数据集是 seq 文件的处理办法
2017-03-17
最近下了一个数据集，是 seq 格式的，第一次处理这种数据。使用了官方提供的 matlab 工具包：https://pdollar.github.io/toolbox/index.html
先下载咯：
然后，添加工具包的路径：
1 Simply unzip, then add all directories to the Matlab path: 2 >> addpath(genpath('path/to/toolbox/')); savepath; 3 4 If needed, run the compile script for the mex files: 5 >> toolboxCompile; 6 Note: 64 bit Windows/Linux/Mac binaries are already included.
然后，进入这个工具包中 video 那个文件夹。可以发现:
利用提供的 seqio.m 文件，就可以完成相应的功能了。这里需要写一个脚本，来调用这个函数。因为不想一个个的处理seq文件：
%% load seq files clc; close all; path = 'F:\dataset\Muliti_Spectal_cvpr2015\视频数据\'; files = dir(path); for i =3:size(files, 1) videoName = files(i).name; videoPath = [path, videoName, '\']; files2 = dir([videoPath, '*.seq']); for j = 1:size(files2, 1) videoname = files2(j).name; seqfile = [videoPath, videoname]; videoname2 = strtok(videoname, '.'); imgSavePath = [videoPath, videoname2 ,'\']; if ~exist(imgSavePath) mkdir(imgSavePath); end Is = seqIo( seqfile, 'toImgs', imgSavePath); end end
其中，最重要的就是 Is = seqio(seqfile, 'toImgs', imgSavePath) 这个命令咯，即：将输入的seq文件 seqfile，转换为 image，保存到 imgSavePath 文件夹当中。
运行后，就可以看到相应的图像生成了，文件也就转换成功了。恭喜！
另外一个问题就是：如何将 label 信息从对应的文件中也提取出来？
写一个脚本，先把自己的文件夹都一层一层的找到，然后调用 vbb 文件，读取，并且存为 txt 文件。
1 %% functions for load vbb files and saved as txt file. 2 function read_vbb() 3 % load vbb files and change it into txt file. 4 clc; clear all; close all; 5 path = 'F:\dataset\Muliti_Spectal_cvpr2015\label_files\'; 6 savePath = 'F:\dataset\Muliti_Spectal_cvpr2015\BBox_annotation\'; 7 files = dir(path); 8 for i = 3:size(files, 1) 9 filename = files(i).name; 10 newpath = [path, filename, '\']; 11 12 vbbfiles = dir([newpath, '*.vbb']); 13 for j = 3:size(vbbfiles, 1) 14 vbbName = vbbfiles(j).name; 15 vbbPath = [newpath, vbbName]; 16 disp(vbbPath); 17 A = vbb('vbbLoad',vbbPath); 18 19 txttmp = strtok(vbbName, '.'); 20 txtFileName = [txttmp, '.txt']; 21 savePath2 = [savePath, filename, '\']; 22 if ~exist(savePath2) 23 mkdir(savePath2); 24 end 25 fid = fopen([savePath2, txtFileName], 'w'); 26 27 for k = 1:A.nFrame 28 iframe = A.objLists(1,k); 29 iframe_data = iframe{1,1}; 30 n1length = length(iframe_data); 31 for kk = 1:n1length 32 iframe_dataj = iframe_data(kk); 33 if iframe_dataj.pos(1) ~= 0 %pos posv 34 fprintf(fid,'%d %f %f %f %f\n', k, iframe_dataj.pos(1),... 35 iframe_dataj.pos(2),iframe_dataj.pos(3),iframe_dataj.pos(4)); 36 end 37 end 38 39 end 40 41 end 42 43 end 44 45 end
需要下面的 关于 vbb 的函数：
1 function varargout = vbb( action, varargin ) 2 % Data structure for video bounding box (vbb) annotations. 3 % 4 % A video bounding box (vbb) annotation stores bounding boxes (bbs) for 5 % objects of interest. The primary difference from a static annotation is 6 % that each object can exist for multiple frames, ie, a vbb annotation not 7 % only provides the locations of objects but also tracking information. A 8 % vbb annotation A is simply a Matlab struct. It contains data per object 9 % (such as a string label) and data per object per frame (such as a bb). 10 % Each object is identified with a unique integer id. 11 % 12 % Data per object (indexed by integer id) includes the following fields: 13 % init - 0/1 value indicating whether object w given id exists 14 % lbl - a string label describing object type (eg: 'pedestrian') 15 % str - the first frame in which object appears (1 indexed) 16 % end - the last frame in which object appears (1 indexed) 17 % hide - 0/1 value indicating object is 'hidden' (used during labeling) 18 % 19 % Data per object per frame (indexed by frame and id) includes: 20 % pos - [l t w h]: bb indicating predicted object extent 21 % posv - [l t w h]: bb indicating visible region (may be [0 0 0 0]) 22 % occl - 0/1 value indicating if bb is occluded 23 % lock - 0/1 value indicating bb is 'locked' (used during labeling) 24 % 25 % vbb contains a number of utility functions for working with an 26 % annotation A, making it generally unnecessary to access the fields of A 27 % directly. The format for accessing the various utility functions is: 28 % outputs = vbb( 'action', inputs ); 29 % Below is a list of utility functions, broken up into 3 categories. 30 % Occasionally more help is available via a call to help "vbb>action". 31 % 32 % %%% init and save/load annotation to/from disk 33 % Create new annotation for given length video 34 % A = vbb( 'init', nFrame, maxObj ) 35 % Generate annotation filename (add .vbb and optionally time stamp) 36 % [fName,ext] = vbb( 'vbbName', fName, [timeStmp], [ext] ) 37 % Save annotation A to fName with optional time stamp (F by default) 38 % vbb('vbbSave', A, fName, [timeStmp] ) 39 % Load annotation from disk: 40 % A = vbb('vbbLoad', fName ) 41 % Save annotation A to fName (in .txt format): 42 % vbb('vbbSaveTxt', A, fName, timeStmp ) 43 % Load annotation from disk (in .txt format): 44 % A = vbb('vbbLoadTxt', fName ) 45 % Export single frame annotations to tarDir/*.txt 46 % vbb( 'vbbToFiles', A, tarDir, [fs], [skip], [f0], [f1] ) 47 % Combine single frame annotations from srcDir/*.txt 48 % [A,fs] = vbb( 'vbbFrFiles', srcDir, [fs] ) 49 % 50 % %%% inspect / alter annotation 51 % Get number of unique objects in annotation 52 % n = vbb( 'numObj', A ) 53 % Get an unused object id (for adding a new object) 54 % [A,id] = vbb( 'newId', A ) 55 % Create a new, empty object (not added to A) 56 % [A,obj] = vbb( 'emptyObj', A, [frame] ) 57 % Get struct with all data from frames s-e for given object 58 % obj = vbb( 'get', A, id, [s], [e] ) 59 % Add object to annotation 60 % A = vbb( 'add', A, obj ) 61 % Remove object from annotation 62 % A = vbb( 'del', A, id ) 63 % Crop or extend object temporally 64 % A = vbb( 'setRng', A, id, s, e ) 65 % Get object information, see above for valid properties 66 % v = vbb( 'getVal', A, id, name, [frmS], [frmE] ) 67 % Set object information, see above for valid properties 68 % A = vbb( 'setVal', A, id, name, v, [frmS], [frmE] ) 69 % 70 % %%% other functions 71 % Visulatization: draw annotation on top of current image 72 % hs = vbb( 'drawToFrame', A, frame ) 73 % Uses seqPlayer to display seq file with overlayed annotations. 74 % vbb( 'vbbPlayer', A, srcName ) 75 % Visulatization: create seq file w annotation superimposed. 76 % vbb( 'drawToVideo', A, srcName, tarName ) 77 % Shift entire annotation by del frames. (useful for synchronizing w video) 78 % A = vbb( 'timeShift', A, del ) 79 % Ensure posv is fully contained in pos 80 % A = vbb( 'swapPosv', A, validate, swap, hide ) 81 % Stats: get stats about annotation 82 % [stats,stats1,logDur] = vbb( 'getStats', A ); 83 % Returns the ground truth bbs for a single frame. 84 % [gt,posv,lbls] = vbb( 'frameAnn', A, frame, lbls, test ) 85 % 86 % USAGE 87 % varargout = vbb( action, varargin ); 88 % 89 % INPUTS 90 % action - string specifying action 91 % varargin - depends on action, see above 92 % 93 % OUTPUTS 94 % varargout - depends on action, see above 95 % 96 % EXAMPLE 97 % 98 % See also BBAPPLY 99 % 100 % Caltech Pedestrian Dataset Version 3.2.1 101 % Copyright 2014 Piotr Dollar. [pdollar-at-gmail.com] 102 % Licensed under the Simplified BSD License [see external/bsd.txt] 103 104 %#ok<*DEFNU> 105 varargout = cell(1,nargout); 106 [varargout{:}] = eval([action '(varargin{:});']); 107 end 108 109 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 110 % init / save / load 111 112 function A = init( nFrame, maxObj ) 113 114 if( nargin<2 || isempty(maxObj) ), maxObj=16; end 115 116 A.nFrame = nFrame; 117 A.objLists = cell(1,nFrame); 118 A.maxObj = maxObj; 119 A.objInit = zeros(1,A.maxObj); 120 A.objLbl = cell(1,A.maxObj); 121 A.objStr = -ones(1,A.maxObj); 122 A.objEnd = -ones(1,A.maxObj); 123 A.objHide = zeros(1,A.maxObj); 124 125 A.altered = false; 126 A.log = 0; 127 A.logLen = 0; 128 end 129 130 function [fName,ext] = vbbName( fName, timeStmp, ext ) 131 if(nargin<2), timeStmp=false; end; if(nargin<3), ext=''; end 132 [d,f,ext1]=fileparts(fName); if(isempty(ext)), ext=ext1; end 133 if(isempty(ext)), ext='.vbb'; end 134 if(timeStmp), f=[f '-' regexprep(datestr(now), ':| ', '-')]; end 135 if(isempty(d)), d='.'; end; fName=[d '/' f ext]; 136 end 137 138 function vbbSave( A, fName, timeStmp ) 139 if(nargin<3), timeStmp=false; end; vers=1.4; %#ok<NASGU> 140 [fName,ext]=vbbName(fName,timeStmp); 141 if(strcmp(ext,'.txt')), vbbSaveTxt(A,fName,timeStmp); return; end 142 A=cleanup(A); save(fName,'A','vers','-v6'); %#ok<NASGU> 143 end 144 145 function A = vbbLoad( fName ) 146 [fName,ext]=vbbName(fName); vers=1.4; 147 if(strcmp(ext,'.txt')), A=vbbLoadTxt(fName); return; end 148 L = load( '-mat', fName ); 149 if( ~isfield(L,'A') || ~isfield(L,'vers') ); 150 error('Not a valid video annoation file.'); 151 end; 152 A = L.A; 153 % .06 -> 1.0 conversion (add log/logLen) 154 if( L.vers==.06 ) 155 L.vers=1.0; A.log=0; A.logLen=0; 156 end 157 % 1.0 -> 1.1 conversion (add trnc field) 158 if( L.vers==1.0 ) 159 L.vers=1.1; 160 for f=1:A.nFrame 161 if(isempty(A.objLists{f})); A.objLists{f}=[]; end 162 for j=1:length(A.objLists{f}); A.objLists{f}(j).trnc=0; end 163 end 164 end 165 % 1.1 -> 1.2 conversion (add hide/posv fields) 166 if( L.vers==1.1 ) 167 L.vers=1.2; 168 for f=1:A.nFrame 169 if(isempty(A.objLists{f})); A.objLists{f}=[]; end 170 for j=1:length(A.objLists{f}); A.objLists{f}(j).posv=[0 0 0 0]; end 171 end 172 A.objHide = zeros(1,A.maxObj); 173 end 174 % 1.2 -> 1.3 conversion (remove trnc field) 175 if( L.vers==1.2 ) 176 L.vers=1.3; 177 for f=1:A.nFrame 178 if(isempty(A.objLists{f})); A.objLists{f}=[]; else 179 A.objLists{f} = rmfield(A.objLists{f},'trnc'); 180 end 181 end 182 end 183 % 1.3 -> 1.4 conversion (remove objAr field) 184 if( L.vers==1.3 ) 185 L.vers=1.4; A = rmfield(A,'objAr'); 186 end 187 % check version 188 if( L.vers~=vers ) 189 er = [num2str(L.vers) ' (current: ' num2str(vers) ')']; 190 error(['Incompatible versions: ' er]); 191 end 192 % order fields 193 order={'nFrame','objLists','maxObj','objInit','objLbl','objStr',... 194 'objEnd','objHide','altered','log','logLen'}; 195 A = orderfields(A,order); 196 A.altered = false; 197 end 198 199 function vbbSaveTxt( A, fName, timeStmp ) 200 if(nargin<3), timeStmp=false; end; vers=1.4; 201 fName=vbbName(fName,timeStmp,'.txt'); 202 A=cleanup(A,0); n=numObj(A); nFrame=A.nFrame; 203 fid=fopen(fName,'w'); assert(fid>0); 204 % write header info to text 205 fp=@(varargin) fprintf(fid,varargin{:}); 206 fp('%% vbb version=%f\n',vers); fp('nFrame=%i n=%i\n',nFrame,n); 207 fp('log=['); fp('%f ',A.log); fp(']\n'); 208 % write each object to text 209 for id=1:n, o=get(A,id); 210 fp('\n-----------------------------------\n'); 211 fp('lbl=''%s'' str=%i end=%i hide=%i\n',o.lbl,o.str,o.end,o.hide); 212 fp('pos =['); fp('%f %f %f %f; ',o.pos'); fp(']\n'); 213 fp('posv=['); fp('%f %f %f %f; ',o.posv'); fp(']\n'); 214 fp('occl=['); fp('%i ',o.occl); fp(']\n'); 215 fp('lock=['); fp('%i ',o.lock); fp(']\n'); 216 end 217 fclose(fid); 218 end 219 220 function A = vbbLoadTxt( fName ) 221 fName=vbbName(fName,0,'.txt'); vers=1.4; 222 if(~exist(fName,'file')), error([fName ' not found']); end 223 try 224 % read in header and create A 225 f=fopen(fName,'r'); s=fgetl(f); v=sscanf(s,'%% vbb version=%f'); 226 if(v~=vers), error('Incompatible versions: %f (current=%f)',v,vers); end 227 s=fgetl(f); r=sscanf(s,'nFrame=%d n=%d'); nFrame=r(1); n=r(2); 228 s=fgetl(f); assert(strcmp(s(1:5),'log=[')); assert(s(end)==']'); 229 log=sscanf(s(6:end-1),'%f ')'; A=init(nFrame,n); 230 % read in each object in turn 231 for id=1:n 232 s=fgetl(f); assert(isempty(s)); 233 s=fgetl(f); assert(strcmp(s,'-----------------------------------')); 234 s=fgetl(f); r=textscan(s,'lbl=%s str=%d end=%d hide=%d'); 235 [A,o]=emptyObj(A,0); o.lbl=r{1}{1}(2:end-1); 236 o.str=r{2}; o.end=r{3}; o.hide=r{4}; 237 s=fgetl(f); assert(strcmp(s(1:6),'pos =[')); assert(s(end)==']'); 238 pos=sscanf(s(7:end-1),'%f %f %f %f;'); o.pos=reshape(pos,4,[])'; 239 s=fgetl(f); assert(strcmp(s(1:6),'posv=[')); assert(s(end)==']'); 240 posv=sscanf(s(7:end-1),'%f %f %f %f;'); o.posv=reshape(posv,4,[])'; 241 s=fgetl(f); assert(strcmp(s(1:6),'occl=[')); assert(s(end)==']'); 242 o.occl=sscanf(s(7:end-1),'%d '); 243 s=fgetl(f); assert(strcmp(s(1:6),'lock=[')); assert(s(end)==']'); 244 o.lock=sscanf(s(7:end-1),'%d '); 245 A=add(A,o); 246 end 247 if(isempty(log)), A.log=0; A.logLen=0; else 248 A.log=log; A.logLen=length(log); end 249 A.altered=false; fclose(f); 250 catch e, fclose(f); throw(e); 251 end 252 end 253 254 function vbbToFiles( A, tarDir, fs, skip, f0, f1 ) 255 % export single frame annotations to tarDir/*.txt 256 nFrm=A.nFrame; fName=@(f) ['I' int2str2(f-1,5) '.txt']; 257 if(nargin<3 || isempty(fs)), for f=1:nFrm, fs{f}=fName(f); end; end 258 if(nargin<4 || isempty(skip)), skip=1; end 259 if(nargin<5 || isempty(f0)), f0=1; end 260 if(nargin<6 || isempty(f1)), f1=nFrm; end 261 if(~exist(tarDir,'dir')), mkdir(tarDir); end 262 for f=f0:skip:f1 263 nObj=length(A.objLists{f}); objs=bbGt('create',nObj); 264 for j=1:nObj 265 o=A.objLists{f}(j); objs(j).lbl=A.objLbl{o.id}; objs(j).occ=o.occl; 266 objs(j).bb=round(o.pos); objs(j).bbv=round(o.posv); 267 end 268 bbGt('bbSave',objs,[tarDir '/' fs{f}]); 269 end 270 end 271 272 function [A,fs] = vbbFrFiles( srcDir, fs ) 273 % combine single frame annotations from srcDir/*.txt 274 if(nargin<2 || isempty(fs)), fs=dir([srcDir '/*.txt']); fs={fs.name}; end 275 nFrm=length(fs); A=init(nFrm); 276 for f=1:nFrm 277 objs = bbGt('bbLoad',[srcDir '/' fs{f}]); 278 for j=1:length(objs) 279 [A,obj]=emptyObj(A,f); o=objs(j); obj.lbl=o.lbl; 280 obj.pos=o.bb; obj.occl=o.occ; obj.posv=o.bbv; A=add(A,obj); 281 end 282 end 283 end 284 285 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 286 % inspect / alter annotation 287 288 function n = numObj( A ) 289 n = sum( A.objInit ); 290 end 291 292 function [A,id] = newId( A ) 293 [val,id] = min( A.objInit ); 294 if( isempty(val) || val~=0 ); 295 A = doubleLen( A ); 296 [val,id] = min( A.objInit ); 297 end 298 assert(val==0); 299 end 300 301 function [A,obj] = emptyObj( A, frame ) 302 [A,id] = newId( A ); 303 obj.id = id; 304 obj.lbl = ''; 305 obj.hide = 0; 306 if(nargin<2) 307 obj.str = -1; 308 obj.end = -1; 309 len = 0; 310 else 311 obj.str = frame; 312 obj.end = frame; 313 len = 1; 314 end 315 obj.pos = zeros(len,4); 316 obj.posv = zeros(len,4); 317 obj.occl = zeros(len,1); 318 obj.lock = ones(len,1); 319 end 320 321 function obj = get( A, id, s, e ) 322 323 assert( 0<id && id<=A.maxObj ); 324 assert( A.objInit(id)==1 ); 325 326 if(nargin<3); s=A.objStr(id); else assert(s>=A.objStr(id)); end; 327 if(nargin<4); e=A.objEnd(id); else assert(e<=A.objEnd(id)); end; 328 329 % get general object info 330 obj.id = id; 331 obj.lbl = A.objLbl{id}; 332 obj.str = s; 333 obj.end = e; 334 obj.hide = A.objHide(id); 335 336 % get per-frame object info 337 len = obj.end-obj.str+1; 338 obj.pos = zeros(len,4); 339 obj.posv = zeros(len,4); 340 obj.occl = zeros(len,1); 341 obj.lock = zeros(len,1); 342 for i=1:len 343 f = obj.str+i-1; 344 objList = A.objLists{f}; 345 obj1 = objList([objList.id]==id); 346 obj.pos(i,:) = obj1.pos; 347 obj.posv(i,:) = obj1.posv; 348 obj.occl(i) = obj1.occl; 349 obj.lock(i) = obj1.lock; 350 end 351 352 end 353 354 function A = add( A, obj ) 355 356 % check id or get new id 357 id = obj.id; 358 if( id==-1 ) 359 [A,id] = newId( A ); 360 else 361 assert( 0<id && id<=A.maxObj ); 362 assert( A.objInit(id)==0 ); 363 end 364 365 % save general object info 366 A.objInit(id) = 1; 367 A.objLbl{id} = obj.lbl; 368 A.objStr(id) = obj.str; 369 A.objEnd(id) = obj.end; 370 A.objHide(id) = obj.hide; 371 372 % save per-frame object info 373 len = obj.end - obj.str + 1; 374 assert( size(obj.pos,1)==len ); 375 assert( size(obj.posv,1)==len ); 376 assert( size(obj.occl,1)==len ); 377 assert( size(obj.lock,1)==len ); 378 for i = 1:len 379 obj1.id = id; 380 obj1.pos = obj.pos(i,:); 381 obj1.posv = obj.posv(i,:); 382 obj1.occl = obj.occl(i); 383 obj1.lock = obj.lock(i); 384 f = obj.str+i-1; 385 A.objLists{f}=[A.objLists{f} obj1]; 386 end 387 388 A = altered( A ); 389 end 390 391 function A = del( A, id ) 392 393 assert( 0<id && id<=A.maxObj ); 394 assert( A.objInit(id)==1 ); 395 396 % delete per-frame object info 397 objStr = A.objStr(id); 398 objEnd = A.objEnd(id); 399 len = objEnd-objStr+1; 400 for i=1:len 401 f = objStr+i-1; 402 objList = A.objLists{f}; 403 objList([objList.id]==id) = []; 404 A.objLists{f} = objList; 405 end 406 407 % delete general object info 408 A.objInit(id) = 0; 409 A.objLbl{id} = []; 410 A.objStr(id) = -1; 411 A.objEnd(id) = -1; 412 A.objHide(id) = 0; 413 414 A = altered( A ); 415 end 416 417 function A = setRng( A, id, s, e ) 418 419 assert( s>=1 && e<=A.nFrame && s<=e && A.objInit(id)==1 ); 420 s0=A.objStr(id); e0=A.objEnd(id); assert( e>=s0 && e0>=s ); 421 if(s==s0 && e==e0), return; end; A.objStr(id)=s; A.objEnd(id)=e; 422 423 if( s0>s ) 424 objs=A.objLists{s0}; obj=objs([objs.id]==id); obj.occl=0; obj.lock=0; 425 for f=s:s0-1, A.objLists{f}=[A.objLists{f} obj]; end 426 elseif( s0<s ) 427 for f=s0:s-1, os=A.objLists{f}; os([os.id]==id)=[]; A.objLists{f}=os; end 428 end 429 430 if( e>e0 ) 431 objs=A.objLists{e0}; obj=objs([objs.id]==id); obj.occl=0; obj.lock=0; 432 for f=e0+1:e, A.objLists{f}=[A.objLists{f} obj]; end 433 elseif( e<e0 ) 434 for f=e+1:e0, os=A.objLists{f}; os([os.id]==id)=[]; A.objLists{f}=os; end 435 end 436 437 A = altered( A ); 438 end 439 440 function v = getVal( A, id, name, frmS, frmE ) 441 442 if(nargin<4); frmS=[]; end; 443 if(nargin<5); frmE=frmS; end; 444 assert(strcmp(name,'init') || A.objInit(id)==1); 445 switch name 446 case 'lbl' 447 assert( isempty(frmS) ); 448 v = A.objLbl{id}; 449 case {'init','str','end','hide'} 450 assert( isempty(frmS) ); 451 name = ['obj' upper(name(1)) name(2:end)]; 452 v = A.(name)(id); 453 case {'pos','posv','occl','lock'} 454 assert( ~isempty(frmS) ); 455 assert( A.objStr(id)<=frmS && frmE<=A.objEnd(id) ); 456 frms = frmS:frmE; len=length(frms); 457 for f=1:len 458 objList = A.objLists{frms(f)}; 459 v1 = objList([objList.id]==id).(name); 460 if( f==1 ); v=repmat(v1,[len 1]); else v(f,:) = v1; end 461 end 462 otherwise 463 error( ['invalid field: ' name] ); 464 end 465 466 end 467 468 function A = setVal( A, id, name, v, frmS, frmE ) 469 470 if(nargin<5); frmS=[]; end; 471 if(nargin<6); frmE=frmS; end; 472 assert( A.objInit(id)==1 ); 473 switch name 474 case 'lbl' 475 assert( isempty(frmS) ); 476 A.objLbl{id} = v; 477 case {'hide'} 478 assert( isempty(frmS) ); 479 name = ['obj' upper(name(1)) name(2:end)]; 480 A.(name)(id) = v; 481 case {'pos','posv','occl','lock'} 482 assert( ~isempty(frmS) ); 483 assert( A.objStr(id)<=frmS && frmE<=A.objEnd(id) ); 484 frms = frmS:frmE; len=length(frms); 485 for f=1:len 486 objList = A.objLists{frms(f)}; 487 objList([objList.id]==id).(name) = v(f,:); 488 A.objLists{frms(f)} = objList; 489 end 490 otherwise 491 error( ['invalid/unalterable field: ' name] ); 492 end 493 494 A = altered( A ); 495 end 496 497 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 498 % other functions 499 500 function hs = drawToFrame( A, frame ) 501 hs=[]; 502 for o=A.objLists{frame} 503 hr = bbApply('draw', o.pos, 'g', 2, '-' ); 504 if(all(o.posv)==0), hrv=[]; else 505 hrv = bbApply('draw', o.posv, 'y', 2, '--' ); 506 end 507 label = [A.objLbl{o.id} ' [' int2str(o.id) ']']; 508 ht = text( o.pos(1), o.pos(2)-10, label ); 509 set( ht, 'color', 'w', 'FontSize', 10, 'FontWeight', 'bold' ); 510 hs = [hs hr ht hrv]; %#ok<AGROW> 511 end 512 end 513 514 function vbbPlayer( A, srcName ) 515 dispFunc=@(f) vbb('drawToFrame',A,f+1); 516 seqPlayer(srcName,dispFunc); 517 end 518 519 function drawToVideo( A, srcName, tarName ) 520 % open video to read and make video to write 521 assert( ~strcmp(srcName,tarName) ); 522 sr=seqIo(srcName,'r'); info=sr.getinfo(); 523 nFrame=info.numFrames; w=info.width; h=info.height; 524 assert(A.nFrame==nFrame); sw=seqIo(tarName,'w',info); 525 % display and write each frame 526 ticId=ticStatus; hf=figure; hAx=axes('parent',hf); 527 hIm=imshow(zeros(h,w,3,'uint8'),'parent',hAx); truesize; 528 529 for i=1:nFrame 530 I=sr.getnext(); set(hIm,'CData',I); hs=drawToFrame(A,i); 531 I=getframe; I=I.cdata; I=I(1:h,1:w,:); 532 sw.addframe(I); delete(hs); tocStatus( ticId, i/nFrame ); 533 end 534 sr.close(); sw.close(); close(hf); 535 end 536 537 function A = timeShift( A, del ) 538 % shift entire annotation by del frames 539 nFrame=A.nFrame; locs=logical(A.objInit); 540 if(del>0), is=locs & A.objStr==1; A.objStr(is)=A.objStr(is)-del; end 541 A.objStr(locs)=min(max(A.objStr(locs)+del,1),nFrame); 542 A.objEnd(locs)=min(max(A.objEnd(locs)+del,1),nFrame); 543 if( del>0 ) % replicate annotation for first frame del times 544 A.objLists=[A.objLists(ones(1,del)) A.objLists(1:end-del)]; 545 else % no annotations for last del frames 546 A.objLists=[A.objLists(1-del:end) cell(1,-del)]; 547 end 548 end 549 550 function A = swapPosv( A, validate, swap, hide ) 551 % Swap pos/posv and ensure pos/posv consistent. 552 % 553 % The visible region of an object (posv) should be a subset of the 554 % predicted region (pos). If the object is not occluded, then posv is the 555 % same as pos (posv=[0 0 0 0] by default and indicates posv not set). 556 % Swapping is used to swap pos and posv, and validating is used to ensure 557 % pos contains posv. In two cases no swapping/validating occurs. If occl==0 558 % for a given bb, posv is set to [0 0 0 0]. If occl==1 and posv=[0 0 0 0], 559 % posv is set to pos. In either case there is no need to swap or validate. 560 % 561 % The validate flag: 562 % validate==-1: posv = intersect(pos,posv) 563 % validate== 0: no validation 564 % validate==+1: pos = union(pos,posv) 565 % If posv is shrunk to 0, it is set to a small bb inside pos. 566 % 567 % The swap flag: 568 % swap==-1: pos and posv are swapped before validating 569 % swap== 0: no swapping 570 % swap==+1: pos and posv are swapped after validating 571 % 572 % The hide flag: 573 % hide==0: set hide attribute to 0 for all objects 574 % hide==1: set hide attribute to 1 iff object is at some point occluded 575 % 576 % Suppose a user has labeled pos in a given video using some annotation 577 % tool. At this point can swap pos and posv and use the same tool in the 578 % same manner to label posv (which is stored in pos). Afterwards, can swap 579 % pos/posv again, and ensure they are consistent, and both end up labeled. 580 % Additionally, in the second labeling phase, we may want to hide any 581 % object that is never occluded. The command to setup labeling of posv is: 582 % A=vbb('swapPosv',A,-1,1,1); % validate (trust pos) THEN swap 583 % Afterwards, to swap back, would use: 584 % A=vbb('swapPosv',A,1,-1,0); % swap THEN validate (trust posv) 585 % While labeling posv only frames where occl is already set should be 586 % altered (the occl flag itself shouldn't be altered). 587 % 588 % USAGE 589 % A = vbb( 'swapPosv', A, validate, swap, hide ) 590 % 591 % INPUTS 592 % A - annotation structure 593 % validate - see above 594 % swap - see above 595 % hide - see above 596 % 597 % OUTPUTS 598 % A - updated annotation 599 % 600 % EXAMPLE 601 % 602 % see also vbb 603 604 for f=1:A.nFrame 605 for i=1:length(A.objLists{f}) 606 o=A.objLists{f}(i); p=o.pos; v=o.posv; vt=[]; 607 % v is trivial - either [0 0 0 0] or same as p, continue 608 if(o.occl==0), vt=[0 0 0 0]; elseif(all(v)==0), vt=p; end 609 if(~isempty(vt)), A.objLists{f}(i).posv=vt; continue; end 610 % optionally swap before validating 611 if(swap==-1), t=p; p=v; v=t; end 612 % validate 613 if( validate==-1 ) 614 v = bbApply('intersect',v,p); 615 if(all(v==0)), v=[p(1:2)+p(3:4)/2 1 1]; end 616 elseif( validate==1 ) 617 p = bbApply('union',v,p); 618 end 619 % optionally swap after validating 620 if(swap==1), t=p; p=v; v=t; end 621 % store results 622 o.pos=p; o.posv=v; A.objLists{f}(i)=o; 623 end 624 end 625 626 if(~hide), A.objHide(:)=0; else 627 for id=find( A.objInit ) 628 occl=vbb('getVal',A,id,'occl',A.objStr(id),A.objEnd(id)); 629 A.objHide(id) = all(occl==0); 630 end 631 end 632 633 end 634 635 function [stats,stats1,logDur] = getStats( A ) 636 637 % get stats of many annotations simultaneously by first merging 638 if(length(A)>1), A=merge(A); end 639 640 % log activity (allows up to .25h of inactivity) 641 nObj0=numObj(A); if(nObj0==0), stats=struct(); return; end 642 log = A.log / 1.1574e-005 / 60 / 60; 643 locs = find( (log(2:end)-log(1:end-1)) > .25 ); 644 logS=log([1 locs+1]); logE=log([locs A.logLen]); 645 logDur = sum(logE-logS); 646 647 % getStats1 on entire annotation 648 stats = getStats1( A ); 649 650 % getStats1 separately for each label 651 lbl0=unique(A.objLbl); nLbl0=length(lbl0); 652 stats1=repmat(getStats1(subset(A,lbl0(1))),1,nLbl0); 653 for i0=2:nLbl0, stats1(i0)=getStats1(subset(A,lbl0(i0))); end 654 655 function stats = getStats1( A ) 656 % unique labels and label counts 657 lbl=unique(A.objLbl); nLbl=length(lbl); lblCnts=zeros(1,nLbl); 658 for i=1:nLbl, lblCnts(i)=sum(strcmp(A.objLbl,lbl{i})); end 659 stats.labels=lbl; stats.labelCnts=lblCnts; 660 % get object lengths 661 nObj=numObj(A); stats.nObj=nObj; 662 len=A.objEnd-A.objStr+1; stats.len=len; 663 % get all per frame info in one flat array ordered by object id 664 nPerFrm=cellfun(@length,A.objLists); stats.nPerFrm=nPerFrm; 665 ols=A.objLists(nPerFrm>0); ols=[ols{:}]; 666 ids=[ols.id]; [ids,order]=sort(ids); ols=ols(order); stats.ids=ids; 667 inds=[0 cumsum(len)]; stats.inds=inds; 668 % get all pos/posv and centers, also first/last frame for each obj 669 pos=reshape([ols.pos],4,[])'; posv=reshape([ols.posv],4,[])'; 670 posS=pos(inds(1:end-1)+1,:); posE=pos(inds(2:end),:); 671 stats.pos=pos; stats.posv=posv; stats.posS=posS; stats.posE=posE; 672 % get object centers and per frame deltas 673 cen=bbApply('getCenter',pos); stats.cen=cen; 674 del=cen(2:end,:)-cen(1:end-1,:); del(inds(2:end),:)=-1; stats.del=del; 675 % get occlusion information 676 occl=(sum(posv,2)>0)'; %occl=[ols.occl]; <--slow 677 occFrac=1-posv(:,3).*posv(:,4)./pos(:,3)./pos(:,4); occFrac(occl==0)=0; 678 occTime=zeros(1,nObj); for i=1:nObj, occTime(i)=mean(occl(ids==i)); end 679 stats.occl=occl; stats.occFrac=occFrac'; stats.occTime=occTime; 680 end 681 682 function A = subset( A, lbls ) 683 % find elements to keep 684 nObj=numObj(A); keep=false(1,nObj); 685 for i=1:length(lbls), keep=keep | strcmp(A.objLbl,lbls{i}); end 686 % clean up objLists by dropping elements 687 frms=find(cellfun('isempty',A.objLists)==0); ols=A.objLists; 688 for f=frms, ols{f}=ols{f}(keep([ols{f}.id])); end, A.objLists=ols; 689 % run cleanup to reorder/drop elements 690 A.objInit=keep; A=cleanup(A,0); 691 end 692 693 function A = merge( AS ) 694 nFrm=0; nObj=0; 695 for i=1:numel(AS) 696 Ai=cleanup(AS(i),0); 697 for f=1:Ai.nFrame 698 for j=1:length(Ai.objLists{f}), 699 Ai.objLists{f}(j).id=Ai.objLists{f}(j).id+nObj; 700 end 701 end 702 Ai.objStr=Ai.objStr+nFrm; Ai.objEnd=Ai.objEnd+nFrm; 703 nFrm=Ai.nFrame+nFrm; Ai.nFrame=nFrm; 704 nObj=nObj+numObj(Ai); AS(i)=Ai; 705 end 706 A.nFrame = nFrm; 707 A.objLists = [AS.objLists]; 708 A.maxObj = sum([AS.maxObj]); 709 A.objInit = [AS.objInit]; 710 A.objLbl = [AS.objLbl]; 711 A.objStr = [AS.objStr]; 712 A.objEnd = [AS.objEnd]; 713 A.objHide = [AS.objHide]; 714 A.altered = false; 715 A.log = sort([AS.log]); 716 A.logLen = sum([AS.logLen]); 717 end 718 end 719 720 function [gt,posv,lbls1] = frameAnn( A, frame, lbls, test ) 721 % Returns the ground truth bbs for a single frame. 722 % 723 % Returns bbs for all object with lbl in lbls. The result is an [nx5] array 724 % where each row is of the form [x y w h ignore]. [x y w h] is the bb and 725 % ignore is a 0/1 flag that indicates regions to be ignored. For each 726 % returned object, the ignore flag is set to 0 if test(lbl,pos,posv)=1 for 727 % the given object. For example, using lbls={'person','people'}, and 728 % test=@(lbl,bb,bbv) bb(4)>100, returns bbs for all 'person' and 'people' 729 % in given frame, and for any objects under 100 pixels tall ignore=1. 730 % 731 % USAGE 732 % [gt,posv,lbls] = vbb( 'frameAnn', A, frame, lbls, [test] ) 733 % 734 % INPUTS 735 % A - annotation structure 736 % frame - the frame index 737 % lbls - cell array of string labels 738 % test - [] ignore = ~test(lbl,pos,posv) 739 % 740 % OUTPUTS 741 % gt - [n x 5] array containg ground truth for frame 742 % posv - [n x 4] bbs of visible regions 743 % lbls - [n x 1] list of object labels 744 % 745 % EXAMPLE 746 % lbls={'person','people'}; test=@(lbl,bb,bbv) bb(4)>100; 747 % [gt,lbls] = vbb( 'frameAnn', A, 200, lbls, test ) 748 749 if( nargin<4 ), test=[]; end; assert(frame<=A.nFrame); 750 ng=length(A.objLists{frame}); ignore=0; 751 gt=zeros(ng,5); posv=zeros(ng,4); lbls1=cell(1,ng); keep=true(1,ng); 752 for g=1:ng 753 o=A.objLists{frame}(g); lbl=A.objLbl{o.id}; 754 if(~any(strcmp(lbl,lbls))), keep(g)=0; continue; end 755 if(~isempty(test)), ignore=~test(lbl,o.pos,o.posv); end 756 gt(g,:)=[o.pos ignore]; lbls1{g}=lbl; posv(g,:)=o.posv; 757 end 758 gt=gt(keep,:); lbls1=lbls1(keep); posv=posv(keep,:); 759 end 760 761 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 762 % helper functions 763 764 function A = doubleLen( A ) 765 maxObj = max(1,A.maxObj); 766 A.objInit = [A.objInit zeros(1,maxObj)]; 767 A.objLbl = [A.objLbl cell(1,maxObj)]; 768 A.objStr = [A.objStr -ones(1,maxObj)]; 769 A.objEnd = [A.objEnd -ones(1,maxObj)]; 770 A.objHide = [A.objHide zeros(1,maxObj)]; 771 A.maxObj = max(1,A.maxObj * 2); 772 773 A = altered( A ); 774 end 775 776 function A = altered( A ) 777 A.altered = true; 778 if( length(A.log)==A.logLen ) 779 A.log = [A.log zeros(1,A.logLen)]; 780 end 781 T = now; sec=1.1574e-005; 782 if( A.logLen>0 && (T-A.log(A.logLen))/sec<1 ) 783 A.log(A.logLen) = T; 784 else 785 A.logLen = A.logLen+1; 786 A.log(A.logLen) = T; 787 end 788 end 789 790 function A = cleanup( A, minn ) 791 % cleanup() Removes placeholder entries from A 792 if( A.maxObj==0 ), return; end 793 if( nargin<2 || isempty(minn) ), minn=1; end 794 % reorder so all initialized objects are first 795 while( 1 ) 796 % find first 0 entry in objInit 797 [val,id0] = min(A.objInit); 798 if( val==1 || id0==A.maxObj ), break; end 799 % find last 1 entry past 0 entry 800 [val,id1]=max(fliplr(A.objInit(id0+1:end))); id1=A.maxObj-id1+1; 801 if(val==0), break; end 802 % swap these two locations 803 A = swap( A, id0, id1 ); 804 end 805 % now discard all uninitialized objects (keep at least minn though) 806 [val,n] = min(A.objInit); n=max(minn,n-1); 807 if( val==0 ) 808 A.maxObj = n; 809 A.objInit = A.objInit(1:n); 810 A.objLbl = A.objLbl(1:n); 811 A.objStr = A.objStr(1:n); 812 A.objEnd = A.objEnd(1:n); 813 A.objHide = A.objHide(1:n); 814 end 815 % discard useless elements in log 816 A.log = A.log(1:A.logLen); 817 end 818 819 function A = swap( A, id1, id2 ) 820 821 A0=A; 822 823 if(A0.objInit(id1)), fs=A0.objStr(id1):A0.objEnd(id1); else fs=[]; end 824 for f=fs, ol=A0.objLists{f}; ol([ol.id]==id1).id=id2; A.objLists{f}=ol; end 825 826 if(A0.objInit(id2)), fs=A0.objStr(id2):A0.objEnd(id2); else fs=[]; end 827 for f=fs, ol=A0.objLists{f}; ol([ol.id]==id2).id=id1; A.objLists{f}=ol; end 828 829 A.objInit(id1) = A0.objInit(id2); A.objInit(id2) = A0.objInit(id1); 830 A.objLbl(id1) = A0.objLbl(id2); A.objLbl(id2) = A0.objLbl(id1); 831 A.objStr(id1) = A0.objStr(id2); A.objStr(id2) = A0.objStr(id1); 832 A.objEnd(id1) = A0.objEnd(id2); A.objEnd(id2) = A0.objEnd(id1); 833 A.objHide(id1) = A0.objHide(id2); A.objHide(id2) = A0.objHide(id1); 834 835 end
执行后，就可以了。如图所示：