mmaction2 指南 (3) 添加新的数据集

2024-01-22 22:54:58

mmaction2 指南

Outline

mmaction2 指南
- 3. 添加新的数据集

3. 添加新的数据集

自定义行为识别数据集

新的数据重新组织到已有格式

重新构建标注文件就行

标注格式一：
rawframe标注【路径，帧数，类别】

some/directory-1 163 1
some/directory-2 122 1
some/directory-3 258 2
some/directory-4 234 2
some/directory-5 295 3
some/directory-6 121 3

标注格式二：
video标注【路径、类别】

some/path/000.mp4 1
some/path/001.mp4 1
some/path/002.mp4 2
some/path/003.mp4 2
some/path/004.mp4 3
some/path/005.mp4 3

标注格式三：

json格式的标注

{
  "video1": {
      "duration_second": 211.53,
      "duration_frame": 6337,
      "annotations": [
          {
              "segment": [
                  30.025882995319815,
                  205.2318595943838
              ],
              "label": "Rock climbing"
          }
      ],
      "feature_frame": 6336,
      "fps": 30.0,
      "rfps": 29.9579255898
  },
  "video2": {
      "duration_second": 26.75,
      "duration_frame": 647,
      "annotations": [
          {
              "segment": [
                  2.578755070202808,
                  24.914101404056165
              ],
              "label": "Drinking beer"
          }
      ],
      "feature_frame": 624,
      "fps": 24.0,
      "rfps": 24.1869158879
  }
}

自定义数据集 class

online转换

继承 BaseDataset 重写 load_annotations(self), evaluate(self, results, metrics, logger) and dump_results(self, results, out) 可参考 RawframeDataset, VideoDataset or ActivityNetDataset.

offline转换
修改数据标注文件，让其符合RawframeDataset, VideoDataset or ActivityNetDataset的调用形式
例子：自己改下标注文件，调用RawframeDataset

...
# dataset settings
dataset_type = 'RawframeDataset'
data_root = 'path/to/your/root'
data_root_val = 'path/to/your/root_val'
ann_file_train = 'data/custom/custom_train_list.txt'
ann_file_val = 'data/custom/custom_val_list.txt'
ann_file_test = 'data/custom/custom_val_list.txt'
...
data = dict(
    videos_per_gpu=32,
    workers_per_gpu=4,
    train=dict(
        type=dataset_type,
        ann_file=ann_file_train,
        ...),
    val=dict(
        type=dataset_type,
        ann_file=ann_file_val,
        ...),
    test=dict(
        type=dataset_type,
        ann_file=ann_file_test,
        ...))
...

自定义数据集的例子

annotation.txt as following

directory,total frames,class
D32_1gwq35E,299,66
-G-5CJ0JkKY,249,254
T4h1bvOd9DA,299,33
4uZ27ivBl00,299,341
0LfESFkfBSw,249,186
-YIsNpBEx6c,299,169

继承BaseDataset,自定义数据类来加载数据
mmaction/datasets/my_dataset.py

import copy
import os.path as osp

import mmcv

from .base import BaseDataset
from .registry import DATASETS


@DATASETS.register_module()
class MyDataset(BaseDataset):

    def __init__(self,
                 ann_file,
                 pipeline,
                 data_prefix=None,
                 test_mode=False,
                 filename_tmpl='img_{:05}.jpg'):
        super(MyDataset, self).__init__(ann_file, pipeline, test_mode)

        self.filename_tmpl = filename_tmpl

    def load_annotations(self):
        video_infos = []
        with open(self.ann_file, 'r') as fin:
            for line in fin:
                if line.startswith("directory"):
                    continue
                frame_dir, total_frames, label = line.split(',')
                if self.data_prefix is not None:
                    frame_dir = osp.join(self.data_prefix, frame_dir)
                video_infos.append(
                    dict(
                        frame_dir=frame_dir,
                        total_frames=int(total_frames),
                        label=int(label)))
        return video_infos

    def prepare_train_frames(self, idx):
        results = copy.deepcopy(self.video_infos[idx])
        results['filename_tmpl'] = self.filename_tmpl
        return self.pipeline(results)

    def prepare_test_frames(self, idx):
        results = copy.deepcopy(self.video_infos[idx])
        results['filename_tmpl'] = self.filename_tmpl
        return self.pipeline(results)

    def evaluate(self,
                 results,
                 metrics='top_k_accuracy',
                 topk=(1, 5),
                 logger=None):
        pass

下一步在配置文件中引入新的数据类型

dataset_A_train = dict(
    type='MyDataset',
    ann_file = ann_file_train,
    pipeline=train_pipeline
)

支持混合数据训练

目前支持repeat dataset；可以用来测试模型是否能在数据集子集上overfit
数据类型RepeatDataset

dataset_A_train = dict(
        type='RepeatDataset',
        times=N,
        dataset=dict(  # This is the original config of Dataset_A
            type='Dataset_A',
            ...
            pipeline=train_pipeline
        )
    )

码农公寓