小白进阶-官方Tensorflow Object_detection API应用(目标检测-安全帽）

2021-10-06 20:38:58

前奏--学生党记录，大佬勿喷

一、准备tensorflow的安装环境

1、Anaconda3(笔者装的minconda，Miniconda，顾名思义，它只包含最基本的内容——python与conda，以及相关的必须依赖项，对于空间要求严格的用户，Miniconda是一种选择。就只包含最基本的东西，其他的库得自己装；Anaconda则是一个打包的集合，里面预装好了conda、某个版本的python、众多packages、科学计算工具等等，就是把很多常用的不常用的库都给你装好了）

2、tensorflow1.x(需要1.12版本以上，笔者装的1.13，对应的cuda和cudnn版本一定要注意匹配，如下图）

3、激活tensorflow环境，按照如下链接进行安装

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md

按照步骤一步步来装，cocoapi也需要装好

二、准备数据集

参考下面链接

https://www.cnblogs.com/zongfa/p/9663649.html

https://blog.csdn.net/qq_17854471/article/details/89786400

三、转换成tensorflow能识别的格式

1、统计xml文件有多少类别

import xml.dom.minidom as xmldom
import os

#voc数据集获取所有标签的所有类别数"
annotation_path="/home/chenxin/下载/Annotations"

annotation_names=[os.path.join(annotation_path,i) for i in os.listdir(annotation_path)]

labels = list()
for names in annotation_names:
    xmlfilepath = names
    domobj = xmldom.parse(xmlfilepath)
    # 得到元素对象
    elementobj = domobj.documentElement
    #获得子标签
    subElementObj = elementobj.getElementsByTagName("object")
    for s in subElementObj:
        label=s.getElementsByTagName("name")[0].firstChild.data
        #print(label)
        if label not in labels:
            labels.append(label)
print(labels)

2、xml转csv格式（只需识别一个类别）

# -*- coding:utf-8 -*-
#!/usr/bin/env python
"""
    @Time: 2020/2/10 22：00
    @Author: chenxin
    @File Name: Only_one_label_xml2csv.py
    @Software: PyCharm
"""

import os
import glob
import pandas as pd
import xml.etree.ElementTree as ET

## xml文件的路径
os.chdir(‘/home/chenxin/models-master/research/object_detection/images/train1‘)
path = ‘/home/chenxin/models-master/research/object_detection/images/train1‘
img_path = ‘/home/chenxin/models-master/research/object_detection/images/train1‘


def xml_to_csv(path):
    xml_list = []
    for xml_file in glob.glob(path + ‘/*.xml‘):  # 返回所有匹配的文件路径列表即对于path目录下的每一个xml文件
        tree = ET.parse(xml_file)#获得xml对应的解析树
        root = tree.getroot()#获得根标签annotation

        for member in root.findall(‘object‘):#对于每一个object标签
           value = (root.find(‘filename‘).text,#在根标签下查找filename标签，并获得其文本信息
                   #int(root.find(‘size‘)[0].text),#在根标签下查找size标签，并获得size的第0个子标签（width）的文本信息，并转化为int
                   #int(root.find(‘size‘)[1].text),#在根标签下查找size标签，并获得size的第1个子标签（width）的文本信息，并转化为int
                   member[0].text,  # 获得object的第0个子标签(name)的文本信息
                   float(member[4][0].text),#获得object的第4个子标签(bndbox),并获得bndbox的第0个子标签（xmin)的文本信息，并转换为int
                   float(member[4][1].text),#获得object的第4个个子标签(bndbox),并获得bndbox的第1个子标签（ymin)的文本信息，并转换为int
                   float(member[4][2].text),#获得object的第4个个子标签(bndbox),并获得bndbox的第2个子标签（xmax)的文本信息，并转换为int
                   float(member[4][3].text)  #获得object的第4个个子标签(bndbox),并获得bndbox的第3个子标签（ymax)的文本信息，并转换为int


                    )
           #  value = (img_path + ‘/‘ + root.find(‘filename‘).text,
           #           int(member[1][0].text),
           #           int(member[1][1].text),
           #           int(member[1][2].text),
           #           int(member[1][3].text),
           #           member[0].text
           #           )

        xml_list.append(value)
            # column_name = [‘filename‘, ‘width‘, ‘height‘, ‘class‘, ‘xmin‘, ‘ymin‘, ‘xmax‘, ‘ymax‘]
        column_name = [‘filename‘, ‘xmin‘, ‘ymin‘, ‘xmax‘, ‘ymax‘, ‘class‘]
        xml_df = pd.DataFrame(xml_list, columns=column_name)
        return xml_df

if __name__ == ‘__main__‘:
        #for folder in [‘train‘,‘test‘]:
        #    image_path = os.path.join(os.getcwd(),(‘images/‘+folder))
        #    xml_path = xml_to_csv(image_path)
        #    xml_df.to_csv((‘images/‘+folder+‘_‘labels.csv‘),index = None
        image_path = path
        xml_df = xml_to_csv(image_path)
        ## 修改文件名称
        xml_df.to_csv((‘/home/chenxin/models-master/research/object_detection/images/train1/‘+‘train1.csv‘), index=None)
        print(‘Successfully converted xml to csv.‘)



3、xml转csv格式（需识别两个及以上类别）

# coding: utf-8
import glob
import pandas as pd
import xml.etree.ElementTree as ET

classes = ["person","hat"]


def xml_to_csv(path):
    train_list = []
    eval_list = []

    for cls in classes:
        xml_list = []
        # 读取注释文件
        for xml_file in glob.glob(path + ‘/*.xml‘):
            tree = ET.parse(xml_file)
            root = tree.getroot()
            for member in root.findall(‘object‘):
                if cls == member[0].text:
                    value = (root.find(‘filename‘).text,
                             int(root.find(‘size‘)[0].text),
                             int(root.find(‘size‘)[1].text),
                             member[0].text,
                             float(member[4][0].text),
                             float(member[4][1].text),
                             float(member[4][2].text),
                             float(member[4][3].text)
                             )
                    xml_list.append(value)

        for i in range(0, int(len(xml_list) * 0.9)):
            train_list.append(xml_list[i])
        for j in range(int(len(xml_list) * 0.9) + 1, int(len(xml_list))):
            eval_list.append(xml_list[j])

    column_name = [‘filename‘, ‘width‘, ‘height‘, ‘class‘, ‘xmin‘, ‘ymin‘, ‘xmax‘, ‘ymax‘]

    # 保存为CSV格式
    train_df = pd.DataFrame(xml_list, columns=column_name)
    eval_df = pd.DataFrame(eval_list, columns=column_name)
    train_df.to_csv(‘/home/chenxin/models-master/research/object_detection/images/data/train531.csv‘, index=None)
    eval_df.to_csv(‘/home/chenxin/models-master/research/object_detection/images/data/eval531.csv‘, index=None)


def main():
    # path = ‘E:\\\data\\\Images‘
    path = r‘/home/chenxin/models-master/research/object_detection/images/train531‘  # path参数更具自己xml文件所在的文件夹路径修改
    xml_to_csv(path)
    print(‘Successfully converted xml to csv.‘)


main()


4、csv转record格式
（终端运行版：python generate_tfrecord.py --csv_input=/home/chenxin/models-master/research/object_detection/images/data/train.csv --output_path=/home/chenxin/models-master/research/object_detection/images/data/train.record）

# generate_tfrecord.py
# -*- coding: utf-8 -*-
#!/usr/bin/env python
"""
    @Time: 2020/2/10 22：00
    @Author: chenxin
    @File Name: csv2tfrecord.py
    @Software: PyCharm
"""

"""
Usage:
  # From tensorflow/models/
  # Create train data:
  python generate_tfrecord.py --csv_input=/home/chenxin/models-master/research/object_detection/images/data/train.csv  --output_path=/home/chenxin/models-master/research/object_detection/images/data/train.record
  # Create test data:
  python generate_tfrecord.py --csv_input=/home/chenxin/models-master/research/object_detection/images/data/test.csv  --output_path=/home/chenxin/models-master/research/object_detection/images/data/test.record
"""


import os
import io
import pandas as pd
import tensorflow as tf

from PIL import Image
from object_detection.utils import dataset_util
from collections import namedtuple,OrderedDict

os.chdir(‘/home/chenxin/models-master/research/object_detection‘)

flags = tf.app.flags
flags.DEFINE_string(‘csv_input‘,‘‘,‘Path to CSV input‘)
flags.DEFINE_string(‘out_path‘,‘‘,‘Path to output TFRecord‘)
FLAGS = flags.FLAGS

def class_text_to_int(row_label):
    if row_label == ‘hat‘:
        return  1
    if row_label == ‘person‘:
        return  2
    else :
        return  3

def split(df,group):
    data = namedtuple(‘data‘,[‘filename‘,‘object‘])
    gb = df.groupby(group)
    return [data(filename,gb.get_group(x)) for filename,x in zip(gb.groups.keys(),gb.groups)]

def create_tf_example(group,path):
    with tf.gfile.GFile(os.path.join(path,‘{}‘.format(group.filename)),‘rb‘) as fid:
        encoded_jpg = fid.read()
    encoded_jpg_io = io.BytesIO(encoded_jpg)
    image = Image.open(encoded_jpg_io)
    width,height = image.size

    filename = group.filename.encode(‘utf8‘)
    image_format = b‘jpg‘
    xmins = []
    xmaxs = []
    ymins = []
    ymaxs = []
    classes_text = []
    classes = []

    for index,row in group.object.iterrows():
        xmins.append(row[‘xmin‘]/width)
        xmaxs.append(row[‘xmax‘]/width)
        ymins.append(row[‘ymin‘]/height)
        ymaxs.append(row[‘ymax‘]/height)
        classes_text.append(row[‘class‘].encode(‘utf8‘))
        classes.append(class_text_to_int(row[‘class‘]))

    tf_example = tf.train.Example(features = tf.train.Features(feature = {
        ‘image/height‘:dataset_util.int64_feature(height),
        ‘image/width‘:dataset_util.int64_feature(width),
        ‘image/filename‘:dataset_util.bytes_feature(filename),
        ‘image/source_id‘:dataset_util.bytes_feature(filename),
        ‘image/encoded‘:dataset_util.bytes_feature(encoded_jpg),
        ‘image/format‘:dataset_util.bytes_feature(image_format),
        ‘image/object/bbox/xmin‘:dataset_util.float_list_feature(xmins),
        ‘image/object/bbox/xmax‘:dataset_util.float_list_feature(xmaxs),
        ‘image/object/bbox/ymin‘:dataset_util.float_list_feature(ymins),
        ‘image/object/bbox/ymax‘:dataset_util.float_list_feature(ymaxs),
        ‘image/object/class/text‘:dataset_util.bytes_list_feature(classes_text),
        ‘image/object/class/label‘:dataset_util.int64_list_feature(classes),
    }))
    return  tf_example

def main(self):
    writer = tf.python_io.TFRecordWriter(FLAGS.output_path)
    path = os.path.join(os.grtcwd(),FLAGS.image_dir)
    examples = pd.read_csv(FLAGS.csv_input)
    grouped = split(examples,‘filename‘)
    for group in grouped:
        tf_example = create_tf_example(group,path)
        writer.write(tf_example.SerialzeToString())

    writer.close()
    output_path = os.path.join(os.getcwd(), FLAGS.output_path)
    #output_path = os.path.join(os.getcwd(),‘images/data‘)
    print(‘Successfully created the TFRecords:{}‘.format(output_path))

if __name__ == ‘__main__‘:
    tf.app.run()

四、配置文件和下载模型

1、 xml位置：/home/chenxin/models-master/research/object_detection/images/xml

2、 xml2csv文件位置：/home/chenxin/models-master/research/object_detection/images/data（自定义）

3、 csv2tfrecord文件位置：/home/chenxin/models-master/research/object_detection/images/data（自定义）

4、在官方提供的model zoo里下载训练好的模型。我们使用ssd_mobilenet_v1_coco，先下载它。

　　在 object_dection文件夹下，解压ssd_mobilenet_v1_coco.tar.gz

　　模型存放位置：mask_rcnn_inception_v2_coco：/home/chenxin/models-master/research/object_detection/

5、建立training文件夹：将ssd_mobilenet_v1_coco.config 放在training 文件夹下

　　training文件位置：/home/chenxin/models-master/research/object_detection/training

　　进行如下更改：

　　5.1、搜索其中的 PATH_TO_BE_CONFIGURED ，将对应的路径改为自己的路径;

　　　　注意最后train input reader和evaluation input reader中label_map_path必须保持一致。

　　5.2、将 num_classes 按照实际情况更改，我的例子中是1；

　　5.3、batch_size 原本是24，我在运行的时候出现显存不足的问题，为了保险起见，改为1，如果1还是出现类似问题的话，建议换电脑……

　　5.4、fine_tune_checkpoint: "ssd_mobilenet_v1_coco/model.ckpt"
　　　from_detection_checkpoint: true

(pipeline.config里面标红的都是需要更改的）

model {
  ssd {
    num_classes: 2
    image_resizer {
      fixed_shape_resizer {
        height: 300
        width: 300
      }
    }
    feature_extractor {
      type: "ssd_mobilenet_v1"
      depth_multiplier: 1.0
      min_depth: 16
      conv_hyperparams {
        regularizer {
          l2_regularizer {
            weight: 3.9999999e-05
          }
        }
        initializer {
          truncated_normal_initializer {
            mean: 0.0
            stddev: 0.029999999
          }
        }
        activation: RELU_6
        batch_norm {
          decay: 0.99970001
          center: true
          scale: true
          epsilon: 0.001
          train: true
        }
      }
      use_depthwise: true
    }
    box_coder {
      faster_rcnn_box_coder {
        y_scale: 10.0
        x_scale: 10.0
        height_scale: 5.0
        width_scale: 5.0
      }
    }
    matcher {
      argmax_matcher {
        matched_threshold: 0.5
        unmatched_threshold: 0.5
        ignore_thresholds: false
        negatives_lower_than_unmatched: true
        force_match_for_each_row: true
      }
    }
    similarity_calculator {
      iou_similarity {
      }
    }
    box_predictor {
      convolutional_box_predictor {
        conv_hyperparams {
          regularizer {
            l2_regularizer {
              weight: 3.9999999e-05
            }
          }
          initializer {
            truncated_normal_initializer {
              mean: 0.0
              stddev: 0.029999999
            }
          }
          activation: RELU_6
          batch_norm {
            decay: 0.99970001
            center: true
            scale: true
            epsilon: 0.001
            train: true
          }
        }
        min_depth: 0
        max_depth: 0
        num_layers_before_predictor: 0
        use_dropout: false
        dropout_keep_probability: 0.80000001
        kernel_size: 3
        box_code_size: 4
        apply_sigmoid_to_scores: false
        use_depthwise: true
      }
    }
    anchor_generator {
      ssd_anchor_generator {
        num_layers: 6
        min_scale: 0.2
        max_scale: 0.94999999
        aspect_ratios: 1.0
        aspect_ratios: 2.0
        aspect_ratios: 0.5
        aspect_ratios: 3.0
        aspect_ratios: 0.33329999
      }
    }
    post_processing {
      batch_non_max_suppression {
        score_threshold: 9.9999999e-09
        iou_threshold: 0.60000002
        max_detections_per_class: 100
        max_total_detections: 100
      }
      score_converter: SIGMOID
    }
    normalize_loss_by_num_matches: true
    loss {
      localization_loss {
        weighted_smooth_l1 {
        }
      }
      classification_loss {
        weighted_sigmoid {
        }
      }
      hard_example_miner {
        num_hard_examples: 3000
        iou_threshold: 0.99000001
        loss_type: CLASSIFICATION
        max_negatives_per_positive: 3
        min_negatives_per_image: 0
      }
      classification_weight: 1.0
      localization_weight: 1.0
    }
  }
}
train_config {
  batch_size: 1
  data_augmentation_options {
    random_horizontal_flip {
    }
  }
  data_augmentation_options {
    ssd_random_crop {
    }
  }
  optimizer {
    rms_prop_optimizer {
      learning_rate {
        exponential_decay_learning_rate {
          initial_learning_rate: 0.0040000002
          decay_steps: 800720
          decay_factor: 0.94999999
        }
      }
      momentum_optimizer_value: 0.89999998
      decay: 0.89999998
      epsilon: 1.0
    }
  }
  fine_tune_checkpoint: "/home/chenxin/models-master/research/object_detection/mask_rcnn_inception_v2_coco/model.ckpt"
  from_detection_checkpoint: true
  num_steps: 20000
}
train_input_reader {
  label_map_path: "/home/chenxin/models-master/research/object_detection/images/data/chenxin.pbtxt"
  tf_record_input_reader {
    input_path: "/home/chenxin/models-master/research/object_detection/images/data/train.record"
  }
}
eval_config {
  num_examples: 20
  max_evals: 10
  retain_original_images: true
}
eval_input_reader {
  label_map_path: "/home/chenxin/models-master/research/object_detection/images/data/chenxin.pbtxt"
  shuffle: false
  num_readers: 1
  tf_record_input_reader {
    input_path: "/home/chenxin/models-master/research/object_detection/images/data/eval.record"
  }
}

6、在前面xml2csv文件夹（/data）下，创建一个 xxx.pbtxt的文本文件(这个id顺序要与前面生成tfrecod的py文件代码顺序一样）

item{
  id:1
  name:‘hat‘
}
item{
  id:2
  name:‘person‘
}

五、训练模型

1、本地GPU训练（本机环境：Ubuntu 18.04），终端进入 object_detection目录下，最新版用model_main.py
2、如果是python3训练，添加list() 到 model_lib.py的大概390行 category_index.values()变成： list(category_index.values())，否则会有 can‘t pickle dict_values ERROR出现
3、准备好所有后，在终端输入如下命令：

python model_main.py --logtostderr --model_dir=/home/chenxin/model-master/research/object_detection/trainning/ --pipeline_config_path=/home/chenxin/model-master/research/object_detection/trainning/ssd_mobilenet_v1_coco.config

4、另开一个终端，同样进入到object_detection目录下，输入：

tensorboard --logdir=training

5、运行一段时间后，我们可以看到我们的training文件夹下已经有模型数据保存了，接下来就可以生成我们的需要的模型文件了，终端在object_detection目录下，输入：

python3 export_inference_graph.py --input_type image_tensor --pipeline_config_path /home/chenxin/models-master/research/object_detection/training/ssd_mobilenet_v1_coco.config --trained_checkpoint_prefix /home/chenxin/models-master/research/object_detection/training/model.ckpt-11945 --output_directory ssd_mobilenet_v1_coco

其中，trained checkpoint 后面接着的数字改为如上图里面的任意一个数字， output为想要将模型存放在何处，新建了一个文件夹xxx_detction 。运行结束后，就可以在xxx_detction文件夹下看到若干文件，有saved_model、checkpoint、frozen_inference_graph.pb等。 .pb结尾的就是最重要的frozen model.

六、测试模型

1、将object_detection目录下的object_detection_tutorial.ipynb打开

2、不用下载模型，下载相关代码可以删除，model name, path to labels , num classes 更改成自己的，download model部分都删去。

3、将训练好的模型放到自定义位置，我的文件位置：/home/chenxin/文档/ssd_mobilenet_v1_coco，测试图片，准备几张放入/home/chenxin/文档/mask_rcnn/test images文件夹中，命名images+数字.jpg的格式，就不用改代码,再在ssd_mobilenet_v1_coco里面新建data/文件夹，将xxx.pbtxt

4、一行更改自己图片的数字序列就好了，range(1,10)，我的图片命名从1至9.

# For the sake of simplicity we will use only 2 images:
# image1.jpg
# image2.jpg
# If you want to test the code with your images, just add path to the images to the TEST_IMAGE_PATHS.
PATH_TO_TEST_IMAGES_DIR = ‘/home/chenxin/文档/ssd_mobilenet_v1_coco/test_images‘
TEST_IMAGE_PATHS = [ os.path.join(PATH_TO_TEST_IMAGES_DIR, ‘image{}.jpg‘.format(i)) for i in range(1, 10) ]