目录

多标签分类

如何使用多标签分类

多标签使用实例

训练

引入库，设置超参数

设置全局参数

生成多分类的标签

切分训练集和验证集

数据增强

设置callback函数

设置模型

训练模型，并保存最终的模型

打印出训练的log

完整代码：

测试

---

多标签分类

multi-label classification problem：多标签分类（或者叫多标记分类），是指一个样本的标签数量不止一个，即一个样本对应多个标签。

如何使用多标签分类

在预测多标签分类问题时，假设隐藏层的输出是[-1.0, 5.0, -0.5, 5.0, -0.5 ]，如果用softmax函数的话，那么输出为：

z = np.array([-1.0, 5.0, -0.5, 5.0, -0.5])print(Softmax_sim(z))# 输出为[ 0.00123281  0.49735104  0.00203256  0.49735104  0.00203256]

通过使用softmax，我们可以清楚地选择标签2和标签4。但我们必须知道每个样本需要多少个标签，或者为概率选择一个阈值。这显然不是我们想要的，因为样本属于每个标签的概率应该是独立的。

对于一个二分类问题，常用的激活函数是sigmoid函数：

ps: sigmoid函数之所以在之前很长一段时间作为神经网络激活函数（现在大家基本都用Relu了），一个很重要的原因是sigmoid函数的导数很容易计算，可以用自身表示：

python 代码为：

import numpy as np

def Sigmoid_sim(x):
    return  1 /(1+np.exp(-x))

a = np.array([-1.0, 5.0, -0.5, 5.0, -0.5])
print(Sigmoid_sim(a))
#输出为： [ 0.26894142  0.99330715  0.37754067  0.99330715  0.37754067]

 

此时，每个标签的概率即是独立的。完整整个模型构建之后，最后一步中最重要的是为模型的编译选择损失函数。在多标签分类中，大多使用binary_crossentropy损失而不是通常在多类分类中使用的categorical_crossentropy损失函数。这可能看起来不合理，但因为每个输出节点都是独立的，选择二元损失，并将网络输出建模为每个标签独立的bernoulli分布。整个多标签分类的模型为：

from keras.models import Model
from keras.layers import Input,Dense

inputs = Input(shape=(10,))
hidden = Dense(units=10,activation='relu')(inputs)
output = Dense(units=5,activation='sigmoid')(hidden)
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

 

多标签使用实例

我们使用最常用的衣服数据集来实现多标签分类，网络模型使用ResNet50。

数据集地址：链接：https://pan.baidu.com/s/1eANXTnWl2nf853IEiLOvWg

提取码：jo4h

我们的数据集由5547张图片组成，它们来自12个不同的种类，包括：

• black_dress（333张图片）
• black_jeans（344张图片）
• black_shirt（436张图片）
• black_shoe（534张图片）
• blue_dress（386张图片）
• blue_jeans（356张图片）
• blue_shirt（369张图片）
• red_dress（384张图片）
• red_shirt（332张图片）
• red_shoe（486张图片）
• white_bag（747张图片）
• white_shoe（840张图片）

我们的卷积神经网络的目标是同时预测颜色和服饰类别。代码使用Tensorflow2.0以上版本编写。下面对我实现算法的代码作讲解：

训练

引入库，设置超参数

# import the necessary packages

from sklearn.preprocessing import MultiLabelBinarizer
from sklearn.model_selection import train_test_split
from imutils import paths
import tensorflow as tf
import numpy as np
import argparse
import random
import pickle
import cv2
import os
from tensorflow.python.keras.applications.resnet import ResNet50
from tensorflow.keras.optimizers import Adam
from tensorflow.python.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
from tensorflow.python.keras.preprocessing.image import ImageDataGenerator, img_to_array

# construct the argument parse and parse the arguments

ap = argparse.ArgumentParser()
ap.add_argument("-d", "--dataset", default='../dataset',
                help="path to input dataset (i.e., directory of images)")
ap.add_argument("-m", "--model", default='model.h5',
                help="path to output model")
ap.add_argument("-l", "--labelbin", default='labelbin',
                help="path to output label binarizer")
ap.add_argument("-p", "--plot", type=str, default="plot.png",
                help="path to output accuracy/loss plot")
args = vars(ap.parse_args())

超参数的解释：

• --dataset：输入的数据集路径。
• --model：输出的Keras序列模型路径。
• --labelbin：输出的多标签二值化对象路径。
• --plot：输出的训练损失及正确率图像路径。

设置全局参数

EPOCHS = 150

INIT_LR = 1e-3

BS = 16

IMAGE_DIMS = (224, 224, 3)

加载数据

print("[INFO] loading images...")

imagePaths = sorted(list(paths.list_images(args["dataset"])))

random.seed(42)

random.shuffle(imagePaths)

# initialize the data and labels

data = []

labels = []

# loop over the input images

for imagePath in imagePaths:

    # load the image, pre-process it, and store it in the data list

    image = cv2.imread(imagePath)

    image = cv2.resize(image, (IMAGE_DIMS[1], IMAGE_DIMS[0]))

    image = img_to_array(image)

    data.append(image)

    # extract set of class labels from the image path and update the

    # labels list

    l = label = imagePath.split(os.path.sep)[-2].split("_")

    labels.append(l)

# scale the raw pixel intensities to the range [0, 1]

data = np.array(data, dtype="float") / 255.0

labels = np.array(labels)

print(labels)

运行结果：

[['red' 'shirt']
 ['black' 'jeans']
 ['black' 'shoe']
 ...
 ['black' 'dress']
 ['black' 'shirt']
 ['white' 'shoe']]
 

生成多分类的标签

print("[INFO] class labels:")

mlb = MultiLabelBinarizer()

labels = mlb.fit_transform(labels)

# loop over each of the possible class labels and show them

for (i, label) in enumerate(mlb.classes_):

print("{}. {}".format(i + 1, label))

print(labels)

通过MultiLabelBinarizer()的fit就可以得到label的编码。我们将类别和生成后的标签打印出来。类别结果如下：

[INFO] class labels:
1. bag
2. black
3. blue
4. dress
5. jeans
6. red
7. shirt
8. shoe
9. white

lables的输出结果如下：

[[0 0 0 ... 1 0 0]
 [0 1 0 ... 0 0 0]
 [0 1 0 ... 0 1 0]
 ...
 [0 1 0 ... 0 0 0]
 [0 1 0 ... 1 0 0]
 [0 0 0 ... 0 1 1]]

 

为了方便大家理解标签，我通过下面的表格说明

	Bag	Black	Blue	Dress	Jeans	Red	Shirt	Shoe	White
[‘red’ ’shirt’]	0	0	0	0	0	1	1	0	0
[‘black’ ’jeans’]	0	1	0	0	1	0	0	0	0
['white' 'shoe']	0	0	0	0	0	0	0	1	1

然后，将MultiLabelBinarizer()训练的模型保存，方便测试时使用。代码如下：

print("[INFO] serializing label binarizer...")

f = open(args["labelbin"], "wb")

f.write(pickle.dumps(mlb))

f.close()

切分训练集和验证集

(trainX, testX, trainY, testY) = train_test_split(data,

                                                  labels, test_size=0.2, random_state=42)

数据增强

# construct the image generator for data augmentation

aug = ImageDataGenerator(rotation_range=25, width_shift_range=0.1,

                         height_shift_range=0.1, shear_range=0.2, zoom_range=0.2,

                         horizontal_flip=True, fill_mode="nearest")

设置callback函数

checkpointer = ModelCheckpoint(filepath='weights_best_Reset50_model.hdf5',

                               monitor='val_accuracy', verbose=1, save_best_only=True, mode='max')

 

reduce = ReduceLROnPlateau(monitor='val_accuracy', patience=10,

                           verbose=1,

                           factor=0.5,

                           min_lr=1e-6)

checkpointer的作用是保存最好的训练模型。reduce动态调整学习率。

设置模型

model = ResNet50(weights=None, classes=len(mlb.classes_))

optimizer = Adam(lr=INIT_LR)

model.compile(loss="binary_crossentropy", optimizer=optimizer,

              metrics=["accuracy"])

训练模型，并保存最终的模型

print("[INFO] training network...")

history = model.fit(

    x=aug.flow(trainX, trainY, batch_size=BS),

    validation_data=(testX, testY),

    steps_per_epoch=len(trainX) // BS,

epochs=EPOCHS, callbacks=[checkpointer, reduce], verbose=1)

# save the model to disk

print("[INFO] serializing network...")

model.save(args["model"], save_format="h5")

打印出训练的log

# plot the training loss and accuracy

loss_trend_graph_path = r"WW_loss.jpg"

acc_trend_graph_path = r"WW_acc.jpg"

import matplotlib.pyplot as plt

 

print("Now,we start drawing the loss and acc trends graph...")

# summarize history for accuracy

fig = plt.figure(1)

plt.plot(history.history["accuracy"])

plt.plot(history.history["val_accuracy"])

plt.title("Model accuracy")

plt.ylabel("accuracy")

plt.xlabel("epoch")

plt.legend(["train", "test"], loc="upper left")

plt.savefig(acc_trend_graph_path)

plt.close(1)

# summarize history for loss

fig = plt.figure(2)

plt.plot(history.history["loss"])

plt.plot(history.history["val_loss"])

plt.title("Model loss")

plt.ylabel("loss")

plt.xlabel("epoch")

plt.legend(["train", "test"], loc="upper left")

plt.savefig(loss_trend_graph_path)

plt.close(2)

print("We are done, everything seems OK...")

完整代码：

# import the necessary packages

from sklearn.preprocessing import MultiLabelBinarizer
from sklearn.model_selection import train_test_split
from imutils import paths
import tensorflow as tf
import numpy as np
import argparse
import random
import pickle
import cv2
import os
from tensorflow.python.keras.applications.resnet import ResNet50
from tensorflow.keras.optimizers import Adam
from tensorflow.python.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
from tensorflow.python.keras.preprocessing.image import ImageDataGenerator, img_to_array

# construct the argument parse and parse the arguments

ap = argparse.ArgumentParser()
ap.add_argument("-d", "--dataset", default='../dataset',
                help="path to input dataset (i.e., directory of images)")
ap.add_argument("-m", "--model", default='model.h5',
                help="path to output model")
ap.add_argument("-l", "--labelbin", default='labelbin',
                help="path to output label binarizer")
ap.add_argument("-p", "--plot", type=str, default="plot.png",
                help="path to output accuracy/loss plot")
args = vars(ap.parse_args())

# initialize the number of epochs to train for, initial learning rate,
# batch size, and image dimensions
EPOCHS = 150
INIT_LR = 1e-3
BS = 16
IMAGE_DIMS = (224, 224, 3)
# disable eager execution
tf.compat.v1.disable_eager_execution()
# grab the image paths and randomly shuffle them
print("[INFO] loading images...")
imagePaths = sorted(list(paths.list_images(args["dataset"])))
random.seed(42)
random.shuffle(imagePaths)
# initialize the data and labels
data = []
labels = []
# loop over the input images
for imagePath in imagePaths:
    # load the image, pre-process it, and store it in the data list
    image = cv2.imread(imagePath)
    image = cv2.resize(image, (IMAGE_DIMS[1], IMAGE_DIMS[0]))
    image = img_to_array(image)
    data.append(image)
    # extract set of class labels from the image path and update the
    # labels list
    l = label = imagePath.split(os.path.sep)[-2].split("_")
    labels.append(l)
# scale the raw pixel intensities to the range [0, 1]
data = np.array(data, dtype="float") / 255.0
labels = np.array(labels)
print("[INFO] data matrix: {} images ({:.2f}MB)".format(
    len(imagePaths), data.nbytes / (1024 * 1000.0)))
# binarize the labels using scikit-learn's special multi-label
# binarizer implementation
print("[INFO] class labels:")
mlb = MultiLabelBinarizer()
labels = mlb.fit_transform(labels)
# loop over each of the possible class labels and show them
for (i, label) in enumerate(mlb.classes_):
    print("{}. {}".format(i + 1, label))
print(labels)
# partition the data into training and testing splits using 80% of
# the data for training and the remaining 20% for testing
(trainX, testX, trainY, testY) = train_test_split(data,
                                                  labels, test_size=0.2, random_state=42)
print("[INFO] serializing label binarizer...")
f = open(args["labelbin"], "wb")
f.write(pickle.dumps(mlb))
f.close()
# construct the image generator for data augmentation
aug = ImageDataGenerator(rotation_range=25, width_shift_range=0.1,
                         height_shift_range=0.1, shear_range=0.2, zoom_range=0.2,
                         horizontal_flip=True, fill_mode="nearest")

checkpointer = ModelCheckpoint(filepath='weights_best_Reset50_model.hdf5',
                               monitor='val_accuracy', verbose=1, save_best_only=True, mode='max')

reduce = ReduceLROnPlateau(monitor='val_accuracy', patience=10,
                           verbose=1,
                           factor=0.5,
                           min_lr=1e-6)
model = ResNet50(weights=None, classes=len(mlb.classes_))
optimizer = Adam(lr=INIT_LR)
model.compile(loss="binary_crossentropy", optimizer=optimizer,
              metrics=["accuracy"])
# train the network
print("[INFO] training network...")
history = model.fit(
    x=aug.flow(trainX, trainY, batch_size=BS),
    validation_data=(testX, testY),
    steps_per_epoch=len(trainX) // BS,
    epochs=EPOCHS, callbacks=[checkpointer, reduce], verbose=1)
# save the model to disk
print("[INFO] serializing network...")
model.save(args["model"], save_format="h5")
# save the multi-label binarizer to disk

# plot the training loss and accuracy
loss_trend_graph_path = r"WW_loss.jpg"
acc_trend_graph_path = r"WW_acc.jpg"
import matplotlib.pyplot as plt

print("Now,we start drawing the loss and acc trends graph...")
# summarize history for accuracy
fig = plt.figure(1)
plt.plot(history.history["accuracy"])
plt.plot(history.history["val_accuracy"])
plt.title("Model accuracy")
plt.ylabel("accuracy")
plt.xlabel("epoch")
plt.legend(["train", "test"], loc="upper left")
plt.savefig(acc_trend_graph_path)
plt.close(1)
# summarize history for loss
fig = plt.figure(2)
plt.plot(history.history["loss"])
plt.plot(history.history["val_loss"])
plt.title("Model loss")
plt.ylabel("loss")
plt.xlabel("epoch")
plt.legend(["train", "test"], loc="upper left")
plt.savefig(loss_trend_graph_path)
plt.close(2)
print("We are done, everything seems OK...")

测试

# import the necessary packages
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.models import load_model
import numpy as np
import argparse
import imutils
import pickle
import cv2
import os
# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-m", "--model", default='weights_best_Reset50_model.hdf5',
	help="path to trained model model")
ap.add_argument("-l", "--labelbin", default='labelbin',
	help="path to label binarizer")
ap.add_argument("-i", "--image", default='../dataset/0.jpg',
	help="path to input image")
args = vars(ap.parse_args())
# load the image
image = cv2.imread(args["image"])
output = imutils.resize(image, width=400)

# pre-process the image for classification
image = cv2.resize(image, (224, 224))
image = image.astype("float") / 255.0
image = img_to_array(image)
image = np.expand_dims(image, axis=0)
# load the trained convolutional neural network and the multi-label
# binarizer
print("[INFO] loading network...")
model = load_model(args["model"])
mlb = pickle.loads(open(args["labelbin"], "rb").read())
# classify the input image then find the indexes of the two class
# labels with the *largest* probability
print("[INFO] classifying image...")
proba = model.predict(image)[0]
idxs = np.argsort(proba)[::-1][:2]
# loop over the indexes of the high confidence class labels
for (i, j) in enumerate(idxs):
	# build the label and draw the label on the image
	label = "{}: {:.2f}%".format(mlb.classes_[j], proba[j] * 100)
	cv2.putText(output, label, (10, (i * 30) + 25),
		cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2)
# show the probabilities for each of the individual labels
for (label, p) in zip(mlb.classes_, proba):
	print("{}: {:.2f}%".format(label, p * 100))
# show the output image
cv2.imshow("Output", output)
cv2.waitKey(0)

参考文章：

 

keras解决多标签分类问题

https://blog.csdn.net/somtian/article/details/79614570

Multi-label classification with Keras

https://www.pyimagesearch.com/2018/05/07/multi-label-classification-with-keras/