Deep learning for visual understanding: A review

视觉理解中的深度学习：回顾

ABSTRACT: Deep learning algorithms are a subset of the machine learning algorithms, which aim at discovering multiple levels of distributed representations. Recently, numerous deep learning algorithms have been proposed to solve traditional artificial intelligence problems. This work aims to review the state-of-the-art in deep learning algorithms in computer vision by highlighting the contributions and challenges from over 210 recent research papers. It first gives an overview of various deep learning approaches and their recent developments, and then briefly describes their applications in diverse vision tasks, such as image classification, object detection, image retrieval, semantic segmentation and human pose estimation. Finally, the paper summarizes the future trends and challenges in designing and training deep neural networks.

摘要：深度学习算法是机器学习算法的一类，其目标是发现多层的分布式表达。最近，提出了众多的深度学习算法来解决传统人工智能问题。本文目标是从210多篇近期的研究论文中通过突出其贡献和挑战来综述最新的机器视觉中的深度学习算法。本文首先给出了众多深度学习算法的概览及其最新进展，然后简要的叙述了在不同视觉任务中的应用，比如图像分类、目标检测、图像检索、语义分割和人体姿势估计。最后，论文总结了设计和训练深度神经网络在未来的趋势和挑战。

1. Introduction  介绍

Deep learning is a subfield of machine learning which attempts to learn high-level abstractions in data by utilizing hierarchical architectures. It is an emerging approach and has been widely applied in traditional artificial intelligence domains, such as semantic parsing [1], transfer learning [2,3], natural language processing [4], computer vision [5,6] and many more. There are mainly three important reasons for the booming of deep learning today: the dramatically increased chip processing abilities (e.g. GPU units), the significantly lowered cost of computing hardware, and the considerable advances in the machine learning algorithms [9].

深度学习是机器学习领域中的一类算法，目标是采用分层的结构来学习数据中的高层抽象特征。这是一种新兴的方法，已经被广泛的应用在传统的人工智能领域，比如语义解析[1]，转移学习[2,3]，自然语言处理[4]，计算机视觉[5,6]以及其他很多方面。有三个主要的原因导致了今天深度学习的蓬勃发展：芯片计算能力的巨大提升（比如GPU单元），计算硬件价格的显著降低，和机器学习算法中的重要进步[9]。

Various deep learning approaches have been extensively reviewed and discussed in recent years [8–12]. Among those Schmidhuber et al. [10] emphasized the important inspirations and technical contributions in a historical timeline format, while Bengio [11] examined the challenges of deep learning research and proposed a few forward-looking research directions. Deep networks have been shown to be successful for computer vision tasks because they can extract appropriate features while jointly performing discrimination [9,13]. In recent ImageNet Large Scale Visual Recognition Challenge (ILSVRC) competitions [189], deep learning methods have been widely adopted by different researchers and achieved top accuracy scores [7].

近年来广泛讨论和回顾过很多深度学习方法[8-12]。其中Schmidhuber等[10]以历史时间线的顺序强调了重要的灵感和技术贡献，而Bengio[11]列举了深度学习研究的挑战并提出了几个以后的研究方向。深度网络已被证明成功应用在计算机视觉任务中，因为它们在提取适当特征的同时进行了区分[9,13]。在最新的ImageNet挑战赛中[189]，深度学习方法已被不同的研究者广泛采用，得到了最高的准确率得分[7]。

This survey is intended to be useful to general neural computing, computer vision and multimedia researchers who are interested in the state-of-the-art in deep learning in computer vision. It provides an overview of various deep learning algorithms and their applications, especially those that can be applied in the computer vision domain.

本文的调查旨在为一般的神经计算、计算机视觉和多媒体研究者中对深度学习在计算机视觉中的最新应用感兴趣的人提供帮助。本文提供了众多深度学习算法及其应用的概览，尤其是那些可以应用在计算机视觉领域中的算法。

The remainder of this paper is organized as follows:

In Section 2, we divide the deep learning algorithms into four categories: Convolutional Neural Networks, Restricted Boltzmann Machines, Autoencoder and Sparse Coding. Some well-known models in these categories as well as their developments are listed. We also describe the contributions and limitations for these models in this section. In Section 3, we describe the achievements of deep learning schemes in various computer vision applications, i.e. image classification, object detection, image retrieval, semantic segmentation and human pose estimation. The results on these applications are shown and compared in the pipeline of their commonly used datasets. In Section 4, along with the success deep learning methods have achieved, we also face several challenges when designing and training the deep networks. In this section, we summarize some major challenges for deep learning, together with the inherent trends that might be developed in the future. In Section 5, we conclude the paper.

本文组织如下：

在第二部分中，我们将深度学习算法分成四类：卷积神经网络，受限玻尔兹曼机，自动编码机和稀疏编码。列出了这些类别中一些著名的模型以及其发展。在这部分中我们还讨论了这些模型的贡献和局限性。在第三部分中，我们描述了深度学习算法在众多计算机视觉应用中的成就，包括，图像分类、目标检测、图像检索、语义分割和人类姿势估计。针对通用的数据集，展示了这些应用的结果并进行了对比。在第四部分，随着深度学习算法取得的成功，我们在设计和训练深度网络时还面临着几个挑战。在这部分中，我们总结了深度学习面临的几个主要挑战，以及将来发展过程中的内在趋势。在第五部分中，我们对本文进行了总结。