参考：登上《Cell》封面的AI医疗影像诊断系统：机器之心专访UCSD张康教授

Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning 2018-2-22 Cell

读《Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning》

没有问题就无法学习：

1. 文中的数据规模是多少？这些数据有什么特点？

2. 为什么要使用迁移学习？

3. 文章是如何验证自己模型的准确性的？

4. 摘要中的每一个结论，在文章都是如何具体论证的？

亮点：

1，首次使用如此庞大的标注好的高质量数据进行迁移学习，来建立人工智能系统；

2，基于光学相干断层成像（OCT）数据有效进行图像黄斑变性和糖尿病视网膜黄斑水肿的识别和严重性定量评估;

3，准确区分患儿胸部X光片上的细菌和病毒性肺炎（差异性分析和准确判断）

目标：

利用迁移学习识别视网膜（OCT）图像中的类别，同时也用迁移学习测试了胸部X光片的肺炎识别。

该研究也通过显示神经网络激活区域的方法向人们提供了机器诊断的可解释性。

真正启动是在去年（2017 年）初。启动到发表总共1年。

机器之心：神经网络的推理是一个「黑箱」，你们提出的新方法是如何解释计算机作出「诊断」的依据的？

张康：我们在视网膜 OCT 图像的研究中加入了「遮挡测试」——通过卷积一个遮挡核心到输入图像上，机器会通过计算预测做出正确诊断最可能的部位，并输出含有高亮色块的「遮挡」图，这些色块就是 AI「认为」的病变部位，得出直观的为临床医生信任的诊断依据。

首先，通过输入大量的数据，神经网络可以获得远超过人类医生的「经验」，计算出超越人类的准确结果，在我们的系统中，我们使用超过 20 万张医学图像，通过不同的疾病分类，最终使用近 11 万张视网膜 OCT 图像训练机器。在眼病方面，能在 30 秒内正确鉴别脉络膜新生血管、糖尿病黄斑水肿、玻璃膜疣以及正常视网膜的 OCT 图像，结果的准确率、敏感度、特异度均在 95% 以上，并能得出与人类相似甚至更高的准确率。其次，计算机对比图像像素与像素之间的差异，观察到人类关注不到的细节，从而得出更精准的判断，且不像人类一样受主观性干扰。另外，我们通过「迁移学习」这种算法，还能诊断不同系统的疾病，比如我们的系统目前还能准确鉴别肺炎和正常胸部 X 线平片，区分肺炎的病原体为细菌还是病毒，准确率可达 90% 以上。

「迁移学习」被认为是一种高效的学习技术，尤其是面临相对有限的训练数据时。相较于其他大多数学习模型的「从零开始」，「迁移学习」利用卷积神经网络（Convolutional Neural Network，CNN）学习已有的已经标记好的预训练网络系统，以医学图像学习为例，该系统会识别预系统中图像的特点，我们再继续导入含有第一层图像相似参数和结构的网络系统，最终构建出终极层级。在我们的系统中，第一层网络就是视网膜 OCT 图像，第二级网络系统使用第一级的图像寻找相应的特点，通过前向传播固定低层图像中的权重，找到已经学习的可辨别的结构，再提取更高层的权重，在其中进行反复的自我调整和反馈、传递，达到学习区分特定类型的图像的目的。我们首次使用如此庞大的标注好的高质量视网膜 OCT 数据进行迁移学习，进行常见视网膜致盲性疾病的检测及推荐治疗手段，得到与人类医生相似甚至更高的准确性。此人工智能系统还可以「举一反三」，将迁移学习用于小儿肺炎诊断。

迁移学习是深度学习的一个自然发展方向，迁移学习能让深度学习变得更加可靠，还能帮我们理解深度学习的模型。比如，我们能够知晓哪部分特征容易迁移，这些特征所对应的是某个领域比较高层、抽象的一些结构型概念。把它们的细节区分开，就能让我们对这个领域的知识表达形成一个更深的理解。这样一来，机器就可以像生物的神经系统一样终身学习，不断地对过去的知识进行总结、归纳，让一个系统越学越快，而且在学习过程中还能发现如何学习。

迁移学习在深度学习上面有极为广阔的应用前景，在图像数据资源有限的医疗领域，更高效、所需图像数量更少的迁移学习，可以说是未来 5 年内 AI 发展的热点以及深度学习成功应用的驱动力。

比如，有一些图像特征较为模糊的图像，如老年黄斑变性，某些较大的玻璃膜疣和脉络膜新生血管非常相似，我们就会偏向于采取更为严重的疾病诊断，因为我们研究的最终目的是帮助病人更可能的推荐给相应的专科医生，从而更快的获得治疗。另外，我们还可以通过我们的想法设定更为贴合实际的过滤器，并按照我们临床医生的需求不断调整；通过「遮挡实验」能够反映机器得出判断的依据。并且，我们的研究还能指导治疗方案的确定。因此我们的研究可能更能达到临床医生想要的效果，并且为临床医生所信任，也许能更快更直接的应用于临床。

学习以下神经网路，适合中国人的视频，简直不要太通俗：迁移学习 Transfer Learning

学习方法及资料：https://github.com/jindongwang/transferlearning

Cell paper数据和代码地址：https://data.mendeley.com/datasets/rscbjbr9sj/2

跑一下代码玩一下！

文章原文解读：

Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning

鉴定医疗诊断和可治疗疾病

Image-based deep learning classifies macular degeneration（[眼科] 黄斑变性） and diabetic retinopathy（[眼科] 糖尿病视网膜病变） using retinal optical coherence tomography（视网膜光学相干断层扫描） images and has potential for generalized applications in biomedical image interpretation and medical decision making.

• An artificial intelligence system using transfer learning（迁移学习） techniques was developed
• It effectively classified images for macular degeneration and diabetic retinopathy
• It also accurately distinguished bacterial and viral pneumonia（细菌和病毒性肺炎） on chest X-rays
• This has potential for generalized high-impact application in biomedical imaging

科普下迁移学习：什么是迁移学习 (Transfer Learning)？这个领域历史发展前景如何？

The implementation of clinical-decision support algorithms for medical imaging faces challenges with reliability and interpretability.

Here, we establish a diagnostic tool based on a deep-learning framework for the screening of patients with common treatable blinding retinal diseases.

Our framework utilizes transfer learning, which trains a neural network with a fraction of the data of conventional approaches（常规方法）. Applying this approach to a dataset of optical coherence tomography images, we demonstrate performance comparable to that of human experts in classifying agerelated macular degeneration and diabetic macular edema. We also provide a more transparent and interpretable diagnosis by highlighting the regions recognized by the neural network. We further demonstrate the general applicability of our AI system for diagnosis of pediatric pneumonia using chest X-ray images.

This tool may ultimately aid in expediting the diagnosis and referral of these treatable conditions, thereby facilitating earlier treatment, resulting in improved clinical outcomes.

Figure 2. Representative Optical Coherence Tomography Images and the Workflow Diagram

结果

The primary application of our transfer learning algorithm was in the diagnosis of retinal OCT images.

Spectral-domain OCT uses light to capture high-resolution in vivo optical cross sections of the retina that can be assembled into three-dimensional-volume images of living retinal tissue.  OCT is critical to guiding the administration of anti-VEGF therapy by providing a clear cross-sectional representation of the retinal pathology in these conditions (Figure 2A), allowing visualization of individual retinal layers, which is impossible with clinical examination by the human eye or by color fundus photography.

主要就干了以下四件事：

Patient and Image Characteristics

数据：We initially obtained 207,130 OCT images. 108,312 images (37,206 with choroidal neovascularization, 11,349 with diabetic macular edema, 8,617 with drusen, and 51,140 normal) from 4,686 patients passed initial image quality review and were used to train the AI system.

After 100 epochs (iterations through the entire dataset), the training was stopped due to the absence of further improvement in both accuracy (Figure 3A) and cross-entropy loss (Figure 3B).

Performance of the Model

We evaluated our AI system in diagnosing the most common blinding retinal diseases.

Comparison of the Model with Human Experts

An independent test set of 1,000 images from 633 patients was used to compare the AI network’s referral decisions with the decisions made by human experts.

Occlusion Testing遮挡测试

We performed an occlusion test on 491 images to identify the areas contributing most to the neural network’s assignment of the predicted diagnosis.

Application of the AI System for Pneumonia Detection Using Chest X-Ray Images

To investigate the generalizability of our AI system in the diagnosis of common diseases, we applied the same transfer learning framework to the diagnosis of pediatric pneumonia.

---

案例二：In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images

• transmitted-light
• fluorescence images

计算机中的标注：预测荧光标签在没被标注的图像中

in-silico-labeling

Seeing More with In Silico Labeling of Microscopy Images

Multi-scale architectures  ：可以用于处理多重输入，平时我们都是输入一张图片，现在我们可以输入多张图片。

In silico labeling, a machine-learning approach, reliably infers fluorescent measurements from transmitted-light（透射光） images of unlabeled fixed or live biological samples.

Microscopy is a central method in life sciences.

Many popular methods, such as antibody labeling, are used to add physical fluorescent labels to specific cellular constituents.

However, these approaches have significant drawbacks, including inconsistency; limitations in the number of simultaneous labels because of spectral overlap; and necessary perturbations of the experiment, such as fixing the cells, to generate the measurement.

Here, we show that a computational machine-learning approach, which we call ‘‘in silico labeling’’ (ISL), reliably predicts some fluorescent labels from transmitted-light images of unlabeled fixed or live biological samples. ISL predicts a range of labels, such as those for nuclei, cell type (e.g., neural), and cell state (e.g., cell death). Because prediction happens in silico, the method is consistent, is not limited by spectral overlap, and does not disturb the experiment. ISL generates biological measurements that would otherwise be problematic or impossible to acquire.

As such, it is unclear whether deep learning approaches would provide a significant and broad-based advance in image analysis and are capable of extracting useful, not readily apparent, information from unlabeled images.

we show it can accurately predict the location and texture of cell nuclei, the health of a cell, the type of cell in a mixture, and the type of subcellular structure.

We also show that the trained network exhibits transfer learning: once trained to predict a set of labels, it could learn new labels with a small number of additional data, resulting in a highly generalizable algorithm, adaptable across experiments.

Training and Testing Datasets for Supervised Machine Learning

Developing Predictive Algorithms with Machine Learning

Network Predictions of Cell Nuclei

Network Predictions of Cell Viability

Network Predictions of Cell Type and Subcellular Process Type

Adapting the Generic Learned Network to New Datasets: Transfer Learning

待续~