DNN的滤波嫁接：Filter Grafting for Deep Neural Networks， CVPR 2020
问题： DNNs have unimportant filters. These filters limit the potential of DNNs since they are identified as having little effect on the network. While filter pruning removes these invalid filters for efficiency consideration, filter grafting re-activates them from an accuracy boosting perspective.
DNN含有一些无效的波滤器，这些波滤器影响了网络性能。当前的剪枝方法直接把这些不重要的波滤器删除，本文的思路非常不同，使用嫁接的思路把它们重新激活。
方法： The activation is processed by grafting external information into invalid filters. To better perform the grafting process, we develop an entropy-based criterion to measure the information of filters and an adaptive weighting strategy for balancing the grafted information among networks. After the grafting operation, the network has very few invalid filters compared with its untouched state, enpowering the model with more representation capacity.
本方法把外部信息嫁接到无效波滤器中。为更好的进行嫁接，我们设计了基于熵的评价准则来度量波滤器中的信息。同时，设计了自适应权重的策略来平衡嫁接到新网络中的信息。嫁接以后，网络将没有无效波滤器，表达能力将显著提高。
效果： Grafted MobileNetV2 outperforms the non-grafted MobileNetV2 by about 7 percent on CIFAR-100 dataset.
嫁接后，MobileNetV2 在CIFAR100数据集上性能提升7%。
代码地址： https://github.com/fxmeng/filter-grafting
语义分割中的动态路由：Learning Dynamic Routing for Semantic Segmentation， CVPR 2020
问题： Previous works intend to handle inputs with various scales in pre-defined static architectures, such as FCN, U-Net, and DeepLab series.
当前的语义分割方法（FCN, U-Net， DeepLab等）处理多种尺度的输入图像时只能在静态的框架下工作。
方法： This paper studies a conceptually new method to alleviate the scale variance in semantic representation, named dynamic routing. The proposed framework generates data-dependent routes, adapting to the scale distribution of each image. To this end, a differentiable gating function, called soft conditional gate, is proposed to select scale transform paths on the fly. In addition, the computational cost can be further reduced in an end-to-end manner by giving budget constraints to the gating function.
为缓解语义分割中的输入图像尺度问题，本文提出了dynamic routing。本方法可以针对输入数据的分布生成网络路径。为了达到这一目标，我们设计soft conditional gate方法用于忙碌的选择尺度转换路径。此外，通过对gating进行限制，计算复杂度可以进一步降低。
效果： Extensive experiments are conducted on Cityscapes and PASCAL VOC 2012 to illustrate the effectiveness of the dynamic framework.
本方法在Cityscapes和PASCAL VOC 2012数据集上改进了分割性能。
代码地址： https://github.com/yanwei-li/DynamicRouting
对抗鲁棒性：Adversarial Robustness: From Self-Supervised Pretraining to Fine-Tuning， CVPR 2020
问题： Pretrained models from self-supervision are prevalently used in fine-tuning downstream tasks faster or for better accuracy. However, gaining robustness from pretraining is left unexplored.
基于自监督的预训练模型经常用于fine-tune下游任务。但是，这些模型的鲁棒性未被有效挖掘。
方法： We introduce adversarial training into self-supervision, to provide general-purpose robust pretrained models for the first time. We find these robust pretrained models can benefit the subsequent fine-tuning in two ways: (1) boosting final model robustness; (2) saving the computation cost.
本文首次将对抗训练引入自监督，来提供泛化能力更强的预训练模型。本文模型用于下游任务时有两个好处：（1）提高了最终模型的鲁棒性；（2）降低了计算复杂度。
效果： The proposed framework achieves large performance margins (e.g., 3.83% on robust accuracy on the CIFAR-10 dataset). Moreover, we find that different self-supervised pretrained models have diverse adversarial vulnerability. It inspires us to ensemble several pretraining tasks, which boosts robustness more. Our ensemble strategy contributes to a further improvement of 3:59% on robust accuracy, while maintaining a slightly higher standard accuracy on CIFAR-10
本方法在CIFAR10上取得了3.83%的性能改善。此外，我们还发现不同的预训练模型含有不同的对抗特性。因此，我们把多个预训练模型集成在一起，在CIFAR10上进一步取得了3.59%的性能提升。
代码地址： https://github.com/TAMU-VITA/Adv-SS-Pretraining