Motivation:

The lack of transparency of the deep  learning models creates key barriers to establishing trusts to the model or effectively troubleshooting classification errors

Common methods on non-security applications:

forward propagation / back propagation / under a blackbox setting

the basic idea is to approximate the local decision boundary using a linear model to infer the important features.

Insights:

A mixture regression model : can approximate both linear and non-linear decision boundaries

Fused Lasso: a panalty term commonly used for capturing frature dependency.

By adding fused lasso to the learning process, the mixture regression model can take features as a group and thus capture the dependency between adjacent features.

Evaluations:

classifying PDF malware: trained on 10000 PDF files

detecting the function start to reverse-engineer  binary code.

Innovation:

Under a  black-box setting :

Give an input data instance x and a classifier such as an RNN,  identify a small set of features that have key contributions to the classification of x.