Deep Learning for NLP

Lecture 2:Introduction to Teano

enter link description here

Neural Networks can be expressed as one long function of vector and matrix operations.

(神经网络可以表示为一个向量和矩阵运算的长函数。)

Common Frameworks(常用框架)

• C/C++
  
  if you need maximum performance,start from scratch(如果你需要最高性能，那么就从头开始编程吧:)
• Matlab
• Caffe
  
  Ported Matlab’s implementation nets to C.(Caffe是纯粹的C++/CUDA架构，支持命令行、Python和MATLAB接口)
  
  Mainly used for machine-vision(主要用于机器视觉。)
• Torch:
  
  Based on Lua,(基于Lua,)
  
  Used by a lot of companies(Google Deep Mind,Facebook,IBM)(被很多公司所使用，包括google深度大脑，facebook，IBM等等)
• Theano
  
  Python based framework(python基本的框架)
  
  Main framework used in the research community(主要框架被用于研究团队)
  
  For comparison: http://fastml.com/torch-vs-theano/

Introduction to Theano

Advantage(优点)

• Python library with tight integration of Numpy(python库Numpy的紧密集成)
  
  Easy syntax for matrix operations(简单的矩阵运算语法)
• Transparent use of GPU(speed-up of up to 140x)(透明的GPU使用(加速140x))
• Efficient symbolic differentiation(Theano computes the gradient)(有效的符号微分(Theano计算梯度))
• Speed and stability optimizations(速度和稳定性优化)
• Calculations are dynamically mapped to C code(计算动态映射到C代码)
  
  We do our computations as fast as we would have written it in C (我们有跟我们用C代码写出来的程序一样快的运算速度)
  
  Great performance(>10 faster than Java in my experiments)(伟大的性能(比在我的实验中用Java还要快10倍))

Disadvantages(缺点)

• Debugging is really hard(调试很困难)

Some note on the installation of Theano(Theano安装的一些注意事项)

• Theano utilizes BLAS (Basic Linear Algebra Subprograms)(Theano利用BLAS(基本线性代数的子程序))
  
  Building blocks for fast vector and matrix operations(速度向量和矩阵运算的构建块)
  
  Often written in Fortran, sometimes in Assembler(经常用Fortran编写，但有时候用Assembler)
• For performance optimization install a BLAS package(为优化性能安装一个BLAS包)
• Benchmark different BLAS packages(不同BLAS包的基准不同)
• I use a manually compiled OpenBlas implementation(我使用一个手动编译OpenBlas来实现)
  
  Installation notes: http://deeplearning.net/software/theano/install_ubuntu.html

Theano – Flow(Theano流程图)

The execution of a Theano script is a bit different(Theano脚本的执行略有不同)

Theano-Computation Graph(Theano计算图)

import theano
import theano.tensor as T

a=T.fvector()
b=T.fvector()
c=T.fmatrix()

res1=a+b
res2=T.dot(res1,c)