Deep Learning
发展历史
Three Steps for Deep Learning
Step 1: Neural Network
Fully Connect Feedforward Network
Deep = Many hidden layers
Matrix Operation
Neural Network
FAQ
Step 2: goodness of function
Step 3: pick the best function
Gradient Descent
Backpropagation
Why Deep?（深度学习的好处）
Modularization
End-to-end Learning
Tips for Deep Learning
1. New activation function
梯度消失
新的激活函数：ReLU
Maxout（Learnable activation function）
Maxout - Training
2. Adaptive Learning Rate
Adagrad
RMSProp（加一个概率α）
Adam （RMSProp + Momentum）
Momentum（动量）
3. Early Stopping
4. Regularization（Weight Decay）
L2 regularization:
L1 regularization:
5. Dropout
Training:
Testing:
