Neural Network
1. Model Summary
At a very simple level, neurons are basically computational units that take inputs (dendrites) as electrical inputs (called “spikes”) that are channeled to outputs (axons). In our model, our dendrites are like the input features
and the output is the result of our hypothesis function. In this model our
input node is sometimes called the “bias unit.” It is always equal to 1. In neural networks, we use the same logistic function as in classification,
, yet we sometimes call it a sigmoid (logistic) activation function. In this situation, our “theta” parameters are sometimes called “weights”.
Visually, a simplistic representation looks like:
three layers: input layer / hidden layer / output layer
: activation unit i in layer j
: Matrix that controls function mapping from j-th layer to (j+1)-th layer
If layer j has units, layer j+1 has units, then size of is
: Number of Layers
: Number of units in l-th layer
Number of Inputs: the dimension of features in
Binary Classification: 1 output unit
K-classes Classification: K output unit
2. Forward Propagation
1) Add
first
2)
3)
— g(x) : Sigmoid
3. Cost Function
Excluding Bias Term
4. Backpropagation Algorithm
error of node j in layer l, then
where
One thing to note: use one training set to train the model at one time!
5. Unrolling Parameters
Enroll them to vectors/Get back:
6.Gradient Checking
When learning, turn off gradient checking!!!
7. Random Initialization
8.Network Architecture
one hidden layer/
more than one hidden layer with same number of units