本文为第一次构建神经网络系列第四篇
探讨如何反向传播计算梯度
系列第一篇:https://blog.csdn.net/qq_37385726/article/details/81740386
系列第二篇:https://blog.csdn.net/qq_37385726/article/details/81742247
系列第三篇:https://blog.csdn.net/qq_37385726/article/details/81744802
系列第四篇:https://blog.csdn.net/qq_37385726/article/details/81745510
系列第五篇:https://blog.csdn.net/qq_37385726/article/details/81748635
目录
1.预构建网络
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
# 1 input image channel, 6 output channels, 5*5 square convolution
# kernel
self.conv1 = nn.Conv2d(in_channels=1, out_channels=32, kernel_size=5, stride=1, padding=2)
self.conv2 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=5, stride=1, padding=2)
# an affine operation: y = Wx + b
self.fc1 = nn.Linear(64 * 8 * 8, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 10)
def forward(self, x):
# max pooling over a (2, 2) window
x = self.conv1(x)
x = F.max_pool2d(F.relu(x), (2, 2)) #32*16*16
# If size is a square you can only specify a single number
x = F.max_pool2d(F.relu(self.conv2(x)), 2) #64*8*8
x = x.view(-1, self.num_flat_features(x))
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
def num_flat_features(self, x):
size = x.size()[1:] # all dimensions except the batch dimension
num_features = 1
for s in size:
num_features *= s
return num_features
net = Net()
网络结构
Net(
(conv1): Conv2d(1, 32, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
(conv2): Conv2d(32, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
(fc1): Linear(in_features=4096, out_features=120, bias=True)
(fc2): Linear(in_features=120, out_features=84, bias=True)
(fc3): Linear(in_features=84, out_features=10, bias=True)
)
2.向网络传入输入,得到输出
- 传入输入的方式
将输入的variable作为参数传入到net中,即net(input)
- 得到输出的方式
输出即为net(input)调用后的返回值
input = Variable(torch.Tensor(1,1,32,32), requires_grad = True)
out = net(input) #将输入作为参数传入网络返回值即为输出
print(out)
输出为 tensor([[-0.1163, 0.0099, 0.0055, -0.0484, 0.1090, -0.0102, -0.1381, 0.0693,
-0.0400, -0.0166]], grad_fn=<ThAddmmBackward>)
3.构建损失函数,计算误差
损失函数定义在nn中,在这里我们使用MSELoss()即最小均方误差作为损失函数
Simply&& Generall
在回归问题上,我们使用 nn.MSELoss() 作为损失函数
在分类问题上,我们使用 nn.CrossEntropyLoss 作为损失函数
先记住一点,在神经网络计算中接受的都是variable而不是tensor,所以在构建target的时候要包装成variable
构造
#Model预测输出
input = Variable(torch.rand(1,1,32,32)) #根据上述网络定义in_channels=1,所以输入为1,shape(32,32)
out = net(input) #将输入作为参数传入网络返回值即为输出
print('out:\n',out)
#对于input目标输出
target = torch.arange(1.0,11.0)
target = Variable(target,requires_grad = True)
print('\n\ntarget:\n',target)
#损失函数
#定义损失函数,选择MSE最小均方误差作为损失函数
loss_func = nn.MSELoss()
#计算误差,误差即为利用损失函数计算out和target间的误差
loss = loss_func(out, target)
print('\n\nloss:\n',loss)
输出
out:
tensor([[-0.0423, 0.0614, 0.0607, 0.0778, 0.0255, -0.0705, 0.0049, 0.0573,
-0.1050, 0.0537]], grad_fn=<ThAddmmBackward>)
target:
tensor([ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10.], requires_grad=True)
loss:
tensor(384.3105, grad_fn=<SumBackward0>)
4. 反向传播,计算梯度
关于反向传播,我们使用backward函数:
预知识可以查看: Pytorch之浅入backward
看过之后,在想为什么要反向传播的,为什么要计算梯度,为什么要清零的,自行了解神经网络BP算法
#基于loss,反向传播,计算梯度
net.zero_grad() #将梯度均置0
loss.backward()
print('\n\ngrad:\n',input.grad)
输出 (loss对于input的梯度)
grad:
tensor([[[[ 1.4667e-02, 1.0938e-02, -1.9200e-03, ..., 3.0458e-02,
-2.6207e-02, 4.9422e-04],
[-1.4665e-02, 1.6973e-02, 3.3222e-03, ..., -1.9088e-02,
-3.2410e-02, 5.6499e-03],
[ 1.6416e-02, -2.5757e-02, 1.3336e-02, ..., -3.5478e-02,
1.8651e-02, -2.6204e-02],
...,
[ 9.0965e-03, -4.8741e-03, -1.5085e-02, ..., -5.3854e-03,
3.5103e-02, -7.2583e-03],
[ 1.5662e-02, 2.1850e-03, 7.3939e-03, ..., -6.1694e-03,
-1.7021e-03, 3.7368e-03],
[ 1.2588e-02, 2.3279e-02, 1.4130e-02, ..., 2.2284e-03,
2.8160e-02, 5.8273e-04]]]])