Written by joezou(邹镇洪), 2019/3/14
项目 内容 这个作业属于课程 人工智能实战 2019 - 北京航空航天大学 这个作业的要求在 Null 这个作业在这些方面帮助我实现目标 练习神经网络基本求导操作
作业正文:
1.请大家根据课堂内容和示例代码完成双变量的反向传播代码,并在博客中给出相应的结果和误差(必做)
结果:分别进行了5,4次迭代,误差分别为0.000005和0.000000
'''
practice backward tansportation for $z = (2w+3b)*(2b+1)$
initial: w = 3, b=4
target: z = 150, loss <= 1e-5
illustration: times:最大循环次数 w,b:方程参数
'''
x(w,b) = 2*w + 3*b
y(b) = 2*b + 1
def backward_1(w,b,times):
div_z = x(w,b) * y(b) - 150
# 反向求导计算第一次权值
w_w = 2 * y(b)
w_b = 3 * y(b) + 2 * x(w,b)
# 更新w,b
loop = 0
while (abs(div_z) >= 1e-5)*(loop <= times):
loop += 1
print("loop = %d, w = %.6f, b = %.6f, z = %.6f, div_z = %.6f" %(loop+1, w, b, div_z+150, div_z))
w -= div_z / w_w / 2
b -= div_z / w_b / 2
div_z = x(w,b) * y(b) - 150
if(loop <= times):
print("Done! loop = %d, w = %.6f, b = %.6f, z = %.6f, div_z = %.6f" %(loop+1, w, b, div_z+150, div_z))
else:
print("Out of time!")
def backward_2(w,b,times):
div_z = x(w,b) * y(b) - 150
loop = 0
while (abs(div_z) >= 1e-5)*(loop <= times):
# 反向求导计算每次权值
w_w = 2 * y(b)
w_b = 3 * y(b) + 2 * x(w,b)
loop += 1
print("loop = %d, w = %.6f, b = %.6f, z = %.6f, div_z = %.6f" %(loop+1, w, b, div_z+150, div_z))
w -= div_z / w_w / 2
b -= div_z / w_b / 2
div_z = x(w,b) * y(b) - 150
if(loop <= times):
print("Done! loop = %d, w = %.6f, b = %.6f, z = %.6f, div_z = %.6f" %(loop+1, w, b, div_z+150, div_z))
else:
print("Out of time!")
if __name__ == "__main__":
backward_1(3,4,100)
backward_2(3,4,100)
运行结果
loop = 1, w = 3.000000, b = 4.000000, z = 162.000000, div_z = 12.000000
loop = 2, w = 2.666667, b = 3.904762, z = 150.181406, div_z = 0.181406
loop = 3, w = 2.661628, b = 3.903322, z = 150.005526, div_z = 0.005526
loop = 4, w = 2.661474, b = 3.903278, z = 150.000170, div_z = 0.000170
Done! loop = 5, w = 2.661469, b = 3.903277, z = 150.000005, div_z = 0.000005
loop = 1, w = 3.000000, b = 4.000000, z = 162.000000, div_z = 12.000000
loop = 2, w = 2.666667, b = 3.904762, z = 150.181406, div_z = 0.181406
loop = 3, w = 2.661519, b = 3.903263, z = 150.000044, div_z = 0.000044
Done! loop = 4, w = 2.661517, b = 3.903263, z = 150.000000, div_z = 0.000000
2. 给出自己的思考和比较(选做)
- 比较:在更新w,b权值的情况下拟合速度更快,且效果更好
- 思考:这是因为我们采用了凸优化问题中的牛顿迭代的思想,当逼近最优解的时候,每次迭代的步长应随着离最优解的距离而增减,以加速拟合