版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/welcom_/article/details/84581844
做ex_2的时候,碰到一个求梯度公式,在此手推一波. 如下:
δθjδJ(θ)=m1i=1∑m(hθ(x)(i)−y(i))xj(i)
- 前提:cost function
J(θ)=m1i=1∑m[−y(i)log(hθ(x(i)))−(1−y(i))log(1−hθ(x(i)))]hypothesis
hθ(x(i))=g(x(i)θ)Logistic function
g(x)=1+exp(−x)1,exp(x)=ex
- 推导过程:
δθjδJ(θ)=−m1(y(i)hθ(x(i))hθ′(x(i))+(1−y(i))1−hθ(x(i))−hθ′(x(i)))
=−m1hθ(x(i))(1−hθ(x(i)))y(i)hθ′(x(i)(1−hθ(x(i)))−(1−y(i))hθ′(x(i))hθ(x(i))
=−m1hθ(x(i))(1−hθ(x(i)))(y(i)−hθ(x(i)))hθ′(x(i))...............(1)
设H(x)=x(i)θ,则hθ′(x(i))=−(1+e−H(x))2eH(x)H′(x).......(2)
hθ(x(i))(1−hθ(x(i)))=(1+e−H(x))2eH(x)..........(3)
H′(x)=θj(i)................(4)
将(2)(3)(4)代入(1)得δθjδJ(θ)=m1xj(i)(hθ(x)(i)−y(i))xj(i)