线性回归函数求导

$模型：h_\theta (x_i) =\theta_1 x_i^1 + \theta_2 x_i^2 + ...... + \theta_j x_i^j = \theta^T x_i\\ 损失函数： J(\theta) = \frac {1}{2m} \sum^m_{i = 1} (h_\theta (x_i) - y_i)^2\\ 求导(复合函数求导链式法则)： \frac {\partial J(\theta)} {\partial_{\theta_j}} =2 \times [ \frac {1}{2m} \sum^m_{i = 1} (h_\theta (x_i) - y_i)] \times [x_i^j]\\ = \frac {1}{m} \sum^m_{i = 1} (h_\theta (x_i) - y_i) x_i^j\\ [(预测值 - 实际值) \times x_i ^j]的期望$

logistics函数求导

返回顶部
$sigmoid函数： g(z) = \frac{1} {1 + e^{-z}}\\ sigmoid求导： \frac{\partial{g(z)}}{\partial_z} = -\frac {e^{-z} \times (-1)}{(1+e^{-z})^2} = \frac {e^{-z}}{(1+e^{-z})^2} = g(z) \times (1 - g(z))\\ 模型： h_\theta(x_i) = g(\theta^T x_i) = \frac{1}{1+e^{-\theta^T x_i}}\\ 损失函数：J(\theta) = -\frac{1}{m} \sum^m_{i = 1} (y_i log(h_\theta(x_i) ) + (1 - y_i) log(1 - h_\theta(x_i)))\\ 或者：J(\theta) = -\frac{1}{m} \sum^m_{i = 1} (y_i log(g(z)) + (1 - y_i) log(1 - g(z)))\\ （其中：\frac { \partial J(\theta)}{\partial g(z)} = -\frac{1}{m} \sum^m_{i = 1} ( \frac{y_i}{g(z)} + \frac{1 - y_i}{1 - g(z)} \times(-1) )= -\frac{1}{m} \sum^m_{i = 1} \frac {y_i - g(z)}{g(z) \times (1 - g(z))}) \\$

复合函数链式求导 ：
$\frac{ \partial J(\theta)}{\partial_{\theta_j}} = \frac { \partial J(\theta)}{\partial g(z)} \times \frac { \partial g(z)}{\partial z} \times \frac { \partial z}{\partial_{\theta_j}}\\ =[ -\frac{1}{m} \sum^m_{i = 1} \frac {y_i - g(z)}{g(z) \times (1 - g(z))}] \times [g(z) \times (1 - g(z))] \times [-x_i^j]\\ = \frac{1}{m} \sum^m_{i = 1} (g(z) - y_i) x_i^j\\ = \frac{1}{m} \sum^m_{i = 1} (h_\theta(x_i) - y_i) x_i^j\\ [(预测值 - 实际值) \times x_i ^j]的期望$

sigmoid函数是logistics单分类问题，softmax函数是logistics多分类问题。
$softmax: h_\theta(x^{(i)}) = \begin{bmatrix} p(y^{(i)} = 1| x^{(i)}; \theta)\\ p(y^{(i)} = 2| x^{(i)}; \theta)\\ .\\ .\\ .\\ p(y^{(i)} = k| x^{(i)}; \theta)\\ \end{bmatrix} = \frac{1}{\sum_{j = 1}^k e^{\theta_j^T x^{(i)}}} \begin{bmatrix} e^{\theta_1^T x^{(i)}}\\ e^{\theta_2^T x^{(i)}}\\ .\\ .\\ .\\ e^{\theta_k^T x^{(i)}}\\ \end{bmatrix}$

结论：

机器学习中线性回归和logistics回归的损失函数求导结果，均为：
$\frac{1}{m} \sum^m_{i = 1} (h_\theta(x_i) - y_i) x_i^j\\ [(预测值 - 实际值) \times x_i ^j]的期望$
形式上统一！

码砖不忘忧国！
望两岸早日实现实质统一！

欢迎关注，敬请点赞！
返回顶部