图解机器学习-前六章总结

本文包含前六章所有未添加遗漏的知识点，以及对前六章涉及的所有知识的一个综合性的总结。

2.1线性模型

> rand('state',0);randn('state',0);
n=50;N=1000;x=linspace(-3,3,n)';X=linspace(-3,3,N)';
pix=pi*x;y=sin(pix)./(pix)+0.1*x+0.05*randn(n,1);
p(:,1)=ones(n,1);P(:,1)=ones(N,1);
for j=1:15
p(:,2*j)=sin(j/2*x);
p(:,2*j+1)=cos(j/2*x);
P(:,2*j)=sin(j/2*X);
P(:,2*j+1)=cos(j/2*X);
end
>> t=p\y;
>> F1=P*t;
>> figure(1);clf;hold on;axis([-2.8 2.8 -0.5 1.2]);
>> plot(x,y,'bo');
>> plot(X,F1,'g-');

>> rand('state',0);randn('state',0);
n=50;N=1000;x=linspace(-3,3,n)';X=linspace(-3,3,N)';
pix=pi*x;y=sin(pix)./(pix)+0.1*x+0.05*randn(n,1);
hh=2*0.3^2;t0=randn(n,1);e=0.1;
for o=1:1000
i=ceil(rand*n);
ki=exp(-x-x(i).^2/hh);
ki=exp(-(x-x(i)).^2/hh);
t=t0-e*ki*(ki'*t0-y(i));
if norm(t-t0)<0.000001,break,end
t0=t;
end
K=exp(-(repmat(X.^2,1,n)+repmat(x.^2',N,1)-2*X*x')/hh);
F=K*t;
figure(1);clf;hold on;axis([-2.8 2.8 -0.5 1.2]);
plot(X,F,'g-');
plot(x,y,'bo');
>>

梯度下降初始：

梯度下降迭代50次效果

梯度下降迭代100次效果

梯度下降迭代150次效果

梯度下降迭代1000次效果

>> rand('state',0);randn('state',0);
n=50;N=1000;x=linspace(-3,3,n)';X=linspace(-3,3,N)';
pix=pi*x;y=sin(pix)./(pix)+0.1*x+0.05*randn(n,1);
p(:,1)=ones(n,1);P(:,1)=ones(N,1);
for j=1:15
p(:,2*j)=sin(j/2*x);
p(:,2*j+1)=cos(j/2*x);
P(:,2*j)=sin(j/2*X);
P(:,2*j+1)=cos(j/2*X);
end
t=p\y;
F1=P*t;
>> t2=(p*diag([ones(1,11) zeros(1,20)]))\y;F2=P*t2;
警告: 秩亏，秩 = 11，tol =  7.850462e-14。 
 
>> figure(1);clf;hold on;axis([-2.8 2.8 -0.8 1.2]);
>> plot(X,F1,'g-');plot(X,F2,'r--');plot(x,y,'bo');
>>

>>  rand('state',0);randn('state',0);
n=50;N=1000;x=linspace(-3,3,n)';X=linspace(-3,3,N)';
pix=pi*x;y=sin(pix)./(pix)+0.1*x+0.05*randn(n,1);
>> x2=x.^2;X2=X.^2;hh=2*0.3^2;l=0.1;
>> k=exp(-(repmat(x2,1,n)+repmat(x2',n,1)-2*x*x')/hh);
>> K=exp(-(repmat(X2,1,n)+repmat(x2',N,1)-2*X*x')/hh);
>> t1=k\y;
>> F1=K*t1;
>> t2=(k^2+l*eye(n))\(k*y);
>> F2=K*t2;
>> figure(1);clf;hold on;
>> axis([-2.8 2.8 -1 1.5]);
>> plot(X,F1,'g-');
>> plot(X,F2,'r--');
>> plot(x,y,'bo');
>>

> rand('state',0);randn('state',0);
n=50;N=1000;x=linspace(-3,3,n)';X=linspace(-3,3,N)';
pix=pi*x;y=sin(pix)./(pix)+0.1*x+0.05*randn(n,1);
>> x2=x.^2;xx=repmat(x2,1,n)+repmat(x2',n,1)-2*x*x';
>> hhs=2*[0.03 0.3 3].^2;ls=[0.0001 0.1 100];
>> m=5;
>> u=floor(m*[0:n-1]/n)+1;
>> u=u(randperm(n));
>> for hk=1:length(hhs)
hh=hhs(hk);k=exp(-xx/hh);
for i=1:m
ki=k(u~=i,:);kc=k(u==i,:);yi=y(u~=i);yc=y(u==i);
for lk=1:length(ls)
l=ls(lk);t=(ki'*ki+l*eye(n))\(ki'*yi);
fc=kc*t;
g(hk,lk,i)=mean((fc-yc).^2);
end,end,end
>> [gl,ggl]=min(mean(g,3),[],2);[ghl ,gghl]=min(gl);
>> L=ls(ggl(gghl));HH=hhs(gghl);
>> K=exp(-(repmat(X.^2,1,n)+repmat(x2',N,1)-2*X*x')/HH);
>> k=exp(-xx/HH);
>> t=(k^2+L*eye(n))\(k*y);
>> F=K*t;
>> figure(1);clf;hold on;axis([-2.8 2.8 -0.7 1.7]);
>> plot(X,F,'g-');
>> plot(x,y,'bo');

未完待续.....to be continued!

使用L1稀疏化之后的数据真的有13个存在，有37个为0值。

-0.394943671729658       1
-1.39338502420644e-14    0
-0.000216387862515260    0
1.98069219617068e-06     0
6.56876527416437e-10     0
3.28062719017625e-07     0
2.46010964496706e-08     0
1.36290741394177e-07     0
-1.63247733755858e-11    0
-6.58898687952171e-14    0
-1.20514520945141e-11    0
-0.188323307346287       2
-0.0980031170554334      3
-1.53215450267066e-05    0
-6.19950402657318e-05    0
-2.79486051434933e-10    0
-3.99354569403842e-11    0
-1.28846135506103e-09    0
1.66702857004136e-14     0
1.09101544113061e-13     0
1.47989100113565e-06     0
0.000683592418516382     0
0.288306370075619        4
0.0183788785782052       5
0.419611253522179        6
0.528762149669693        7
0.000426063669160926     0
4.80873845140238e-09     0
4.00168586803585e-10     0
1.59295327643168e-06     0
0.116161192164022        8
0.275461063088218        9
1.24769125246363e-05     0
7.99427392654602e-06     0
1.12876992522508e-07     0
1.16185503038789e-11     0
-2.84657395980191e-11    0
-1.59582481153204e-05    0
-0.00147702546757086     10
-0.0808426721733398      11
-7.20269627546754e-07    0
-5.09433533853781e-11    0
2.37781464377136e-12     0
5.93898574415400e-11     0
0.332074870952451        12
0.0739794152593247       13
1.29197098402304e-07     0
2.06947748382139e-07     0
2.86510243361181e-06     0
7.39855166146442e-08     0

没有稀疏化的矩阵L2最小二乘法的数据表示为下，50个数据就是50：

-0.220328583978388
-0.162115131799812
-0.0836043085659182
-0.0149713084719319
0.0372908554835615
0.0715834614360994
0.0707003802941173
0.0219409748218776
-0.0514080720867747
-0.0999204950825484
-0.0933031345034189
-0.0499006553217768
-0.0184393827223330
-0.0276299002726696
-0.0545689395036345
-0.0520871388764327
-0.00855141571338680
0.0346374015016090
0.0335295490978071
-0.00227690687677037
-0.0157041311764674
0.0399447176655191
0.156398537409784
0.276846241154809
0.333995749818333
0.289432805454773
0.164299613803733
0.0398186831130090
-0.00112724366606240
0.0474470879749669
0.110621612439549
0.118021498437080
0.0740943922827923
0.0320205190064917
0.0205610340580442
0.0211996883601706
0.00796356295312131
-0.0188293785983502
-0.0439986529934651
-0.0590639688541648
-0.0578427504434347
-0.0258493626216256
0.0455744558402019
0.133146213474800
0.180954566754125
0.141026649000196
0.0288685861385103
-0.0653362228978380
-0.0456863729331597
0.0955861153656629

 rand('state',0);randn('state',0);
>> n=50; N=1000; x=linspace(-3,3,n)'; X=linspace(-3,3,N)';
pix=pi*x; y=sin(pix)./(pix)+0.1*x+0.2*randn(n,1);

hh=2*0.3^2; l=0.1; t0=randn(n,1); x2=x.^2;
k=exp(-(repmat(x2,1,n)+repmat(x2',n,1)-2*x*x')/hh);
k2=k^2; ky=k*y;
for o=1:1000
  t=(k2+l*pinv(diag(abs(t0))))\ky;
  if norm(t-t0)<0.001, break, end
  t0=t;
end
K=exp(-(repmat(X.^2,1,n)+repmat(x2',N,1)-2*X*x')/hh);
F=K*t;

figure(1); clf; hold on; axis([-2.8 2.8 -1 1.5]);
plot(X,F,'g-'); plot(x,y,'bo');

huber效果出奇的好，我定义了两个异值点，但是未循环和循环之后的结果居然是一样的。。。。。

 rand('state',0);randn('state',0);
n=10;N=1000;x=linspace(-3,3,n)';X=linspace(-3,3,N)';
y=x+0.2*randn(n,1);y(4)=-4; y(5)=5;
p(:,1)=ones(n,1);p(:,2)=x;t0=p\y;e=1;
>> for o=1:3
r=abs(p*t0-y); w=ones(n,1); w(r>e)=e./r(r>e);
t=(p'*(repmat(w,1,2).*p)\(p'*(w.*y)));
if norm(t-t0)<0.001,break,end
t0=t;
end
>> P(:,1)=ones(N,1);P(:,2)=X;F=P*t;
>> figure(1);clf;hold on; axis([-4 4 -4.5 3.5]);
plot(X,F,'g-');plot(x,y,'bo');

一次未循环得到的结果

循环了三次之后的结果

 rand('state',0);randn('state',0);
n=50;N=1000;x=linspace(-3,3,n)';X=linspace(-3,3,N)';
pix=pi*x;y=sin(pix)./(pix)+0.1*x+0.2*randn(n,1);
y(n/2)=-0.5;
>> hh=2*0.3^2;l=0.1;e=0.1;t0=randn(n,1);x2=x.^2;
>> k=exp(-(repmat(x2,1,n)+repmat(x2',n,1)-2*x*x')/hh);
 for o=1:1000
r=abs(k*t0-y);w=ones(n,1);w(r>e)=e./r(r>e);
Z=k*(repmat(w,1,n).*k)+l*pinv(diag(abs(t0)));
 t=(Z+0.000001*eye(n))\(k*(w.*y));
if norm(t-t0)<0.001,break,end
t0=t;
end
 K=exp(-(repmat(X.^2,1,n)+repmat(x2',N,1)-2*X*x')/hh);
>> F=K*t;
>> figure(1);clf;hold on;axis([-2.8 2.8 -1 1.5]);
>> plot(X,F,'g-');plot(x,y,'bo');

图解机器学习-前六章总结

猜你喜欢