1.Bayesian Approach
-
Consider a nested sequence of model classes
P1⊂P2⊂P3⊂⋯
-
ML decision rule:
m^=argmmax{p∈Pmmaxp(y)}=argmmax{amaxpy∣x,H(y∣a,Hm)}
2. Laplace’s Method
-
连续分布
p×(x)=Zpp0(x)
-
用 taylor 级数近似似然函数
lnp0(x)≈lnp(x^)+(x−x^)dxdlnp0(x)∣∣∣∣x=x^+21(x−x^)2dx2d2lnp0(x)∣∣∣∣x=x^p0(x)≈p0(x^)exp[−21Jy=y(x^)(x−x^)2]
3. Bayes Information Criterion
- MAP decision rule:
m^=argmmaxpy∣H(y∣Hm)
其中
py∣H(y∣Hm)=∫py∣x,H(y∣x,Hm)px∣H(x∣Hm)dx
令
q0(x)=py∣x,H(y∣x,Hm)px∣H(x∣Hm)∝px∣y,H(x∣y,Hm)
可以有
py∣H(y∣H)=∫q0(x)dx≈py∣x,H(y∣x^,H)px∣H(x^∣H)2πJy−1(x^)
其中最后一项为 Occam’s razor factor
其他内容请看:
统计推断(一) Hypothesis Test
统计推断(二) Estimation Problem
统计推断(三) Exponential Family
统计推断(四) Information Geometry
统计推断(五) EM algorithm
统计推断(六) Modeling
统计推断(七) Typical Sequence
统计推断(八) Model Selection
统计推断(九) Graphical models
统计推断(十) Elimination algorithm
统计推断(十一) Sum-product algorithm