多项分布
假设一次实验有 K 种结果，每种结果对应的概率分别为： p 1 , … , p K p_1, \ldots, p_K p1,…,pK，且满足：
p 1 + p 2 + ⋯ + p K = 1 p_1 + p_2 + \cdots + p_K = 1 p1+p2+⋯+pK=1
则进行 M 次实验，每种结果分别出现 m 1 , … , m K m_1, \ldots, m_K m1,…,mK 的概率为：
P ( m 1 , … , m K ; p ) = C M m 1 p 1 m 1 ⋅ C M − m 1 m 2 p 2 m 2 ⋯ C m K m K p K m K = ( C M m 1 C M − m 1 m 2 ⋯ C m K m K ) ( p 1 m 1 p 2 m 2 ⋯ p K m K ) = M ! m 1 ! m 2 ! … m K ! p 1 m 1 p 2 m 2 ⋯ p K m K \begin{aligned} P(m_1,\ldots,m_K; \mathbf{p}) =&C_M^{m_1} p_1^{m_1} \cdot C_{M-m_1}^{m_2} p_2^{m_2} \cdots C_{m_K}^{m_K} p_K^{m_K} \\\\ =& ( C_M^{m_1} C_{M-m_1}^{m_2} \cdots C_{m_K}^{m_K}) (p_1^{m_1} p_2^{m_2}\cdots p_K^{m_K}) \\\\ =& \frac{M!}{m_1!m_2!\ldots m_K!}p_1^{m_1} p_2^{m_2}\cdots p_K^{m_K} \end{aligned} P(m1,…,mK;p)===CMm1p1m1⋅CM−m1m2p2m2⋯CmKmKpKmK(CMm1CM−m1m2⋯CmKmK)(p1m1p2m2⋯pKmK)m1!m2!…mK!M!p1m1p2m2⋯pKmK
狄利克雷分布
狄利克雷分布记为：
D i r ( p ⃗ ; α ⃗ ) = 1 Δ ( α ⃗ ) ∏ i = 1 K p i α i − 1 Dir(\vec{p};\vec{\alpha}) = \frac{1}{\Delta(\vec{\alpha}) }\prod_{i=1}^Kp_i^{\alpha_i-1} Dir(p;α)=Δ(α)1i=1∏Kpiαi−1
其中 Δ ( α ⃗ ) \Delta(\vec{\alpha}) Δ(α) 为归一化常数，由分布的性质，可想而知：
∫ ∑ i = 1 K p i = 1 D i r ( p ⃗ ; α ⃗ ) d p ⃗ = 1 \int_{\sum_{i=1}^Kp_i=1} Dir(\vec{p};\vec{\alpha}) d\vec{p} = 1 ∫∑i=1Kpi=1Dir(p;α)dp=1
即
Δ ( α ⃗ ) = ∫ ∑ i = 1 K p i = 1 ∏ i = 1 K p i α i − 1 d p ⃗ \Delta(\vec{\alpha}) = \int_{\sum_{i=1}^Kp_i=1} \prod_{i=1}^Kp_i^{\alpha_i-1} d\vec{p} Δ(α)=∫∑i=1Kpi=1i=1∏Kpiαi−1dp
Dirichlet-Multinomial 共轭
假设多项分布的参数 p ⃗ \vec{p} p 的先验分布为狄利克雷分布： p ⃗ ∼ D i r ( p ⃗ ; α ⃗ ) \vec{p} \sim Dir(\vec{p};\vec{\alpha}) p∼Dir(p;α)
然后又做了 N = ∑ i = 1 K n i N = \sum_{i=1}^K n_i N=∑i=1Kni 次实验(记为事件Z)，每种结果分别出现 n 1 , … , n K n_1, \ldots, n_K n1,…,nK 次，求多项分布的后验概率：
P ( p ⃗ ∣ Z ) = P ( p ⃗ , Z ) P ( Z ) = P ( Z ∣ p ⃗ ) P ( p ⃗ ) ∫ P ( Z ∣ p ⃗ ) P ( p ⃗ ) d p ⃗ = N ! n 1 ! n 2 ! … n K ! p 1 n 1 p 2 n 2 ⋯ p K n K 1 Δ ( α ⃗ ) ∏ i = 1 K p i α i − 1 ∫ N ! n 1 ! n 2 ! … n K ! p 1 n 1 p 2 n 2 ⋯ p K n K 1 Δ ( α ⃗ ) ∏ i = 1 K p i α i − 1 d p ⃗ = ∏ i = 1 K p i n i + α i − 1 ∫ ∏ i = 1 K p i n i + α i − 1 d p ⃗ = ∏ i = 1 K p i n i + α i − 1 Δ ( α ⃗ + n ⃗ ) \begin{aligned} P(\vec{p} | Z) &= \frac{P(\vec{p} ,Z)}{P(Z)} \\\\ &= \frac{P(Z|\vec{p})P(\vec{p})}{\int P(Z|\vec{p})P(\vec{p}) d\vec{p}} \\\\ &= \frac{ \frac{N!}{n_1!n_2!\ldots n_K!}p_1^{n_1} p_2^{n_2}\cdots p_K^{n_K} \frac{1}{\Delta(\vec{\alpha}) }\prod_{i=1}^Kp_i^{\alpha_i-1}}{\int \frac{N!}{n_1!n_2!\ldots n_K!}p_1^{n_1} p_2^{n_2}\cdots p_K^{n_K} \frac{1}{\Delta(\vec{\alpha}) }\prod_{i=1}^Kp_i^{\alpha_i-1} d\vec{p}} \\\\ &= \frac{\prod_{i=1}^Kp_i^{n_i+\alpha_i-1}}{\int \prod_{i=1}^Kp_i^{n_i+\alpha_i-1} d\vec{p}} \\\\ &= \frac{\prod_{i=1}^Kp_i^{n_i+\alpha_i-1}}{\Delta(\vec{\alpha}+\vec{n})} \end{aligned} P(p∣Z)=P(Z)P(p,Z)=∫P(Z∣p)P(p)dpP(Z∣p)P(p)=∫n1!n2!…nK!N!p1n1p2n2⋯pKnKΔ(α)1∏i=1Kpiαi−1dpn1!n2!…nK!N!p1n1p2n2⋯pKnKΔ(α)1∏i=1Kpiαi−1=∫∏i=1Kpini+αi−1dp∏i=1Kpini+αi−1=Δ(α+n)∏i=1Kpini+αi−1
即后验分布服从新的狄利克雷分布： D i r ( p ⃗ ; α ⃗ + n ⃗ ) Dir(\vec{p};\vec{\alpha}+\vec{n}) Dir(p;α+n)