Word likelihood
ProvedNestedCRP.word_likelihoodbayesian-nonparametricsp2o-gran-per-chapterp2o-plan-paperp2o-v1topic-models
Fix a tree, level allocations and a document . Then
where is the probability of the words of the other documents and
Here and count the words in the node of at level , with and without those of . This is the likelihood factor of Eq. (4).
Differences from the source:
- encoding: the conditional p(w_d | c, w_{-d}, z, η) → a product identity between word masses; reason: equivalent to the conditional, and needs no division
- encoding: levels and sticks are 1-based → levels and sticks are 0-based; reason: Lean indexing; level ℓ here is the paper's level ℓ + 1, and no instance changes
- encoding: a vocabulary of V words, nonempty implicitly → [NeZero V], that is V ≥ 1; reason: nondegeneracy only: at V = 0 the symmetric Dirichlet on the empty simplex is undefined and the Γ(Vη) = Γ(0) factors of the path weights are poles; no instance of the source is excluded
Preamble
import Mathlib import Definitions.Def_NestedCRP_Hyper import Definitions.Def_NestedCRP_pathWeight import Definitions.Def_NestedCRP_topicMass
Formal statement
namespace NestedCRP
/-- The likelihood `p(w_d | c, w_{-d}, z, η)` of §5.2: the probability of the corpus is the product
over the levels up to `max(z_d)` of the ratios of Dirichlet normalizers, times the probability of
the words of the other documents. Blei–Griffiths–Jordan, *The nested Chinese restaurant process and Bayesian nonparametric
inference of topic hierarchies*, J. ACM 57(2), Article 7 (2010), §5.2, second display, p.7:17. -/
theorem word_likelihood {V D : ℕ} [NeZero V] (H : Hyper) (C : Corpus V D) (T : Tree D)
(z : C.Levels) (d : Fin D) :
topicMass H.eta C T z =
(∏ ℓ ∈ Finset.range (Finset.univ.sup (z d) + 1), levelRatio H.eta C z T d ℓ) *
topicMassOn H.eta C T z (Finset.univ.filter fun s => s.1 ≠ d) := by sorry
end NestedCRP
Source
Blei–Griffiths–Jordan, The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies, J. ACM 57(2), Article 7 (2010), https://doi.org/10.1145/1667053.1667056, §5.2, second display, p.7:17
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.