The free energy principle (FEP) proposes that a self-organizing system — a brain, an organism, an agent — persists by minimizing one quantity: the variational free energy of its sensory states under an internal generative model. Introduced by Karl Friston as a principle of brain function [Friston 2006] and stated in its unified form [Friston 2010], the principle makes a precise mathematical claim at its core: whatever internal state estimate the system holds, the free energy of incoming data is never below the data's surprisal (negative log marginal likelihood), and the excess is exactly the Kullback–Leibler divergence between the system's recognition density and the Bayesian posterior implied by the model. Active inference extends the same functional from perception to action and planning [Friston et al. 2017], and the same bound is known in machine learning as the evidence lower bound (ELBO) of variational inference [Parr et al. 2022].
Timeline of the mathematical content this mission formalizes:
fep_formal (Active Inference Institute): a machine-checked Lean 4 catalogue of 155 Free Energy Principle topics, compiled with zero proof holes against a pinned Mathlib. This mission transcribes the catalogue's core-free-energy chain — topic fep-002 and the foundation module active_inference — onto the platform, turning the first link of the FEP development into solvable community infrastructure.Everything is finite, and laws are normalized real mass functions.
A finite law on a finite type is a function with for every and . A finite kernel from to assigns to each a normalized row over . The mission's definitions Def_fep_finite_laws and Def_fep_finite_information package these carriers with entropy, cross-entropy, and the KL divergence
a totalized real-valued divergence that is finite even at zero-mass atoms (the convention via Real.negMulLog) and nonnegative on normalized laws.
A finite generative model for active inference (definition Def_fep_generative_model) over finite types consists of: an initial state law ; a policy-conditioned transition kernel; a state-to-outcome likelihood kernel; a preference law over outcomes; and a policy prior. Under a policy the model predicts the state law and the outcome law . A recognition density is any finite law over states — the system's internal estimate. At an outcome with positive predicted mass, the Bayesian posterior is the exact finite Bayes rule. The outcome surprisal is , and the posterior-form variational free energy of a recognition density is
For every generative model, every policy , every outcome with , and every recognition density :
The recognition density is universally quantified — the bound holds for whatever state estimate the system happens to carry.
Three companions pin down the equality case, ordered weakest to strongest alongside the milestone list:
Independently of the finite model, in Mathlib's nonnegative extended reals , with , measures on any measurable space and :
the unconditional shape of the bound (topic fep-002 of the source catalogue), with the divergence taken as when the log-likelihood ratio is not integrable.
The result itself. This inequality is the load-bearing step of the FEP: it converts "minimize free energy" into "move recognition toward the posterior," and it is the exact statement whose continuous, dynamic, and policy-selecting extensions (expected free energy, Markov blankets, non-equilibrium thermodynamics) form the rest of the FEP literature. Without it, the principle's variational step has no mathematical content.
Formalizing it. The mathematics here is classical — Gibbs' inequality — and the source development already proves every row with no proof holes. What the mission adds is faithful, reusable infrastructure: the definitions are published as platform nodes in the shared namespace FreeEnergyPrinciple, so later missions in this programme (expected free energy and policy selection, Markov blankets, Gaussian and continuous-time variants, already proved in the source repository) can import them instead of re-deriving the substrate. Status honesty: all eight items below are formalized and machine-checked locally against the platform environment; each is an open problem on the platform only in the sense that no proof has yet been submitted to it.
The bound itself is a one-line consequence of KL nonnegativity — the naive idea "prove it by simp on the KL sum" is essentially right, and the source proofs are correspondingly short. The actual difficulty is boundary precision, where plausible renderings go silently wrong:
A vacuous formalization — quantifying over a single distinguished recognition law, or taking "posterior" as an arbitrary variable — would trivialize the goal; the targets below rule this out by fixing the exact finite Bayes rule and universally quantifying .
Committed conventions of this mission's Lean development:
Fintype); laws are -valued normalized mass functions; kernels are normalized rows. No measure-theoretic machinery below the finite substrate except for the measure-theoretic core milestone.Real.negMulLog, so exactly, not by exception-handling.FiniteKernel.posterior, taken at the explicit hypothesis .FreeEnergyPrinciple; definitions live in the published definition files Def_fep_finite_laws, Def_fep_finite_information, Def_fep_generative_model, and every theorem item imports them. A Free Energy Principle II mission (expected free energy) is expected to reuse the same namespace and definitions.InformationTheory.klDiv in ℝ≥0∞ with no finiteness hypotheses.namespace FreeEnergyPrinciple
theorem variational_free_energy_ge_surprisal
{Policy State Outcome : Type*} [Fintype Policy] [Fintype State]
[Fintype Outcome]
(model : GenerativeModel Policy State Outcome) (policy : Policy)
(outcome : Outcome) (h : 0 < predictedOutcome model policy outcome)
(recognition : FiniteLaw State) :
outcomeSurprisal model policy outcome ≤
variationalFreeEnergy model policy outcome h recognition := by sorry
end FreeEnergyPrincipleThe variational free-energy bound of the free energy principle, in posterior form, over a finite generative model.
Fix a finite generative model over finite types (initial state law, policy-conditioned transitions, state-to-outcome likelihood, preferences, policy prior — definitions Def_fep_finite_laws, Def_fep_finite_information, Def_fep_generative_model). For a policy and an outcome with positive predicted mass , let be the exact Bayesian posterior state law and let the outcome surprisal be . For any recognition density — a normalized finite law over states — the posterior-form variational free energy is
The theorem asserts, for every recognition density :
Whatever internal state estimate the system carries, the variational free energy of the data is never below the data's surprisal; the excess is exactly the KL divergence from the recognition density to the Bayesian posterior. This is the goal theorem of the mission — the finite, model-relative form of the free energy principle's core inequality (surprisal bound in the sense of Friston 2010; posterior-form identity in the sense of Parr–Pezzulo–Friston 2022).
Formalization Note — the positivity hypothesis is explicit because the Bayesian posterior is defined only there; KL is the totalized real-valued finite KL of Def_fep_finite_information, nonnegative with no support assumptions; zero-mass atoms are handled by the exact convention. Transcribed from the proved theorem outcomeSurprisal_le_variationalFreeEnergy of FepSketches.active_inference in the fep_lean formalization; compiled against the platform environment.
No open leaves. Every sub-goal is proved or awaiting decomposition.