As `β → ∞` the optimal value returns to the Chebyshev average `ψ(N)/N`.
ProvedRLHF.vonMangoldt_high_temperature_limitaether-catalognumbertheory
As β → ∞ the optimal value returns to the Chebyshev average ψ(N)/N.
theorem RLHF.vonMangoldt_high_temperature_limit{N : ℕ} (hN : 0 < N) :
haveI : Nonempty (Fin N) := Fin.pos_iff_nonempty.mp hN
Tendsto (fun β => freeEnergy β (vonMangoldtReward N) (unifRef N)) atTop
(𝓝 (chebyshevPsiFin N / (N : ℝ))) := by sorry
Formalization Note Transplanted verbatim from the Aether Catalog source NumberTheory/RLHFTemperatureLimits.lean; the statement is byte-identical to the source declaration, elaborated with autoImplicit disabled in the platform environment.
Preamble
-- Thm stub generated from NumberTheory/RLHFTemperatureLimits.lean
import Mathlib
import Definitions.Def_NumberTheory_RLHFTemperatureLimits
import Definitions.Def_NumberTheory_RLHFTemperatureSpectrum
/-!
# Zero- and infinite-temperature limits of the RLHF free energy
Continuing `NumberTheory.RLHFTemperatureSpectrum`, we quantify the two endpoints of the
free-energy spectrum `V(β) = β log Z(β)`:
* `RLHF.kl_budget` — the aligned policy cannot collapse: `β · KL(π_β ‖ p) ≤ max r − min r`.
* `RLHF.freeEnergy_ge_point` — `V(β) ≥ r y + β log p y` for every response `y`.
* `RLHF.tendsto_freeEnergy_zero_temperature` — as `β → 0⁺`, `V(β) → max r`
(greedy reward maximization).
* `RLHF.freeEnergy_le_high_temperature` and `RLHF.tendsto_freeEnergy_high_temperature`
— as `β → ∞`, `V(β) → 𝔼_p[r]` (the SFT reference), with the explicit rate
`V(β) ≤ min r + e^{(max r − min r)/β} (𝔼_p[r] − min r)`.
Arithmetic payoff (`RLHF.vonMangoldt_zero_temperature_limit`): for the von Mangoldt reward
on `{1, …, N}` the zero-temperature limit of the RLHF free energy equals `log P` where `P`
is the **largest prime ≤ N**, while the infinite-temperature limit is the Chebyshev average
`ψ(N)/N`. The whole alignment spectrum of this reward model is thus pinned between two
classical prime-counting quantities.
-/
open RLHF
open Finset ArithmeticFunction Filter Topology
variable {Ω : Type*} [Fintype Ω] [Nonempty Ω]
/-! ## 1. Pointwise lower bounds and the KL budget -/
/-! ## 2. The zero-temperature (greedy) limit -/
/-! ## 3. The high-temperature (reference) limit -/
/-! ## 4. Arithmetic endpoint: the largest prime below `N` -/Formal statement
theorem RLHF.vonMangoldt_high_temperature_limit{N : ℕ} (hN : 0 < N) :
haveI : Nonempty (Fin N) := Fin.pos_iff_nonempty.mp hN
Tendsto (fun β => freeEnergy β (vonMangoldtReward N) (unifRef N)) atTop
(𝓝 (chebyshevPsiFin N / (N : ℝ))) := by sorrySource