Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Theorem 1 — a bounded solution of (1) makes the minimizing rule of C'' optimal over C, with average cost g

Proved
DermanDenumerable.AvgCost.theorem1

by mikedeng1 · Oct 5, 2026 · Mathlib 0df444a (Lean v4.33.1)

average-costdynamic-programmingp2o-batch-pfp2bp2o-gran-per-chapterp2o-plan-paperp2o-v1

Assume conditions (A) and (B): the state space III is denumerable, each state iii has finitely many decisions, and the costs wikw_{ik}wik​ are bounded. Suppose a number ggg and a bounded set of numbers {vj}j∈I\{v_j\}_{j \in I}{vj​}j∈I​ satisfy

g+vi=min⁡k{wik+∑j∈Iqij(k)vj},i∈I.(1)g + v_i = \min_k \Big\{ w_{ik} + \sum_{j\in I} q_{ij}(k) v_j \Big\}, \qquad i \in I. \tag{1}g+vi​=kmin​{wik​+j∈I∑​qij​(k)vj​},i∈I.(1)

Then a rule R∗∈C′′R^* \in C''R∗∈C′′ prescribing at each state iii a decision that minimizes the right side of (1) exists, and every such rule satisfies

g=QR∗(i)≤QR(i)g = Q_{R^*}(i) \le Q_R(i)g=QR∗​(i)≤QR​(i)

for every initial state iii and every rule R∈CR \in CR∈C (history-dependent and randomized).

This is the verification theorem of the paper: a bounded solution of the average-cost optimality equation identifies the optimal average cost and an optimal stationary deterministic rule. Theorems 2, 3 and 4 all conclude by reducing to it.

Formalization Note. "R* is the rule which prescribes the decision that minimizes" is stated for every choice of minimizing decisions, as the proof allows ("if there are several minimizing decisions, let kik_iki​ be any one of them"). The bounded vvv is essential: on an infinite state space unbounded solutions of (1) do not certify optimality.

Preamble
import Mathlib
import Definitions.Def_DermanDenumerable_AvgCost_Model
open scoped Topology
open Filter SennottDP.AvgFinite
Formal statement
namespace DermanDenumerable.AvgCost

/-- Derman (1966), Theorem 1, p. 1548: under (A) and (B), if a bounded `{g, v_j}` solves (1), then a
rule `R* ∈ C''` prescribing at each state a decision minimizing the right side of (1) exists, and
every such rule satisfies `g = Q_{R*}(i) ≤ Q_R(i)` for every initial state `i` and every rule
`R ∈ C`. -/
theorem theorem1 {S Act : Type} [Countable S] (M : MDC S Act)
    (w : S → Act → ℝ) (hB : CostBounded M w) (g : ℝ) (v : S → ℝ) (hv : BddFun v)
    (h1 : IsACOE M w g v) :
    (∃ e : StationaryPolicy M, IsMinimizer M w v e) ∧
      ∀ e : StationaryPolicy M, IsMinimizer M w v e →
        ∀ i : S, avgCostR e.toPolicy w i = g ∧ ∀ R : Policy M, g ≤ avgCostR R w i := by sorry

end DermanDenumerable.AvgCost
Source
Derman, Denumerable State Markovian Decision Processes—Average Cost Criterion, Ann. Math. Statist. 37(6) (1966), p. 1548, Theorem 1
Human review
  • Endorsed by Shuze Chen · Oct 6, 2026

    Confirmed by the moderator at approval.

  • Endorsed by mikedeng1 · Oct 6, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me