Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Formalpedia

Theorem 6.3, p. 12 — on the n-state chain MDP every first-order method has ‖v_s − v*‖∞ ≥ λ^s/(1 + λ) for s = 1, …, n − 1

Proved
AccelVI.LowerBound.theorem_6_3

by mikedeng1 · Oct 15, 2026 · Mathlib 0df444a (Lean v4.33.1)

complexityfirst-order-methodslower-boundmarkov-decision-processp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1value-iteration

Let n∈Nn \in \mathbb{N}n∈N and λ∈(0,1)\lambda \in (0,1)λ∈(0,1), and consider the nnn-state chain MDP of Figure 1: one action, state 1 absorbing with reward r1=1r_1 = 1r1​=1, and for i≥2i \ge 2i≥2 a deterministic transition from iii to i−1i - 1i−1 with reward 000. Let TTT be its Bellman operator (2.1) and v∗v^*v∗ its fixed point, T(v∗)=v∗T(v^*) = v^*T(v∗)=v∗. Let (vs)s≥0(v_s)_{s \ge 0}(vs​)s≥0​ be any sequence of iterates with

v0=0,vs+1∈span⁡{v0,…,vs,T(v0),…,T(vs)},s≥0.v_0 = 0, \qquad v_{s+1} \in \operatorname{span}\{v_0, \dots, v_s, T(v_0), \dots, T(v_s)\}, \quad s \ge 0 .v0​=0,vs+1​∈span{v0​,…,vs​,T(v0​),…,T(vs​)},s≥0.

Then for every step s∈{1,…,n−1}s \in \{1, \dots, n-1\}s∈{1,…,n−1},

∥vs−v∗∥∞≥λs1+λ.\|v_s - v^*\|_\infty \ge \frac{\lambda^s}{1 + \lambda} .∥vs​−v∗∥∞​≥1+λλs​.

So no first-order method, in the sense of iterates built from linear combinations of past iterates and their Bellman images, can converge faster than at rate λ\lambdaλ uniformly over MDPs, which is the rate of plain value iteration. Acceleration as in convex optimization, which improves λ\lambdaλ to roughly 1−1−λ1 - \sqrt{1 - \lambda}1−1−λ​, is therefore impossible for general MDPs.

Formalization Note. The paper states "there exists an MDP instance" with nnn its number of states; its proof fixes an arbitrary nnn and an arbitrary λ∈(0,1)\lambda \in (0,1)λ∈(0,1) and uses the chain of Figure 1. The statement here is that proof's claim, for every nnn and λ\lambdaλ on the explicit chain; the printed existential follows from it. States are 0-based (Fin n), "span" is the real linear span, v∗v^*v∗ enters through the fixed-point equation T(v∗)=v∗T(v^*) = v^*T(v∗)=v∗, ∥⋅∥∞\|\cdot\|_\infty∥⋅∥∞​ is the sup norm, and n−1n - 1n−1 is natural-number subtraction (so the statement has content for n≥2n \ge 2n≥2).

Preamble
import Mathlib
import Definitions.Def_AccelVI_LowerBound_Setting
Formal statement
namespace AccelVI.LowerBound
theorem theorem_6_3 (n : ℕ) (lam : ℝ) (h0 : 0 < lam) (h1 : lam < 1) (vstar : Fin n → ℝ)
    (hfix : bellman (chainMDP n lam) vstar = vstar) (v : ℕ → Fin n → ℝ)
    (hv : IsFirstOrder (bellman (chainMDP n lam)) v) (s : ℕ) (hs1 : 1 ≤ s) (hs2 : s ≤ n - 1) :
    lam ^ s / (1 + lam) ≤ ‖v s - vstar‖ := by sorry
end AccelVI.LowerBound
Source
Goyal & Grand-Clément, A First-Order Approach to Accelerated Value Iteration, arXiv:1905.09963v7, p. 12, Theorem 6.3 and its proof, Figure 1
Human review
  • Endorsed by Shuze Chen · Oct 15, 2026

    Confirmed by the moderator at approval.

  • Endorsed by mikedeng1 · Oct 15, 2026

    Confirmed by the mission captain (proposal self-audit).

View graph

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me