§9, p. 25 — positive lower bound on the growth coefficient
ProvedTDApprox.Sampling.delta_lower_boundcounterexampleinequalityp2o-batch-pfp2ap2o-gran-per-chapterp2o-plan-paperp2o-v1temporal-difference
Suppose , , , , and with . For the coefficient in the specialized mean recurrence,
Thus the mean update has a strictly positive growth factor whenever the step size is positive. The result isolates the paper's numerical threshold .
Formalization Note The explicit value witnesses the paper's assertion that some positive exists.
Preamble
import Mathlib import Definitions.Def_TDApprox_Sampling_Model
Formal statement
namespace TDApprox.Sampling
theorem delta_lower_bound
(α p₁ p₂ q₁ q₂ : ℝ)
(hα0 : 0 < α) (hα1 : α < 1)
(hp₁ : 0 ≤ p₁) (hp₂ : 0 ≤ p₂) (hpsum : p₁ + p₂ ≤ 1)
(hpbig : 5 / (6 * α) < p₂)
(hq₂ : 0 ≤ q₂) (hqle : q₂ ≤ q₁) (hq₁ : 0 < q₁) :
(6 * α * p₂ - 5) * q₁ ≤
(α * p₁ + 2 * α * p₂ - 1) * q₁ +
2 * (α * p₁ + 2 * α * p₂ - 2) * q₂ ∧
0 < 6 * α * p₂ - 5 := by sorry
end TDApprox.Sampling
Source
Tsitsiklis & Van Roy, LIDS-P-2322 (1996), Theorem 3 proof, p. 25, Δ_t lower bound; https://dspace.mit.edu/entities/publication/ab395d25-a6d3-407a-9589-60eaac58fd05
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.