(4.6)–(4.8), proof of Theorem 4.1, p. 18 — the Fenchel coupling to a dominated strategy grows at least linearly
ProvedReinfRegGames.Extinction.eq_4_8Consider a finite game, a penalty function for every player , and an orbit of the reinforcement learning dynamics (RL). Fix a player and two mixed strategies , and let be a real number such that
Let be the cross-coupling (4.6). Then for every ,
With this is (4.8). When is strictly dominated by , and the Fenchel coupling between and the score of player diverges to , which by Proposition C.4 drives play away from .
Formalization Note The minimum is the largest admissible , so stating the bound for every admissible is equivalent to stating it for , without a real infimum. Here is the expected payoff of player when is replaced by . The orbit is an orbit of (RL) in the differential form of the Model definition.
import Mathlib import Definitions.Def_agt_games import Definitions.Def_ReinfRegGames_Extinction_Model
namespace ReinfRegGames.Extinction
theorem eq_4_8 {ι : Type*} [Fintype ι] [DecidableEq ι] {A : ι → Type*}
[∀ k, Fintype (A k)] [∀ k, DecidableEq (A k)]
(u : ι → (∀ k, A k) → ℝ) (h : ∀ k, (A k → ℝ) → ℝ) (K : ι → ℝ)
(hpen : ∀ k, IsPenalty (h k) (K k))
(y x : ℝ → ∀ k, A k → ℝ) (horb : IsRLOrbit u h y x)
(k : ι) (p p' : A k → ℝ) (hp : AGT.IsLottery p) (hp' : AGT.IsLottery p') (δ : ℝ)
(hδ : ∀ z : ∀ ℓ, A ℓ → ℝ, AGT.IsMixedProfile z →
δ ≤ AGT.expectedPayoff u (Function.update z k p') k -
AGT.expectedPayoff u (Function.update z k p) k) :
∀ t : ℝ, 0 ≤ t →
(h k p - h k p' - y 0 k ⬝ᵥ (p - p')) + δ * t ≤ fenchelCoupling (h k) p (y t k) := by sorry
end ReinfRegGames.Extinction
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.