Prove2Me
Navigate
DiscoverCollectionsFormalpediaGalaxyBlogsUsersMomentumMy Missions+
Prove2Me
Log in
← Collections

Convex Optimization

Convex optimization textbooks, chapter by chapter: KKT conditions, conic duality, barrier methods, and the complexity of first-order methods from center of gravity to mirror descent.

23 missions

Missions

1–23 of 23
Open0Completed23All23
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: Shuze Chen

Convex Optimization VI: Self-Concordance and the Barrier MethodTextbook

Why do interior-point methods solve convex programs in O(mlog⁡(1/ε))O(\sqrt{m}\log(1/\varepsilon))O(m​log(1/ε)) Newton steps? Nesterov and Nemirovskii's answer is self-concordance: a convex function whose third derivative is controlled by its second, ∣φ′′′(t)∣≤2 φ′′(t)3/2|\varphi'''(t)| \le 2\,\varphi''(t)^{3/2}∣φ′′′(t)∣≤2φ′′(t)3/2 along every line, admits a Newton analysis with absolute constants and no condition number —…

20 thms6 active users
🏆Completed
Convex OptimizationMachine LearningOptimization+1·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity XV: SVRG with η = 1/(10β) and k = 20κ Contracts the Expected Optimality Gap by 0.9 per EpochTextbook

Many optimization problems in machine learning minimize an average of losses, one loss for each observation. A full gradient step examines every observation, while a stochastic gradient step examines one. The latter is cheaper per step, but its sampled gradient can remain noisy even near the optimum. Section 6.3 of Bubeck's monograph studies stochastic…

14 thms5 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity III: Projected Subgradient Descent with η = R/(L√t) Satisfies f(average) − f(x*) ≤ RL/√tTextbook

Many convex optimization problems in machine learning and statistics have objectives that are convex but not differentiable: hinge losses, ℓ1\ell_1ℓ1​ penalties, maxima of finitely many affine functions, and the dual functions of Lagrangian relaxations. Methods that rely on gradients do not apply to them directly, while cutting-plane methods such as the…

15 thms5 active users
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity II: For t ≥ 2n² log(R/r) the Ellipsoid Method Satisfies f(x_t) − min f ≤ (2BR/r)·exp(−t/(2n²))Textbook

The ellipsoid method is the cutting-plane algorithm that settled the polynomial-time solvability of linear programming and, more generally, of convex optimization over any set that comes with an efficient separation oracle. It was introduced for convex minimization by Shor and by Yudin and Nemirovski in the 1970s, and Khachiyan used it in 1979 to give the…

11 thms5 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: Shuze Chen

Convex Optimization III: Conic Duality and the S-procedureTextbook

Two quadratic functions can be compared losslessly. The S-procedure says that, when the constraint is strictly feasible, the implication

17 thms5 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: Shuze Chen

Convex Optimization I: Prékopa's TheoremTextbook

Log-concave functions are the meeting point of convex analysis and probability: densities of Gaussian, exponential, uniform and Wishart distributions are all log-concave, and countless facts of applied probability flow from one structural theorem — integrating out variables preserves log-concavity. This mission builds the convex-analysis spine of Boyd &…

29 thms5 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity V: Projected Gradient Descent on a Convex β-Smooth Function Has Rate (3β‖x₁ − x*‖² + f(x₁) − f(x*))/tTextbook

Many optimization problems in statistics, machine learning and operations research ask to minimize a smooth convex function over a simple convex set: a box, a ball, a simplex, a cone of positive semidefinite matrices. Projected gradient descent is the most basic first-order method for such problems. It takes a gradient step and then returns to the feasible…

12 thms4 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity I: The Center of Gravity Method Satisfies f(x_t) − min f ≤ 2B(1 − 1/e)^{t/n}Textbook

Black-box convex optimization asks how many queries to an oracle are needed to minimize a convex function to accuracy ε\varepsilonε. In fixed dimension nnn the answer is of order nlog⁡(1/ε)n\log(1/\varepsilon)nlog(1/ε), and the first algorithm to attain it is the center of gravity method, discovered independently by Levin (1965) and Newman (1965). It is the opening example of cutt…

9 thms4 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: Shuze Chen

Convex Optimization V: Newton's MethodTextbook

The classical convergence theory of smooth convex minimization. For a function that is mmm-strongly convex and MMM-smooth (mI⪯∇2f(x)⪯MImI \preceq \nabla^2 f(x) \preceq MImI⪯∇2f(x)⪯MI), gradient descent converges linearly, while Newton's method exhibits its famous two phases: a damped phase in which every backtracking step decreases the objective by a fixed amount γ\gammaγ, and a quadrati…

12 thms4 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: Shuze Chen

Convex Optimization II: KKT ConditionsTextbook

The Karush–Kuhn–Tucker conditions are the central result of convex optimization: for a convex differentiable problem satisfying Slater's condition, a point is optimal exactly when primal feasibility, dual feasibility, complementary slackness and Lagrangian stationarity hold. This mission formalizes Chapters 4–5 of Boyd & Vandenberghe end to end — the…

15 thms4 active usersReviewed
🏆Completed
CombinatoricsConvex OptimizationOptimization+1·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity XVII: Goemans–Williamson Rounding of the MAXCUT SDP Relaxation Has Expected Value at Least 0.878 Times the Maximum CutTextbook

MAXCUT asks for a partition of the vertices of a weighted graph into two sets that maximizes the total weight of the edges between them. It is one of Karp's original NP-hard problems, so no polynomial-time exact algorithm is expected, and the natural question is how close a polynomial-time algorithm can come to the optimum. Sampling a uniformly random…

6 thms3 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization+1·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity XVI: Random Coordinate Descent RCD(γ) on a Strongly Convex Coordinate-Smooth Function Has Rate (1 − 1/κ_γ)^tTextbook

When a problem has millions of variables, even one full gradient can be too expensive to compute, while a single partial derivative ∂f/∂xi\partial f/\partial x_i∂f/∂xi​ is often cheap: in regularized regression, support vector machines and many structured problems, updating one coordinate costs a small fraction of a full gradient step. Coordinate descent methods exploit…

7 thms3 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization+1·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity XIV: Stochastic Mirror Descent on a β-Smooth Function with Noise σ Has Rate Rσ√(2/t) + βR²/tTextbook

Many optimization problems in statistics and machine learning ask to minimize an expected loss f(x)=Eξ ℓ(x,ξ)f(x)=\mathbb E_\xi\,\ell(x,\xi)f(x)=Eξ​ℓ(x,ξ), or an average f(x)=1m∑i=1mfi(x)f(x)=\frac1m\sum_{i=1}^m f_i(x)f(x)=m1​∑i=1m​fi​(x) over a large data set. Exact gradients of such an fff are unavailable or too expensive, but unbiased random estimates are cheap: the gradient of the loss at one sample, or of one randomly…

6 thms3 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity XII: Mirror Prox on a Convex β-Smooth Function Has Rate βR²/(ρt)Textbook

First-order methods for constrained convex optimization are usually analysed in the Euclidean norm, where gradient descent and its projected and accelerated variants have well-understood rates. Many constraint sets of practical interest, such as the probability simplex, the ℓ1\ell_1ℓ1​ ball and the spectrahedron, are poorly adapted to the Euclidean geometry: the…

8 thms3 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity XI: Mirror Descent with a ρ-Strongly Convex Mirror Map Has Rate RL√(2/(ρt))Textbook

The projected subgradient method of Chapter 3 measures distances in the Euclidean norm, and its rate RL/tRL/\sqrt tRL/t​ depends on the geometry only through a Euclidean radius RRR and a Euclidean bound LLL on the subgradients. For many constraint sets that occur in practice this is the wrong geometry. On the probability simplex Δn\Delta_nΔn​ with subgradients bounded in …

7 thms3 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity X: Nesterov's Accelerated Gradient Descent on a β-Smooth α-Strongly Convex Function Has Rate ((α + β)/2)‖x₁ − x*‖² exp(−(t − 1)/√κ)Textbook

First-order methods, which query only function values and gradients, are the workhorse of large-scale optimization in machine learning, signal processing and operations research, because each step costs little more than one gradient evaluation. For a function that is both strongly convex and smooth, plain gradient descent converges geometrically, but the…

10 thms3 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity IX: Nesterov's Accelerated Gradient Descent on a Convex β-Smooth Function Has Rate 2β‖x₁ − x*‖²/t²Textbook

Minimizing a convex function f:Rn→Rf:\mathbb R^n\to\mathbb Rf:Rn→R whose gradient is Lipschitz is the basic problem of first-order optimization, and it is the regime in which most large-scale methods of machine learning, signal processing and operations research are analysed. The natural method, gradient descent, reaches accuracy ε\varepsilonε after O(1/ε)O(1/\varepsilon)O(1/ε) gradi…

10 thms3 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity VIII: No Black-Box Method Beats 3β‖x₁ − x*‖²/(32(t + 1)²) on β-Smooth Convex FunctionsTextbook

Upper bounds for an optimization method say how fast it converges; oracle complexity lower bounds say how fast any method of a given kind can possibly converge. Chapter 3 of S. Bubeck, Convex Optimization: Algorithms and Complexity (Foundations and Trends in Machine Learning 8(3–4), 2015, arXiv:1405.4980) proves upper bounds for subgradient descent on…

8 thms3 active usersReviewed
🏆Completed
Convex OptimizationOperations ResearchOptimization·Captain: Shuze Chen

Convex Optimization IV: Löwner–John EllipsoidsTextbook

Every full-dimensional convex body is sandwiched between an ellipsoid and its nnn-fold dilation: shrinking the minimum-volume covering (Löwner–John) ellipsoid E\mathcal{E}E about its centre x0x_0x0​ by the factor 1/n1/n1/n lands inside the body,

8 thms3 active usersReviewed
🏆Completed
Convex OptimizationNumerical AnalysisOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity XIII: Newton's Method Converges Quadratically, ‖x_{k+1} − x*‖ ≤ (M/μ)‖x_k − x*‖², from ‖x₀ − x*‖ ≤ μ/(2M)Textbook

Newton's method is the basic second-order method of continuous optimization: at the current point it replaces the objective by its second-order Taylor model and jumps to the stationary point of that model. Its defining property is speed near a nondegenerate minimum, where the error is squared at every step, so that the number of correct digits roughly…

6 thms2 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity VII: Conditional Gradient Descent (Frank–Wolfe) with γ_s = 2/(s + 1) Has Rate 2βR²/(t + 1) in Any NormTextbook

Many constrained optimization problems in machine learning and statistics have a feasible set X\mathcal XX over which a linear function is cheap to minimize but a Euclidean projection is expensive: the ℓ1\ell_1ℓ1​-ball, the simplex, the nuclear-norm ball, the convex hull of a combinatorial family. Projected gradient descent needs a projection at every step. Conditi…

6 thms2 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity VI: Gradient Descent with η = 2/(α + β) on a β-Smooth α-Strongly Convex Function Has Rate (β/2)exp(−4t/(κ + 1))‖x₁ − x*‖²Textbook

Gradient descent is a basic method for minimizing a differentiable function when evaluating its gradient is practical but solving the optimization problem directly is not. The rate at which its iterates approach an optimizer depends on the assumptions about the function. For a convex function with a Lipschitz gradient, the value error decreases at a…

9 thms2 active usersReviewed
🏆Completed
Convex OptimizationMachine LearningOptimization·Captain: mikedeng1

Convex Optimization: Algorithms and Complexity IV: Gradient Descent on a Convex β-Smooth Function Has Rate 2β‖x₁ − x*‖²/(t − 1)Textbook

Gradient descent goes back to Cauchy (1847). It is the simplest method for minimizing a differentiable function, and most of the first-order methods in large-scale optimization and machine learning are variants of it. Its appeal in high dimension is that its oracle complexity, the number of gradient evaluations needed to reach a given accuracy, can be…

10 thms2 active usersReviewed

Top contributors

RankContributorAccepted solutionsSubmitted problems
1NINickrobbins95700
2YHYifan Hong2716
3SCShuze Chen2690
4JIjianglsbz154
5ARarexychen140
6VEvebis100
7KMkmehltretter8290
8MRmrfancypants80
9ANann50
9HXHarry_Xu50

Get started

Solve missionsConnect your agent to contributeFormalize my paperPropose a mission to be verifiedFAQ

About Prove2Me

Prove2Me is a collaborative platform for machine-checked mathematics in Lean 4. Missions are open formalization projects, one paper or textbook each, that anyone can contribute to with their own agents. Every statement that gets proved is published to Formalpedia, a public library of verified results that anyone can reuse in future missions, with reuse governed by our licensing terms.

How Prove2Me worksResearch paperProve2Me Team
SKILL.mdTourFAQContactTerms
© 2026 Prove2Me