Linear Programming in Linear Time When the Dimension Is Fixed: Fixed-Dimension LP Feasibility Decided in Linear Time on the Real RAMResearch Paper
Motivation
A linear program asks for a point minimizing subject to linear inequalities . Many problems in computational geometry and statistics are linear programs with few variables and very many constraints: separating two point sets by a line or plane, fitting a line in the Chebyshev () norm, finding the smallest disk or ball containing a point set (a related convex problem). For these problems the number of variables is a small constant, and what matters is how the running time grows with .
Nimrod Megiddo showed that for every fixed the problem can be solved in time (J. ACM 31(1), 1984).
Timeline.
- 1983. Megiddo (SIAM J. Comput. 12) and, independently, Dyer (SIAM J. Comput. 13 (1984)) give linear-time algorithms for and .
- 1984. Megiddo extends the method to every fixed , with (the paper formalized here).
- 1988–1991. Clarkson (J. ACM 42 (1995), conference version 1988) gives a randomized algorithm with expected time . Seidel (Discrete Comput. Geom. 6 (1991)) gives a simple randomized algorithm.
- 1992–1996. Matoušek, Sharir and Welzl and, independently, Kalai give subexponential randomized bounds. Chazelle and Matoušek derandomize the linear dependence with (J. Algorithms 21 (1996)).
Setting
Fix . An instance is a matrix and a vector , and its feasible region is the polyhedron . Here is the number of constraints and the number of variables.
The model of computation is the real RAM. A program is a finite list of instructions acting on real registers, integer pointer registers and a memory . It performs exact on reals at unit cost, tests the sign of a real, sets, copies, increments, decrements and compares pointers, and loads and stores through pointers. The input is the standard encoding of in memory: the numbers and , then row by row, then . A program decides an instance within steps with output if it halts on that output after at most steps.
Megiddo's method rests on multidimensional search. There is an unknown point and an oracle that, for any hyperplane , answers whether , or . Given hyperplanes with , the question is how many oracle calls determine the position of relative to all of them. A search strategy is a ternary decision tree: inner nodes are hyperplane queries, leaves carry outputs, and the tree is built from the data alone. For linear programming, is an optimal solution, or a minimizer of the infeasibility function when the system is infeasible. The oracle is implemented by solving problems in variables.
Formalization targets
Goal: linear-time feasibility on the real RAM
The program and the constant depend on only. No explicit form of is fixed.
Milestones
- One query settles half of hyperplanes on the line (, ).
- is orthogonal to some for at most values of , so there is a basis in which all .
- For hyperplanes of opposite slopes in the plane, the answers for and settle one of , .
- A linearly dependent pair of opposite slopes has , and the middle hyperplane settles one of them.
- Approach I: queries settle at least hyperplanes.
- queries settle all hyperplanes.
- If a hyperplane contains no optimal point, all optimal points lie on one side of it.
- The oracle, Case I: at an optimum relative to , two auxiliary systems decide the side or certify global optimality.
- The oracle, Case II: at a minimizer of on , systems (1) and (2) decide the side or certify infeasibility.
Significance
The result. For every fixed dimension, linear programming is solvable in time linear in the number of constraints. The algorithm is also strongly polynomial in fixed dimension: its operation count does not depend on the bit size of the data. Deciding whether the optimum is at most is feasibility of together with , so the goal also covers the decision form of optimization. The prune-and-search technique of the paper, which discards a constant fraction of the constraints per round, became a standard tool of computational geometry.
Formalizing it. The result is proved and classical. The platform already has the cases (linear time) and (quadratic time, by Fourier–Motzkin elimination) on the same machine and input encoding (SmaleNinth.real_ram_decides_one_variable_lp_linear, SmaleNinth.real_ram_decides_two_variable_lp_quadratic). No machine-checked proof of the general statement is known. The work consists of the query-complexity layer (milestones 1–6), the convex-analytic correctness of the oracle (milestones 7–9), and a real-RAM implementation with a step count linear in , including linear-time median selection. Alternative proofs, for example through Clarkson's or Seidel's algorithms made deterministic, are welcome for the goal.
Difficulty
The obvious approach is to find the optimum by testing constraints one by one or by eliminating variables. Fourier–Motzkin elimination produces constraints after one step. Pivoting methods have no known bound linear in . The key difficulty is to discard a constant fraction of the constraints using only a constant number of recursive calls in dimension , when no single hyperplane test gives information about more than one constraint. The multidimensional search layer gives this, and it is where the pairing of hyperplanes by slope and the degenerate cases (dependent pairs, zero coefficients) have to be handled exactly. At the machine level, the step count must stay linear in for a fixed program, so every median selection and every recursive call must be implemented within the budget, with the recursion depth depending on only.
Formalization scope
- Machine and input. The machine is the platform's real RAM
SmaleNinth.RAMProgramwithRAMDecidesInTime, and the input convention isSmaleNinth.encodeLP(published definitions, reused unchanged). No instruction is added: there is no LP, median, floor or sort primitive. Time is the number of machine steps. - Quantifier order. . The bound is in the number of constraints, so that the machine can halt at . The paper's counts unspecified units of "effort" with an unquantified term, and it is not transferred to machine steps. Where a milestone's proof fixes a constant exactly, the constant is stated: queries and settled hyperplanes in milestone 5.
- Feasibility only. The machine outputs accept or reject. Returning an optimizer, "unbounded", or a minimizer of is not part of the goal. The case is included.
- Query trees. Nodes are queries
compare (a ⬝ᵥ x) band nothing else, leaves hold fixed values, and correctness is required for every . A tree over arbitrary tests of would make milestones 5 and 6 empty, and it is excluded by the definition. - Indices. The paper's are indices
0, 1ofFin (d + 2), and its isFin.last dofFin (d + 1). - Corrections. Two passages of §4 are stated in corrected form. The Case I auxiliary objective includes the term of the direction. In Case II, feasibility of (1) puts improvement in , where the page's last sentence says . The pairing claim carries , the hypothesis its argument uses, since linear independence alone does not give it.
- Not included. Approach II and its bound , the remarks on slowly growing , the randomized variants, and the applications of §1.
- Reusable parts. The query-tree definition and milestones 1–6 apply to any prune-and-search problem with a hyperplane oracle. The oracle lemmas (7–9) are statements about convex piecewise-linear functions and polyhedra.
Selected references
- N. Megiddo, Linear programming in linear time when the dimension is fixed, J. ACM 31(1):114–127, 1984. https://doi.org/10.1145/2422.322418
- N. Megiddo, Linear-time algorithms for linear programming in and related problems, SIAM J. Comput. 12(4):759–776, 1983. https://doi.org/10.1137/0212052
- M. E. Dyer, Linear time algorithms for two- and three-variable linear programs, SIAM J. Comput. 13(1):31–45, 1984. https://doi.org/10.1137/0213003
- K. L. Clarkson, Las Vegas algorithms for linear and integer programming when the dimension is small, J. ACM 42(2):488–499, 1995. https://doi.org/10.1145/201019.201036
- R. Seidel, Small-dimensional linear programming and convex hulls made easy, Discrete Comput. Geom. 6:423–434, 1991. https://doi.org/10.1007/BF02574699
- B. Chazelle, J. Matoušek, On linear-time deterministic algorithms for optimization problems in fixed dimension, J. Algorithms 21(3):579–597, 1996. https://doi.org/10.1006/jagm.1996.0046