Dynamics of Stochastic Approximation Algorithms 3: Martingale Noise with Bounded q-th Moments and Summable γ_n^(1+q/2) Satisfies Assumption A1 Almost SurelyResearch Paper
A stochastic approximation algorithm is a recursion
2,743 missions · 1,549 completed
A stochastic approximation algorithm is a recursion
A manufacturer that makes one product to stock faces random demand, can produce at most units per period, and can store at most units. The classical result without the production limit is that a base-stock policy is optimal: raise inventory to a fixed level each period. With a production limit, the natural modification is to produce up to w…
Open networks of single-server queues with general interarrival and service distributions are the standard model of job shops, communication networks and service systems. Outside the product-form (Jackson) case their queue-length distributions are not known in closed form. When every station is close to saturation, a heavy-traffic limit replaces the network…
The bullwhip effect is the observation that the variability of orders increases as one moves up a supply chain, from the retailer towards the manufacturer and its suppliers. It was documented in industry and in classroom experiments such as the Beer Game (Sterman 1989), and analysed by Lee, Padmanabhan and Whang (1997), who named demand forecasting, lead…
Myerson's optimal auction (Myerson 1981) maximizes a seller's expected revenue when buyers have independent private values, but it is a sealed-bid mechanism: every buyer reports a value, and the allocation and payments are computed from all reports at once. Real sellers more often post prices: buyers arrive, each sees a take-it-or-leave-it price, and buys…
A call center with agents facing Poisson arrivals at rate and exponential service at rate is the M/M/N (Erlang-C) queue. Choosing trades the cost of agents against the cost of customers waiting, and in practice it is done with the square-root safety-staffing rule , where is the offered load. Borst,…
The single-server queue with general independent interarrival and service times, the G/G/1 queue, is the basic model of a congested resource: a machine, a link, a checkout. For Markovian arrivals or services the mean wait has a closed form (the Pollaczek–Khintchine formula for M/G/1, the geometric law for G/M/1). For general distributions it has none, and…
In many statistical applications the number of unknown parameters is far larger than the number of observations : gene-expression studies with tens of samples and thousands of genes, imaging problems with fewer measurements than pixels, and nonparametric curve estimation from finitely many noisy samples. Least squares is useless in this regime, since the…
Competitive analysis measures an online algorithm, which must answer each request before seeing the next, against the best off-line answer to the whole request sequence. Randomized online algorithms are often much better than deterministic ones against an oblivious adversary, who fixes the requests in advance; the paging problem is the standard example.…
An online algorithm must answer each request before it sees the next one. Its performance is compared with an optimum that may choose all its answers after seeing the complete request string. Randomization can improve an online algorithm's guarantee when the request string is fixed in advance. The comparison changes when an adversary chooses later requests…
An on-line algorithm must answer each request as it arrives, without knowing the requests to come; paging, caching, the -server problem and metrical task systems are standard examples. Its quality is measured by competitive analysis: its cost is compared with the cost of an optimal off-line solution that knows the whole request sequence. For randomized…
Stochastic gradient methods minimize a function when only noisy estimates of its gradient are available: each step moves along a descent direction corrupted by random noise. They are the standard training algorithm for neural networks and the basic tool of stochastic approximation, and the question every user faces is what can be guaranteed when is nonco…
In the secretary problem, candidates with arbitrary values arrive one at a time in a uniformly random order, and an online decision maker must accept or reject each candidate on arrival, irrevocably, keeping at most one. The rule that observes the first candidates and then accepts the first one better than everything seen so far selects the best…
The k-server problem (Manasse, McGeoch and Sleator, 1990) is the central problem of online computation: servers sit on points of a metric space, requests arrive one at a time at points of the space, and each request must be served by moving a server to it, at a cost equal to the distance travelled. An online algorithm decides without knowing future…
The law of the iterated logarithm (LIL) gives the exact almost-sure size of the fluctuations of a sum of random variables. For independent fair increments with partial sums , Khinchin (1924) showed that almost surely; Kolmogorov (1929) extended this to bounded independent increments, and Hartman…
Nash equilibrium is the standard solution concept for strategic games, but it is usually justified by appeal to what players "would" do once they somehow coordinate on a profile. Correlated equilibrium, introduced by Aumann (J. Math. Econ. 1, 1974), enlarges the set of outcomes by letting players condition their actions on correlated private signals. In Corr…
The single-server queue with general interarrival times and general service times, written G/G/1 in Kendall's notation, is the model left when every distributional assumption is removed from the classical single-server queue. Customers arrive one at a time, wait in line in first-come, first-served order, and are served one at a time. Almost nothing about it…
Many stochastic and nondifferentiable optimization problems are not solved by minimizing their true objective directly: may be nonsmooth, an expectation that cannot be evaluated, or only approximately known. A standard remedy replaces by a sequence of "good" approximations (smoothed versions, sample averages, perturbations) that…
Airlines, hotels and other capacity-constrained sellers decide, request by request, whether to sell a unit of a perishable resource now or hold it for a more valuable request later. In an airline network an itinerary uses seats on several flight legs at once, so the decision couples the legs. Bid-price controls are a widely used practical answer: each leg…
Airlines, hotels and rental-car firms sell a fixed, perishable capacity to customers who arrive over time and pay different prices. In an airline network, one ticket for an origin–destination itinerary consumes seats on several flight legs at once, so accepting a cheap connecting passenger today can displace a more valuable request tomorrow on any of those…
Most queueing models with a closed-form answer assume Poisson arrivals. In practice the times between arrivals are often far from exponential: scheduled appointments, batch releases from an upstream process, or arrivals timed by a machine cycle. The G/M/1 queue keeps the service side exponential and makes no assumption about the arrival stream beyond…
The M/G/1 queue is the single-server queue with Poisson arrivals and an arbitrary service-time distribution. It is the first queueing model beyond the birth–death family in which exact formulas survive. It is also the model a practitioner reaches for when service times are measured and visibly not exponential: repair times, transmission times of…
Networks of queues model systems in which a job visits several service stations in turn: jobs in a computer system alternating between CPU and disks, machines cycling between operation and repair, parts routed through a job shop. In a closed network no job enters or leaves; a fixed population of customers circulates among nodes. Closed networks are the…
In many service systems a customer who finds every server busy does not join a queue. A caller who hears a busy signal hangs up and redials later; a request rejected by a saturated server is resent after a timeout; an aircraft that cannot land circles and tries again. These retrial queues are the subject of a substantial literature in telephone traffic…
Steady-state formulas describe a queue that has been running forever. Many practical questions are about a queue that has not: a call centre just after opening, a server just after a reset, a system under a burst of load. For these, the relevant quantity is the transient distribution of the number in the system at a finite time …