Markov Decision Processes (Puterman) VIII: Average Optimality for Semi-Markov ModelsTextbook
A controller may make decisions whenever a machine fails, a customer arrives, or a service finishes. The time between two decisions then depends on the state and the chosen action. Counting decisions alone does not measure elapsed operating time: a policy earning a large reward per decision may spend so long between decisions that its reward per unit time…