On the Optimality of (s, S) Policies

Eugene A. Feinberg

Department of Applied Mathematics and Statistics

State University of New York at Stony Brook

Mark E. Lewis

Cornell University
School of Operations Research and Information Engineering
Ithaca, New York 14853

This paper describes results on the existence of optimal policies and convergence properties of optimal actions for discounted and average-cost Markov Decision Processes with weakly continuous transition probabilities. It is possible that cost functions are unbounded and action sets are not compact. The results are applied to stochastic periodic-review inventory control problems, for which they imply the existence of stationary optimal policies and certain optimality properties. The optimality of (s, S) policies is proved by using dynamic programming equations for discounted costs and the vanishing discount factor approach for average costs per unit time.