M. Puterman, “Markov Decision Processes: Discrete Stochastic Dynamic Programming,” John Wiley & Sons, Inc, New York, 1994.

has been cited by the following article:

• TITLE: Structural Properties of Optimal Scheduling Policies for Wireless Data Transmission

  AUTHORS: Nomesh Bolia, Vidyadhar Kulkarni

  KEYWORDS: MDP; Scheduling; Structural Properties

  JOURNAL NAME: International Journal of Communications, Network and System Sciences, Vol.5 No.10, October 31, 2012

  ABSTRACT: We analyze a cell with a fixed number of users in a time period network. The base station schedules to serve at most one user in a given time period based on information about the available data rates and other parameter(s) for all the users in the cell. We consider infinitely backlogged queues and model the system as a Markov Decision Process (MDP) and prove the monotonicity of the optimal policy with respect to the "starvation age" and the available data rate. For this, we consider both the discounted as well as the long-run average criterion. The proofs of the monotonicity properties serve as good illustrations of analyzing MDPs with respect to their optimal solutions.