A semimartingale characterization of average optimal stationary policies for Markov decision processes

[摘要] This paper deals with discrete-time Markov decision processes withBorel state and action spaces. The criterion to be minimized isthe average expected costs, and the costs may haveneitherupper nor lowerbounds. In our former paper (to appear in Journalof Applied Probability),weakerconditions are proposedto ensure the existence of average optimal stationary policies. Inthis paper, we further study some properties of optimal policies.Under theseweakerconditions, we not only obtain twonecessary and sufficient conditions for optimal policies, but alsogive a “semimartingale characterization” of an average optimalstationary policy.

[学科分类] 应用数学