关键词: 计算机应用, 半Markov决策过程, 等价无穷小生成子, 异步数值迭代

Abstract: For the problem of value iteration (VI) optimization of the compact action set in semiMarkov decision process, a unified standard VI algorithm directly based on the equivalent infinitesimal generator under both discount and average criteria was proposed with the proof of convergence. In addition, two unified asynchronous VI algorithms, GaussSeidel and stochastic, especially the stochastic asynchronous VI algorithm based on the sample path simulation under both discount and average criteria was discussed with improvement by using the performance potential theory. Comparison of each algorithm through a numerical example was given, which shows that the proposed algorithms are applicable to the continuoustime Markov decision processes.

Key words: computer application, semiMarkov decision process, equivalent infinitesimal generator, asynchronous value iteration