Markov reward model
In probability theory, a Markov reward model or Markov reward process is a stochastic process which extends either a Markov chain or continuous-time Markov chain by adding a reward rate to each state. An additional variable records the reward accumulated up to the current time.[1] Features of interest in the model include expected reward at a given time and expected time to accumulate a given reward.[2] The model appears in Ronald A. Howard's book.[3] The models are often studied in the context of Markov decision processes where a decision strategy can impact the rewards received.
The Markov Reward Model Checker tool can be used to numerically compute transient and stationary properties of Markov reward models.
Markov chain
![[icon]](/wiki/File:Wiki_letter_w_cropped.svg?lang=en){kind=small} | This section is empty. You can help by adding to it. (October 2013) |
Continuous-time Markov chain
The accumulated reward at a time t can be computed numerically over the time domain or by evaluating the linear hyperbolic system of equations which describe the accumulated reward using transform methods or finite difference methods.[4]
References
- ^ Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1007/978-1-4615-1387-2_2, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with
|doi=10.1007/978-1-4615-1387-2_2instead. - ^ Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1007/978-3-642-11492-2_10, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with
|doi=10.1007/978-3-642-11492-2_10instead. - ^ Howard, R.A. (1971). Dynamic Probabilistic Systems, Vol II: Semi-Markov and Decision Processes. New York: Wiley. ISBN 0471416657.
- ^ Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1016/0377-2217(89)90335-4, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with
|doi=10.1016/0377-2217(89)90335-4instead.
 | This probability-related article is a stub. You can help Wikipedia by expanding it. |