Markov reward model

In probability theory, a Markov reward model is a stochastic process which extends either a Markov chain or continuous-time Markov chain by adding a reward rate to each state. An additional variable records the reward accumulated up to the current time.^[1] Features of interest in the model include expected reward at a given time and expected time to accumulate a given reward.^[2] The model appears in Ronald A. Howard's book.^[3]

The Markov Reward Model Checker tool can be used to numerically compute transient and stationary properties of Markov reward models.

Markov chain

Continuous-time Markov chain

The accumulated reward can be computed numerically over the time domain or by evaluating the linear hyperbolic system of equations which describe the accumulated reward using transform methods or finite difference methods.^[4]

References

^ Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1007/978-1-4615-1387-2_2, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with |doi=10.1007/978-1-4615-1387-2_2 instead.
^ Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1007/978-3-642-11492-2_10, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with |doi=10.1007/978-3-642-11492-2_10 instead.
^ Howard, R.A. (1971). Dynamic Probabilistic Systems, Vol II: Semi-Markov and Deccision Processes. New York: Wiley. ISBN 0471416657.
^ Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1016/0377-2217(89)90335-4, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with |doi=10.1016/0377-2217(89)90335-4 instead.

This probability-related article is a stub. You can help Wikipedia by expanding it.

[1] Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1007/978-1-4615-1387-2_2, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with |doi=10.1007/978-1-4615-1387-2_2 instead.

[2] Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1007/978-3-642-11492-2_10, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with |doi=10.1007/978-3-642-11492-2_10 instead.

[3] Howard, R.A. (1971). Dynamic Probabilistic Systems, Vol II: Semi-Markov and Deccision Processes. New York: Wiley. ISBN 0471416657.

[4] Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1016/0377-2217(89)90335-4, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with |doi=10.1016/0377-2217(89)90335-4 instead.

[1]

[2]

[3]

[4]