Markov strategy - Wikipedia

In game theory, a Markov strategy^[1] is a strategy that depends only on the current state of the game, rather than the full history of past actions. The state summarizes all relevant past information needed for decision-making. For example, in a repeated game, the state could be the outcome of the most recent round or any summary statistic that captures the strategic situation or recent sequence of play.^[2]

A profile of Markov strategies forms a Markov perfect equilibrium if it constitutes a Nash equilibrium in every possible state of the game. Markov strategies are widely used in dynamic and stochastic games, where the state evolves over time according to probabilistic rules.

Although the concept is named after Andrey Markov due to its reliance on the Markov property^[3]—the idea that only the current state matters—the strategy concept itself was developed much later in the context of dynamic game theory.

References

[edit]

^ "First Links in the Markov Chain". American Scientist. 2017-02-06. Retrieved 2017-02-06.
^ Fudenberg, Drew (1995). Game Theory. Cambridge, MA: The MIT Press. pp. 501–40. ISBN 0-262-06141-4.
^ Sack, Harald (2022-06-14). "Andrey Markov and the Markov Chains". SciHi Blog. Retrieved 2017-11-23.

v t e Topics of game theory
Definitions	Asynchrony Bayesian regret Best response Bounded rationality Cheap talk Complete contract Complete information Complete mixing Confrontation analysis Congestion game Conjectural variation Contingent cooperator Coopetition Determinacy Dynamic inconsistency Escalation of commitment Extensive-form game Farsightedness First-player and second-player win Game complexity Game form Game semantics Game theory Game tree Hierarchy of beliefs Imperfect information Incomplete information Information set Mechanism design Move by nature Mutual knowledge Normal-form game No-win situation Non-credible threat Null move Outcome Payoff Perfect information Perfect recall Ply Preference Rationality Sequential game Simultaneous action selection Simultaneous game Solved game Strategic dominance Strategic form Strategic interaction Strategy Succinct game Topological game Tragedy of the commons
Equilibrium concepts	Backward induction Bayes correlated equilibrium Bayesian efficiency Bayesian game Bayesian Nash equilibrium Berge equilibrium Bertrand–Edgeworth model Coalition-proof Nash equilibrium Core Correlated equilibrium Edgeworth price cycle Epsilon-equilibrium Evolutionarily stable strategy Gibbs equilibrium Implementation theory Incentive compatibility Incomplete contracts Inequity aversion Individual rationality Iterated elimination of dominated strategies Hierarchy of beliefs Markov perfect equilibrium Mertens-stable equilibrium Nash equilibrium One-shot deviation principle Open-loop model Pareto efficiency Participation constraint Payoff dominance Perfect Bayesian equilibrium Price of anarchy Program equilibrium Proper equilibrium Quantal response equilibrium Quasi-perfect equilibrium Rational agent Rationalizability Rationalizable strategy Risk dominance Satisfaction equilibrium Self-confirming equilibrium Sequential equilibrium Shapley value Strong Nash equilibrium Subgame perfection Trembling hand equilibrium
Strategies	Appeasement Behavioral strategy Bid shading Cheap talk Collusion Commitment device De-escalation Deterrence Markov strategy Dominant strategies Escalation Fictitious play Focal point Forward induction Grim trigger Hobbesian trap Markov strategy Max-dominated strategy Mixed strategy Pairing strategy Pure strategy Strategy-stealing argument Tit for tat
Classes of games	Asymmetric game Auction Bargaining problem Cooperative game Deterministic game Differential game Finite game Global game Infinite game Intransitive game n-player game Large Poisson game Market game Mean-field game Mean payoff game Minimum effort game Multi-stage game Network game Non-cooperative game Potential game Quantum refereed game Repeated game Screening game Signaling game Stochastic game Strictly determined game Supermodular game Symmetric game Zero-sum game
Games	All-pay auction Battle of the sexes Bertrand competition Blotto game Centipede game Chess Chicken Coordination game Cournot competition Deadlock Dictator game Trust game Diner's dilemma Dollar auction Checkers El Farol Bar problem Electronic mail game Fair cake-cutting Fair division Gift-exchange game Go Guess 2/3 of the average Induction puzzles Infinite chess Keynesian beauty contest Kuhn poker Lewis signaling game Matching pennies Nash bargaining game Obligationes Optional prisoner's dilemma Pirate game Prisoner's dilemma Princess and monster game Public goods game Pursuit game Rendezvous problem Rock paper scissors Stackelberg competition Stag hunt Traveler's dilemma Ultimatum game Volunteer's dilemma War of attrition
Theorems & theories	Arrow's impossibility theorem Aumann's agreement theorem Brouwer fixed-point theorem Competitive altruism Folk theorem Gibbard–Satterthwaite theorem Gibbs lemma Glicksberg's theorem Kakutani fixed-point theorem Kuhn's theorem Minimax theorem Myerson–Satterthwaite theorem Nash's theorem Negamax theorem One-shot deviation principle Parrondo's paradox Prim–Read theory Purification theorem Rational ignorance Rational irrationality Revelation principle Revenue equivalence theorem Shapley's axioms Sperner's lemma Sprague–Grundy theorem Zermelo's theorem
Key figures	Albert W. Tucker Alvin E. Roth Amos Tversky Antoine Augustin Cournot Ariel Rubinstein Claude Shannon Daniel Kahneman David Gale David K. Levine David M. Kreps Donald B. Gillies Drew Fudenberg Eric Maskin Harold W. Kuhn Herbert Simon Herbert Scarf Hervé Moulin Jean Tirole Jean-François Mertens Jennifer Tour Chayes John Conway John Nash John Harsanyi John Maynard Smith John von Neumann Ken Binmore Kenneth Arrow Kenneth Binmore Leonid Hurwicz Lloyd Shapley Martin Shubik Melvin Dresher Merrill M. Flood Olga Bondareva Oskar Morgenstern Paul Milgrom Peyton Young Reinhard Selten Robert Aumann Robert Axelrod Robert B. Wilson Roger Myerson Samuel Bowles Suzanne Scotchmer Thomas Schelling William Vickrey
Search optimizations	Alpha–beta pruning Aspiration window Expectiminimax Game tree search Lazy SMP Lemke–Howson algorithm max^n algorithm Minimax algorithm Monte Carlo tree search Negamax Paranoid algorithm Principal variation search
Subfields	Algorithmic game theory Behavioral game theory Combinatorial game theory Compositional game theory Confrontation analysis Contract theory Cooperative game Drama theory Evolutionary game theory Graphical game theory Heresthetic Implementation theory Mean-field game theory Negotiation theory Quantum game theory
Miscellaneous	Bertrand paradox Chainstore paradox Computational complexity of games Glossary of game theory Helly metric Lemke–Howson algorithm List of game theorists List of games in game theory Multi-agent system PPAD-complete

This game theory article is a stub. You can help Wikipedia by expanding it.