Odds algorithm

Odds-algorithm

The odds-algorithm is a mathematical method to compute optimal strategies for a class of problems which belong to the domain of optimal stopping. Its solution determines the odds-strategy, and the importance of the odds-strategy lies in its optimality, as explained below.

The odds-algorithm applies to a class of problems called last-success- problems. Formally the objective is to maximize the probability of identifying in a sequence of sequentially observed independent events a last specific event. This identification must be done on-line, that is, at the time of observation. No recall on preceding observations is permitted. Usually a specific event is defined by the decision maker as an event which is of true interest to her/him in the view of "stopping" in order to take a well-defined action. Such problems are encountered in several real-world situations.

Examples

Two different situations exemplify the interest in maximizing the probability to stop on a last specific event.

1. Suppose a car is advertised for sale (best offer) on the internet. n people respond and ask to see the car. They all insist that they would need the immediate decision whether their offer is accepted or not. Define an offer interesting, coded 1 say, if it is better than all preceding offers, and coded 0 otherwise. The offers will form a random sequence of 0's and 1's. Only 1's are of interest to the seller, and with each 1 he/she may fear that there will be no further 1's. It follows from the definition that the very last 1 is the highest offer. Maximizing the probability of selling on the last 1 therefore means maximizing the probability of selling best.

2. A physician, using a special treatment, may use the code 1 for a successful treatment, 0 otherwise. Treating a sequence of n patients with the same treatment he/she wants to minimize unnecessary sufferings, and, at the same time, obtain all successes in the sequence of patients. Stopping on the last 1 in such a random sequence of 0's and 1's means to realize this objective. Since the physician has no prophetical power, his/her objective translates into the goal of finding a strategy maximizing the probability of stopping on the last 1.

Definitions

Consider a sequence of n independent events. Associate with this sequence another sequence $I_{1},\,I_{2},\,...,\,I_{n}$ with values 1 or 0. Here $\,I_{k}=1$ stands for the event that the kth observation is interesting (as defined by the decision maker), and $\,I_{k}=0$ for non-interesting. Let $\,p_{k}=P(\,I_{k}\,=1)$ be the probability that the kth event is interesting. Further let $\,q_{k}=\,1-p_{k}$ and $\,r_{k}=p_{k}/q_{k}$. Note that $\,r_{k}$ represents the odds of the kth event turning out to be interesting, explaining the name of the odds-algorithm.

Algorithmic procedure of the odds-algorithm

The odds-algorithm sums up the odds in reverse order

$\,r_{n}\,+r_{n-1}\,+r_{n-2}\,+...$,

until this sum reaches or exceeds the value 1 for the first time. If this happens at index $\,s$, it saves $\,s$ and the corresponding sum

$\,R_{s}=\,r_{n}\,+r_{n-1}\,+r_{n-2}\,+...+r_{s}$.

If the sum of the odds does not reach 1, it sets $\,s=1$. At the same time it computes

$\,Q_{n}=q_{n}q_{n-1}...q_{s}$. The output is

  a)   $\,s$ (stopping threshold)

  b)   $\,w=Q_{s}R_{s}$ (win probability)

Odds-Strategy

The odds-strategy is the rule to observe the events one after the other and to stop on the first interesting event from index s onwards (if any), where s is the stopping threshold of output a).

The importance of the odds-strategy, and hence of the odds-algorithm, lies in the following Odds-Theorem.

Odds-Theorem

The Odds-theorem states that

  i)    The odds-strategy is optimal, that is, it maximizes the probability of stopping on the last 1.

  ii)   The win probability of the odds-strategy equals   $\,w=Q_{s}R_{s}$

  iii)  If $\,R_{s}\geq \,1$, the win probability $\,w$ is always at least $\,1/e=0.378...$, and this lower bound is best possible.

Features of the Odds-algorithm

The odds-algorithm computes the optimal strategy and the optimal win probability at the same time. Also, the number of operations of the odds-algorithm is (sub)linear in n. Hence no quicker algorithm can possibly exist for all sequences, so that the odds-algorithm is, at the same time, optimal as an algorithm.

Applications

Applications reach from medical questions in clinical trials over sales problems, secretary problems, portfolio selection, (one-way) search strategies, trajectory problems and the parking problem to problems in on-line Maintenance and others.

There exists, in the same spirit, an Odds-Theorem for continuous-time arrival processes with independent increments such as the Poisson process (Bruss(2000)). In some cases, the odds are not necessarily known in advance (as in Example 2 above) so that the application of the odds-algorithm is not directly possible. In this case each step can use sequential estimates of the odds. This is meaningful, if the number of unknown parameters is not large compared with the number n of observations. The question of optimality is then more complicated, however, and requires additional studies. Generalizations of the odds-algorithm allow for different rewards for failing to stop and wrong stops as well as replacing independence assumptions by weaker ones (Ferguson (2008))

See also

• secretary problem
• clinical trial
• Maintenance,repair and operations
• House selling Problem
• Parking problem

References

• F. Thomas Bruss Sum the Odds to One and Stop, Annals of Probability Vol. 28, 1384-1391 (2000).
• --- A note on Bounds for the Odds-Theorem of Optimal Stopping, Annals of Probability Vol. 31, 1859-1862, (2003).
• --- The art of a right decision. Newsletter of the European Mathematical Society , Issue 62, 14-20, (2006).
• T.S. Ferguson (2008, unpublished)
• Mitsushi Tamaki:Optimal Stopping on Trajectories and the Ballot Problem, Journal of Applied Probability Vol. 38, 946-959 (2001).
• Shoo-Ren Hsiao and Jiing-Ru. Yang: Selecting the Last Success in Markov-Dependent Trials, Journal of Applied Probability, Vol. 93, 271-281, (2002.)
• B. Iung, E. Levrat and E. Thomas: Odds-Algorithm - based Opportunistic Maintenance Task Execution for Preserving Product Conditions, CIRP-Annals Maintenance, Vol. 56, Issue 1, 13-16, (2007).