Optimization problem that (very) loosely parallels

[Overseer55]

I came up with this “game” yesterday evening after reading (on 2+2) a post from a well respected individual that “it is possible to poker perfectly, but the definition of ‘perfectly’ changes all the time”.

I find the "game" to be fairly interesting. Also, I think it is a reasonable model for optimizing "perfect play" in a game where the definition of "perfect play" is constantly changing.

I call it the "triple-half" game.

Specifications:
Let game G consist of 100 trials. At the beginning of each trial, you are given a $1000 “trial balance” and are guaranteed that the trial is “fair”. Each trial consists of a potentially unlimited number of rounds. At each round, you are allowed to move your “trial balance” to your “overall balance” or you can risk the “trial balance” in the hopes of winning more. If you decide to risk the money, two events happen (in succession). First, a random number, n, between 0 and 1 is chosen. If n is less than p, the trial becomes “unfair”. Second, a coin is flipped and your trial balance changes. If the result is heads, your trial balance triples. If the result is tails, your trial balance is reduced by 50%. If the trial is in “fair” mode, the coin has an equal chance of landing on heads and tails. If the trial is in “unfair” mode, the coin will always land on tails.

Goal:
Determine an optimal strategy to maximize your overall balance.

Clarifications:
1) Once a given trial becomes “unfair”, there is no way to make it “fair” again.
2) You do not know when a given trial becomes “unfair”.
3) You do not know p at the beginning of the game.

If my analysis is correct,

1) It is possible to optimally play the game if you know p (using some form of Bayesian analysis).
2) Every trial allows you to refine your estimation of p.
3) By playing more rounds in a trial that you suspect to be unfair (not necessarily the first one) you increase your ability to estimate p.
4) Assuming that 'p' is uniformally distributed from [0,1) at the beginning of the game may not be optimal.

I may be out in left-field with this one...but, hopefully someone else in the world will find this one interesting. BTW, I have taken a significant number of university level math/probability/statistics courses...so, feel free to throw any formulas at me.

Thanks in advance.

- Mark

[AaronBrown]

I think you need to specify p > 1/3, otherwise the expectation is infinite which will screw things up. If there is even a tiny chance of a p <= 1/3, it's worth risking any finite amount of money, so you would keep flipping forever on all trials.

If you knew p, and it was greater than 1/3, the strategy of flipping until you get your first tail has expected value $1,000*(1+p)/(6p - 2). This is greater than $1,000 if p < 3/5. If you knew p > 3/5, you wouldn't make even one flip. However, you might flip even knowing there was some probability p > 3/5, because the expectation there is not negative infinity.

If you knew p > 3/5, then all flips have negative expectation.

It seems to me likely that the optimal strategy is to flip until you get k tails in a row. You might adjust k as you go from trial to trial and get more information about p. For fixed p, it's easy to compute the expected value of this strategy for any k. On the 100th trial, you use your knowledge of p to pick the best k. On the 99th trial, you might want to use a larger k to get more information for the 100th trial. Working backward, you should be able to get the optimal strategy, give some initial prior about p.