The following is a very interesting problem copied from p.132 of “Elements of Information Theory” by Cover and Thomas. I’ve been reading information theory recently, and I’m finding it very fascinating.
Example 6.3.1 (Red and Black):
In this example, cards replace horses and the outcomes become more predictable as time goes on.
Consider the case of betting on the color of the next card in a deck of
26 red and 26 black cards. Bets are placed on whether the next card will
be red or black, as we go through the deck. We also assume the game
pays a-for-l, that is, the gambler gets back twice what he bets on the
right color. These are fair odds if red and black are equally probable.
We consider two alternative betting schemes:
1. If we bet sequentially, we can calculate the conditional probability of the next card and bet proportionally. Thus we should bet on (red, black) for the first card, and for the second card, if the first card is black, etc.
2. Alternatively, we can bet on the entire sequence of 52 cards at once. There are possible sequences of 26 red and 26 black cards, all of them equally likely. Thus proportional betting implies that we put of our money on each of these sequences and let each bet “ride.”
We will argue that these procedures are equivalent. For example, half the sequences of 52 cards start with red, and so the proportion of money bet on sequences that start with red in scheme 2 is also one half, agreeing with the proportion used in the first scheme. In general, we can verify that betting of the money on each of the possible outcomes will at each stage give bets that are proportional to the probability of red and black at that stage. Since we bet of the wealth on each possible outtut sequence, and a bet on a sequence increases wealth by a factor of 2^52 on the observed sequence and 0 on all the others, the resulting wealth is:
Rather interestingly, the return does not depend on the actual sequence. This is like the AEP in that the return is the same for all sequences. All sequences are typical in this sense.
It’s amazing that the strategy which bets everything at the start can beat one which bets after seeing which cards are no longer in the deck. The calculation of resulting wealth of the 2nd betting scheme, which results in 9.08, is transparent but I didn’t see how it could be exactly equal to the resulting wealth of the other betting scheme.
Here is an excel spreadsheet I made to simulate the first betting scheme: excel 2007 | older 2003, (functions in 2003 version may not work). On the first sheet, it randomly generates a sequence of draws from the deck, according to actual probabilities, and calculates the wealth you would have after the deck runs out. You can see the resulting wealth at the bottom (cell H54) of the “wealth” column- it’s always 9.08. It’s very surprising to me and I don’t fully understand how the two results can match. I’m new to the field though.
Tell me if you have any insight or a similar example that makes the paradox less paradoxical.
![Why Overfitting is More Dangerous than Just Poor Accuracy [PART 1]](https://www.smartdatacollective.com/wp-content/uploads/2014/06/why-overfitting-more-dangerous-just-poor-accuracy-part-i-330x220.png)
