Information Theory Betting

March 1, 2009
52 Views

The following is a very interesting problem copied from p.132 of “Elements of Information Theory” by Cover and Thomas. I’ve been reading information theory recently, and I’m finding it very fascinating.

Example 6.3.1 (Red and Black):
In this example, cards replace horses and the outcomes become more predictable as time goes on.

Consider the case of betting on the color of the next card in a deck of
26 red and 26 black cards. Bets are placed on whether the next card will
be red or black, as we go through the deck. We also assume the game
pays a-for-l, that is, the gambler gets back twice what he bets on the
right color. These are fair odds if red and black are equally probable.

We consider two alternative betting schemes:
1. If we bet sequentially, we can calculate the conditional probability of the next card and bet proportionally. Thus we should bet on (red, black) for the first card, and for the second card, if the first card is black, etc.
2. Alternatively, we can bet on the entire sequence of 52 cards at once. There are possible sequences of 26 red and 26 black cards, all of them equally likely. Thus proportional betting implies that we put of our money on each of these sequences


The following is a very interesting problem copied from p.132 of “Elements of Information Theory” by Cover and Thomas. I’ve been reading information theory recently, and I’m finding it very fascinating.

Example 6.3.1 (Red and Black):
In this example, cards replace horses and the outcomes become more predictable as time goes on.

Consider the case of betting on the color of the next card in a deck of
26 red and 26 black cards. Bets are placed on whether the next card will
be red or black, as we go through the deck. We also assume the game
pays a-for-l, that is, the gambler gets back twice what he bets on the
right color. These are fair odds if red and black are equally probable.

We consider two alternative betting schemes:
1. If we bet sequentially, we can calculate the conditional probability of the next card and bet proportionally. Thus we should bet on (red, black) for the first card, and for the second card, if the first card is black, etc.
2. Alternatively, we can bet on the entire sequence of 52 cards at once. There are possible sequences of 26 red and 26 black cards, all of them equally likely. Thus proportional betting implies that we put of our money on each of these sequences and let each bet “ride.”

We will argue that these procedures are equivalent. For example, half the sequences of 52 cards start with red, and so the proportion of money bet on sequences that start with red in scheme 2 is also one half, agreeing with the proportion used in the first scheme. In general, we can verify that betting of the money on each of the possible outcomes will at each stage give bets that are proportional to the probability of red and black at that stage. Since we bet of the wealth on each possible outtut sequence, and a bet on a sequence increases wealth by a factor of 2^52 on the observed sequence and 0 on all the others, the resulting wealth is:

Rather interestingly, the return does not depend on the actual sequence. This is like the AEP in that the return is the same for all sequences. All sequences are typical in this sense.

It’s amazing that the strategy which bets everything at the start can beat one which bets after seeing which cards are no longer in the deck. The calculation of resulting wealth of the 2nd betting scheme, which results in 9.08, is transparent but I didn’t see how it could be exactly equal to the resulting wealth of the other betting scheme.

Here is an excel spreadsheet I made to simulate the first betting scheme: excel 2007 | older 2003, (functions in 2003 version may not work). On the first sheet, it randomly generates a sequence of draws from the deck, according to actual probabilities, and calculates the wealth you would have after the deck runs out. You can see the resulting wealth at the bottom (cell H54) of the “wealth” column- it’s always 9.08. It’s very surprising to me and I don’t fully understand how the two results can match. I’m new to the field though.

Tell me if you have any insight or a similar example that makes the paradox less paradoxical.

You may be interested

How SAP Hana is Driving Big Data Startups
Big Data
298 shares3,060 views
Big Data
298 shares3,060 views

How SAP Hana is Driving Big Data Startups

Ryan Kh - July 20, 2017

The first version of SAP Hana was released in 2010, before Hadoop and other big data extraction tools were introduced.…

Data Erasing Software vs Physical Destruction: Sustainable Way of Data Deletion
Data Management
60 views
Data Management
60 views

Data Erasing Software vs Physical Destruction: Sustainable Way of Data Deletion

Manish Bhickta - July 20, 2017

Physical Data destruction techniques are efficient enough to destroy data, but they can never be considered eco-friendly. On the other…

10 Simple Rules for Creating a Good Data Management Plan
Data Management
69 shares664 views
Data Management
69 shares664 views

10 Simple Rules for Creating a Good Data Management Plan

GloriaKopp - July 20, 2017

Part of business planning is arranging how data will be used in the development of a project. This is why…