Lesson 4 Sampling With Replacement
Motivating Example
Recall Galileo’s problem from Lesson 1. He wanted to know whether a sum of 9 or a sum of 10 was more likely when 3 dice are rolled. In fact, Galileo’s peers reasoned that the two events should be equally likely, since there are six ways to get a sum of 9
\[\begin{align*} 3 + 3 + 3 & & 2 + 3 + 4 & & 2 + 2 + 5 & & 1 + 4 + 4 & & 1 + 3 + 5 & & 1 + 2 + 6 \end{align*}\]
and also six ways to get a sum of 10:
\[\begin{align*} 3 + 3 + 4 & & 2 + 4 + 4 & & 2 + 3 + 5 & & 2 + 2 + 6 & & 1 + 4 + 5 & & 1 + 3 + 6 \end{align*}\]
But gamblers of the day knew better. From experience, they knew that a sum of 10 was more likely than a sum of 9. But where did Galileo’s peers go wrong in their reasoning?
Discussion
Recall from Lesson 3 that the three rolls of a die can be modeled as \(n=3\) draws with replacement from the box \[ \fbox{$\fbox{1}\ \fbox{2}\ \fbox{3}\ \fbox{4}\ \fbox{5}\ \fbox{6}$}. \] Galileo’s peers showed that there are 6 ways to get a sum of 9 if you ignore the order in which the tickets were drawn. (Notice that they included 2 + 3 + 5, but not 3 + 5 + 2 and other orderings of the same three rolls.) They also showed that there are 6 ways to get a sum of 10. Why does this not guarantee the probabilities of the two events are the same?
Notice that this case (draws with replacement, order doesn’t matter) is one that we have not studied yet.
| with replacement | without replacement | |
|---|---|---|
| order matters | \(N^n\) | \(\frac{N!}{(N-n)!}\) |
| order doesn’t matter | ??? | \(\binom{N}{n} = \frac{N!}{n!(N-n)!}\) |
To settle the question, let’s go back to counting ordered outcomes.
- The outcome \(2 + 3 + 4\) corresponds to \(3! = 6\) outcomes, when you account for the possible orderings:
- \(2 + 3 + 4\)
- \(2 + 4 + 3\)
- \(3 + 2 + 4\)
- \(3 + 4 + 2\)
- \(4 + 2 + 3\)
- \(4 + 3 + 2\)
- On the other hand, the outcome \(2 + 2 + 5\) can only be reordered \(3\) different ways:
- \(2 + 2 + 5\)
- \(2 + 5 + 2\)
- \(5 + 2 + 2\)
- And the outcome \(3 + 3 + 3\) only has one possible ordering.
In other words, when we draw with replacement, the same ticket can be drawn more than once, and repetitions reduce the number of ways that tickets can be reordered. The problem with simply counting the number of outcomes is that the 6 outcomes \[\begin{align*} 3 + 3 + 3 & & 2 + 3 + 4 & & 2 + 2 + 5 & & 1 + 4 + 4 & & 1 + 3 + 5 & & 1 + 2 + 6 \end{align*}\] are not all equally likely. \(2 + 3 + 4\) is twice as likely as \(2 + 2 + 5\) and six times as likely as \(3 + 3 + 3\). When draws are made with replacement, only ordered outcomes are equally likely.
This was Galileo’s insight. When he took into account the number of possible orderings associated with each unordered outcome: \[\begin{array}{rr} \hline \text{Unordered Outcome} & \text{Possible Orderings} \\ \hline 3 + 3 + 3 & \text{1 way}\ \\ 2 + 3 + 4 & \text{6 ways} \\ 2 + 2 + 5 & \text{3 ways} \\ 1 + 4 + 4 & \text{3 ways} \\ 1 + 3 + 5 & \text{6 ways} \\ 1 + 2 + 6 & + \text{6 ways} \\ \hline & \text{25 ways} \end{array}\]he found that the probability of a 9 was actually \[ \frac{25}{216}. \] (Remember that there are \(6^3 = 216\) equally likely outcomes when order matters.)
Repeating the calculation for the probability of a 10, Galileo showed that the two probabilities were indeed different.
Examples
- Here’s a different illustration of the fact that not all unordered outcomes are
equally likely when draws are made with replacement. In the Pick 3 Lotto, a winning number is
chosen between 000 to 999. Contestants win if the digits in their chosen number matches
the winning number, in any order.
- What is your chance of winning if you bet on 053?
- What is your chance of winning if you bet on 055?
- What is your chance of winning if you bet on 555?
- Complete the solution to Galileo’s problem. What is the probability that the sum is 10 when 3 fair dice are rolled? How does this compare with the probability that the sum is 9?
Bonus Material
You will rarely need to count the unordered ways that \(n\) tickets can be drawn with replacement, since the unordered outcomes are not equally likely. However, in case you are curious how this can be calculated, the following video explains how. This video is completely optional!