Lesson 18 Joint Distributions
Motivating Example
Xavier and Yolanda head to the roulette table at a casino. They both place bets on red on 3 spins of the roulette wheel before Xavier has to leave. After Xavier leaves, Yolanda places bets on red on 2 more spins of the wheel. Let \(X\) be the number of bets that Xavier wins and \(Y\) be the number that Yolanda wins.
We know that \(X\) follows a \(\text{Binomial}(n=3, p=\frac{18}{38})\) distribution so its p.m.f. is \[ f(x) = \binom{3}{x} \left( \frac{18}{38} \right)^x \left(1 - \frac{18}{38} \right)^{3-x}, \] which we can write in tabular form as \[ \begin{array}{r|cccc} x & 0 & 1 & 2 & 3 \\ \hline f(x) & .1458 & .3936 & .3543 & .1063 \\ \end{array}. \]
We also know that \(Y\) follows a \(\text{Binomial}(n=5, p=\frac{18}{38})\) distribution so its p.m.f. is \[ \begin{array}{r|cccc} y & 0 & 1 & 2 & 3 & 4 & 5\\ \hline f(y) & .0404 & .1817 & .3271 & .2944 & .1325 & .0238 \end{array}. \]
But this does not tell us how \(X\) and \(Y\) are related to each other. In fact, the two random variables have a very distinctive relationship. For example, \(Y\) must be greater than or equal to \(X\), since Yolanda made the same three bets that Xavier did, plus two more. In this lesson, we will learn a way to describe the distribution of two (or more) random variables.
Theory
Example 18.1 Let’s work out the joint p.m.f. of \(X\), the number of bets that Xavier wins, and \(Y\), the number of bets that Yolanda wins. To do this, we will lay out the values of \(f(x, y)\) in a table, like the following. \[\begin{equation} \begin{array}{rr|cccc} & 5 & f(0, 5) & f(1, 5) & f(2, 5) & f(3, 5) \\ & 4 & f(0, 4) & f(1, 4) & f(2, 4) & f(3, 4) \\ y & 3 & f(0, 3) & f(1, 3) & f(2, 3) & f(3, 3) \\ & 2 & f(0, 2) & f(1, 2) & f(2, 2) & f(3, 2) \\ & 1 & f(0, 1) & f(1, 1) & f(2, 1) & f(3, 1) \\ & 0 & f(0, 0) & f(1, 0) & f(2, 0) & f(3, 0) \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array} \tag{18.2} \end{equation}\]
First, we observe that it is impossible for Xavier to win more bets than Yolanda—since Yolanda makes all the bets that Xavier does (plus two more). Therefore, we know that \(f(x,y) = 0\) if \(x > y\). We also know that Yolanda can at most two more bets than Xavier. So \(f(x, y) = 0\) if \(y > x + 2\). Therefore, we can fill in half of the entries in the table with zeroes. \[ \begin{array}{rr|cccc} & 5 & 0 & 0 & 0 & f(3, 5) \\ & 4 & 0 & 0 & f(2, 4) & f(3, 4) \\ y & 3 & 0 & f(1, 3) & f(2, 3) & f(3, 3) \\ & 2 & f(0, 2) & f(1, 2) & f(2, 2) & 0 \\ & 1 & f(0, 1) & f(1, 1) & 0 & 0 \\ & 0 & f(0, 0) & 0 & 0 & 0 \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array} \]
Let’s calculate one of the non-zero joint probabilities: \[ f(2, 3) = P(X=2 \text{ and } Y=3). \] In order for Xavier to win twice and Yolanda to win 3 times, there must have been 2 reds in the first 3 spins and 1 red in the 2 spins after that. Because the first 3 spins and the next 2 spins are independent, we can multiply the two binomial probabilities: \[\begin{align*} f(2, 3) &= P(\text{2 reds in first 3 spins} \text{ and } \text{1 red in next 2 spins}) \\ &= \binom{3}{2} \left( \frac{18}{38} \right)^2 \left( 1 - \frac{18}{38} \right)^{1} \cdot \binom{2}{1} \left( \frac{18}{38} \right)^1 \left( 1 - \frac{18}{38} \right)^{1} \\ &\approx .1766 \end{align*}\]
As one more example, let’s calculate \[ f(1, 1) = P(X=1 \text{ and } Y=1). \] In order for Xavier and Yolanda to each win once, there must have been 1 red in the first 3 spins and 0 reds in the next two. Since the first 3 spins and the next 2 spins are independent, we can multiply their probabilities: \[\begin{align*} f(1, 1) &= P(\text{1 red in first 3 spins} \text{ and } \text{0 reds in next 2 spins}) \\ &= \binom{3}{1} \left( \frac{18}{38} \right)^1 \left( 1 - \frac{18}{38} \right)^2 \cdot \binom{2}{0} \left( \frac{18}{38} \right)^0 \left( 1 - \frac{18}{38} \right)^2 \\ &\approx .1090 \end{align*}\]
The process for calculating the other probabilities in the table is the same. We look at what each event \(\{ X=x \text{ and } Y=y\}\) implies about the number of reds in the first 3 spins and the number of reds in the next 2 spins; because the first 3 spins and the next 2 spins are independent, we can multiply their probabilities. The completed table looks like this: \[ \begin{array}{rr|cccc} & 5 & 0 & 0 & 0 & .0238 \\ & 4 & 0 & 0 & .0795 & .0530 \\ y & 3 & 0 & .0883 & .1766 & .0294 \\ & 2 & .0327 & .1963 & .0981 & 0 \\ & 1 & .0727 & .1090 & 0 & 0 \\ & 0 & .0404 & 0 & 0 & 0 \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array} \]
Notice that all of the probabilities in this table add up to \(1.0\) (up to rounding error).Solution. The first 3 tosses and the next 3 tosses are independent. Therefore, we know that \[ f(x, y) = P(X=x \text{ and } Y=y) = P(X=x) \cdot P(Y=y) \] for any value \(x\) and \(y\).
Since \(X\) and \(Y\) are both \(\text{Binomial}(n=3, N_1=1, N_0=1)\), we have a formula for their p.m.f.s: \[\begin{align*} P(X=x) &= \frac{\binom{3}{x}}{2^3} & P(Y=y) &= \frac{\binom{3}{y}}{2^3}. \end{align*}\]
Therefore, we can write the joint p.m.f. as a formula: \[ f(x, y) = \frac{\binom{3}{x}}{2^3}\cdot \frac{\binom{3}{y}}{2^3}, 0 \leq x, y \leq 3. \] This is equivalent to writing the probabilities in a table: \[ \begin{array}{rr|cccc} & 3 & .0156 & .0469 & .0469 & .0156 \\ y & 2 & .0469 & .1406 & .1406 & .0469 \\ & 1 & .0469 & .1406 & .1406 & .0469 \\ & 0 & .0156 & .0469 & .0469 & .0156 \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array}. \]
To use this joint p.m.f. to calculate \(P(X + Y \leq 2)\), we add up \(f(x, y)\) for values \(x\) and \(y\) satisfying \(x + y \leq 2\). As a formula, we can express this as: \[ P(X + Y \leq 2) = \underset{x, y:\ x + y \leq 2}{\sum\sum} f(x, y). \] But it is easier to see what probabilities we need to add in the table: \[ \begin{array}{rr|cccc} & 3 & .0156 & .0469 & .0469 & .0156 \\ y & 2 & \fbox{.0469} & .1406 & .1406 & .0469 \\ & 1 & \fbox{.0469} & \fbox{.1406} & .1406 & .0469 \\ & 0 & \fbox{.0156} & \fbox{.0469} & \fbox{.0469} & .0156 \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array}. \] The answer is: \[\begin{align*} P(X + Y \leq 2) &= f(0, 0) + f(1, 0) + f(0, 1) + f(0, 2) + f(1, 1) + f(2, 0) \\ &= .0156 + .0469 + .0469 + .0469 + .1406 + .0469 \\ &= .3438. \end{align*}\]
Of course, we could have obtained this answer without deriving the joint p.m.f. of \(X\) and \(Y\). The random variable \(X + Y\) represents the number of heads in 6 tosses of a fair coin, which we know follows a binomial distribution. Therefore, we can calculate \(P(X + Y \leq 2)\) using the c.d.f. of a binomial distribution:## 0.3437500000000001We get the same answer.
Solution. First, observe that \(N\) is a random variable that can take on values from \(0\) and \(\infty\). We cannot possibly write down all of the probabilities in a table. We will try to write the joint p.m.f. as a formula.
We are interested in: \[ f(n, x) = P(N = n \text{ and } X = x). \] By the multiplication rule (Theorem 6.1), we have \[ P(N = n \text{ and } X = x) = P(N = n) P(X = x | N = n). \]
Once we know how many eggs there are, then \(X\) follows a binomial distribution. In other words, given \(\{ N = n \}\), \(X\) is a \(\text{Binomial}(n, p)\) random variable. Therefore, \[\begin{align*} P(N = n) P(X = x | N = n) &= e^{-\mu} \frac{\mu^n}{n!} \cdot \binom{n}{x} p^x (1-p)^{n-x} \\ &= e^{-\mu} \frac{(\mu p)^x}{x!} \frac{(\mu (1-p))^{n-x}}{(n-x)!}. \end{align*}\] This formula is only valid when \(x \leq n\), since we cannot have more baby chickens than eggs. So, we can specify the p.m.f. as \[ f(n, x) = \begin{cases} e^{-\mu} \frac{(\mu p)^x}{x!} \frac{(\mu (1-p))^{n-x}}{(n-x)!} & 0 \leq x \leq n < \infty \\ 0 & \text{otherwise} \end{cases}. \]Essential Practice
Two tickets are drawn from a box with \(N_1\) \(\fbox{1}\)s and \(N_0\) \(\fbox{0}\)s. Let \(X\) be the number of \(\fbox{1}\)s on the first draw and \(Y\) be the number of \(\fbox{1}\)s on the second draw. (Note that \(X\) and \(Y\) can only be 0 or 1.)
- Find the joint p.m.f. of \(X\) and \(Y\) when the draws are made with replacement.
- Find the joint p.m.f. of \(X\) and \(Y\) when the draws are made without replacement.
(You may specify the joint p.m.f. as a table or a formula, whichever is more convenient for you.)
Two fair, six-sided dice are rolled. Let \(S\) be the smaller of the numbers on the two dice. Let \(L\) be the larger of the numbers on the two dice. (Note: If doubles are rolled, then \(S = L\).)
- Find the joint p.m.f. of \(S\) and \(L\).
- Use the joint p.m.f. to calculate \(P(S + L = 2)\).
- Repeat for \(P(S + L = k)\), \(k = 3, 4, 5, \ldots, 12\). Why does the answer make sense?
A fair coin is tossed 3 times. Let \(X\) be the number of heads in these three tosses. Let \(Y\) be the number of tails in these three tosses. Find the joint p.m.f. of \(X\) and \(Y\). (You may specify the joint p.m.f. as a table or a formula, whichever is more convenient for you.)
At Diablo Canyon nuclear plant, radioactive particles hit a Geiger counter according to a Poisson process with a rate of 3.5 particles per second. Let \(X\) be the number of particles detected in the first 2 seconds. Let \(Y\) be the number of particles detected in the second after that (i.e., the 3rd second). Find the joint p.m.f. of \(X\) and \(Y\). (You may specify the joint p.m.f. as a table or a formula, whichever is more convenient for you.)
Additional Exercises
A fair coin is tossed 4 times. Let \(X\) be the number of heads in the first three tosses. Let \(Y\) be the number of heads in the last three tosses. Find the joint p.m.f. of \(X\) and \(Y\). (Hint: There are only \(2^4 = 16\) equally likely outcomes when you toss 4 coins. If you are unable to calculate the probabilities using rules we have learned, just list all the possible outcomes!)
At Diablo Canyon nuclear plant, radioactive particles hit a Geiger counter according to a Poisson process with a rate of 3.5 particles per second. Let \(X\) be the number of particles detected in the first 2 seconds. Let \(Z\) be the number of particles detected in the first 3 seconds. Find the joint p.m.f. of \(X\) and \(Z\).