# Lesson 18 Joint Distributions

## Motivating Example

Xavier and Yolanda head to the roulette table at a casino. They both place bets on red on 3 spins of the roulette wheel before Xavier has to leave. After Xavier leaves, Yolanda places bets on red on 2 more spins of the wheel. Let $$X$$ be the number of bets that Xavier wins and $$Y$$ be the number that Yolanda wins.

We know that $$X$$ follows a $$\text{Binomial}(n=3, p=\frac{18}{38})$$ distribution so its p.m.f. is $f(x) = \binom{3}{x} \left( \frac{18}{38} \right)^x \left(1 - \frac{18}{38} \right)^{3-x},$ which we can write in tabular form as $\begin{array}{r|cccc} x & 0 & 1 & 2 & 3 \\ \hline f(x) & .1458 & .3936 & .3543 & .1063 \\ \end{array}.$

We also know that $$Y$$ follows a $$\text{Binomial}(n=5, p=\frac{18}{38})$$ distribution so its p.m.f. is $\begin{array}{r|cccc} y & 0 & 1 & 2 & 3 & 4 & 5\\ \hline f(y) & .0404 & .1817 & .3271 & .2944 & .1325 & .0238 \end{array}.$

But this does not tell us how $$X$$ and $$Y$$ are related to each other. In fact, the two random variables have a very distinctive relationship. For example, $$Y$$ must be greater than or equal to $$X$$, since Yolanda made the same three bets that Xavier did, plus two more. In this lesson, we will learn a way to describe the distribution of two (or more) random variables.

## Theory

Definition 18.1 The joint distribution of two random variables $$X$$ and $$Y$$ is described by the joint p.m.f. $\begin{equation} f(x, y) = P(X=x \text{ and } Y=y). \tag{18.1} \end{equation}$

Example 18.1 Let’s work out the joint p.m.f. of $$X$$, the number of bets that Xavier wins, and $$Y$$, the number of bets that Yolanda wins. To do this, we will lay out the values of $$f(x, y)$$ in a table, like the following. $\begin{equation} \begin{array}{rr|cccc} & 5 & f(0, 5) & f(1, 5) & f(2, 5) & f(3, 5) \\ & 4 & f(0, 4) & f(1, 4) & f(2, 4) & f(3, 4) \\ y & 3 & f(0, 3) & f(1, 3) & f(2, 3) & f(3, 3) \\ & 2 & f(0, 2) & f(1, 2) & f(2, 2) & f(3, 2) \\ & 1 & f(0, 1) & f(1, 1) & f(2, 1) & f(3, 1) \\ & 0 & f(0, 0) & f(1, 0) & f(2, 0) & f(3, 0) \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array} \tag{18.2} \end{equation}$

First, we observe that it is impossible for Xavier to win more bets than Yolanda—since Yolanda makes all the bets that Xavier does (plus two more). Therefore, we know that $$f(x,y) = 0$$ if $$x > y$$. We also know that Yolanda can at most two more bets than Xavier. So $$f(x, y) = 0$$ if $$y > x + 2$$. Therefore, we can fill in half of the entries in the table with zeroes. $\begin{array}{rr|cccc} & 5 & 0 & 0 & 0 & f(3, 5) \\ & 4 & 0 & 0 & f(2, 4) & f(3, 4) \\ y & 3 & 0 & f(1, 3) & f(2, 3) & f(3, 3) \\ & 2 & f(0, 2) & f(1, 2) & f(2, 2) & 0 \\ & 1 & f(0, 1) & f(1, 1) & 0 & 0 \\ & 0 & f(0, 0) & 0 & 0 & 0 \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array}$

Let’s calculate one of the non-zero joint probabilities: $f(2, 3) = P(X=2 \text{ and } Y=3).$ In order for Xavier to win twice and Yolanda to win 3 times, there must have been 2 reds in the first 3 spins and 1 red in the 2 spins after that. Because the first 3 spins and the next 2 spins are independent, we can multiply the two binomial probabilities: \begin{align*} f(2, 3) &= P(\text{2 reds in first 3 spins} \text{ and } \text{1 red in next 2 spins}) \\ &= \binom{3}{2} \left( \frac{18}{38} \right)^2 \left( 1 - \frac{18}{38} \right)^{1} \cdot \binom{2}{1} \left( \frac{18}{38} \right)^1 \left( 1 - \frac{18}{38} \right)^{1} \\ &\approx .1766 \end{align*}

As one more example, let’s calculate $f(1, 1) = P(X=1 \text{ and } Y=1).$ In order for Xavier and Yolanda to each win once, there must have been 1 red in the first 3 spins and 0 reds in the next two. Since the first 3 spins and the next 2 spins are independent, we can multiply their probabilities: \begin{align*} f(1, 1) &= P(\text{1 red in first 3 spins} \text{ and } \text{0 reds in next 2 spins}) \\ &= \binom{3}{1} \left( \frac{18}{38} \right)^1 \left( 1 - \frac{18}{38} \right)^2 \cdot \binom{2}{0} \left( \frac{18}{38} \right)^0 \left( 1 - \frac{18}{38} \right)^2 \\ &\approx .1090 \end{align*}

The process for calculating the other probabilities in the table is the same. We look at what each event $$\{ X=x \text{ and } Y=y\}$$ implies about the number of reds in the first 3 spins and the number of reds in the next 2 spins; because the first 3 spins and the next 2 spins are independent, we can multiply their probabilities. The completed table looks like this: $\begin{array}{rr|cccc} & 5 & 0 & 0 & 0 & .0238 \\ & 4 & 0 & 0 & .0795 & .0530 \\ y & 3 & 0 & .0883 & .1766 & .0294 \\ & 2 & .0327 & .1963 & .0981 & 0 \\ & 1 & .0727 & .1090 & 0 & 0 \\ & 0 & .0404 & 0 & 0 & 0 \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array}$

Notice that all of the probabilities in this table add up to $$1.0$$ (up to rounding error).
Example 18.2 (Coin Tosses) A fair coin is tossed 6 times. Let $$X$$ be the number of heads in the first 3 tosses. Let $$Y$$ be the number of heads in the last 3 tosses. Calculate the joint p.m.f. of $$X$$ and $$Y$$, and use it to calculate $$P(X + Y \leq 2)$$.

Solution. The first 3 tosses and the next 3 tosses are independent. Therefore, we know that $f(x, y) = P(X=x \text{ and } Y=y) = P(X=x) \cdot P(Y=y)$ for any value $$x$$ and $$y$$.

Since $$X$$ and $$Y$$ are both $$\text{Binomial}(n=3, N_1=1, N_0=1)$$, we have a formula for their p.m.f.s: \begin{align*} P(X=x) &= \frac{\binom{3}{x}}{2^3} & P(Y=y) &= \frac{\binom{3}{y}}{2^3}. \end{align*}

Therefore, we can write the joint p.m.f. as a formula: $f(x, y) = \frac{\binom{3}{x}}{2^3}\cdot \frac{\binom{3}{y}}{2^3}, 0 \leq x, y \leq 3.$ This is equivalent to writing the probabilities in a table: $\begin{array}{rr|cccc} & 3 & .0156 & .0469 & .0469 & .0156 \\ y & 2 & .0469 & .1406 & .1406 & .0469 \\ & 1 & .0469 & .1406 & .1406 & .0469 \\ & 0 & .0156 & .0469 & .0469 & .0156 \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array}.$

To use this joint p.m.f. to calculate $$P(X + Y \leq 2)$$, we add up $$f(x, y)$$ for values $$x$$ and $$y$$ satisfying $$x + y \leq 2$$. As a formula, we can express this as: $P(X + Y \leq 2) = \underset{x, y:\ x + y \leq 2}{\sum\sum} f(x, y).$ But it is easier to see what probabilities we need to add in the table: $\begin{array}{rr|cccc} & 3 & .0156 & .0469 & .0469 & .0156 \\ y & 2 & \fbox{.0469} & .1406 & .1406 & .0469 \\ & 1 & \fbox{.0469} & \fbox{.1406} & .1406 & .0469 \\ & 0 & \fbox{.0156} & \fbox{.0469} & \fbox{.0469} & .0156 \\ \hline & & 0 & 1 & 2 & 3\\ & & & & x \end{array}.$ The answer is: \begin{align*} P(X + Y \leq 2) &= f(0, 0) + f(1, 0) + f(0, 1) + f(0, 2) + f(1, 1) + f(2, 0) \\ &= .0156 + .0469 + .0469 + .0469 + .1406 + .0469 \\ &= .3438. \end{align*}

Of course, we could have obtained this answer without deriving the joint p.m.f. of $$X$$ and $$Y$$. The random variable $$X + Y$$ represents the number of heads in 6 tosses of a fair coin, which we know follows a binomial distribution. Therefore, we can calculate $$P(X + Y \leq 2)$$ using the c.d.f. of a binomial distribution:
from symbulate import *
Binomial(n=6, p=0.5).cdf(2)
## 0.3437500000000001

Example 18.3 (Chicken and the Egg) The number of eggs laid by a hen, $$N$$, is a $$\text{Poisson}(\mu)$$ random variable. Each egg hatches with probability $$p$$, independently of any other egg. Let $$X$$ be the number of eggs that hatch into baby chickens. Find the joint p.m.f. of $$N$$ and $$X$$.

Solution. First, observe that $$N$$ is a random variable that can take on values from $$0$$ and $$\infty$$. We cannot possibly write down all of the probabilities in a table. We will try to write the joint p.m.f. as a formula.

We are interested in: $f(n, x) = P(N = n \text{ and } X = x).$ By the multiplication rule (Theorem 6.1), we have $P(N = n \text{ and } X = x) = P(N = n) P(X = x | N = n).$

Once we know how many eggs there are, then $$X$$ follows a binomial distribution. In other words, given $$\{ N = n \}$$, $$X$$ is a $$\text{Binomial}(n, p)$$ random variable. Therefore, \begin{align*} P(N = n) P(X = x | N = n) &= e^{-\mu} \frac{\mu^n}{n!} \cdot \binom{n}{x} p^x (1-p)^{n-x} \\ &= e^{-\mu} \frac{(\mu p)^x}{x!} \frac{(\mu (1-p))^{n-x}}{(n-x)!}. \end{align*} This formula is only valid when $$x \leq n$$, since we cannot have more baby chickens than eggs. So, we can specify the p.m.f. as $f(n, x) = \begin{cases} e^{-\mu} \frac{(\mu p)^x}{x!} \frac{(\mu (1-p))^{n-x}}{(n-x)!} & 0 \leq x \leq n < \infty \\ 0 & \text{otherwise} \end{cases}.$

## Essential Practice

1. Two tickets are drawn from a box with $$N_1$$ $$\fbox{1}$$s and $$N_0$$ $$\fbox{0}$$s. Let $$X$$ be the number of $$\fbox{1}$$s on the first draw and $$Y$$ be the number of $$\fbox{1}$$s on the second draw. (Note that $$X$$ and $$Y$$ can only be 0 or 1.)

1. Find the joint p.m.f. of $$X$$ and $$Y$$ when the draws are made with replacement.
2. Find the joint p.m.f. of $$X$$ and $$Y$$ when the draws are made without replacement.

(You may specify the joint p.m.f. as a table or a formula, whichever is more convenient for you.)

2. Two fair, six-sided dice are rolled. Let $$S$$ be the smaller of the numbers on the two dice. Let $$L$$ be the larger of the numbers on the two dice. (Note: If doubles are rolled, then $$S = L$$.)

1. Find the joint p.m.f. of $$S$$ and $$L$$.
2. Use the joint p.m.f. to calculate $$P(S + L = 2)$$.
3. Repeat for $$P(S + L = k)$$, $$k = 3, 4, 5, \ldots, 12$$. Why does the answer make sense?
3. A fair coin is tossed 3 times. Let $$X$$ be the number of heads in these three tosses. Let $$Y$$ be the number of tails in these three tosses. Find the joint p.m.f. of $$X$$ and $$Y$$. (You may specify the joint p.m.f. as a table or a formula, whichever is more convenient for you.)

4. At Diablo Canyon nuclear plant, radioactive particles hit a Geiger counter according to a Poisson process with a rate of 3.5 particles per second. Let $$X$$ be the number of particles detected in the first 2 seconds. Let $$Y$$ be the number of particles detected in the second after that (i.e., the 3rd second). Find the joint p.m.f. of $$X$$ and $$Y$$. (You may specify the joint p.m.f. as a table or a formula, whichever is more convenient for you.)

1. A fair coin is tossed 4 times. Let $$X$$ be the number of heads in the first three tosses. Let $$Y$$ be the number of heads in the last three tosses. Find the joint p.m.f. of $$X$$ and $$Y$$. (Hint: There are only $$2^4 = 16$$ equally likely outcomes when you toss 4 coins. If you are unable to calculate the probabilities using rules we have learned, just list all the possible outcomes!)
2. At Diablo Canyon nuclear plant, radioactive particles hit a Geiger counter according to a Poisson process with a rate of 3.5 particles per second. Let $$X$$ be the number of particles detected in the first 2 seconds. Let $$Z$$ be the number of particles detected in the first 3 seconds. Find the joint p.m.f. of $$X$$ and $$Z$$.