First recitation
Contents
- The main line of this chapter is the three parts of a probability space, operations on events, and inclusion-exclusion.
- For independence problems, keep pairwise independence and mutual independence separate. It is easy to mix them up in a first course.
- For distribution functions, check four things one by one: monotonicity, right-continuity, limits at infinity, and jumps.
Tip. For identities involving events, first check the set identity, then take probabilities.
Exercise 1.1
This section is about sample spaces, fields of events, probability measures, and finite inclusion-exclusion. In Jordan's formula, the key point is how many times each intersection is counted.
Toss two fair coins. Write down the three parts of the probability space in detail.
Proof
, , and .
Compare the probabilities of the following two events: "in 3 tosses of a fair die, at least one 6 appears" and "in 6 tosses of a fair die, at least two 6's appear."
Proof
, while . Comparing them shows that the first probability is larger.
Let be a -algebra, and let . Prove that and .
Proof
This follows from and .
Prove Jordan's formula.
Proof
Use induction. The case is clear. For the induction step,
Prove that , and discuss when equality holds.
Proof
One way is to use and the equality condition in Cauchy--Schwarz. We can also give a direct proof. Write and .
Then
It is enough to check the bound when takes its largest and smallest possible values.
Assume without loss of generality that . Then
So the inequality holds when takes its largest possible value.
If , then , and
If , then , and
Also , so by the elementary inequality,
Thus the inequality holds when takes its smallest possible value as well.
Therefore the original inequality holds. Equality holds if and only if
Let be a sequence of events such that for all . Prove that
Proof
Look at the complement. By De Morgan's law,
Since , we have for every . By countable subadditivity,
So , and hence
Exercise 1.2
Here we need to distinguish pairwise independence from mutual independence. The prime-number example shows that in a finite uniform probability space, arithmetic can strongly restrict the form of independent events.
Toss a fair coin times. Let be the event that the -th toss and the -th toss have the same outcome. Prove that the events are pairwise independent but not mutually independent.
Proof
For two different pairs, the only nontrivial case is when the two pairs share one index, for example and . Then
If the pairs are disjoint, independence is immediate from independence of the tosses. Thus the events are pairwise independent. But they are not mutually independent. For example,
whereas
Let be prime, and put the uniform probability model on . If two events and are independent, prove that at least one of is either or .
Proof
Suppose instead that both and are nonempty proper subsets of . If they are independent, then . Let , , and . Then , so . Hence or , contradicting the assumption that and are nonempty proper subsets.
There are urns. The -th urn contains red balls and blue balls. Choose one urn at random, and then draw two balls without replacement. Find the probabilities of the following events: (1) the second ball is blue; (2) the second ball is blue given that the first ball is blue.
Proof
Let and . Similarly,
Therefore .
One hundred passengers board a plane with exactly 100 seats. Each passenger has an assigned seat. The first passenger chooses a seat uniformly at random. Passenger 2 sits in their own seat if it is empty; otherwise, they choose uniformly from the remaining empty seats. Passenger 3 follows the same rule, and so on until all passengers have boarded. What is the probability that the last passenger sits in their own seat?
Proof
If passenger 1 sits in their own seat, then passenger will certainly sit in their own seat. Otherwise, suppose passenger 1 sits in seat . Then passengers 2 through all sit in their own seats. If passenger sits in seat 1, then passenger will still get their own seat. If passenger sits in seat , then passengers through all sit in their own seats. After that, plays the same role as did before. So we only need to look at the first person who sits in seat 1 or seat .
Let be the index of this person. Given , that person is equally likely to choose seat 1 or seat . Hence
Exercise 1.3
This section is mainly about Bernoulli trials and recursions with finitely many states. Once the state variables are written clearly, the recursion usually becomes natural.
Two table-tennis players, A and B, play against each other. Player A has a higher chance of winning each game than player B. There are two possible match formats: best of 3 games, or best of 5 games. Which format is more favorable to player A?
Proof
Let . In a best-of-3 match, . In a best-of-5 match, . Therefore
This is positive when .
Child tosses fair coins, and child tosses fair coins. Find the probability that gets more heads than .
Proof
Let be the number of heads in the first tosses of , let be the result of the extra toss of , and let be the number of heads in the tosses of . Let the desired event be . By the law of total probability,
By symmetry,
Thus the answer is . One can also compute it directly:
The second-to-last equality holds because the two sums together count all equally likely outcomes of the two sets of coin tosses. The first sum counts the cases where gets more heads; the second counts the cases where gets at least as many heads as .
Players A and B take turns rolling a fair die. A rolls first. A keeps rolling until a 1 appears, then B starts rolling. B keeps rolling until a 1 appears, then A starts again, and so on. Find the probability that the -th roll is made by A.
Proof
Let be the probability that the -th roll is made by A, and let be the probability that it is made by B. Then
So the recursion is , and therefore
Exercise 1.4
For problems about distribution functions, focus on monotonicity, right-continuity, the two limits at infinity, and jumps. The size of a jump is the probability mass at that point.
Let and be distribution functions. (1) For , prove that and are distribution functions. (2) Prove that and are distribution functions.
Proof
It is enough to check monotonicity, right-continuity, and normalization. In particular, is the distribution function of the minimum of independent random variables with common distribution function . If , then
Let be a random variable, and define . Prove that is left-continuous on , and express in terms of .
Proof
Here we used continuity of probability measures. Also . Since
we get
Let be random variables. (1) Prove that and are also random variables. (2) Prove that and are also random variables.
Proof
Use the following identities: , , for and is empty for , and for and is empty for .
Suppose the distribution function of a random variable is , . Find the constants and .
Proof
We must have and . Solving these two equations gives and .
Problems
This section puts inclusion-exclusion, Euler's product formula, and medians together. A useful habit is to rewrite the object in terms of events or indicator functions.
Prove Bonferroni's inequality:
Proof
Write probabilities as expectations of indicator functions: . Let and . For a fixed , let , the number of sets among that contain . Then:
-
If , then .
-
If , then .
-
If , then and .
So . Taking expectations gives the inequality. The same method also gives the more general Bonferroni inequalities. If is odd, then
If is even, then
On the positive integers , define the probability measure
For a positive integer , let .
For any distinct primes , prove that are mutually independent.
Use probability to prove Euler's formula:
where are all primes.
Under the probability measure , choose two positive integers independently. Prove that the probability that and are coprime is
Note.
converges, so the probability measure is well-defined.
Proof
Using the least common multiple property of distinct primes,
Thus are mutually independent. Therefore
Finally, let be independent random variables with this common distribution. Then
Define a real number to be a median of a distribution function if
Prove that every distribution function has at least one median, and that the set of all medians is a closed interval.
Proof
Define
Claim that and that the set of medians is .
If there were some with , then , which would force , a contradiction. For every , we have since , and since . Thus every is a median.
If and were a median, then , which would imply , impossible. Similarly, no can be a median. Hence the set of medians is exactly .
End-of-chapter checklist
- The original problems and solutions in this chapter come from the corresponding TeX source file.
- You can first read only the problem boxes, write down the key identities, and then open the proofs or solutions.
- If a result uses independence, countable additivity, a change of variables, or a moment condition, it is worth marking that point explicitly.