Introduction to Probability and Statistics

9.1 Some preliminary set theory#

Probability is built on sets, so before any probability appears we need to be fluent with the language of sets. Most of this will be revision, but the notation matters enormously later — almost every probability rule in this chapter is really a set identity in disguise.

Example. The set $A = \{1,2,3\}$ has elements $1$, $2$ and $3$; so $1 \in A$ but $4 \notin A$.

Example. The set $A = \{1,2,3\}$ has eight subsets; for instance $\{2,3\} \subseteq A$. In general a set with $n$ elements has $2^n$ subsets, since each element is independently either in or out — this doubling is worth remembering, as it reappears when we count outcomes.

In any given problem there is a fixed set containing everything under discussion, called the universal set $S$. For three-dimensional vector geometry it is usually $S = \mathbb{R}^3$; in probability it will be the set of all possible outcomes.

Example. Let $S$ be all students enrolled in MATH1231, let $A$ be those aged 20 or over, and let $B$ be those who own a bicycle. Then $A \cap B$ is the students aged 20 or over who own a bicycle; $A \cup B$ is those who are aged 20 or over, own a bicycle, or both; $A^c$ is those under 20; and $A - B$ is those aged 20 or over who do not own a bicycle. Note that "or" in mathematics is always inclusive — $A \cup B$ includes the students who satisfy both conditions. English "or" is frequently exclusive, and that mismatch causes real errors when translating word problems.

Example. The sets $A_1 = \{1,3\}$, $A_2 = \{2\}$ and $A_3 = \{4,5\}$ partition $B = \{1,2,3,4,5\}$.

Basically, a partition chops a set into non-overlapping pieces that between them use up everything. This is the single most important idea in the chapter: every major probability rule — the addition rule, the law of total probability, Bayes' theorem — works by partitioning a set and adding up the pieces, which is only legitimate because the pieces do not overlap.

Two identities get used constantly and are worth proving once.

Proof. We prove the first; the second is identical with the roles of $\cup$ and $\cap$ swapped. For any $x \in S$,

$$\begin{align*} x \in (A\cup B)^c &\iff x \notin A \cup B \\ &\iff \text{it is not the case that } (x \in A \text{ or } x \in B) \\ &\iff x \notin A \text{ and } x \notin B \\ &\iff x \in A^c \text{ and } x \in B^c \\ &\iff x \in A^c \cap B^c. \end{align*}$$

Since the two sets have exactly the same elements, they are equal. $\blacksquare$

Basically, De Morgan says that negating an "or" gives an "and", and vice versa. In probability language: "neither $A$ nor $B$ happened" is the same event as "not $A$ and not $B$", and "not both" is the same as "at least one failed". Getting this backwards is a very common source of wrong answers in "at least one" problems.

Proof. Adding $|A|$ and $|B|$ counts every element of $A \cap B$ exactly twice — once as a member of $A$ and once as a member of $B$ — while every other element of $A \cup B$ is counted once. Subtracting $|A\cap B|$ removes the double count. $\blacksquare$

Example. Of $20$ music students, $7$ play guitar, $8$ play piano, and $3$ play both. How many play at least one of the two, and how many play neither?
Let $G$ and $P$ be the sets of guitarists and pianists, so $|G| = 7$, $|P| = 8$ and $|G \cap P| = 3$. Then

$$\begin{align*} |G \cup P| &= |G| + |P| - |G\cap P| \\ &= 7 + 8 - 3 \\ &= 12, \end{align*}$$

and by De Morgan the students who play neither form $(G\cup P)^c$, of size $20 - 12 = 8$. Therefore $12$ students play at least one instrument and $8$ play neither. Notice that $7 + 8 = 15 \neq 12$; forgetting to subtract the overlap is the classic error, and it is exactly the error the addition rule for probabilities is designed to prevent.

9.2 Probability#

9.2.1 Sample space and probability axioms#

An experiment is any procedure with an uncertain outcome.

Example. Tossing a coin is an experiment with sample space $S = \{H, T\}$.

Example. Tossing a coin three times and recording the ordered sequence has sample space

$$S = \{HHH, HHT, HTH, HTT, THH, THT, TTH, TTT\},$$

with $|S| = 2^3 = 8$. The event $A$ that exactly two heads are tossed is the subset

$$A = \{HHT, HTH, THH\}.$$

The choice of sample space is part of modelling, not something handed to you. If we only cared about the number of heads we could have taken $S = \{0,1,2,3\}$ — but then the outcomes would not be equally likely, which usually makes life harder. Choose a sample space in which the outcomes are equally likely whenever you can.

Basically, probability is a way of measuring how big an event is compared with the whole sample space: never negative, the whole space has measure $1$, and non-overlapping pieces add. Condition (d) requires disjointness and is worthless without it — this is where almost every incorrect probability calculation goes wrong.

Proof. Statement 1 follows for finite $S$ by induction on $|A|$ using axiom (d), since the singletons $\{a\}$ for $a \in A$ are pairwise disjoint and partition $A$. Statement 3 is statement 1 applied to $A = S$, together with $P(S) = 1$. For statement 2, suppose $P(\{a\}) = p$ for every $a \in S$. By statement 3,

$$1 = \sum_{a\in S}P(\{a\}) = \sum_{a \in S}p = |S|\,p,$$

so $p = \frac{1}{|S|}$. Then by statement 1,

$$P(A) = \sum_{a\in A}P(\{a\}) = \sum_{a\in A}\frac{1}{|S|} = \frac{|A|}{|S|}. \qquad \blacksquare$$

Statement 2 is the formula you have used since school — probability equals favourable outcomes over total outcomes. But it is only valid when the outcomes are equally likely, and it is a theorem, not a definition. Applying it to a sample space with unequally likely outcomes is a serious error.

Example. Pick a ball at random from a bag containing $3$ red and $7$ blue balls. Taking $S$ to be the ten individual balls, each equally likely, the event $R$ that the ball is red has $|R| = 3$, so

$$P(R) = \frac{|R|}{|S|} = \frac{3}{10}.$$

Therefore the probability of drawing red is $0.3$. Notice that had we taken $S = \{\text{red}, \text{blue}\}$ we could not have used the formula, since those two outcomes are not equally likely.

9.2.2 Rules for probabilities#

Proof.

1. The sets $A - B$, $A\cap B$ and $B - A$ partition $A \cup B$, so by axiom (d),

$$P(A\cup B) = P(A-B) + P(A\cap B) + P(B-A).$$

Also $A - B$ and $A \cap B$ partition $A$, giving $P(A-B) = P(A) - P(A\cap B)$, and similarly $P(B-A) = P(B) - P(A\cap B)$. Substituting,

$$P(A\cup B) = \big(P(A) - P(A\cap B)\big) + P(A\cap B) + \big(P(B) - P(A\cap B)\big) = P(A)+P(B)-P(A\cap B).$$

2. The sets $A$ and $A^c$ partition $S$, so $P(A) + P(A^c) = P(S) = 1$.
3. If $A \subseteq B$ then $A$ and $B - A$ partition $B$, so $P(B) = P(A) + P(B-A) \geq P(A)$ by axiom (a). $\blacksquare$

Notice that this is exactly the inclusion–exclusion principle with $|\cdot|$ replaced by $P$, and for the same reason: the overlap gets counted twice and must be removed once.

The complement rule is the most useful computational tool in the whole chapter. Whenever a question asks for the probability of "at least one" of something, computing $P(\text{none})$ and subtracting from $1$ is almost always dramatically easier, because "none" is a single intersection while "at least one" is a messy union.

Example. What is the probability that at least two of $n$ people share a birthday?
Ignoring leap years, let $Y$ be the $365$ days of the year. The sample space of all possible birthday lists is

$$S_n = \{(b_1,\dots,b_n) : b_1,\dots,b_n \in Y\}, \qquad |S_n| = 365^n,$$

with all outcomes equally likely. Let $A$ be the event that at least two people share a birthday. Computing $P(A)$ directly would require a horrible union over all pairs, so use the complement: $A^c$ is the event that all $n$ birthdays are distinct, and such a list is an ordered selection of $n$ different days,

$$|A^c| = 365 \times 364 \times \cdots \times (365 - n + 1).$$

Therefore

$$P(A) = 1 - P(A^c) = 1 - \frac{365 \times 364 \times \cdots \times (365-n+1)}{365^n}.$$

Evaluating this gives famously counter-intuitive numbers:

So with just $23$ people the odds are already better than even. Therefore the answer is $1 - \frac{365!}{(365-n)!\,365^n}$. The intuition that misleads people here is comparing themselves to everyone else ($n-1$ comparisons) rather than counting all $\binom{n}{2}$ pairs, which for $n=23$ is $253$ pairs — plenty of chances for a match.

Example. In a town, $80\%$ of the population has comprehensive car insurance, $60\%$ has house insurance, and $50\%$ has both. What proportion has at least one, and what proportion has neither?
Let $C$ and $H$ be the events of having car and house cover, so $P(C) = 0.8$, $P(H) = 0.6$ and $P(C\cap H) = 0.5$. By the addition rule,

$$\begin{align*} P(C\cup H) &= P(C) + P(H) - P(C\cap H) \\ &= 0.8 + 0.6 - 0.5 \\ &= 0.9, \end{align*}$$

and by De Morgan together with the complement rule,

$$P(C^c\cap H^c) = P\big((C\cup H)^c\big) = 1 - 0.9 = 0.1.$$

Therefore $90\%$ have at least one policy and $10\%$ have neither.

9.2.3 Conditional probabilities#

Often we learn something partway through, and want to update a probability in the light of it. If we know that $B$ has occurred, then $B$ effectively becomes the new sample space, and we should measure $A$ only by the part of it lying inside $B$.

Basically, we restrict attention to $B$ and rescale so that $B$ has total probability $1$; dividing by $P(B)$ is exactly that rescaling. The condition $P(B) > 0$ is not a technicality — conditioning on an impossible event is meaningless.

Rearranging gives the form used most often in practice:

$$\boxed{P(A\cap B) = P(A\mid B)\,P(B) = P(B\mid A)\,P(A).}$$

Example. We roll a die. Let $A$ be the event of rolling a six and $B$ the event of rolling an even number. Find $P(A\mid B)$ and $P(B\mid A)$.
Here $A = \{6\}$ and $B = \{2,4,6\}$, so $A\cap B = \{6\}$, and

$$\begin{align*} P(A\mid B) &= \frac{P(A\cap B)}{P(B)} = \frac{1/6}{3/6} = \frac13, \\ P(B\mid A) &= \frac{P(A\cap B)}{P(A)} = \frac{1/6}{1/6} = 1. \end{align*}$$

Therefore $P(A\mid B) = \frac13$ and $P(B\mid A) = 1$. Note how different the two are: $P(A\mid B) \neq P(B\mid A)$ in general, and confusing the two is so common in law and medicine that it has a name — the prosecutor's fallacy. Knowing the roll is even makes a six more likely than before; knowing it is a six makes it certainly even.

Example. A bag contains $3$ red and $3$ blue balls. Two balls are drawn without replacement. Find the probability that both are red.
Let $R_1$ and $R_2$ be the events that the first and second balls are red. Before any draw, $P(R_1) = \frac36 = \frac12$. Given the first ball was red, only $5$ balls remain of which $2$ are red, so $P(R_2\mid R_1) = \frac25$. Therefore

$$P(R_1\cap R_2) = P(R_2\mid R_1)P(R_1) = \frac25\times\frac12 = \frac15.$$

Drawing without replacement is what makes the draws dependent; with replacement, $P(R_2\mid R_1)$ would still be $\frac12$ and the answer would be $\frac14$. Always check which of the two a question intends.

Conditioning becomes really powerful when combined with a partition.

Proof. Since the $B_i$ partition $S$, the sets $A\cap B_1, \dots, A\cap B_n$ are pairwise disjoint and their union is $A$. By axiom (d) applied repeatedly and then the definition of conditional probability,

$$P(A) = \sum_{i=1}^{n}P(A\cap B_i) = \sum_{i=1}^{n}P(A\mid B_i)P(B_i). \qquad \blacksquare$$

Basically, to find the probability of $A$, split the world into cases, work out the chance of $A$ in each case, and average those weighted by how likely each case is. This is exactly what a tree diagram does: each path through the tree is one term of the sum, and the probability of a path is the product of the probabilities along its edges.

Proof. By the definition of conditional probability twice over,

$$P(B_j\mid A) = \frac{P(A\cap B_j)}{P(A)} = \frac{P(A\mid B_j)P(B_j)}{P(A)},$$

and expanding the denominator by the law of total probability gives the result. $\blacksquare$

Basically, Bayes' theorem reverses a conditional probability: it converts "the chance of the evidence given the cause" into "the chance of the cause given the evidence", which is almost always the direction you actually want. The following example is the standard illustration of how badly intuition fails here, and it is worth studying carefully.

Example. A diagnostic test for disease $X$ correctly indicates the disease $99\%$ of the time in people who have it, and correctly returns negative $98\%$ of the time in people who do not. Suppose $2\%$ of the population has the disease. Find the probability that a person who tests positive does not have the disease.
Let $D$ be the event of having the disease and $T_p$ the event of testing positive, with $T_n = T_p^c$. We are given

$$P(D) = 0.02, \quad P(T_p\mid D) = 0.99, \quad P(T_n\mid D^c) = 0.98, \quad \text{so } P(T_p\mid D^c) = 0.02.$$

The events $D$ and $D^c$ partition $S$, so by Bayes' theorem,

$$\begin{align*} P(D^c\mid T_p) &= \frac{P(T_p\mid D^c)P(D^c)}{P(T_p\mid D^c)P(D^c) + P(T_p\mid D)P(D)} \\ &= \frac{0.02\times 0.98}{0.02\times 0.98 + 0.99\times 0.02} \\ &= \frac{0.0196}{0.0196 + 0.0198} \\ &= \frac{0.0196}{0.0394} \\ &\approx 0.497. \end{align*}$$

Therefore almost half of all positive results are false, despite the test being $98$–$99\%$ accurate. The reason is that the disease is rare: out of $10\,000$ people, about $200$ have it and $198$ of those test positive, while of the $9800$ healthy people $2\%$ — that is, $196$ people — also test positive. The two groups are almost the same size, so a positive result is nearly a coin flip.

By contrast, a negative result is extremely informative:

$$\begin{align*} P(D\mid T_n) &= \frac{P(T_n\mid D)P(D)}{P(T_n\mid D)P(D) + P(T_n\mid D^c)P(D^c)} \\ &= \frac{0.01\times 0.02}{0.01\times0.02 + 0.98\times0.98} \\ &= \frac{0.0002}{0.9606} \\ &\approx 0.000208. \end{align*}$$

So someone testing negative almost certainly does not have the disease. The lesson is that the accuracy of a test tells you very little on its own; the base rate $P(D)$ matters just as much, and ignoring it is called the base rate fallacy. In practice this is why a cheap screening test is followed up by a more expensive confirmatory one.

Example. Modify the previous example by supposing a fraction $x$ of the population has disease $X$, and that $5\%$ of a large random sample tests positive. What percentage of the population has the disease?
By the law of total probability,

$$\begin{align*} 0.05 = P(T_p) &= P(T_p\mid D)P(D) + P(T_p\mid D^c)P(D^c) \\ &= 0.99x + 0.02(1-x) \\ &= 0.97x + 0.02. \end{align*}$$

Solving, $0.97x = 0.03$, so

$$x = \frac{0.03}{0.97} \approx 0.031.$$

Therefore about $3.1\%$ of the population has the disease. Notice that the observed positive rate of $5\%$ substantially overstates the true prevalence of $3.1\%$, again because of false positives among the large healthy majority.

9.2.4 Statistical independence#

Sometimes learning that $B$ occurred tells us nothing at all about $A$. That deserves a definition.

When $P(B) > 0$ this is equivalent to $P(A\mid B) = P(A)$, which is the more intuitive form: conditioning on $B$ leaves the probability of $A$ unchanged. The product form is preferred as the definition because it is symmetric in $A$ and $B$ and does not require $P(B) > 0$.

Independent and mutually exclusive are completely different things, and confusing them is the most common conceptual error in this chapter. In fact they are almost opposites: if $A$ and $B$ are mutually exclusive with $P(A), P(B) > 0$, then $P(A\cap B) = P(\varnothing) = 0$ but $P(A)P(B) > 0$, so they are necessarily dependent. Intuitively, if $A$ and $B$ cannot both happen, then learning $A$ occurred tells you a great deal about $B$ — namely that it did not occur.

Example. Roll a die; let $A$ be rolling a six and $B$ rolling an even number. Are $A$ and $B$ independent?
Here $P(A) = \frac16$, $P(B) = \frac12$ and $P(A\cap B) = P(\{6\}) = \frac16$. But

$$P(A)P(B) = \frac16\times\frac12 = \frac{1}{12} \neq \frac16 = P(A\cap B),$$

so they are not independent. This matches the earlier calculation that $P(A\mid B) = \frac13 \neq \frac16 = P(A)$.

Example. Roll a die twice. For each $i$, let $X_i$ be the event that the first roll gives $i$ and $Y_i$ that the second roll gives $i$. Then $P(X_i) = P(Y_j) = \frac16$, and since each of the $36$ ordered pairs is equally likely,

$$P(X_i\cap Y_j) = \frac{1}{36} = \frac16\times\frac16 = P(X_i)P(Y_j),$$

so $X_i$ and $Y_j$ are independent for all $i$ and $j$ — as they should be, since the die has no memory.

For more than two events, independence has to be required for every sub-collection, not just for pairs.

So $A$, $B$, $C$ are mutually independent exactly when all four of these hold:

$$\begin{align*} P(A\cap B) &= P(A)P(B), \\ P(A\cap C) &= P(A)P(C), \\ P(B\cap C) &= P(B)P(C), \\ P(A\cap B\cap C) &= P(A)P(B)P(C). \end{align*}$$

Pairwise independence does not imply mutual independence. The standard counterexample: draw a ball at random from a bag of four balls marked $0, 1, 2, 3$, and let $A$, $B$, $C$ be the events that the ball is $1$ or $3$, that it is $2$ or $3$, and that it is $0$ or $3$. Each has probability $\frac12$, and each pairwise intersection is $\{3\}$ with probability $\frac14 = \frac12\times\frac12$, so the events are pairwise independent. But $A\cap B\cap C = \{3\}$ also has probability $\frac14$, whereas $P(A)P(B)P(C) = \frac18$. So the three are pairwise but not mutually independent.

Basically, independence survives complementing — if $A$ tells you nothing about $B$, then $A$ tells you nothing about "not $B$" either. This is what lets us compute "at least one" probabilities by complementing each event separately.

Example. (Reliability.) A system has three components that fail independently, with failure probabilities $0.1$, $0.2$ and $0.3$. Find the probability the system works, if it needs (a) all three components working, and (b) at least one component working.
Let $F_1, F_2, F_3$ be the failure events, so the working events $F_i^c$ have probabilities $0.9$, $0.8$ and $0.7$ and are mutually independent by the theorem above.

(a) All three working:

$$P(F_1^c\cap F_2^c\cap F_3^c) = 0.9\times0.8\times0.7 = 0.504.$$

(b) At least one working is the complement of all three failing, so

$$\begin{align*} P(\text{at least one works}) &= 1 - P(F_1\cap F_2\cap F_3) \\ &= 1 - 0.1\times0.2\times0.3 \\ &= 1 - 0.006 \\ &= 0.994. \end{align*}$$

Therefore the series system works with probability $0.504$ and the parallel system with probability $0.994$. Notice how much redundancy buys you — and notice that (b) was computed via the complement, because "at least one" as a union would have needed the full three-set inclusion–exclusion formula.

9.3 Random variables#

Describing events as subsets of $S$ gets clumsy fast, and it gives us nothing to do arithmetic with. The fix is to attach a number to each outcome.

A random variable is neither random nor a variable; it is a function. The name is entirely historical and actively misleading, so it is worth fixing the correct picture early: $X$ takes an outcome $s \in S$ and returns a number $X(s)$. The randomness lives in which $s$ occurs, not in $X$.

Example. Toss a coin, with $S = \{H, T\}$. Define $X(H) = 1$ and $X(T) = 0$. Then $X$ counts the heads.

Example. Toss a coin three times and let $X$ count the number of heads. Then $X(HHT) = 2$, $X(TTT) = 0$, and so on. The event "exactly two heads" is now written compactly as $\{X = 2\}$, meaning $\{s \in S : X(s) = 2\} = \{HHT, HTH, THH\}$, and

$$P(X = 2) = \frac{3}{8}.$$

This notation is the whole point: $\{X = 2\}$, $\{X \leq 2\}$ and $\{X > 0\}$ are all just events, described far more conveniently than by listing outcomes.

9.3.1 Discrete random variables#

By the summation theorem, the $p_k$ are non-negative and sum to $1$; these two conditions characterise a valid probability distribution, and checking that your probabilities sum to $1$ is the cheapest error check available.

Example. Toss a coin three times and let $X$ count the heads. Since all eight outcomes are equally likely and the numbers of outcomes giving $0,1,2,3$ heads are $1,3,3,1$:

and indeed $\frac18+\frac38+\frac38+\frac18 = 1$.

Example. Roll a die twice and let $X$ be the sum of the two rolls. The sample space has $36$ equally likely ordered pairs, and counting those summing to each value:

The numerators sum to $36$, as they must. Therefore $7$ is the most likely total, with probability $\frac16$.

9.3.2 The mean and variance of a discrete random variable#

Random variables let us do arithmetic on outcomes. The two numbers we care about most are the weighted average of the values, and a measure of how spread out they are.

Basically, $E(X)$ is the long-run average value of $X$ if the experiment were repeated many times, with each possible value weighted by how often it occurs. The expected value need not be a value $X$ can actually take — the expected number of heads in three tosses is $\frac32$, which is impossible on any single trial.

Example. Toss a coin three times and let $X$ count the heads. Using the table above,

$$\begin{align*} E(X) &= 0\times\tfrac18 + 1\times\tfrac38 + 2\times\tfrac38 + 3\times\tfrac18 \\ &= \tfrac{0+3+6+3}{8} = \tfrac{12}{8} = \tfrac32. \end{align*}$$

Therefore $E(X) = \frac32$, which is exactly what symmetry suggests: half of three tosses.

This is more useful than it looks: it says you can compute $E(g(X))$ without first finding the distribution of $g(X)$, which would usually be a nuisance. In particular $E(X^2) = \sum_k x_k^2p_k$.

A warning that is worth a lot of marks: in general $E(g(X)) \neq g(E(X))$. For instance with the coin example, $E(X^2) = \frac{0+3+12+9}{8} = 3$ while $\big(E(X)\big)^2 = \frac94$. The two are only equal when $g$ is linear.

Basically, the variance is the average squared distance from the mean; squaring stops positive and negative deviations from cancelling. The standard deviation then undoes the squaring so that the answer is back in the original units, which is why it is the more interpretable of the two.

Proof. Write $\mu = E(X)$. Expanding the square,

$$\begin{align*} \operatorname{Var}(X) &= \sum_k(x_k-\mu)^2p_k \\ &= \sum_k(x_k^2 - 2x_k\mu + \mu^2)p_k \\ &= \sum_kx_k^2p_k - 2\mu\sum_kx_kp_k + \mu^2\sum_kp_k \\ &= E(X^2) - 2\mu\cdot\mu + \mu^2\cdot 1 \\ &= E(X^2) - \big(E(X)\big)^2. \qquad \blacksquare \end{align*}$$

This is the formula to use for hand calculation, since it needs only two sums rather than a subtraction inside every term. As a free sanity check, $\operatorname{Var}(X) \geq 0$ always; if your computation gives a negative variance you have made an arithmetic error, most often by squaring the mean before rather than after averaging.

Example. Toss a coin three times and let $X$ count the heads. We found $E(X) = \frac32$ and $E(X^2) = 3$, so

$$\operatorname{Var}(X) = 3 - \left(\frac32\right)^2 = 3 - \frac94 = \frac34.$$

Therefore $\operatorname{Var}(X) = \frac34$ and $\operatorname{SD}(X) = \frac{\sqrt3}{2} \approx 0.866$.

Example. A random variable $X$ has the distribution

Find its mean, variance and standard deviation.
First check the probabilities sum to $0.2+0.06+0.2+0.04+0.5 = 1$. Then

$$\begin{align*} E(X) &= 0(0.2) + 1(0.06) + 2(0.2) + 4(0.04) + 7(0.5) \\ &= 0 + 0.06 + 0.4 + 0.16 + 3.5 \\ &= 4.12, \\ E(X^2) &= 0^2(0.2) + 1^2(0.06) + 2^2(0.2) + 4^2(0.04) + 7^2(0.5) \\ &= 0 + 0.06 + 0.8 + 0.64 + 24.5 \\ &= 26.0, \end{align*}$$

so

$$\operatorname{Var}(X) = 26.0 - (4.12)^2 = 26.0 - 16.9744 = 9.0256,$$

and $\operatorname{SD}(X) = \sqrt{9.0256} \approx 3.004$. Therefore the values of $X$ sit on average about $3$ away from the mean of $4.12$. (The printed course notes give the variance here as $9.256$; that is a typo — $26 - 4.12^2 = 9.0256$, and their own conclusion that the standard deviation is roughly $3$ matches the corrected figure.)

Proof. For the mean,

$$E(aX+b) = \sum_k(ax_k+b)p_k = a\sum_kx_kp_k + b\sum_kp_k = aE(X)+b.$$

For the variance, using the mean result,

$$\begin{align*} \operatorname{Var}(aX+b) &= E\Big(\big(aX+b - E(aX+b)\big)^2\Big) \\ &= E\Big(\big(aX + b - aE(X) - b\big)^2\Big) \\ &= E\Big(a^2\big(X-E(X)\big)^2\Big) \\ &= a^2\operatorname{Var}(X), \end{align*}$$