Linear Transformations

7.1 Introduction to linear maps

Recall that a function $f:X\to Y$ is a rule which assigns exactly one element $y=f(x)$ of the codomain $Y$ to each element $x$ of the domain $X$. The element $x$ is called an argument of the function, and $y=f(x)$ is called the image of $x$ under $f$.

Linear maps are a special class of functions where both the domain and the codomain are vector spaces, and where the function "preserves" the two fundamental vector space operations; adding two vectors then applying the map gives the same result as applying the map first and then adding, and the same goes for scalar multiplication.

Basically, a linear map is a function where the order of operations does not matter; you can "add then map" or "map then add", "scale then map" or "map then scale", and you always land in the same place. The vector space operations pass straight through the function untouched. Geometric operations like rotating, reflecting and projecting behave exactly like this, and so do calculus operations like differentiating and integrating, which is why linear maps show up everywhere.

Example. Show that the function $T:\mathbb{R}\to \mathbb{R}$ defined by $T(x)=a_0+a_{1}x$, where $a_0,a_1\in \mathbb{R}$ are constants, is a linear map if and only if $a_0=0$.

The domain and codomain are both $\mathbb{R}$, which is a vector space, so we check the two conditions. For $x,x^{\prime }\in \mathbb{R}$,

$$\begin{array}{rl}T(x+x^{\prime })& =a_0+a_1(x+x^{\prime }),\end{array}$$

whereas

$$\begin{array}{rl}T(x)+T(x^{\prime })& =(a_0+a_{1}x)+(a_0+a_1{x}^{\prime })\\ & =2a_0+a_1(x+x^{\prime }).\end{array}$$

The two expressions are equal if and only if $a_0=2a_0$, that is, if and only if $a_0=0$. In that case we check the scalar multiplication condition:

$$\begin{array}{rl}T(\lambda x)& =a_1(\lambda x)\\ & =\lambda (a_{1}x)\\ & =\lambda T(x).\end{array}$$

Therefore, $T$ is a linear map if and only if $a_0=0$.

Notice how $y=a_0+a_{1}x$ is the equation of a line in ${\mathbb{R}}^2$; the equation of a line defines a linear map if and only if the line passes through the origin. It is important to remember that a "linear polynomial" like $T(x)=3x+2$ is NOT a linear map; the constant term must be zero.

Example. Show that the function $T:{\mathbb{R}}^3\to {\mathbb{R}}^2$ defined by

$$T(\mathbf{x})=\left(\begin{array}{c}-5x_2+4x_3\\ x_1+2x_3\end{array}\right)\text{for}\mathbf{x}=\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right)\in {\mathbb{R}}^3,$$

is a linear map.

Both ${\mathbb{R}}^3$ and ${\mathbb{R}}^2$ are vector spaces, so we just check the two conditions.

Addition condition. For $\mathbf{x},{\mathbf{x}}^{\prime }\in {\mathbb{R}}^3$,

$$\begin{array}{rl}T(\mathbf{x}+{\mathbf{x}}^{\prime })& =\left(\begin{array}{c}-5(x_2+x_2^{\prime })+4(x_3+x_3^{\prime })\\ (x_1+x_1^{\prime })+2(x_3+x_3^{\prime })\end{array}\right)\\ & =\left(\begin{array}{c}-5x_2+4x_3\\ x_1+2x_3\end{array}\right)+\left(\begin{array}{c}-5x_2^{\prime }+4x_3^{\prime }\\ x_1^{\prime }+2x_3^{\prime }\end{array}\right)\\ & =T(\mathbf{x})+T({\mathbf{x}}^{\prime }).\end{array}$$

Scalar multiplication condition. For $\mathbf{x}\in {\mathbb{R}}^3$ and $\lambda \in \mathbb{R}$,

$$\begin{array}{rl}T(\lambda \mathbf{x})& =\left(\begin{array}{c}-5(\lambda x_2)+4(\lambda x_3)\\ \lambda x_1+2(\lambda x_3)\end{array}\right)\\ & =\lambda \left(\begin{array}{c}-5x_2+4x_3\\ x_1+2x_3\end{array}\right)\\ & =\lambda T(\mathbf{x}).\end{array}$$

Both conditions hold, and therefore $T$ is a linear map.

Quick ways to show a map is NOT linear

Every linear map is forced to behave nicely on the zero vector and on negatives.

Proof. Since $0\mathbf{v}=0$ in any vector space, the scalar multiplication condition gives

$$\begin{array}{rl}T(0)& =T(0\mathbf{v})\\ & =0T(\mathbf{v})\\ & =0.\end{array}$$

Similarly, since $-\mathbf{v}=(-1)\mathbf{v}$,

$$\begin{array}{rl}T(-\mathbf{v})& =T((-1)\mathbf{v})\\ & =(-1)T(\mathbf{v})\\ & =-T(\mathbf{v}).■\end{array}$$

Basically, a linear map must send the zero vector of the domain to the zero vector of the codomain. This gives you the fastest possible "not linear" test: plug in $0$ first, and if you do not get $0$ out, you are done. If $T(0)=0$ does hold, hunt for a specific numerical counterexample to one of the two conditions instead (same idea as disproving closure in subspace questions).

Example. Show that the function $T:{\mathbb{R}}^2\to \mathbb{R}$ defined by $T\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)=4x_1+3(x_2-6)$ is not linear.

We have

$$\begin{array}{rl}T\left(\begin{array}{c}0\\ 0\end{array}\right)& =4(0)+3(0-6)\\ & =-18\\ & \ne 0.\end{array}$$

Since $T(0)\ne 0$, $T$ is not a linear map.

Example. Show that the function $T:\mathbb{R}\to \mathbb{R}$ defined by $T(x)=x^2$ is not linear.

Here $T(0)=0$, so the quick test tells us nothing; we look for a numerical counterexample to the scalar multiplication condition:

$$\begin{array}{rl}T(2\times 3)& =36,\\ 2T(3)& =2\times 9=18.\end{array}$$

Since $T(2\times 3)\ne 2T(3)$, $T$ is not a linear map.

The converse of Proposition 1 is NOT true; a function can pass both the $T(0)=0$ and $T(-\mathbf{v})=-T(\mathbf{v})$ tests and still fail to be linear.

Example. The function $T(x)=x^3$ satisfies $T(0)=0$ and $T(-x)=-T(x)$, but

$$\begin{array}{rl}T(2\times 1)& =8,\\ 2T(1)& =2,\end{array}$$

so $T$ is not a linear map. Passing the quick tests is necessary but never sufficient.

The one-condition test and linear combinations

The two conditions in the definition can be merged into a single condition, which halves the amount of algebra you have to do in a "show that $T$ is linear" question.

Proof. If $T$ is linear, then by the addition condition followed by the scalar multiplication condition (used twice),

$$\begin{array}{rl}T({\lambda }_1{\mathbf{v}}_1+{\lambda }_2{\mathbf{v}}_2)& =T({\lambda }_1{\mathbf{v}}_1)+T({\lambda }_2{\mathbf{v}}_2)\\ & ={\lambda }_{1}T({\mathbf{v}}_1)+{\lambda }_{2}T({\mathbf{v}}_2).\end{array}$$

Conversely, if the combined condition holds, then choosing ${\lambda }_1={\lambda }_2=1$ recovers the addition condition, and choosing ${\lambda }_2=0$ recovers the scalar multiplication condition. $■$

Example. Show that the function $T:{\mathbb{R}}^2\to {\mathbb{R}}^3$ defined by

$$T(\mathbf{x})=\left(\begin{array}{c}3x_1-x_2\\ 4x_2\\ 5x_1+6x_2\end{array}\right)\text{for}\mathbf{x}=\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)\in {\mathbb{R}}^2,$$

is a linear map.

For $\mathbf{x},{\mathbf{x}}^{\prime }\in {\mathbb{R}}^2$ and $\lambda ,\mu \in \mathbb{R}$, we have $\lambda \mathbf{x}+\mu {\mathbf{x}}^{\prime }=\left(\begin{array}{c}\lambda x_1+\mu x_1^{\prime }\\ \lambda x_2+\mu x_2^{\prime }\end{array}\right)$, and hence

$$\begin{array}{rl}T(\lambda \mathbf{x}+\mu {\mathbf{x}}^{\prime })& =\left(\begin{array}{c}3(\lambda x_1+\mu x_1^{\prime })-(\lambda x_2+\mu x_2^{\prime })\\ 4(\lambda x_2+\mu x_2^{\prime })\\ 5(\lambda x_1+\mu x_1^{\prime })+6(\lambda x_2+\mu x_2^{\prime })\end{array}\right)\\ & =\lambda \left(\begin{array}{c}3x_1-x_2\\ 4x_2\\ 5x_1+6x_2\end{array}\right)+\mu \left(\begin{array}{c}3x_1^{\prime }-x_2^{\prime }\\ 4x_2^{\prime }\\ 5x_1^{\prime }+6x_2^{\prime }\end{array}\right)\\ & =\lambda T(\mathbf{x})+\mu T({\mathbf{x}}^{\prime }).\end{array}$$

Therefore, by Theorem 2, $T$ is a linear map.

An induction argument extends the one-condition test from two vectors to any finite linear combination.

In other words, linear maps preserve linear combinations. This has a very useful consequence: if you know what $T$ does to a basis, you know what $T$ does to everything, because every vector is a unique linear combination of the basis vectors.

Example. Let $T:{\mathbb{R}}^3\to {\mathbb{R}}^2$ be a linear map with

$$T\left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)=\left(\begin{array}{c}3\\ 7\end{array}\right),T\left(\begin{array}{c}0\\ 1\\ 0\end{array}\right)=\left(\begin{array}{c}-5\\ 6\end{array}\right),T\left(\begin{array}{c}0\\ 0\\ 1\end{array}\right)=\left(\begin{array}{c}-2\\ 8\end{array}\right).$$

Find the function value at $\mathbf{x}=\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right)$.

Since $\mathbf{x}=x_1{\mathbf{e}}_1+x_2{\mathbf{e}}_2+x_3{\mathbf{e}}_3$, Theorem 3 gives

$$\begin{array}{rl}T(\mathbf{x})& =x_{1}T({\mathbf{e}}_1)+x_{2}T({\mathbf{e}}_2)+x_{3}T({\mathbf{e}}_3)\\ & =x_1\left(\begin{array}{c}3\\ 7\end{array}\right)+x_2\left(\begin{array}{c}-5\\ 6\end{array}\right)+x_3\left(\begin{array}{c}-2\\ 8\end{array}\right)\\ & =\left(\begin{array}{c}3x_1-5x_2-2x_3\\ 7x_1+6x_2+8x_3\end{array}\right).\end{array}$$

Therefore, three function values were enough to pin down the whole map.

Example. Show that the function $T:{\mathbb{R}}^3\to {\mathbb{R}}^2$ with the same three values as above, but which also satisfies $T\left(\begin{array}{c}1\\ 1\\ 1\end{array}\right)=\left(\begin{array}{c}-4\\ 20\end{array}\right)$, is not a linear map.

If $T$ were linear, then since $\left(\begin{array}{c}1\\ 1\\ 1\end{array}\right)={\mathbf{e}}_1+{\mathbf{e}}_2+{\mathbf{e}}_3$, Theorem 3 would force

$$\begin{array}{rl}T\left(\begin{array}{c}1\\ 1\\ 1\end{array}\right)& =T({\mathbf{e}}_1)+T({\mathbf{e}}_2)+T({\mathbf{e}}_3)\\ & =\left(\begin{array}{c}3\\ 7\end{array}\right)+\left(\begin{array}{c}-5\\ 6\end{array}\right)+\left(\begin{array}{c}-2\\ 8\end{array}\right)\\ & =\left(\begin{array}{c}-4\\ 21\end{array}\right).\end{array}$$

But we are told $T\left(\begin{array}{c}1\\ 1\\ 1\end{array}\right)=\left(\begin{array}{c}-4\\ 20\end{array}\right)\ne \left(\begin{array}{c}-4\\ 21\end{array}\right)$, and hence $T$ is not a linear map.

Common exam traps

The strategy for "is $T$ linear?" questions: first check the domain and codomain are actually vector spaces, then check $T(0)=0$, then try small specific vectors (especially $\lambda =-1$), and only commit to the general algebra of Theorem 2 once nothing has broken.

Example (trap: translation). Is $T:{\mathbb{R}}^2\to {\mathbb{R}}^2$ defined by $T(\mathbf{x})=\mathbf{x}+\left(\begin{array}{c}1\\ 1\end{array}\right)$ a linear map?

No;

$$\begin{array}{rl}T\left(\begin{array}{c}0\\ 0\end{array}\right)& =\left(\begin{array}{c}1\\ 1\end{array}\right)\\ & \ne \left(\begin{array}{c}0\\ 0\end{array}\right).\end{array}$$

Therefore a translation is never a linear map (unless you translate by $0$), even though sliding the plane around feels like the most "linear" thing imaginable.

Example (trap: absolute value). Is $T:\mathbb{R}\to \mathbb{R}$ defined by $T(x)=|x|$ a linear map?

Here $T(0)=0$, so the quick test passes. Try $\lambda =-1$:

$$\begin{array}{rl}T((-1)\times 1)& =|-1|=1,\\ (-1)T(1)& =-1.\end{array}$$

Since $1\ne -1$, the scalar multiplication condition fails and $T$ is not linear. Whenever a formula folds negatives back onto positives, $\lambda =-1$ is the counterexample to reach for.

Example (trap: determinant). Is $T:M_{22}(\mathbb{R})\to \mathbb{R}$ defined by $T(A)=\det (A)$ a linear map?

The domain and codomain are vector spaces and $\det (O)=0$, so we test the conditions on the identity matrix $I$:

$$\begin{array}{rl}T(2I)& =\det \left(\begin{array}{cc}2& 0\\ 0& 2\end{array}\right)=4,\\ 2T(I)& =2\times 1=2.\end{array}$$

Since $4\ne 2$, $T$ is not linear. In fact $\det (\lambda A)={\lambda }^2\det (A)$ for $2\times 2$ matrices, so the determinant scales quadratically, not linearly.

Example (trap: the domain matters). Is $S:[-1,1]\to \mathbb{R}$ defined by $S(x)=5x$ a linear map?

No, and not because of the formula; the domain $[-1,1]$ is not a vector space (it is not closed under addition or scalar multiplication), so $S$ is disqualified before we even check the conditions. The function $T:\mathbb{R}\to \mathbb{R}$ with the same rule $T(x)=5x$ is linear. Always check that the domain and codomain are vector spaces before checking the two conditions.

7.2 Linear maps from ${\mathbb{R}}^n$ to ${\mathbb{R}}^m$ and $m\times n$ matrices

If you look back at the examples in the previous section, every linear map from ${\mathbb{R}}^n$ to ${\mathbb{R}}^m$ we wrote down could have been written as $T(\mathbf{x})=A\mathbf{x}$ for some $m\times n$ matrix $A$. This is no accident; in this section we show that every matrix defines a linear map and, conversely, every linear map from ${\mathbb{R}}^n$ to ${\mathbb{R}}^m$ is a matrix map.

Proof. This follows straight from the matrix arithmetic rules of 1131. For all $\mathbf{x},{\mathbf{x}}^{\prime }\in {\mathbb{R}}^n$ and $\lambda \in \mathbb{R}$,

$$\begin{array}{rl}{T}_A(\mathbf{x}+{\mathbf{x}}^{\prime })& =A(\mathbf{x}+{\mathbf{x}}^{\prime })\\ & =A\mathbf{x}+A{\mathbf{x}}^{\prime }\\ & =T_A(\mathbf{x})+T_A({\mathbf{x}}^{\prime }),\end{array}$$

and

$$\begin{array}{rl}{T}_A(\lambda \mathbf{x})& =A(\lambda \mathbf{x})\\ & =\lambda (A\mathbf{x})\\ & =\lambda T_A(\mathbf{x}).■\end{array}$$

Example. Describe the linear map $T_A$ such that $T_A(\mathbf{x})=A\mathbf{x}$ for the matrix

$$A=\left(\begin{array}{cc}3& 4\\ -1& 0\\ -5& 6\end{array}\right).$$

Since $A$ has $3$ rows and $2$ columns, the domain is ${\mathbb{R}}^2$ and the codomain is ${\mathbb{R}}^3$ (an $m\times n$ matrix eats vectors of length $n$ and spits out vectors of length $m$), and

$$T_A\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)=A\mathbf{x}=\left(\begin{array}{c}3x_1+4x_2\\ -x_1\\ -5x_1+6x_2\end{array}\right).$$

Therefore every matrix defines a linear map just by multiplication.

The converse direction is the important one.

Proof. Every $\mathbf{x}\in {\mathbb{R}}^n$ can be written uniquely as $\mathbf{x}=x_1{\mathbf{e}}_1+\cdots +x_n{\mathbf{e}}_n$. By Theorem 3 of Section 7.1,

$$\begin{array}{rl}T(\mathbf{x})& =T(x_1{\mathbf{e}}_1+\cdots +x_n{\mathbf{e}}_n)\\ & =x_{1}T({\mathbf{e}}_1)+\cdots +x_{n}T({\mathbf{e}}_n)\\ & =x_1{\mathbf{a}}_1+\cdots +x_n{\mathbf{a}}_n\\ & =A\mathbf{x},\end{array}$$

where the last step uses the fact (Proposition 3 of Section 6.4) that a linear combination of the columns of $A$ is exactly the matrix product $A\mathbf{x}$. $■$

The take-away formula is

$$\boxed{A=(T({\mathbf{e}}_1)|T({\mathbf{e}}_2)|\cdots |T({\mathbf{e}}_n))},$$

i.e. the $j$​th column of the matrix is the image of the $j$​th standard basis vector. This is worth internalising; it is how you will construct every geometric matrix in Section 7.3.

Example. Find a matrix $A$ such that $T(\mathbf{x})=A\mathbf{x}$ for the linear map $T:{\mathbb{R}}^3\to {\mathbb{R}}^2$ defined by

$$T\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right)=\left(\begin{array}{c}3x_1-5x_2+6x_3\\ 5x_2+31x_3\end{array}\right).$$

We compute the images of the standard basis vectors:

$$\begin{array}{rl}T({\mathbf{e}}_1)& =\left(\begin{array}{c}3\\ 0\end{array}\right),\\ T({\mathbf{e}}_2)& =\left(\begin{array}{c}-5\\ 5\end{array}\right),\\ T({\mathbf{e}}_3)& =\left(\begin{array}{c}6\\ 31\end{array}\right).\end{array}$$

Therefore, placing these as columns,

$$A=\left(\begin{array}{ccc}3& -5& 6\\ 0& 5& 31\end{array}\right).$$

An alternative (often faster) method is to notice that the components of $T(\mathbf{x})$ look like the left-hand side of a system of linear equations, and simply read off the coefficient matrix.

Example. Find a matrix $A$ such that $T(\mathbf{x})=A\mathbf{x}$ for the linear map $T:{\mathbb{R}}^4\to {\mathbb{R}}^3$ defined by

$$T\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\\ x_4\end{array}\right)=\left(\begin{array}{c}2x_1-3x_2+4x_3-5x_4\\ -2x_1+3x_4\\ x_1-5x_2+6x_3-8x_4\end{array}\right).$$

Reading off the coefficients row by row (remembering to put a $0$ wherever a variable is missing),

$$A=\left(\begin{array}{cccc}2& -3& 4& -5\\ -2& 0& 0& 3\\ 1& -5& 6& -8\end{array}\right).$$

Therefore $T(\mathbf{x})=A\mathbf{x}$ for all $\mathbf{x}\in {\mathbb{R}}^4$; both methods must always give the same matrix.

Example (images of non-standard vectors). A linear map $T:{\mathbb{R}}^2\to {\mathbb{R}}^2$ satisfies

$$T\left(\begin{array}{c}1\\ 1\end{array}\right)=\left(\begin{array}{c}2\\ 0\end{array}\right)\text{and}T\left(\begin{array}{c}1\\ -1\end{array}\right)=\left(\begin{array}{c}0\\ 4\end{array}\right).$$

Find the matrix of $T$ with respect to the standard basis.

The trap here is that we were not given $T({\mathbf{e}}_1)$ and $T({\mathbf{e}}_2)$; we have to build them, using the fact that $T$ preserves linear combinations. Since ${\mathbf{e}}_1=\frac{1}{2}\left(\begin{array}{c}1\\ 1\end{array}\right)+\frac{1}{2}\left(\begin{array}{c}1\\ -1\end{array}\right)$,

$$\begin{array}{rl}T({\mathbf{e}}_1)& =\frac{1}{2}T\left(\begin{array}{c}1\\ 1\end{array}\right)+\frac{1}{2}T\left(\begin{array}{c}1\\ -1\end{array}\right)\\ & =\frac{1}{2}\left(\begin{array}{c}2\\ 0\end{array}\right)+\frac{1}{2}\left(\begin{array}{c}0\\ 4\end{array}\right)\\ & =\left(\begin{array}{c}1\\ 2\end{array}\right).\end{array}$$

Similarly, ${\mathbf{e}}_2=\frac{1}{2}\left(\begin{array}{c}1\\ 1\end{array}\right)-\frac{1}{2}\left(\begin{array}{c}1\\ -1\end{array}\right)$, and hence

$$\begin{array}{rl}T({\mathbf{e}}_2)& =\frac{1}{2}\left(\begin{array}{c}2\\ 0\end{array}\right)-\frac{1}{2}\left(\begin{array}{c}0\\ 4\end{array}\right)\\ & =\left(\begin{array}{c}1\\ -2\end{array}\right).\end{array}$$

Therefore

$$A=\left(\begin{array}{cc}1& 1\\ 2& -2\end{array}\right),$$

and a quick sanity check confirms $A\left(\begin{array}{c}1\\ 1\end{array}\right)=\left(\begin{array}{c}2\\ 0\end{array}\right)$ as required. This works because $\left\{\left(\begin{array}{c}1\\ 1\end{array}\right),\left(\begin{array}{c}1\\ -1\end{array}\right)\right\}$ is a basis of ${\mathbb{R}}^2$; by Theorem 4, values on a basis determine the map completely.

7.3 Geometric examples of linear transformations

Many of the standard geometric operations on ${\mathbb{R}}^2$ and ${\mathbb{R}}^3$ are linear maps: stretching, compressing, reflecting, rotating and projecting. For each of them, the Matrix Representation Theorem gives a mechanical recipe for the matrix: work out where ${\mathbf{e}}_1$ and ${\mathbf{e}}_2$ land, and make those the columns.

A useful general fact first: linear maps send lines to lines (or squash them to points).

Proof. A line is the set $\{\mathbf{x}\in {\mathbb{R}}^n:\mathbf{x}=\mathbf{a}+\lambda \mathbf{v},\lambda \in \mathbb{R}\}$ with $\mathbf{v}\ne 0$. By Theorem 2 of Section 7.1, $T(\mathbf{a}+\lambda \mathbf{v})=T(\mathbf{a})+\lambda T(\mathbf{v})$, so the image is $\{\mathbf{y}\in {\mathbb{R}}^m:\mathbf{y}=T(\mathbf{a})+\lambda T(\mathbf{v}),\lambda \in \mathbb{R}\}$; this is a line when $T(\mathbf{v})\ne 0$ and the single point $T(\mathbf{a})$ otherwise. $■$

The same argument shows a line segment maps to the segment between the images of the endpoints, which is why it is enough to transform the vertices of a shape and then reconnect them.

Stretching and dilation

A $2\times 2$ diagonal matrix with positive entries,

$$A=\left(\begin{array}{cc}{\lambda }_1& 0\\ 0& {\lambda }_2\end{array}\right)\text{with}{\lambda }_1,{\lambda }_2>0,$$

multiplies the first component by ${\lambda }_1$ and the second by ${\lambda }_2$; each axis direction is stretched (if ${\lambda }_i>1$) or compressed (if ${\lambda }_i<1$) independently. When ${\lambda }_1={\lambda }_2=\lambda$ the whole plane is scaled uniformly by $\lambda$ (a dilation).

Example. Apply $A=\left(\begin{array}{cc}2& 0\\ 0& \frac{1}{2}\end{array}\right)$ to the point $(3,4)$.

$$\begin{array}{rl}A\left(\begin{array}{c}3\\ 4\end{array}\right)& =\left(\begin{array}{c}2(3)+0(4)\\ 0(3)+\frac{1}{2}(4)\end{array}\right)\\ & =\left(\begin{array}{c}6\\ 2\end{array}\right).\end{array}$$

Therefore the point is pushed out to twice the horizontal distance and squashed to half the vertical distance.

Reflections

Example. Reflection in the $x_1$-axis sends $\mathbf{x}=\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)$ to ${\mathbf{x}}^{\prime }=\left(\begin{array}{c}{x}_1\\ -x_2\end{array}\right)$, and is represented by

$$A=\left(\begin{array}{cc}1& 0\\ 0& -1\end{array}\right),$$

since $A\mathbf{x}=\left(\begin{array}{c}{x}_1\\ -x_2\end{array}\right)={\mathbf{x}}^{\prime }$. Note that finding a matrix which performs the transformation is itself the proof that the transformation is linear. Similarly, reflection in the $x_2$-axis has matrix $\left(\begin{array}{cc}-1& 0\\ 0& 1\end{array}\right)$.

Example (matrix from a geometric description). Find the matrix of the reflection in the line $x_2=x_1$ in ${\mathbb{R}}^2$.

We only need the images of the standard basis vectors. Reflecting in the line $x_2=x_1$ swaps the two coordinate axes, so

$$\begin{array}{rl}T({\mathbf{e}}_1)& =\left(\begin{array}{c}0\\ 1\end{array}\right),\\ T({\mathbf{e}}_2)& =\left(\begin{array}{c}1\\ 0\end{array}\right).\end{array}$$

Therefore

$$A=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right),$$

and as a check, $A\left(\begin{array}{c}3\\ 1\end{array}\right)=\left(\begin{array}{c}1\\ 3\end{array}\right)$, which is exactly the mirror image of $(3,1)$ in that line. This "track ${\mathbf{e}}_1$ and ${\mathbf{e}}_2$, then check on one extra point" routine handles basically every geometric matrix question.

Rotations

Example. Let $R_{\alpha }:{\mathbb{R}}^2\to {\mathbb{R}}^2$ rotate every point anticlockwise about the origin by an angle $\alpha$. Rotation is linear; rotating $\mathbf{a}+\mathbf{b}$ gives the same result as rotating $\mathbf{a}$ and $\mathbf{b}$ separately and then adding (the whole parallelogram rotates rigidly), and the same for scaling. To find the matrix, rotate the standard basis vectors: ${\mathbf{e}}_1$ sits at angle $0$ on the unit circle and moves to angle $\alpha$, while ${\mathbf{e}}_2$ sits at angle $\frac{\pi }{2}$ and moves to angle $\frac{\pi }{2}+\alpha$, so

$$R_{\alpha }({\mathbf{e}}_1)=\left(\begin{array}{c}\cos \alpha \\ \sin \alpha \end{array}\right)\text{and}{R}_{\alpha }({\mathbf{e}}_2)=\left(\begin{array}{c}-\sin \alpha \\ \cos \alpha \end{array}\right).$$

Therefore the rotation matrix for angle $\alpha$ is

$$\boxed{A_{\alpha }=\left(\begin{array}{cc}\cos \alpha & -\sin \alpha \\ \sin \alpha & \cos \alpha \end{array}\right)}.$$

Example. Find the matrix for rotation by $\frac{\pi }{3}$ anticlockwise, and rotate the point $(2,0)$.

$$A_{\pi /3}=\left(\begin{array}{cc}\cos \frac{\pi }{3}& -\sin \frac{\pi }{3}\\ \sin \frac{\pi }{3}& \cos \frac{\pi }{3}\end{array}\right)=\left(\begin{array}{cc}\frac{1}{2}& -\frac{\sqrt{3}}{2}\\ \frac{\sqrt{3}}{2}& \frac{1}{2}\end{array}\right).$$

Then

$$\begin{array}{rl}{A}_{\pi /3}\left(\begin{array}{c}2\\ 0\end{array}\right)& =\left(\begin{array}{c}\frac{1}{2}(2)-\frac{\sqrt{3}}{2}(0)\\ \frac{\sqrt{3}}{2}(2)+\frac{1}{2}(0)\end{array}\right)\\ & =\left(\begin{array}{c}1\\ \sqrt{3}\end{array}\right).\end{array}$$

Therefore $(2,0)$ rotates to $(1,\sqrt{3})$, which still has length $2$, as a rotation should preserve.

Projections and the dot product

Example. Recall from 1131 that the projection of $\mathbf{x}\in {\mathbb{R}}^n$ onto a fixed non-zero vector $\mathbf{b}\in {\mathbb{R}}^n$ is

$${\text{proj}}_{\mathbf{b}}\mathbf{x}=\frac{\mathbf{x}\cdot \mathbf{b}}{|\mathbf{b}{|}^2}\mathbf{b}.$$

Show that $T(\mathbf{x})={\text{proj}}_{\mathbf{b}}\mathbf{x}$ is a linear map.

Rather than arguing geometrically, we use the algebraic properties of the dot product. For all $\mathbf{x},{\mathbf{x}}^{\prime }\in {\mathbb{R}}^n$,

$$\begin{array}{rl}T(\mathbf{x}+{\mathbf{x}}^{\prime })& =\frac{(\mathbf{x}+{\mathbf{x}}^{\prime })\cdot \mathbf{b}}{|\mathbf{b}{|}^2}\mathbf{b}\\ & =\frac{\mathbf{x}\cdot \mathbf{b}+{\mathbf{x}}^{\prime }\cdot \mathbf{b}}{|\mathbf{b}{|}^2}\mathbf{b}\\ & =T(\mathbf{x})+T({\mathbf{x}}^{\prime }),\end{array}$$

and for all $\lambda \in \mathbb{R}$,

$$\begin{array}{rl}T(\lambda \mathbf{x})& =\frac{(\lambda \mathbf{x})\cdot \mathbf{b}}{|\mathbf{b}{|}^2}\mathbf{b}\\ & =\lambda \left(\frac{\mathbf{x}\cdot \mathbf{b}}{|\mathbf{b}{|}^2}\mathbf{b}\right)\\ & =\lambda T(\mathbf{x}).\end{array}$$

Therefore $T$ is a linear map.

Example. Find the matrix of the projection onto $\mathbf{b}=\left(\begin{array}{c}1\\ 2\end{array}\right)$ in ${\mathbb{R}}^2$.

Here $|\mathbf{b}{|}^2=1^2+2^2=5$. The columns of the matrix are the projections of ${\mathbf{e}}_1$ and ${\mathbf{e}}_2$:

$$\begin{array}{rl}T({\mathbf{e}}_1)& =\frac{{\mathbf{e}}_1\cdot \mathbf{b}}{5}\mathbf{b}=\frac{1}{5}\left(\begin{array}{c}1\\ 2\end{array}\right),\\ T({\mathbf{e}}_2)& =\frac{{\mathbf{e}}_2\cdot \mathbf{b}}{5}\mathbf{b}=\frac{2}{5}\left(\begin{array}{c}1\\ 2\end{array}\right).\end{array}$$

Therefore

$$A=\frac{1}{5}\left(\begin{array}{cc}1& 2\\ 2& 4\end{array}\right),$$

and as a check, for $\mathbf{x}=\left(\begin{array}{c}3\\ 1\end{array}\right)$ we have $\mathbf{x}\cdot \mathbf{b}=5$, so ${\text{proj}}_{\mathbf{b}}\mathbf{x}=\left(\begin{array}{c}1\\ 2\end{array}\right)$, and indeed

$$\begin{array}{rl}A\left(\begin{array}{c}3\\ 1\end{array}\right)& =\frac{1}{5}\left(\begin{array}{c}3+2\\ 6+4\end{array}\right)\\ & =\left(\begin{array}{c}1\\ 2\end{array}\right).\end{array}$$

Therefore the matrix reproduces the projection formula exactly.

Example. For a fixed $\mathbf{b}\in {\mathbb{R}}^n$, the function $T:{\mathbb{R}}^n\to \mathbb{R}$ defined by $T(\mathbf{x})=\mathbf{b}\cdot \mathbf{x}$ is a linear map; the proof is the same dot-product algebra as for the projection. Taking $\mathbf{b}={\mathbf{e}}_i$ gives the map which picks out the $i$​th component of $\mathbf{x}$, so "read off a coordinate" is itself a linear map.

7.4 Subspaces associated with linear maps

There are two important subspaces attached to every linear map: the kernel (the set of vectors squashed to zero, living in the domain) and the image (the set of all function values, living in the codomain). Informally, the kernel measures what the map destroys and the image measures what the map can actually reach.

7.4.1 The kernel of a map

For matrix maps the definition specialises to something very familiar.

Basically, finding the kernel of a matrix is nothing new; it is just solving $A\mathbf{x}=0$ by Gaussian elimination like in 1131. Showing a specific vector lies in the kernel is even easier: multiply and see if you get $0$.

Example. Let $A=\left(\begin{array}{cc}1& 2\\ 3& 6\end{array}\right)$ and $\mathbf{x}=\left(\begin{array}{c}2\\ -1\end{array}\right)$. Then

$$\begin{array}{rl}A\mathbf{x}& =\left(\begin{array}{c}1(2)+2(-1)\\ 3(2)+6(-1)\end{array}\right)\\ & =\left(\begin{array}{c}0\\ 0\end{array}\right),\end{array}$$

and hence $\mathbf{x}\in \ker (A)$. Also, $0\in \ker (T)$ for every linear map $T$, since $T(0)=0$; the kernel is never empty.

Example. Find the kernel of

$$A=\left(\begin{array}{cccc}1& 4& 2& 7\\ 3& 6& 0& 15\\ 2& -4& -8& 2\end{array}\right).$$

We solve $A\mathbf{x}=0$. Row reducing,

$$\begin{array}{rlrl}\left(\begin{array}{cccc}1& 4& 2& 7\\ 3& 6& 0& 15\\ 2& -4& -8& 2\end{array}\right)& \sim \left(\begin{array}{cccc}1& 4& 2& 7\\ 0& -6& -6& -6\\ 0& -12& -12& -12\end{array}\right)& & (R_2\to R_2-3R_1,R_3\to R_3-2R_1)\\ & \sim \left(\begin{array}{cccc}1& 4& 2& 7\\ 0& -6& -6& -6\\ 0& 0& 0& 0\end{array}\right)=U& & (R_3\to R_3-2R_2).\end{array}$$

Columns $3$ and $4$ are non-leading, so set $x_3={\lambda }_1$ and $x_4={\lambda }_2$. Back substitution into row $2$ gives

$$\begin{array}{rl}-6x_2-6{\lambda }_1-6{\lambda }_2& =0\\ x_2& =-{\lambda }_1-{\lambda }_2,\end{array}$$

and then row $1$ gives

$$\begin{array}{rl}{x}_1& =-4x_2-2x_3-7x_4\\ & =4{\lambda }_1+4{\lambda }_2-2{\lambda }_1-7{\lambda }_2\\ & =2{\lambda }_1-3{\lambda }_2.\end{array}$$

Hence

$$\begin{array}{rl}\mathbf{x}& =\left(\begin{array}{c}2{\lambda }_1-3{\lambda }_2\\ -{\lambda }_1-{\lambda }_2\\ {\lambda }_1\\ {\lambda }_2\end{array}\right)\\ & ={\lambda }_1\left(\begin{array}{c}2\\ -1\\ 1\\ 0\end{array}\right)+{\lambda }_2\left(\begin{array}{c}-3\\ -1\\ 0\\ 1\end{array}\right),\end{array}$$

and therefore

$$\ker (A)=\left\{\mathbf{x}\in {\mathbb{R}}^4:\mathbf{x}={\lambda }_1\left(\begin{array}{c}2\\ -1\\ 1\\ 0\end{array}\right)+{\lambda }_2\left(\begin{array}{c}-3\\ -1\\ 0\\ 1\end{array}\right)\text{ for }{\lambda }_1,{\lambda }_2\in \mathbb{R}\right\}.$$

Geometrically, the kernel is a plane through the origin in ${\mathbb{R}}^4$.

The word "subspace" in the section title is justified by the following theorem.

Proof. We use the Subspace Theorem. The kernel contains $0$, since $T(0)=0$. Suppose $\mathbf{v},{\mathbf{v}}^{\prime }\in \ker (T)$ and $\lambda \in \mathbb{F}$. Then

$$\begin{array}{rl}T(\mathbf{v}+{\mathbf{v}}^{\prime })& =T(\mathbf{v})+T({\mathbf{v}}^{\prime })\\ & =0+0\\ & =0,\end{array}$$

and

$$\begin{array}{rl}T(\lambda \mathbf{v})& =\lambda T(\mathbf{v})\\ & =\lambda 0\\ & =0,\end{array}$$

so $\ker (T)$ is closed under addition and scalar multiplication, and is therefore a subspace of $V$. $■$

Since the kernel is a subspace, it has a dimension, and that dimension gets a special name.

For a matrix map $T_A$ we have $\ker (T_A)=\ker (A)$ (the conditions $T_A(\mathbf{x})=0$ and $A\mathbf{x}=0$ are literally the same equation), and the nullity can be read off from the row-echelon form.

Example (continued). For the $3\times 4$ matrix $A$ above, the two vectors

$$\left\{\left(\begin{array}{c}2\\ -1\\ 1\\ 0\end{array}\right),\left(\begin{array}{c}-3\\ -1\\ 0\\ 1\end{array}\right)\right\}$$

span $\ker (A)$, and they are linearly independent (look at the third and fourth entries; the only combination giving $0$ there is ${\lambda }_1={\lambda }_2=0$). Therefore they form a basis for $\ker (A)$, and $\text{nullity}(A)=2$, matching the two non-leading columns of $U$.

This follows because the columns are independent exactly when $A\mathbf{x}=0$ has only the zero solution, i.e. when $\ker (A)=\{0\}$.

7.4.2 Image

We have met this set several times before in disguise. Since $A\mathbf{x}$ is a linear combination of the columns of $A$, the image is exactly the span of the columns, and it is also the set of right-hand sides $\mathbf{b}$ for which $A\mathbf{x}=\mathbf{b}$ is solvable:

$$\boxed{\text{im}(A)=\text{col}(A)=\text{span}(\text{columns of }A)=\{\mathbf{b}\in {\mathbb{R}}^m:A\mathbf{x}=\mathbf{b}\text{ has a solution}\}}.$$

Basically, every question about the image of a matrix is secretly a linear equations question you already know how to solve.

Example (continued). Find conditions on $\mathbf{b}$ for $\mathbf{b}\in \text{im}(A)$, where $A$ is the same $3\times 4$ matrix as before.

We ask when $A\mathbf{x}=\mathbf{b}$ has a solution for $\mathbf{b}=\left(\begin{array}{c}{b}_1\\ b_2\\ b_3\end{array}\right)$. Reducing the augmented matrix with a general right-hand side,

$$\begin{array}{rl}\left(\begin{array}{ccccc}1& 4& 2& 7& b_1\\ 3& 6& 0& 15& b_2\\ 2& -4& -8& 2& b_3\end{array}\right)& \sim \left(\begin{array}{ccccc}1& 4& 2& 7& b_1\\ 0& -6& -6& -6& b_2-3b_1\\ 0& -12& -12& -12& b_3-2b_1\end{array}\right)\\ & \sim \left(\begin{array}{ccccc}1& 4& 2& 7& b_1\\ 0& -6& -6& -6& b_2-3b_1\\ 0& 0& 0& 0& 4b_1-2b_2+b_3\end{array}\right).\end{array}$$

The system has a solution if and only if the last row is consistent, that is, if and only if

$$4b_1-2b_2+b_3=0.$$

Therefore $\text{im}(A)$ is the plane through the origin in ${\mathbb{R}}^3$ with normal $\left(\begin{array}{c}4\\ -2\\ 1\end{array}\right)$; a two-dimensional subspace of ${\mathbb{R}}^3$.

Proof. We have $0=T(0)\in \text{im}(T)$. If $\mathbf{w}=T(\mathbf{v})$ and ${\mathbf{w}}^{\prime }=T({\mathbf{v}}^{\prime })$ are in the image and $\lambda \in \mathbb{F}$, then

$$\begin{array}{rl}\mathbf{w}+{\mathbf{w}}^{\prime }& =T(\mathbf{v})+T({\mathbf{v}}^{\prime })\\ & =T(\mathbf{v}+{\mathbf{v}}^{\prime }),\end{array}$$

which is a function value of the vector $\mathbf{v}+{\mathbf{v}}^{\prime }\in V$, and

$$\begin{array}{rl}\lambda \mathbf{w}& =\lambda T(\mathbf{v})\\ & =T(\lambda \mathbf{v}),\end{array}$$

which is a function value of $\lambda \mathbf{v}\in V$. Hence the image is closed under both operations, and by the Subspace Theorem it is a subspace of $W$. $■$

Example (continued). Find $\text{rank}(A)$ and a basis for $\text{im}(A)$.

The row-echelon form $U$ has leading entries in columns $1$ and $2$, so $\text{rank}(A)=2$, and a basis for $\text{im}(A)$ is given by columns $1$ and $2$ of the original matrix $A$: