Taylor Series

4.1 Taylor polynomials

Polynomials are the nicest functions there are. You can evaluate one with a finite number of additions and multiplications, you can differentiate and integrate one in your head, and the results are polynomials again. Almost nothing else is that well behaved, which raises an obvious question: given some function that is not a polynomial, can we find a polynomial that behaves nearly the same?

The motivating problem is embarrassingly concrete. Recall that $f(x)=e^x$ is defined as the inverse of $\ln$, and $\ln$ is itself defined by an integral. So how does one actually evaluate $e^{0.1}$? (If the answer is "use a calculator", then we simply ask how the calculator does it.)

One approach is to set $y=e^{0.1}$, so that we must solve $\ln y=0.1$, i.e. the integral equation

$${\int }_1^y\frac{dt}{t}=0.1,$$

guessing a value of $y$ and checking it with Riemann sums. This is clearly a hopeless way to go about it.

A far better idea is one you have already met: approximate $f$ locally by a linear function, using the fact that the tangent line hugs the graph near the point of contact. For $f(x)=e^x$ the tangent at $0$ is

$$p_1(x)=1+x,$$

so $e^x\approx 1+x$ near $0$, giving $e^{0.1}\approx 1.1$. The thing to notice is why $p_1$ was the right line: it is the unique degree-$1$ polynomial whose value and gradient at $0$ match those of $f$,

$$p_1(0)=f(0),p_1^{\prime }(0)=f^{\prime }(0).$$

That immediately suggests how to do better. Ask for a degree-$2$ polynomial $p_2(x)=b_0+b_{1}x+b_2{x}^2$ matching the value, gradient and concavity of $f$ at $0$:

$$p_2(0)=f(0),p_2^{\prime }(0)=f^{\prime }(0),p_2^{\prime \prime }(0)=f^{\prime \prime }(0).$$

Differentiating, $p_2^{\prime }(x)=b_1+2b_{2}x$ and $p_2^{\prime \prime }(x)=2b_2$, so evaluating at $0$ gives $b_0=1$, $b_1=1$ and $2b_2=1$. Hence

$$p_2(x)=1+x+\frac{x^2}{2},$$

and $e^{0.1}\approx p_2(0.1)=1.105$, which is noticeably better. Continuing with a cubic $p_3$ and matching one more derivative gives a better approximation again.

Basically, each extra derivative you force to agree buys you one more order of contact with the graph, and the approximation hugs the curve over a wider interval. Running this argument for a general degree $n$ gives the pattern: if $p_n(x)={\sum }_{k=0}^n{c}_k{x}^k$ then $p_n^{(k)}(0)=k!c_k$, so matching $p_n^{(k)}(0)=f^{(k)}(0)$ forces $c_k=\frac{f^{(k)}(0)}{k!}$. There is nothing special about the point $0$, so we make the general definition.

The factor of $\frac{1}{k!}$ is not decoration; it is exactly what cancels the $k!$ produced by differentiating $x^k$ down to a constant $k$ times. Forgetting it is the single most common slip in this section.

Example. Find the $n$​th Maclaurin polynomial of $f(x)=e^x$.
Every derivative of $e^x$ is $e^x$, so $f^{(k)}(0)=e^0=1$ for every $k$. Therefore

$$p_n(x)=\sum _{k=0}^n\frac{x^k}{k!}=1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+\cdots +\frac{x^n}{n!},$$

which is exactly the pattern we found by hand above.

Example. Find the Maclaurin polynomials of $f(x)=\sin x$.
The derivatives cycle with period four, which is what makes this one clean:

and then it repeats. So every even-index coefficient vanishes and the odd ones alternate in sign:

$$p_n(x)=x-\frac{x^3}{3!}+\frac{x^5}{5!}-\frac{x^7}{7!}+\cdots (n\text{ odd}).$$

Therefore the Maclaurin polynomials of $\sin$ contain only odd powers. Notice this had to happen; $\sin$ is an odd function, and the Taylor polynomial about $0$ of an odd function contains only odd powers, while that of an even function contains only even powers. Checking this parity is a free sanity check on any Maclaurin computation.

Example. Find the $n$​th Maclaurin polynomial of $g(x)=\ln (1+x)$.
Differentiating repeatedly,

$$\begin{array}{rl}{g}^{\prime }(x)& =(1+x{)}^{-1},\\ g^{\prime \prime }(x)& =-(1+x{)}^{-2},\\ g^{\prime \prime \prime }(x)& =2(1+x{)}^{-3},\\ g^{(4)}(x)& =-6(1+x{)}^{-4},\end{array}$$

and in general $g^{(k)}(x)=(-1{)}^{k-1}(k-1)!(1+x{)}^{-k}$ for $k\ge 1$, so $g^{(k)}(0)=(-1{)}^{k-1}(k-1)!$. Dividing by $k!$ leaves just $\frac{(-1{)}^{k-1}}{k}$, and since $g(0)=\ln 1=0$,

$$q_n(x)=x-\frac{x^2}{2}+\frac{x^3}{3}-\frac{x^4}{4}+\cdots +(-1{)}^{n-1}\frac{x^n}{n}.$$

Therefore the coefficients decay only like $\frac{1}{k}$, not like $\frac{1}{k!}$. Keep this contrast with $e^x$ in mind — it is the whole reason the next section exists. The factorial in the denominator of the exponential's coefficients crushes the terms very fast; a mere $\frac{1}{k}$ does not, and we will see that $\ln (1+x)$ pays for this with a much smaller range of usefulness.

Example. Find the Taylor polynomial of degree $3$ for $f(x)=\sqrt{x}$ about $a=4$, and use it to estimate $\sqrt{4.2}$.
Here we expand about $4$ rather than $0$, because $4$ is a perfect square close to $4.2$. Differentiating,

$$\begin{array}{rlrl}f(x)& =x^{1/2},& f(4)& =2,\\ f^{\prime }(x)& =\frac{1}{2}{x}^{-1/2},& f^{\prime }(4)& =\frac{1}{4},\\ f^{\prime \prime }(x)& =-\frac{1}{4}{x}^{-3/2},& f^{\prime \prime }(4)& =-\frac{1}{32},\\ f^{\prime \prime \prime }(x)& =\frac{3}{8}{x}^{-5/2},& f^{\prime \prime \prime }(4)& =\frac{3}{256}.\end{array}$$

So

$$p_3(x)=2+\frac{1}{4}(x-4)-\frac{1}{64}(x-4{)}^2+\frac{1}{512}(x-4{)}^3,$$

using $\frac{f^{\prime \prime }(4)}{2!}=-\frac{1}{64}$ and $\frac{f^{\prime \prime \prime }(4)}{3!}=\frac{3}{256\times 6}=\frac{1}{512}$. Putting $x=4.2$,

$$\begin{array}{rl}{p}_3(4.2)& =2+\frac{0.2}{4}-\frac{0.04}{64}+\frac{0.008}{512}\\ & =2+0.05-0.000625+0.000015625\\ & =\mathrm{2.049390625.}\end{array}$$

The true value is $\sqrt{4.2}=2.04939015\dots$, so we are right to seven decimal places. Therefore $\sqrt{4.2}\approx 2.0493906$. Choosing the centre $a$ well is most of the battle: expanding about $0$ would have been useless here, since $\sqrt{x}$ is not even differentiable at $0$.

4.2 Taylor's theorem

All of the last section was optimism. We produced polynomials that agree with $f$ to high order at a single point and then cheerfully evaluated them somewhere else, with no guarantee whatsoever about how wrong the answer might be.

Suppose a calculator with a ten-digit display uses Taylor polynomials to compute $e^{0.1}$. What degree should it use so that every displayed digit is correct? And crucially — is there a way to answer that without already knowing the decimal expansion of $e^{0.1}$? If we had to know the answer to bound the error, the whole enterprise would be circular.

Two cautionary examples show how badly intuition can fail here. For $f(x)=\sin x$, the Maclaurin polynomials $p_n$ approximate $f(7)$ well once $n$ reaches about $19$ or $21$ — even though $7$ is nowhere near $0$, and even though the intermediate polynomials are wildly wrong there. But for $g(x)=\ln (1+x)$, the Maclaurin polynomials approximate $g$ beautifully on roughly $(-0.7,0.7)$, tolerably on $(0.7,1)$, and to the right of $1$ they are useless — and worse, the higher-degree polynomials are further from $g$ there than the lower-degree ones. So "take more terms" is not a universally safe strategy, and we need something rigorous.

What we want is an exact expression for the difference $f(x)-p_n(x)$. Remarkably, repeated integration by parts delivers one. Suppose $f$ has $n+1$ continuous derivatives on an open interval $I$ containing $0$, and fix $x\in I$. Start from the fundamental theorem of calculus,

$${\int }_0^x{f}^{\prime }(t)dt=f(x)-f(0).\text{\hspace{1em}(1)}$$

Now evaluate that same integral by parts, with the deliberately odd choice $v=-(x-t)$ (any antiderivative of $1$ will do, and this one is engineered to vanish at $t=x$):

$$\begin{array}{rlrl}u& =f^{\prime }(t),& v& =-(x-t),\\ \frac{du}{dt}& =f^{\prime \prime }(t),& \frac{dv}{dt}& =1,\end{array}$$

so

$$\begin{array}{rl}{\int }_0^x{f}^{\prime }(t)dt& =[-f^{\prime }(t)(x-t){]}_0^x+{\int }_0^x{f}^{\prime \prime }(t)(x-t)dt\\ & =f^{\prime }(0)x+{\int }_0^x{f}^{\prime \prime }(t)(x-t)dt.\end{array}\text{\hspace{1em}(2)}$$

Comparing $(1)$ and $(2)$,

$$f(x)=f(0)+f^{\prime }(0)x+{\int }_0^x{f}^{\prime \prime }(t)(x-t)dt,$$

where the first two terms are exactly $p_1(x)$ and the integral is exactly the error. Integrating by parts again with $u=f^{\prime \prime }(t)$ and $v=-\frac{1}{2}(x-t{)}^2$ gives

$$f(x)=f(0)+f^{\prime }(0)x+\frac{f^{\prime \prime }(0)}{2!}{x}^2+\frac{1}{2!}{\int }_0^x{f}^{\prime \prime \prime }(t)(x-t{)}^{2}dt,$$

and now the pattern is clear. After $n$ steps we have $p_n$ plus a single leftover integral.

So the error in $f(x)\approx p_n(x)$ is exactly $R_{n+1}(x)$ — no approximation, no hand-waving. The catch is that this integral is usually impossible to evaluate, since it involves the very function we could not handle in the first place. Fortunately there is a much more usable form.

Basically, the remainder looks exactly like "the next term of the Taylor polynomial", except that the derivative is evaluated at some mystery point $c$ instead of at $a$. In almost every problem it is impossible to find $c$ exactly — and you never need to. You only need to know where $c$ lives, namely between $a$ and $x$, which is enough to bound $|f^{(n+1)}(c)|$ and hence bound the error. That is the entire technique of this section.

Notice that Taylor's theorem with Lagrange remainder generalises the mean value theorem: taking $n=0$ gives $f(x)-f(a)=f^{\prime }(c)(x-a)$, i.e.

$$\frac{f(x)-f(a)}{x-a}=f^{\prime }(c)$$

for some $c$ between $a$ and $x$, which is the MVT exactly.

Example. Let $f(x)=\cos x$. Using the second Taylor polynomial about $0$, estimate $f\left(\frac{1}{5}\right)$ and find an upper bound for the error.
The second Taylor polynomial is

$$\begin{array}{rl}{p}_2(x)& =f(0)+f^{\prime }(0)x+\frac{f^{\prime \prime }(0)}{2!}{x}^2\\ & =\cos 0-(\sin 0)x-\frac{\cos 0}{2!}{x}^2\\ & =1-\frac{x^2}{2},\end{array}$$

so $\cos \frac{1}{5}\approx p_2\left(\frac{1}{5}\right)=1-\frac{1}{50}=\frac{49}{50}$. For the error, apply the Lagrange formula with $n=2$, noting $f^{\prime \prime \prime }(x)=\sin x$:

$$\begin{array}{rl}|\text{error}|& =\left|f(1/5)-p_2(1/5)\right|\\ & =|R_3(1/5)|\\ & =\left|\frac{f^{\prime \prime \prime }(c)}{3!}{\left(\frac{1}{5}\right)}^3\right|(\text{for some }c\in [0,1/5])\\ & =\frac{|\sin c|}{6}\times \frac{1}{125}\\ & \le \frac{1}{6}\times \frac{1}{125}(\text{since }|\sin c|\le 1)\\ & =\frac{1}{750}\approx \mathrm{0.001333.}\end{array}$$

Therefore $\cos \frac{1}{5}\approx \frac{49}{50}$ with error at most $\frac{1}{750}$.

That bound is quite crude, because $|\sin c|\le 1$ throws away everything we know about where $c$ is. Using instead the sharper inequality $\sin t<t$ for $t>0$, we get $\sin c<c<\frac{1}{5}$, and hence

$$|\text{error}|<\frac{1}{6}\times \frac{1}{5}\times \frac{1}{125}=\frac{1}{3750}\approx \mathrm{0.0002667.}$$

(The true error is about $0.0000666$, so even this is conservative.) Any valid bound on $|f^{(n+1)}(c)|$ over the relevant interval gives a valid error bound; the tighter your bound on the derivative, the tighter your answer, and marks are usually given for a correct bound rather than the tightest one.

Example. A calculator with a ten-digit display uses a Maclaurin polynomial to estimate $e^{0.1}$. What degree polynomial guarantees that every displayed digit is correct?
Let $f(x)=e^x$. By Taylor's theorem, $e^x=p_n(x)+R_{n+1}(x)$ with

$$p_n(x)=1+x+\frac{x^2}{2!}+\cdots +\frac{x^n}{n!},R_{n+1}(x)=\frac{e^c}{(n+1)!}{x}^{n+1}$$

for some $c$ between $0$ and $x$. To be sure of ten displayed digits it suffices that

$$|R_{n+1}(0.1)|<10^{-10}.$$

Now bound the remainder without ever computing $e^{0.1}$:

$$\begin{array}{rl}|R_{n+1}(0.1)|& =\frac{e^c}{(n+1)!}(0.1{)}^{n+1}(\text{for some }c\in [0,0.1])\\ & \le \frac{e^{0.1}}{(n+1)!}(0.1{)}^{n+1}(\text{since }\exp \text{ is increasing})\\ & <\frac{2}{(n+1)!}(0.1{)}^{n+1}(\text{since }{e}^{0.1}<3^{0.1}<3^{1/2}<2)\\ & =\frac{2}{(n+1)!}10^{-(n+1)}.\end{array}$$

Since $\frac{2}{(n+1)!}<1$ whenever $n\ge 1$, a very crude conclusion is $|R_{n+1}(0.1)|<10^{-(n+1)}$, which already gives $n=9$. But we can do better by not throwing the factorial away. Testing values:

Therefore $n=6$ is the first degree that works, and the calculator should display the first ten digits of $p_6(0.1)$. Notice that at no point did we use the decimal expansion of $e^{0.1}$ — which was exactly the requirement, and is what makes error bounds genuinely useful rather than circular.

4.2.1 Classifying stationary points

Taylor's theorem also repairs a gap you may have noticed in the second derivative test. The function $f(x)=(x-3{)}^4$ has a stationary point at $3$, but $f^{\prime \prime }(3)=0$, so the second derivative test is silent — even though it is obvious from the formula that $3$ is a minimum. The fix is to keep differentiating until something is non-zero.

Sketch proof. Assume additionally that $f^{(k)}$ exists near $a$ and is continuous at $a$. Taylor's theorem with Lagrange remainder gives

$$\begin{array}{rl}f(x)& =f(a)+f^{\prime }(a)(x-a)+\frac{f^{\prime \prime }(a)}{2!}(x-a{)}^2+\cdots +\frac{f^{(k-1)}(a)}{(k-1)!}(x-a{)}^{k-1}+\frac{f^{(k)}(c)}{k!}(x-a{)}^k\\ & =f(a)+0+0+\cdots +0+\frac{f^{(k)}(c)}{k!}(x-a{)}^k\\ & =f(a)+\frac{f^{(k)}(c)}{k!}(x-a{)}^k\end{array}$$

for some $c$ between $x$ and $a$, since every middle term vanishes by hypothesis. For case (i): if $k$ is even then $(x-a{)}^k>0$ for $x\ne a$; and since $f^{(k)}(a)>0$ with $f^{(k)}$ continuous at $a$, we have $f^{(k)}(c)>0$ for all $x$ (and hence $c$) sufficiently close to $a$. Combining, $f(x)\ge f(a)$ near $a$, so $a$ is a local minimum. Cases (ii) and (iii) go the same way. $■$

Basically, near $a$ the function looks exactly like $f(a)+(\text{constant})(x-a{)}^k$, so the shape of the stationary point is decided entirely by the parity of $k$ and the sign of that constant — an even power sits on one side of its vertex, an odd power passes straight through.

Example. You are given that $3$ is a stationary point of

$$f(x)=x^7-17x^6+101x^5-229x^4+3x^3+621x^2-297x-567.$$

Classify it.
Differentiating and evaluating at $3$ gives

$$f^{\prime }(3)=f^{\prime \prime }(3)=f^{\prime \prime \prime }(3)=0,f^{(4)}(3)=-1536\ne 0,$$

so the corollary applies with $k=4$. Since $k$ is even and $f^{(4)}(3)<0$, we are in case (ii). Therefore $3$ is a local maximum point of $f$. Note how useless the ordinary second derivative test was here; it would have reported $f^{\prime \prime }(3)=0$ and stopped.

4.2.2 Some questions arising from Taylor's theorem

Return to the calculator problem. We showed

$$e^{0.1}=1+(0.1)+\frac{(0.1{)}^2}{2!}+\frac{(0.1{)}^3}{3!}+\cdots +\frac{(0.1{)}^n}{n!}+R_{n+1}(0.1),\text{\hspace{1em}(3)}$$

with

$$0<|R_{n+1}(0.1)|<\frac{2}{10^{n+1}(n+1)!}.$$

That bound plainly shrinks to nothing as $n$ grows, so it seems reasonable to expect $R_{n+1}(0.1)\to 0$, and letting $n\to \infty$ in $(3)$ suggests the striking identity

$$e^{0.1}=1+(0.1)+\frac{(0.1{)}^2}{2!}+\frac{(0.1{)}^3}{3!}+\frac{(0.1{)}^4}{4!}+\cdots ,$$

an infinite sum equalling an exact value. But every word of that sentence needs justification, which sets the agenda for the rest of the chapter:

• What does $R_{n+1}(0.1)\to 0$ as $n\to \infty$ actually mean? More generally, given a list of numbers $a_1,a_2,a_3,\dots$, how do we pin down the limiting behaviour of $a_n$? This is Section 4.3.
• What does an infinite sum mean, and how do we tell whether one converges to a real number or diverges? This is Sections 4.4 and 4.5.
• For which $x$ does ${\lim }_{n\to \infty }{R}_{n+1}(x)=0$, and when the infinite series converges, does it converge to $f(x)$ itself? The $\ln (1+x)$ picture warns that the answer is not always yes. This is Sections 4.6 to 4.8.

4.3 Sequences

A sequence is a real-valued function defined on (a subset of) the natural numbers. So $f:\mathbb{N}\to \mathbb{R}$ with $f(n)=n^2$ is a sequence; but for sequences we traditionally write $a_n$ rather than $a(n)$, and denote the whole object by $\{a_n\}$ or $\{a_n{\}}_{n=0}^{\infty }$.

Example. The Fibonacci sequence is defined recursively by $a_0=0$, $a_1=1$ and $a_{n+2}=a_{n+1}+a_n$, giving $\{0,1,1,2,3,5,8,13,\dots \}$. Not every sequence comes with a formula for $a_n$.

4.3.1 Describing the limiting behaviour of sequences

Our objective is to describe what $a_n$ does as $n\to \infty$. There are two broad outcomes: either $a_n$ approaches a finite number $L$, in which case $\{a_n\}$ is convergent and we write ${\lim }_{n\to \infty }{a}_n=L$; or it does not, in which case $\{a_n\}$ is divergent. Divergent sequences subdivide:

• if $a_n\to \infty$, the sequence diverges to infinity;
• if $a_n\to -\infty$, it diverges to negative infinity;
• if $\{a_n\}$ has no limit but stays bounded, it is boundedly divergent;
• if none of the above, it is unboundedly divergent.

Example. Describe the behaviour of each sequence as $n\to \infty$.
(a) $a_n=n^3$; (b) $a_n=\sin (n\pi /2)$; (c) $a_n=\frac{3n^2}{n^2+4n+3}$; (d) $a_n=(-1{)}^n{2}^n$.

(a) $a_n\to \infty$, so $\{a_n\}$ diverges to infinity.

(b) The terms run $\{0,1,0,-1,0,1,0,-1,\dots \}$, which is bounded but has no limit; so $\{a_n\}$ is boundedly divergent.

(c) Dividing top and bottom by $n^2$,

$$\frac{3n^2}{n^2+4n+3}=\frac{3}{1+4/n+3/n^2}\to \frac{3}{1+0+0}=3,$$

so $\{a_n\}$ converges to $3$.

(d) The terms run $\{1,-2,4,-8,16,-32,\dots \}$, which is unbounded, and the even terms head to $+\infty$ while the odd terms head to $-\infty$. Therefore $\{a_n\}$ is unboundedly divergent.

Basically, draw any horizontal band of half-width $\epsilon$ around the height $L$, however thin; the definition says the sequence eventually enters that band and never leaves. The crucial logical order is that $\epsilon$ comes first and $M$ is allowed to depend on it — a thinner band is allowed to need a longer wait.

4.3.2 Techniques for calculating limits of sequences

Most of the machinery you learned for limits of functions transfers directly.

Every one of these requires the individual limits to exist first. Writing $\lim (a_n{b}_n)=\lim a_n\times \lim b_n$ when one factor diverges is meaningless, and is where marks go missing.

Example. Let $a_n=\sqrt{n^2+4n}-n$. Find the limiting behaviour of $a_n$.
This is an $\infty -\infty$ form, so the limit laws do not apply directly; we rationalise, exactly as we would for the corresponding function limit:

$$\begin{array}{rl}{a}_n& =\sqrt{n^2+4n}-n\\ & =\frac{\left(\sqrt{n^2+4n}-n\right)\left(\sqrt{n^2+4n}+n\right)}{\sqrt{n^2+4n}+n}(\text{multiplying by the conjugate})\\ & =\frac{n^2+4n-n^2}{\sqrt{n^2+4n}+n}(\text{difference of two squares})\\ & =\frac{4n}{\sqrt{n^2+4n}+n}\\ & =\frac{4}{\sqrt{1+4/n}+1}(\text{dividing top and bottom by }n).\end{array}$$

Since $4/n\to 0$, the limit laws now apply and $a_n\to \frac{4}{1+1}=2$. Therefore $\{a_n\}$ converges to $2$. (Numerically, $a_{10000}=1.9998$, which agrees.)

This last one is what lets us use l'Hôpital's rule on sequences, which is otherwise illegal — you cannot differentiate a function of a discrete variable. The implication only runs one way. If the function limit fails to exist, the sequence limit may still exist perfectly well; for instance $f(x)=\sin (\pi x)$ has no limit as $x\to \infty$, but $a_n=\sin (\pi n)=0$ for every integer $n$, so $a_n\to 0$.

Example. Let $a_n={\left(1+\frac{1}{n}\right)}^n$. Find ${\lim }_{n\to \infty }{a}_n$.
Set $f(x)={\left(1+\frac{1}{x}\right)}^x$, which is the corresponding function. This is a $1^{\infty }$ indeterminate form, so take logarithms:

$$\ln f(x)=x\ln \left(1+\frac{1}{x}\right)=\frac{\ln (1+1/x)}{1/x},$$

which is a $\frac{0}{0}$ form as $x\to \infty$. By l'Hôpital's rule,

$$\begin{array}{rl}\underset{x\to \infty }{\lim }\ln f(x)& =\underset{x\to \infty }{\lim }\frac{\frac{1}{1+1/x}\cdot \left(-\frac{1}{x^2}\right)}{-\frac{1}{x^2}}\\ & =\underset{x\to \infty }{\lim }\frac{1}{1+1/x}\\ & =1.\end{array}$$

Since $\exp$ is continuous, $f(x)\to e^1=e$, and hence by the function-to-sequence proposition, $a_n\to e$. Therefore ${\lim }_{n\to \infty }{\left(1+\frac{1}{n}\right)}^n=e$. (Checking, $a_{100000}=2.718268\dots$, against $e=2.718281\dots$)

Example. Find ${\lim }_{n\to \infty }\frac{\sin (n^2+n)}{n}$.
We cannot say anything sensible about $\sin (n^2+n)$ itself, but we do not need to; we only need that it is trapped. Since $-1\le \sin \theta \le 1$ for all $\theta$,

$$-\frac{1}{n}\le \frac{\sin (n^2+n)}{n}\le \frac{1}{n}$$

for all $n\ge 1$, and both outer sequences tend to $0$. Therefore by the pinching theorem the limit is $0$. Whenever a bounded but wildly behaved factor is divided by something growing, reach for pinching; trying to compute the oscillating part is a trap.

It is worth having a feel for the relative speeds at which standard sequences grow, since a great many limits are settled by nothing more than knowing which of two expressions wins:

$$\boxed{\ln n\ll n^p\ll c^n\ll n!\ll n^n(p>0,c>1),}$$

where $\ll$ means the left-hand side is eventually negligible compared with the right. In particular $\frac{c^n}{n!}\to 0$ for every fixed $c$, which is precisely the fact that makes the exponential's remainder term vanish.

Basically, an increasing sequence with a ceiling has nowhere to go but up towards some limit; it cannot oscillate, because it is monotonic, and it cannot escape, because it is bounded. This is the one convergence result that does not require you to know the limit in advance, which makes it invaluable for recursively defined sequences where no formula for $a_n$ exists.

4.3.3 [X] Suprema and infima

The monotone convergence theorem rests on a deep property of $\mathbb{R}$ that we should state properly.

This is an axiom of the real numbers, not a theorem — and it is exactly what $\mathbb{Q}$ lacks. The rationals less than $\sqrt{2}$ are bounded above but have no least rational upper bound, which is precisely the hole that $\mathbb{R}$ was built to fill.

Example. Find the greatest lower bound and least upper bound of $\{a_n{\}}_{n=1}^{\infty }$ where $a_n=\frac{1}{n}$.
The terms are $1,\frac{1}{2},\frac{1}{3},\dots$, all positive and all at most $1$. The least upper bound is $1$, and it is attained at $n=1$. The greatest lower bound is $0$: every $a_n>0$, so $0$ is a lower bound; and no positive number $ϵ$ can be a lower bound, since $\frac{1}{n}<ϵ$ once $n>\frac{1}{ϵ}$. Therefore $\sup =1$ and $\inf =0$. Notice the infimum is not attained — a supremum or infimum need not be a member of the sequence, which is exactly why "maximum" and "supremum" are different words.

Given this axiom, the monotone convergence theorem is nearly immediate for an increasing bounded sequence: let $L$ be the least upper bound of $\{a_n\}$. For any $\epsilon >0$, $L-\epsilon$ is not an upper bound, so some $a_M>L-\epsilon$; and since the sequence increases and is capped by $L$, every $n>M$ has $L-\epsilon <a_M\le a_n\le L$, giving $|a_n-L|<\epsilon$.

4.4 Infinite series

We can now say what an infinite sum means. The idea is the only sensible one available: add up finitely many terms, and see whether those running totals settle down.

A series is not a sum; it is the limit of a sequence of sums. Every question about series is secretly a question about the sequence $\{s_n\}$, and keeping that in mind resolves most of the confusion in this topic.

Example. Determine whether the geometric series ${\sum }_{k=0}^{\infty }{r}^k$ converges, and find its sum when it does.
The partial sums have a closed form. For $r\ne 1$,

$$\begin{array}{rl}{s}_n& =1+r+r^2+\cdots +r^n,\\ r{s}_n& =r+r^2+\cdots +r^n+r^{n+1},\\ s_n-r{s}_n& =1-r^{n+1},\\ s_n& =\frac{1-r^{n+1}}{1-r}.\end{array}$$

Now $r^{n+1}\to 0$ if $|r|<1$, and $\{r^{n+1}\}$ diverges if $|r|>1$ or $r=-1$; while for $r=1$ we have $s_n=n+1\to \infty$. Therefore

$$\boxed{\sum _{k=0}^{\infty }{r}^k=\frac{1}{1-r}\text{ if }|r|<1,\text{ and the series diverges if }|r|\ge 1.}$$

This is the single most useful series in the course, both in its own right and as the comparison against which other series are judged.

Example. Evaluate ${\sum }_{k=1}^{\infty }\frac{1}{k(k+1)}$.
Partial fractions (see Section 2.4) give $\frac{1}{k(k+1)}=\frac{1}{k}-\frac{1}{k+1}$, so the partial sums telescope:

$$\begin{array}{rl}{s}_n& =\sum _{k=1}^n\left(\frac{1}{k}-\frac{1}{k+1}\right)\\ & =\left(1-\frac{1}{2}\right)+\left(\frac{1}{2}-\frac{1}{3}\right)+\cdots +\left(\frac{1}{n}-\frac{1}{n+1}\right)\\ & =1-\frac{1}{n+1},\end{array}$$

since every interior term cancels with its neighbour. Hence $s_n\to 1$. Therefore ${\sum }_{k=1}^{\infty }\frac{1}{k(k+1)}=1$. Geometric and telescoping series are essentially the only ones whose exact sum you can find by hand; for everything else the realistic goal is deciding convergence, not evaluating.

Proof. Apply the corresponding limit laws to the sequences of partial sums, noting that the $n$​th partial sum of $\sum (a_k+b_k)$ is exactly $s_n+t_n$ where $s_n,t_n$ are those of $\sum a_k$ and $\sum b_k$. $■$

It is worth recording one structural remark. While every term contributes to the value of a convergent series, only the tail matters for the question of convergence: the first hundred, thousand or billion terms are irrelevant to whether the series converges, since changing finitely many terms changes every partial sum past a point by the same fixed constant. This is why the tests below are all allowed to say "for all sufficiently large $k$".

4.5 Tests for series convergence

4.5.1 Some preliminary results on series summation

Since exact sums are almost never available, we need tests that decide convergence from the shape of $a_k$ alone. The first one is free.

4.5.2 The $k$​th term divergence test

Proof. Suppose $\sum a_k$ converges to $L$, and let $s_n$ be the $n$​th partial sum, so $s_n\to L$ and also $s_{n-1}\to L$ (it is the same sequence shifted). Then

$$a_n=s_n-s_{n-1}\to L-L=0.■$$

This is just the contrapositive of the previous theorem, and it is the cheapest test there is — always try it first. But the converse is emphatically false: $a_k\to 0$ does NOT imply that $\sum a_k$ converges. The harmonic series below is the standard counterexample, and mis-using this test in the wrong direction is the most common error in the entire chapter.

Example. Determine whether ${\sum }_{k=0}^{\infty }\frac{k}{\sqrt{k^2+2k}}$ converges.
Looking at the terms,

$$\frac{k}{\sqrt{k^2+2k}}=\frac{1}{\sqrt{1+2/k}}\to 1\ne 0$$

as $k\to \infty$. Therefore the series diverges by the $k$​th term test. Intuitively we are adding infinitely many numbers each close to $1$, so the partial sums grow without bound.

4.5.3 The integral test

Basically, the terms $a_k$ are the areas of rectangles of width $1$ and height $f(k)$, and those rectangles can be lined up to sit just under or just over the graph of $f$. Sliding them one way gives ${\sum }_{k=2}^n{a}_k\le {\int }_1^{n}f$, and the other way gives ${\int }_1^{n}f\le {\sum }_{k=1}^{n-1}{a}_k$; so the sum and the integral are trapped within one term of each other and must live or die together. The hypotheses matter — $f$ must be positive and decreasing, or the rectangle picture collapses.

Example. For which $p$ does the $p$-series ${\sum }_{k=1}^{\infty }\frac{1}{k^p}$ converge?
Take $f(x)=x^{-p}$, which is positive and decreasing on $[1,\infty )$ when $p>0$. For $p\ne 1$,

$${\int }_1^{\infty }{x}^{-p}dx=\underset{b\to \infty }{\lim }{\left[\frac{x^{1-p}}{1-p}\right]}_1^b=\underset{b\to \infty }{\lim }\frac{b^{1-p}-1}{1-p},$$

which converges (to $\frac{1}{p-1}$) exactly when $1-p<0$, i.e. $p>1$. For $p=1$,

$${\int }_1^{\infty }\frac{dx}{x}=\underset{b\to \infty }{\lim }\ln b=\infty .$$

For $p\le 0$ the terms do not even tend to $0$. Therefore

$$\boxed{\sum _{k=1}^{\infty }\frac{1}{k^p}\text{ converges if and only if }p>1.}$$

The case $p=1$ deserves its own name: ${\sum }_{k=1}^{\infty }\frac{1}{k}$ is the harmonic series, and it diverges even though its terms tend to $0$. It does so extraordinarily slowly — the partial sums grow like $\ln n$, so summing the first hundred thousand terms gives only about $12.09$ — which is exactly why the $k$​th term test cannot possibly be a test for convergence.

4.5.4 The comparison test

Proof. The partial sums of a positive series are increasing. In case (i), they are also bounded above by $\sum b_k$, so they converge by monotone convergence. Case (ii) is the contrapositive of (i) with the roles swapped. $■$

Basically, a smaller positive series cannot escape past a convergent one, and a bigger one cannot stay below a divergent one. Get the direction right: bounding your series above by a convergent one proves convergence, and below by a divergent one proves divergence. The other two combinations tell you nothing at all.

Example. Determine whether ${\sum }_{k=1}^{\infty }\frac{1}{k^2+3k}$ converges.
For $k\ge 1$ we have $k^2+3k>k^2>0$, so

$$0<\frac{1}{k^2+3k}<\frac{1}{k^2},$$

and $\sum \frac{1}{k^2}$ converges as a $p$-series with $p=2>1$. Therefore the series converges by comparison.

Example. Determine whether ${\sum }_{k=2}^{\infty }\frac{1}{\ln k}$ converges.
Since $\ln k<k$ for all $k\ge 2$, we have $\frac{1}{\ln k}>\frac{1}{k}>0$, and $\sum \frac{1}{k}$ diverges as the harmonic series. Therefore the series diverges by comparison. Notice the direction: we bounded below by something divergent.

4.5.5 [X] The limit form of the comparison test

Finding an exact inequality is often fiddly, and there is a version that only needs the terms to be of the same order.

Example. Determine whether ${\sum }_{k=1}^{\infty }\frac{2k^2+5}{4k^4-3k+1}$ converges.
For large $k$ the terms behave like $\frac{2k^2}{4k^4}=\frac{1}{2k^2}$, so compare with $b_k=\frac{1}{k^2}$: