Week 2 notes

Week 2 The Harmonic Series. Rearrangements. Series with positive and negative terms

Version 2025/02/04 Week 2 in PDF All notes in PDF To other weeks

We begin the chapter with a series which is an example to many results in this course. In particular, it shows that a positive series with $lim_{n \to \infty} \frac{a_{n + 1}}{a_{n}} = 1$ may be divergent:

Proposition 2.1: the Harmonic Series is divergent.

The following series, called the harmonic series, is divergent:

$1 + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + \dots = \sum_{n = 1}^{\infty} \frac{1}{n} = + \infty .$

Proof. We have

$s_{2 n} - s_{n} = \frac{1}{n + 1} + \frac{1}{n + 2} + \dots + \frac{1}{2 n} \geq n \times \frac{1}{2 n} = \frac{1}{2} .$

Hence $s_{2} \geq s_{1} + \frac{1}{2},$ $s_{4} \geq s_{2} + \frac{1}{2} \geq s_{1} + \frac{2}{2},$ and, continuing, we get $s_{2^{n}} \geq s_{1} + \frac{n}{2} .$ This shows that the sequence $(s_{n})_{n \geq 1}$ is unbounded. □

Comment: we can see that the partial sums $s_{n}$ of the harmonic series diverge to $+ \infty$ but “slowly”. How large must $n$ be so that $s_{n} \geq 10$ ? Do you need to add up more than a thousand terms of the harmonic series to get a sum of at least $10$ ? Open the spreadsheet which tabulates partial sums of the harmonic series and scroll down to find out!

Rearrangements of a series with non-negative terms

When we add up finitely many numbers, the answer does not depend on the order of summands. Yet for infinite series this is more intricate: putting terms in a different order gives a very different sequence of partial sums. Let us formally define a rearrangement.

Definition: rearrangement.

A series $\sum_{n = 1}^{\infty} b_{n}$ is called a rearrangement of a series $\sum_{n = 1}^{\infty} a_{n}$ if there exists a bijective function $𝜎 : N \to N$ such that $b_{n} = a_{𝜎 (n)}$ for all $n .$

We now prove that rearranging a non-negative series does not change the sum.

Theorem 2.2: the rearrangement theorem for non-negative series.

Suppose that all terms in a series $a_{1} + a_{2} + \dots$ are non-negative. Then all rearrangements of this series have the same sum, and if the series $a_{1} + a_{2} + \dots$ is divergent, then all rearrangements are divergent.

Proof. Let $𝜎 : N \to N$ be a bijection so that $a_{𝜎 (1)} + a_{𝜎 (2)} + \dots$ is a rearrangement of the original series. We let

$s_{n} = a_{1} + a_{2} + \dots + a_{n}, t_{n} = a_{𝜎 (1)} + a_{𝜎 (2)} + \dots + a_{𝜎 (n)}$

be the $n$ th partial sum of the series, respectively, rearranged series. Denote by $M (n)$ the largest among the indices $𝜎 (1), \dots, 𝜎 (n) .$ Then $a_{𝜎 (1)}, a_{𝜎 (2)}, \dots, a_{𝜎 (n)}$ is a sublist of the list $a_{1}, \dots, a_{M (n)}$ of non-negative real numbers, and so

$t_{n} \leq s_{M (n)} .$

If $\sum_{n = 1}^{\infty} a_{n}$ is convergent with sum $S < + \infty,$ then $s_{M (n)} \leq S,$ and so $t_{n} \leq S,$ for all $n .$ By the boundedness test, Theorem 1.4, the rearranged series is convergent with

$\sum_{n = 1}^{\infty} a_{𝜎 (n)} \leq \sum_{n = 1}^{\infty} a_{n},$

which proves that rearranging a non-negative series cannot increase its sum.

But the series $\sum_{n = 1}^{\infty} a_{n}$ can also be viewed as a rearrangement of $\sum_{n = 1}^{\infty} a_{𝜎 (n)},$ via the bijective function $𝜎^{- 1} .$ Since rearranging cannot increase the sum, we must conclude that

$\sum_{n = 1}^{\infty} a_{n} \leq \sum_{n = 1}^{\infty} a_{𝜎 (n)} .$

From the two inequalities we conclude that the series and its rearrangement have the same sum. Finally, our observation that $\sum_{n = 1}^{\infty} a_{n}$ is a rearrangement of $\sum_{n = 1}^{\infty} a_{𝜎 (n)}$ proves, by contrapositive, the implication “ $\sum_{n = 1}^{\infty} a_{n}$ is divergent $\Rightarrow$ $\sum_{n = 1}^{\infty} a_{𝜎 (n)}$ is divergent”. □

Summation of double series

For theoretical and practical reasons, we would like to be able to calculate a sum of all numbers in a double series, which is defined as an array infinite in two directions,

$\begin{matrix} a_{00} & a_{01} & a_{02} & a_{03} & \dots \\ a_{10} & a_{11} & a_{12} & a_{13} & \dots \\ a_{20} & a_{21} & a_{22} & a_{23} & \dots \\ a_{30} & a_{31} & a_{32} & a_{33} & \dots \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋱ \end{matrix}$

where $a_{m, n} \in R$ for all $m, n \geq 0.$ Double series allow several methods of summation. First, we can enumerate all terms of a double series by non-negative integers, which gives a (single) series. (Two such enumerations will be shown later in Figure 2.2.) Secondly, let

• ${RowSum}_{m} =$ the sum of the infinite series $a_{m 0} + a_{m 1} + a_{m 2} + \dots$ if it exists;
• ${ColumnSum}_{n} =$ the sum of the infinite series $a_{0 n} + a_{1 n} + a_{2 n} + \dots$ if it exists;
• ${DiagSum}_{d} =$ the finite sum $a_{d 0} + a_{d - 1, 1} + \dots + a_{0 d},$ equivalently $\sum_{m + n = d} a_{m, n};$
• ${SquareSum}_{b \times b} = \sum {a_{m, n} : 0 \leq m \leq b, 0 \leq n \leq b} .$

Figure 2.1 illustrates how some of these sums are calculated.

(doubly infinite array with rows, columns and diagonals circled)

The next result considers summing a double series by enumeration, by squares, by diagonals, by rows and by columns. If the terms are all non-negative, the methods will return the same answer:

Proposition 2.3: summation of double series with non-negative terms.

Suppose that a double series $(a_{m, n})$ has non-negative terms, and a way to enumerate all these terms gives a single series with sum $S .$ Then:

1. all ways to enumerate the terms $a_{m, n}$ will result in series with sum $S;$
2. $lim_{b \to \infty} {SquareSum}_{b \times b} = S$ and $\sum_{d = 0}^{\infty} {DiagSum}_{d} = S;$
3. $\sum_{m = 0}^{\infty} {RowSum}_{m} = S$ and $\sum_{n = 0}^{\infty} {ColumnSum}_{n} = S .$

Proof. 1. All the different ways of arranging the terms $a_{m, n}$ in a single series are rearrangements of one another, hence must have the same sum, $S,$ by Theorem 2.2.

2. To show that $\sum_{d = 0}^{\infty} {DiagSum}_{d} = S$ and $lim_{b \to \infty} {SquareSum}_{b \times b} = S,$ we consider two special ways to enumerate the $a_{m, n},$ shown in Figure 2.2.

Figure 2.2: Two examples of enumerating terms of a double series

In Figure 2.2(A), partial sums of the resulting single series (which, by part 1., converge to $S$ ) contain a subsequence of sums of the form ${DiagSum}_{0} + {DiagSum}_{1} + \dots + {DiagSum}_{d} .$ All subsequences of a convergent sequence have the same limit, so this subsequence must also converge to $S .$

Figure 2.2(B) similarly shows that $lim_{b \to \infty} {SquareSum}_{b \times b}$ must be $S .$

3. The top row of the $b \times b$ square sum is $a_{00} + a_{01} + \dots + a_{0 b} .$ This is a partial sum which is less than or equal to the infinite sum ${RowSum}_{0} .$ In the same way, the remaining rows of ${SquareSum}_{b \times b}$ are bounded above by ${RowSum}_{1}, \dots, {RowSum}_{b} .$ Therefore,

${SquareSum}_{b \times b} \leq \sum_{m = 0}^{b} {RowSum}_{m} .$

Taking the limit as $b \to \infty,$ we have

$S = lim_{b \to \infty} {SquareSum}_{b \times b} \leq \sum_{m = 0}^{\infty} {RowSum}_{m} .$

It remains to show that the opposite inequality, $\sum_{m = 0}^{\infty} {RowSum}_{m} \leq S,$ also holds. Since the infinite sum $\sum_{m = 0}^{\infty} {RowSum}_{m}$ is the limit of partial sums, it is enough to show that for every fixed $M,$ the sum $\sum_{m = 0}^{M} {RowSum}_{m}$ is at most $S .$

By definition of the row sums, we have

${RowSum}_{0} = lim_{n \to \infty} a_{00} + a_{01} + \dots + a_{0 n},$
${RowSum}_{1} = lim_{n \to \infty} a_{10} + a_{11} + \dots + a_{1 n},$
$⋮$
${RowSum}_{M} = lim_{n \to \infty} a_{M, 0} + a_{M, 1} + \dots + a_{M, n} .$

Adding together these limits, we have

$\sum_{m = 0}^{M} {RowSum}_{m} = lim_{n \to \infty} (sum over the M \times n rectangle) .$

Yet any rectangle is contained in a $b \times b$ square for large enough $b,$ and by part 2., sums over all squares are $\leq S .$ Hence $\sum_{m = 0}^{M} {RowSum}_{m}$ is a limit of a sequence bounded above by $S,$ which means that $\sum_{m = 0}^{M} {RowSum}_{m} \leq S .$

We have proved that $S \leq \sum_{m = 0}^{\infty} {RowSum}_{m} \leq S,$ so $\sum_{m = 0}^{\infty} {RowSum}_{m} = S .$ Finally, the argument for the column sums, $\sum_{n = 0}^{\infty} {ColumnSum}_{n} = S,$ is the same as for row sums, and we omit it. The Proposition is proved. □

Alert: changing the order of summation.

In analysis, passing from sum by rows to sum by columns is known as changing the order of summation. We have proved that

$\sum_{m = 0}^{\infty} (\sum_{n = 0}^{\infty} a_{m, n}) = \sum_{n = 0}^{\infty} (\sum_{m = 0}^{\infty} a_{m, n}),$

assuming that all $a_{m, n}$ are non-negative. The equality does not hold in general, see example below. Finding conditions (such as non-negativity) which allow changing the order of infinite summation is an important problem in Analysis.

Example: if we drop the assumption that all $a_{m, n}$ are non-negative, the “sum of the double series” may depend on the method of summation. Consider the double series

$\begin{array}{rrrrrr} 1 & - 1 & 0 & 0 & 0 & \dots \\ 0 & 1 & - 1 & 0 & 0 & \dots \\ 0 & 0 & 1 & - 1 & 0 & \dots \\ 0 & 0 & 0 & 1 & - 1 & \dots \\ 0 & 0 & 0 & 0 & 1 & \dots \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋮ & ⋱ \end{array} a_{m, n} = {\begin{cases} 1, & if m = n, \\ - 1, & if m = n - 1, \\ 0, & otherwise . \end{cases}$

We have ${RowSum}_{m} = 0$ for all $m .$ Yet ${ColumnSum}_{0} = 1$ and ${ColumnSum}_{n} = 0$ for all $n \geq 1,$ and so

$\sum_{m = 0}^{\infty} (\sum_{n = 0}^{\infty} a_{m, n}) = 0, \sum_{n = 0}^{\infty} (\sum_{m = 0}^{\infty} a_{m, n}) = 1.$

Series with terms of different signs. The Nullity Test

We will now develop several convergence tests for infinite series without the assumption that all terms are non-negative. Our first test can only show divergence:

Theorem 2.4: the nullity test for divergence.

If $a_{n} ↛ 0$ as $n \to \infty,$ then the series $\sum_{n = 1}^{\infty} a_{n}$ is divergent.

Proof. Write $s_{n}$ for the $n$ th partial sum. Assume that the series is convergent, i.e., $lim_{n \to \infty} s_{n} = s$ for some real number $s .$ Then $lim_{n \to \infty} s_{n - 1} = s$ as well. By AoL of convergent sequences, $lim_{n \to \infty} (s_{n} - s_{n - 1}) = s - s = 0.$ But $s_{n} - s_{n - 1} = a_{n}$ and so $lim_{n \to \infty} a_{n} = 0.$ We proved:

the series $\sum_{n = 1}^{\infty} a_{n}$ is convergent $\Rightarrow$ $lim_{n \to \infty} a_{n} = 0,$

which is the contrapositive of, hence is equivalent to, the statement of the Theorem. □

Example: application of the nullity test.

Show that the series (a) $\sum_{n \geq 1} \frac{n}{n + 1}$ and (b) $\sum_{n \geq 0} (- 1)^{n}$ are divergent.

Solution: (a) $lim_{n \to \infty} \frac{n}{n + 1} = lim_{n \to \infty} \frac{1}{1 + 1 / n} = 1 \neq 0$ so by the Nullity Test, the series is divergent. (b) $lim_{n \to \infty} (- 1)^{n}$ does not exist and in particular is not $0,$ so $\sum_{n \geq 0} (- 1)^{n}$ is also a divergent series.

Remark: if $a_{n} \to 0$ as $n \to \infty,$ the Nullity Test is inconclusive: the series $\sum_{n = 1}^{\infty} a_{n}$ may still be divergent. The Harmonic Series is an example.

Algebra of infinite sums

The next result allows us to construct new convergent series out of existing examples.

Proposition 2.5: Algebra of Infinite Sums (AoIS).

If $\sum_{n = 1}^{\infty} a_{n}$ is convergent with sum $S$ and $\sum_{n = 1}^{\infty} b_{n}$ is convergent with sum $T,$ then for any real numbers $𝜆, 𝜇$ the series $\sum_{n = 1}^{\infty} (𝜆 a_{n} + 𝜇 b_{n})$ is convergent with sum $𝜆 S + 𝜇 T .$

Proof. The $n$ th partial sum of the series $\sum_{n = 1}^{\infty} (𝜆 a_{n} + 𝜇 b_{n})$ is $(𝜆 a_{1} + 𝜇 b_{1}) + \dots + (𝜆 a_{n} + 𝜇 b_{n}) .$ This is a finite sum, so we can rearrange to get $𝜆 s_{n} + 𝜇 t_{n},$ where $s_{n} = a_{1} + \dots + a_{n}$ and $t_{n} = b_{1} + \dots + b_{n}$ (partial sums). We are given that $s_{n} \to S$ and $t_{n} \to T$ as $n \to \infty,$ so by AoL of convergent sequences, $𝜆 s_{n} + 𝜇 t_{n} \to 𝜆 S + 𝜇 T$ as claimed. □

Alert: no $\times$ or $/$ for series.

Unlike the Algebra of Limits of convergent sequences, AoIS does not allow multiplication or division of series.

Absolute convergence

The following definition comes from “absolute value”, the old name for the modulus $| a | .$

Definition: absolutely convergent.

The series $\sum_{n = 1}^{\infty} a_{n}$ is absolutely convergent if $\sum_{n = 1}^{\infty} | a_{n} | < + \infty .$

The next very strong test can establish convergence of many series.

Theorem 2.6: Absolute Convergence Theorem; the Infinite Triangle Inequality.

Suppose the series $\sum_{n = 1}^{\infty} a_{n}$ is absolutely convergent. Then:

1. there are non-negative $p_{n}, q_{n}$ such that $\sum_{n = 1}^{\infty} p_{n} < + \infty,$ $\sum_{n = 1}^{\infty} q_{n} < + \infty$ and $a_{n} = p_{n} - q_{n}$ for all $n;$
2. $\sum_{n = 1}^{\infty} a_{n}$ is convergent;
3. $| \sum_{n = 1}^{\infty} a_{n} | \leq \sum_{n = 1}^{\infty} | a_{n} |$ (infinite triangle inequality).

Proof. For a real number $a,$ denote

$a^{+} = {\begin{cases} a, & if a \geq 0, \\ 0, & if a < 0, \end{cases}$ $a^{-} = {\begin{cases} 0, & if a \geq 0, \\ - a, & if a < 0. \end{cases}$

1. Assume that the series $\sum a_{n}$ is absolutely convergent, meaning that $\sum_{n = 1}^{\infty} | a_{n} |$ has finite sum $M < + \infty .$ Put $p_{n} = a_{n}^{+}$ and $q_{n} = a_{n}^{-}$ for all $n .$ Then we have

$0 \leq p_{n}, q_{n} \leq | a_{n} |, a_{n} = p_{n} - q_{n}, | a_{n} | = p_{n} + q_{n} .$

By Comparison Test, Theorem 1.5, the series $\sum_{n = 1}^{\infty} p_{n}$ is convergent, with some finite sum $P \leq M,$ and likewise $\sum_{n = 1}^{\infty} q_{n} = Q \leq M .$

2. By AoIS, the series $\sum_{n = 1}^{\infty} a_{n} = \sum_{n = 1}^{\infty} (p_{n} - q_{n})$ is convergent with sum $P - Q .$

3. Since $0 \leq P, Q \leq M,$ we have $| P - Q | \leq M,$ which reads $| \sum_{n = 1}^{\infty} a_{n} | \leq \sum_{n = 1}^{\infty} | a_{n} | .$ □

Remark. How to test the series $\sum_{n = 1}^{\infty} | a_{n} |$ for convergence? For example, we can use the Comparison Test, the Ratio Test or the Root Test.

The next result shows that rearrangements of absolutely convergent series are as well-behaved as rearrangements of non-negative series.

Claim 2.7: rearrangements of absolutely convergent series and double series.

(a) All rearrangements of absolutely convergent $\sum_{n} a_{n}$ converge with the same sum.

(b) If $\sum_{m, n} | a_{m, n} | < + \infty,$ summing the double series $(a_{m, n})$ by rows, by columns, by diagonals and by squares gives the same answer. In particular, changing the order of summation is allowed: $\sum_{m = 0}^{\infty} \sum_{n = 0}^{\infty} a_{m, n} = \sum_{n = 0}^{\infty} \sum_{m = 0}^{\infty} a_{m, n} .$

Explanation: (a) If $\sum a_{n}$ is absolutely convergent, Theorem 2.6 gives us non-negative $p_{n}$ and $q_{n}$ such that $a_{n} = p_{n} - q_{n}$ for all $n$ and $\sum p_{n} = P < + \infty,$ $\sum q_{n} = Q < + \infty .$ Then $a_{𝜎 (n)} = p_{𝜎 (n)} - q_{𝜎 (n)}$ and so by AoIS, $\sum a_{𝜎 (n)} = (\sum p_{𝜎 (n)}) - (\sum q_{𝜎 (n)}) .$ Yet by Rearrangement Theorem 2.2 for non-negative series, $\sum p_{𝜎 (n)} = P$ and $\sum q_{𝜎 (n)} = Q,$ and so $\sum a_{𝜎 (n)} = P - Q,$ regardless of the rearrangement.

(b) Similarly to (a), write $a_{m, n} = p_{m, n} - q_{m, n}$ with non-negative $p_{m, n}$ and $q_{m, n}$ and $\sum_{m, n} p_{m, n} = P < + \infty,$ $\sum_{m, n} q_{m, n} = Q < + \infty .$ Then by Proposition 2.3 and AoIS, all methods of summation of the double series $a_{m, n}$ will return the answer $P - Q .$

The Alternating Series Test

If a series with positive and negative terms is not absolutely convergent, it might still satisfy the assumptions of the next test which shows convergence.

Theorem 2.8: The Alternating Series Test.

Let $a_{1}, a_{2}, \dots \geq 0$ be a decreasing sequence which tends to zero. Then the series $a_{1} - a_{2} + a_{3} - a_{4} + \dots = \sum_{n = 1}^{\infty} (- 1)^{n + 1} a_{n}$ is convergent.

Proof. Similarly to the Absolute Convergence Theorem, the idea is to write the given series as a linear combination of two convergent series; yet in this case, we cannot use two series with non-negative terms. Consider two series:

$\begin{array}{lccccccc} Series 1: & (a_{1} - a_{2}) & + 0 & + (a_{3} - a_{4}) & + 0 & + (a_{5} - a_{6}) & + 0 & + \dots \\ Series 2: & a_{2} & - a_{2} & + a_{4} & - a_{4} & + a_{6} & - a_{6} & + \dots \end{array}$

To obtain a partial sum of Series 1, we start with $a_{1},$ subtract $a_{2},$ add $a_{3},$ subtract $a_{4},$ add $a_{5}$ etc. By assumption, $a_{2}, a_{3}, \dots$ decrease, so each time we subtract more than we add; hence all partial sums are bounded above by $a_{1} .$ Series 1 has non-negative terms, hence is convergent by Boundedness Test, Theorem 1.4.

Series 2 has partial sums $a_{2},$ $0,$ $a_{4},$ $0,$ $a_{6},$ $0$ and so on. The $n$ th partial sum is between $0$ and $a_{n},$ and $a_{n} \to 0$ as $n \to \infty .$ By Sandwich Rule the partial sums have limit $0,$ hence Series 2 is convergent.

The required series $a_{1} - a_{2} + a_{3} - a_{4} + \dots$ is obtained by adding Series 2 to Series 1, hence is convergent by Algebra of Infinite Sums, Proposition 2.5. □

Example: Alternating Harmonic Series.

Show that $1 - \frac{1}{2} + \frac{1}{3} - \frac{1}{4} + \dots = \sum_{n = 1}^{\infty} \frac{(- 1)^{n + 1}}{n}$ is a convergent series.

Solution. $(\frac{1}{n})_{n \geq 1}$ is a decreasing sequence which has limit $0.$ So by the Alternating Series Test, Theorem 2.8, the series is convergent.

Remark: we are not ready to calculate the sum of the Alternating Harmonic Series just yet. Methods from this course will allow us to prove that its sum is $\ln (2) .$

Conditional convergence. Rearrangements

We proved that absolute convergence implies convergence. Is the converse true? That is, if a series converges, does it have to be absolutely convergent? Here is a counterexample:

Example: a convergent series which is not absolutely convergent.

Show that the (convergent) alternating harmonic series $1 - \frac{1}{2} + \frac{1}{3} - \frac{1}{4} + \dots$ is not absolutely convergent.

Solution. The series is convergent by the Alternating Series Test. But the series

$1 + | - \frac{1}{2} | + | \frac{1}{3} | + | - \frac{1}{4} | + \dots$ $=$ $1 + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + \dots,$

made up of absolute values, is the Harmonic Series which, as we proved, is divergent.

Series with this property have a special name:

Definition: conditionally convergent.

A series is conditionally convergent if it is convergent but not absolutely convergent.

Rearrangements of conditionally convergent series do not have the same sum:

Theorem 2.9: Riemann’s rearrangement theorem.

Suppose $\sum_{n = 1}^{\infty} a_{n}$ is a conditionally convergent series. Then for any real number $𝛼$ there is a rearrangement of $\sum_{n = 1}^{\infty} a_{n}$ which converges with sum $𝛼 .$ There is also a rearrangement which diverges. □

We do not go through the proof of this striking result in class, but it may later be added as an appendix to this week’s notes (not examinable).

Version 2025/02/04 Week 2 in PDF All notes in PDF To other weeks