Week 5 The dual code. Syndrome decoding

Version 2023-11-01. To PDF of this week only Open the whole set of notes in PDF

Synopsis. Every linear code $C$ has a dual code, $C^{⊥},$ and check matrices. While a generator matrix $G$ is used to encode messages into codevectors, a check matrix $H$ serves to detect errors — and to correct them using syndrome decoding.

Motivation. The inner product of vectors

Let $C$ be a linear code. Given a received vector $\underset{―}{y},$ how to test whether $\underset{―}{y} \in C$ ? Storing all codevectors of $C$ is not an option for codes of large length and dimension, whose use is dictated by modern applications to low-noise channels. Storing just a generator matrix $G$ of $C$ is better in terms of storage space, but testing whether $\underset{―}{y}$ is in the row space of $G$ can be computationally demanding.

Some codes, however, are defined by a single checksum — recall the even weight code and the ISBN-10 code. A checksum of a given vector is easy to compute.

Extending the checksum approach, we introduce a check matrix which generates the dual code. It turns out that this construction helps to correct errors as well (not just detect). The first notion we need is:

Definition: inner product

For $\underset{―}{u}, \underset{―}{v} \in F_{q}^{n},$ the scalar (element of $F_{q})$ defined as $\underset{―}{u} \cdot \underset{―}{v} = \sum_{i = 1}^{n} u_{i} v_{i}$ is called the inner product of the vectors $\underset{―}{u}$ and $\underset{―}{v} .$

Example: some inner products in $F_{2}^{3}$

For $111, 101 \in F_{2}^{3},$ one has

$111 \cdot 111 = 1^{2} + 1^{2} + 1^{2} = 1, 111 \cdot 101 = 1 \cdot 1 + 1 \cdot 0 + 1 \cdot 1 = 0, 101 \cdot 101 = 1^{2} + 0^{2} + 1^{2} = 0.$

If $C \subset F_{q}^{n}$ is a set, $\underset{―}{v} \in F_{q}^{n},$ we may write $\underset{―}{v} \cdot C$ to denote the set ${\underset{―}{v} \cdot \underset{―}{c} ∣ \underset{―}{c} \in C} .$

Properties of the inner product

(1) Expression as a matrix product: $\underset{―}{u} \cdot \underset{―}{v} = \underset{―}{u} {\underset{―}{v}}^{T} .$

Explanation: we write elements of $F_{q}^{n}$ as row vectors. Thus, $\underset{―}{u}$ is a row vector $(u_{1}, \dots, u_{n}),$ and ${\underset{―}{v}}^{T}$ is the transpose of $\underset{―}{v},$ so a column vector $(\begin{matrix} v_{1} \\ v_{2} \\ ⋮ \\ v_{n} \end{matrix}) .$ Multiplying $\underset{―}{u},$ an $1 \times n$ matrix, and ${\underset{―}{v}}^{T},$ an $n \times 1$ matrix, we obtain a $1 \times 1$ matrix, which we identify with a scalar in $F_{q} .$

(2) Symmetry: $\underset{―}{u} \cdot \underset{―}{v} = \underset{―}{v} \cdot \underset{―}{u} .$ (Explanation: this is easily seen from the definition.)

(3) Bilinearity: for a scalar $λ \in F_{q}$ we have $(\underset{―}{u} + λ \underset{―}{w}) \cdot \underset{―}{v} = \underset{―}{u} \cdot \underset{―}{v} + λ (\underset{―}{w} \cdot \underset{―}{v})$ and $\underset{―}{u} \cdot (\underset{―}{v} + λ w) = \underset{―}{u} \cdot \underset{―}{v} + λ (\underset{―}{u} \cdot \underset{―}{w}) .$ (Explanation: from linear algebra, the matrix product in $\underset{―}{u} {\underset{―}{v}}^{T}$ is bilinear.)

(4) Non-degeneracy: $\underset{―}{u} \cdot F_{q}^{n} = {0},$ if and only if $\underset{―}{u} = 0.$

Explanation: let ${\underset{―}{ϵ}}_{i} = (0, \dots, 0, 1, 0, \dots, 0)$ be the vector with $i$ th symbol $1$ and all other symbols $0.$ Then $\underset{―}{u} \cdot {\underset{―}{ϵ}}_{i} = u_{i} .$ So if $\underset{―}{u} \cdot F_{q}^{n} = {0},$ then in particular $\underset{―}{u} \cdot {\underset{―}{ϵ}}_{i} = 0$ hence $u_{i} = 0,$ for all $i,$ meaning that $\underset{―}{u}$ is the zero vector. And if $\underset{―}{u} = \underset{―}{0},$ then $\underset{―}{u} \cdot \underset{―}{c} = 0$ for all $\underset{―}{c} \in F_{q}^{n} .$

The dual code

Definition: dual code

Given a code $C \subseteq F_{q}^{n},$ we define the dual code $C^{⊥}$ as

$C^{⊥} = {\underset{―}{v} \in F_{q}^{n} ∣ \underset{―}{v} \cdot C = {0}} .$

We can say that $C^{⊥}$ consists of all vectors orthogonal to the code $C$ (where $\underset{―}{v}$ orthogonal to $C$ means $\underset{―}{v} \cdot C = {0}) .$

Exercise. Using bilinearity of the inner product, show that $C^{⊥}$ is a linear code.

Recall that $Rep (n, F_{2}) = {00 \dots 0, 11 \dots 1} \subseteq F_{2}^{n}$ is the binary repetition code of length $n .$ We now work out the code $Rep (n, F_{2})^{⊥}$ using the definition.

By definition, $Rep (n, F_{2})^{⊥} = {\underset{―}{v} \in F_{2}^{n} ∣ \underset{―}{v} \cdot 00 \dots 0 = 0, \underset{―}{v} \cdot 11 \dots 1 = 0} .$ The first condition, $\underset{―}{v} \cdot \underset{―}{0} = 0$ is vacuous (holds for all vectors $\underset{―}{v} \in F_{2}^{n}) .$ The second condition, $\underset{―}{v} \cdot 11 \dots 1,$ means $v_{1} + v_{2} + \dots + v_{n} = 0$ in $F_{2},$ i.e., $\underset{―}{v} \in E_{n},$ the binary even weight code of length $n .$ Thus:

Example: the dual code of the binary repetition code

$Rep (n, F_{2})^{⊥} = E_{n} .$

Check matrices

Definition: check matrix

A check matrix for a linear code $C$ means a generator matrix for $C^{⊥} .$

One sometimes says parity check matrix (the term arose from applications of binary codes).

Theorem 5.1: properties of the dual code and a check matrix

If $C \subseteq F_{q}^{n}$ is a linear code of dimension $k,$ then:

i. $\dim C^{⊥} = n - k;$
ii. $C = {\underset{―}{v} \in F_{q}^{n} : \underset{―}{v} H^{T} = \underset{―}{0}}$ for any check matrix $H$ of $C .$

Proof. We recall the Rank-Nullity Theorem from Linear Algebra: if $M$ is a matrix with $n$ columns, then

$rank (M) + \dim Nullspace (M) = n,$

where $rank (M)$ is the dimension of the span of the rows of $M,$ and $Nullspace (M)$ can be written as ${\underset{―}{v} \in F_{q}^{n} : M {\underset{―}{v}}^{T} = \overset{―}{0}} .$

i. Consider the matrix $[\begin{matrix} C \end{matrix}]$ made up of all codevectors of $C$ used as rows. The $Nullspace ([\begin{matrix} C \end{matrix}])$ is the set ${\underset{―}{v} : [\begin{matrix} C \end{matrix}] {\underset{―}{v}}^{T} = \overset{―}{0}} .$ Note that the column vector $[\begin{matrix} C \end{matrix}] {\underset{―}{v}}^{T}$ is $[\begin{matrix} {\underset{―}{c}}_{1} {\underset{―}{v}}^{T} \\ {\underset{―}{c}}_{2} {\underset{―}{v}}^{T} \\ ⋮ \end{matrix}] = [\begin{matrix} {\underset{―}{c}}_{1} \cdot \underset{―}{v} \\ {\underset{―}{c}}_{2} \cdot \underset{―}{v} \\ ⋮ \end{matrix}],$ which is zero if and only if the inner product $\underset{―}{c} \cdot \underset{―}{v}$ is $0$ for all rows $\underset{―}{c}$ of $[\begin{matrix} C \end{matrix}],$ i.e., for all codevectors $\underset{―}{c}$ of $C .$ By definition of the dual code, this happens exactly when $\underset{―}{v} \in C^{⊥},$ so $Nullspace ([\begin{matrix} C \end{matrix}]) = C^{⊥} .$ By rank-nullity, $\dim C^{⊥} = n - rank ([\begin{matrix} C \end{matrix}]) .$ Since the rows of $[\begin{matrix} C \end{matrix}]$ span $C,$ one has $rank ([\begin{matrix} C \end{matrix}]) = \dim C = k$ and so $\dim C^{⊥} = n - k .$

ii. By definition $H$ generates the code $C^{⊥};$ so by i., $H$ has $n - k$ rows, $H = [\begin{matrix} {\underset{―}{r}}_{1} \\ ⋮ \\ {\underset{―}{r}}_{n - k} \end{matrix}] .$ Thus, $rank (H) = \dim C^{⊥} = n - k,$ and so by rank-nullity, $\dim Nullspace (H) = n - (n - k) = k .$ Note that $C \subseteq Nullspace (H) :$ indeed, if $\underset{―}{c} \in C,$ then ${\underset{―}{r}}_{i} {\underset{―}{c}}^{T} = {\underset{―}{r}}_{i} \cdot \underset{―}{c} = 0$ for all $i$ because ${\underset{―}{r}}_{i} \in C^{⊥},$ which means that $H {\underset{―}{c}}^{T} = \overset{―}{0} .$ Since $\dim C = \dim Nullspace (H),$ it follows that $C = Nullspace (H),$ which is ${\underset{―}{v} : H {\underset{―}{v}}^{T} = \overset{―}{0}} .$

The law $(A B)^{T} = B^{T} A^{T}$ for the product of matrices implies that $(\underset{―}{v} H^{T})^{T} = H {\underset{―}{v}}^{T},$ and so $H {\underset{―}{v}}^{T}$ is zero iff $\underset{―}{v} H^{T}$ is. Thus, $C = {\underset{―}{v} \in F_{q}^{n} : \underset{―}{v} H^{T} = \underset{―}{0}}$ as claimed. □

The syndrome of a vector

Definition: syndrome

Let $H$ be a check matrix for a linear code $C \subseteq F_{q}^{n} .$ Let $\underset{―}{y} \in F_{q}^{n} .$ The vector

$S (\underset{―}{y}) = \underset{―}{y} H^{T}$

is called the syndrome of $\underset{―}{y} .$ The linear map $S : F_{q}^{n} \to F_{q}^{n - k}$ is the syndrome map.

Proposition 5.2: syndromes of vectors in the same coset

Let $S$ be a syndrome map for a linear code $C \subseteq F_{q}^{n} .$ If $\underset{―}{v}, \underset{―}{y} \in F_{q}^{n},$

• $S (\underset{―}{v}) = S (\underset{―}{y})$ $⟺$ $\underset{―}{v}, \underset{―}{y}$ are in the same coset of $C;$
• $S (\underset{―}{v}) = \underset{―}{0}$ $⟺$ $\underset{―}{v} \in C .$

Proof. $\underset{―}{y} H^{T} = \underset{―}{v} H^{T}$ $⟺$ $(\underset{―}{y} - \underset{―}{v}) H^{T} = \underset{―}{0}$ $⟺$ $\underset{―}{y} - \underset{―}{v} \in C .$ By definition of cosets, this means that $\underset{―}{v}$ is in the coset of $\underset{―}{y} .$ In particular, $S (\underset{―}{v}) = \underset{―}{0}$ means $\underset{―}{v} \in \underset{―}{0} + C = C .$ □

The use of syndromes in error detection and correction

If $S (\underset{―}{y}) \neq 0,$ $\underset{―}{y}$ is not a codevector, so the syndrome map detects errors in a received vector.

To correct errors, we need to construct a decoder for the linear code $C .$ If we know a check matrix $H$ for $C,$ we can improve the standard array decoder for $C .$ We will write the same decoder differently; it will require much less memory but more calculations.

Algorithm 5.3: the syndrome decoder

Preparation. Construct a table of syndromes, with $q^{n - k}$ rows, of the form

.

Coset leader ${\underset{―}{a}}_{i}$	$S ({\underset{―}{a}}_{i})$

Start with the top row: the codeword $\underset{―}{0}$ and its syndrome $S (\underset{―}{0}) = \underset{―}{0} .$

At each step, choose a vector ${\underset{―}{a}}_{i} \in F_{q}^{n}$ of smallest weight such that $S ({\underset{―}{a}}_{i})$ does not appear in the table; then ${\underset{―}{a}}_{i}$ is a coset leader of a new coset.

Decoding.

• Receive a vector $\underset{―}{y} \in F_{q}^{n} .$
• Calculate $S (\underset{―}{y}) = \underset{―}{y} H^{T} .$
• In the table, find ${\underset{―}{a}}_{i}$ with $S ({\underset{―}{a}}_{i}) = S (\underset{―}{y}) .$ Then ${\underset{―}{a}}_{i}$ is the coset leader of $\underset{―}{y} + C .$
• Return $DECODE (\underset{―}{y}) = \underset{―}{y} - {\underset{―}{a}}_{i} .$

Remark. The syndrome decoder is based on a choice of one coset leader in every coset. This is the same as for the standard array decoder.

In fact, if the same coset leaders are chosen in both decoders, both decoders with yield the same function $DECODE : F_{q}^{n} \to C .$ They differ only in the way this function is computed.

The number of arithmetic operations required to calculate the syndrome $S (\underset{―}{y}) = \underset{―}{y} H^{T}$ can be of order $n^{2},$ whereas the standard array decoder requires $\sim n$ operations to look up a vector. On the other hand, the amount of memory required by the syndrome decoder is proportional to $q^{n - k}$ which is better than $q^{n}$ for the standard array. The advantage is especially significant for codes with high code rate $\frac{k}{n} .$

Nevertheless, for codes which have more algebraic structure (than just linear codes), decoding algorithms exist which require even less storage, but the computation complexity is higher compared to syndrome decoding. Some examples will appear from the next chapter onwards.

Example: example of syndrome decoding

Let $C$ be the binary linear code with check matrix $H = [\begin{array}{cccccc} 0 & 0 & 1 & 0 & 0 & 0 \\ 1 & 0 & 0 & 1 & 0 & 0 \\ 1 & 1 & 0 & 0 & 1 & 0 \\ 0 & 1 & 0 & 0 & 0 & 1 \end{array}] .$

(a) Construct the table of syndromes for $C$ using the matrix $H .$
(b) Using the table of syndromes, decode the received vector $\underset{―}{y} = 111111.$

Solution.

(a) When calculating syndromes, it is useful to observe that the syndrome of a vector $0 \dots 010 \dots 0$ (with $1$ in position $i$ and $0$ s elsewhere) is equal to the $i$ th column of $H,$ transposed.

The syndrome map is linear, so the syndrome of a sum of two vectors is the sum of their syndromes, etc.

For example, $S (011000) = 0011 + 1000 = 1011$ (the sum of the second and the third columns of $H,$ transposed).

.
vector	syndrome	leader?
$000000$	$0000$	yes
$000001$	$0001$	yes
$000010$	$0010$	yes
$000100$	$0100$	yes
$001000$	$1000$	yes
$010000$	$0011$	yes
$100000$	$0110$	yes

All vectors of weight $1$ have different syndromes, so they all are coset leaders. We need more coset leaders, hence we start looking at vectors of weight $2,$ then weight $3 :$

.

$000011$	$0011$	no, syndrome already in the table
$000101$	$0101$	yes
$001001$	$1001$	yes
$001010$	$1010$	yes
$001100$	$1100$	yes
$010100$	$0111$	yes
$011000$	$1011$	yes
$101000$	$1110$	yes
$001101$	$1101$	yes
$011100$	$1111$	yes

When we try a vector, say of weight $2,$ and find that is syndrome is already in the table, we ignore that vector and try another one.

We found $16 = 2^{6 - 2}$ coset leaders so we stop.

(b) $S (111111) = 1010$ which is the syndrome of the coset leader $001010$ in the table. Therefore, $DECODE (111111) = 111111 - 001010 = 110101.$