Skip to content
Geometry and Algebra

Matrices: operations, the row-by-column product and the inverse

Why the row-by-column product is composition, properties and non-commutativity, special matrices and the transpose, the 2×2 inverse and computing it with Gauss-Jordan.

4 min read 0 comments

A matrix is a table of numbers, but above all it is a compact way of writing transformations and systems. In this lesson we cover matrix operations, why the row-by-column product is defined the way it is, special matrices and how to compute the inverse.

Definitions

An m\times n matrix has m rows and n columns; the entry in row i and column j is written a_{ij}. If m = n the matrix is square of order n. Real m\times n matrices form the set M_{m,n}(\mathbb R).

Addition and multiplication by a scalar are done entry by entry, and they have the same properties as for vectors: M_{m,n}(\mathbb R) is a vector space of dimension mn.

The row-by-column product

If A is m\times n and B is n\times p, the product AB is the m\times p matrix with

    \[ (AB)_{ij} = \sum_{k=1}^{n} a_{ik}\,b_{kj} \]

that is, the dot product of row i of A with column j of B. The number of columns of A must equal the number of rows of B.

    \[ \begin{pmatrix} 1 & 2\\ 3 & 4 \end{pmatrix}\begin{pmatrix} 5 & 6\\ 7 & 8 \end{pmatrix} = \begin{pmatrix} 1\cdot 5 + 2\cdot 7 & 1\cdot 6 + 2\cdot 8\\ 3\cdot 5 + 4\cdot 7 & 3\cdot 6 + 4\cdot 8 \end{pmatrix} = \begin{pmatrix} 19 & 22\\ 43 & 50 \end{pmatrix} \]

Why it is defined this way

A matrix A transforms a vector \mathbf x into the vector A\mathbf x. If we first apply B and then A, we get A(B\mathbf x). Writing out the components:

    \[ \big(A(B\mathbf x)\big)_i = \sum_k a_{ik}\,(B\mathbf x)_k = \sum_k a_{ik}\sum_j b_{kj}x_j = \sum_j\Big(\sum_k a_{ik}b_{kj}\Big)x_j \]

The coefficient in brackets is exactly (AB)_{ij}: the matrix product is the composition of the transformations. It follows at once that the product is associative, (AB)C = A(BC), because composition of functions is.

The product is not commutative

In general AB \ne BA, even when both exist. Rotating by 90° and then reflecting in the x axis does not give the same result as reflecting and then rotating. Other differences from numbers:

  • AB = 0 does not imply A = 0 or B = 0: for example \begin{pmatrix}0 & 1\\ 0 & 0\end{pmatrix}^2 = 0;
  • from AB = AC you cannot “cancel” A, unless A is invertible.

Special matrices

  • Identity I_n: 1 on the diagonal, 0 elsewhere. AI = IA = A.
  • Diagonal: zeros off the diagonal. They multiply entry by entry along the diagonal, and their powers are trivial.
  • Triangular, upper or lower: zeros below (or above) the diagonal. The echelon form of Gaussian elimination is upper triangular.
  • Transpose A^T: rows and columns swapped, (A^T)_{ij} = a_{ji}. We have (AB)^T = B^T A^T (mind the order).
  • Symmetric: A^T = A. They appear everywhere, from covariance matrices to the stiffness matrices of structures, and have special properties we will meet with the spectral theorem.

The inverse matrix

A square matrix A is invertible if there is A^{-1} with AA^{-1} = A^{-1}A = I. If it exists it is unique, and the system A\mathbf x = \mathbf b has the single solution \mathbf x = A^{-1}\mathbf b.

We have (AB)^{-1} = B^{-1}A^{-1} and (A^T)^{-1} = (A^{-1})^T. A matrix is invertible if and only if its determinant is non-zero (see the lesson on the determinant), that is if and only if its rank is n.

The 2×2 case. If ad - bc \ne 0:

    \[ \begin{pmatrix} a & b\\ c & d \end{pmatrix}^{-1} = \frac{1}{ad - bc}\begin{pmatrix} d & -b\\ -c & a \end{pmatrix} \]

Check by multiplying: the product gives (ad - bc)\,I, divided by ad - bc.

Computing the inverse with Gauss-Jordan

Place the identity next to A and apply elementary operations until A becomes I: A^{-1} appears on the right.

    \[ \left(\begin{array}{cc|cc} 2 & 1 & 1 & 0\\ 5 & 3 & 0 & 1 \end{array}\right) \xrightarrow{R_2 - \frac52 R_1} \left(\begin{array}{cc|cc} 2 & 1 & 1 & 0\\ 0 & \frac12 & -\frac52 & 1 \end{array}\right) \xrightarrow[R_1 - R_2]{2R_2} \left(\begin{array}{cc|cc} 2 & 0 & 6 & -2\\ 0 & 1 & -5 & 2 \end{array}\right) \xrightarrow{\frac12 R_1} \left(\begin{array}{cc|cc} 1 & 0 & 3 & -1\\ 0 & 1 & -5 & 2 \end{array}\right) \]

So A^{-1} = \begin{pmatrix} 3 & -1\\ -5 & 2 \end{pmatrix}, as the 2×2 formula with ad - bc = 1 confirms.

Why it works. Each elementary operation amounts to multiplying on the left by an elementary matrix E_k. If E_k\cdots E_1 A = I, then the product E_k\cdots E_1 is exactly A^{-1}, and applied to I it gives A^{-1}. If during the process a row on the left becomes all zeros, A is not invertible.

A practical tip

To solve A\mathbf x = \mathbf b on a computer you never compute A^{-1}: it costs about three times as much as Gaussian elimination and is numerically less accurate. The inverse is a theoretical tool, useful for writing formulas; in programs you use A\backslash b in MATLAB or numpy.linalg.solve.

Related articles

Subscribe
Notify of
guest
0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x
()
x