Here’s Looking at You, Matrix Multiplication

Given a real n\times n matrix A and a vector \vec{x}\in\mathbb{R}^n, A\vec{x} produces an n dimensional vector \vec{b}. This is a stretched, rotated version of \vec{x}. Two-dimensional matrix multiplication is easily visualized on a plane.

Two dimensional case: n=2

Given matrix A = \begin{bmatrix} a & b \\ c & d\\ \end{bmatrix}, multiplying by vector \vec{x} = \begin{bmatrix} x \\ y\\ \end{bmatrix} produces \vec{b}= \begin{bmatrix} ax +by\\ cx+dy\\ \end{bmatrix} .

Say we wanted to find a matrix to rotate a vector \begin{bmatrix} x \\ y\\ \end{bmatrix} 90 degrees clockwise. The desired result of multiplication will be \begin{bmatrix} y \\ -x\\ \end{bmatrix} . Studying the formula above, this is accomplished when a=0, b=1, c=-1, d=0 .

Therefore given any \vec{x} \in \mathbb{R}^2, left-multiplying by A = \begin{bmatrix} 0& 1 \\ -1 & 0\\ \end{bmatrix} produces the rotated vector. The animation below shows the result of successive matrix multiplications. Of course, rotating by 90 degrees four times produces a cycle, since after four turns you end up where you started.

The arrows start to pile up on themselves.

Matrix multiplication can also be used to scale the components of \vec{x}.

A = \begin{bmatrix} 0.9& 0 \\ 0 & 0.8\\ \end{bmatrix} scales the x and y components of \vec{x} by 0.9 and 0.8 respectively. Since one component shrinks faster than the others, the ends of the arrows representing \vec{x} form a curved path.

Sometimes, repeated multiplication will rotate and stretch the vector outwards. Here A = \begin{bmatrix} 1 & -0.8 \\ 0.8 & 1\\ \end{bmatrix}.

Of course, there exist matrices such that the vector will converge to the origin. Can you think of one?

Application to Politics

Imagine x and y represent the percentage of voters in a district with allegiance to the Democratic and Republican parties, respectively (and assume no third parties exist).

Assume that every year, 2% of Republicans switch their allegiance to the Democratic party, and 4% of Democrats become Republicans. The percentage of voters in each party after a year can be computed with a single matrix multiplication:

\begin{bmatrix} x_{\text{next}} \\ y_{\text{next}} \\ \end{bmatrix} = \begin{bmatrix} 0.96 & 0.02 \\ 0.04 & 0.98\\ \end{bmatrix} \begin{bmatrix} x \\ y \\ \end{bmatrix}

The distribution of voters after k years can be calculated simply with k successive matrix multiplications. Below is a visualization of this process, starting with an even distribution of allegiances.

The arrow converges to (33.\bar{3}, 66.\bar{6}) no matter the initial voter allegiances.

Notice that by our model, x+y is always equal to 100, so this process does not shrink the vector to zero or stretch it endlessly. Also, this arrow cannot leave the first quadrant, since the percentage of voters is always a nonnegative number.

Perhaps a more natural way to convey the same information is using a bar chart animation.

Extension to Higher-Dimensional Matrices

Since high-dimensional arrows are hard to plot, it is more convenient to visualize this process with multiple bar charts when dealing with many dimensions.

Instead of the example with two political parties above, consider a situation using 4 political affiliations. Let x,y,z,a represent the percent support for the MORENA, PAN, PRI, and independent candidates, respectively in some Mexican state. Assume the affiliation percentages change from one month to the next according to the following law. (What does this mean in practical terms?)

\begin{bmatrix} x_{\text{next}} \\ y_{\text{next}} \\ z _{\text{next}} \\ a _{\text{next}} \\ \end{bmatrix} = \begin{bmatrix} 0.85 & 0.05 & 0.1 & 0.4 \\ 0.05 & 0.9 & 0.05& 0.05 \\ 0.05 & 0.03 & 0.75 & 0.05 \\ 0.05 & 0.02 & 0.1 & 0.5 \\ \end{bmatrix} \begin{bmatrix} x \\ y \\ z\\ a\\ \end{bmatrix}

The results of successive multiplication produce a victory for the MORENA party.


Two resources have inspired today’s blog post:

  1. Gilbert Strang’s fantastic textbook, Linear Algebra and its Applications, for introducing me to the idea of modelling population dynamics using matrix multiplication.
  2. The Essence of Linear Algebra video series by 3Blue1Brown. Grant Sanderson is a master at explaining mathematics using visual aids and his videos informed my understanding of linear algebra.
  3. As always, the code is available on GitHub. I used VirtualDub and Handbrake to produce the animations as MP4 files.

Leave a Reply

Your email address will not be published.