Fall 2024
MATH 204: Introduction to Linear Algebra
Branko Ćurgus

Tuesday, November 19, 2024

---

• Recommended exercises for Section 5.3 are 2, 3, 5, 8, 9, 12, 13, 16, 18, 20, 23, 24.
• Today we discussed the following theorem:

  Theorem. Let $n \in \mathbb{N}$ and let $A$ be an $n\!\times\!n$ matrix. The matrix $A$ is diagonalizable if and only if there exists a basis of $\mathbb{R}^n$ which consists of eigenvectors of $A.$

  The above formulation of this important theorem is short. It requires the understanding of the definition of a diagonalizable matrix and the understanding of the definition of a basis of $\mathbb{R}^n$ and the understanding of the concept of an eigenvector of $A.$ A long statement of the theorem which incorporates all these definitions is as follows:

  Theorem. Let $n \in \mathbb{N}$ and let $A$ be an $n\!\times\!n$ matrix. The following two statements are equivalent:

  (a) There exist an invertible $n\!\times\!n$ matrix $P$ and a diagonal $n\!\times\!n$ matrix $D$ such that $A= PDP^{-1}.$

  (b) There exist linearly independent vectors $\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n$ in $\mathbb{R}^n$ and real numbers $\lambda_1, \lambda_2,\ldots,\lambda_n$ such that $A \mathbf{v}_k = \lambda_k \mathbf{v}_k$ for all $k\in \{1,\ldots,n\}.$

• Proof of (b)⇒(a).  Assume (b). That is, assume that there exist linearly independent vectors $\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n$ in $\mathbb{R}^n$ and real numbers $\lambda_1, \lambda_2,\ldots,\lambda_n$ such that \[ A \mathbf{v}_k = \lambda_k \mathbf{v}_k \quad \text{for all} \quad k\in \{1,\ldots,n\}. \] The preceding displayed expression contains $n$ vector equations. These $n$ vector equations can be expressed as one matrix equation \[ \Bigl[ A \mathbf{v}_1 \ A \mathbf{v}_2 \ \cdots \ A \mathbf{v}_n \Bigr] = \Bigl[ \lambda_1 \mathbf{v}_1 \ \lambda_2 \mathbf{v}_2 \ \cdots \ \lambda_n \mathbf{v}_n \Bigr]. \] By the definition of the matrix multiplication the preceding matrix equality can be expressed as \[ A \Bigl[ \mathbf{v}_1 \ \mathbf{v}_2 \ \cdots \ \mathbf{v}_n \Bigr] = \Bigl[ \mathbf{v}_1 \ \mathbf{v}_2 \ \cdots \ \mathbf{v}_n \Bigr] \left[\!\begin{array}{cccc} \lambda_1 & 0 & \cdots & 0 \\ 0 & \lambda_1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda_n \end{array}\!\right]. \] The preceding matrix equality can be written as \[ A P = P D, \] where we set \[ P = \Bigl[ \mathbf{v}_1 \ \mathbf{v}_2 \ \cdots \ \mathbf{v}_n \Bigr] \quad \text{and} \quad D = \left[\!\begin{array}{cccc} \lambda_1 & 0 & \cdots & 0 \\ 0 & \lambda_1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda_n \end{array}\!\right]. \] Since we assume that $\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n$ are linearly independent vectors in $\mathbb{R}^n$ the matrix $P$ is invertible by the Invertible Matrix Theorem. Therefore, the matrix $P^{-1}$ exists. Multiplying the equality \[ AP=PD \] from the right by $P^{-1}$ we get \[ A = PDP^{-1}. \] This proves (a).
• Now we recall Example 1 from Monday, November 18, 2024. We calculated that \begin{align*} \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] \left[\! \begin{array}{c} 1 \\ 1 \\ 3 \end{array} \!\right] & = 1 \left[\! \begin{array}{c} 1 \\ 1 \\ 3 \end{array} \!\right], \\ \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & = 2 \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right], \\ \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right]\left[\! \begin{array}{c} 1 \\ 0 \\ 1 \end{array} \!\right] & = 2 \left[\! \begin{array}{c} 1 \\ 0 \\ 1 \end{array} \!\right]. \end{align*} and \[ \left[\! \begin{array}{crc|ccc} 1 & -1 & 1 & 1 & 0 & 0\\ 1 & 1 & 0 & 0 & 1 & 0 \\ 3 & 0 & 1 & 0 & 0 & 1 \end{array} \!\right] \sim \cdots \sim \left[\! \begin{array}{ccc|rrc} 1 & 0 & 0 & -1 & -1 & 1\\ 0 & 1 & 0 & 1 & 2 & -1 \\ 0 & 0 & 1 & 3 & 3 & -2 \end{array} \!\right]. \]
• In the preceding item we have that \[ A = \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] \] is a $3\!\times\!3$ matrix. The vectors \[ \mathbf{v}_1 = \left[\! \begin{array}{c} 1 \\ 1 \\ 3 \end{array} \!\right], \quad \mathbf{v}_2 = \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right], \quad \mathbf{v}_3 = \left[\! \begin{array}{r} 1 \\ 0 \\ 1 \end{array} \!\right] \] are linearly independent vectors in $\mathbb{R}^3$ which are eigenvectors of $A$ such that \[ A \mathbf{v}_1 = 1 \mathbf{v}_1, \quad A \mathbf{v}_2 = 2 \mathbf{v}_2, \quad A \mathbf{v}_3 = 2 \mathbf{v}_3. \] Based on the proof presented above the preceding three vector equations can be written as one matrix equation as follows (this is the equation $AP=PD$ in the proof) \[ \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] \left[\! \begin{array}{crc} 1 & -1 & 1\\ 1 & 1 & 0 \\ 3 & 0 & 1 \end{array} \!\right] = \left[\! \begin{array}{crc} 1 & -1 & 1\\ 1 & 1 & 0 \\ 3 & 0 & 1 \end{array} \!\right] \left[\! \begin{array}{crc} 1 & 0 & 0\\ 0 & 2 & 0 \\ 0 & 0 & 2 \end{array} \!\right] \] Multiplying the preceding matrix equation with the inverse calculated in the previous item \[ \left[\! \begin{array}{crc} 1 & -1 & 1\\ 1 & 1 & 0 \\ 3 & 0 & 1 \end{array} \!\right]^{-1} = \left[\! \begin{array}{rrr} -1 & -1 & 1 \\ 1 & 2 & -1 \\ 3 & 3 & -2 \end{array} \!\right], \] we get the diagonalization of the matrix $A$: \[ \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] = \left[\! \begin{array}{rrr} 1 & -1 & 1 \\ 1 & 1 & 0 \\ 3 & 0 & 1 \end{array} \!\right] \left[\! \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 2 \end{array} \!\right] \left[\! \begin{array}{rrr} -1 & -1 & 1 \\ 1 & 2 & -1 \\ 3 & 3 & -2 \end{array} \!\right] . \] Notice that this is a specific numerical matrix equality and it takes just a little bit of patience to verify whether it is true or not.

---

Monday, November 18, 2024

---

• For Section 5.2 do 1-8, 11, 12, 14, 15, (in all these problems you can find eigenvectors as well) 9, 13, 18, 19, 20, 21, 24, 25, 27.
• Example 1. In this item I will illustrate how to calculate eigenvalues and the corresponding eigenspaces of a specific $3\!\times\!3$ matrix. Consider the matrix \[ A = \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] . \]
  • First we find the characteristic polynomial of this matrix. The characteristic polynomial is the determinant of the following matrix: \[ A - \lambda I_3 = \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] - \left[\! \begin{array}{rrr} \lambda & 0 & 0 \\ 0 & \lambda & 0 \\ 0 & 0 & \lambda \end{array} \!\right] = \left[\! \begin{array}{ccc} 3-\lambda & 1 & -1 \\ 1 & 3-\lambda & -1 \\ 3 & 3 & -1-\lambda \end{array} \!\right] \] Next we calculate this determinant: \begin{align*} \left|\! \begin{array}{ccc} 3-\lambda & 1 & -1 \\ 1 & 3-\lambda & -1 \\ 3 & 3 & -1-\lambda \end{array} \!\right| &= \left|\! \begin{array}{ccc} 2-\lambda & -2+\lambda & 0 \\ 1 & 3-\lambda & -1 \\ 3 & 3 & -1-\lambda \end{array} \!\right| \\[5pt] &= \left|\! \begin{array}{ccc} 2-\lambda & 0 & 0 \\ 1 & 4-\lambda & -1 \\ 3 & 6 & -1-\lambda \end{array} \!\right| \\[5pt] &= (2-\lambda) \bigl( (4-\lambda)(-1-\lambda) + 6 \bigr) \\[5pt] & = (2-\lambda)\bigl(\lambda^2 - 3 \lambda + 2\bigr) \\[5pt] & = -(\lambda - 2)^2 ( \lambda - 1) \end{align*} (At the first equality sign, we subtracted the second row from the first. At the second equality sign, we added the first column to the second. These operations do not change the value of a determinant.)
  • Thus the eigenvalues of the matrix $A$ are $1$ and $2.$
  • Next we will find the eigenspace corresponding to the eigenvalue $1.$ For that we need to find the nullspace of the matrix \[ A - 1 I_3 = \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] - \left[\! \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \!\right] = \left[\! \begin{array}{ccc} 2 & 1 & -1 \\ 1 & 2 & -1 \\ 3 & 3 & -2 \end{array} \!\right]. \] So, we row reduce this matrix: \[ \left[\! \begin{array}{ccc} 2 & 1 & -1 \\ 1 & 2 & -1 \\ 3 & 3 & -2 \end{array} \!\right] \sim \left[\! \begin{array}{ccc} 1 & 2 & -1 \\ 0 & 3 & -1 \\ 0 & 3 & -1 \end{array} \!\right] \sim \left[\! \begin{array}{ccc} 1 & 2 & -1 \\ 0 & 1 & -1/3 \\ 0 & 0 & 0 \end{array} \!\right] \sim \left[\! \begin{array}{ccc} 1 & 0 & -1/3 \\ 0 & 1 & -1/3 \\ 0 & 0 & 0 \end{array} \!\right]. \] Thus, the eigenspace is the subspace \[ \left\{ \left[\! \begin{array}{c} s/3 \\ s/3 \\ s \end{array} \!\right] \ : \ s \in \mathbb{R} \right\} = \operatorname{Span} \left\{ \left[\! \begin{array}{c} 1 \\ 1 \\ 3 \end{array} \!\right] \right\}. \] Hence one eigenvector is $\left[\! \begin{array}{c} 1 \\ 1 \\ 3 \end{array} \!\right].$
  • Next we will find the eigenspace corresponding to the eigenvalue $2.$ For that we need to find the nullspace of the matrix \[ A - 2 I_3 = \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] - \left[\! \begin{array}{rrr} 2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 2 \end{array} \!\right] = \left[\! \begin{array}{rrr} 1 & 1 & -1 \\ 1 & 1 & -1 \\ 3 & 3 & -3 \end{array} \!\right]. \] So, we row reduce this matrix: \[ \left[\! \begin{array}{rrr} 1 & 1 & -1 \\ 1 & 1 & -1 \\ 3 & 3 & -3 \end{array} \!\right] \sim \left[\! \begin{array}{rrr} 1 & 1 & -1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{array} \!\right]. \] Thus, the eigenspace is the subspace \[ \left\{ \left[\! \begin{array}{c} -s + t \\ s \\ t \end{array} \!\right] \ : \ s, t \in \mathbb{R} \right\} = \operatorname{Span} \left\{ \left[\! \begin{array}{c} -1 \\ 1 \\ 0 \end{array} \!\right], \left[\! \begin{array}{c} 1 \\ 0 \\ 1 \end{array} \!\right] \right\}. \]
  • Hence, two linearly independent eigenvectors corresponding to the eigenvalue $2$ are \[ \left[\! \begin{array}{c} -1 \\ 1 \\ 0 \end{array} \!\right] \qquad \text{and} \qquad \left[\! \begin{array}{c} 1 \\ 0 \\ 1 \end{array} \!\right]. \]
  • The magic of what we found by now is that we found a basis of $\mathbb{R}^3$ which consists of eigenvectors of $A:$ \[ \left[\! \begin{array}{c} 1 \\ 1 \\ 3 \end{array} \!\right], \quad \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right], \quad \left[\! \begin{array}{c} 1 \\ 0 \\ 1 \end{array} \!\right]. \]
  • It is easy to verify whether these are really eigenvectors: \begin{align*} \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] \left[\! \begin{array}{c} 1 \\ 1 \\ 3 \end{array} \!\right] & = 1 \left[\! \begin{array}{c} 1 \\ 1 \\ 3 \end{array} \!\right], \\ \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right] \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & = 2 \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right], \\ \left[\! \begin{array}{rrr} 3 & 1 & -1 \\ 1 & 3 & -1 \\ 3 & 3 & -1 \end{array} \!\right]\left[\! \begin{array}{c} 1 \\ 0 \\ 1 \end{array} \!\right] & = 2 \left[\! \begin{array}{c} 1 \\ 0 \\ 1 \end{array} \!\right]. \end{align*} Yes, they are.
  • I also made the claim that the three eigenvectors are linearly independent. Let us verify that as well. \[ \left[\! \begin{array}{crc|ccc} 1 & -1 & 1 & 1 & 0 & 0\\ 1 & 1 & 0 & 0 & 1 & 0 \\ 3 & 0 & 1 & 0 & 0 & 1 \end{array} \!\right] \sim \cdots \sim \left[\! \begin{array}{ccc|rrc} 1 & 0 & 0 & -1 & -1 & 1\\ 0 & 1 & 0 & 1 & 2 & -1 \\ 0 & 0 & 1 & 3 & 3 & -2 \end{array} \!\right]. \] We know that the right-hand side matrix in the Reduced Row Echelon Form is the inverse of the matrix whose columns are the eigenvectors. To verify the row reduction above, we calculate: \[ \left[\! \begin{array}{rrr} 1 & -1 & 1 \\ 1 & 1 & 0 \\ 3 & 0 & 1 \end{array} \!\right] \left[\! \begin{array}{rrr} -1 & -1 & 1 \\ 1 & 2 & -1 \\ 3 & 3 & -2 \end{array} \!\right] = \left[\! \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \!\right]. \]
• Example 2. In this item I will illustrate how to calculate eigenvalues and the corresponding eigenspaces of a specific $4\!\times\!4$ matrix. The purpose is to demonstrate a matrix that is not diagonalizable. Consider the matrix \[ A = \left[\! \begin{array}{rrrr} 0 & 0 & -1 & -1 \\ -1 & 0 & 0 & 0 \\ 2 & 1 & 2 & 1 \\ -2 & -1 & -1 & 0 \end{array} \!\right] . \]
  • First we find the characteristic polynomial of this matrix. The characteristic polynomial is the determinant of the following matrix: \[ A - \lambda I_4 = \left[\! \begin{array}{rrrr} 0 & 0 & -1 & -1 \\ -1 & 0 & 0 & 0 \\ 2 & 1 & 2 & 1 \\ -2 & -1 & -1 & 0 \end{array} \!\right] - \left[\! \begin{array}{rrrr} \lambda & 0 &0 & 0 \\ 0 & \lambda & 0 & 0 \\ 0 & 0 & \lambda & 0 \\ 0 & 0 & 0 & \lambda \end{array} \!\right] = \left[\! \begin{array}{cccc} -\lambda & 0 & -1 & -1 \\ -1 & -\lambda & 0 & 0 \\ 2 & 1 & 2-\lambda & 1 \\ -2 & -1 & -1 & -\lambda \end{array} \!\right] \] Next we calculate the determinant of the preceding matrix: \begin{align*} \left|\! \begin{array}{cccc} -\lambda & 0 & -1 & -1 \\ -1 & -\lambda & 0 & 0 \\ 2 & 1 & 2-\lambda & 1 \\ -2 & -1 & -1 & -\lambda \end{array} \!\right| & = -\lambda \left|\begin{array}{ccc} -\lambda & 0 & 0 \\ 1 & 2-\lambda & 1 \\ -1 & -1 & -\lambda \end{array} \right| - \left| \begin{array}{ccc} -1 & -\lambda & 0 \\ 2 & 1 & 1 \\ -2 & -1 & -\lambda \end{array} \right| + \left| \begin{array}{ccc} -1 & -\lambda & 0 \\ 2 & 1 & 2-\lambda \\ -2 & -1 & -1 \end{array} \right| \\ & = \lambda^2 ( \lambda^2 -2 \lambda + 1 ) - \bigl( \lambda -1 +2 \lambda - 2 \lambda^2 \bigr) + \bigl( \lambda - 1 + 2 \lambda - 2 \lambda^2 \bigr) \\ & = \lambda^2 ( \lambda^2 -2 \lambda + 1 ) \\ & = \lambda^2 ( \lambda - 1 )^2 \end{align*}
  • Thus the eigenvalues of the matrix $A$ are $0$ and $1.$
  • Next we will find the eigenspace corresponding to the eigenvalue $0.$ For that we need to find the nullspace of the matrix \[ A - 0 I_4 = \left[\! \begin{array}{rrrr} 0 & 0 & -1 & -1 \\ -1 & 0 & 0 & 0 \\ 2 & 1 & 2 & 1 \\ -2 & -1 & -1 & 0 \end{array} \!\right] \]
  • So, we row reduce the matrix $A:$ \[ \left[\! \begin{array}{rrrr} 0 & 0 & -1 & -1 \\ -1 & 0 & 0 & 0 \\ 2 & 1 & 2 & 1 \\ -2 & -1 & -1 & 0 \end{array} \!\right] \sim \cdots \sim \left[\! \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & -1 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \end{array} \!\right] \] Thus, the eigenspace is the subspace \[ \left\{ \left[\! \begin{array}{c} 0 \\ s \\ -s \\ s \end{array} \!\right] \ : \ s \in \mathbb{R} \right\} = \operatorname{Span} \left\{ \left[\! \begin{array}{r} 0 \\ 1 \\ -1 \\ 1 \end{array} \!\right] \right\}. \]
  • Thus, eigenspace corresponding to the eigenvalue $0$ is one-dimensional.
  • Next we will find the eigenspace corresponding to the eigenvalue $1.$ For that we need to find the nullspace of the matrix \[ A - 1 I_4 = \left[\! \begin{array}{rrrr} 0 & 0 & -1 & -1 \\ -1 & 0 & 0 & 0 \\ 2 & 1 & 2 & 1 \\ -2 & -1 & -1 & 0 \end{array} \!\right] - \left[\! \begin{array}{rrrr} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{array} \!\right] = \left[\! \begin{array}{rrrr} -1 & 0 & -1 & -1 \\ -1 & -1 & 0 & 0 \\ 2 & 1 & 1 & 1 \\ -2 & -1 & -1 & -1 \end{array} \!\right] \]
  • So, we row reduce the precedig matrix: \[ \left[\! \begin{array}{rrrr} -1 & 0 & -1 & -1 \\ -1 & -1 & 0 & 0 \\ 2 & 1 & 1 & 1 \\ -2 & -1 & -1 & -1 \end{array} \!\right] \sim \cdots \sim \left[\! \begin{array}{rrrr} 1 & 0 & 1 & 1 \\ 0 & 1 & -1 & -1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array} \!\right] \] Thus, the eigenspace corresponding to the eigenvalue $1$ is the subspace \[ \left\{ \left[\! \begin{array}{c} -s-t \\ s+t \\ s \\ t \end{array} \!\right] \ : \ s, t \in \mathbb{R} \right\} = \operatorname{Span} \left\{ \left[\! \begin{array}{r} -1 \\ 1 \\ 1 \\ 0 \end{array} \!\right], \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \\ 1 \end{array} \!\right] \right\}. \]
  • Thus, the eigenspace corresponding to the eigenvalue $1$ is two-dimensional.
  • Since we found all eigenspaces of the $4\!\times\!4$ matrix $A$ and these eigenspaces have dimensions $1$ and $2$, we conclude that we can have at most three linearly independent eigenvectors. Consequently, we can not have a basis for $\mathbb R^4$ which consists of eigenvectors of $A.$ This shows directly that the matrix $A$ is not diagonalizable.

---

Tuesday, November 12, 2024

---

• We discussed Section 5.1. Suggested problems for Section 5.1: 1, 3, 4, 5, 6, 8, 11, 15, 16, 17, 19, 20, 24-27, 29, 30, 31.
• A related Wikipedia link: Eigenvalue, eigenvector and eigenspace.
• Below are animations of different matrices in action. In each scene the navy blue vector is the image of the sea green vector under the multiplication by a matrix $A$. For easier visualization of the action the heads of vectors leave traces.
• Just looking at the movies you can guess what are the eigenvalues and eigenvectors of the featured matrix. In particular it is easy to see whether an eigenvalue is positive, negative, zero, or a complex number, ... You can also approximately calculate which matrix is featured in each movie.

Place the cursor over the image to start the animation.

---

Saturday, November 9, 2024 (updated)

---

• In this file(updated), you will find sample problems formatted similarly to those on the exam. Note that some sample problems contain more items than would appear on the actual exam; I will include at most three small items per problem. Check the website on Monday for any updates to this sample file.

---

Friday, November 8, 2024

---

• Before I present the method of finding the inverse of an invertible matrix using cofactors, I point out that a more efficient way to find the inverse of an invertible $3\!\times\!3$ matrix is the method presented earlier where we calculated the inverse $A^{-1}$ by row reducing the $3\times 6$ matrix $[A | I_3]$.
• Let \[ A = \left[\! \begin{array}{cc} a & b \\ c & d \end{array} \!\right]. \] Then without any conditions on the matrix $A$ we have \[ \left[\! \begin{array}{cc} a & b \\ c & d \end{array} \!\right] \left[\! \begin{array}{rr} d & -b \\ -c & a \end{array} \!\right] = (ad-bc) \left[\! \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \!\right]. \] Therefore $A$ is invertable if and only if $\det(A) = ad-bc \neq 0.$ If $\det(A) = ad-bc \neq 0,$ then \[ A^{-1} = \frac{1}{ad-bc} \left[\! \begin{array}{rr} d & -b \\ - c & a \end{array} \!\right] = \frac{1}{ad-bc} \left[\! \begin{array}{rr} d & -c \\ -b & a \end{array} \!\right]^{\top} \]
• It is truly amazing that the same pattern holds for a $3\!\times\!3$ matrix. We just have to take into account that the definition of the determinant for a $3\!\times\!3$ matrix is more complicated and the determinant can be expressed as three different looking cofactor expensions.
• Let \[ A = \left[\! \begin{array}{rrr} a & b & c \\ d & e & f \\ g & h & i \end{array} \!\right]. \] Recall that \begin{align*} \require{bbox} \det(A) & = a \left(\bbox[yellow]{\left|\! \begin{array}{cc} e & f \\ h & i \end{array} \!\right|}\right) + b \left(\bbox[yellow]{-\left|\! \begin{array}{cc} d & f \\ g & i \end{array} \!\right|}\right) + c \left(\bbox[yellow]{\left|\! \begin{array}{cc} d & e \\ g & h \end{array} \!\right|}\right) \\ & = d \left(\bbox[cyan]{-\left|\! \begin{array}{cc} b & c \\ h & i \end{array} \!\right|} \right) + e \left(\bbox[cyan]{\left|\! \begin{array}{cc} a & c \\ g & i \end{array} \!\right|}\right) + f \left(\bbox[cyan]{-\left|\! \begin{array}{cc} a & b \\ g & h \end{array} \!\right|}\right) \\ & = g \left(\bbox[#FF88FF]{\left|\! \begin{array}{cc} b & c \\ e & f \end{array} \!\right|}\right) + h \left(\bbox[#FF88FF]{-\left|\! \begin{array}{cc} a & c \\ d & f \end{array} \!\right|}\right) + i \left(\bbox[#FF88FF]{\left|\! \begin{array}{cc} a & b \\ d & e \end{array} \!\right|}\right) \\ & = aei + bfg +cdh - afh - bdi - ceg. \end{align*}
• Then without any assumptions on the $3\!\times\!3$ matrix $A$ we have \[ \require{bbox} \left[\! \begin{array}{rrr} a & b & c \\ d & e & f \\ g & h & i \end{array} \!\right] \left[\! \begin{array}{rrr} \bbox[yellow]{\left|\! \begin{array}{cc} e & f \\ h & i \end{array} \!\right|} & \bbox[cyan]{ -\left|\! \begin{array}{cc} b & c \\ h & i \end{array} \!\right|} & \bbox[#FF88FF]{ \left|\! \begin{array}{cc} b & c \\ e & f \end{array} \!\right|} \\[6pt] \bbox[yellow]{ -\left|\! \begin{array}{cc} d & f \\ g & i \end{array} \!\right|} & \bbox[cyan]{ \left|\! \begin{array}{cc} a & c \\ g & i \end{array} \!\right|} & \bbox[#FF88FF]{ -\left|\! \begin{array}{cc} a & c \\ d & f \end{array} \!\right|} \\[6pt] \bbox[yellow]{ \left|\! \begin{array}{cc} d & e \\ g & h \end{array} \!\right|} & \bbox[cyan]{ -\left|\! \begin{array}{cc} a & b \\ g & h \end{array} \!\right|} & \bbox[#FF88FF]{ \left|\! \begin{array}{cc} a & b \\ d & e \end{array} \!\right|} \end{array} \!\right] = \] (here we use the definition of matrix multiplication to calculate the product) \[ = \require{bbox} \left[\! \begin{array}{ccc} \bbox[yellow]{ a \left|\! \begin{array}{cc} e & f \\ h & i \end{array} \!\right| - b \left|\! \begin{array}{cc} d & f \\ g & i \end{array} \!\right| + c \left|\! \begin{array}{cc} d & e \\ g & h \end{array} \!\right|} \ & \bbox[cyan]{ - a \left|\! \begin{array}{cc} b & c \\ h & i \end{array} \!\right| + b \left|\! \begin{array}{cc} a & c \\ g & i \end{array} \!\right| - c \left|\! \begin{array}{cc} a & b \\ g & h \end{array} \!\right|} \ & \bbox[#FF88FF]{ a \left|\! \begin{array}{cc} b & c \\ e & f \end{array} \!\right| - b \left|\! \begin{array}{cc} a & c \\ d & f \end{array} \!\right| + c \left|\! \begin{array}{cc} a & b \\ d & e \end{array} \!\right|} \\[14pt] \bbox[yellow]{ d \left|\! \begin{array}{cc} e & f \\ h & i \end{array} \!\right| - e \left|\! \begin{array}{cc} d & f \\ g & i \end{array} \!\right| + f \left|\! \begin{array}{cc} d & e \\ g & h \end{array} \!\right|} \ & \bbox[cyan]{ - d \left|\! \begin{array}{cc} b & c \\ h & i \end{array} \!\right| + e \left|\! \begin{array}{cc} a & c \\ g & i \end{array} \!\right| - f \left|\! \begin{array}{cc} a & b \\ g & h \end{array} \!\right|} \ & \bbox[#FF88FF]{ d \left|\! \begin{array}{cc} b & c \\ e & f \end{array} \!\right| - e \left|\! \begin{array}{cc} a & c \\ d & f \end{array} \!\right| + f \left|\! \begin{array}{cc} a & b \\ d & e \end{array} \!\right|} \\[14pt] \bbox[yellow]{ g \left|\! \begin{array}{cc} e & f \\ h & i \end{array} \!\right| - h \left|\! \begin{array}{cc} d & f \\ g & i \end{array} \!\right| + i \left|\! \begin{array}{cc} d & e \\ g & h \end{array} \!\right|} \ & \bbox[cyan]{ - g \left|\! \begin{array}{cc} b & c \\ h & i \end{array} \!\right| + h \left|\! \begin{array}{cc} a & c \\ g & i \end{array} \!\right| - i \left|\! \begin{array}{cc} a & b \\ g & h \end{array} \!\right|} \ & \bbox[#FF88FF]{ g \left|\! \begin{array}{cc} b & c \\ e & f \end{array} \!\right| - h \left|\! \begin{array}{cc} a & c \\ d & f \end{array} \!\right| + i \left|\! \begin{array}{cc} a & b \\ d & e \end{array} \!\right|} \end{array} \!\right] = \] (now we notice that each colored entry in the preceding matrix is a cofactor expension of a $3\!\times\!3$ determinant; so each colored entry we write as a determinant; so the entry of the preceding matrix are $3\!\times\!3$ determinants) \[ = \left[\! \begin{array}{ccc} \bbox[yellow]{\left|\! \begin{array}{ccc} a & b & c \\ d & e & f \\ g & h & i \end{array} \!\right|} & \bbox[cyan]{\left|\! \begin{array}{ccc} a & b & c \\ a & b & c \\ g & h & i \end{array} \!\right|} & \bbox[#FF88FF]{ \left|\! \begin{array}{ccc} a & b & c \\ d & e & f \\ a & b & c \end{array} \!\right|} \\[6pt] \bbox[yellow]{ \left|\! \begin{array}{ccc} d & e & f \\ d & e & f \\ g & h & i \end{array} \!\right| } & \bbox[cyan]{\left|\! \begin{array}{ccc} a & b & c \\ d & e & f \\ g & h & i \end{array} \!\right|} & \bbox[#FF88FF]{ \left|\! \begin{array}{ccc} a & b & c \\ d & e & f \\ d & e & f \end{array} \!\right|} \\[6pt] \bbox[yellow]{\left|\! \begin{array}{ccc} g & h & i \\ d & e & f \\ g & h & i \end{array} \!\right|} & \bbox[cyan]{ \left|\! \begin{array}{ccc} a & b & c \\ g & h & i \\ g & h & i \end{array} \!\right|} & \bbox[#FF88FF]{ \left|\! \begin{array}{ccc} a & b & c \\ d & e & f \\ g & h & i \end{array} \!\right| } \end{array} \!\right] = \] (now looking carefully at each of nine determinants, we see that on the diagonal we have $\det(A)$ and off-diagonal we have determinants with two identical rows, so these are equal to $0$) \[ =\left[\! \begin{array}{ccc} \bbox[yellow]{ \det(A)} & \bbox[cyan]{0} & \bbox[#FF88FF]{0} \\[6pt] \bbox[yellow]{ 0} & \bbox[cyan]{\det(A)} & \bbox[#FF88FF]{0} \\[6pt] \bbox[yellow]{0} & \bbox[cyan]{0} & \bbox[#FF88FF]{\det(A)} \end{array} \!\right] = \det(A) \left[\! \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \!\right]. \]
• In conclusion, without any conditions on the $3\!\times\!3$ matrix $A$ we have \[ \require{bbox} \left[\! \begin{array}{rrr} a & b & c \\ d & e & f \\ g & h & i \end{array} \!\right] \left[\! \begin{array}{rrr} \bbox[yellow]{\left|\! \begin{array}{cc} e & f \\ h & i \end{array} \!\right|} & \bbox[cyan]{ -\left|\! \begin{array}{cc} b & c \\ h & i \end{array} \!\right|} & \bbox[#FF88FF]{ \left|\! \begin{array}{cc} b & c \\ e & f \end{array} \!\right|} \\[6pt] \bbox[yellow]{ -\left|\! \begin{array}{cc} d & f \\ g & i \end{array} \!\right|} & \bbox[cyan]{ \left|\! \begin{array}{cc} a & c \\ g & i \end{array} \!\right|} & \bbox[#FF88FF]{ -\left|\! \begin{array}{cc} a & c \\ d & f \end{array} \!\right|} \\[6pt] \bbox[yellow]{ \left|\! \begin{array}{cc} d & e \\ g & h \end{array} \!\right|} & \bbox[cyan]{ -\left|\! \begin{array}{cc} a & b \\ g & h \end{array} \!\right|} & \bbox[#FF88FF]{ \left|\! \begin{array}{cc} a & b \\ d & e \end{array} \!\right|} \end{array} \!\right] = \det(A) \left[\! \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \!\right]. \]
• The preceding formula shows that $A$ is invertible if and only if $\det(A) \neq 0.$ If $\det(A) \neq 0,$ then \[ A^{-1} = \frac{1}{\det(A)} \left[\! \begin{array}{rrr} \bbox[yellow]{\left|\! \begin{array}{cc} e & f \\ h & i \end{array} \!\right|} & \bbox[cyan]{ -\left|\! \begin{array}{cc} b & c \\ h & i \end{array} \!\right|} & \bbox[#FF88FF]{ \left|\! \begin{array}{cc} b & c \\ e & f \end{array} \!\right|} \\[6pt] \bbox[yellow]{ -\left|\! \begin{array}{cc} d & f \\ g & i \end{array} \!\right|} & \bbox[cyan]{ \left|\! \begin{array}{cc} a & c \\ g & i \end{array} \!\right|} & \bbox[#FF88FF]{ -\left|\! \begin{array}{cc} a & c \\ d & f \end{array} \!\right|} \\[6pt] \bbox[yellow]{ \left|\! \begin{array}{cc} d & e \\ g & h \end{array} \!\right|} & \bbox[cyan]{ -\left|\! \begin{array}{cc} a & b \\ g & h \end{array} \!\right|} & \bbox[#FF88FF]{ \left|\! \begin{array}{cc} a & b \\ d & e \end{array} \!\right|} \end{array} \!\right]. \] Notice that in the matrix on the right we have cofactors of the matrix $A.$
• The preceding formula for the inverse is commonly written with the transpose of the matrix involving the cofactors. \[ A^{-1} = \frac{1}{\det(A)} \left[\! \begin{array}{rrr} \bbox[yellow]{\left|\! \begin{array}{cc} e & f \\ h & i \end{array} \!\right|} & \bbox[yellow]{-\left|\! \begin{array}{cc} d & f \\ g & i \end{array} \!\right|} & \bbox[yellow]{\left|\! \begin{array}{cc} d & e \\ g & h \end{array} \!\right|} \\ \bbox[cyan]{-\left|\! \begin{array}{cc} b & c \\ h & i \end{array} \!\right|} & \bbox[cyan]{\left|\! \begin{array}{cc} a & c \\ g & i \end{array} \!\right|} & \bbox[cyan]{-\left|\! \begin{array}{cc} a & b \\ g & h \end{array} \!\right|} \\ \bbox[#FF88FF]{\left|\! \begin{array}{cc} b & c \\ e & f \end{array} \!\right|} & \bbox[#FF88FF]{-\left|\! \begin{array}{cc} a & c \\ d & f \end{array} \!\right|} & \bbox[#FF88FF]{\left|\! \begin{array}{cc} a & b \\ d & e \end{array} \!\right|} \end{array} \!\right]^{\mathbf{\top}}. \] Please recall the definition of the cofactors of $A$. With the definition of the cofactors the above formula for the inverse is simply \[ A^{-1} = \frac{1}{\det(A)} \left[\! \begin{array}{rrr} \bbox[yellow]{C_{11}} & \bbox[yellow]{C_{21}} & \bbox[yellow]{C_{31}} \\ \bbox[cyan]{C_{12}} & \bbox[cyan]{C_{22}} & \bbox[cyan]{C_{32}} \\ \bbox[#FF88FF]{C_{13}} & \bbox[#FF88FF]{C_{23}} & \bbox[#FF88FF]{C_{33}} \end{array} \!\right]^{\mathbf{\top}}. \]
• The pattern presented above for a $3\!\times\!3$ matrix holds for any $n\!\times\!n$ matrix: \[ \left[\! \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{array} \!\right] \left[\! \begin{array}{cccc} C_{11} & C_{21} & \cdots & C_{n1} \\ C_{12} & C_{22} & \cdots & C_{n2} \\ \vdots & \vdots & \ddots & \vdots \\ C_{1n} & C_{2n} & \cdots & C_{nn} \end{array} \!\right] = \det(A) I_n. \] A justification for this identity is the definition of determinant using the cofactor expansions along the rows of $A$ and the fact that a determinant with two identical rows is equal to $0.$
• Let us calculate the inverse of a specific $3\!\times\!3$ matrix. Consider \[ A = \left[\! \begin{array}{ccc} 1 & 2 & 6 \\ 3 & 5 & 7 \\ 4 & 8 & 9 \end{array} \!\right]. \] Let us first evaluate the determinant of this matrix in three different ways, by performing the cofactor expansion along each column: \begin{align*} \det A & = 1*(\bbox[yellow]{+(-11)})+2*(\bbox[yellow]{-(-1)})+6*(\bbox[yellow]{+(4)}) \\ & = 3*(\bbox[cyan]{-(-30)})+5*(\bbox[cyan]{+(-15)})+7*(\bbox[cyan]{-(0)}) \\ & = 4*(\bbox[#FF88FF]{+(-16)})+8*(\bbox[#FF88FF]{-(-11)})+9*(\bbox[#FF88FF]{+(-1)}) \\ & = 15. \end{align*} Since we are doing this three different ways, there is a lot of checking going on in the above calculations.
• Based on the three cofactor expansions above we can write the inverse of the given matrix $A:$ \[ A^{-1} = \frac{1}{15} \left[\! \begin{array}{rrr} \bbox[yellow]{-11} & \bbox[cyan]{30} & \bbox[#FF88FF]{-16} \\ \bbox[yellow]{1} & \bbox[cyan]{-15} & \bbox[#FF88FF]{11} \\ \bbox[yellow]{4} & \bbox[cyan]{0} & \bbox[#FF88FF]{-1} \end{array} \!\right]. \] To verify, calculate \[ \left[\! \begin{array}{ccc} 1 & 2 & 6 \\ 3 & 5 & 7 \\ 4 & 8 & 9 \end{array} \!\right] \left[\! \begin{array}{rrr} \bbox[yellow]{-11} & \bbox[cyan]{30} & \bbox[#FF88FF]{-16} \\ \bbox[yellow]{1} & \bbox[cyan]{-15} & \bbox[#FF88FF]{11} \\ \bbox[yellow]{4} & \bbox[cyan]{0} & \bbox[#FF88FF]{-1} \end{array} \!\right] = \left[\! \begin{array}{rrr} \bbox[yellow]{15} & \bbox[cyan]{0} & \bbox[#FF88FF]{0} \\ \bbox[yellow]{0} & \bbox[cyan]{15} & \bbox[#FF88FF]{0} \\ \bbox[yellow]{0} & \bbox[cyan]{0} & \bbox[#FF88FF]{15} \end{array} \!\right] \]
• Suggested problems for Section 3.3: 2, 4, 6, 7, 9, 13, 14, 18, 19-24, 27, 30, 31, 32.

---

Thursday, November 7, 2024

---

• We discussed Section 3.2 Properties of determinants. Suggested problems for Section 3.2: 5, 7, 9, 11, 16-20 (even), 21, 25, 31, 33, 34, 35, 40c

• Let \(m \in \mathbb{N}\). Let \(i,j \in \mathbb{N}\) be such that \(i \lt j \lt n\). In this item I want to prove that the determinant of the elementary matrix obtained from the identity matrix by exchanging the positions of \(i\)-th and \(j\)-th row equals to \(-1\).

  Below is a "click-by-click" proof of the fact that the determinant of the elementary matrix obtained from the identity matrix by exchanging two rows (or, equivalently two columns) equals to $-1.$ There are nine steps in this proof. I describe each step below.
  1. This is the determinant that we want to calculate.
  2. I emphasize that the $i$-th and $j$-th row in the identity matrix are interchanged.
  3. We will calculate the $n\!\times\!n$ determinant by cofactor expansion along the $i$-th row.
  4. Since the only nonzero entry in the $i$-th row is at the $j$-th place, the cofactor expansion equals $(-1)^{i+j}$ multiplied by an $(n-1)\times(n-1)$ determinant.
  5. We will calculate the $(n-1)\!\times\!(n-1)$ determinant using cofactor expansion along the $(j-1)$-st row. Notice that the only nonzero entry in the $(j-1)$-st row is $1$ which is at $i$-th position.
  6. The previous $(n-1)\!\times\!(n-1)$ determinant calculates to $(-1)^{i+j-1}$ multiplied by the $(n-2)\!\times\!(n-2)$ determinant of the identity matrix.
  7. The $(n-2)\!\times\!(n-2)$ determinant of the identity matrix calculates to $1$.
  8. $(-1)^{i+j}(-1)^{i+j-1} = (-1)^{2i+2j-1}$.
  9. $(-1)^{2i+2j-1} = -1$.

  All entries left blank in the determinant below are zeros.
  Click on the image for a step by step proof.

  
• A different way to prove that the determinant of the elementary matrix obtained from the identity matrix by exchanging two rows (or, equivalently two columns) equals to $-1$ is by first observing the following fact
  Let $n$ be a positive integer greater than $2$ and let $k\in\{1,2,\ldots,n\}.$ Let $A$ be an $n\!\times\!n$ matrix in which $k$-th row ($k$-th column, respectively) is the $k$-th row ($k$-th column, respectively) of the identity matrix $I_n.$ Then the determinant of $A$ equals to the determinant of the $(n-1)\!\times\!(n-1)$ matrix obtained from $A$ by removing its $k$-th row and its $k$-th column.

  This fact follows from the cofactor expansion calculation of a determinant, Theorem 1 in Section 3.1.

  • For example, with the third row being the third row of the identity matrix $I_4:$ \[ \require{bbox} \left| \begin{array}{cccc} a & b & \bbox[yellow]{c} & d \\ e & f & \bbox[yellow]{g} & h \\ \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{1} & \bbox[yellow]{0} \\ i & j & \bbox[yellow]{k} & l \end{array} \right| = (-1)^{3+3} \left| \begin{array}{ccc} a & b & d \\ e & f & h \\ i & j & l \end{array} \right| =\left| \begin{array}{ccc} a & b & d \\ e & f & h \\ i & j & l \end{array} \right|. \]
  • This procedure is repeatable as many times as many rows or columns of the identity matrix we have in a square matrix. That is
    Let $n$ be a positive integer greater than $2,$ $l \in \{1,2,\ldots,n\}$ and let $k_1,\ldots,k_l\in\{1,2,\ldots,n\}$ be distinct. Let $A$ be an $n\!\times\!n$ matrix with the following property:

    For every $j \in \{1,\ldots,l\}$ we have that $k_j$-th row is the $k_j$-th row of the identity matrix $I_n$ OR $k_j$-th column is the $k_j$-th column of the identity matrix $I_n$.

    Then the determinant of $A$ equals to the determinant of the $(n-l)\!\times\!(n-l)$ matrix obtained from $A$ by removing all of its rows with indexes $k_1,\ldots,k_l$ and all of its columns with indexes $k_1,\ldots,k_l.$
  • For example, with the third row being the third row of the identity matrix $I_6$ and the fifth column being the fifth column of the identity matrix $I_6:$ (here $l = 2$ and $k_1 = 3, k_2 = 5$) \begin{align*} \require{bbox} \left| \begin{array}{cccccc} a & b & \bbox[yellow]{c} & d & 0 & e \\ f & g & \bbox[yellow]{h} & i & 0 & j \\ \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{1} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0} \\ k & l & \bbox[yellow]{m} & n & 0 & o \\ p & q & \bbox[yellow]{r} & s & 1 & t \\ u & v & \bbox[yellow]{w} & x & 0 & y \end{array} \right| & = \left| \begin{array}{ccccc} a & b & d & 0 & e \\ f & g & i & 0 & j \\ k & l & n & 0 & o \\ p & q & s & 1 & t \\ u & v & x & 0 & y \end{array} \right| \\[6pt] & = \left| \begin{array}{ccccc} a & b & d & \bbox[yellow]{0} & e \\ f & g & i & \bbox[yellow]{0} & j \\ k & l & n & \bbox[yellow]{0} & o \\ \bbox[yellow]{p} & \bbox[yellow]{q} & \bbox[yellow]{s} & \bbox[yellow]{1} & \bbox[yellow]{t} \\ u & v & x & \bbox[yellow]{0} & y \end{array} \right| \\[6pt] & = \left| \begin{array}{cccc} a & b & d & e \\ f & g & i & j \\ k & l & n & o \\ u & v & x & y \end{array} \right|. \end{align*}
  Or, with the third row being the third row of the identity matrix $I_6$ and the second column being the second column of the identity matrix $I_6$ and the fifth column being the fifth column of the identity matrix $I_6:$ (here $l = 3$ and $k_1 = 2, k_2 = 3, k_3 = 5$) \begin{equation*} \require{bbox} \left| \begin{array}{cccccc} a & \bbox[yellow]{0} & \bbox[yellow]{b} & c & \bbox[yellow]{0} & d \\ \bbox[yellow]{e} & \bbox[yellow]{1} & \bbox[yellow]{f} & \bbox[yellow]{g} & \bbox[yellow]{0} & \bbox[yellow]{h} \\ \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{1} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0} \\ i & \bbox[yellow]{0} & \bbox[yellow]{j} & k & \bbox[yellow]{0} & l \\ \bbox[yellow]{m} & \bbox[yellow]{0} & \bbox[yellow]{n} & \bbox[yellow]{o} & \bbox[yellow]{1} & \bbox[yellow]{p} \\ q & \bbox[yellow]{0} & \bbox[yellow]{r} & s & \bbox[yellow]{0} & t \end{array} \right| = \left| \begin{array}{ccc} a & c & d \\ i & k & l \\ q & s & t \end{array} \right|. \end{equation*}
  • Let $n$ be a positive integer greater than $2$ and $i,j \in \{1,2,\ldots,n\}$ are such that $i \lt j.$ For the $n\!\times\!n$ elementary matrix in which $i$-th and $j$-th rows exchanged places in the identity matrix $I_n$, then all the rows and the columns with indexes different from $i$ and $j$ can be removed. Thus, the determinant of this elementary matrix equals \[ \left| \begin{array}{cc} 0 & 1 \\ 1 & 0 \end{array} \right| = -1. \]
  • The following \(7\times 7\) determinant is obtained from the identity matrix by exchanging the third and the fifth row. Below, at each step we remove a row and a column which comes from the corresponding identity matrix: (at the first step, the first row and column, then again the first row and column, then the second row and column, then again the second row and column, then, finally, the third row and column) \begin{align*} \require{bbox} \left| \begin{array}{ccccccc} \bbox[yellow]{1} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0} \\ \bbox[yellow]{0} & 1 & 0 & 0 & 0 & 0 & 0\\ \bbox[yellow]{0} & 0 & 0 & 0 & 0 & 1 & 0\\ \bbox[yellow]{0} & 0 & 0 & 1 & 0 & 0 & 0\\ \bbox[yellow]{0} & 0 & 0 & 0 & 1 & 0 & 0\\ \bbox[yellow]{0} & 0 & 1 & 0 & 0 & 0 & 0 \\ \bbox[yellow]{0} & 0 & 0 & 0 & 0 & 0 & 1 \end{array} \right| &= \left| \begin{array}{cccccc} \bbox[yellow]{1} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0}\\ \bbox[yellow]{0} & 0 & 0 & 0 & 1 & 0\\ \bbox[yellow]{0} & 0 & 1 & 0 & 0 & 0\\ \bbox[yellow]{0} & 0 & 0 & 1 & 0 & 0\\ \bbox[yellow]{0} & 1 & 0 & 0 & 0 & 0 \\ \bbox[yellow]{0} & 0 & 0 & 0 & 0 & 1 \end{array} \right| \\[6pt] &= \left| \begin{array}{ccccc} 0 & \bbox[yellow]{0} & 0 & 1 & 0\\ \bbox[yellow]{0} & \bbox[yellow]{1} & \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{0}\\ 0 & \bbox[yellow]{0} & 1 & 0 & 0\\ 1 & \bbox[yellow]{0} & 0 & 0 & 0 \\ 0 & \bbox[yellow]{0} & 0 & 0 & 1 \end{array} \right| \\[6pt] &= \left| \begin{array}{cccc} 0 & \bbox[yellow]{0} & 1 & 0\\ \bbox[yellow]{0} & \bbox[yellow]{1} & \bbox[yellow]{0} & \bbox[yellow]{0}\\ 1 & \bbox[yellow]{0} & 0 & 0 \\ 0 & \bbox[yellow]{0} & 0 & 1 \end{array} \right| \\[6pt] &= \left| \begin{array}{cccc} 0 & 1 & \bbox[yellow]{0}\\ 1 & 0 & \bbox[yellow]{0} \\ \bbox[yellow]{0} & \bbox[yellow]{0} & \bbox[yellow]{1} \end{array} \right| \\[6pt] & = \left| \begin{array}{cc} 0 & 1 \\ 1 & 0 \end{array} \right|\\[6pt] & = -1. \end{align*}
• Look at Wikipedia's Matrix page. It contains some stuff that we did and much that we didn't do.

---

Tuesday, November 5, 2024

---

• We did Section 3.1 today. Suggested problems for Section 3.1: 1, 3, 5, 9, 11, 13, 17, 20, 21, 22, 25-32, 37, 41, 42

---

Monday, November 4, 2024

---

• Today in class we explored a problem inspired by the picture below. No numbers are given just the picture. In the picture below we are given two bases of $\mathbb{R}^2$, one blue and one purple: \[ \color{blue}{\mathcal B} = \bigl\{ \color{blue}{\mathbf{b}_1}, \color{blue}{\mathbf{b}_2} \bigr\}, \quad \color{purple}{\mathcal C} = \bigl\{ \color{purple}{\mathbf{c}_1}, \color{purple}{\mathbf{c}_2} \bigr\} \] For each basis a coordinate grid is shown in the corresponding color. The coordinate grids are drown in the increments of 1/10 with the multiples of 1/2 emphasized with slightly thicker lines. Based on the information provided in the picture give good estimates for the change of coordinates matrices: \[ \underset{\color{purple}{\mathcal C}\leftarrow\color{blue}{\mathcal{B}}}{P}, \qquad \underset{\color{blue}{\mathcal{B}}\leftarrow\color{purple}{\mathcal C}}{P}. \] Why are the red points in the picture exceptional? How you can use the red points to verify whether your change of coordinates matrices are correct?
• Suggested exercises for Section 4.7: Change of Basis are 2, 3, 4, 6, 8, 9, 11, 12, 19, 20.
• A brief review of the Change of Coordinates Matrix (this is my preferred name) follows. Let $m, n \in \mathbb{N}$ and $m\leq n$. Let $\mathcal{H}$ be a subspace of $\mathbb{R}^n$ and let \[ \mathcal{A} = \bigl\{\mathbf{a}_1,\ldots,\mathbf{a}_m\bigr\} \] and \[ \mathcal{B} = \bigl\{\mathbf{b}_1,\ldots,\mathbf{b}_m\bigr\} \] be two bases of $\mathcal{H}.$ By definition of a basis this implies that \[ \mathcal{H} = \operatorname{Span}\bigl\{\mathbf{a}_1,\ldots,\mathbf{a}_m\bigr\} = \operatorname{Span}\bigl\{\mathbf{b}_1,\ldots,\mathbf{b}_m\bigr\} \] and both \[ \mathcal{A} = \bigl\{\mathbf{a}_1,\ldots,\mathbf{a}_m\bigr\} \quad \text{and} \quad \mathcal{B} = \bigl\{\mathbf{b}_1,\ldots,\mathbf{b}_m\bigr\} \] are linearly independent. We proved in class that the change of coordinates matrix $\displaystyle\underset{\mathcal{B}\leftarrow\mathcal{A}}{P}$ is given by \[ \underset{\mathcal{B}\leftarrow\mathcal{A}}{P} = \Bigl[ \bigl[\mathbf{a}_1\bigr]_{\mathcal{B}} \ \cdots \ \bigl[ \mathbf{a}_m\bigr]_{\mathcal{B}} \Bigr] \] and analogously \[ \underset{\mathcal{A}\leftarrow\mathcal{B}}{P} = \Bigl[ \bigl[\mathbf{b}_1\bigr]_{\mathcal{A}} \ \cdots \ \bigl[ \mathbf{b}_m\bigr]_{\mathcal{A}} \Bigr]. \] But, how to calculate \[ \bigl[\mathbf{a}_1\bigr]_{\mathcal{B}}, \ldots,\bigl[ \mathbf{a}_m\bigr]_{\mathcal{B}}? \] Let us look at \[ \bigl[\mathbf{a}_1\bigr]_{\mathcal{B}} = \left[\!\begin{array}{c} x_1 \\ \vdots \\ x_m \end{array}\!\right]. \] To find the real numbers $x_1, \ldots, x_m$ we have to solve the nonhomogeneous vector equation \[ x_1 \mathbf{b}_1 + x_2 \mathbf{b}_2 + \cdots + x_m \mathbf{b}_m = \mathbf{a}_1. \] To solve the preceding equation we row reduce \[ \Bigl[\!\begin{array}{cccc|c} \mathbf{b}_1 & \mathbf{b}_2 & \cdots & \mathbf{b}_m & \mathbf{a}_1\end{array}\!\Bigr]. \] Since the vectors $\mathbf{b}_1, \mathbf{b}_2, \ldots, \mathbf{b}_m$ are linearly independent, the Reduced Row Echelon Form of the preceding augmented matrix has the following form \[ \left[\!\begin{array}{cccc|c} 1 & 0 & \cdots & 0 & \text{the solution for} \ x_1 \\ 0 & 1 & \cdots & 0 & \text{the solution for} \ x_2 \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & \cdots & 1 & \text{the solution for} \ x_m \\ 0 & 0 & \cdots & 0 & 0 \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & \cdots & 0 & 0 \end{array}\!\right]. \] Notice that in the preceding matrix the bottom zero rows are present only in the case when $n \gt m.$ If $n \gt m$, then there are exactly $n-m$ rows of zeros. Also notice that the above system must be consistent since the vector $\mathbf{a}_1$ is in the span of the vectors $\mathbf{b}_1, \mathbf{b}_2, \ldots, \mathbf{b}_m.$

  ---

  To solve the nonhomogeneous vector equations \[ x_1 \mathbf{b}_1 + x_2 \mathbf{b}_2 + \cdots + x_m \mathbf{b}_m = \mathbf{a}_2, \quad \ldots, \quad x_1 \mathbf{b}_1 + x_2 \mathbf{b}_2 + \cdots + x_m \mathbf{b}_m = \mathbf{a}_m, \] we just build the bigger augmented matrix: \[ \Bigl[\!\begin{array}{cccc|cccc} \mathbf{b}_1 & \mathbf{b}_2 & \cdots & \mathbf{b}_m & \mathbf{a}_1 & \mathbf{a}_2 & \cdots & \mathbf{a}_m \end{array}\!\Bigr]. \] Since the vectors $\mathbf{b}_1, \ldots, \mathbf{b}_m$ are linearly independent, the RREF off the matrix whose columns are the vectors of $\mathcal{B}$ consists of the identity matrix $I_m$ and $n-m$ zero rows at the bottom if $n\gt m.$ Therefore \[ \Bigl[\!\begin{array}{ccc|ccc} \mathbf{b}_1 & \cdots & \mathbf{b}_m & \mathbf{a}_1 & \cdots & \mathbf{a}_m \end{array}\!\Bigr] \sim \cdots \sim \left[\! \begin{array}{c|c} I_m & \underset{\mathcal{B}\leftarrow\mathcal{A}}{P} \\ 0 & 0 \end{array} \!\right]. \] In the preceding RREF, the zero matrices at the bottom are present only if $n-m \gt 0.$ Then, if $n-m \gt 0,$ these matrices are of the size $(n-m)\!\times\!m;$ they both have $m$ columns and $n-m$ rows consisting of zeros.
• In the next example we are given two bases of a two-dimensional subspace of $\mathbb{R}^4$ and we are asked to find a change of coordinate matrices between these two bases: \[ \mathcal{H} = \operatorname{Span}\left\{\left[\!\begin{array}{c} 1 \\ 2 \\ 1 \\ 3 \end{array}\!\right], \left[\!\begin{array}{c} 2 \\ 3 \\ 1 \\ 5 \end{array}\!\right]\right\} = \operatorname{Span}\left\{\left[\!\begin{array}{c} 2 \\ 5 \\ 3 \\ 7 \end{array}\!\right], \left[\!\begin{array}{r} 1 \\ 0 \\ -1 \\ 1 \end{array}\!\right]\right\}. \] Set \[ \mathcal{A} = \left\{\left[\!\begin{array}{c} 1 \\ 2 \\ 1 \\ 3 \end{array}\!\right], \left[\!\begin{array}{c} 2 \\ 3 \\ 1 \\ 5 \end{array}\!\right]\right\}, \qquad \mathcal{B} = \left\{\left[\!\begin{array}{c} 2 \\ 5 \\ 3 \\ 7 \end{array}\!\right], \left[\!\begin{array}{r} 1 \\ 0 \\ -1 \\ 1 \end{array}\!\right]\right\}. \] To calculate $\displaystyle\underset{\mathcal{B}\leftarrow\mathcal{A}}{P}$ we need to row reduce the matrix \[ \left[\!\begin{array}{cr|cc} 2 & 1 & 1 & 2 \\ 5 & 0 & 2 & 3 \\ 3 & -1 & 1 & 1 \\ 7 & 1 & 3 & 5 \end{array}\!\right] \] The RREF of the preceding matrix will certainly include fractions. Therefore we rather find $\displaystyle\underset{\mathcal{A}\leftarrow\mathcal{B}}{P}$ for which we need to row reduce (without fractions) \[ \left[\!\begin{array}{cc|cr} 1 & 2 & 2 & 1 \\ 2 & 3 & 5 & 0 \\ 1 & 1 & 3 & -1 \\ 3 & 5 & 7 & 1 \end{array}\!\right] \sim \cdots \sim \left[\!\begin{array}{cc|rr} 1 & 0 & 4 & -3 \\ 0 & 1 & -1 & 2 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array}\!\right]. \] Hence \[ \underset{\mathcal{A}\leftarrow\mathcal{B}}{P} = \left[\!\begin{array}{rr} 4 & -3 \\ -1 & 2 \end{array}\!\right]. \] Let us verify this calculation. Is it true that: \[ \left[\!\begin{array}{c} 2 \\ 5 \\ 3 \\ 7 \end{array}\!\right] = (4) \left[\!\begin{array}{c} 1 \\ 2 \\ 1 \\ 3 \end{array}\!\right] + (-1)\left[\!\begin{array}{c} 2 \\ 3 \\ 1 \\ 5 \end{array}\!\right], \qquad \left[\!\begin{array}{r} 1 \\ 0 \\ -1 \\ 1 \end{array}\!\right] = (-3) \left[\!\begin{array}{c} 1 \\ 2 \\ 1 \\ 3 \end{array}\!\right] + (2) \left[\!\begin{array}{c} 2 \\ 3 \\ 1 \\ 5 \end{array}\!\right]? \] Yes. Therefore the following equalities are correct: \[ \bigl[ \mathbf{b}_1\bigr]_{\mathcal{A}} = \left[\!\begin{array}{r} 4 \\ -1 \end{array}\!\right], \qquad \bigl[ \mathbf{b}_2\bigr]_{\mathcal{A}} = \left[\!\begin{array}{r} -3 \\ 2 \end{array}\!\right]. \] Hence, $\displaystyle\underset{\mathcal{A}\leftarrow\mathcal{B}}{P}$ is correct.
  
  We have \[ \underset{\mathcal{B}\leftarrow\mathcal{A}}{P} = \left(\underset{\mathcal{A}\leftarrow\mathcal{B}}{P}\right)^{-1} = \frac{1}{5} \left[\!\begin{array}{rr} 2 & 3 \\ 1 & 4 \end{array}\!\right]. \] Verify this: \[ \left[\!\begin{array}{c} 1 \\ 2 \\ 1 \\ 3 \end{array}\!\right] = \frac{2}{5}\left[\!\begin{array}{c} 2 \\ 5 \\ 3 \\ 7 \end{array}\!\right] + \frac{1}{5} \left[\!\begin{array}{r} 1 \\ 0 \\ -1 \\ 1 \end{array}\!\right], \qquad \left[\!\begin{array}{c} 2 \\ 3 \\ 1 \\ 5 \end{array}\!\right] = \frac{3}{5} \left[\!\begin{array}{c} 2 \\ 5 \\ 3 \\ 7 \end{array}\!\right] + \frac{4}{5} \left[\!\begin{array}{r} 1 \\ 0 \\ -1 \\ 1 \end{array}\!\right]. \] True. Therefore the following equalities are correct: \[ \bigl[ \mathbf{a}_1\bigr]_{\mathcal{B}} = \frac{1}{5}\left[\!\begin{array}{r} 2 \\ 1 \end{array}\!\right], \qquad \bigl[ \mathbf{a}_2\bigr]_{\mathcal{B}} = \frac{1}{5} \left[\!\begin{array}{r} 3 \\ 4 \end{array}\!\right]. \] Hence, $\displaystyle\underset{\mathcal{B}\leftarrow\mathcal{A}}{P}$ is correct.

---

Saturday, November 2, 2024 (updated)

---

• Today, I dedicate this post to what I consider to be a fundamental theorem in linear algebra—one that, surprisingly, the author of our textbook doesn’t even present as a theorem. Here is the relevant snippet from the book. I will use this snippet to highlight some differences in notation that I use in class and on this website. I denote subspaces using script letters; thus, instead of \(H\) as used in the textbook, I write \(\mathcal{H}\) to denote a subspace. Additionally, for the vector \(\mathbf{0}\), I always indicate the space it belongs to: \[ \mathbf{0}_2 = \left[\! \begin{array}{c} 0 \\ 0 \end{array} \!\right] \in \mathbb{R}^2, \quad \mathbf{0}_3 = \left[\! \begin{array}{c} 0 \\ 0 \\ 0 \end{array} \!\right] \in \mathbb{R}^3, \quad \mathbf{0}_4 = \left[\! \begin{array}{c} 0 \\ 0 \\ 0 \\ 0 \end{array} \!\right] \in \mathbb{R}^4. \] In general, I denote the zero vector in \(\mathbb{R}^n\) by \(\mathbf{0}_n\).

• Today, I will share several important theorems and aim to explain each proof in full detail. I view mathematical proofs as consisting of three parts: the claim, the background knowledge needed, and the proof itself, which I usually break down into several steps.

  As I mentioned earlier, each theorem is an implication that can be summarized as: "If \( p \), then \(q\)," where \( p \) represents the assumptions of the theorem and \(q\) represents the conclusion. To help clarify the structure of proofs, I often color-code the assumptions and relevant background facts in green, a color symbolizing approachability and friendliness, and I color the conclusion in red. This color scheme highlights the initial mystery in the connection between the assumptions (green) and the conclusion (red), emphasizing that the proofer’s task is to bridge this gap by constructing a logical path. In this language of colors, the proofer’s task is to create a green path of logical steps, starting with the assumptions, using the background knowledge as individual stepping stones, and ultimately reaching the red conclusion. In a way, the goal of the proof is to "greenify" the red, completing the logical connection.

• Theorem below is presented in Exercise 27 and Exercise 28 in Section 2.9 (page 161) in the textbook. My presentation is different, but the content is the same. In fact, I prove the contrapositives of the statements presented in these exercises.

  Theorem 1. Let $n, p, q \in \mathbb{N}.$ Let $F$ be an $n\!\times\!p$ matrix and let $G$ be an $n\!\times\!q$ matrix. If \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\operatorname{Col}(F) \subseteq \operatorname{Col}(G)} \quad \text{and} \quad \bbox[lightgreen, 6px, border:3px solid green]{\text{the columns of} \ \ F \ \ \text{are linearly independent}}, \end{equation*} then \begin{equation*} \bbox[#FFC0C0, 6px, border:3px solid red]{p \leq q}. \end{equation*}

  Background Knowledge is as follows.

  BK0. The definition of matrix-vector multiplication, the definition of matrix multiplication.

  BK1. Let \(\mathbb{y} \in \mathbb{R}^n\). Then \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\mathbb{y} \in \operatorname{Col}(G) \quad \text{if and only if} \quad \text{there exists} \ \ \mathbb{x} \in \mathbb{R}^q \ \ \text{such that} \ \ \mathbb{y} = G \mathbb{x}}. \end{equation*}

  BK2. Let \(H\) be an \(q\times p\) matrix and let \(\mathbb{x} \in \mathbb{R}^p\). Then \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{ \begin{array}{c} \text{the columns of} \ \ H \ \ \text{are linearly independent} \quad \\ \quad\quad\quad\quad \quad \text{if and only if}\quad \quad H\mathbf{x} = \mathbf{0}_q \ \Rightarrow \ \mathbf{x} = \mathbf{0}_p. \end{array}} \end{equation*} Notice that the implication \(H\mathbf{x} = \mathbf{0}_q \ \Rightarrow \ \mathbf{x} = \mathbf{0}_p\) is stated in English as: The homogeneous equation \(H\mathbf{x} = \mathbf{0}_q\) has only the trivial solution.

  BK3. Let \(H\) be an \(q\times p\) matrix. The following implication holds: \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{ \text{If the columns of} \ H \ \text{are linearly independent, then} \ \ p \leq q. } \end{equation*} In English, this implication can be stated as: If the columns of a matrix are linearly independent, then the number of rows is greater than or equal to the number of columns.

  Proof. Let \(\mathbf{f}_1,\ldots, \mathbf{f}_p \in \mathbb{R}^n\) be the columns of \(F\) and let \(\mathbf{g}_1,\ldots, \mathbf{g}_q \in \mathbb{R}^n\) be the columns of \(G\). That is \[ F = \bigl[ \mathbf{f}_1 \ \cdots \ \mathbf{f}_p \bigr], \qquad G = \bigl[ \mathbf{g}_1 \ \cdots \ \mathbf{g}_q \bigr]. \]

  Step 1. Since for every \(j\in\{1,\ldots,p\}\) we have \(\mathbf{f}_j \in \operatorname{Col}(F)\), the assumption \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\operatorname{Col}(F) \subseteq \operatorname{Col}(G)} \end{equation*} implies that \(\mathbf{f}_j \in \operatorname{Col}(G)\). Since for every \(j\in\{1,\ldots,p\}\) we have \(\mathbf{f}_j \in \operatorname{Col}(G)\), by background knowledge BK1 we conclude that for every \(j\in\{1,\ldots,p\}\) there exists \(\mathbf{h}_j \in \mathbb{R}^q\) such that \(\mathbf{f}_j = G\mathbf{h}_j\). Set \[ H = \bigl[ \mathbf{h}_1 \ \cdots \ \mathbf{h}_p \bigr]. \] Then \(H\) is \(q\times p\) matrix and by background knowledge BK0 we have \begin{align*} GH & = G\bigl[ \mathbf{h}_1 \ \cdots \ \mathbf{h}_p \bigr] \\ & = \bigl[ G\mathbf{h}_1 \ \cdots \ G \mathbf{h}_p \bigr] \\ & = \bigl[ \mathbf{f}_1 \ \cdots \ \mathbf{f}_p \bigr] \\ & = F. \end{align*} Thus, there exists an \(q\times p\) matrix \(H\) such that \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{F = GH}. \end{equation*}

  Step 2. In this step we will prove that the columns of \(H\) are linearly independent. For that proof we use background knowledge BK2. We will prove that for \(\mathbf{x} \in \mathbb{R}^p\) the following implication holds: \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{H\mathbf{x} = \mathbf{0}_q} \quad \Rightarrow \quad \bbox[#FFC0C0, 6px, border:3px solid red]{\mathbf{x} = \mathbf{0}_p}. \end{equation*} Here is the proof of this implication. (Notice how I start from green and use only known green stuff to arrive at red.) Assume \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{H\mathbf{x} = \mathbf{0}_q}. \end{equation*} Apply matrix \(G\) to both sides of the preceding green equality to get \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{GH\mathbf{x} = G\mathbf{0}_q}. \end{equation*} By Step 1 we have \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{F = GH}. \end{equation*} and by BK0 we have \(G\mathbf{0}_q = \mathbf{0}_n\). Hence \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{F\mathbf{x} = \mathbf{0}_n}. \end{equation*} By assumption \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\text{the columns of} \ \ F \ \ \text{are linearly independent}}. \end{equation*} By background knowledge BK2 we deduce that \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{F\mathbf{x} = \mathbf{0}_n \quad \Rightarrow \quad \mathbf{x} = \mathbf{0}_p}. \end{equation*} In conclusion, the implication \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{H\mathbf{x} = \mathbf{0}_q} \quad \Rightarrow \quad \bbox[#FFC0C0, 6px, border:3px solid red]{\mathbf{x} = \mathbf{0}_p} \end{equation*} is proved. By background knowledge BK2 the last implication yields that \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\text{the columns of} \ \ H \ \ \text{are linearly independent}}. \end{equation*}

  Step 3. By the final result of Step 2 \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\text{the columns of} \ \ H \ \ \text{are linearly independent}}. \end{equation*} and by background knowledge BK3 \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{ \text{If the columns of} \ H \ \text{are linearly independent, then} \ \ p \leq q. } \end{equation*} we conclude \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{p \leq q}. \end{equation*}

  This completes the proof.

• In this item I state Theorem 1 in English without using a single mathematical symbol.

  Theorem 1. If two matrices have the same column space and one of them has linearly independent columns, then the matrix with linearly independent columns has no more columns than the other matrix. In particular, if two matrices have the same column space and both have linearly independent columns, then the matrices have the same size.

• In this item I prove the fundamental theorem in linear algebra. First I state the theorem in English:

  Theorem 2.  With \(n\in\mathbb{N}\), any two bases of a subspace of \(\mathbb{R}^n\) have the same number of elements.

  To proceed with a proof, it is essential to state the theorem in full mathematical detail. Note that I have formulated the theorem as an implication.

  Theorem 2  Let $n, m, k \in \mathbb{N},$ and let $\mathcal{H}$ be a subspace of $\mathbb{R}^n$ such that \(\mathcal{H} \neq \{\mathbf{0}_n\}\). If the following two assumptions are satisfied:

   A1.  Vectors $\mathbf{a}_1, \ldots, \mathbf{a}_k$ form a basis for $\mathcal{H}$, (notice that there are $k$ vectors in this basis),

   A2.  Vectors $\mathbf{b}_1, \ldots, \mathbf{b}_m$ form a basis for $\mathcal{H}$, (notice that there are $m$ vectors in this basis),

  then the following claim is true:

   C.  $m=k$.

  Background Knowledge is as follows.

  BK0. The definition of a basis for a subspace, the definition of a column space.

  BK1. Theorem 1.

  Proof. Introduce an \(n\times k\) matrix \(A\) whose columns are $\mathbf{a}_1, \ldots, \mathbf{a}_k$ and \(n\times m\) matrix \(B\) whose columns are $\mathbf{b}_1, \ldots, \mathbf{b}_m$. That is \[ A = \bigl[ \mathbf{a}_1 \ \cdots \ \mathbf{a}_k \bigr], \qquad B = \bigl[ \mathbf{b}_1 \ \cdots \ \mathbf{b}_m \bigr]. \]

  Step 1. By background knowledge BK0 we have that \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\mathcal{H} = \operatorname{Col}(A) = \operatorname{Col}(B)}, \end{equation*} \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\text{the columns of} \ \ A \ \ \text{are linearly independent}}, \end{equation*} and \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\text{the columns of} \ \ B \ \ \text{are linearly independent}}. \end{equation*}

  Step 2. In this step we apply BK1, that is Theorem 1 to the matrix \(A\) in the role of \(F\) and \(B\) in the role of \(G\). Since by Step 1 we have \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\operatorname{Col}(A) = \operatorname{Col}(B)} \quad \text{and} \quad \bbox[lightgreen, 6px, border:3px solid green]{\text{the columns of} \ \ A \ \ \text{are linearly independent}}, \end{equation*} we deduce \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{k \leq m}. \end{equation*}

  Step 3. In this step we apply BK1, that is Theorem 1 to the matrix \(B\) in the role of \(F\) and \(A\) in the role of \(G\). Since by Step 1 we have \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{\operatorname{Col}(B) = \operatorname{Col}(A)} \quad \text{and} \quad \bbox[lightgreen, 6px, border:3px solid green]{\text{the columns of} \ \ B \ \ \text{are linearly independent}}, \end{equation*} we deduce \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{m \leq k}. \end{equation*}

  Step 4. In Step 2 and Step 3 we proved \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{k \leq m} \quad \text{and} \quad \bbox[lightgreen, 6px, border:3px solid green]{m \leq k}. \end{equation*} Consequently \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{k = m}. \end{equation*}

  This completes the proof.

---

Thursday, October 31, 2024

---

• Let $m, n \in \mathbb{N}.$ Let $A$ be an $n\!\times\!m$ matrix. The matrix $A$ has $n$ rows and $m$ columns. The columns of $A$ are vectors in $\mathbb{R}^n.$ The transpose of $A$, denoted by \(A^\top\), is an \(m\!\times\!n\) matrix. The matrix \(A^\top\) has $m$ rows and $n$ columns. The columns of \(A^\top\) are vectors in \(\mathbb{R}^m\). The columns of $A^\top$ have the identical entries as the rows of \(A\), just instead of being in rows they are written as columns. Two fundamental subspaces associated with the matrix $A$ are \begin{alignat*}{2} \operatorname{Col}(A) & = \operatorname{Row}(A^\top) \quad & & \left\{ \begin{array}{l} \text{this space is the span of the columns of} \ \ A; \\ \text{since each column of $A$ is a vector in} \ \mathbb{R}^n, \\ \text{this space is a subspace of} \ \ \mathbb{R}^n \end{array} \right. \\[10pt] \operatorname{Row}(A) &= \operatorname{Col}(A^\top) \quad & & \left\{ \begin{array}{l} \text{this space is the span of the columns of} \ A^\top; \\ \text{since each column of $A^\top$ is a vector in} \ \mathbb{R}^m, \\ \text{this space is a subspace of} \ \ \mathbb{R}^m \end{array} \right. \\[10pt] \operatorname{Nul}(A) & = \bigl\{ \mathbf{x} \in \mathbb{R}^m \, : \, & & A\mathbf{x} = \mathbf{0}_n \bigr\} \quad \text{this space is a subspace of} \ \ \mathbb{R}^m, \\[10pt] \operatorname{Nul}(A^\top) & = \bigl\{ \mathbf{y} \in \mathbb{R}^n \, : \, & & A^\top\mathbf{y} = \mathbf{0}_m \bigr\} \quad \text{this space is a subspace of} \ \ \mathbb{R}^n \end{alignat*} Thus \begin{equation*} \bbox[lightgreen, 6px, border:3px solid green]{ \begin{array}{cll} \operatorname{Row}(A) = \operatorname{Col}(A^\top) \quad & \text{and} \quad \operatorname{Nul}(A) \quad & \text{are subspaces of} \quad \mathbb{R}^m\\[3pt] \operatorname{Col}(A) = \operatorname{Row}(A^\top) \quad & \text{and} \quad \operatorname{Nul}(A^\top) \quad & \text{are subspaces of} \quad \mathbb{R}^n \end{array} } \end{equation*}

• The concept of column space of a matrix is introduced inn Section 2.8. The concept of row space of a matrix is introduced in Section 4.6 in subsection entitled The Row Space. You can learn how to find a basis of the row space in An Ode to Reduced Row Echelon Form.

  The rank theorem is covered both in Section 2.9 in subsection Dimension of a Subspace and in subsection The Rank Theorem in Section 4.6. Read both.

  Suggested problems related to $\operatorname{Col}(A)$ and $\operatorname{Nul}(A)$ of a given matrix $A$ from Section 4.5 are: 3, 6, 12, 13, 15, 18.

  More problems about $\operatorname{Col}(A)$, $\operatorname{Row}(A)$ and $\operatorname{Nul}(A)$ of a given matrix $A$ are in Section 4.6: 3, 4, 5, 6, 7, 8, 9, 11, 13, 15, 17, 18, 27, 28, 29.

  Suggested problems for Section 2.9: 1, 2, 3, 5, 6, 7, 9, 11, 12, 14, 15, 19, 20, 21, 23, 24, and in particular 27 and 28.

• All relevant properties of a column space its basis and dimension, of a row space, its basis and dimension, and of a null space its basis and dimension, you can find in my
  An Ode to Reduced Row Echelon Form
• Below I show two examples of coordinate systems.
• In the picture below we show a dark green basis in $\mathbb{R}^2$ and the corresponding coordinate system.
• In the picture below we show two dark green vectors in $\mathbb{R}^3$. These vectors span a two-dimensional subspace of $\mathbb{R}^3.$ These two dark green vectors form a basis of this subspace. In the picture below I show the corresponding coordinate system. In the picture above, I used the vectors \[ \left[\begin{array}{c} 1 \\ -3 \\ 1 \end{array}\right], \qquad \left[\begin{array}{c} -4 \\ 1 \\ 2 \end{array}\right]. \]

---

Tuesday, October 29, 2024

---

• Today we started Section 2.8 Subspaces of $\mathbb{R}^n.$ Suggested problems for Section 2.8: 5, 6, 8, 9, 10, 11-20, 24, 25, 26, 30, 31-36.

  Pay attention to the definitions of

  • subspace of \(\mathbb{R}^n\), page 148.
  • column space of a matrix, page 149.
  • row space of a matrix, page 233 in Section 4.6.
  • basis for a subspace \(\mathcal{H}\) of \(\mathbb{R}^n\), page 150.

• Let me summarize the concepts of column space and row space.

  Let $m, n \in \mathbb{N}.$ Let $A$ be an $n\!\times\!m$ matrix. The matrix $A$ has $n$ rows and $m$ columns. The columns of $A$ are vectors in $\mathbb{R}^n.$ The transpose of $A$, denoted by \(A^\top\), is an \(m\!\times\!n\) matrix. The matrix \(A^\top\) has $m$ rows and $n$ columns. The columns of \(A^\top\) are vectors in \(\mathbb{R}^m\). The columns of $A^\top$ have the identical entries as the rows of \(A\), just instead of being in rows they are written as columns. Two fundamental subspaces associated with the matrix $A$ are \begin{alignat*}{2} \operatorname{Col}(A) & = \operatorname{Row}(A^\top) \quad & & \left\{ \begin{array}{l} \text{this space is the span of the columns of} \ \ A \\ \text{this space is a subspace of} \ \ \mathbb{R}^n, \\ \text{since each column of $A$ is a vector in} \ \mathbb{R}^n \end{array} \right. \\ \operatorname{Row}(A) &= \operatorname{Col}(A^\top) \quad & & \left\{ \begin{array}{l} \text{this space is the span of the columns of} \ A^\top \\ \text{this space is a subspace of} \ \ \mathbb{R}^m, \\ \text{since each column of $A^\top$ is a vector in} \ \mathbb{R}^m \end{array} \right. \end{alignat*}

• Important theorems related to these concepts are:

  Theorem 13.(page 152) Let $m$ and $n$ be positive integers, and let $A$ be an $n\!\times\!m$ matrix. The column space of the matrix \(A\), denoted by \(\operatorname{Col}(A)\), is a subspace of \(\mathbb{R}^n\). The pivot columns of the matrix \(A\) form a basis for the column space of \(A\).

  Theorem 13.(page 233) Let $m$ and $n$ be positive integers, and let $A$ be an $n\!\times\!m$ matrix. The row spaces of the matrix \(A\), denoted by \(\operatorname{Row}(A)\), is a subspace of \(\mathbb{R}^m\). The row spaces of the matrix \(A\) and the row space of its Reduced Row Echelon Form are the same. The nonzero rows of the Reduced Row Echelon Form of \(A\) form a basis for the row space of \(A\).

• You will understand the above theorems better, after reading my writing about Reduced Row Echelon Form:
  An Ode to Reduced Row Echelon Form

---

Monday, October 28, 2024

---

• Today we finished Section 2.2: The inverse of a matrix; suggested problems are: 1, 4, 5, 7, 12, 13, 21, 22, 23, 24, 28, 32, 33, 34, 38. In all bold face problems the textbook asks you to "explain why" something is true. In the text of the book and in class we offer proofs. Please try to organize your solutions as proofs, not explanations. State clearly what is known from theorems in the book and use logical reasoning to deduce what the problem claims.
• The most interesting implication in the Invertibel Matrix Theorem is (b)$\Rightarrow$(a). In that implication we need to prove:
  Let $n\in\mathbb{N}$ and let $A$ be an $n\!\times\!n$ matrix. If the RREF of $A$ is $I_n$, then $A$ is invertible.

  I posted a proof of this implication on Saturday. I post it again because of its importance and since I think that the proof here is better than the proof given in the textbook; see Theorem 7.

  The proof presented below uses only two facts: elementary matrices are invertible and matrix multiplication is associative. I also like that the proof below proves the invertibility by using the definition of invertibility. The proof below was suggested to me by a student during Fall Quarter 2021. Unfortunately I forgot who it was. In any case, please think on your own about each proof. You can come up with new proofs.

  In the proof below we need to prove that a matrix is invertible. For that it is useful to recall the definition of invertibility.

  Definition.  Let $n\in \mathbb{N}$ and let $I_n$ be the $n\!\times\!n$ identity matrix. An $n\!\times\!n$ matrix $A$ is said to be invertible if there exists an $n\!\times\!n$ matrix $C$ such that \[ CA = I_n \quad \text{and} \quad AC = I_n. \]
  Proof.

  • Step 1.  Let $n\in\mathbb{N}$ and let $A$ be an $n\!\times\!n$ matrix. Assume that the Reduced Row Echelon Form of $A$ is $I_n.$ This means that there exists a positive integer $p$ and $p$ row operators that row reduce $A$ to $I_n.$ Since each row operation can be represented by matrix multiplication from the left by an elementary matrix, there exist elementary matrices $E_1, E_2,\ldots, E_p$ such that the row reduction of the matrix $A$ looks like \[ A \sim E_1 A \sim E_2(E_1 A) \sim \cdots \sim E_p(E_{p-1} \cdots E_1A) = I_n. \] Since matrix multiplication is associative we have \[ (E_p \cdots E_1) A = I_n. \] Thus, we can set $C = E_p \cdots E_1$ in the definition of an invertible matrix and we have $CA = I_n.$
  • Step 2.  Since the matrices $E_1, E_2, \ldots, E_{p-1}, E_p$ are elementary matrices, they are invertible. Therefore we have the matrices $(E_1)^{-1}, (E_2)^{-1}, \ldots, (E_{p-1})^{-1}, (E_p)^{-1}.$ Recall the equality \[ (E_p \cdots E_1) A = I_n. \] from Step 1. Now we multiply the preceding equality from the left by $(E_p)^{-1}$ to get \[ (E_p)^{-1}(E_p E_{p-1} \cdots E_2 E_1) A = (E_p)^{-1} I_n. \] Since matrix multiplication is associative we get \[ (E_{p-1} \cdots E_2 E_1) A = (E_p)^{-1}. \] Next we keep multiplying the preceding equality consecutively by $(E_{p-1})^{-1}, \ldots, (E_2)^{-1}$ to get \[ E_1 A =(E_2)^{-1} \cdots (E_{p-1})^{-1} (E_p)^{-1}. \] Finally we multiply the preceding equality by $(E_1)^{-1}$ to get \[ A =(E_1)^{-1} (E_2)^{-1} \cdots (E_{p-1})^{-1} (E_p)^{-1}. \] Now we multiply the preceding equality from the right with $p$ matrices $E_p, E_{p-1},\ldots, E_2, E_1$ to get \[ A (E_p E_{p-1} \cdots E_2 E_1) = I_n. \] Recall that in Step 1 we set $C = E_p E_{p-1} \cdots E_2 E_1.$ Thus, the last displayed equality reads $AC = I_n.$ To summarize, we have proved that the following two matrix equalities hold \[ C A = I_n \quad \text{and} \quad A C = I_n. \] Therefore, by the definition of invertible matrix $A$ is invertible.
  • QED. 
• To fully understand the proof above, it is a good idea to work out a complete example.
• Consider the following problem:
  • Determine whether the matrix $A = \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 2 & 2 & 1 \end{array}\right]$ is invertible.
  • If $A$ is invertible find its inverse $A^{-1}$.
• The first step is to row reduce $A$ to RREF. \begin{align*} \left[\!\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 2 & 2 & 1 \end{array}\right] & \sim \left[\!\begin{array}{rrr} 1 & 1 & 0 \\ -1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \\ \end{align*}
• To understand the proof of the implication
  Let $n\in\mathbb{N}$ and let $A$ be an $n\!\times\!n$ matrix. If the RREF of $A$ is $I_n$, then $A$ is invertible.
  we need to understand the row reduction algorithm in the language of matrix multiplication with elementary matrices.
• Each step in a row reduction can be achieved by multiplication by an elementary matrix.

  Step	the row operation	the elementary matrix	the inverse of elementary matrix
  1st	The second row is replaced by the the sum of the second row and the third row multiplied by (-1)	$E_1 = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & -1 \\ 0 & 0 & 1 \end{array}\right]$	$E_1^{-1} = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{array}\right]$
  2nd	The second row scaled (multiplied) by (-1)	$E_2 = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{array}\right]$	$E_2^{-1} = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{array}\right]$
  3rd	The third row is replaced by the the sum of the third row and the first row multiplied by $(-2)$	$E_3 = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -2 & 0 & 1 \end{array}\right]$	$E_3^{-1} = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 2 & 0 & 1 \end{array}\right]$
  4th	The first row is replaced by the sum of the first row and the second row multiplied by (-1)	$E_4 = \left[\!\begin{array}{rrr} 1 & -1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right]$	$E_4^{-1} = \left[\!\begin{array}{rrr} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right]$
  5th	The first row and the second row are interchanged	$E_5 = \left[\!\begin{array}{rrr} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right]$	$E_5^{-1} = \left[\!\begin{array}{rrr} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right]$

• Next we use the elementary matrices $E_1,$ $E_2,$ $E_3,$ $E_4$ and $E_5$ to reconstruct the row reduction above: \begin{align*} E_1 A & = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & -1 \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 2 & 2 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 1 & 0 \\ -1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right], \\ E_2 (E_1 A) & = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 1 & 0 \\ -1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right], \\ E_3 (E_2 E_1 A) & =\left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -2 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right], \\ E_4 (E_3 E_2 E_1 A) & = \left[\!\begin{array}{rrr} 1 & -1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right] = \left[\!\begin{array}{rrr} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right], \\ E_5 (E_4 E_3 E_2 E_1 A) & = \left[\!\begin{array}{rrr} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right] = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right]. \end{align*}
• Since matrix multiplication is associative, we have \[ (E_5 E_4 E_3 E_2 E_1) A = I_3. \] This equality is important, since I can set \(C= E_5 E_4 E_3 E_2 E_1\) and I have proved the first property \(CA = I_3\) in the definition of an invertible matrix.
• To prove the second property of \(C\), that is \(AC = I_3\), I start from already proven equality \[ (E_5 E_4 E_3 E_2 E_1) A = I_3 \] and multiply both sides on the left, first by $E_5^{-1}$, to get \[ E_4 E_3 E_2 E_1 A = E_5^{-1}, \] then by $E_4^{-1}$, to get \[ E_3 E_2 E_1 A = E_5^{-1}E_4^{-1}, \] then by $E_3^{-1}$, \[ E_2 E_1 A = E_5^{-1}E_4^{-1}E_3^{-1}, \] then by $E_2^{-1}$, to get \[ E_1 A = E_5^{-1}E_4^{-1}E_3^{-1}E_2^{-1}, \] then by $E_1^{-1}$, to finally we get \[ A = E_1^{-1} E_2^{-1} E_3^{-1} E_4^{-1} E_5^{-1}. \] This is an important equlity since it tells us that \(A\) can be written as a product of elementary matrices.
• To prove that \(AC = I_3\), we start with \[ A = E_1^{-1} E_2^{-1} E_3^{-1} E_4^{-1} E_5^{-1}, \] and multiply from the right by \(E_5\), then by \(E_4\), then by \(E_3\), then by \(E_2\), then by \(E_1\), to get \[ A (E_5 E_4 E_3 E_2 E_1) = I_3. \] Now recall that \(C= E_5 E_4 E_3 E_2 E_1\), and we have proved that \(AC= I_3\).
• Let me summarize: the row reduction of \(A\) produced five elementary matrices, which we used to define \(C\). From the row reduction algorithm, we concluded that \(CA = I_3\). Then, using matrix algebra and the facts that elementary matrices are invertible and matrix multiplication is associative, we proved that \(AC = I_3\). By the definition of an invertible matrix, we concluded that \(A\) is invertible.
• I hope this specific example helps clarify the abstract proof given above and in Saturday’s post.

---

• In the preceding items we establishe the equality: \[ A = E_1^{-1} E_2^{-1} E_3^{-1} E_4^{-1} E_5^{-1}. \] This is interesting, we represented \(A\) as a product of elementary matrices.

  Let us verify this claim: \begin{align*} E_5^{-1} & = \left[\!\begin{array}{rrr} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right], \\ E_4^{-1} E_5^{-1} & = \left[\!\begin{array}{rrr} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right] = \left[\!\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right], \\ E_3^{-1} \bigl(E_4^{-1} E_5^{-1}\bigr) & =\left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 2 & 0 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right], \\ E_2^{-1} \bigl(E_3^{-1} E_4^{-1} E_5^{-1}\bigr) & = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 1 & 0 \\ -1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right], \\ E_1^{-1} \bigl(E_2^{-1} E_3^{-1} E_4^{-1} E_5^{-1}\bigr) & = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 1 & 0 \\ -1 & 0 & 0 \\ 2 & 2 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 2 & 2 & 1 \end{array}\right] \end{align*} Thus, we confirmed that \[ A = \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 2 & 2 & 1 \end{array}\right] = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 2 & 0 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{array}\right] \]

---

• What we presented above is of theoretical importance. But, just to calculate the inverse of \(A\) we use the algorithm presented in the next item.
• To calculate the inverse $A^{-1}$ we row reduce the $3\times 6$ matrix $[A | I_3]$: \begin{align*} \left[\!\begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ 1 & 2 & 1 & 0 & 1 & 0 \\ 2 & 2 & 1 & 0 & 0 & 1 \end{array}\right] & \sim \left[\!\begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ -1 & 0 & 0 & 0 & 1 & -1 \\ 2 & 2 & 1 & 0 & 0 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 & -1 & 1 \\ 2 & 2 & 1 & 0 & 0 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 1 & 1 & 0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 & -1 & 1 \\ 0 & 0 & 1 & -2 & 0 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 0 & 1 & 0 & 1 & 1 & -1 \\ 1 & 0 & 0 & 0 & -1 & 1 \\ 0 & 0 & 1 & -2 & 0 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 1 & 0 & 0 & 0 & -1 & 1 \\ 0 & 1 & 0 & 1 & 1 & -1 \\ 0 & 0 & 1 & -2 & 0 & 1 \end{array}\right] \\ \end{align*}
• We pause here to celebrate the fact that we found the inverse of the matrix $A.$ Let us verify this claim: \[ \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 2 & 2 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 0 & -1 & 1 \\ 1 & 1 & -1 \\ -2 & 0 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \]

• After reading this post you should be able to solve a problem stated as follows:

  Consider the matrix $M = \left[\begin{array}{rrr} 3 & 3 & 2 \\ 3 & 2 & 1 \\ 2 & 1 & 0 \end{array}\right]$.

  • Determine whether it is possible to write the matrix $M$ as a product of elementary matrices.
  • If you claim that it is possible to write $M$ as a product of elementary matrices, then find elementary matrices whose product is $M$. If you claim that it is not possible to write $M$ as a product of elementary matrices, justify your answer.
• The matrix $A = \left[\begin{array}{rrr} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 2 & 2 & 1 \end{array}\right]$ whose inverse we found above and the matrix $M=\left[\begin{array}{rrr} 3 & 3 & 2 \\ 3 & 2 & 1 \\ 2 & 1 & 0 \end{array}\right]$ which I assigned in the problem above are very special matrices. These matrices have integer entries and all the entries of their inverses are also integers. In exercises it is often convenient to have such matrices: matrices with integer entries whose inverse is also a matrix with integer entries. Such matrices are called unimodular matrices. I was surprised that there are a lot of such matrices. Here is a pdf file with all unimodular matrices with entries $-1,0,1,2,3$ whose inverses have entries among the ten digits and their opposites. I have omitted the matrices with 3 or more zeros. This pdf file is huge, 3119 pages with over 80000 matrices. If you ever need a unimodular matrix I hope you find it here. You can use these matrices to practice row reduction since for every matrix listed in this file I list its inverse.

---

Saturday, Octoer 26, 2026

---

• It is a good idea to review Problems 23, 24, 25 from Section 2.1 before reading this section. Suggested problems for Section 2.3: 1, 3, 5, 8, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 26, 27, 33.
• Statements in Problems 23, 24, 25 in Section 2.1, Theorems in Section 2.2 and all statements in the Invertible Matrix Theorem can be expressed as implications.
  • In general, whenever you try to prove something, you should formulate the statement of the problem as a clear implication. That is, as a statement of the form
    If              , then              .
    In the preceding displayed implication the green space is replaced by a mathematical property which is assumed and the red space is replaced by a mathematical property which is to be proved.
  • When talking abstractly about logical statements we denote the green space by a letter, say $p$ and the red space by another letter, say $q$. Then we write implication as
    If $p$, then $q$.
  • The logical symbol for the implication is $\Rightarrow$ as in
    $p \Rightarrow q$
  • The negation of the implication
    $p \Rightarrow q$
    is
    The negation
    $p$ and $\neg q$.
    Here $\neg q$ stands for the negation of the statement $q.$ As always, a good question is: Why is this the negation of the implication $p\Rightarrow q$? For an answer, I would argue that when we think about propositions carefully our brains are wired to think that way. You can try on your favorite implication. I will illustrate with a popular implication: "If life is present on a planet, then water is present on that planet." Or briefly, "If life, then water." This claim sounds plausible from our experience. However, what would it take for this claim to turn out to be False? First identify $p$ and $q$ in this claim: $p$ is "life is present on a planet" and $q$ is "water is present on a planet." To demonstrate that "If life, then water" is False we would need to find a planet on which life is present, but water is not present. Thus $p$ is true and $q$ is false, or, equivalently $p$ and $\neg q$ are both true.
  • With any implication you should be aware of its contrapositive. The contrapositive of the implication $p \Rightarrow q$ involves the negations of $q$ and $p$. Mathematical notation of the negation of $q$ is $\neg q$ and the negation of $p$ is $\neg p$. The contrapositive of the implication $p \Rightarrow q$ is
    The contrapositive
    $\neg q \Rightarrow \neg p$
    The contrapositive is equivalent to the original implication.
    Why is the contrapositive equivalent to the original implication? Look at the negation of the contrapositive:
    The negation of the contrapositive
    $\neg q$ and $p$.
    Compare this box to the negation of the implication $p \Rightarrow q$. They are identical.
    I must admit that I made a shortcut in stating the negation of the contrapositive. The true negation of the contrapositive is: $\neg q$ and $\neg(\neg p)$ are both true. However, $\neg(\neg p)$ is equal to $p.$ Hence, the negation of the contrapositive is: $\neg q$ and $p$ are both true; which is identical to the negation of the original implication $p\Rightarrow q.$
  • Recall that the converse of the implication $p \Rightarrow q$ is the implication $q \Rightarrow p$. Also recall that the converse can be wrong when the original proposition is true.
  • Almost all mathematical statements can be stated as implications. A symbolic way to represent an implication is $p \Rightarrow q$, where $p$ and $q$ are mathematical statements. There are many ways to say $p \Rightarrow q$ in English.
    • If $p$, then $q$.
    • $q$ if $p$.
    • $p$ is sufficient for $q$.
    • $q$ is necessary for $p$.
    • A sufficient condition for $q$ is $p$.
    • A necessary condition for $p$ is $q$.
    • $p$ implies $q$.
    • $p$ only if $q$.
    • $q$ whenever $p$.
    • $q$ follows from $p$.
    The concepts of necessary and sufficient conditions are commonly used in sciences. If you google "water is necessary for life" Google will respond with hundreds of thousands of hits. In our terminology, "water is necessary for life" means "if life, then water". Interestingly, if you google "life is sufficient for water" there will be no hits; although, the statements "water is necessary for life" and "life is sufficient for water" are equivalent.
  • Recall that the converse of the implication $p \Rightarrow q$ is the implication $q \Rightarrow p$. Also recall that the converse can be false when the original implication is true, like in my favorite example "If it rains, then the Red Square is wet." Like with all things in life, this validity of this statement is debatable (but I would argue that it is expensive to make this statement false). However, the converse is definitely not true. It is easy to make the converse false. However, I do not recommend that we do that without the approval of the Red Square authorities.
  • Logic is essential for any mathematics class. I my essay Brief Review of Mathematical Logic, I introduce the basics of Mathematical Logic.
• All this logic introduction is for the better understanding of the Invertible Matrix Theorem. For easier comprehension of the twelve parts of the Invertible Matrix Theorem I created this picture You should first understand the proofs of the implications corresponding to the green arrows in the picture above. Sometimes the proofs of the green implications are just restatements of the definitions. The next step is understanding the implications corresponding to the blue arrows. Finally, below I present the proofs of the implications corresponding to the orange arrows. Both of these proofs involve the fact that the process of the row reduction to the RREF can be represented as a product of a sequence of elementary matrices.
• Now proof of the implication (g)$\Rightarrow$(c). That is we need to prove:
  If for all $\mathbf{b} \in \mathbb{R}^n$ the matrix equation $A\mathbf{x} = \mathbf{b}$ is consistent, then $A$ has $n$ pivot columns.
  I will prove the contrapositive:
  If $A$ has $m$ pivot columns and $m \lt n$, then there exists $\mathbf{b} \in \mathbb{R}^n$ such that $A\mathbf{x} = \mathbf{b}$ is not consistent.
  Proof.

  • Step 1.  Assume that $A$ has $m$ pivot columns and $m \lt n.$ Then there are $m$ pivot positions in the RREF of $A.$ Since each pivot position occupies its own row, there are $m$ rows in the RREF of $A$ which contain pivot positions. The rows which contain the pivot positions are at the top of the RREF of $A.$ The remaining rows at the bottom of the RREF of $A$ are zero rows. Since $m \lt n$, the bottom row of the RREF of $A$ is a zero row.
  • Step 2.  Let $q \in\mathbb{N}.$ Assume that the RREF of $A$ has been obtained by performing $q$ row operations. Denote by $E_1,\ldots,E_q$ the corresponding elementary matrices. Then The matrix $E_q \cdots E_1 A$ is the RREF of $A$.
  • Step 3.  From Step 2 and Step 1 we conclude that The bottom row of the matrix $E_q \cdots E_1 A$ is the zero row. Therefore, The matrix equation $(E_q \cdots E_1 A) \mathbf{x} = \mathbf{e}_{n,n}$ is not consistent.
  • Step 4.  We know that changing a system of linear equations by performing row operations does not change the solution set of linear system.
    Therefore, the matrix equation $(E_q \cdots E_1 A) \mathbf{x} = \mathbf{e}_{n,n}$ and the matrix equation $ A \mathbf{x} = (E_1)^{-1} \cdots (E_q)^{-1}\mathbf{e}_{n,n}$ have the same solution set.
  • Step 5.  From Step 3 and Step 4 we deduce that the matrix equation $A \mathbf{x} = (E_1)^{-1} \cdots (E_q)^{-1}\mathbf{e}_{n,n}$ is not consistent.
  • Step 6.  From Step 5 we deduce that there exists $\mathbf{b} = (E_1)^{-1} \cdots (E_q)^{-1}\mathbf{e}_{n,n} \in \mathbb{R}^n$ such that the matrix equation $A \mathbf{x} = \mathbf{b}$ is not consistent.
  • QED. 
• Next, I present a proof of the implication (b)$\Rightarrow$(a). That is we need to prove:
  Let $n\in\mathbb{N}$ and let $A$ be an $n\!\times\!n$ matrix. If the RREF of $A$ is $I_n$, then $A$ is invertible.

  This implication is proved in Theorem 7 in the textbook, but I prefer to give another proof which uses only the facts that elementary matrices are invertible and that matrix multiplication is associative. I also like that the proof below proves the invertibility by using the definition of invertibility. The proof below was suggested to me by a student during Fall Quarter 2021. Unfortunately I forgot who it was. In any case, please think on your own about each proof. You can come up with new proofs.

  In the proof below we need to prove that a matrix is invertible. For that it is useful to recall the definition of invertibility.

  Definition.  Let $n\in \mathbb{N}$ and let $I_n$ be the $n\!\times\!n$ identity matrix. An $n\!\times\!n$ matrix $A$ is said to be invertible if there exists an $n\!\times\!n$ matrix $C$ such that \[ CA = I_n \quad \text{and} \quad AC = I_n. \]
  Proof.

  • Step 1.  Let $n\in\mathbb{N}$ and let $A$ be an $n\!\times\!n$ matrix. Assume that the Reduced Row Echelon Form of $A$ is $I_n.$ This means that there exists a positive integer $p$ and $p$ row operators that row reduce $A$ to $I_n.$ Since each row operation can be represented by matrix multiplication from the left by an elementary matrix, there exist elementary matrices $E_1, E_2,\ldots, E_p$ such that the row reduction of the matrix $A$ looks like \[ A \sim E_1 A \sim E_2(E_1 A) \sim \cdots \sim E_p(E_{p-1} \cdots E_1A) = I_n. \] Since matrix multiplication is associative we have \[ (E_p \cdots E_1) A = I_n. \] Thus, we can set $C = E_p \cdots E_1$ in the definition of an invertible matrix and we have $CA = I_n.$
  • Step 2.  Since the matrices $E_1, E_2, \ldots, E_{p-1}, E_p$ are elementary matrices, they are invertible. Therefore we have the matrices $(E_1)^{-1}, (E_2)^{-1}, \ldots, (E_{p-1})^{-1}, (E_p)^{-1}.$ Recall the equality \[ (E_p \cdots E_1) A = I_n. \] from Step 1. Now we multiply the preceding equality from the left by $(E_p)^{-1}$ to get \[ (E_p)^{-1}(E_p E_{p-1} \cdots E_2 E_1) A = (E_p)^{-1} I_n. \] Since matrix multiplication is associative we get \[ (E_{p-1} \cdots E_2 E_1) A = (E_p)^{-1}. \] Next we keep multiplying the preceding equality consecutively by $(E_{p-1})^{-1}, \ldots, (E_2)^{-1}$ to get \[ E_1 A =(E_2)^{-1} \cdots (E_{p-1})^{-1} (E_p)^{-1}. \] Finally we multiply the preceding equality by $(E_1)^{-1}$ to get \[ A =(E_1)^{-1} (E_2)^{-1} \cdots (E_{p-1})^{-1} (E_p)^{-1}. \] Now we multiply the preceding equality from the right with $p$ matrices $E_p, E_{p-1},\ldots, E_2, E_1$ to get \[ A (E_p E_{p-1} \cdots E_2 E_1) = I_n. \] Recall that in Step 1 we set $C = E_p E_{p-1} \cdots E_2 E_1.$ Thus, the last displayed equality reads $AC = I_n.$ To summarize, we have proved that the following two matrix equalities hold \[ C A = I_n \quad \text{and} \quad A C = I_n. \] Therefore, by the definition of invertible matrix $A$ is invertible.
  • QED. 
• I think that the best way to prove the implication (d)$\Rightarrow$(c) is to prove its contrapositive. You can try this proof on your own. It is a blue proof.

---

Friday, October 25, 2024

---

• Today we did preliminaries for Section 2.3 Characterizations of invertible matrices. It is a good idea to review Exercises 23, 24, 25 from Section 2.1 before reading this section. Suggested problems for Section 2.3: 1, 3, 5, 8, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 26, 27, 33.
• I will post the about Invertible Matrix Theorem tomorrow. Below I post about row reduction and its connection with elementary matrices.
• Recall the row reduction that we posted yesterday to find the inverse of $\displaystyle \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ -1 & 1 & 1 \end{array}\right]$: \begin{align*} \left[\!\begin{array}{rrr|rrr} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 2 & 1 & 0 & 1 & 0 \\ -1 & 1 & 1 & 0 & 0 & 1 \end{array}\right] & \sim \left[\!\begin{array}{rrr|rrr} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 2 & 1 & 0 & 1 & 0 \\ 0 & 1 & 0 & 1 & 0 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 0 & 1 \\ 0 & 2 & 1 & 0 & 1 & 0 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & -2 & 1 & -2 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 1 & 0 & 0 & -1 & 1 & -2 \\ 0 & 1 & 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & -2 & 1 & -2 \end{array}\right] \\ \end{align*}
• Now recall that each step in a row reduction can be achieved by multiplication by an elementary matrix.

  Step	the row operation	the elementary matrix	the inverse of elementary matrix
  1st	The third row is replaced by the the sum of the first row and the third row	$E_1 = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 1 & 0 & 1 \end{array}\right]$	$E_1^{-1} = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -1 & 0 & 1 \end{array}\right]$
  2nd	The third row and the second row are interchanged	$E_2 = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{array}\right]$	$E_2^{-1} = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{array}\right]$
  3rd	The third row is replaced by the the sum of the third row and the second row multiplied by $(-2)$	$E_3 = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -2 & 1 \end{array}\right]$	$E_3^{-1} = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 2 & 1 \end{array}\right]$
  4th	The first row is replaced by the the sum of the first row and the third row	$E_4 = \left[\!\begin{array}{rrr} 1 & 0 & 1 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right]$	$E_4^{-1} = \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right]$

• Next we use the elementary matrices $E_1$, $E_2$, $E_3$ and $E_4$ to reconstruct the row reduction above: \begin{align*} E_1 A & = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 1 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ -1 & 1 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ 0 & 1 & 0 \end{array}\right], \\ E_2 (E_1 A) & = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{array}\right] \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ 0 & 1 & 0 \end{array}\right] = \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0\\ 0 & 2 & 1 \end{array}\right], \\ E_3 (E_2 E_1 A) & =\left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -2 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0\\ 0 & 2 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0\\ 0 & 0 & 1 \end{array}\right], \\ E_4 (E_3 E_2 E_1 A) & = \left[\!\begin{array}{rrr} 1 & 0 & 1 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0\\ 0 & 0 & 1 \end{array}\right] = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right]. \end{align*}
• Since matrix multiplication is associative, we have \[ (E_4 E_3 E_2 E_1) A = I_3. \]
• The reason that I wrote all the above detailed steps is to show that we can write $A$ as a product of elementary matrices. Recall the preceding equality: \[ (E_4 E_3 E_2 E_1) A = I_3. \] Multiplying the last equality consecutively by $E_4^{-1}$, $E_3^{-1}$, $E_2^{-1}$, $E_1^{-1}$ we get \[ A = E_1^{-1} E_2^{-1} E_3^{-1} E_4^{-1}. \] Let us verify this claim: \begin{align*} E_4^{-1}& = \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right], \\ E_3^{-1} E_4^{-1} & = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 2 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] = \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0 \\ 0 & 2 & 1 \end{array}\right], \\ E_2^{-1} \bigl(E_3^{-1} E_4^{-1}\bigr) & =\left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{array}\right] \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0 \\ 0 & 2 & 1 \end{array}\right] = \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ 0 & 1 & 0 \end{array}\right], \\ E_1^{-1} \bigl(E_2^{-1} E_3^{-1} E_4^{-1}\bigr) & = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -1 & 0 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ 0 & 1 & 0 \end{array}\right] = \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ -1 & 1 & 1 \end{array}\right]. \end{align*} Thus, we confirmed that \[ A = \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ -1 & 1 & 1 \end{array}\right] = \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -1 & 0 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{array}\right] \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 2 & 1 \end{array}\right] \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \]
• The importance of the preceding item is that we found an expression for the matrix $A$ as a product of elementary matrices. In fact, the reasoning presented above for a specific $3\!\times\!3$ matrix $A$ holds for an arbitrary $n\!\times\!n$ matrix $A$ which can be row reduced to the identity matrix $I_n.$ The above reasoning is used as a model for a proof of the following important implication:

  If Reduced Row Echelon Form of $A$ is $I_n$, then $A$ is invertible.

  This implication is proved in Theorem 7 in Section 2.2.
• After reading this post you should be able to solve a problem stated as follows: Consider the matrix $M = \left[\begin{array}{rrr} 3 & 3 & 2 \\ 3 & 2 & 1 \\ 2 & 1 & 0 \end{array}\right]$.
  • Determine whether it is possible to write the matrix $M$ as a product of elementary matrices.
  • If you claim that it is possible to write $M$ as a product of elementary matrices, then find elementary matrices whose product is $M$. If you claim that it is not possible to write $M$ as a product of elementary matrices, justify your answer.

  ---

Thursday, October 24, 2024

---

• Today we started Section 2.2: The inverse of a matrix; suggested problems are: 1, 4, 5, 7, 12, 13, 21, 22, 23, 24, 28, 32, 33, 34, 38. In all bold face problems, the textbook asks you to "explain why" something is true. In the text of the book and in class we offer proofs. Please try to organize your solutions as proofs, not explanations. State clearly what is known from theorems in the book and use logical reasoning to deduce what the problem claims.
• My approach to the inverse of a $2\!\times\!2$ matrix is a little different from the approach in the textbook. I look at the process of seeking: Given a $2\!\times\!2$ matrix $\displaystyle \left[\!\begin{array}{cc} a & b \\ c & d \end{array}\!\right]$ we seek another $2\!\times\!2$ matrix $\displaystyle \left[\!\begin{array}{cc} x & y \\ z & w \end{array}\!\right]$ such that \[ \left[\!\begin{array}{cc} a & b \\ c & d \end{array}\!\right]\left[\!\begin{array}{cc} x & y \\ z & w \end{array}\!\right] = \left[\!\begin{array}{cc} ax+bz & ay+bw \\ cx+dz & cy+dw \end{array}\!\right] = \left[\!\begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array}\!\right]. \] At the first step we focus on the zeros and seek $y$ and $w$ such that $ay+bw = 0$, and $x$ and $z$ such that $cx+dz = 0.$ One of the simplest solution of $ay+bw = 0$ is $a(-b)+b\,a = 0$ and one of the simplest solution of $cx+dz = 0$ is $c\,d+d(-b) = 0.$ (This might take some guessing since you have another choice of $c(-d)+d\,b = 0.$) Now try $\displaystyle \left[\!\begin{array}{cc} d & -b \\ -c & a \end{array}\!\right]$: \[ \left[\!\begin{array}{cc} a & b \\ c & d \end{array}\!\right]\left[\!\begin{array}{cc} d & -b \\ -c & a \end{array}\!\right] = \left[\!\begin{array}{cc} ad-bc & 0 \\ 0 & c(-b)+da \end{array}\!\right] = (ad-bc) \left[\!\begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array}\!\right]. \] From the last equality we see that if $ad-bc \neq 0$, then the matrix $\displaystyle \left[\!\begin{array}{cc} a & b \\ c & d \end{array}\!\right]$ is invertible and its inverse is \[ \displaystyle \left[\!\begin{array}{cc} a & b \\ c & d \end{array}\!\right]^{-1} = \frac{1}{ad-bc}\left[\!\begin{array}{cc} d & -b \\ -c & a \end{array}\!\right]. \] My point here is that you do not have to memorize the inverse, you can construct it on your own just playing with the equations $ay+bw = 0$ and $cx+dz = 0$ and their simple solutions.
  
  The number $ad-bc$ is called the determinant of the matrix $\displaystyle \left[\!\begin{array}{cc} a & b \\ c & d \end{array}\!\right]$. We write \[ \det \left[\!\begin{array}{cc} a & b \\ c & d \end{array}\!\right] = \left|\!\begin{array}{cc} a & b \\ c & d \end{array}\!\right| = ad-bc. \]
• It is important for you to perfect calculation of the inverse of an invertible matrix. For a $2\!\times\!2$ matrix one should use Theorem 4, that is the method explained in the preceding item. For a $3\!\times\!3$ matrix one uses the method presented in Example 7.
• Consider the following problem:
  • Determine whether the matrix $A = \left[\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ -1 & 1 & 1 \end{array}\right]$ is invertible.
  • If $A$ is invertible find its inverse $A^{-1}$.
• To answer the first question in the problem we use the implication

  If RREF of $A$ is $I_3$, then $A$ is invertible.

  This implication is proved in Theorem 7 in Section 2.2 . This proof is important!
• So, we just row reduce $A$ to RREF and that will answer the first question in the exercise: \begin{align*} \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ -1 & 1 & 1 \end{array}\right] & \sim \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 2 & 1 \\ 0 & 1 & 0 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0 \\ 0 & 2 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr} 1 & 0 & -1 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] \\ \end{align*}
• The row reduction above shows that $A$ is row equivalent to $I_3$. Therefore, by Theorem 7 in Section 2.2 , $A$ is invertible.
• To calculate the inverse $A^{-1}$ we row reduce the $3\times 6$ matrix $[A | I_3]$: \begin{align*} \left[\!\begin{array}{rrr|rrr} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 2 & 1 & 0 & 1 & 0 \\ -1 & 1 & 1 & 0 & 0 & 1 \end{array}\right] & \sim \left[\!\begin{array}{rrr|rrr} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 2 & 1 & 0 & 1 & 0 \\ 0 & 1 & 0 & 1 & 0 & 1 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 0 & 1 \\ 0 & 2 & 1 & 0 & 1 & 0 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 1 & 0 & -1 & 1 & 0 & 0 \\ 0 & 1 & 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & -2 & 1 & -2 \end{array}\right] \\ & \sim \left[\!\begin{array}{rrr|rrr} 1 & 0 & 0 & -1 & 1 & -2 \\ 0 & 1 & 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & -2 & 1 & -2 \end{array}\right] \\ \end{align*}
• The matrix whose inverse we found above is a very special matrix. It is a matrix whose entries are integers and all the entries of its inverse are integers. In exercises it is often convenient to have such matrices: matrices with integer entries whose inverse is also a matrix with integer entries. Such matrices are called unimodular matrices. I was surprised that there are a lot of such matrices. Here is a pdf file with all unimodular matrices with entries $-1,0,1,2,3$ whose inverses have entries among the ten digits and their opposites. I have omitted the matrices with 3 or more zeros. This pdf file is huge, 3119 pages with over 80000 matrices. If you ever need a unimodular matrix I hope you find it here. You can use these matrices to practice row reduction since for every matrix listed in this file I list its inverse.

---

Tuesday, October 22, 2024

---

• We finished Section 2.1: Matrix Operations and started Section 2.2: The Inverse Of a Matrix

• Please pay special attention to problems 27, 28. The pattern that you are asked to explore in these problems we encountered in explorations related to Matrix Multiplication. But here that pattern is put in context of vectors from the same space.

  • For example if we are given two vectors in \(\mathbb{R}^4\): \[ \mathbf{u} = \left[\! \begin{array}{c} 4 \\ 3\\ 2 \\ 1 \end{array} \!\right], \quad \mathbf{v} = \left[\! \begin{array}{c} a \\ b \\ c \\ d \end{array} \!\right]. \] Notice that these vectors are \(4\times 1\) matrices. Therefore, their transposes are \(1\times 4\) matrices \[ \mathbf{u}^\top = \bigl[\! \begin{array}{cccc} 4 & 3 & 2 & 1 \end{array} \!\bigr], \quad \mathbf{v}^\top = \bigl[\! \begin{array}{c} a & b & c & d \end{array} \!\bigr]. \]
  • Now we can calculate \(1\times 1\) matrices, which are just scalars, and which I called "rotated Ts" in my handwritten posts: \begin{align*} \mathbf{u}^\top \mathbf{u} & = \bigl[\! \begin{array}{cccc} 4 & 3 & 2 & 1 \end{array} \!\bigr] \left[\! \begin{array}{c} 4 \\ 3\\ 2 \\ 1 \end{array} \!\right] = 4^2 + 3^2 + 2^2 + 1^2 = 30, \\ \mathbf{u}^\top \mathbf{v} & = \bigl[\! \begin{array}{cccc} 4 & 3 & 2 & 1 \end{array} \!\bigr] \left[\! \begin{array}{c} a \\ b \\ c \\ d \end{array} \!\right] = 4a+3b+2c+1d, \\ \mathbf{v}^\top \mathbf{u} & = \bigl[\! \begin{array}{c} a & b & c & d \end{array} \!\bigr] \left[\! \begin{array}{c} 4 \\ 3\\ 2 \\ 1 \end{array} \!\right] = 4a + 3b + 2c + 1 d, \\ \mathbf{v}^\top \mathbf{v} & = \bigl[\! \begin{array}{c} a & b & c & d \end{array} \!\bigr] \left[\! \begin{array}{c} a \\ b \\ c \\ d \end{array} \!\right] = a^2 + b^2 + c^2 + d^2. \end{align*} Such products are called inner products or dot products.
  • We can also calculate \(4\times 4\) matrices: \begin{align*} \mathbf{u} \mathbf{u}^\top & = \left[\! \begin{array}{c} 4 \\ 3\\ 2 \\ 1 \end{array} \!\right] \bigl[\! \begin{array}{cccc} 4 & 3 & 2 & 1 \end{array} \!\bigr] = \left[\! \begin{array}{cccc} 16 & 12 & 8 & 4 \\ 12 & 9 & 6 & 3 \\ 8 & 6 & 4 & 2 \\ 4 & 3 & 2 & 1 \end{array} \!\right], \\ \mathbf{v} \mathbf{u}^\top & = \left[\! \begin{array}{c} a \\ b \\ c \\ d \end{array} \!\right] \bigl[\! \begin{array}{cccc} 4 & 3 & 2 & 1 \end{array} \!\bigr] = \left[\! \begin{array}{cccc} 4a & 3a & 2a & 1a \\ 4b & 3b & 2b & 1b \\ 4c & 3c & 2c & 1c \\ 4d & 3d & 2d & 1d \end{array} \!\right] , \\ \mathbf{u} \mathbf{v}^\top & = \left[\! \begin{array}{c} 4 \\ 3\\ 2 \\ 1 \end{array} \!\right] \bigl[\! \begin{array}{c} a & b & c & d \end{array} \!\bigr] = \left[\! \begin{array}{cccc} 4a & 4b & 4c & 4d \\ 3a & 3b & 3c & 3d \\ 2a & 2b & 2c & 2d \\ 1a & 1b & 1c & 1d \end{array} \!\right], \\ \mathbf{v} \mathbf{v}^\top & = \left[\! \begin{array}{c} a \\ b \\ c \\ d \end{array} \!\right] \bigl[\! \begin{array}{c} a & b & c & d \end{array} \!\bigr] = \left[\! \begin{array}{cccc} aa & ab & ac & ad \\ ba & bb & bc & bd \\ ca & cb & cc & cd \\ da & db & dc & dd \end{array} \!\right]. \end{align*} Such products are called outer products.
• Please pay special attention to problems 22, 23, 24, 25 from Section 2.1. In each of these problems you are asked to prove an implication.
• For each of these problems, and in general whenever you try to prove something, you should rewrite the statement of the problem as a clear implication. That is, as a statement of the form
  If              , then              .
  In the preceding displayed implication the green space is replaced by a mathematical property which is assumed and the red space is replaced by a mathematical property which is to be proved. When talking abstractly about logical statements we denote the green space by a letter, say p and the red space by another letter, say q. Then we write an implication as
  If $p$, then $q$.
  The logical symbol for the implication is $\Rightarrow$ as in
  $p \Rightarrow q$
  With any implication you should be aware of its contrapositive. The contrapositive of the implication $p \Rightarrow q$ involves the negations of $q$ and $p$. Mathematical notation of the negation of $q$ is $\neg q$ and the negation of $p$ is $\neg p$. The contrapositive of the implication $p \Rightarrow q$ is
  The contrapositive
  $\neg q \Rightarrow \neg p$
  The contrapositive is equivalent to the original implication.

• Recall that the converse of the implication $p \Rightarrow q$ is the implication $q \Rightarrow p$. Also recall that the converse can be wrong when the original proposition is true.
• For Problem 22 in Section 2.1 write down the converse and answer whether it is true or false. Justify your answer.
• For Problem 24 in Section 2.1 write down the converse and answer whether it is true or false. Justify your answer. This problem is closely related to Problem 26 in Section 2.1.

---

Monday, October 21, 2024 (updated)

---

• Here is my another attempt to illustrate how matrix multiplication works. Notice that in the illustration below I lifted matrix \(A\) into a gray cloud. I did this to be able to "cross" the rows of \(A\) with the columns of \(B\) to get the colored rotated Ts, understanding of which is essential for understanding the matrix multiplication algorithm.
• In this item I will illustrate the multiplication of two matrices by emphasising the pattern of "rotated Ts" between $\mathbb{R}^3$ vectors: \[ \require{bbox} \bbox[#7FBFBF, 8px, border: 3px solid teal]{ \left[\! \begin{array}{rrr} 1 & 3 & 2 \\ 2 & 0 & 1 \\ 2 & 1 & 1 \\ 1 & 4 & 2 \end{array} \!\right] \left[\! \begin{array}{rrrrr} 1 & 0 & -1 & 0 & 1 \\ 0 & 1 & 1 & 0 & 1 \\ 0 & 0 & 0 & 1 & -1 \\ \end{array} \!\right] = \left[\! \begin{array}{rrrrr} 1 & 3 & 2 & 2 & 2 \\ 2 & 0 & -2 & 1 & 1 \\ 2 & 1 & -1 & 1 & 2 \\ 1 & 4 & 3 & 2 & 3 \end{array} \!\right] }. \] Here the left matrix in the product is a $4\times\!3$ matrix with the rows: \[ \left[\! \begin{array}{r} 1 \\ 3 \\ 2 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 2 \\ 0 \\ 1 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 2 \\ 1 \\ 1 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 1 \\ 4 \\ 2 \end{array} \!\right], \] and right matrix in the product is a $3\times\!5$ matrix with the columns: \[ \left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right], \qquad \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right]. \] It is useful to think of the matrix multiplication as follows (as you read through the matrix-vector arithmetic below always look for dot products in different forms, always): \begin{align*} & \left[\!\! \begin{array}{r} \phantom{\biggl|\biggr.} \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr] \\ \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr] \\ \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr] \\ \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr] \end{array} \!\right] \left[\! \begin{array}{rrrrr} \left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \end{array} \!\right] \\ & = \left[\! \begin{array}{rrrrr} \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \\ \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \\ \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \\ \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \\ \end{array} \!\right] \\ & = \left[\! \begin{array}{ccccc} 1\!\cdot\!1 + 3\!\cdot\!0 + 2\!\cdot\!0 & 1\!\cdot\!0 + 3\!\cdot\!1 + 2\!\cdot\!0 & 1\!\cdot\!(-1) + 3\!\cdot\!1 + 2\!\cdot\!0 & 1\!\cdot\!0 + 3\!\cdot\!0 + 2\!\cdot\!1 & 1\!\cdot\!1 + 3\!\cdot\!1 + 2\!\cdot\!(-1) \\ 2\!\cdot\!1 + 0\!\cdot\!0 + 1\!\cdot\!0 & 2\!\cdot\!0 + 0\!\cdot\!1 + 1\!\cdot\!0 & 2\!\cdot\!(-1) + 0\!\cdot\!1 + 1\!\cdot\!0 & 2\!\cdot\!0 + 0\!\cdot\!0 + 1\!\cdot\!1 & 2\!\cdot\!1 + 0\!\cdot\!1 + 1\!\cdot\!(-1) \\ 2\!\cdot\!1 + 1\!\cdot\!0 + 1\!\cdot\!0 & 2\!\cdot\!0 + 1\!\cdot\!1 + 1\!\cdot\!0 & 2\!\cdot\!(-1) + 1\!\cdot\!1 + 1\!\cdot\!0 & 2\!\cdot\!0 + 1\!\cdot\!0 + 1\!\cdot\!1 & 2\!\cdot\!1 + 1\!\cdot\!1 + 1\!\cdot\!(-1) \\ 1\!\cdot\!1 + 4\!\cdot\!0 + 2\!\cdot\!0 & 1\!\cdot\!0 + 4\!\cdot\!1 + 2\!\cdot\!0 & 1\!\cdot\!(-1) + 4\!\cdot\!1 + 2\!\cdot\!0 & 1\!\cdot\!0 + 4\!\cdot\!0 + 2\!\cdot\!1 & 1\!\cdot\!1 + 4\!\cdot\!1 + 2\!\cdot\!(-1) \end{array} \!\right] \\ &= \left[\! \begin{array}{rrrrr} 1 & 3 & 2 & 2 & 2 \\ 2 & 0 & -2 & 1 & 1 \\ 2 & 1 & -1 & 1 & 2 \\ 1 & 4 & 3 & 2 & 3 \end{array} \!\right]. \end{align*}

• Here is my attempt to improve Figures 2 and 3 in Section 2.1: Matrix Multiplication (page 96).

  At the bottom of the snippet that I present below is a proof that for all vectors \(\mathbf{x} \in \mathbb{R}^n\) we have \((AB)\mathbf{x} = A(B\mathbf{x})\). In the proof, I use the definition of the matrix product \(AB\), the definition of the matrix vector multiplication, the linearity property of the matrix-vector multiplication, and the definition of the matrix-vector multiplication.

  This proof is given in the book below Figures 2 and 3 on pages 96 and 97. I present it here in hope that you will enjoy it and understand it better in color.

  

---

Friday, October 18, 2024

---

• Today we discussed Section 2.1: Matrix operations. For now, the objective is to learn matrix multiplication and how it is connected to the Reduced Row Echelon Form. Suggested problems for Section 2.1: 1, 4, 5, 7, 9, 10, 11, 12, 17, 20, 22, 23, 24, 25, 26, 27, 28, 34. The jewel of this section is Matrix Multiplication. It is a beautiful and rich concept.
• Here is my attempt to illustrate matrix multiplication with colors:
• Here is another attempt to illustrate matrix multiplication with colors. This time the number the matrix $A$ on the left has $n$ columns and the matrix $B$ on the right has $n$ rows. I illustrated this by representing the rows of $A$ as wide narrow rectangles and by representing the columns of $B$ as tall narrow rectangles.
• In this item I will illustrate the multiplication of two matrices below by emphasising the pattern between $\mathbb{R}^3$ vectors: \[ \require{bbox} \bbox[#7FBFBF, 8px, border: 3px solid teal]{ \left[\! \begin{array}{rrr} 1 & 3 & 2 \\ 2 & 0 & 1 \\ 2 & 1 & 1 \\ 1 & 4 & 2 \end{array} \!\right] \left[\! \begin{array}{rrrrr} 1 & 0 & -1 & 0 & 1 \\ 0 & 1 & 1 & 0 & 1 \\ 0 & 0 & 0 & 1 & -1 \\ \end{array} \!\right] = \left[\! \begin{array}{rrrrr} 1 & 3 & 2 & 2 & 2 \\ 2 & 0 & -2 & 1 & 1 \\ 2 & 1 & -1 & 1 & 2 \\ 1 & 4 & 3 & 2 & 3 \end{array} \!\right] }. \] Here the left matrix in the product is a $4\times\!3$ matrix with the rows: \[ \left[\! \begin{array}{r} 1 \\ 3 \\ 2 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 2 \\ 0 \\ 1 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 2 \\ 1 \\ 1 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 1 \\ 4 \\ 2 \end{array} \!\right], \] and right matrix in the product is a $3\times\!5$ matrix with the columns: \[ \left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right], \qquad \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right], \qquad \left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right]. \] It is useful to think of the matrix multiplication as follows (as you read through the matrix-vector arithmetic below always look for dot products in different forms, always): \begin{align*} & \left[\!\! \begin{array}{r} \phantom{\biggl|\biggr.} \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr] \\ \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr] \\ \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr] \\ \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr] \end{array} \!\right] \left[\! \begin{array}{rrrrr} \left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \end{array} \!\right] \\ & = \left[\! \begin{array}{rrrrr} \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 3 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \\ \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 0 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \\ \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 2 & 1 & 1 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \\ \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 0 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} -1 \\ 1 \\ 0 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 0 \\ 0 \\ 1 \end{array} \!\right] & \bigl[\!\begin{array}{ccc} 1 & 4 & 2 \end{array} \!\bigr]\!\left[\! \begin{array}{r} 1 \\ 1 \\ -1 \end{array} \!\right] \\ \end{array} \!\right] \\ & = \left[\! \begin{array}{ccccc} 1\!\cdot\!1 + 3\!\cdot\!0 + 2\!\cdot\!0 & 1\!\cdot\!0 + 3\!\cdot\!1 + 2\!\cdot\!0 & 1\!\cdot\!(-1) + 3\!\cdot\!1 + 2\!\cdot\!0 & 1\!\cdot\!0 + 3\!\cdot\!0 + 2\!\cdot\!1 & 1\!\cdot\!1 + 3\!\cdot\!1 + 2\!\cdot\!(-1) \\ 2\!\cdot\!1 + 0\!\cdot\!0 + 1\!\cdot\!0 & 2\!\cdot\!0 + 0\!\cdot\!1 + 1\!\cdot\!0 & 2\!\cdot\!(-1) + 0\!\cdot\!1 + 1\!\cdot\!0 & 2\!\cdot\!0 + 0\!\cdot\!0 + 1\!\cdot\!1 & 2\!\cdot\!1 + 0\!\cdot\!1 + 1\!\cdot\!(-1) \\ 2\!\cdot\!1 + 1\!\cdot\!0 + 1\!\cdot\!0 & 2\!\cdot\!0 + 1\!\cdot\!1 + 1\!\cdot\!0 & 2\!\cdot\!(-1) + 1\!\cdot\!1 + 1\!\cdot\!0 & 2\!\cdot\!0 + 1\!\cdot\!0 + 1\!\cdot\!1 & 2\!\cdot\!1 + 1\!\cdot\!1 + 1\!\cdot\!(-1) \\ 1\!\cdot\!1 + 4\!\cdot\!0 + 2\!\cdot\!0 & 1\!\cdot\!0 + 4\!\cdot\!1 + 2\!\cdot\!0 & 1\!\cdot\!(-1) + 4\!\cdot\!1 + 2\!\cdot\!0 & 1\!\cdot\!0 + 4\!\cdot\!0 + 2\!\cdot\!1 & 1\!\cdot\!1 + 4\!\cdot\!1 + 2\!\cdot\!(-1) \end{array} \!\right] \\ &= \left[\! \begin{array}{rrrrr} 1 & 3 & 2 & 2 & 2 \\ 2 & 0 & -2 & 1 & 1 \\ 2 & 1 & -1 & 1 & 2 \\ 1 & 4 & 3 & 2 & 3 \end{array} \!\right]. \end{align*}

---

Monday, October 14, 2024

---

• Suggested problems for Section 1.8: 1-4, 12, 13-17, 19, 20, 25, 27, 28.
• Suggested problems for Section 1.9: 1, 3, 4, 5, 7, 8, 11, 12, 18, 19, 23 Notice that we did not discuss Definitions on page 76 and the content after these definitions.
• On each in-class exam, I assign four problems. Often these problems have several parts (a), (b), and sometimes (c) parts. In this PDF file you can find some problems in the form that they could appear on the exam.
• In the textbook the authors list several reflections transformations across some popular lines like: $x_2 = 0$ (this is $x_1$-axis), $x_1 = 0$ (this is $x_2$-axis), $x_2 = x_1$ (this is the diagonal of the first and the third quadrant), and $x_2 = -x_1$ (this is the diagonal of the second and the fourth quadrant). But why not establish the standard matrix of the reflection across the line which makes the angle $\theta$ with the positive $x_1$-semi-axis? It is not more difficult than the standard matrix of the rotation by the angle $\theta$ in counterclockwise direction, which is \[ \left[\! \begin{array}{cc} \cos(\theta) & -\sin(\theta) \\[5pt] \sin(\theta) & \cos(\theta) \end{array}\!\right] \]
• The picture below will help us determine the standard matrix of the reflection across the blue line which makes angle $\theta$ with the positive $x_1$-semi-axis. Denote this reflection by \[ F:\mathbb{R}^2 \to \mathbb{R}^2. \] To determine the standard matrix of this reflection we need to calculate the coordinates of the vectors \[ F \mathbf{e}_1 = F \left[\! \begin{array}{c} 1 \\[3pt] 0 \end{array}\!\right] \quad \text{and} \quad F \mathbf{e}_2 = F \left[\! \begin{array}{c} 0 \\[3pt] 1 \end{array}\!\right] \] In the picture below the vector $F \mathbf{e}_1$ is dark orange and the vector $F \mathbf{e}_2$ dark purple.
  • By the definition of the reflection $F$, the angle formed by the light orange coordinate vector $\mathbf{e}_1$ and its reflection the dark orange vector $F \mathbf{e}_1$ is $2\theta.$ Therefore, \[ F \mathbf{e}_1 = F \left[\! \begin{array}{c} 1 \\[3pt] 0 \end{array}\!\right] = \left[\! \begin{array}{c} \cos(2\theta) \\[3pt] \sin(2\theta) \end{array}\!\right]. \]
  • In this item we calculate the coordinates of the dark purple vector $F \mathbf{e}_2$ which is the reflection of the light purple coordinate vector $\mathbf{e}_2.$ First, we observe that the light gray right triangles in the picture below are congruent. This claim follows from the fact that the hypothenuse of each of these triangles is of length $1$ and the triangles have the same angle $2\theta$ at the origin. That the angle at the origin of the top light gray triangle is $2\theta$ follows from the definition of the reflection. To calculate the angle at the origin for the lower light gray triangle we calculate the angle between the light purple coordinate vector $\mathbf{e}_2$ and its reflection, the dark purple vector $F \mathbf{e}_2;$ since the angle between the light purple coordinate vector $\mathbf{e}_2$ and the blue line is $\displaystyle \frac{\pi}{2} - \theta$, the angle between is $\mathbf{e}_2$ and $F \mathbf{e}_2$ is \[ 2 \left(\frac{\pi}{2} - \theta\right) = \pi - 2 \theta. \] Since the sum of this angle and the angle at the origin for the lower light gray triangle equals $\pi,$ we have proven that the angle at the origin for the lower light gray triangle equals $2\theta.$ In the preceding item we proved that the horizontal side of the top light gray right triangle is $\cos(2\theta)$ and its vertical side is $\sin(2\theta)$. As a consequence, the horizontal side of the lower light gray right triangle is $\sin(2\theta)$ and its vertical side is $\cos(2\theta)$. Adjusting for the positioning in the coordinate system we get \[ F \mathbf{e}_2 = F \left[\! \begin{array}{c} 0 \\[3pt] 1 \end{array}\!\right] = \left[\! \begin{array}{c} \sin(2\theta) \\[3pt] -\cos(2\theta) \end{array}\!\right]. \] Finally we have that the standard matrix of the reflection across the blue line which makes an angle $\theta$ with the positive $x_1$-semi-axis is \[ \left[\! \begin{array}{cc} \cos(2\theta) & \sin(2\theta) \\[5pt] \sin(2\theta) & -\cos(2\theta) \end{array}\!\right]. \] Enjoy Reflections!
  Enjoy Reflections!

• In the following examples, I demonstrate the geometric interpretation of various matrices. The textbook provides additional illustrations for a broader range of matrices.

  In the examples below, the happy face is designed by the heads of the following vectors: For the face I used the circle centered at the head of the vector \(\left[\! \begin{array}{c} 1 \\ 1 \end{array} \!\right]\) and with radius \(4/5\). That is the set of the following vectors \[ \Biggl\{ \left[\! \begin{array}{c} 1 \\ 1 \end{array} \!\right] + \frac{4}{5} \left[\! \begin{array}{c} \cos t \\ \sin t \end{array} \!\right] \, : \, t \in [0, 2 \pi) \Biggr\}. \] The navy eyes are at the heads of the following two vectors \[ \frac{1}{5} \left[\! \begin{array}{c} 4 \\ 7 \end{array} \!\right], \quad \frac{1}{5} \left[\! \begin{array}{c} 6 \\ 7 \end{array} \!\right]. \] The red smile, I used the set of the following vectors \[ \Biggl\{ \left[\! \begin{array}{c} 1 \\ 1/2 \end{array} \!\right] + \frac{1}{2} \left[\! \begin{array}{c} 2 t \\ 3 t^2 \end{array} \!\right] \, : \, t \in \left[-\frac{1}{3},\frac{1}{3}\right] \Biggr\}. \]

• The matrix resulting in the image transformation below is \[ A = \left[\! \begin{array}{cc} 1 & 1 \\ 0 & 1 \end{array} \!\right]. \] This transformation is called shear.
• Below is an illustration of a rotation about the origin by the angle \(\theta = \dfrac{5 \pi}{8}\) counterclockwise. Recall that the rotation matrix about the origin by the angle \(\theta\) is \[ R(\theta) = \left[\! \begin{array}{cc} \cos \theta & - \sin \theta \\ \sin \theta & \cos \theta \end{array} \!\right]. \] Thus, the matrix used in the following illustration is \[ \left[\! \begin{array}{cc} \cos\left(\dfrac{5 \pi}{8}\right) & - \sin\left(\dfrac{5 \pi}{8}\right) \\[5pt] \sin\left(\dfrac{5 \pi}{8}\right) & \cos\left(\dfrac{5 \pi}{8}\right) \end{array} \!\right] = - \frac{1}{2} \left[\! \begin{array}{cc} \sqrt{2-\sqrt{2}} & \sqrt{2+\sqrt{2}} \\[5pt] - \sqrt{2+\sqrt{2}} & \sqrt{2-\sqrt{2}} \end{array} \!\right] . \]
• In this item we show another rotation. It is the rotation about the origin by the angle \(\theta = \dfrac{\pi}{8}\) counterclockwise. The matrix used in the following illustration is \[ \left[\! \begin{array}{cc} \cos\left(\dfrac{\pi}{8}\right) & - \sin\left(\dfrac{\pi}{8}\right) \\[5pt] \sin\left(\dfrac{\pi}{8}\right) & \cos\left(\dfrac{\pi}{8}\right) \end{array} \!\right] = \frac{1}{2} \left[\! \begin{array}{cc} \sqrt{2+\sqrt{2}} & - \sqrt{2-\sqrt{2}} \\[5pt] \sqrt{2-\sqrt{2}} & \sqrt{2+\sqrt{2}} \end{array} \!\right] . \] Notice the relationship between the rotation used in this item and the rotation in the preceding item: The angle used in the rotation in the preceding item is the sum of the angle in this item and \(\pi/2\): \[ \dfrac{5 \pi}{8} = \dfrac{\pi}{2} + \dfrac{\pi}{8}. \] In the language of rotations, the rotation in the preceding item can be obtained by performing the rotation from this item and then performing the rotation by the angle \(\theta = \pi/2\). Notice that the standard matrix of the rotation by the angle \(\theta = \pi/2\) is \[ \left[\! \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \!\right]. \] Later on we will establish a close connection between the following three matrices: \[ \frac{1}{2} \left[\! \begin{array}{cc} -\sqrt{2-\sqrt{2}} & -\sqrt{2+\sqrt{2}} \\[5pt] \sqrt{2+\sqrt{2}} & -\sqrt{2-\sqrt{2}} \end{array} \!\right] , \quad \left[\! \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \!\right], \quad \frac{1}{2} \left[\! \begin{array}{cc} \sqrt{2+\sqrt{2}} & - \sqrt{2-\sqrt{2}} \\[5pt] \sqrt{2-\sqrt{2}} & \sqrt{2+\sqrt{2}} \end{array} \!\right]. \] To eliminate the suspense, the first matrix is the product of the second and the third matrix. We will define matrix product next week.
• The purple line in the image below makes an angle of \(\theta = \pi/8\) with the horizontal axis. The image illustrates the reflection transformation across the purple line. As shown in an earlier item, the reflection matrix used in the image below is \[ \left[\! \begin{array}{cc} \cos\left(2\theta\right) & \sin\left(2\theta\right) \\[5pt] \sin\left(2\theta\right) & -\cos\left(2\theta\right) \end{array} \!\right] = \left[\! \begin{array}{cc} \cos\left(\dfrac{\pi}{4}\right) & \sin\left(\dfrac{\pi}{4}\right) \\[5pt] \sin\left(\dfrac{\pi}{4}\right) & -\cos\left(\dfrac{\pi}{4}\right) \end{array} \!\right] = \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} \\[5pt] \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right]. \]
• The teal line in the image below makes an angle of \(\theta = 7\pi/8\) with the positive horizontal semi-axis. The image illustrates the reflection transformation across the teal line. As shown in an earlier item, the reflection matrix used in the image below is \[ \left[\! \begin{array}{cc} \cos\left(2\theta\right) & \sin\left(2\theta\right) \\[5pt] \sin\left(2\theta\right) & -\cos\left(2\theta\right) \end{array} \!\right] = \left[\! \begin{array}{cc} \cos\left(\dfrac{7\pi}{4}\right) & \sin\left(\dfrac{7\pi}{4}\right) \\[5pt] \sin\left(\dfrac{7\pi}{4}\right) & -\cos\left(\dfrac{7\pi}{4}\right) \end{array} \!\right] = \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right]. \]
• An interesting problem is to find the standard matrix of the linear transformation obtained by first reflecting across the teal line, then reflecting across the purple line, in the two preceding items. How would we find that standard matrix? To find that standard matrix we need to track the images of the standard coordinate vectors \[ \mathbf{e}_1 = \left[\! \begin{array}{c} 1 \\ 0 \end{array} \!\right] \quad \text{and} \quad \mathbf{e}_2 = \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right]. \] Consider first the vector \(\mathbf{e}_1 = \left[\! \begin{array}{c} 1 \\ 0 \end{array} \!\right]\): Reflecting this vector across the teal line we get the vector: \[ \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right] \left[\! \begin{array}{c} 1 \\ 0 \end{array} \!\right] = \left[\! \begin{array}{c} \frac{\sqrt{2}}{2} \\[5pt] - \frac{\sqrt{2}}{2} \end{array} \!\right]. \] Reflecting the last vector across the purple line we get the vector: \[ \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} \\[5pt] \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right] \left[\! \begin{array}{c} \frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2}\end{array} \!\right] = \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right]. \] Now, consider the vector \(\mathbf{e}_2 = \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right]\): Reflecting this vector across the teal line we get the vector: \[ \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right] \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right] = \left[\! \begin{array}{c} -\frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2} \end{array} \!\right]. \] Reflecting the last vector across the purple line we get the vector: \[ \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} \\[5pt] \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right] \left[\! \begin{array}{c} -\frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2} \end{array} \!\right] = \left[\! \begin{array}{c} -1 \\ 0 \end{array} \!\right]. \] Thus, we have that the linear transformation obtained by first reflecting across the teal line, then reflecting across the purple line maps \[ \left[\! \begin{array}{c} 1 \\ 0 \end{array} \!\right] \quad \text{to} \quad \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right] \qquad \text{and} \qquad \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right] \quad \text{to} \quad \left[\! \begin{array}{c} -1 \\ 0 \end{array} \!\right]. \] Hence, the standard matrix of the linear transformation obtained by first reflecting across the purple line, then reflecting across the teal line is \[ \left[\! \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \!\right]. \] We have seen this matrix before. This is the rotation matrix by the angle \(\theta = \pi/2\). That is \[ \left[\! \begin{array}{cc} \cos\left(\dfrac{\pi}{2}\right) & -\sin\left(\dfrac{\pi}{2}\right) \\[5pt] \sin\left(\dfrac{\pi}{2}\right) & \cos\left(\dfrac{\pi}{2}\right) \end{array} \!\right] = \left[\! \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \!\right]. \] In conclusion, in this example, the composition of two reflections is a rotation. We will later prove this fact for any two rotations.
• The image below illustrates the linear transformation from \(\mathbb{R}^2\) into \(\mathbb{R}^3\) given by the matrix \[ A = \left[\! \begin{array}{cc} 0 & 0 \\[5pt] 1 & 0 \\ 0 & 1 \end{array} \!\right]. \]
• The image below illustrates the linear transformation from \(\mathbb{R}^2\) into \(\mathbb{R}^3\) given by the matrix \[ A = \left[\! \begin{array}{cc} 1 & -1 \\[5pt] 2 & 0 \\ 1 & 1 \end{array} \!\right]. \]

---

Sunday, October 13, 2024

---

• Here is my attempt to illustrate matrix-vector multiplication with colors:
• We did a part of Section 1.8 on Friday. We will continue on Monday and also do Section 1.9.
• Suggested problems for Section 1.8: 1-4, 12, 13-17, 19, 20, 25, 27, 28

---

Thursday, October 10, 2024

---

• We discussed Section 1.7: Linear independence today. Suggested problems: 2, 4, 5, 8, 10, 11, 17, 21, 23, 24, 25, 26, 27, 28, 29, 32, 33, 34, 35, 36, 37, 38, 39, 40 and read the next item.

• Problems 41, 42, 43, 44 in Section 1.7: Linear independence are very interesting and important. The matrices in these problems are not easy to row reduce by hand, so the textbook recommends that we use a calculator. Below I calculated RREFs for the matrices given in Problems 41 and 42. Based on these RREFs you should be able to answer Problems 41, 42, 43, 44.

  Problem 41 \[ \left[ \begin{array}{rrrrrr} 8 & -3 & 0 & -7 & 2 \\ -9 & 4 & 5 & 11 & -7 \\ 6 & -2 & 2 & -4 & 4 \\ 5 & -1 & 7 & 0 & 10 \\ \end{array} \right] \sim \quad \cdots \quad \sim \left[ \begin{array}{ccccc} 1 & 0 & 3 & 1 & 0 \\ 0 & 1 & 8 & 5 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 \\ \end{array} \right] \]

  Problem 42 \[ \left[ \begin{array}{rrrrrr} 12 & 10 & -6 & -3 & 7 & 10 \\ -7 & -6 & 4 & 7 & -9 & 5 \\ 9 & 9 & -9 & -5 & 5 & -1 \\ -4 & -3 & 1 & 6 & -8 & 9 \\ 8 & 7 & -5 & -9 & 11 & -8 \\ \end{array} \right] \sim \quad \cdots \quad \sim \left[ \begin{array}{rrrrrr} 1 & 0 & 2 & 0 & 2 & 0 \\ 0 & 1 & -3 & 0 & -2 & 0 \\ 0 & 0 & 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ \end{array} \right] \]

• Below I am offering a simpler matrix and more detailed guidance for the calculations needed for Problems 41, 42, 43, 44 in Section 1.7. This is my version of these four problems.
  Problem. Consider the following $4\times 6$ matrix \[ A = \left[ \begin{array}{cccccc} 1 & 1 & 4 & 1 & 0 & 6 \\ 2 & 1 & 3 & 0 & 1 & 4 \\ 3 & 1 & 2 & 1 & 0 & 8 \\ 4 & 1 & 1 & 0 & 1 & 6 \\ \end{array} \right]. \] Denote the columns of $A$ by $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_3,$ $\mathbf{a}_4,$ $\mathbf{a}_5,$ and $\mathbf{a}_6.$ That is, \[ \mathbf{a}_1 = \left[ \begin{array}{cccccc} 1 \\ 2 \\ 3 \\ 4 \\ \end{array} \right], \ \mathbf{a}_2 = \left[ \begin{array}{c} 1\\ 1 \\ 1 \\ 1 \\ \end{array} \right], \ \mathbf{a}_3 = \left[ \begin{array}{c} 4 \\ 3 \\ 2 \\ 1 \\ \end{array} \right], \ \mathbf{a}_4 = \left[ \begin{array}{c} 1 \\ 0 \\ 1 \\ 0 \\ \end{array} \right], \ \mathbf{a}_5 = \left[ \begin{array}{c} 0\\ 1\\ 0\\ 1 \\ \end{array} \right], \ \mathbf{a}_6 = \left[ \begin{array}{c} 6 \\ 4 \\ 8 \\ 6 \\ \end{array} \right]. \] Answer the following questions:

  1. Find the Reduced Row Echelon Form of $A$. (Please make sure that it is done correctly.)
  2. Is the vector $\mathbf{a}_1$ linearly independent? Explain using the definition.
  3. Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2$ linearly independent? Explain using the definition.
  4. Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_3$ linearly independent? Explain using the definition. If the answer is No, then express $\mathbf{a}_3$ as a linear combination of $\mathbf{a}_1$ and $\mathbf{a}_2.$ That is, find the real numbers $\alpha_1$ and $\alpha_2$ such that \[ \mathbf{a}_3 = \alpha_1 \mathbf{a}_1+ \alpha_2 \mathbf{a}_2. \] Make a record of the numbers $\alpha_1$ and $\alpha_2.$
  5. Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_4$ linearly independent? Explain using the definition.
  6. Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_4,$ $\mathbf{a}_5$ linearly independent? Explain using the definition. If the answer is No, then express $\mathbf{a}_5$ as a linear combination of $\mathbf{a}_1,$ $\mathbf{a}_2$ and $\mathbf{a}_4.$ That is, find the real numbers $\beta_1,$ $\beta_2$ and $\beta_4$ such that \[ \mathbf{a}_5 = \beta_1 \mathbf{a}_1+ \beta_2 \mathbf{a}_2 + \beta_4 \mathbf{a}_4. \] Make a record of the numbers $\beta_1,$ $\beta_2$ and $\beta_4.$
  7. Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_4,$ $\mathbf{a}_6$ linearly independent? Explain using the definition. If the answer is No, then express $\mathbf{a}_6$ as a linear combination of $\mathbf{a}_1,$ $\mathbf{a}_2$ and $\mathbf{a}_4.$ That is, find the real numbers $\gamma_1,$ $\gamma_2$ and $\gamma_4$ such that \[ \mathbf{a}_6 = \gamma_1 \mathbf{a}_1+ \gamma_2 \mathbf{a}_2 + \gamma_4 \mathbf{a}_4. \] Make a record of the numbers $\gamma_1,$ $\gamma_2$ and $\gamma_4.$
  8. The moral of this problem is that all the information you found about the linear independence and dependence of the columns of $A$ is contained in the Reduced Row Echelon Form of $A$. I would like you to carefully review your findings and discover how are all the facts about the columns of $A$ that you found in this problem encoded in the Reduced Row Echelon Form of $A$.
• Let me answer some of the questions asked in the above boxed Problem.
  1. The row reduction algorithm applied to the matrix $A$ gives its Reduced Row Echelon Form: \[ A = \Bigl[ \begin{array}{cccccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 & \mathbf{a}_4 & \mathbf{a}_5 & \mathbf{a}_6 \end{array} \Bigr] = \left[ \begin{array}{cccccc} 1 & 1 & 4 & 1 & 0 & 6 \\ 2 & 1 & 3 & 0 & 1 & 4 \\ 3 & 1 & 2 & 1 & 0 & 8 \\ 4 & 1 & 1 & 0 & 1 & 6 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & -1 & 0 & 0 & 1 \\ 0 & 1 & 5 & 0 & 1 & 2 \\ 0 & 0 & 0 & 1 & -1 & 3 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ \end{array} \right]. \] Remark We obtained the preceding RREF by performing row operations on the rows of $A.$ It is important to realize that the same row operations would lead to the following Reduced Row Echelon Forms: \[ \Bigl[ \begin{array}{cc} \mathbf{a}_1 & \mathbf{a}_2 \end{array} \Bigr] = \left[ \begin{array}{cc} 1 & 1 \\ 2 & 1 \\ 3 & 1 \\ 4 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \\ 0 & 0 \\ 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{ccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 \end{array} \Bigr] = \left[ \begin{array}{ccc} 1 & 1 & 4 \\ 2 & 1 & 3 \\ 3 & 1 & 2 \\ 4 & 1 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & -1 \\ 0 & 1 & 5 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{ccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_4 \end{array} \Bigr] = \left[ \begin{array}{ccc} 1 & 1 & 1 \\ 2 & 1 & 0 \\ 3 & 1 & 1 \\ 4 & 1 & 0 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{cccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_4 & \mathbf{a}_5 \end{array} \Bigr] = \left[ \begin{array}{cccc} 1 & 1 & 1 & 0 \\ 2 & 1 & 0 & 1 \\ 3 & 1 & 1 & 0 \\ 4 & 1 & 0 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & -1 \\ 0 & 0 & 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{cccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_4 & \mathbf{a}_5 \end{array} \Bigr] = \left[ \begin{array}{cccc} 1 & 1 & 1 & 0 \\ 2 & 1 & 0 & 1 \\ 3 & 1 & 1 & 0 \\ 4 & 1 & 0 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & -1 \\ 0 & 0 & 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{cccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_4 & \mathbf{a}_6 \end{array} \Bigr] = \left[ \begin{array}{cccc} 1 & 1 & 1 & 6 \\ 2 & 1 & 0 & 4 \\ 3 & 1 & 1 & 8 \\ 4 & 1 & 0 & 6 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & 0 & 1 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 0 \\ \end{array} \right]. \]
  2. The vector $\mathbf{a}_1$ is linearly independent. Recall the definition of linear independence. A vector $\mathbf{v}_1$ is linearly independent if the homogeneous equation $x_1 \mathbf{v}_1 = \mathbf{0}$ has only the trivial solution. Consider the homogeneous equation $x_1 \mathbf{a}_1 = \mathbf{0}.$ The preceding equation is equivalent to \[ x_1 \left[ \begin{array}{cccccc} 1 \\ 2 \\ 3 \\ 4 \\ \end{array} \right] = \left[ \begin{array}{cccccc} 0 \\ 0 \\ 0 \\ 0 \\ \end{array} \right], \] which in turn is equivalent to $x_1 = 0.$ Hence the equation $x_1 \mathbf{a}_1 = \mathbf{0}$ has only the trivial solution. This proves that the vector $\mathbf{a}_1$ is linearly independent.
  3. The vectors $\mathbf{a}_1$ and $\mathbf{a}_2$ are linearly independent. To prove this claim we need to prove that the homogeneous vector equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{0}$ has only the trivial solution. Since we already know that \[ \Bigl[ \begin{array}{cc} \mathbf{a}_1 & \mathbf{a}_2 \end{array} \Bigr] = \left[ \begin{array}{cc} 1 & 1 \\ 2 & 1 \\ 3 & 1 \\ 4 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \\ 0 & 0 \\ 0 & 0 \\ \end{array} \right], \] the equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{0}$ is equivalent to the equation \[ x_1 \left[ \begin{array}{c} 1 \\ 0 \\ 0 \\ 0 \\ \end{array} \right] + x_2 \left[ \begin{array}{c} 0 \\ 1 \\ 0 \\ 0 \\ \end{array} \right] = \left[ \begin{array}{c} 0 \\ 0 \\ 0 \\ 0 \\ \end{array} \right] \] and the last equation is equivalent to $x_1 = 0$ and $x_2 =0$. Hence $\mathbf{a}_1$ and $\mathbf{a}_2$ are linearly independent.
  4. The vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_3$ are not linearly independent. These vectors are linearly dependent. We claim that the homogeneous vector equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + x_3 \mathbf{a}_3 = \mathbf{0}$ has infinitely many solution. Since we already know that \[ \Bigl[ \begin{array}{ccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 \end{array} \Bigr] = \left[ \begin{array}{ccc} 1 & 1 & 4 \\ 2 & 1 & 3 \\ 3 & 1 & 2 \\ 4 & 1 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccr} 1 & 0 & -1 \\ 0 & 1 & 5 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} \right], \] the equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + x_3 \mathbf{a}_3= \mathbf{0}$ is equivalent to the equation \[ x_1 \left[ \begin{array}{c} 1 \\ 0 \\ 0 \\ 0 \\ \end{array} \right] + x_2 \left[ \begin{array}{c} 0 \\ 1 \\ 0 \\ 0 \\ \end{array} \right] + x_2 \left[ \begin{array}{r} -1 \\ 5 \\ 0 \\ 0 \\ \end{array} \right]= \left[ \begin{array}{c} 0 \\ 0 \\ 0 \\ 0 \\ \end{array} \right]. \] In the last equation the variable $x_3$ is free. All solutions of this equation are given by $x_1 = s,$ $x_2 = -5s,$ $x_3 = s$ where $s$ is any real number. For $s=1$ we get a specific linear dependence relation \[ 1 \mathbf{a}_1 + (-5) \mathbf{a}_2 + 1 \mathbf{a}_3 = \mathbf{0}. \] A different way to get the same relation is to solve the nonhomogeneous equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{a}_3.$ It is very interesting to observe that we already have everything that we need to solve this equation. We already know that \[ \Bigl[ \begin{array}{cc|c} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 \end{array} \Bigr] = \left[ \begin{array}{ccc} 1 & 1 & 4 \\ 2 & 1 & 3 \\ 3 & 1 & 2 \\ 4 & 1 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{cc|r} 1 & 0 & -1 \\ 0 & 1 & 5 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} \right]. \] Therefore, $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{a}_3$ is equivalent to the equation \[ x_1 \left[ \begin{array}{c} 1 \\ 0 \\ 0 \\ 0 \\ \end{array} \right] + x_2 \left[ \begin{array}{c} 0 \\ 1 \\ 0 \\ 0 \\ \end{array} \right] = \left[ \begin{array}{r} -1 \\ 5 \\ 0 \\ 0 \\ \end{array} \right]. \] Hence the only solution of the nonhomogeneous equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{a}_3$ is $x_1 = -1$ and $x_2 = 5.$ That is \[ \mathbf{a}_3 = (-1) \mathbf{a}_1 +5 \mathbf{a}_2. \] Verify! \[ \left[ \begin{array}{r} 4 \\ 3 \\ 2 \\ 1 \\ \end{array} \right] = (-1) \left[ \begin{array}{c} 1 \\ 2 \\ 3 \\ 4 \\ \end{array} \right] + 5 \left[ \begin{array}{c} 1 \\ 1 \\ 1 \\ 1 \\ \end{array} \right]. \]
  5. Similar to III.
  6. Similar to IV.
  7. Similar to IV and VI.
  8. Would you please think about this? The connections that are present in RREF are beautiful. We will discuss these many times in future classes.

---

Wednesday, October 9, 2024

---

---

Tuesday, October 8, 2024 (updated with color coded matrices, three dimensions still to come)

---

• We discussed Section 1.6: Applications of Linear Systems. We did four applications. Two from this section: Balancing Chemical Equations and Network Flow. Suggested problems for Section 1.6: 5-8, 11-14. Please pay attention to Problem 11.
• The third application was finding an approximation for the steady-state temperature distribution of a thin plate when the temperature around the boundary is known. This is explained in Problems 33 and 34 in Section 1.1. These problems explain how to find an approximation for the steady-state temperature distribution of a thin plate when the temperature around the boundary is known. In Problems 33 and 34, approximations for the steady-state temperature distribution are found at only four points. However, this problem gives a blueprint on how to find the approximations at any rectangular grid of points. Below, I do it for a \(2\times 3\) grid of six points and a \(3\times 4\) grid of twelve points
  • Consider the following grid which represents a heated plate: Temperatures at green points are known and given by the indexed symbols \(T\), while the temperatures at six red points are unknown. The symbols for the corresponding unknowns are given next to red points. The temperature at each red point is the average of the temperatures at the closest points. Writing down these six equalities we get the following nonhomogeneous linear system: \begin{align*} x_{1,1} & = \frac{1}{4} \bigl(T_{0,1}+T_{1,0}+x_{1,2} +x_{2,1}\bigr) \\ x_{1,2} & = \frac{1}{4} \bigl(T_{0,2}+x_{1,1}+T_{1,3}+x_{2,2}\bigr) \\ x_{2,1} & = \frac{1}{4} \bigl(x_{1,1}+T_{2,0}+x_{2,2}+x_{3,1}\bigr) \\ x_{2,2} & = \frac{1}{4} \bigl(x_{1,2}+x_{2,1}+T_{2,3}+x_{3,2}\bigr) \\ x_{3,1} & = \frac{1}{4} \bigl(x_{2,1}+T_{3,0}+x_{3,2}+T_{4,1}\bigr)\\ x_{3,2} & = \frac{1}{4} \bigl(x_{2,2}+x_{3,1}+T_{3,3}+T_{4,2} \bigr) \end{align*} Let us now write this system in matrix form: \begin{equation*} \left[\! \begin{array}{rrrrrr} 1 &-\frac{1}{4} & -\frac{1}{4} & 0 & 0 & 0 \\[5pt] -\frac{1}{4} & 1 & 0 & -\frac{1}{4} & 0 & 0 \\[5pt] -\frac{1}{4} & 0 & 1 & -\frac{1}{4} & -\frac{1}{4} & 0 \\[5pt] 0 & -\frac{1}{4} & -\frac{1}{4} & 1 & 0 & -\frac{1}{4} \\[5pt] 0 & 0 & -\frac{1}{4} & 0 & 1 & -\frac{1}{4} \\[5pt] 0 & 0 & 0 & -\frac{1}{4} & -\frac{1}{4} & 1 \\[5pt] \end{array} \!\right] \left[\!\begin{array}{c} x_{1,1} \\[5pt] x_{1,2} \\[5pt] x_{2,1} \\[5pt] x_{2,2} \\[5pt] x_{3,1} \\[5pt] x_{3,2} \end{array}\! \right] = \frac{1}{4} \left[\!\begin{array}{c} T_{0,1}+ T_{1,0} \\[5pt] T_{0,2}+ T_{1,3} \\[5pt] T_{2,0} \\[5pt] T_{2,3} \\[5pt] T_{3,0}+ T_{4,1} \\[5pt] T_{3,3}+ T_{4,2} \end{array}\! \right] \end{equation*} The above matrix has special block structure that I will try to emphasize with colors. I also ignore all zero entries. \begin{equation*} \left[\! \begin{array}{ccc} \bbox[#50FF50]{\begin{array}{r@{\hspace{-2pt}}r@{\hspace{-2pt}}} 1 & -\frac{1}{4} \\[5pt] -\frac{1}{4} & 1 \end{array}} & \bbox[#FFFF25]{\begin{array}{rr} -\frac{1}{4} & \\[5pt] & -\frac{1}{4} \end{array}} & \begin{array}{rr} & \\[5pt] & \end{array} \\ \bbox[#FFFF25]{\begin{array}{rr} -\frac{1}{4} & \\[5pt] & -\frac{1}{4} \end{array}} & \bbox[#50FF50]{\begin{array}{rr} 1 & -\frac{1}{4} \\[5pt] -\frac{1}{4} & 1 \end{array}} & \bbox[#FFFF25]{\begin{array}{rr} -\frac{1}{4} & \\[5pt] & -\frac{1}{4} \end{array}} \\ \begin{array}{rr} & \\[5pt] & \end{array} & \bbox[#FFFF25]{\begin{array}{rr} -\frac{1}{4} & \\[5pt] & -\frac{1}{4} \end{array}} & \bbox[#50FF50]{\begin{array}{rr} 1 & -\frac{1}{4} \\[5pt] -\frac{1}{4} & 1 \end{array}} \end{array} \!\right] \end{equation*} With the specific values \[ T_{0, 1} = 1, \quad T_{0, 2} = 3, \quad T_{1, 3} = 3, \quad T_{2, 3} = 2, \quad T_{3, 3} = 1, \quad T_{4, 2} = 2, \quad T_{4, 1} = 4, \quad T_{3, 0}= 3, \quad T_{2, 0} = 2, \quad T_{1, 0}= 1, \] the augmented matrix is \[ \left[\!\begin{array}{cccccc|c} 1 & -\frac{1}{4} & -\frac{1}{4} & 0 & 0 & 0 & \frac{1}{2} \\[5pt] -\frac{1}{4} & 1 & 0 & -\frac{1}{4} & 0 & 0 & \frac{3}{2} \\[5pt] -\frac{1}{4} & 0 & 1 & -\frac{1}{4} & -\frac{1}{4} & 0 & \frac{1}{2} \\[5pt] 0 & -\frac{1}{4} & -\frac{1}{4} & 1 & 0 & -\frac{1}{4} & \frac{1}{2} \\[5pt] 0 & 0 & -\frac{1}{4} & 0 & 1 & -\frac{1}{4} & \frac{7}{4} \\[5pt] 0 & 0 & 0 & -\frac{1}{4} & -\frac{1}{4} & 1 & \frac{3}{4} \\[5pt] \end{array}\! \right] \] and Mathematica row reduces it to the following Reduced Row Echelon Form \[ \left[\! \begin{array}{cccccc|c} 1 & 0 & 0 & 0 & 0 & 0 & \frac{173}{105} \\ 0 & 1 & 0 & 0 & 0 & 0 & \frac{257}{105} \\ 0 & 0 & 1 & 0 & 0 & 0 & \frac{15}{7} \\ 0 & 0 & 0 & 1 & 0 & 0 & \frac{15}{7} \\ 0 & 0 & 0 & 0 & 1 & 0 & \frac{292}{105} \\ 0 & 0 & 0 & 0 & 0 & 1 & \frac{208}{105} \\ \end{array}\! \right] \] Thus the solution, written as a matrix is \[ \left[\! \begin{array}{cc} x_{1,1} & x_{1,2}\\ x_{2,1} & x_{2,2} \\ x_{3,1} & x_{3,2} \end{array}\! \right] = \left[\! \begin{array}{cc} \frac{173}{105} & \frac{257}{105}\\ \frac{15}{7} & \frac{15}{7} \\ \frac{292}{105} & \frac{208}{105} \end{array}\! \right] \approx \left[\! \begin{array}{cc} 1.64762 & 2.44762 \\ 2.14286 & 2.14286 \\ 2.78095 & 1.98095 \\ \end{array}\! \right]. \]
  • Consider the following grid which represents a heated plate: Temperatures at green points are known and given by the indexed symbols \(T\), while the temperatures at six red points are unknown. The symbols for the corresponding unknowns are given next to red points. The temperature at each red point is the average of the temperatures at the closest points. Writing down these six equalities we get the following nonhomogeneous linear system: \begin{align*} x_{1,1} &= \frac{1}{4} \bigl( T_{0,1}+T_{1,0}+x_{1,2}+x_{2,1} \bigr)\\ x_{1,2} & =\frac{1}{4} \bigl( T_{0,2}+x_{1,1}+x_{1,3}+x_{2,2} \bigr)\\ x_{1,3} & =\frac{1}{4} \bigl( T_{0,3}+x_{1,2}+x_{1,4}+x_{2,3} \bigr)\\ x_{1,4} & =\frac{1}{4} \bigl( T_{0,4}+x_{1,3}+T_{1,5}+x_{2,4} \bigr)\\ x_{2,1} & =\frac{1}{4} \bigl( x_{1,1}+T_{2,0}+x_{2,2}+x_{3,1} \bigr)\\ x_{2,2} &=\frac{1}{4} \bigl( x_{1,2}+x_{2,1}+x_{2,3}+x_{3,2} \bigr)\\ x_{2,3} &=\frac{1}{4} \bigl( x_{1,3}+x_{2,2}+x_{2,4}+x_{3,3} \bigr)\\ x_{2,4} & =\frac{1}{4}\bigl( x_{1,4}+x_{2,3}+T_{2,5}+x_{3,4} \bigr) \\ x_{3,1} & =\frac{1}{4} \bigl( x_{2,1} + T_{3,0}+x_{3,2}+T_{4,1} \bigr) \\ x_{3,2} & =\frac{1}{4} \bigl( x_{2,2}+x_{3,1}+x_{3,3}+T_{4,2}\bigr)\\ x_{3,3}& =\frac{1}{4} \bigl( x_{2,3}+x_{3,2}+x_{3,4}+T_{4,3}\bigr)\\ x_{3,4}&=\frac{1}{4} \bigl( x_{2,4}+x_{3,3}+T_{3,5}+T_{4,4}\bigr) \end{align*} Let us now write this system in matrix form: \begin{equation*} \left[\! \begin{array}{rrrrrrrrrrrr} 1 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\[5pt] -\frac{1}{4} & 1 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 0 & 0 & 0 & 0 & 0 & 0 \\[5pt] 0 & -\frac{1}{4} & 1 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 0 & 0 & 0 & 0 & 0 \\[5pt] 0 & 0 & -\frac{1}{4} & 1 & 0 & 0 & 0 & -\frac{1}{4} & 0 & 0 & 0 & 0 \\[5pt] -\frac{1}{4} & 0 & 0 & 0 & 1 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 0 & 0 & 0 \\[5pt] 0 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 1 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 0 & 0 \\[5pt] 0 & 0 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 1 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 0 \\[5pt] 0 & 0 & 0 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 1 & 0 & 0 & 0 & -\frac{1}{4} \\[5pt] 0 & 0 & 0 & 0 & -\frac{1}{4} & 0 & 0 & 0 & 1 & -\frac{1}{4} & 0 & 0 \\[5pt] 0 & 0 & 0 & 0 & 0 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 1 & -\frac{1}{4} & 0 \\[5pt] 0 & 0 & 0 & 0 & 0 & 0 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 1 & -\frac{1}{4} \\[5pt] 0 & 0 & 0 & 0 & 0 & 0 & 0 & -\frac{1}{4} & 0 & 0 & -\frac{1}{4} & 1 \\[5pt] \end{array} \!\right] \left[\!\begin{array}{c} x_{1,1} \\[5pt] x_{1,2} \\[5pt] x_{1,3} \\[5pt] x_{1,4} \\[5pt] x_{2,1} \\[5pt] x_{2,2} \\[5pt] x_{2,3} \\[5pt] x_{2,4} \\[5pt] x_{3,1} \\[5pt] x_{3,2} \\[5pt] x_{3,3} \\[5pt] x_{3,4} \end{array}\! \right] = \frac{1}{4} \left[\!\begin{array}{c} T_{0,1}+T_{1,0}\\[5pt] T_{0,2} \\[5pt] T_{0,3} \\[5pt] T_{0,4}+T_{1,5}\\[5pt] T_{2,0} \\[5pt] 0 \\[5pt] 0 \\[5pt] T_{2,5}\\[5pt] T_{3,0}+T_{4,1}\\[5pt] T_{4,2}\\[5pt] T_{4,3} \\[5pt] T_{3,5}+T_{4,4} \end{array}\! \right] \end{equation*} The above matrix has special block structure that I will try to emphasize with colors. I also ignore all zero entries. \begin{equation*} \left[\! \begin{array}{ccc} \bbox[#50FF50]{\begin{array}{rrrr} 1 & -\frac{1}{4} & & \\[5pt] -\frac{1}{4} & 1 & -\frac{1}{4} & \\[5pt] & -\frac{1}{4} & 1 & -\frac{1}{4} \\[5pt] & & -\frac{1}{4} & 1 \end{array}} & \bbox[#FFFF25]{\begin{array}{rrrr} -\frac{1}{4} & & & \\[5pt] & -\frac{1}{4} & & \\[5pt] & & -\frac{1}{4} & \\[5pt] & & & -\frac{1}{4} \end{array}} & \begin{array}{rrrr} & & & \\[5pt] & & & \\[5pt] & & & \\[5pt] & & & \end{array} \\ \bbox[#FFFF25]{\begin{array}{rrrr} -\frac{1}{4} & & & \\[5pt] & -\frac{1}{4} & & \\[5pt] & & -\frac{1}{4} & \\[5pt] & & & -\frac{1}{4} \end{array}} & \bbox[#50FF50]{\begin{array}{rrrr} 1 & -\frac{1}{4} & & \\[5pt] -\frac{1}{4} & 1 & -\frac{1}{4} & \\[5pt] & -\frac{1}{4} & 1 & -\frac{1}{4} \\[5pt] & & -\frac{1}{4} & 1 \end{array}} & \bbox[#FFFF25]{\begin{array}{rrrr} -\frac{1}{4} & & & \\[5pt] & -\frac{1}{4} & & \\[5pt] & & -\frac{1}{4} & \\[5pt] & & & -\frac{1}{4} \end{array}} \\ \begin{array}{rrrr} & & & \\[5pt] & & & \\[5pt] & & & \\[5pt] & & & \end{array} & \bbox[#FFFF25]{\begin{array}{rrrr} -\frac{1}{4} & & & \\[5pt] & -\frac{1}{4} & & \\[5pt] & & -\frac{1}{4} & \\[5pt] & & & -\frac{1}{4} \end{array}} & \bbox[#50FF50]{\begin{array}{rrrr} 1 & -\frac{1}{4} & & \\[5pt] -\frac{1}{4} & 1 & -\frac{1}{4} & \\[5pt] & -\frac{1}{4} & 1 & -\frac{1}{4} \\[5pt] & & -\frac{1}{4} & 1 \end{array}} \end{array} \!\right] \end{equation*}
  • The following grid has \(20\times 30 = 600\) unknowns and \(2\times (20+30) = 100\) given temperature values.

    I did not try it, but I believe that Mathematica would solve the corresponding system of \(600\) equations with \(600\) unknowns relatively quickly and that would give a good approximation of the heat distribution in this plate, The advantage of this system us that it has a lot of zeros. The matrix of this system is of the size \(600\times 600\). Thus it has \(360,000\) entries. Based on the patterns that we observed for small matrices above, I estimate that this \(600\times 600\) matrix has less than \(3,000\) nonzero entries. That is less than 1 in 120 entries is nonzero; that is less than 0.083%.

    Matrices with a small percentage of nonzero entries are called sparse matrices. Mathematicians have developed super efficient methods of doing calculations with sparse matrices of huge size.

  • In Math 430 we show how to find the steady-state temperature distribution in a thin plate when the temperature around the boundary is known using partial differential equations.

• My favorite application of vectors: COLORS. In fact, I love this application so much that I wrote a webpage to celebrate it: Color Cube.

  One exercise in this context would be to ask you to find three colors which are between teal and yellow, one in the middle between teal and yellow and the other two in the middle between teal and the mid-color and in the middle of mid-color and yellow.

---

Monday, October 7, 2024

---

• Today we finished Section 1.4: The Matrix Equation: $A\mathbf{x} = \mathbf{b}$ and Section 1.5: Solution Sets Of Linear Systems.

• Section 1.5 talks about writing solution sets of linear systems in parametric vector form.

  We explained the relationship between the solution set of the homogeneous equation \[ \color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0} \] and the solution set of a consistent nonhomogeneous equation \[ \color{green}{A}\color{red}{\mathbf{x}} = \color{green}{\mathbf{b}}. \] This is explained in Theorem 6 in the book. Please recognize how this theorem is reflected when the solution of $A \mathbf x = \mathbf b$ is written in parametric vector form.

  Suggested problems for Section 1.5: 1, 3, 5, 6, 7, 9, 11, 12, 13-16, 19, 21, 23, 24, 26, 29, 32, 35, 37-40. When you write a formula for the solution of a nonhomogeneous equation in parametric form, try to recognize a particular solution $\color{purple}{\mathbf{p}}$ of the nonhomogeneous equation and a span of one, or two, or three vectors which is the solution of the corresponding homogeneous equation.

• Let $m, n \in \mathbb{N}.$ Let $\color{green}{A}$ be an $m\times n$ matrix. A few points about the solution set of a homogeneous equation \[ \color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}. \] The preceding matrix equation can be rewritten as a homogenous system with $m$ equations and $n$ unknowns. The unknowns are collected in the vector $\color{red}{\mathbf{x}} \in \mathbb{R}^n.$ Here $\mathbf{0}$ is the zero vector in $\mathbb{R}^m.$
  • The homogeneous equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ always has a solution (it is consistent). One solution is always the zero vector in $\mathbb{R}^n.$
  • Another explanation why the homogeneous equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ always has a solution is to consider its augmented matrix \[ \bigl[\begin{array}{c|c} \color{green}{A} & \mathbf{0} \end{array} \bigr]. \] As we perform row reduction, the last column will always be $\mathbf{0}.$ Therefore the last column of the reduced row echelon form of the matrix $\bigl[\begin{array}{c|c} \color{green}{A} & \mathbf{0} \end{array} \bigr]$ will not be a pivot column. Hence the matrix equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ is consistent.
  • We have the following dichotomy for the solution:
    • The zero vector in $\mathbb{R}^n$ is a unique solution. This case occurs if and only if each column of the reduced row echelon form of $\color{green}{A}$ is a pivot column. In other words, there are no free variables.
    • There are infinitely many solutions. This case occurs if and only if there are free variables. In this case the solution set can be written as a span of several vectors. In fact, we can write the solution set as a span of as many vectors as there are free variables.
• For example, let $\color{green}{A}$ be a $3\times 4$ matrix and assume that \[ \bigl[\begin{array}{c|c} \color{green}{A} & \mathbf{0} \end{array} \bigr] \sim \cdots \sim \left[\!\begin{array}{rrrr|r} 1 & -2 & 0 & -1 & 0 \\ 0 & 0 & 1 & 2 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array}\! \right] \] In this case we have two free variables and the solution set of the matrix equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ can be written as a span of two vectors in $\mathbb{R}^4.$ To find those vectors we need to solve the corresponding system of linear equations: \begin{alignat*}{10} &x_1 & & - &&2 x_2 & & && && && - && && x_4 && = &&0\\ & & & && & & && &&x_3 && + && 2 && x_4 && = &&0 \end{alignat*} The solution of this system in parametric vector form is \[ \left[\!\begin{array}{c} x_1 \\ x_2 \\ x_3 \\ x_4 \end{array}\! \right] = \left[\!\begin{array}{c} 2 s + t \\ s \\ -2 t \\ t \end{array}\! \right] = \left[\!\begin{array}{c} 2 s + 1 t \\ 1 s + 0 t \\ 0 s -2 t \\ 0 s + 1 t \end{array}\! \right] = s \left[\!\begin{array}{c} 2 \\ 1 \\ 0 \\ 0 \end{array}\! \right] + t \left[\!\begin{array}{r} 1 \\ 0 \\ -2 \\ 1 \end{array}\! \right], \] where $s$ and $t$ are arbitrary real numbers. Hence, the solution set of the homogeneous matrix equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ can be written as \[ \operatorname{Span}\left\{ \left[\!\begin{array}{c} 2 \\ 1 \\ 0 \\ 0 \end{array}\! \right], \left[\!\begin{array}{r} 1 \\ 0 \\ -2 \\ 1 \end{array}\! \right] \right\}. \]

• At the beginning of class today you asked me about my favorite color. The answer is TEAL. I love using colors on my websites. On my websites, or in Mathematica, to chose colors I always use RGB (red-green-blue) color encoding. That is expression which encodes each color as a triple of numbers. In HTML the triple for TEAL is written as a #008080. Here HTML uses two three two digit numbers in hexadecimal number system. The hexadecimal digits are 0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F. HTML encodes colors with triples of integers between 0, which they write as 00 and 255 which in hexadecimal number system is FF. In hexadecimal number system, the number 80 stands for 128, half-way between 0 and 255.

  It is hard to see a vector in #008080. However, in CSS, the triple for TEAL can be written as rgb(0%,50%,50%). This is getting closer to a vector, when you view percentages as their fractional values. So, the vector (0%,50%,50%) is (0,1/2,1/2). So, each color can be identified by a vector whose components are real numbers between 0 and 1.

  So I started the class by talking about the colors in relation to linear algebra. I love the application of vectors to COLORS so much that I wrote a webpage to celebrate it: Color Cube.

• Did you work on Problem 32 in Section 1.3? I find this problem interesting. I like it so much that I decided to rewrite it and give more information in the associated pictures. I do it in the next item.

  An interesting feature of the Problem in the next item is that you do not need to know the specific coordinates of the vectors in the picture to answer the questions. You only need to record the linear relationships among vectors that are clear from the given grids: the vectors $\color{green}{\mathbf{a}_3},$ and $\color{#00FF00}{\mathbf{b}}$ are linear combinations of $\color{green}{\mathbf{a}_1},$ $\color{green}{\mathbf{a}_2}.$ At this point you can solve items (i) and (iii) in Problem in the next item. After you solved item (iii), you can reconstruct (ii) But, you will have more information to answer solve (ii) as we learn more about RREF. An important note: The Problem in the next item will be on the final assignment. So, solving it early, and ask for clarifications if there is something not clear, assures success.

  The content of Section 1.5 is very useful for the Problem below.

• Problem  Consider the vectors $\color{green}{\mathbf{a}_1},$ $\color{green}{\mathbf{a}_2},$ $\color{green}{\mathbf{a}_3},$ and $\color{#00FF00}{\mathbf{b}}$ in $\mathbb{R}^2$ as shown in the picture below. Answer the following questions and justify your answers.
  1. Is the vector equation \[ \color{red}{x_1} \color{green}{\mathbf{a}_1} + \color{red}{x_2} \color{green}{\mathbf{a}_2} + \color{red}{x_3} \color{green}{\mathbf{a}_3} = \color{#00FF00}{\mathbf{b}} \] consistent or inconsistent? Explain your reasoning.
  2. Find the reduced row echelon form of the augmented matrix \[ \left[\begin{array}{ccc|c} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \color{green}{\mathbf{a}_3} & \color{#00FF00}{\mathbf{b}} \end{array} \right]. \]
  3. Find the solution set of the vector equation \[ \color{red}{x_1} \color{green}{\mathbf{a}_1} + \color{red}{x_2} \color{green}{\mathbf{a}_2} + \color{red}{x_3} \color{green}{\mathbf{a}_3} = \color{#00FF00}{\mathbf{b}} \] and represent the solution set in parametric vector form.
  If you can solve item (ii) in the above problem, then the solution of item (iii) is clear. However, one strategy is to use what we learned in Section 1.5 to solve c. first and then from c. deduce what b. must be.

---

Saturday, October 5, 2024

---

• We finished Section 1.4 and started Section 1.5. Suggested problems for Section 1.4: 1, 5, 13, 14, 15, 16,17-20, 21, 22, 23, 24, 25, 26, 32, 34, 35, 36 and for Section 1.5: 1, 3, 5, 6, 7, 9, 11, 12, 13-16, 19, 21, 23, 24, 26, 29, 32, 35, 37-40.
• Notice that Problems 25 26, 27, and 28 in Section 1.1 are related to Theorem 4 in Section 1.4. These are interesting deep problems. Also notice that how these problems are related to the concept of span. I wrote about this in an item on October 3; the box starting with "Find a relationship between the coordinates ..."
• On Friday, I illustrated content of Sections 1.4 and 1.5 on the following modification of an example that we did before: \begin{alignat*}{5} \require{amsmath} 3 \color{red}{x_{1}} & + & 0 \color{red}{x_{2}} & + & (-1) \color{red}{x_{3}} & + & 1 \color{red}{x_{4}} & = & -2& \\ 1 \color{red}{x_{1}} & + & 1 \color{red}{x_{2}} & + & (-2) \color{red}{x_{3}} & + & 2 \color{red}{x_{4}} & = & 3& \\ 2 \color{red}{x_{1}} & + & (-1) \color{red}{x_{2}} & + & 1 \color{red}{x_{3}} & + & 3 \color{red}{x_{4}} & = & 8 & \\ \end{alignat*} Let us rewrite the above system as a vector equation: \begin{equation*} \color{red}{x_{1}} \left[\! \begin{array}{r} 3 \\ 1\\ 2 \end{array} \!\right] + \color{red}{x_{2}} \left[\! \begin{array}{r} 0 \\1\\ -1 \end{array} \!\right] + \color{red}{x_{3}} \left[\! \begin{array}{r} -1 \\-2\\ 1 \end{array} \!\right] + \color{red}{x_{4}} \left[\! \begin{array}{r} 1 \\ 2\\ 3 \end{array} \!\right] = \left[\! \begin{array}{r} -2 \\ 3 \\ 8 \end{array} \!\right] . \end{equation*} Let us rewrite the above vector equation as a matrix equation: \begin{equation*} \left[\!\begin{array}{rrrr} 3 & 0 & -1 & 1 \\ 1 & 1 & -2 & 2 \\ 2 & -1 & 1 & 3 \end{array}\! \right] \left[\! \begin{array}{c} \color{red}{x_{1}} \\ \color{red}{x_{2}}\\ \color{red}{x_{3}} \\ \color{red}{x_{4}} \end{array} \!\right] = \left[\! \begin{array}{r} -2 \\ 3 \\ 8 \end{array} \!\right] . \end{equation*} To solve any of the any of the above, the system, the vector equation, the matrix equation we form the augmented matrix and row reduce it: \begin{equation*} \left[\!\begin{array}{rrrr|r} 3 & 0 & -1 & 1 & -2 \\ 1 & 1 & -2 & 2 & 3 \\ 2 & -1 & 1 & 3 & 8 \end{array}\! \right] \ \ \sim \quad \cdots \quad \sim \ \ \left[\!\begin{array}{rrrr|r} 1 & 0 & -\frac{1}{3} & 0 & -\frac{7}{4} \\ 0 & 1 & -\frac{5}{3} & 0 & -\frac{7}{4} \\ 0 & 0 & 0 & 1 & \frac{13}{4} \end{array}\! \right] \end{equation*} Thus, the solution exists since the augmented column is not a pivot column. There is one free variable, so \(\color{red}{x_{3}} = s\) is free. To find the solution, we write the system corresponding to the RREF: \begin{alignat*}{5} \require{amsmath} 1 \color{red}{x_{1}} & + & 0 \color{red}{x_{2}} & + & (-\tfrac{1}{3}) \color{red}{x_{3}} & + & 0 \color{red}{x_{4}} & = & -\tfrac{7}{4}& \\ 0 \color{red}{x_{1}} & + & 1 \color{red}{x_{2}} & + & (-\tfrac{5}{3}) \color{red}{x_{3}} & + & 0 \color{red}{x_{4}} & = & -\tfrac{7}{4}& \\ 0 \color{red}{x_{1}} & + & 0 \color{red}{x_{2}} & + & 0 \color{red}{x_{3}} & + & 1 \color{red}{x_{4}} & = & \tfrac{13}{4} & \\ \end{alignat*} Hence, the solution is \begin{alignat*}{3} \color{red}{x_{1}} & = & -\tfrac{7}{4} & + & \tfrac{1}{3} s & \\ \color{red}{x_{2}} & = & -\tfrac{7}{4} & + & \tfrac{5}{3} s & \\ \color{red}{x_{3}} & = & 0 & + & 1 s & \\ \color{red}{x_{4}} & = & \tfrac{13}{4} & + & 0 s & \end{alignat*} where \(s\) is any real number. The simplest way to write this solution is in parametric vector form: \[ \color{red}{\mathbf{x}} = \left[\! \begin{array}{r} -\tfrac{7}{4} \\ -\tfrac{7}{4} \\ 0 \\ \tfrac{13}{4} \end{array} \!\right] + \left[\! \begin{array}{r} \tfrac{1}{3} s \\ \tfrac{5}{3} s \\ 1 s \\ 0 s \end{array} \!\right] = \frac{1}{4} \left[\! \begin{array}{r} - 7 \\ - 7 \\ 0 \\ 13 \end{array} \!\right] + s \frac{1}{3} \left[\! \begin{array}{r} 1 \\ 5 \\ 3 \\ 0 \end{array} \!\right] = \mathbf{p} + s \mkern 3mu \mathbf{v}, \] where we set \[ \mathbf{p} = \frac{1}{4} \left[\! \begin{array}{r} - 7 \\ - 7 \\ 0 \\ 13 \end{array} \!\right] \quad \text{and} \quad \mathbf{v} = \frac{1}{3} \left[\! \begin{array}{r} 1 \\ 5 \\ 3 \\ 0 \end{array} \!\right]. \]
• I used Wolfram Mathematica to help me with the row reduction in the preceding item.
• Computer algebra system Wolfram Mathematica will be very useful in this class. To get started with Mathematica see my Mathematica page. Please watch the videos that are on my Mathematica page. Watching the movies is very helpful to get started with Mathematica efficiently! Mathematica is available in the computer labs in BH 215, HH 233 and in the Math Center BH 209/211A.
• In class I created a small Mathematica notebook 20241004.nb in which I performed the above row reduction. I also created 20241004.pdf, for you to enjoy this file without access to Mathematica.

---

Thursday, October 3, 2024

---

• Today we discussed Section 1.3: Vector Equations. Suggested problems for Section 1.3: 1, 5, 9, 15, 17, 18, 19, 21-25, 32. We are a liitle late. We did mention how Section 1.3: The Matrix Equation \(A \mathbf{x} = \mathbf{b}\) ties into this. More about it tomorrow.
• To internalize algebraic operations with vectors it is useful to draw pictures in $\mathbb{R}^2$ (the Cartesian plane) and $\mathbb{R}^3$ (the Euclidean space). That is what I will do below.
• Consider two specific vectors in $\mathbb{R}^3$ \[ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]} \quad \text{and} \quad \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \] as given in the picture below: Now add three more gray vectors to this picture These vectors are \[ \left[\! \begin{array}{r} 2 \\ 1 \\ -4 \end{array} \!\right] \quad \text{and} \quad \left[\! \begin{array}{r} 6 \\ -7 \\ 0 \end{array} \!\right] \quad \text{and} \quad \left[\! \begin{array}{r} -7 \\ -1 \\ 5 \end{array} \!\right] . \]
• We ask the question: Is the vector $\left[\! \begin{array}{r} 2 \\ 1 \\ -4 \end{array} \!\right]$ in the span of the vectors $\bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}$ and $\bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]}$? Or, the same question in mathematical notation: Is the following statement true \[ \left[\begin{array}{r} 2 \\ 1 \\ -4 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}? \] And two more questions: Is the following statement true \[ \left[\begin{array}{r} 6 \\ -7 \\ 0 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}? \qquad \left[\begin{array}{r} -7 \\ -1 \\ 5 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}? \]
• Really interesting aspect of linear algebra is that these three questions are just vector equations in disguise. In fact, we are asking if the following vector equations have solutions (are consistent): \[ x_1 \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]} +x_2 \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} = \left[\begin{array}{r} 2 \\ 1 \\ -4 \end{array} \right], \quad x_1 \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]} +x_2 \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} = \left[\begin{array}{r} 6 \\ -7 \\ 0 \end{array} \right], \quad x_1 \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]} +x_2 \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} = \left[\begin{array}{r} -7 \\ -1 \\ 5 \end{array} \right]. \] In turn, these vector equations are in fact systems of linear equations, like we did on the first day of classes:

  System 1	Systems 2	Systems 3
  \begin{alignat*}{8} &x_1 & & - 4 &x_2 & &=&& 2\\ -3 &x_1 & & + &x_2 & & =&& 1 \\ &x_1 & & + 2 &x_2 & & =&& -4 \end{alignat*}	\begin{alignat*}{8} &x_1 & & - 4 &x_2 & &=&& 6\\ -3 &x_1 & & + &x_2 & & =&&-7 \\ &x_1 & & + 2 &x_2 & & =&& 0 \end{alignat*}	\begin{alignat*}{8} &x_1 & & - 4 &x_2 & &=&& -7\\ -3 &x_1 & & + &x_2 & & =&& -1 \\ &x_1 & & + 2 &x_2 & & =&& 5 \end{alignat*}

  These systems are solved by row reducing the augmented matrices. But, this is huge, we can do three row reductions in one go: \begin{align*} \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ -3 & 1 & 1 & -7& -1 \\ 1 & 2 & -4& 0& 5 \end{array}\! \right] & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ 0 & -11 & 7 & 11 & -22 \\ 0 & 6 & -6& -6 & 12 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ 0 & 1 & -\frac{7}{11} & -1 & 2 \\ 0 & 1 & -1& -1 & 2 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ 0 & 1 & -\frac{7}{11} & -1 & 2 \\ 0 & 0 & -\frac{4}{11}& 0 & 0 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ 0 & 1 & -\frac{7}{11} & -1 & 2 \\ 0 & 0 & 1 & 0 & 0 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 0 & 6& -7 \\ 0 & 1 & 0 & -1 & 2 \\ 0 & 0 & 1 & 0 & 0 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & 0 & 0 & 2 & 1 \\ 0 & 1 & 0 & -1 & 2 \\ 0 & 0 & 1 & 0 & 0 \end{array}\! \right] \qquad \text{this matrix is in RREF} \end{align*}
• Conclusions:
  • Since the the first augmented column of the RREF is a pivot column and this augmented column corresponds to System 1, we conclude that System 1 is inconsistent. Therefore, it is true that \[ \left[\begin{array}{r} 2 \\ 1 \\ -4 \end{array} \right] \not\in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}. \]
  • Since the second augmented column of the RREF is not a pivot column and this augmented column corresponds to System 2, we conclude that System 2 is consistent. From the RREF we can read even more \[ \left[\begin{array}{r} 6 \\ -7 \\ 0 \end{array} \right] = 2 \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]} - \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]}. \] Therefore, it is true that \[ \left[\begin{array}{r} 6 \\ -7 \\ 0 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}. \]
  • Since the third augmented column of the RREF is not a pivot column and this augmented column corresponds to System 3, we conclude that System 3 is consistent. From the RREF we can read even more \[ \left[\begin{array}{r} -7 \\ -1 \\ 5 \end{array} \right] = \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]} + 2 \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]}. \] Therefore, it is true that \[ \left[\begin{array}{r} -7 \\ -1 \\ 5 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}. \]
• The questions which are answered in the previous items can be answered by stating a universal problem:
  Find a relationship between the coordinates $b_1,$ $b_2$ and $b_3$ such that the following relationship is satisfied: \[ \left[\begin{array}{c} b_1 \\ b_2 \\ b_3 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}. \]
  The answer to this question is very similar to what we did previously. Instead of specific coordinates in the augmented column we do the row reduction with $b_1,$ $b_2$ and $b_3:$ \begin{align*} \left[\!\begin{array}{rr|l} 1 & -4 & b_1 \\ -3 & 1 & b_2 \\ 1 & 2 & b_3 \end{array}\! \right] & \sim \left[\!\begin{array}{rr|l} 1 & -4 & b_1 \\ 0 & -11 & 3b_1 + b_2 \\ 0 & 6 &-b_1 + b_3 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|l} 1 & -4 & b_1 \\ 0 & 1 & -\frac{3}{11} b_1 -\frac{1}{11} b_2 \\ 0 & 6 &-b_1 + b_3 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|l} 1 & 0 & -\frac{1}{11} b_1 - \frac{4}{11} b_2 \\ 0 & 1 & -\frac{3}{11} b_1 - \frac{1}{11} b_2 \\ 0 & 0 & \frac{7}{11}b_1 + \frac{6}{11} b_2 + b_3 \end{array}\! \right] \\ \end{align*} The last matrix is in row echelon form. For the last matrix to be in reduced row echelon form we must have \[ \frac{7}{11}b_1 + \frac{6}{11} b_2 + b_3 = \frac{1}{11} \bigl(7 b_1 + 6 b_2 + 11 b_3\bigr) = 0. \] If \[ \frac{7}{11}b_1 + \frac{6}{11} b_2 + b_3 = \frac{1}{11} \bigl(7 b_1 + 6 b_2 + 11 b_3\bigr) \neq 0, \] then the augmented column of the last matrix is a pivot column and the corresponding system is inconsistent. Therefore we can make the following claim: \[ \left[\begin{array}{c} b_1 \\ b_2 \\ b_3 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\} \quad \text{if and only if} \quad 7b_1 + 6 b_2 + 11 b_3 = 0. \] Provided that $7b_1 + 6 b_2 + 11 b_3 = 0$, with the last matrix being in reduced row echelon form, we can even give the following relationship: \[ 11 \left[\begin{array}{c} b_1 \\ b_2 \\ b_3 \end{array} \right] = (- b_1 - 4 b_2) \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]} + (-3 b_1 - b_2) \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]}, \] holds if and only if $7b_1 + 6 b_2 + 11 b_3 = 0.$ The last claim can be verified algebraically.
• The picture below is an attempt to illustrate \[ \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\} \] geometrically. In the picture below this span is represented by the yellow translucent plane in $\mathbb{R}^3$. In this plane I placed a grid imposed by the green and the blue vector. To indicate that there are many vectors in this plane, I placed nine gray vectors in the picture all of which are in the span. From the calculation in the preceding item we know that a vector $\left[\begin{array}{c} b_1 \\ b_2 \\ b_3 \end{array} \right]$ belongs to this plane if and only if $7b_1 + 6 b_2 + 11 b_3 = 0.$ This equation is an equation of the translucent yellow plane.

---

• Above, we focused primarily on practical calculations. However, the key takeaway from today's presentation is that three concepts—a system of linear equations, a linear vector equation, and a matrix equation—are mathematically equivalent.

  In the following items, I will provide all the details behind the reasoning for this claim.

• Let \( m \) and \( n \) be positive integers. Using set notation, we write: \( m, n \in \mathbb{N} \). This is read as: \( m \) and \( n \) are elements of the set of positive integers.
  • Linear System. Consider a linear system with $m$ linear equations each with $n$ unknowns. A linear system with $m$ equations each with $n$ unknowns can be written as follows: \begin{equation} \label{eq:LS} \tag{LS} \require{amsmath} \begin{array}{ccccccccc} \color{green}{a_{11}} \color{red}{x_{1}} & + & \color{green}{a_{12}} \color{red}{x_{2}} & + & \cdots & + & \color{green}{a_{1n}} \color{red}{x_{n}} & = & \color{green}{b_{1}} \\ \color{green}{a_{21}} \color{red}{x_{1}} & + & \color{green}{a_{22}} \color{red}{x_{2}} & + & \cdots & + & \color{green}{a_{2n}} \color{red}{x_{n}} & = & \color{green}{b_{2}} \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ \color{green}{a_{m1}} \color{red}{x_{1}} & + & \color{green}{a_{m2}} \color{red}{x_{2}} & + & \cdots & + & \color{green}{a_{mn}} \color{red}{x_{n}} & = & \color{green}{b_{m}} \end{array} \end{equation}
  • Steps towards Vector Equation. These \(m\) equations we can write as one vector equation \begin{equation*} \require{amsmath} \left[\!\begin{array}{c} \color{green}{a_{11}} \color{red}{x_{1}}+ \color{green}{a_{12}} \color{red}{x_{2}} + \cdots + \color{green}{a_{1n}} \color{red}{x_{n}} \\ \color{green}{a_{21}} \color{red}{x_{1}} + \color{green}{a_{22}} \color{red}{x_{2}} + \cdots + \color{green}{a_{2n}} \color{red}{x_{n}} \\ \vdots \\ \color{green}{a_{m1}} \color{red}{x_{1}} + \color{green}{a_{m2}} \color{red}{x_{2}} + \cdots + \color{green}{a_{mn}} \color{red}{x_{n}} \end{array} \!\right] = \left[\! \begin{array}{c} \color{green}{b_{1}} \\ \color{green}{b_{2}} \\ \vdots \\ \color{green}{b_{m}} \end{array} \!\right] \end{equation*} Using the definition of the vector addition, the vector on the left-hand side of the preceding equation can be written as a sum of \(n\) vectors: \begin{equation*} \left[\!\begin{array}{c} \color{red}{x_{1}} \color{green}{a_{11}} \\ \color{red}{x_{1}} \color{green}{a_{21}} \\ \vdots \\ \color{red}{x_{1}}\color{green}{a_{m1}} \end{array} \!\right] + \left[\!\begin{array}{c} \color{red}{x_{2}} \color{green}{a_{12}} \\ \color{red}{x_{2}} \color{green}{a_{22}} \\ \vdots \\ \color{red}{x_{2}}\color{green}{a_{m2}} \end{array}\!\right] + \cdots + \left[\!\begin{array}{c} \color{red}{x_{n}} \color{green}{a_{1n}} \\ \color{red}{x_{n}} \color{green}{a_{2n}} \\ \vdots \\ \color{red}{x_{n}}\color{green}{a_{mn}} \end{array}\!\right] = \left[\! \begin{array}{c} \color{green}{b_{1}} \\ \color{green}{b_{2}} \\ \vdots \\ \color{green}{b_{m}} \end{array}\!\right] \end{equation*} Using the definition of vector scaling, we can factor out the common red scalar in each of the \(n\) vector on the left-hand side of the preceding equation: \begin{equation*} \color{red}{x_{1}} \left[\! \begin{array}{c} \color{green}{a_{11}} \\ \color{green}{a_{21}} \\ \vdots \\ \color{green}{a_{m1}} \end{array} \!\right] + \color{red}{x_{2}} \left[\! \begin{array}{c} \color{green}{a_{12}} \\ \color{green}{a_{22}} \\ \vdots \\ \color{green}{a_{m2}} \end{array} \!\right] + \cdots + \color{red}{x_{n}} \left[\! \begin{array}{c} \color{green}{a_{1n}} \\ \color{green}{a_{2n}} \\ \vdots \\ \color{green}{a_{mn}} \end{array} \!\right] = \left[\! \begin{array}{c} \color{green}{b_{1}} \\ \color{green}{b_{2}} \\ \vdots \\ \color{green}{b_{m}} \end{array} \!\right] \end{equation*}
  • Vector Equation.Now we introduce short boldface names for \(n+1\) vectors in the space \(\mathbb{R}^m\): \begin{equation*} \color{green}{\mathbf{a}_1} = \left[\! \begin{array}{c} \color{green}{a_{11}} \\ \color{green}{a_{21}} \\ \vdots \\ \color{green}{a_{m1}} \end{array} \!\right], \quad \color{green}{\mathbf{a}_2} = \left[\! \begin{array}{c} \color{green}{a_{12}} \\ \color{green}{a_{22}} \\ \vdots \\ \color{green}{a_{m2}} \end{array} \!\right], \quad \cdots , \quad \color{green}{\mathbf{a}_n} = \left[\! \begin{array}{c} \color{green}{a_{1n}} \\ \color{green}{a_{2n}} \\ \vdots \\ \color{green}{a_{mn}} \end{array} \!\right], \quad \color{green}{\mathbf{b}} = \left[\! \begin{array}{c} \color{green}{b_{1}} \\ \color{green}{b_{2}} \\ \vdots \\ \color{green}{b_{m}} \end{array} \!\right]. \end{equation*} Notice that the vectors in the preceding displayed expression are vectors in \(\mathbb{R}^m\). Using these vectors we can rewrite the system \eqref{eq:LS} as one vector equation: \begin{equation} \label{eq:VE} \tag{VE} \color{red}{x_{1}} \color{green}{\mathbf{a}_1} + \color{red}{x_{2}} \color{green}{\mathbf{a}_2} + \cdots + \color{red}{x_{n}} \color{green}{\mathbf{a}_n} = \color{green}{\mathbf{b}}. \end{equation}
  • Matrix Equation. One more step: We introduce a \(m\times n\) matrix, call it \(\color{green}{A}\), whose columns are the vectors \(\color{green}{\mathbf{a}_1}, \color{green}{\mathbf{a}_2}, \ldots, \color{green}{\mathbf{a}_n}\), which, as we know are vectors in \(\mathbb{R}^m\): \begin{equation*} \color{green}{A} = \left[\!\begin{array}{cccc} \color{green}{a_{11}} & \color{green}{a_{12}} & \cdots & \color{green}{a_{1n}} \\ \color{green}{a_{21}} & \color{green}{a_{22}} & \cdots & \color{green}{a_{2n}} \\ \vdots & \vdots & \ddots & \vdots \\ \color{green}{a_{m1}} & \color{green}{a_{m2}} & \cdots & \color{green}{a_{mn}} \\ \end{array}\! \right] = \bigl[ \begin{array}{cccc} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \cdots & \color{green}{\mathbf{a}_n} \end{array} \bigr] \end{equation*} We also introduce a red vector with \(n\) components, that is a vector in \(\mathbb{R}^n\), which consists of the unknowns \(\color{red}{x_{1}}, \color{red}{x_{2}}, \ldots, \color{red}{x_{n}}\): \begin{equation*} \color{red}{\mathbf{x}} = \left[\! \begin{array}{c} \color{red}{x_{1}} \\ \color{red}{x_{2}} \\ \vdots \\ \color{red}{x_{n}} \end{array} \!\right]. \end{equation*} Using the definition of matrix-vector multiplication we can write the vector equation \eqref{eq:VE} as a matrix equation \begin{equation} \label{eq:ME} \tag{ME} \color{green}{A} \color{red}{\mathbf{x}} = \color{green}{\mathbf{b}}. \end{equation} The connection between the vector equation \eqref{eq:VE} and the matrix equation \eqref{eq:ME} is the definition of the matrix-vector multiplication: the product of the matrix \begin{equation*} \color{green}{A} = \left[\!\begin{array}{cccc} \color{green}{a_{11}} & \color{green}{a_{12}} & \cdots & \color{green}{a_{1n}} \\ \color{green}{a_{21}} & \color{green}{a_{22}} & \cdots & \color{green}{a_{2n}} \\ \vdots & \vdots & \ddots & \vdots \\ \color{green}{a_{m1}} & \color{green}{a_{m2}} & \cdots & \color{green}{a_{mn}} \\ \end{array}\! \right] = \bigl[ \begin{array}{cccc} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \cdots & \color{green}{\mathbf{a}_n} \end{array} \bigr] \end{equation*} and the vector \begin{equation*} \color{red}{\mathbf{x}} = \left[\! \begin{array}{c} \color{red}{x_{1}} \\ \color{red}{x_{2}} \\ \vdots \\ \color{red}{x_{n}} \end{array} \!\right] \end{equation*} equals the linear combination of the columns of \(\color{green}{A} = \bigl[ \begin{array}{cccc} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \cdots & \color{green}{\mathbf{a}_n} \end{array} \bigr]\) which are scaled by scalars \(\color{red}{x_{1}}, \color{red}{x_{2}}, \ldots, \color{red}{x_{n}}\). That is \begin{equation*} \require{amsmath} \require{graphicx} \color{green}{A} \color{red}{\mathbf{x}} = \bigl[ \begin{array}{cccc} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \cdots & \color{green}{\mathbf{a}_n} \end{array} \bigr] \left[\! \begin{array}{c} \color{red}{x_{1}} \\ \color{red}{x_{2}} \\ \vdots \\ \color{red}{x_{n}} \end{array} \!\right] \overset{\text{definition}}{=\mkern -1.5mu=\mkern -1.5mu=\mkern -1.5mu=} \ \color{red}{x_{1}} \color{green}{\mathbf{a}_1} + \color{red}{x_{2}} \color{green}{\mathbf{a}_2} + \cdot + \color{red}{x_{n}} \color{green}{\mathbf{a}_n}. \end{equation*}
  • Finally: To solve any of the three equations: the linear system in \eqref{eq:LS}, the linear vector equation \eqref{eq:VE}, or the matrix equation \eqref{eq:ME}, we form the augmented matrix: \begin{equation} \label{eq:AM} \tag{AM} \left[\!\begin{array}{cccc|c} \color{green}{a_{11}} & \color{green}{a_{12}} & \cdots & \color{green}{a_{1n}} & \color{green}{b_{1}} \\ \color{green}{a_{21}} & \color{green}{a_{22}} & \cdots & \color{green}{a_{2n}} & \color{green}{b_{2}} \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ \color{green}{a_{m1}} & \color{green}{a_{m2}} & \cdots & \color{green}{a_{mn}} & \color{green}{b_{m}} \\ \end{array}\! \right] \end{equation} and row reduce it.

    It is important to note that the above augmented matrix \eqref{eq:AM} is completely green matrix with $m$ rows and $n+1$ columns. The last column is the augmented column.

    It is important to note that the above augmented matrix \eqref{eq:AM} is completely green matrix with $m$ rows and $n+1$ columns. The last column is the augmented column.

  ---

---

Wednesday, October 2, 2024

---

• We have moved on to Section 1.3: Vector Equations, but since the Reduced Row Echelon Form of a matrix is a key concept introduced in this class, I will summarize its important aspects in today's post.
• In the items below I will define the Reduced Row Echelon Form (RREF) of a matrix.
• A useful link is Wikipedia's Row echelon form.
• A matrix whose all entries are zero is called a zero matrix. A row of a matrix is said to be a zero row if all the entries in that row are zero. The leftmost nonzero entry of a nonzero row is called the leading entry of a row. The zeros preceding the leading entry are called the leading zeros of a row. All entries of a zero row are leading zeros.
• This is a restatement of Wikipidia's definition of reduced row echelon form:
  Each zero matrix is in Reduced Row Echelon Form. A nonzero matrix is in Reduced Row Echelon Form if it satisfies the following three conditions:

  • Each nonzero row has strictly more leading zeros then the row above it.
  • The leading entry of each nonzero row is equal to $1$.
  • The leading entry of each nonzero row is the only nonzero entry in its column.
• Some consequences of the definition of Reduced Row Echelon Form in the preceding item are:
  • If a nonzero matrix is in Reduced Row Echelon Form, then all its zero rows are at the bottom.
  • If a nonzero matrix is in Reduced Row Echelon Form, then its first row has the fewest leading zeros. The second row has more leading zeros than the first row, the third row has more leading zeros than the second row, and so on, ...
  • Let $m, n, k \in \mathbb{N}.$ If a nonzero $m\!\times\!n$ matrix is in Reduced Row Echelon Form and it has $k$ leading entries, then its columns containing the leading entries are the first $k$ columns of the identity matrix $I_m$ in exactly the same order as in the identity matrix $I_m$. In other words, the pivot columns are the first $k$ columns of the identity matrix $I_m$.
  To illustrate the last item ➢ in the preceding box, I will construct an example of a nonzero $5\!\times\!7$ matrix in Reduced Row Echelon Form. First choose the number of of pivot columns. This can be any positive integer between $1$ and $\min\{5,7\} = 5.$ Let us choose $3.$ The pivot columns are the first three columns of the identity matrix $I_5$ \[ \left[ \begin{array}{ccccc} 1 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ \end{array} \right], \] that is the columns \[ \left[ \begin{array}{c} 1 \\ 0 \\ 0 \\ 0 \\ 0 \\ \end{array} \right], \qquad \left[ \begin{array}{c} 0 \\ 1 \\ 0 \\ 0 \\ 0 \\ \end{array} \right], \qquad \left[ \begin{array}{c} 0 \\ 0 \\ 1 \\ 0 \\ 0 \\ \end{array} \right]. \] Next we need to choose the positions for these three columns in a $5\!\times\!7$ matrix. Since a $5\!\times\!7$ matrix has $7$ columns we can choose any $3$ positions of the possible positions $\{1,2,3,4,5,6,7\}.$ So choose, $2,$ $4,$ and $7$ \[ \left[ \begin{array}{ccccccc} \Box & 1 & \Box & 0 & \Box & \Box & 0 \\ \Box & 0 & \Box & 1 & \Box & \Box & 0 \\ \Box & 0 & \Box & 0 & \Box & \Box & 1 \\ \Box & 0 & \Box & 0 & \Box & \Box & 0 \\ \Box & 0 & \Box & 0 & \Box & \Box & 0 \\ \end{array} \right] \] Since the $1$s are the leading entries in their rows, we must substitute the $\Box$s preceding $1$s with zeros \[ \left[ \begin{array}{ccccccc} 0 & 1 & \Box & 0 & \Box & \Box & 0 \\ 0 & 0 & 0 & 1 & \Box & \Box & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ \Box & 0 & \Box & 0 & \Box & \Box & 0 \\ \Box & 0 & \Box & 0 & \Box & \Box & 0 \\ \end{array} \right] \] Since the fourth and the fifth row do not have leading entries, they must be zero rows: \[ \left[ \begin{array}{ccccccc} 0 & 1 & \Box & 0 & \Box & \Box & 0 \\ 0 & 0 & 0 & 1 & \Box & \Box & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \end{array} \right] \] The last matrix is in Reduced Row Echelon Form where $\Box$s can be replaced by arbitrary real numbers. For example \[ \left[ \begin{array}{ccccccc} 0 & 1 & -2 & 0 & 3 & -1 & 0 \\ 0 & 0 & 0 & 1 & 2 & 4 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \end{array} \right] \]
• To illustrate the definition of the reduced row echelon form I made a picture of a $12\times 25$ matrix (that is a matrix with $12$ rows and $25$ columns which is in reduced row echelon form. To emphasize the role of the leading zeros, I colored them blue. To emphasize the role of leading $1$s, I colored them green. I also colored in green the each pivot column. The entries of the matrix that are not specifically written as $0$s or $1$s can be any real numbers. Please notice how blue leading zeros form steps which climb from right to left. Also notice that each stair hosts a pivot position, that is the leading $1$ in a nonzero row.
• A very good exercise in understanding RREF (reduced row echelon form) is to write down all $2\!\times\!3$ matrices which are in RREF. There are seven of them. Below I list the matrix with no pivot columns, then all matrices with one pivot column, then the matrices with two pivot columns. Those are all possibilities since each pivot column has pivot position occupied by the leading entry which needs its own row and we are studying matrices with only two rows. In the matrices below stars ($*$) stand for arbitrary real numbers. \begin{align*} & \begin{bmatrix} 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 0 & 1 \\ 0 & 0 & 0 \end{bmatrix},\quad \begin{bmatrix} 0 & 1 & * \\ 0 & 0 & 0 \end{bmatrix},\quad \begin{bmatrix} 1 & * & * \\ 0 & 0 & 0 \end{bmatrix},\\ & \begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}, \quad \begin{bmatrix} 1 & * & 0 \\ 0 & 0 & 1 \end{bmatrix}, \quad \begin{bmatrix} 1 & 0 & * \\ 0 & 1 & * \end{bmatrix}, \quad \end{align*}
• Here are all possible $3\!\times\!4$ matrices in RREF (reduced row echelon form). There are fifteen of them. \begin{align*} & \begin{bmatrix} 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 0 & 0 & 1 & * \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix},\ \begin{bmatrix} 0 & 1 & * & * \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 1 & * & * & * \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix},\\ & \begin{bmatrix} 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 0 & 1 & * & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 1 & * & * & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 0 & 1 & 0 & * \\ 0 & 0 & 1 & * \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 1 & * & 0 & * \\ 0 & 0 & 1 & * \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 1 & 0 & * & * \\ 0 & 1 & * & * \\ 0 & 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}, \ \begin{bmatrix} 1 & * & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix},\ \begin{bmatrix} 1 & 0 & * & 0 \\ 0 & 1 & * & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}, \ \begin{bmatrix} 1 & 0 & 0 & * \\ 0 & 1 & 0 & * \\ 0 & 0 & 1 & * \end{bmatrix}, \end{align*}
• Here are all possible $4\!\times\!3$ matrices in RREF (reduced row echelon form). There are eight of them. \begin{align*} & \begin{bmatrix} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \quad \begin{bmatrix} 0 & 1 & * \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix},\quad \begin{bmatrix} 1 & * & * \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \quad \begin{bmatrix} 1 & * & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \quad \begin{bmatrix} 1 & 0 & * \\ 0 & 1 & * \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{bmatrix}. \end{align*}
• Notice that in all the matrices listed in the preceding item the bottom row is the zero row. By removing that zero row we would get all possible $3\!\times\!3$ matrices in RREF (reduced row echelon form). Thus, there is eight different types of $3\!\times\!3$ matrices in RREF (reduced row echelon form). This is true in general: If $m\geq n$, than the number of different types of $m\!\times\!n$ matrices in RREF is the same as the number of different types of $n\!\times\!n$ matrices in RREF.
• Another good exercise is to think of the above matrices as augmented matrices of systems of equations and state for each corresponding system whether it has: No Solutions (NS), Unique Solution (US), Infinitely Many Solutions (MS).

---

• Another interesting exercise, but for Math 309, is to calculate how many $m\!\times\!n$ matrices are in RREF. Just for the record: Let $k$ be a nonnegative integer and let $m, n$ be positive integers such that $k \leq m \leq n.$ It turns out that there are exactly \[ \displaystyle \binom{n}{k} = \frac{n!}{k!(n-k)!} \] matrices of size $m\!\times\!n$ with $k$ pivots (leading $1$s). The symbol $\displaystyle \binom{n}{k}$ is the standard notation for the binomial coefficient $n$-choose-$k$. Since an arbitrary $m\!\times\!n$ matrix can have $k \in \{0,1,\ldots,m\}$ pivots, it follows that the number of possible RREF types for an $m\!\times\!n$ matrix is \[ \sum_{k=0}^m \binom{n}{k}. \] For example, with $m=3$ and $n=4$, as seen above, there are \[ \binom{4}{0} + \binom{4}{1} + \binom{4}{2} + \binom{4}{3} = 1 + 4 + 6 + 4 = 15. \] possible RREF types of matrices. It is interesting to observe that for an $n\!\times\!n$ matrix there are \[ \sum_{k=0}^n \binom{n}{k} = \binom{n}{0} + \binom{n}{1} + \cdots + \binom{n}{n-1} + \binom{n}{n} = 2^n \] possible RREF types. If you take Math 309 you will understand the background of these formulas.
• In conclusion, for an $m\!\times\!n$ matrix with $m\lt n$ there are \[ \sum_{k=0}^m \binom{n}{k} \] possible RREF types and if $m \geq n$ there are \[ 2^n \] possible RREF types.

---

Tuesday, October 1, 2024

---

• This post is lengthy because it repeats all the arguments from last week's classes. Please read it carefully, especially the conclusion. I've also illustrated the equations with pictures, using the fact that that the graph of a linear equation in three variables is a plane in three-dimensional space \(\mathbb{R}^3\). If you're not yet familiar with this setting, please revisit this post when we cover vectors in \(\mathbb{R}^3\). Most of the post is independent of the geometric illustrations.
• Read Section 1.1 and Section 1.2. Assigned problems for Section 1.1: 3, 6, 7, 5, 9, 11,12, 16, 17, 18, 21, 22, 23, 24, 25, 27, 31, 33, 34. Assigned problems for Section 1.2: 3, 5, 6, 7, 8, 12, 17-31.
• Today we solved the following system of linear equations: \begin{alignat*}{8} &x_1 & & + &&x_2 & & - &&2 &&x_3 &&= -&&5\\ 2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\ 3 &x_1 & & && & & - && && x_3 &&= &&3 \end{alignat*} I will give all the details of this solution and illustrate wth pictures. This system is solved by row reducing its augmented matrix: \begin{align*} \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \\ 2 & -1 & 1 & 8 \\ 3 & 0 & -1 & 3 \end{array}\! \right] & \sim \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \\ 0 & -3 & 5 & 18 \\ 0 & -3 & 5 & 18 \end{array}\! \right] \quad \begin{array}{l} \text{(two row replacements)} \end{array} \\ & \sim \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \\ 0 & -3 & 5 & 18 \\ 0 & 0 & 0 & 0 \end{array}\! \right] \quad \begin{array}{l} \text{(one row replacements)} \end{array} \\ & \sim \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \\ 0 & 1 & -5/3 & -6 \\ 0 & 0 & 0 & 0 \end{array}\! \right] \quad \begin{array}{l} \text{(one row scaling)} \\[-5pt] \text{(and one row replacements to obtain} \\[-6pt] \text{the matrix below)} \end{array} \\ & \sim \left[\!\begin{array}{crr|r} 1 & 0 & -1/3 & 1 \\ 0 & 1 & -5/3 & -6 \\ 0 & 0 & 0 & 0 \end{array}\! \right] \quad \begin{array}{l} \text{This matrix is in Reduced Row Echelon Form.} \\[-5pt] \text{I always like to row reduce a matrix} \\[-5pt] \text{to its RREF - Reduced Row Echelon Form.} \\[-5pt] \text{For the definition of RREF see the end of this post.} \end{array} \end{align*} Why are the matrices presented above row equivalent?
  • On the first line, the two matrices are row equivalent since the matrix on the right is obtained from the matrix on the left by performing two row operations. The first row operation was a row replacement: the second row $\left[\!\begin{array}{crr|r} 2 & -1 & 1 & 8 \end{array}\! \right]$ is replaced by the sum of itself and the multiple of the first row $ \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right]$ by $-2$: \begin{align*} \left[\!\begin{array}{crr|r} 2 & -1 & 1 & 8 \end{array}\! \right] + (-2) \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right] & = \left[\!\begin{array}{crr|r} 2 & -1 & 1 & 8 \end{array}\! \right] + \left[\!\begin{array}{crr|r} -2 & -2 & 4 & 10 \end{array}\! \right] \\ & = \left[\!\begin{array}{crr|r} 2 - 2 & -1 - 2 & 1 + 4 & 8 + 10 \end{array}\! \right] \\ & = \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] \end{align*} The second row operation was again a row replacement: the third row $\left[\!\begin{array}{crr|r} 3 & 0 & -1 & 3 \end{array}\! \right]$ is replaced by the sum of itself and the multiple of the first row $ \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right]$ by $-3$: \begin{align*} \left[\!\begin{array}{crr|r} 3 & 0 & -1 & 3 \end{array}\! \right] + (-3) \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right] & = \left[\!\begin{array}{crr|r} 3 & 0 & -1 & 3 \end{array}\! \right] + \left[\!\begin{array}{crr|r} -3 & -3 & 6 & 15 \end{array}\! \right] \\ & = \left[\!\begin{array}{crr|r} 3 - 3 & 0 - 3 & -1 + 6 & 3 + 15 \end{array}\! \right] \\ & = \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] \end{align*} Thinking about the corresponding system of linear equations, what we achieved at this step is that we eliminated unknown $x_1$ from the second and the third equation.
  • The matrix on the second line is row equivalent to the matrix above it since it is obtained from the matrix above it by performing a row replacement: the third row $\left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right]$ is replaced by the sum of itself and the second row $ \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right]$ multiplied by $-1$: \begin{align*} \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] + (-1) \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] & = \left[\!\begin{array}{crr|r} 0 & 0 & 0 & 0 \end{array}\! \right] \end{align*} Thinking about the corresponding system of linear equations, what we achieved at this step is that we reduced the number of equations from three to two.
  • The matrix on the third line is row equivalent to the matrix above itself since it is obtained from the matrix above itself by performing a row scaling: the second row $\left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right]$ is replaced by the multiple of itself $-1/3$: \[ (-1/3) \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] = \left[\!\begin{array}{crr|r} 0 & 1 & -5/3 & -6 \end{array}\! \right] \] Thinking about the corresponding system of linear equations, what we achieved at this step is that we now have the coefficient $1$ with the unknown $x_2$ in the second equation.
  • The matrix on the fourth line is row equivalent to the matrix above itself since it is obtained from the matrix above itself by performing a row replacement: the first row $\left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right]$ is replaced by the sum of itself and the second row $ \left[\!\begin{array}{crr|r} 0 & 1 & -5/3 & -6 \end{array}\! \right]$ multiplied by $-1$: \begin{align*} \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right] + (-1) \left[\!\begin{array}{crr|r} 0 & 1 & -5/3 & -6 \end{array}\! \right] & = \left[\!\begin{array}{crr|r} 1 & 0 & -1/3 & 1 \end{array}\! \right] \end{align*} Thinking about the corresponding system of linear equations, what we achieved at this step is that we eliminated unknown $x_2$ from the first equation.
  In conclusion, the matrices \[ \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \\ 2 & -1 & 1 & 8 \\ 3 & 0 & -1 & 3 \end{array}\! \right] \sim \left[\!\begin{array}{crr|r} 1 & 0 & -1/3 & 1 \\ 0 & 1 & -5/3 & -6 \\ 0 & 0 & 0 & 0 \end{array}\! \right] \] are row equivalent since the matrix on the right is obtained from the matrix on the left by performing five row operations: first two are row replacements, then a row replacement again, then row scaling and finally one more row replacement.
• Since the matrices \[ \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \\ 2 & -1 & 1 & 8 \\ 3 & 0 & -1 & 3 \end{array}\! \right] \sim \left[\!\begin{array}{crr|r} 1 & 0 & -1/3 & 1 \\ 0 & 1 & -5/3 & -6 \\ 0 & 0 & 0 & 0 \end{array}\! \right] \] are row equivalent, the corresponding systems are equivalent systems. That is, the solution set of the given system is the same as the solution set of the system \begin{alignat*}{8} &x_1 & & && & & - &&1/3 &&\, x_3 &&= &&1\\ & & & &&x_2 & & - &&5/3 &&\, x_3 &&= -&&6 \end{alignat*} In the preceding system there is no restriction on $x_3$; $x_3$ is so-called free variable. I like introducing a new name for the free variable, say $s$. Then the solution of the preceding system is given by \begin{equation*} x_1 = \frac{1}{3} s + 1, \quad x_2 = \frac{5}{3} s - 6, \quad x_3 = s, \end{equation*} where $s$ is an arbitrary real number. In set notation, $s \in \mathbb{R}.$ For example, with $s=3$ we obtain that \[ x_1 = 2, \quad x_2 = -1, \quad x_3 = 3, \] is a solution to the given system.
• It is always fun to verify whether the solution realy satisfies the given system. Substitute $x_1 = \frac{1}{3} s + 1,$ $x_2 = \frac{5}{3} s - 6$ and $x_3 = s,$ and see what we get: \begin{alignat*}{8} &(1/3 \, s + 1) & & + &&(5/3 \, s - 6) & & - &&2 &&s &&= 2s - 5 +2s = -&&5\\ 2 &(1/3 \, s + 1) & & - &&(5/3 \, s - 6) & & + && &&s &&= -s + 8 + s = &&8 \\ 3 &(1/3 \, s + 1) & & && & & - && && s &&= s + 3 - s = &&3 \end{alignat*} Hence we found all the solutions of the given system.
• Finally, let us illustrate the system \begin{alignat*}{8} &x_1 & & + &&x_2 & & - &&2 &&x_3 &&= -&&5\\ 2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\ 3 &x_1 & & && & & - && && x_3 &&= &&3 \end{alignat*} geometrically in $x_1x_2x_3$-space, that is in the space $\mathbb{R}^3.$ Each equation is represented by a plane. The first equation is represented by the brown plane, the second equation is represented by the blue plane and the third equation is represented by the green plane.
• A natural question is: How are row operations reflected in the illustration of the system with planes?

  Given System	Systems 2 and 3	"Row Reduced" System
  \begin{alignat*}{8} &x_1 & & + &&x_2 & & - &&2 &&x_3 &&= -&&5\\ 2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\ 3 &x_1 & & && & & - && && x_3 &&= &&3 \end{alignat*}	\begin{alignat*}{8} &x_1 & & + &&x_2 & & - &&\phantom{5/} 2&&x_3 &&= -&&5\\ & & & &&x_2 & & - &&5/3 &&x_3 &&= -&&6 \\ & & & && & & && && && && \end{alignat*}	\begin{alignat*}{8} &x_1 & & && & & - &&1/3 &&x_3 &&= &&1\\ & & & \phantom{+} &&x_2 & & - &&5/3 &&x_3 &&= -&&6 \\ & & & && & & && && && && \end{alignat*}

  The systems that correspond to the row equivalent matrices on lines 2 and 3 in the row reduction above are illustrated by the same picture; the planes do not change. The system corresponding to the last matrix in Reduced Row Echelon Form is represented by the simplest planes. What is the simplicity of these planes? The first plane is parallel to the coordinate axis $x_2.$ The second plane is parallel to the coordinate axis $x_1.$ (Being parallel to coordinate ases is a very desirable feature for a plane.)
• Let us change the system just by changing one number on the left hand side of the first equation: \begin{alignat*}{8} &x_1 & & + &&x_2 & & - &&2 &&x_3 &&= &&5\\ 2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\ 3 &x_1 & & && & & - && && x_3 &&= &&3 \end{alignat*} The only difference is that in the first equation $-5$ is replaced by $5$. So, the same row operations that we used above will row reduce the corresponding augmented matrix: \begin{align*} \left[\!\begin{array}{crr|r} 1 & 1 & -2 & 5 \\ 2 & -1 & 1 & 8 \\ 3 & 0 & -1 & 3 \end{array}\! \right] & \sim \left[\!\begin{array}{crr|r} 1 & 1 & -2 & 5 \\ 0 & -3 & 5 & -2 \\ 0 & -3 & 5 & -12 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{crr|r} 1 & 1 & -2 & 5 \\ 0 & -3 & 5 & -2 \\ 0 & 0 & 0 & -10 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{crr|r} 1 & 1 & -2 & 5 \\ 0 & 1 & -5/3 & 1/3 \\ 0 & 0 & 0 & 1 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{crr|r} 1 & 0 & -1/3 & 14/3 \\ 0 & 1 & -5/3 & 1/3 \\ 0 & 0 & 0 & 1 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{crr|r} 1 & 0 & -1/3 & 0 \\ 0 & 1 & -5/3 & 0 \\ 0 & 0 & 0 & 1 \end{array}\! \right] \end{align*} The preceding row reduction proves that the following two linear systems are equivalent:

  \begin{alignat*}{8} &x_1 & & + &&x_2 & & - &&2 &&x_3 &&= &&5\\ 2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\ 3 &x_1 & & && & & - && && x_3 &&= &&3 \end{alignat*}

  \begin{alignat*}{8} &x_1 & & && & & - 1/3 && &&x_3 &&= &&0\\ & & & &&x_2 & & - 5/3 && &&x_3 &&= &&0 \\ 0 &x_1 & & + 0 & & x_2 & & + \ 0 && && x_3 &&= &&1 \end{alignat*}

  The last equation in the system on the right is written with zeros for us to see that the resulting equation does not have a solution. Hence the above system on the right is inconsistent and thus, the given system is inconsistent.
  Below is the corresponding illustration in $\mathbb{R}^3.$ The change in the illustration is that a parallel plane (the new brown plane) has replaced the brown plane in the original illustration. The new brown plane does not pass through the intersection of the blue and green plane. Thus, no point lies in all three planes.

---

Wednesday, September 25, 2024

---

• The information sheet and the class calendar.
• We will start with Section 1.1 Systems of Linear Equations. Suggested problems are 3, 6, 7, 5, 9, 11, 12, 16, 17, 18, 21, 22, 23, 24, 25, 27, 31, 33, 34.
• Just reading Section 1.1 you will experience that linear algebra is a subject full of new terminology:
  • linear equation,
  • coefficients of a linear equation,
  • system of linear equations or linear system or just system,
  • a solution of a linear system,
  • the solution set of a linear system,
  • consistent linear system,
  • inconsistent linear system,
  • equivalent systems,
  • coefficient matrix of a system,
  • augmented matrix of a system,
  • elementary row operations,
  • row-equivalent matrices.
• Learning the terminology of a field is crucial to getting to understand that field. This is true in general and particularly true in Linear Algebra. Therefore I started a webpage with Glossary of Linear Algebra Terms with my comments.
• What is the oldest linear algebra problem?
  • Clay tablet VAT 8389 from the Old Babylonian period, from 2000 to 1600 BC, contains what is believed to be the earliest word problem that can be interpreted as a system of linear equations:
    Total area of two fields is 1800 sar, the rent for one is 2 silà of grain per 3 sar, for the other is 1 silà per 2 sar, the total rent on the first exceeds that on the other by by 500 silà. What is the area of each plot?
    This blog has the picture of clay tablet VAT 8389 and more details about it.
    
    A translation of this word problem into a system of linear equations is as follows: \begin{alignat*}{4} &x_1 & &\ + &x_2 & = 1800 \\ \tfrac{2}{3} &x_1 & &- \tfrac{1}{2} &x_2 & = \phantom{1}500. \end{alignat*}
  • Problem 40 of the Rhind papyrus which is dated to 1550 BC is:
    Divide 100 hekats of barley among 5 men so that the common difference is the same and so that the sum of the two smallest is 1/7 the sum of the three largest.
    Historians suspect that the scribe Ahmes copied the Rhind papyrus from a now-lost text from around 1850 BC. Further, this lost text might have been copied from an even older text from around 2500 BC. Therefore, the preceding problem could be the oldest known linear algebra problem.
    
    Denote by $x_1$ the smallest number and by $x_2$ the common difference. After simplification, the above problem translates into the following system of linear equations: \begin{alignat*}{5} 5 &x_1 & & + 10 &x_2 & = 100 \\ \tfrac{11}{7} &x_1 & & - \phantom{1}\tfrac{2}{7} &x_2 & = \phantom{10}0. \end{alignat*}
  • Most importantly for us, the oldest known treatment of systems of linear equations from antiquity which resembles the methods that we will use in this class is in Chapter 8 of the Chinese textbook Nine Chapters of the Mathematical Art which is at least 1800 years old.
    From 3 top-grade rice paddies, 2 medium-grade, and 1 low-grade, the combined yield is 39 dou of grain. From 2 top-grade, 3 medium-grade, and 1 low-grade, the combined yield is 34 dou of grain. From 1 top-grade, 2 medium-grade, and 3 low-grade, the combined yield is 26 dou of grain. How much dou does one bundle of each grade yield?
    Denote by $x_1$ the yield of the top-grade rice paddy, by $x_2$ the yield of the medium-grade, and by $x_3$ the yield of the low-grade rice paddy. Then the above problem translates into the following system of linear equations: \begin{alignat*}{7} 3 &x_1 & & + 2 &x_2 & + & x_3 = 39 \\ 2 &x_1 & & + 3 &x_2 & + & x_3 = 34 \\ &x_1 & & + 2 &x_2 & + 3 & x_3 = 26 \end{alignat*}
• If the history of mathematics might inspire you to study mathematics with more enthusiasm, below I link to some websites with more about the history of Linear Algebra.
  • Early History of Linear Algebra by Roger Hart
  • History of matrices
  • History of abstract vector spaces
  • Solving a System of Linear Equations Using Ancient Chinese Methods by Mary Flagg
• My comment on the history of mathematics:
  Different civilizations have created mathematical knowledge throughout history and sometimes passed that knowledge among themselves. A significant aspect of the growth of mathematical knowledge was that succeeding civilizations recognized the value of the knowledge created by preceding civilizations. Just the fact that I am writing this here is my recognition and appreciation of the contribution of the ancient civilizations to Linear Algebra.

---