- Suggested problems for Section 1.8: 1-4, 12, 13-17, 19, 20, 25, 27, 28.
- Suggested problems for Section 1.9: 1, 3, 4, 5, 7, 8, 11, 12, 18, 19, 23 Notice that we did not discuss Definitions on page 76 and the content after these definitions.
- On each in-class exam, I assign four problems. Often these problems have several parts (a), (b), and sometimes (c) parts. In this PDF file you can find some problems in the form that they could appear on the exam.
- In the textbook the authors list several reflections transformations across some popular lines like: $x_2 = 0$ (this is $x_1$-axis), $x_1 = 0$ (this is $x_2$-axis), $x_2 = x_1$ (this is the diagonal of the first and the third quadrant), and $x_2 = -x_1$ (this is the diagonal of the second and the fourth quadrant). But why not establish the standard matrix of the reflection across the line which makes the angle $\theta$ with the positive $x_1$-semi-axis? It is not more difficult than the standard matrix of the rotation by the angle $\theta$ in counterclockwise direction, which is \[ \left[\! \begin{array}{cc} \cos(\theta) & -\sin(\theta) \\[5pt] \sin(\theta) & \cos(\theta) \end{array}\!\right] \]
-
The picture below will help us determine the standard matrix of the reflection across the blue line which makes angle $\theta$ with the positive $x_1$-semi-axis. Denote this reflection by
\[
F:\mathbb{R}^2 \to \mathbb{R}^2.
\]
To determine the standard matrix of this reflection we need to calculate the coordinates of the vectors
\[
F \mathbf{e}_1 = F \left[\! \begin{array}{c}
1 \\[3pt]
0
\end{array}\!\right] \quad \text{and} \quad F \mathbf{e}_2 = F \left[\! \begin{array}{c}
0 \\[3pt]
1
\end{array}\!\right]
\]
In the picture below the vector $F \mathbf{e}_1$ is dark orange and the vector $F \mathbf{e}_2$ dark purple.
- By the definition of the reflection $F$, the angle formed by the light orange coordinate vector $\mathbf{e}_1$ and its reflection the dark orange vector $F \mathbf{e}_1$ is $2\theta.$ Therefore, \[ F \mathbf{e}_1 = F \left[\! \begin{array}{c} 1 \\[3pt] 0 \end{array}\!\right] = \left[\! \begin{array}{c} \cos(2\theta) \\[3pt] \sin(2\theta) \end{array}\!\right]. \]
- In this item we calculate the coordinates of the dark purple vector $F \mathbf{e}_2$ which is the reflection of the light purple coordinate vector $\mathbf{e}_2.$ First, we observe that the light gray right triangles in the picture below are congruent. This claim follows from the fact that the hypothenuse of each of these triangles is of length $1$ and the triangles have the same angle $2\theta$ at the origin. That the angle at the origin of the top light gray triangle is $2\theta$ follows from the definition of the reflection. To calculate the angle at the origin for the lower light gray triangle we calculate the angle between the light purple coordinate vector $\mathbf{e}_2$ and its reflection, the dark purple vector $F \mathbf{e}_2;$ since the angle between the light purple coordinate vector $\mathbf{e}_2$ and the blue line is $\displaystyle \frac{\pi}{2} - \theta$, the angle between is $\mathbf{e}_2$ and $F \mathbf{e}_2$ is \[ 2 \left(\frac{\pi}{2} - \theta\right) = \pi - 2 \theta. \] Since the sum of this angle and the angle at the origin for the lower light gray triangle equals $\pi,$ we have proven that the angle at the origin for the lower light gray triangle equals $2\theta.$ In the preceding item we proved that the horizontal side of the top light gray right triangle is $\cos(2\theta)$ and its vertical side is $\sin(2\theta)$. As a consequence, the horizontal side of the lower light gray right triangle is $\sin(2\theta)$ and its vertical side is $\cos(2\theta)$. Adjusting for the positioning in the coordinate system we get \[ F \mathbf{e}_2 = F \left[\! \begin{array}{c} 0 \\[3pt] 1 \end{array}\!\right] = \left[\! \begin{array}{c} \sin(2\theta) \\[3pt] -\cos(2\theta) \end{array}\!\right]. \] Finally we have that the standard matrix of the reflection across the blue line which makes an angle $\theta$ with the positive $x_1$-semi-axis is \[ \left[\! \begin{array}{cc} \cos(2\theta) & \sin(2\theta) \\[5pt] \sin(2\theta) & -\cos(2\theta) \end{array}\!\right]. \] Enjoy Reflections!
Enjoy Reflections!
-
In the following examples, I demonstrate the geometric interpretation of various matrices. The textbook provides additional illustrations for a broader range of matrices.
In the examples below, the happy face is designed by the heads of the following vectors: For the face I used the circle centered at the head of the vector \(\left[\! \begin{array}{c} 1 \\ 1 \end{array} \!\right]\) and with radius \(4/5\). That is the set of the following vectors \[ \Biggl\{ \left[\! \begin{array}{c} 1 \\ 1 \end{array} \!\right] + \frac{4}{5} \left[\! \begin{array}{c} \cos t \\ \sin t \end{array} \!\right] \, : \, t \in [0, 2 \pi) \Biggr\}. \] The navy eyes are at the heads of the following two vectors \[ \frac{1}{5} \left[\! \begin{array}{c} 4 \\ 7 \end{array} \!\right], \quad \frac{1}{5} \left[\! \begin{array}{c} 6 \\ 7 \end{array} \!\right]. \] The red smile, I used the set of the following vectors \[ \Biggl\{ \left[\! \begin{array}{c} 1 \\ 1/2 \end{array} \!\right] + \frac{1}{2} \left[\! \begin{array}{c} 2 t \\ 3 t^2 \end{array} \!\right] \, : \, t \in \left[-\frac{1}{3},\frac{1}{3}\right] \Biggr\}. \]
-
The matrix resulting in the image transformation below is
\[
A = \left[\! \begin{array}{cc} 1 & 1 \\ 0 & 1 \end{array} \!\right].
\]
This transformation is called shear.
-
Below is an illustration of a rotation about the origin by the angle \(\theta = \dfrac{5 \pi}{8}\) counterclockwise. Recall that the rotation matrix about the origin by the angle \(\theta\) is
\[
R(\theta) = \left[\! \begin{array}{cc} \cos \theta & - \sin \theta \\ \sin \theta & \cos \theta \end{array} \!\right].
\]
Thus, the matrix used in the following illustration is
\[
\left[\! \begin{array}{cc} \cos\left(\dfrac{5 \pi}{8}\right) & - \sin\left(\dfrac{5 \pi}{8}\right) \\[5pt]
\sin\left(\dfrac{5 \pi}{8}\right) & \cos\left(\dfrac{5 \pi}{8}\right) \end{array} \!\right] =
- \frac{1}{2} \left[\! \begin{array}{cc} \sqrt{2-\sqrt{2}} & \sqrt{2+\sqrt{2}} \\[5pt]
- \sqrt{2+\sqrt{2}} & \sqrt{2-\sqrt{2}} \end{array} \!\right] .
\]
-
In this item we show another rotation. It is the rotation about the origin by the angle \(\theta = \dfrac{\pi}{8}\) counterclockwise. The matrix used in the following illustration is
\[
\left[\! \begin{array}{cc} \cos\left(\dfrac{\pi}{8}\right) & - \sin\left(\dfrac{\pi}{8}\right) \\[5pt]
\sin\left(\dfrac{\pi}{8}\right) & \cos\left(\dfrac{\pi}{8}\right) \end{array} \!\right] =
\frac{1}{2} \left[\! \begin{array}{cc}
\sqrt{2+\sqrt{2}} & - \sqrt{2-\sqrt{2}} \\[5pt]
\sqrt{2-\sqrt{2}} & \sqrt{2+\sqrt{2}} \end{array} \!\right] .
\]
Notice the relationship between the rotation used in this item and the rotation in the preceding item: The angle used in the rotation in the preceding item is the sum of the angle in this item and \(\pi/2\):
\[
\dfrac{5 \pi}{8} = \dfrac{\pi}{2} + \dfrac{\pi}{8}.
\]
In the language of rotations, the rotation in the preceding item can be obtained by performing the rotation from this item and then performing the rotation by the angle \(\theta = \pi/2\). Notice that the standard matrix of the rotation by the angle \(\theta = \pi/2\) is
\[
\left[\! \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \!\right].
\]
Later on we will establish a close connection between the following three matrices:
\[
\frac{1}{2} \left[\! \begin{array}{cc} -\sqrt{2-\sqrt{2}} & -\sqrt{2+\sqrt{2}} \\[5pt]
\sqrt{2+\sqrt{2}} & -\sqrt{2-\sqrt{2}} \end{array} \!\right] , \quad
\left[\! \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \!\right],
\quad
\frac{1}{2} \left[\! \begin{array}{cc}
\sqrt{2+\sqrt{2}} & - \sqrt{2-\sqrt{2}} \\[5pt]
\sqrt{2-\sqrt{2}} & \sqrt{2+\sqrt{2}} \end{array} \!\right].
\]
To eliminate the suspense, the first matrix is the product of the second and the third matrix. We will define matrix product next week.
-
The purple line in the image below makes an angle of \(\theta = \pi/8\) with the horizontal axis. The image illustrates the reflection transformation across the purple line. As shown in an earlier item, the reflection matrix used in the image below is
\[
\left[\! \begin{array}{cc} \cos\left(2\theta\right) & \sin\left(2\theta\right) \\[5pt]
\sin\left(2\theta\right) & -\cos\left(2\theta\right) \end{array} \!\right]
= \left[\! \begin{array}{cc} \cos\left(\dfrac{\pi}{4}\right) & \sin\left(\dfrac{\pi}{4}\right) \\[5pt]
\sin\left(\dfrac{\pi}{4}\right) & -\cos\left(\dfrac{\pi}{4}\right) \end{array} \!\right] =
\left[\! \begin{array}{cc}
\frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} \\[5pt]
\frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right].
\]
-
The teal line in the image below makes an angle of \(\theta = 5\pi/8\) with the positive horizontal semi-axis. The image illustrates the reflection transformation across the teal line. As shown in an earlier item, the reflection matrix used in the image below is
\[
\left[\! \begin{array}{cc} \cos\left(2\theta\right) & \sin\left(2\theta\right) \\[5pt]
\sin\left(2\theta\right) & -\cos\left(2\theta\right) \end{array} \!\right]
= \left[\! \begin{array}{cc} \cos\left(\dfrac{5\pi}{4}\right) & \sin\left(\dfrac{5\pi}{4}\right) \\[5pt]
\sin\left(\dfrac{5\pi}{4}\right) & -\cos\left(\dfrac{5\pi}{4}\right) \end{array} \!\right] =
\left[\! \begin{array}{cc}
\frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \\[5pt]
-\frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right].
\]
- An interesting problem is to find the standard matrix of the linear transformation obtained by first reflecting across the teal line, then reflecting across the purple line, in the two preceding items. How would we find that standard matrix? To find that standard matrix we need to track the images of the standard coordinate vectors \[ \mathbf{e}_1 = \left[\! \begin{array}{c} 1 \\ 0 \end{array} \!\right] \quad \text{and} \quad \mathbf{e}_2 = \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right]. \] Consider first the vector \(\mathbf{e}_1 = \left[\! \begin{array}{c} 1 \\ 0 \end{array} \!\right]\): Reflecting this vector across the teal line we get the vector: \[ \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right] \left[\! \begin{array}{c} 1 \\ 0 \end{array} \!\right] = \left[\! \begin{array}{c} \frac{\sqrt{2}}{2} \\[5pt] - \frac{\sqrt{2}}{2} \end{array} \!\right]. \] Reflecting the last vector across the purple line we get the vector: \[ \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} \\[5pt] \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right] \left[\! \begin{array}{c} \frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2}\end{array} \!\right] = \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right]. \] Now, consider the vector \(\mathbf{e}_2 = \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right]\): Reflecting this vector across the teal line we get the vector: \[ \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right] \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right] = \left[\! \begin{array}{c} -\frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2} \end{array} \!\right]. \] Reflecting the last vector across the purple line we get the vector: \[ \left[\! \begin{array}{cc} \frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} \\[5pt] \frac{\sqrt{2}}{2} & -\frac{\sqrt{2}}{2} \end{array} \!\right] \left[\! \begin{array}{c} -\frac{\sqrt{2}}{2} \\[5pt] -\frac{\sqrt{2}}{2} \end{array} \!\right] = \left[\! \begin{array}{c} -1 \\ 0 \end{array} \!\right]. \] Thus, we have that the linear transformation obtained by first reflecting across the teal line, then reflecting across the purple line maps \[ \left[\! \begin{array}{c} 1 \\ 0 \end{array} \!\right] \quad \text{to} \quad \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right] \qquad \text{and} \qquad \left[\! \begin{array}{c} 0 \\ 1 \end{array} \!\right] \quad \text{to} \quad \left[\! \begin{array}{c} -1 \\ 0 \end{array} \!\right]. \] Hence, the standard matrix of the linear transformation obtained by first reflecting across the purple line, then reflecting across the teal line is \[ \left[\! \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \!\right]. \] We have seen this matrix before. This is the rotation matrix by the angle \(\theta = \pi/2\). That is \[ \left[\! \begin{array}{cc} \cos\left(\dfrac{\pi}{2}\right) & -\sin\left(\dfrac{\pi}{2}\right) \\[5pt] \sin\left(\dfrac{\pi}{2}\right) & \cos\left(\dfrac{\pi}{2}\right) \end{array} \!\right] = \left[\! \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \!\right]. \] In conclusion, the composition of two reflections is a rotation. And the angle of the rotation is the difference of the angles of the first and second reflections. We will prove this later.
-
The image below illustrates the linear transformation from \(\mathbb{R}^2\) into \(\mathbb{R}^3\) given by the matrix
\[
A = \left[\! \begin{array}{cc} 0 & 0 \\[5pt]
1 & 0 \\ 0 & 1 \end{array} \!\right].
\]
-
The image below illustrates the linear transformation from \(\mathbb{R}^2\) into \(\mathbb{R}^3\) given by the matrix
\[
A = \left[\! \begin{array}{cc} 1 & -1 \\[5pt]
2 & 0 \\ 1 & 1 \end{array} \!\right].
\]
-
Here is my attempt to illustrate matrix-vector multiplication with colors:
- We did a part of Section 1.8 on Friday. We will continue on Monday and also do Section 1.9.
- Suggested problems for Section 1.8: 1-4, 12, 13-17, 19, 20, 25, 27, 28
- We discussed Section 1.7: Linear independence today. Suggested problems: 2, 4, 5, 8, 10, 11, 17, 21, 23, 24, 25, 26, 27, 28, 29, 32, 33, 34, 35, 36, 37, 38, 39, 40 and read the next item.
-
Problems 41, 42, 43, 44 in Section 1.7: Linear independence are very interesting and important. The matrices in these problems are not easy to row reduce by hand, so the textbook recommends that we use a calculator. Below I calculated RREFs for the matrices given in Problems 41 and 42. Based on these RREFs you should be able to answer Problems 41, 42, 43, 44.
Problem 41 \[ \left[ \begin{array}{rrrrrr} 8 & -3 & 0 & -7 & 2 \\ -9 & 4 & 5 & 11 & -7 \\ 6 & -2 & 2 & -4 & 4 \\ 5 & -1 & 7 & 0 & 10 \\ \end{array} \right] \sim \quad \cdots \quad \sim \left[ \begin{array}{ccccc} 1 & 0 & 3 & 1 & 0 \\ 0 & 1 & 8 & 5 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 \\ \end{array} \right] \]
Problem 42 \[ \left[ \begin{array}{rrrrrr} 12 & 10 & -6 & -3 & 7 & 10 \\ -7 & -6 & 4 & 7 & -9 & 5 \\ 9 & 9 & -9 & -5 & 5 & -1 \\ -4 & -3 & 1 & 6 & -8 & 9 \\ 8 & 7 & -5 & -9 & 11 & -8 \\ \end{array} \right] \sim \quad \cdots \quad \sim \left[ \begin{array}{rrrrrr} 1 & 0 & 2 & 0 & 2 & 0 \\ 0 & 1 & -3 & 0 & -2 & 0 \\ 0 & 0 & 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ \end{array} \right] \]
-
Below I am offering a simpler matrix and more detailed guidance for the calculations needed for Problems 41, 42, 43, 44 in Section 1.7. This is my version of these four problems.
Problem. Consider the following $4\times 6$ matrix \[ A = \left[ \begin{array}{cccccc} 1 & 1 & 4 & 1 & 0 & 6 \\ 2 & 1 & 3 & 0 & 1 & 4 \\ 3 & 1 & 2 & 1 & 0 & 8 \\ 4 & 1 & 1 & 0 & 1 & 6 \\ \end{array} \right]. \] Denote the columns of $A$ by $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_3,$ $\mathbf{a}_4,$ $\mathbf{a}_5,$ and $\mathbf{a}_6.$ That is, \[ \mathbf{a}_1 = \left[ \begin{array}{cccccc} 1 \\ 2 \\ 3 \\ 4 \\ \end{array} \right], \ \mathbf{a}_2 = \left[ \begin{array}{c} 1\\ 1 \\ 1 \\ 1 \\ \end{array} \right], \ \mathbf{a}_3 = \left[ \begin{array}{c} 4 \\ 3 \\ 2 \\ 1 \\ \end{array} \right], \ \mathbf{a}_4 = \left[ \begin{array}{c} 1 \\ 0 \\ 1 \\ 0 \\ \end{array} \right], \ \mathbf{a}_5 = \left[ \begin{array}{c} 0\\ 1\\ 0\\ 1 \\ \end{array} \right], \ \mathbf{a}_6 = \left[ \begin{array}{c} 6 \\ 4 \\ 8 \\ 6 \\ \end{array} \right]. \] Answer the following questions:
- Find the Reduced Row Echelon Form of $A$. (Please make sure that it is done correctly.)
- Is the vector $\mathbf{a}_1$ linearly independent? Explain using the definition.
- Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2$ linearly independent? Explain using the definition.
- Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_3$ linearly independent? Explain using the definition. If the answer is No, then express $\mathbf{a}_3$ as a linear combination of $\mathbf{a}_1$ and $\mathbf{a}_2.$ That is, find the real numbers $\alpha_1$ and $\alpha_2$ such that \[ \mathbf{a}_3 = \alpha_1 \mathbf{a}_1+ \alpha_2 \mathbf{a}_2. \] Make a record of the numbers $\alpha_1$ and $\alpha_2.$
- Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_4$ linearly independent? Explain using the definition.
- Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_4,$ $\mathbf{a}_5$ linearly independent? Explain using the definition. If the answer is No, then express $\mathbf{a}_5$ as a linear combination of $\mathbf{a}_1,$ $\mathbf{a}_2$ and $\mathbf{a}_4.$ That is, find the real numbers $\beta_1,$ $\beta_2$ and $\beta_4$ such that \[ \mathbf{a}_5 = \beta_1 \mathbf{a}_1+ \beta_2 \mathbf{a}_2 + \beta_4 \mathbf{a}_4. \] Make a record of the numbers $\beta_1,$ $\beta_2$ and $\beta_4.$
- Are the vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_4,$ $\mathbf{a}_6$ linearly independent? Explain using the definition. If the answer is No, then express $\mathbf{a}_6$ as a linear combination of $\mathbf{a}_1,$ $\mathbf{a}_2$ and $\mathbf{a}_4.$ That is, find the real numbers $\gamma_1,$ $\gamma_2$ and $\gamma_4$ such that \[ \mathbf{a}_6 = \gamma_1 \mathbf{a}_1+ \gamma_2 \mathbf{a}_2 + \gamma_4 \mathbf{a}_4. \] Make a record of the numbers $\gamma_1,$ $\gamma_2$ and $\gamma_4.$
- The moral of this problem is that all the information you found about the linear independence and dependence of the columns of $A$ is contained in the Reduced Row Echelon Form of $A$. I would like you to carefully review your findings and discover how are all the facts about the columns of $A$ that you found in this problem encoded in the Reduced Row Echelon Form of $A$.
-
Let me answer some of the questions asked in the above boxed Problem.
- The row reduction algorithm applied to the matrix $A$ gives its Reduced Row Echelon Form: \[ A = \Bigl[ \begin{array}{cccccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 & \mathbf{a}_4 & \mathbf{a}_5 & \mathbf{a}_6 \end{array} \Bigr] = \left[ \begin{array}{cccccc} 1 & 1 & 4 & 1 & 0 & 6 \\ 2 & 1 & 3 & 0 & 1 & 4 \\ 3 & 1 & 2 & 1 & 0 & 8 \\ 4 & 1 & 1 & 0 & 1 & 6 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & -1 & 0 & 0 & 1 \\ 0 & 1 & 5 & 0 & 1 & 2 \\ 0 & 0 & 0 & 1 & -1 & 3 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ \end{array} \right]. \] Remark We obtained the preceding RREF by performing row operations on the rows of $A.$ It is important to realize that the same row operations would lead to the following Reduced Row Echelon Forms: \[ \Bigl[ \begin{array}{cc} \mathbf{a}_1 & \mathbf{a}_2 \end{array} \Bigr] = \left[ \begin{array}{cc} 1 & 1 \\ 2 & 1 \\ 3 & 1 \\ 4 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \\ 0 & 0 \\ 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{ccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 \end{array} \Bigr] = \left[ \begin{array}{ccc} 1 & 1 & 4 \\ 2 & 1 & 3 \\ 3 & 1 & 2 \\ 4 & 1 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & -1 \\ 0 & 1 & 5 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{ccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_4 \end{array} \Bigr] = \left[ \begin{array}{ccc} 1 & 1 & 1 \\ 2 & 1 & 0 \\ 3 & 1 & 1 \\ 4 & 1 & 0 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{cccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_4 & \mathbf{a}_5 \end{array} \Bigr] = \left[ \begin{array}{cccc} 1 & 1 & 1 & 0 \\ 2 & 1 & 0 & 1 \\ 3 & 1 & 1 & 0 \\ 4 & 1 & 0 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & -1 \\ 0 & 0 & 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{cccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_4 & \mathbf{a}_5 \end{array} \Bigr] = \left[ \begin{array}{cccc} 1 & 1 & 1 & 0 \\ 2 & 1 & 0 & 1 \\ 3 & 1 & 1 & 0 \\ 4 & 1 & 0 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & -1 \\ 0 & 0 & 0 & 0 \\ \end{array} \right] \] and \[ \Bigl[ \begin{array}{cccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_4 & \mathbf{a}_6 \end{array} \Bigr] = \left[ \begin{array}{cccc} 1 & 1 & 1 & 6 \\ 2 & 1 & 0 & 4 \\ 3 & 1 & 1 & 8 \\ 4 & 1 & 0 & 6 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccrcrc} 1 & 0 & 0 & 1 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 0 \\ \end{array} \right]. \]
- The vector $\mathbf{a}_1$ is linearly independent. Recall the definition of linear independence. A vector $\mathbf{v}_1$ is linearly independent if the homogeneous equation $x_1 \mathbf{v}_1 = \mathbf{0}$ has only the trivial solution. Consider the homogeneous equation $x_1 \mathbf{a}_1 = \mathbf{0}.$ The preceding equation is equivalent to \[ x_1 \left[ \begin{array}{cccccc} 1 \\ 2 \\ 3 \\ 4 \\ \end{array} \right] = \left[ \begin{array}{cccccc} 0 \\ 0 \\ 0 \\ 0 \\ \end{array} \right], \] which in turn is equivalent to $x_1 = 0.$ Hence the equation $x_1 \mathbf{a}_1 = \mathbf{0}$ has only the trivial solution. This proves that the vector $\mathbf{a}_1$ is linearly independent.
- The vectors $\mathbf{a}_1$ and $\mathbf{a}_2$ are linearly independent. To prove this claim we need to prove that the homogeneous vector equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{0}$ has only the trivial solution. Since we already know that \[ \Bigl[ \begin{array}{cc} \mathbf{a}_1 & \mathbf{a}_2 \end{array} \Bigr] = \left[ \begin{array}{cc} 1 & 1 \\ 2 & 1 \\ 3 & 1 \\ 4 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \\ 0 & 0 \\ 0 & 0 \\ \end{array} \right], \] the equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{0}$ is equivalent to the equation \[ x_1 \left[ \begin{array}{c} 1 \\ 0 \\ 0 \\ 0 \\ \end{array} \right] + x_2 \left[ \begin{array}{c} 0 \\ 1 \\ 0 \\ 0 \\ \end{array} \right] = \left[ \begin{array}{c} 0 \\ 0 \\ 0 \\ 0 \\ \end{array} \right] \] and the last equation is equivalent to $x_1 = 0$ and $x_2 =0$. Hence $\mathbf{a}_1$ and $\mathbf{a}_2$ are linearly independent.
- The vectors $\mathbf{a}_1,$ $\mathbf{a}_2,$ $\mathbf{a}_3$ are not linearly independent. These vectors are linearly dependent. We claim that the homogeneous vector equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + x_3 \mathbf{a}_3 = \mathbf{0}$ has infinitely many solution. Since we already know that \[ \Bigl[ \begin{array}{ccc} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 \end{array} \Bigr] = \left[ \begin{array}{ccc} 1 & 1 & 4 \\ 2 & 1 & 3 \\ 3 & 1 & 2 \\ 4 & 1 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{ccr} 1 & 0 & -1 \\ 0 & 1 & 5 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} \right], \] the equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + x_3 \mathbf{a}_3= \mathbf{0}$ is equivalent to the equation \[ x_1 \left[ \begin{array}{c} 1 \\ 0 \\ 0 \\ 0 \\ \end{array} \right] + x_2 \left[ \begin{array}{c} 0 \\ 1 \\ 0 \\ 0 \\ \end{array} \right] + x_2 \left[ \begin{array}{r} -1 \\ 5 \\ 0 \\ 0 \\ \end{array} \right]= \left[ \begin{array}{c} 0 \\ 0 \\ 0 \\ 0 \\ \end{array} \right]. \] In the last equation the variable $x_3$ is free. All solutions of this equation are given by $x_1 = s,$ $x_2 = -5s,$ $x_3 = s$ where $s$ is any real number. For $s=1$ we get a specific linear dependence relation \[ 1 \mathbf{a}_1 + (-5) \mathbf{a}_2 + 1 \mathbf{a}_3 = \mathbf{0}. \] A different way to get the same relation is to solve the nonhomogeneous equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{a}_3.$ It is very interesting to observe that we already have everything that we need to solve this equation. We already know that \[ \Bigl[ \begin{array}{cc|c} \mathbf{a}_1 & \mathbf{a}_2 & \mathbf{a}_3 \end{array} \Bigr] = \left[ \begin{array}{ccc} 1 & 1 & 4 \\ 2 & 1 & 3 \\ 3 & 1 & 2 \\ 4 & 1 & 1 \\ \end{array} \right] \sim \cdots \sim \left[ \begin{array}{cc|r} 1 & 0 & -1 \\ 0 & 1 & 5 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} \right]. \] Therefore, $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{a}_3$ is equivalent to the equation \[ x_1 \left[ \begin{array}{c} 1 \\ 0 \\ 0 \\ 0 \\ \end{array} \right] + x_2 \left[ \begin{array}{c} 0 \\ 1 \\ 0 \\ 0 \\ \end{array} \right] = \left[ \begin{array}{r} -1 \\ 5 \\ 0 \\ 0 \\ \end{array} \right]. \] Hence the only solution of the nonhomogeneous equation $x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = \mathbf{a}_3$ is $x_1 = -1$ and $x_2 = 5.$ That is \[ \mathbf{a}_3 = (-1) \mathbf{a}_1 +5 \mathbf{a}_2. \] Verify! \[ \left[ \begin{array}{r} 4 \\ 3 \\ 2 \\ 1 \\ \end{array} \right] = (-1) \left[ \begin{array}{c} 1 \\ 2 \\ 3 \\ 4 \\ \end{array} \right] + 5 \left[ \begin{array}{c} 1 \\ 1 \\ 1 \\ 1 \\ \end{array} \right]. \]
- Similar to III.
- Similar to IV.
- Similar to IV and VI.
- Would you please think about this? The connections that are present in RREF are beautiful. We will discuss these many times in future classes.
- We discussed Section 1.6: Applications of Linear Systems. We did four applications. Two from this section: Balancing Chemical Equations and Network Flow. Suggested problems for Section 1.6: 5-8, 11-14. Please pay attention to Problem 11.
-
The third application was finding an approximation for the steady-state temperature distribution of a thin plate when the temperature around the boundary is known. This is explained in Problems 33 and 34 in Section 1.1. These problems explain how to find an approximation for the steady-state temperature distribution of a thin plate when the temperature around the boundary is known. In Problems 33 and 34, approximations for the steady-state temperature distribution are found at only four points. However, this problem gives a blueprint on how to find the approximations at any rectangular grid of points. Below, I do it for a \(2\times 3\) grid of six points and a \(3\times 4\) grid of twelve points
-
Consider the following grid which represents a heated plate:
-
Consider the following grid which represents a heated plate:
-
The following grid has \(20\times 30 = 600\) unknowns and \(2\times (20+30) = 100\) given temperature values.
I did not try it, but I believe that Mathematica would solve the corresponding system of \(600\) equations with \(600\) unknowns relatively quickly and that would give a good approximation of the heat distribution in this plate, The advantage of this system us that it has a lot of zeros. The matrix of this system is of the size \(600\times 600\). Thus it has \(360,000\) entries. Based on the patterns that we observed for small matrices above, I estimate that this \(600\times 600\) matrix has less than \(3,000\) nonzero entries. That is less than 1 in 120 entries is nonzero; that is less than 0.083%.
Matrices with a small percentage of nonzero entries are called sparse matrices. Mathematicians have developed super efficient methods of doing calculations with sparse matrices of huge size.
- In Math 430 we show how to find the steady-state temperature distribution in a thin plate when the temperature around the boundary is known using partial differential equations.
-
Consider the following grid which represents a heated plate:
-
My favorite application of vectors: COLORS. In fact, I love this application so much that I wrote a webpage to celebrate it: Color Cube.
One exercise in this context would be to ask you to find three colors which are between teal and yellow, one in the middle between teal and yellow and the other two in the middle between teal and the mid-color and in the middle of mid-color and yellow.
- Today we finished Section 1.4: The Matrix Equation: $A\mathbf{x} = \mathbf{b}$ and Section 1.5: Solution Sets Of Linear Systems.
-
Section 1.5 talks about writing solution sets of linear systems in parametric vector form.
We explained the relationship between the solution set of the homogeneous equation \[ \color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0} \] and the solution set of a consistent nonhomogeneous equation \[ \color{green}{A}\color{red}{\mathbf{x}} = \color{green}{\mathbf{b}}. \] This is explained in Theorem 6 in the book. Please recognize how this theorem is reflected when the solution of $A \mathbf x = \mathbf b$ is written in parametric vector form.
Suggested problems for Section 1.5: 1, 3, 5, 6, 7, 9, 11, 12, 13-16, 19, 21, 23, 24, 26, 29, 32, 35, 37-40. When you write a formula for the solution of a nonhomogeneous equation in parametric form, try to recognize a particular solution $\color{purple}{\mathbf{p}}$ of the nonhomogeneous equation and a span of one, or two, or three vectors which is the solution of the corresponding homogeneous equation.
-
Let $m, n \in \mathbb{N}.$ Let $\color{green}{A}$ be an $m\times n$ matrix. A few points about the solution set of a homogeneous equation
\[
\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}.
\]
The preceding matrix equation can be rewritten as a homogenous system with $m$ equations and $n$ unknowns. The unknowns are collected in the vector $\color{red}{\mathbf{x}} \in \mathbb{R}^n.$ Here $\mathbf{0}$ is the zero vector in $\mathbb{R}^m.$
- The homogeneous equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ always has a solution (it is consistent). One solution is always the zero vector in $\mathbb{R}^n.$
- Another explanation why the homogeneous equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ always has a solution is to consider its augmented matrix \[ \bigl[\begin{array}{c|c} \color{green}{A} & \mathbf{0} \end{array} \bigr]. \] As we perform row reduction, the last column will always be $\mathbf{0}.$ Therefore the last column of the reduced row echelon form of the matrix $\bigl[\begin{array}{c|c} \color{green}{A} & \mathbf{0} \end{array} \bigr]$ will not be a pivot column. Hence the matrix equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ is consistent.
-
We have the following dichotomy for the solution:
- The zero vector in $\mathbb{R}^n$ is a unique solution. This case occurs if and only if each column of the reduced row echelon form of $\color{green}{A}$ is a pivot column. In other words, there are no free variables.
- There are infinitely many solutions. This case occurs if and only if there are free variables. In this case the solution set can be written as a span of several vectors. In fact, we can write the solution set as a span of as many vectors as there are free variables.
- For example, let $\color{green}{A}$ be a $3\times 4$ matrix and assume that \[ \bigl[\begin{array}{c|c} \color{green}{A} & \mathbf{0} \end{array} \bigr] \sim \cdots \sim \left[\!\begin{array}{rrrr|r} 1 & -2 & 0 & -1 & 0 \\ 0 & 0 & 1 & 2 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array}\! \right] \] In this case we have two free variables and the solution set of the matrix equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ can be written as a span of two vectors in $\mathbb{R}^4.$ To find those vectors we need to solve the corresponding system of linear equations: \begin{alignat*}{10} &x_1 & & - &&2 x_2 & & && && && - && && x_4 && = &&0\\ & & & && & & && &&x_3 && + && 2 && x_4 && = &&0 \end{alignat*} The solution of this system in parametric vector form is \[ \left[\!\begin{array}{c} x_1 \\ x_2 \\ x_3 \\ x_4 \end{array}\! \right] = \left[\!\begin{array}{c} 2 s + t \\ s \\ -2 t \\ t \end{array}\! \right] = \left[\!\begin{array}{c} 2 s + 1 t \\ 1 s + 0 t \\ 0 s -2 t \\ 0 s + 1 t \end{array}\! \right] = s \left[\!\begin{array}{c} 2 \\ 1 \\ 0 \\ 0 \end{array}\! \right] + t \left[\!\begin{array}{r} 1 \\ 0 \\ -2 \\ 1 \end{array}\! \right], \] where $s$ and $t$ are arbitrary real numbers. Hence, the solution set of the homogeneous matrix equation $\color{green}{A}\color{red}{\mathbf{x}} = \mathbf{0}$ can be written as \[ \operatorname{Span}\left\{ \left[\!\begin{array}{c} 2 \\ 1 \\ 0 \\ 0 \end{array}\! \right], \left[\!\begin{array}{r} 1 \\ 0 \\ -2 \\ 1 \end{array}\! \right] \right\}. \]
-
At the beginning of class today you asked me about my favorite color. The answer is TEAL. I love using colors on my websites. On my websites, or in Mathematica, to chose colors I always use RGB (red-green-blue) color encoding. That is expression which encodes each color as a triple of numbers. In HTML the triple for TEAL is written as a #008080. Here HTML uses two three two digit numbers in hexadecimal number system. The hexadecimal digits are 0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F. HTML encodes colors with triples of integers between 0, which they write as 00 and 255 which in hexadecimal number system is FF. In hexadecimal number system, the number 80 stands for 128, half-way between 0 and 255.
It is hard to see a vector in #008080. However, in CSS, the triple for TEAL can be written as rgb(0%,50%,50%). This is getting closer to a vector, when you view percentages as their fractional values. So, the vector (0%,50%,50%) is (0,1/2,1/2). So, each color can be identified by a vector whose components are real numbers between 0 and 1.
So I started the class by talking about the colors in relation to linear algebra. I love the application of vectors to COLORS so much that I wrote a webpage to celebrate it: Color Cube.
-
Did you work on Problem 32 in Section 1.3? I find this problem interesting. I like it so much that I decided to rewrite it and give more information in the associated pictures. I do it in the next item.
An interesting feature of the Problem in the next item is that you do not need to know the specific coordinates of the vectors in the picture to answer the questions. You only need to record the linear relationships among vectors that are clear from the given grids: the vectors $\color{green}{\mathbf{a}_3},$ and $\color{#00FF00}{\mathbf{b}}$ are linear combinations of $\color{green}{\mathbf{a}_1},$ $\color{green}{\mathbf{a}_2}.$ At this point you can solve items (i) and (iii) in Problem in the next item. After you solved item (iii), you can reconstruct (ii) But, you will have more information to answer solve (ii) as we learn more about RREF. An important note: The Problem in the next item will be on the final assignment. So, solving it early, and ask for clarifications if there is something not clear, assures success.
The content of Section 1.5 is very useful for the Problem below.
-
Problem
Consider the vectors $\color{green}{\mathbf{a}_1},$ $\color{green}{\mathbf{a}_2},$ $\color{green}{\mathbf{a}_3},$ and $\color{#00FF00}{\mathbf{b}}$ in $\mathbb{R}^2$ as shown in the picture below.
- Is the vector equation \[ \color{red}{x_1} \color{green}{\mathbf{a}_1} + \color{red}{x_2} \color{green}{\mathbf{a}_2} + \color{red}{x_3} \color{green}{\mathbf{a}_3} = \color{#00FF00}{\mathbf{b}} \] consistent or inconsistent? Explain your reasoning.
- Find the reduced row echelon form of the augmented matrix \[ \left[\begin{array}{ccc|c} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \color{green}{\mathbf{a}_3} & \color{#00FF00}{\mathbf{b}} \end{array} \right]. \]
- Find the solution set of the vector equation \[ \color{red}{x_1} \color{green}{\mathbf{a}_1} + \color{red}{x_2} \color{green}{\mathbf{a}_2} + \color{red}{x_3} \color{green}{\mathbf{a}_3} = \color{#00FF00}{\mathbf{b}} \] and represent the solution set in parametric vector form.
- We finished Section 1.4 and started Section 1.5. Suggested problems for Section 1.4: 1, 5, 13, 14, 15, 16,17-20, 21, 22, 23, 24, 25, 26, 32, 34, 35, 36 and for Section 1.5: 1, 3, 5, 6, 7, 9, 11, 12, 13-16, 19, 21, 23, 24, 26, 29, 32, 35, 37-40.
- Notice that Problems 25 26, 27, and 28 in Section 1.1 are related to Theorem 4 in Section 1.4. These are interesting deep problems. Also notice that how these problems are related to the concept of span. I wrote about this in an item on October 3; the box starting with "Find a relationship between the coordinates ..."
- On Friday, I illustrated content of Sections 1.4 and 1.5 on the following modification of an example that we did before: \begin{alignat*}{5} \require{amsmath} 3 \color{red}{x_{1}} & + & 0 \color{red}{x_{2}} & + & (-1) \color{red}{x_{3}} & + & 1 \color{red}{x_{4}} & = & -2& \\ 1 \color{red}{x_{1}} & + & 1 \color{red}{x_{2}} & + & (-2) \color{red}{x_{3}} & + & 2 \color{red}{x_{4}} & = & 3& \\ 2 \color{red}{x_{1}} & + & (-1) \color{red}{x_{2}} & + & 1 \color{red}{x_{3}} & + & 3 \color{red}{x_{4}} & = & 8 & \\ \end{alignat*} Let us rewrite the above system as a vector equation: \begin{equation*} \color{red}{x_{1}} \left[\! \begin{array}{r} 3 \\ 1\\ 2 \end{array} \!\right] + \color{red}{x_{2}} \left[\! \begin{array}{r} 0 \\1\\ -1 \end{array} \!\right] + \color{red}{x_{3}} \left[\! \begin{array}{r} -1 \\-2\\ 1 \end{array} \!\right] + \color{red}{x_{4}} \left[\! \begin{array}{r} 1 \\ 2\\ 3 \end{array} \!\right] = \left[\! \begin{array}{r} -2 \\ 3 \\ 8 \end{array} \!\right] . \end{equation*} Let us rewrite the above vector equation as a matrix equation: \begin{equation*} \left[\!\begin{array}{rrrr} 3 & 0 & -1 & 1 \\ 1 & 1 & -2 & 2 \\ 2 & -1 & 1 & 3 \end{array}\! \right] \left[\! \begin{array}{c} \color{red}{x_{1}} \\ \color{red}{x_{2}}\\ \color{red}{x_{3}} \\ \color{red}{x_{4}} \end{array} \!\right] = \left[\! \begin{array}{r} -2 \\ 3 \\ 8 \end{array} \!\right] . \end{equation*} To solve any of the any of the above, the system, the vector equation, the matrix equation we form the augmented matrix and row reduce it: \begin{equation*} \left[\!\begin{array}{rrrr|r} 3 & 0 & -1 & 1 & -2 \\ 1 & 1 & -2 & 2 & 3 \\ 2 & -1 & 1 & 3 & 8 \end{array}\! \right] \ \ \sim \quad \cdots \quad \sim \ \ \left[\!\begin{array}{rrrr|r} 1 & 0 & -\frac{1}{3} & 0 & -\frac{7}{4} \\ 0 & 1 & -\frac{5}{3} & 0 & -\frac{7}{4} \\ 0 & 0 & 0 & 1 & \frac{13}{4} \end{array}\! \right] \end{equation*} Thus, the solution exists since the augmented column is not a pivot column. There is one free variable, so \(\color{red}{x_{3}} = s\) is free. To find the solution, we write the system corresponding to the RREF: \begin{alignat*}{5} \require{amsmath} 1 \color{red}{x_{1}} & + & 0 \color{red}{x_{2}} & + & (-\tfrac{1}{3}) \color{red}{x_{3}} & + & 0 \color{red}{x_{4}} & = & -\tfrac{7}{4}& \\ 0 \color{red}{x_{1}} & + & 1 \color{red}{x_{2}} & + & (-\tfrac{5}{3}) \color{red}{x_{3}} & + & 0 \color{red}{x_{4}} & = & -\tfrac{7}{4}& \\ 0 \color{red}{x_{1}} & + & 0 \color{red}{x_{2}} & + & 0 \color{red}{x_{3}} & + & 1 \color{red}{x_{4}} & = & \tfrac{13}{4} & \\ \end{alignat*} Hence, the solution is \begin{alignat*}{3} \color{red}{x_{1}} & = & -\tfrac{7}{4} & + & \tfrac{1}{3} s & \\ \color{red}{x_{2}} & = & -\tfrac{7}{4} & + & \tfrac{5}{3} s & \\ \color{red}{x_{3}} & = & 0 & + & 1 s & \\ \color{red}{x_{4}} & = & \tfrac{13}{4} & + & 0 s & \end{alignat*} where \(s\) is any real number. The simplest way to write this solution is in parametric vector form: \[ \color{red}{\mathbf{x}} = \left[\! \begin{array}{r} -\tfrac{7}{4} \\ -\tfrac{7}{4} \\ 0 \\ \tfrac{13}{4} \end{array} \!\right] + \left[\! \begin{array}{r} \tfrac{1}{3} s \\ \tfrac{5}{3} s \\ 1 s \\ 0 s \end{array} \!\right] = \frac{1}{4} \left[\! \begin{array}{r} - 7 \\ - 7 \\ 0 \\ 13 \end{array} \!\right] + s \frac{1}{3} \left[\! \begin{array}{r} 1 \\ 5 \\ 3 \\ 0 \end{array} \!\right] = \mathbf{p} + s \mkern 3mu \mathbf{v}, \] where we set \[ \mathbf{p} = \frac{1}{4} \left[\! \begin{array}{r} - 7 \\ - 7 \\ 0 \\ 13 \end{array} \!\right] \quad \text{and} \quad \mathbf{v} = \frac{1}{3} \left[\! \begin{array}{r} 1 \\ 5 \\ 3 \\ 0 \end{array} \!\right]. \]
- I used Wolfram Mathematica to help me with the row reduction in the preceding item.
- Computer algebra system Wolfram Mathematica will be very useful in this class. To get started with Mathematica see my Mathematica page. Please watch the videos that are on my Mathematica page. Watching the movies is very helpful to get started with Mathematica efficiently! Mathematica is available in the computer labs in BH 215, HH 233 and in the Math Center BH 209/211A.
- In class I created a small Mathematica notebook 20241004.nb in which I performed the above row reduction. I also created 20241004.pdf, for you to enjoy this file without access to Mathematica.
- Today we discussed Section 1.3: Vector Equations. Suggested problems for Section 1.3: 1, 5, 9, 15, 17, 18, 19, 21-25, 32. We are a liitle late. We did mention how Section 1.3: The Matrix Equation \(A \mathbf{x} = \mathbf{b}\) ties into this. More about it tomorrow.
- To internalize algebraic operations with vectors it is useful to draw pictures in $\mathbb{R}^2$ (the Cartesian plane) and $\mathbb{R}^3$ (the Euclidean space). That is what I will do below.
-
Consider two specific vectors in $\mathbb{R}^3$
\[
\bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1
\end{array} \!\right]} \quad \text{and} \quad \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2
\end{array} \!\right]}
\]
as given in the picture below:
- We ask the question: Is the vector $\left[\! \begin{array}{r} 2 \\ 1 \\ -4 \end{array} \!\right]$ in the span of the vectors $\bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}$ and $\bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]}$? Or, the same question in mathematical notation: Is the following statement true \[ \left[\begin{array}{r} 2 \\ 1 \\ -4 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}? \] And two more questions: Is the following statement true \[ \left[\begin{array}{r} 6 \\ -7 \\ 0 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}? \qquad \left[\begin{array}{r} -7 \\ -1 \\ 5 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}? \]
-
Really interesting aspect of linear algebra is that these three questions are just vector equations in disguise. In fact, we are asking if the following vector equations have solutions (are consistent):
\[
x_1 \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1
\end{array} \!\right]} +x_2 \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2
\end{array} \!\right]} = \left[\begin{array}{r} 2 \\ 1 \\ -4
\end{array} \right], \quad
x_1 \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1
\end{array} \!\right]} +x_2 \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2
\end{array} \!\right]} = \left[\begin{array}{r} 6 \\ -7 \\ 0
\end{array} \right], \quad
x_1 \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1
\end{array} \!\right]} +x_2 \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2
\end{array} \!\right]} = \left[\begin{array}{r} -7 \\ -1 \\ 5
\end{array} \right].
\]
In turn, these vector equations are in fact systems of linear equations, like we did on the first day of classes:
These systems are solved by row reducing the augmented matrices. But, this is huge, we can do three row reductions in one go: \begin{align*} \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ -3 & 1 & 1 & -7& -1 \\ 1 & 2 & -4& 0& 5 \end{array}\! \right] & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ 0 & -11 & 7 & 11 & -22 \\ 0 & 6 & -6& -6 & 12 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ 0 & 1 & -\frac{7}{11} & -1 & 2 \\ 0 & 1 & -1& -1 & 2 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ 0 & 1 & -\frac{7}{11} & -1 & 2 \\ 0 & 0 & -\frac{4}{11}& 0 & 0 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 2 & 6& -7 \\ 0 & 1 & -\frac{7}{11} & -1 & 2 \\ 0 & 0 & 1 & 0 & 0 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & -4 & 0 & 6& -7 \\ 0 & 1 & 0 & -1 & 2 \\ 0 & 0 & 1 & 0 & 0 \end{array}\! \right] \\ & \sim \left[\!\begin{array}{rr|rrr} 1 & 0 & 0 & 2 & 1 \\ 0 & 1 & 0 & -1 & 2 \\ 0 & 0 & 1 & 0 & 0 \end{array}\! \right] \qquad \text{this matrix is in RREF} \end{align*}System 1 Systems 2 Systems 3 \begin{alignat*}{8} &x_1 & & - 4 &x_2 & &=&& 2\\ -3 &x_1 & & + &x_2 & & =&& 1 \\ &x_1 & & + 2 &x_2 & & =&& -4 \end{alignat*} \begin{alignat*}{8} &x_1 & & - 4 &x_2 & &=&& 6\\ -3 &x_1 & & + &x_2 & & =&&-7 \\ &x_1 & & + 2 &x_2 & & =&& 0 \end{alignat*} \begin{alignat*}{8} &x_1 & & - 4 &x_2 & &=&& -7\\ -3 &x_1 & & + &x_2 & & =&& -1 \\ &x_1 & & + 2 &x_2 & & =&& 5 \end{alignat*} -
Conclusions:
- Since the the first augmented column of the RREF is a pivot column and this augmented column corresponds to System 1, we conclude that System 1 is inconsistent. Therefore, it is true that \[ \left[\begin{array}{r} 2 \\ 1 \\ -4 \end{array} \right] \not\in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}. \]
- Since the second augmented column of the RREF is not a pivot column and this augmented column corresponds to System 2, we conclude that System 2 is consistent. From the RREF we can read even more \[ \left[\begin{array}{r} 6 \\ -7 \\ 0 \end{array} \right] = 2 \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]} - \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]}. \] Therefore, it is true that \[ \left[\begin{array}{r} 6 \\ -7 \\ 0 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}. \]
- Since the third augmented column of the RREF is not a pivot column and this augmented column corresponds to System 3, we conclude that System 3 is consistent. From the RREF we can read even more \[ \left[\begin{array}{r} -7 \\ -1 \\ 5 \end{array} \right] = \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]} + 2 \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]}. \] Therefore, it is true that \[ \left[\begin{array}{r} -7 \\ -1 \\ 5 \end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1 \end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2 \end{array} \!\right]} \right\}. \]
The questions which are answered in the previous items can be answered by stating a universal problem:
-
The picture below is an attempt to illustrate
\[
\operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1
\end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2
\end{array} \!\right]} \right\}
\]
geometrically. In the picture below this span is represented by the yellow translucent plane in $\mathbb{R}^3$. In this plane I placed a grid imposed by the green and the blue vector. To indicate that there are many vectors in this plane, I placed nine gray vectors in the picture all of which are in the span.
-
Above, we focused primarily on practical calculations. However, the key takeaway from today's presentation is that three concepts—a system of linear equations, a linear vector equation, and a matrix equation—are mathematically equivalent.
In the following items, I will provide all the details behind the reasoning for this claim.
-
Let \( m \) and \( n \) be positive integers. Using set notation, we write: \( m, n \in \mathbb{N} \). This is read as: \( m \) and \( n \) are elements of the set of positive integers.
- Linear System. Consider a linear system with $m$ linear equations each with $n$ unknowns. A linear system with $m$ equations each with $n$ unknowns can be written as follows: \begin{equation} \label{eq:LS} \tag{LS} \require{amsmath} \begin{array}{ccccccccc} \color{green}{a_{11}} \color{red}{x_{1}} & + & \color{green}{a_{12}} \color{red}{x_{2}} & + & \cdots & + & \color{green}{a_{1n}} \color{red}{x_{n}} & = & \color{green}{b_{1}} \\ \color{green}{a_{21}} \color{red}{x_{1}} & + & \color{green}{a_{22}} \color{red}{x_{2}} & + & \cdots & + & \color{green}{a_{2n}} \color{red}{x_{n}} & = & \color{green}{b_{2}} \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ \color{green}{a_{m1}} \color{red}{x_{1}} & + & \color{green}{a_{m2}} \color{red}{x_{2}} & + & \cdots & + & \color{green}{a_{mn}} \color{red}{x_{n}} & = & \color{green}{b_{m}} \end{array} \end{equation}
- Steps towards Vector Equation. These \(m\) equations we can write as one vector equation \begin{equation*} \require{amsmath} \left[\!\begin{array}{c} \color{green}{a_{11}} \color{red}{x_{1}}+ \color{green}{a_{12}} \color{red}{x_{2}} + \cdots + \color{green}{a_{1n}} \color{red}{x_{n}} \\ \color{green}{a_{21}} \color{red}{x_{1}} + \color{green}{a_{22}} \color{red}{x_{2}} + \cdots + \color{green}{a_{2n}} \color{red}{x_{n}} \\ \vdots \\ \color{green}{a_{m1}} \color{red}{x_{1}} + \color{green}{a_{m2}} \color{red}{x_{2}} + \cdots + \color{green}{a_{mn}} \color{red}{x_{n}} \end{array} \!\right] = \left[\! \begin{array}{c} \color{green}{b_{1}} \\ \color{green}{b_{2}} \\ \vdots \\ \color{green}{b_{m}} \end{array} \!\right] \end{equation*} Using the definition of the vector addition, the vector on the left-hand side of the preceding equation can be written as a sum of \(n\) vectors: \begin{equation*} \left[\!\begin{array}{c} \color{red}{x_{1}} \color{green}{a_{11}} \\ \color{red}{x_{1}} \color{green}{a_{21}} \\ \vdots \\ \color{red}{x_{1}}\color{green}{a_{m1}} \end{array} \!\right] + \left[\!\begin{array}{c} \color{red}{x_{2}} \color{green}{a_{12}} \\ \color{red}{x_{2}} \color{green}{a_{22}} \\ \vdots \\ \color{red}{x_{2}}\color{green}{a_{m2}} \end{array}\!\right] + \cdots + \left[\!\begin{array}{c} \color{red}{x_{n}} \color{green}{a_{1n}} \\ \color{red}{x_{n}} \color{green}{a_{2n}} \\ \vdots \\ \color{red}{x_{n}}\color{green}{a_{mn}} \end{array}\!\right] = \left[\! \begin{array}{c} \color{green}{b_{1}} \\ \color{green}{b_{2}} \\ \vdots \\ \color{green}{b_{m}} \end{array}\!\right] \end{equation*} Using the definition of vector scaling, we can factor out the common red scalar in each of the \(n\) vector on the left-hand side of the preceding equation: \begin{equation*} \color{red}{x_{1}} \left[\! \begin{array}{c} \color{green}{a_{11}} \\ \color{green}{a_{21}} \\ \vdots \\ \color{green}{a_{m1}} \end{array} \!\right] + \color{red}{x_{2}} \left[\! \begin{array}{c} \color{green}{a_{12}} \\ \color{green}{a_{22}} \\ \vdots \\ \color{green}{a_{m2}} \end{array} \!\right] + \cdots + \color{red}{x_{n}} \left[\! \begin{array}{c} \color{green}{a_{1n}} \\ \color{green}{a_{2n}} \\ \vdots \\ \color{green}{a_{mn}} \end{array} \!\right] = \left[\! \begin{array}{c} \color{green}{b_{1}} \\ \color{green}{b_{2}} \\ \vdots \\ \color{green}{b_{m}} \end{array} \!\right] \end{equation*}
- Vector Equation.Now we introduce short boldface names for \(n+1\) vectors in the space \(\mathbb{R}^m\): \begin{equation*} \color{green}{\mathbf{a}_1} = \left[\! \begin{array}{c} \color{green}{a_{11}} \\ \color{green}{a_{21}} \\ \vdots \\ \color{green}{a_{m1}} \end{array} \!\right], \quad \color{green}{\mathbf{a}_2} = \left[\! \begin{array}{c} \color{green}{a_{12}} \\ \color{green}{a_{22}} \\ \vdots \\ \color{green}{a_{m2}} \end{array} \!\right], \quad \cdots , \quad \color{green}{\mathbf{a}_n} = \left[\! \begin{array}{c} \color{green}{a_{1n}} \\ \color{green}{a_{2n}} \\ \vdots \\ \color{green}{a_{mn}} \end{array} \!\right], \quad \color{green}{\mathbf{b}} = \left[\! \begin{array}{c} \color{green}{b_{1}} \\ \color{green}{b_{2}} \\ \vdots \\ \color{green}{b_{m}} \end{array} \!\right]. \end{equation*} Notice that the vectors in the preceding displayed expression are vectors in \(\mathbb{R}^m\). Using these vectors we can rewrite the system \eqref{eq:LS} as one vector equation: \begin{equation} \label{eq:VE} \tag{VE} \color{red}{x_{1}} \color{green}{\mathbf{a}_1} + \color{red}{x_{2}} \color{green}{\mathbf{a}_2} + \cdots + \color{red}{x_{n}} \color{green}{\mathbf{a}_n} = \color{green}{\mathbf{b}}. \end{equation}
- Matrix Equation. One more step: We introduce a \(m\times n\) matrix, call it \(\color{green}{A}\), whose columns are the vectors \(\color{green}{\mathbf{a}_1}, \color{green}{\mathbf{a}_2}, \ldots, \color{green}{\mathbf{a}_n}\), which, as we know are vectors in \(\mathbb{R}^m\): \begin{equation*} \color{green}{A} = \left[\!\begin{array}{cccc} \color{green}{a_{11}} & \color{green}{a_{12}} & \cdots & \color{green}{a_{1n}} \\ \color{green}{a_{21}} & \color{green}{a_{22}} & \cdots & \color{green}{a_{2n}} \\ \vdots & \vdots & \ddots & \vdots \\ \color{green}{a_{m1}} & \color{green}{a_{m2}} & \cdots & \color{green}{a_{mn}} \\ \end{array}\! \right] = \bigl[ \begin{array}{cccc} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \cdots & \color{green}{\mathbf{a}_n} \end{array} \bigr] \end{equation*} We also introduce a red vector with \(n\) components, that is a vector in \(\mathbb{R}^n\), which consists of the unknowns \(\color{red}{x_{1}}, \color{red}{x_{2}}, \ldots, \color{red}{x_{n}}\): \begin{equation*} \color{red}{\mathbf{x}} = \left[\! \begin{array}{c} \color{red}{x_{1}} \\ \color{red}{x_{2}} \\ \vdots \\ \color{red}{x_{n}} \end{array} \!\right]. \end{equation*} Using the definition of matrix-vector multiplication we can write the vector equation \eqref{eq:VE} as a matrix equation \begin{equation} \label{eq:ME} \tag{ME} \color{green}{A} \color{red}{\mathbf{x}} = \color{green}{\mathbf{b}}. \end{equation} The connection between the vector equation \eqref{eq:VE} and the matrix equation \eqref{eq:ME} is the definition of the matrix-vector multiplication: the product of the matrix \begin{equation*} \color{green}{A} = \left[\!\begin{array}{cccc} \color{green}{a_{11}} & \color{green}{a_{12}} & \cdots & \color{green}{a_{1n}} \\ \color{green}{a_{21}} & \color{green}{a_{22}} & \cdots & \color{green}{a_{2n}} \\ \vdots & \vdots & \ddots & \vdots \\ \color{green}{a_{m1}} & \color{green}{a_{m2}} & \cdots & \color{green}{a_{mn}} \\ \end{array}\! \right] = \bigl[ \begin{array}{cccc} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \cdots & \color{green}{\mathbf{a}_n} \end{array} \bigr] \end{equation*} and the vector \begin{equation*} \color{red}{\mathbf{x}} = \left[\! \begin{array}{c} \color{red}{x_{1}} \\ \color{red}{x_{2}} \\ \vdots \\ \color{red}{x_{n}} \end{array} \!\right] \end{equation*} equals the linear combination of the columns of \(\color{green}{A} = \bigl[ \begin{array}{cccc} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \cdots & \color{green}{\mathbf{a}_n} \end{array} \bigr]\) which are scaled by scalars \(\color{red}{x_{1}}, \color{red}{x_{2}}, \ldots, \color{red}{x_{n}}\). That is \begin{equation*} \require{amsmath} \require{graphicx} \color{green}{A} \color{red}{\mathbf{x}} = \bigl[ \begin{array}{cccc} \color{green}{\mathbf{a}_1} & \color{green}{\mathbf{a}_2} & \cdots & \color{green}{\mathbf{a}_n} \end{array} \bigr] \left[\! \begin{array}{c} \color{red}{x_{1}} \\ \color{red}{x_{2}} \\ \vdots \\ \color{red}{x_{n}} \end{array} \!\right] \overset{\text{definition}}{=\mkern -1.5mu=\mkern -1.5mu=\mkern -1.5mu=} \ \color{red}{x_{1}} \color{green}{\mathbf{a}_1} + \color{red}{x_{2}} \color{green}{\mathbf{a}_2} + \cdot + \color{red}{x_{n}} \color{green}{\mathbf{a}_n}. \end{equation*}
-
Finally: To solve any of the three equations: the linear system in \eqref{eq:LS}, the linear vector equation \eqref{eq:VE}, or the matrix equation \eqref{eq:ME}, we form the augmented matrix:
\begin{equation} \label{eq:AM} \tag{AM}
\left[\!\begin{array}{cccc|c}
\color{green}{a_{11}} & \color{green}{a_{12}} & \cdots & \color{green}{a_{1n}} & \color{green}{b_{1}} \\
\color{green}{a_{21}} & \color{green}{a_{22}} & \cdots & \color{green}{a_{2n}} & \color{green}{b_{2}} \\
\vdots & \vdots & \ddots & \vdots & \vdots \\
\color{green}{a_{m1}} & \color{green}{a_{m2}} & \cdots & \color{green}{a_{mn}} & \color{green}{b_{m}} \\
\end{array}\! \right]
\end{equation}
and row reduce it.
It is important to note that the above augmented matrix \eqref{eq:AM} is completely green matrix with $m$ rows and $n+1$ columns. The last column is the augmented column.
It is important to note that the above augmented matrix \eqref{eq:AM} is completely green matrix with $m$ rows and $n+1$ columns. The last column is the augmented column.
Find a relationship between the coordinates $b_1,$ $b_2$ and $b_3$ such that the following relationship is satisfied:
\[
\left[\begin{array}{c} b_1 \\ b_2 \\ b_3
\end{array} \right] \in \operatorname{Span}\left\{ \bbox[#50FF50]{\left[\! \begin{array}{r} 1 \\ -3 \\ 1
\end{array} \!\right]}, \bbox[#8080FF]{\left[\! \begin{array}{r} -4 \\ 1 \\ 2
\end{array} \!\right]} \right\}.
\]
- We have moved on to Section 1.3: Vector Equations, but since the Reduced Row Echelon Form of a matrix is a key concept introduced in this class, I will summarize its important aspects in today's post.
- In the items below I will define the Reduced Row Echelon Form (RREF) of a matrix.
- A useful link is Wikipedia's Row echelon form.
- A matrix whose all entries are zero is called a zero matrix. A row of a matrix is said to be a zero row if all the entries in that row are zero. The leftmost nonzero entry of a nonzero row is called the leading entry of a row. The zeros preceding the leading entry are called the leading zeros of a row. All entries of a zero row are leading zeros.
- This is a restatement of Wikipidia's definition of reduced row echelon form:
Each zero matrix is in Reduced Row Echelon Form. A nonzero matrix is in Reduced Row Echelon Form if it satisfies the following three conditions:
- Each nonzero row has strictly more leading zeros then the row above it.
- The leading entry of each nonzero row is equal to $1$.
- The leading entry of each nonzero row is the only nonzero entry in its column.
- Some consequences of the definition of Reduced Row Echelon Form in the preceding item are:
- If a nonzero matrix is in Reduced Row Echelon Form, then all its zero rows are at the bottom.
- If a nonzero matrix is in Reduced Row Echelon Form, then its first row has the fewest leading zeros. The second row has more leading zeros than the first row, the third row has more leading zeros than the second row, and so on, ...
- Let $m, n, k \in \mathbb{N}.$ If a nonzero $m\!\times\!n$ matrix is in Reduced Row Echelon Form and it has $k$ leading entries, then its columns containing the leading entries are the first $k$ columns of the identity matrix $I_m$ in exactly the same order as in the identity matrix $I_m$. In other words, the pivot columns are the first $k$ columns of the identity matrix $I_m$.
-
To illustrate the definition of the reduced row echelon form I made a picture of a $12\times 25$ matrix (that is a matrix with $12$ rows and $25$ columns which is in reduced row echelon form. To emphasize the role of the leading zeros, I colored them blue. To emphasize the role of leading $1$s, I colored them green. I also colored in green the each pivot column. The entries of the matrix that are not specifically written as $0$s or $1$s can be any real numbers. Please notice how blue leading zeros form steps which climb from right to left. Also notice that each stair hosts a pivot position, that is the leading $1$ in a nonzero row.
- A very good exercise in understanding RREF (reduced row echelon form) is to write down all $2\!\times\!3$ matrices which are in RREF. There are seven of them. Below I list the matrix with no pivot columns, then all matrices with one pivot column, then the matrices with two pivot columns. Those are all possibilities since each pivot column has pivot position occupied by the leading entry which needs its own row and we are studying matrices with only two rows. In the matrices below stars ($*$) stand for arbitrary real numbers. \begin{align*} & \begin{bmatrix} 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 0 & 1 \\ 0 & 0 & 0 \end{bmatrix},\quad \begin{bmatrix} 0 & 1 & * \\ 0 & 0 & 0 \end{bmatrix},\quad \begin{bmatrix} 1 & * & * \\ 0 & 0 & 0 \end{bmatrix},\\ & \begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}, \quad \begin{bmatrix} 1 & * & 0 \\ 0 & 0 & 1 \end{bmatrix}, \quad \begin{bmatrix} 1 & 0 & * \\ 0 & 1 & * \end{bmatrix}, \quad \end{align*}
- Here are all possible $3\!\times\!4$ matrices in RREF (reduced row echelon form). There are fifteen of them. \begin{align*} & \begin{bmatrix} 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 0 & 0 & 1 & * \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix},\ \begin{bmatrix} 0 & 1 & * & * \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 1 & * & * & * \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix},\\ & \begin{bmatrix} 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 0 & 1 & * & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 1 & * & * & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 0 & 1 & 0 & * \\ 0 & 0 & 1 & * \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 1 & * & 0 & * \\ 0 & 0 & 1 & * \\ 0 & 0 & 0 & 0 \end{bmatrix}, \ \begin{bmatrix} 1 & 0 & * & * \\ 0 & 1 & * & * \\ 0 & 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}, \ \begin{bmatrix} 1 & * & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix},\ \begin{bmatrix} 1 & 0 & * & 0 \\ 0 & 1 & * & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}, \ \begin{bmatrix} 1 & 0 & 0 & * \\ 0 & 1 & 0 & * \\ 0 & 0 & 1 & * \end{bmatrix}, \end{align*}
- Here are all possible $4\!\times\!3$ matrices in RREF (reduced row echelon form). There are eight of them. \begin{align*} & \begin{bmatrix} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \quad \begin{bmatrix} 0 & 1 & * \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix},\quad \begin{bmatrix} 1 & * & * \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \quad \begin{bmatrix} 1 & * & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \quad \begin{bmatrix} 1 & 0 & * \\ 0 & 1 & * \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}, \\ & \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{bmatrix}. \end{align*}
- Notice that in all the matrices listed in the preceding item the bottom row is the zero row. By removing that zero row we would get all possible $3\!\times\!3$ matrices in RREF (reduced row echelon form). Thus, there is eight different types of $3\!\times\!3$ matrices in RREF (reduced row echelon form). This is true in general: If $m\geq n$, than the number of different types of $m\!\times\!n$ matrices in RREF is the same as the number of different types of $n\!\times\!n$ matrices in RREF.
- Another good exercise is to think of the above matrices as augmented matrices of systems of equations and state for each corresponding system whether it has: No Solutions (NS), Unique Solution (US), Infinitely Many Solutions (MS).
- Another interesting exercise, but for Math 309, is to calculate how many $m\!\times\!n$ matrices are in RREF. Just for the record: Let $k$ be a nonnegative integer and let $m, n$ be positive integers such that $k \leq m \leq n.$ It turns out that there are exactly \[ \displaystyle \binom{n}{k} = \frac{n!}{k!(n-k)!} \] matrices of size $m\!\times\!n$ with $k$ pivots (leading $1$s). The symbol $\displaystyle \binom{n}{k}$ is the standard notation for the binomial coefficient $n$-choose-$k$. Since an arbitrary $m\!\times\!n$ matrix can have $k \in \{0,1,\ldots,m\}$ pivots, it follows that the number of possible RREF types for an $m\!\times\!n$ matrix is \[ \sum_{k=0}^m \binom{n}{k}. \] For example, with $m=3$ and $n=4$, as seen above, there are \[ \binom{4}{0} + \binom{4}{1} + \binom{4}{2} + \binom{4}{3} = 1 + 4 + 6 + 4 = 15. \] possible RREF types of matrices. It is interesting to observe that for an $n\!\times\!n$ matrix there are \[ \sum_{k=0}^n \binom{n}{k} = \binom{n}{0} + \binom{n}{1} + \cdots + \binom{n}{n-1} + \binom{n}{n} = 2^n \] possible RREF types. If you take Math 309 you will understand the background of these formulas.
- In conclusion, for an $m\!\times\!n$ matrix with $m\lt n$ there are \[ \sum_{k=0}^m \binom{n}{k} \] possible RREF types and if $m \geq n$ there are \[ 2^n \] possible RREF types.
- This post is lengthy because it repeats all the arguments from last week's classes. Please read it carefully, especially the conclusion. I've also illustrated the equations with pictures, using the fact that that the graph of a linear equation in three variables is a plane in three-dimensional space \(\mathbb{R}^3\). If you're not yet familiar with this setting, please revisit this post when we cover vectors in \(\mathbb{R}^3\). Most of the post is independent of the geometric illustrations.
- Read Section 1.1 and Section 1.2. Assigned problems for Section 1.1: 3, 6, 7, 5, 9, 11,12, 16, 17, 18, 21, 22, 23, 24, 25, 27, 31, 33, 34. Assigned problems for Section 1.2: 3, 5, 6, 7, 8, 12, 17-31.
-
Today we solved the following system of linear equations:
\begin{alignat*}{8}
&x_1 & & + &&x_2 & & - &&2 &&x_3 &&= -&&5\\
2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\
3 &x_1 & & && & & - && && x_3 &&= &&3
\end{alignat*}
I will give all the details of this solution and illustrate wth pictures. This system is solved by row reducing its augmented matrix:
\begin{align*}
\left[\!\begin{array}{crr|r}
1 & 1 & -2 & -5 \\
2 & -1 & 1 & 8 \\
3 & 0 & -1 & 3
\end{array}\! \right] & \sim
\left[\!\begin{array}{crr|r}
1 & 1 & -2 & -5 \\
0 & -3 & 5 & 18 \\
0 & -3 & 5 & 18
\end{array}\! \right] \quad \begin{array}{l}
\text{(two row replacements)}
\end{array} \\
& \sim
\left[\!\begin{array}{crr|r}
1 & 1 & -2 & -5 \\
0 & -3 & 5 & 18 \\
0 & 0 & 0 & 0
\end{array}\! \right] \quad \begin{array}{l}
\text{(one row replacements)} \end{array} \\
& \sim
\left[\!\begin{array}{crr|r}
1 & 1 & -2 & -5 \\
0 & 1 & -5/3 & -6 \\
0 & 0 & 0 & 0
\end{array}\! \right] \quad \begin{array}{l}
\text{(one row scaling)} \\[-5pt]
\text{(and one row replacements to obtain} \\[-6pt]
\text{the matrix below)}
\end{array} \\
& \sim
\left[\!\begin{array}{crr|r}
1 & 0 & -1/3 & 1 \\
0 & 1 & -5/3 & -6 \\
0 & 0 & 0 & 0
\end{array}\! \right] \quad \begin{array}{l}
\text{This matrix is in Reduced Row Echelon Form.} \\[-5pt]
\text{I always like to row reduce a matrix} \\[-5pt]
\text{to its RREF - Reduced Row Echelon Form.} \\[-5pt]
\text{For the definition of RREF see the end of this post.}
\end{array}
\end{align*}
Why are the matrices presented above row equivalent?
- On the first line, the two matrices are row equivalent since the matrix on the right is obtained from the matrix on the left by performing two row operations. The first row operation was a row replacement: the second row $\left[\!\begin{array}{crr|r} 2 & -1 & 1 & 8 \end{array}\! \right]$ is replaced by the sum of itself and the multiple of the first row $ \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right]$ by $-2$: \begin{align*} \left[\!\begin{array}{crr|r} 2 & -1 & 1 & 8 \end{array}\! \right] + (-2) \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right] & = \left[\!\begin{array}{crr|r} 2 & -1 & 1 & 8 \end{array}\! \right] + \left[\!\begin{array}{crr|r} -2 & -2 & 4 & 10 \end{array}\! \right] \\ & = \left[\!\begin{array}{crr|r} 2 - 2 & -1 - 2 & 1 + 4 & 8 + 10 \end{array}\! \right] \\ & = \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] \end{align*} The second row operation was again a row replacement: the third row $\left[\!\begin{array}{crr|r} 3 & 0 & -1 & 3 \end{array}\! \right]$ is replaced by the sum of itself and the multiple of the first row $ \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right]$ by $-3$: \begin{align*} \left[\!\begin{array}{crr|r} 3 & 0 & -1 & 3 \end{array}\! \right] + (-3) \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right] & = \left[\!\begin{array}{crr|r} 3 & 0 & -1 & 3 \end{array}\! \right] + \left[\!\begin{array}{crr|r} -3 & -3 & 6 & 15 \end{array}\! \right] \\ & = \left[\!\begin{array}{crr|r} 3 - 3 & 0 - 3 & -1 + 6 & 3 + 15 \end{array}\! \right] \\ & = \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] \end{align*} Thinking about the corresponding system of linear equations, what we achieved at this step is that we eliminated unknown $x_1$ from the second and the third equation.
- The matrix on the second line is row equivalent to the matrix above it since it is obtained from the matrix above it by performing a row replacement: the third row $\left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right]$ is replaced by the sum of itself and the second row $ \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right]$ multiplied by $-1$: \begin{align*} \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] + (-1) \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] & = \left[\!\begin{array}{crr|r} 0 & 0 & 0 & 0 \end{array}\! \right] \end{align*} Thinking about the corresponding system of linear equations, what we achieved at this step is that we reduced the number of equations from three to two.
- The matrix on the third line is row equivalent to the matrix above itself since it is obtained from the matrix above itself by performing a row scaling: the second row $\left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right]$ is replaced by the multiple of itself $-1/3$: \[ (-1/3) \left[\!\begin{array}{crr|r} 0 & -3 & 5 & 18 \end{array}\! \right] = \left[\!\begin{array}{crr|r} 0 & 1 & -5/3 & -6 \end{array}\! \right] \] Thinking about the corresponding system of linear equations, what we achieved at this step is that we now have the coefficient $1$ with the unknown $x_2$ in the second equation.
- The matrix on the fourth line is row equivalent to the matrix above itself since it is obtained from the matrix above itself by performing a row replacement: the first row $\left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right]$ is replaced by the sum of itself and the second row $ \left[\!\begin{array}{crr|r} 0 & 1 & -5/3 & -6 \end{array}\! \right]$ multiplied by $-1$: \begin{align*} \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \end{array}\! \right] + (-1) \left[\!\begin{array}{crr|r} 0 & 1 & -5/3 & -6 \end{array}\! \right] & = \left[\!\begin{array}{crr|r} 1 & 0 & -1/3 & 1 \end{array}\! \right] \end{align*} Thinking about the corresponding system of linear equations, what we achieved at this step is that we eliminated unknown $x_2$ from the first equation.
- Since the matrices \[ \left[\!\begin{array}{crr|r} 1 & 1 & -2 & -5 \\ 2 & -1 & 1 & 8 \\ 3 & 0 & -1 & 3 \end{array}\! \right] \sim \left[\!\begin{array}{crr|r} 1 & 0 & -1/3 & 1 \\ 0 & 1 & -5/3 & -6 \\ 0 & 0 & 0 & 0 \end{array}\! \right] \] are row equivalent, the corresponding systems are equivalent systems. That is, the solution set of the given system is the same as the solution set of the system \begin{alignat*}{8} &x_1 & & && & & - &&1/3 &&\, x_3 &&= &&1\\ & & & &&x_2 & & - &&5/3 &&\, x_3 &&= -&&6 \end{alignat*} In the preceding system there is no restriction on $x_3$; $x_3$ is so-called free variable. I like introducing a new name for the free variable, say $s$. Then the solution of the preceding system is given by \begin{equation*} x_1 = \frac{1}{3} s + 1, \quad x_2 = \frac{5}{3} s - 6, \quad x_3 = s, \end{equation*} where $s$ is an arbitrary real number. In set notation, $s \in \mathbb{R}.$ For example, with $s=3$ we obtain that \[ x_1 = 2, \quad x_2 = -1, \quad x_3 = 3, \] is a solution to the given system.
- It is always fun to verify whether the solution realy satisfies the given system. Substitute $x_1 = \frac{1}{3} s + 1,$ $x_2 = \frac{5}{3} s - 6$ and $x_3 = s,$ and see what we get: \begin{alignat*}{8} &(1/3 \, s + 1) & & + &&(5/3 \, s - 6) & & - &&2 &&s &&= 2s - 5 +2s = -&&5\\ 2 &(1/3 \, s + 1) & & - &&(5/3 \, s - 6) & & + && &&s &&= -s + 8 + s = &&8 \\ 3 &(1/3 \, s + 1) & & && & & - && && s &&= s + 3 - s = &&3 \end{alignat*} Hence we found all the solutions of the given system.
-
Finally, let us illustrate the system
\begin{alignat*}{8}
&x_1 & & + &&x_2 & & - &&2 &&x_3 &&= -&&5\\
2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\
3 &x_1 & & && & & - && && x_3 &&= &&3
\end{alignat*}
geometrically in $x_1x_2x_3$-space, that is in the space $\mathbb{R}^3.$ Each equation is represented by a plane. The first equation is represented by the brown plane, the second equation is represented by the blue plane and the third equation is represented by the green plane.
-
A natural question is: How are row operations reflected in the illustration of the system with planes?
The systems that correspond to the row equivalent matrices on lines 2 and 3 in the row reduction above are illustrated by the same picture; the planes do not change. The system corresponding to the last matrix in Reduced Row Echelon Form is represented by the simplest planes. What is the simplicity of these planes? The first plane is parallel to the coordinate axis $x_2.$ The second plane is parallel to the coordinate axis $x_1.$ (Being parallel to coordinate ases is a very desirable feature for a plane.)Given System Systems 2 and 3 "Row Reduced" System \begin{alignat*}{8} &x_1 & & + &&x_2 & & - &&2 &&x_3 &&= -&&5\\ 2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\ 3 &x_1 & & && & & - && && x_3 &&= &&3 \end{alignat*} \begin{alignat*}{8} &x_1 & & + &&x_2 & & - &&\phantom{5/} 2&&x_3 &&= -&&5\\ & & & &&x_2 & & - &&5/3 &&x_3 &&= -&&6 \\ & & & && & & && && && && \end{alignat*} \begin{alignat*}{8} &x_1 & & && & & - &&1/3 &&x_3 &&= &&1\\ & & & \phantom{+} &&x_2 & & - &&5/3 &&x_3 &&= -&&6 \\ & & & && & & && && && && \end{alignat*} 
-
Let us change the system just by changing one number on the left hand side of the first equation:
\begin{alignat*}{8}
&x_1 & & + &&x_2 & & - &&2 &&x_3 &&= &&5\\
2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\
3 &x_1 & & && & & - && && x_3 &&= &&3
\end{alignat*}
The only difference is that in the first equation $-5$ is replaced by $5$. So, the same row operations that we used above will row reduce the corresponding augmented matrix:
\begin{align*}
\left[\!\begin{array}{crr|r}
1 & 1 & -2 & 5 \\
2 & -1 & 1 & 8 \\
3 & 0 & -1 & 3
\end{array}\! \right] & \sim
\left[\!\begin{array}{crr|r}
1 & 1 & -2 & 5 \\
0 & -3 & 5 & -2 \\
0 & -3 & 5 & -12
\end{array}\! \right] \\
& \sim
\left[\!\begin{array}{crr|r}
1 & 1 & -2 & 5 \\
0 & -3 & 5 & -2 \\
0 & 0 & 0 & -10
\end{array}\! \right] \\
& \sim
\left[\!\begin{array}{crr|r}
1 & 1 & -2 & 5 \\
0 & 1 & -5/3 & 1/3 \\
0 & 0 & 0 & 1
\end{array}\! \right] \\
& \sim
\left[\!\begin{array}{crr|r}
1 & 0 & -1/3 & 14/3 \\
0 & 1 & -5/3 & 1/3 \\
0 & 0 & 0 & 1
\end{array}\! \right] \\
& \sim
\left[\!\begin{array}{crr|r}
1 & 0 & -1/3 & 0 \\
0 & 1 & -5/3 & 0 \\
0 & 0 & 0 & 1
\end{array}\! \right]
\end{align*}
The preceding row reduction proves that the following two linear systems are equivalent:
The last equation in the system on the right is written with zeros for us to see that the resulting equation does not have a solution. Hence the above system on the right is inconsistent and thus, the given system is inconsistent.\begin{alignat*}{8} &x_1 & & + &&x_2 & & - &&2 &&x_3 &&= &&5\\ 2 &x_1 & & - &&x_2 & & + && &&x_3 &&= &&8 \\ 3 &x_1 & & && & & - && && x_3 &&= &&3 \end{alignat*} \begin{alignat*}{8} &x_1 & & && & & - 1/3 && &&x_3 &&= &&0\\ & & & &&x_2 & & - 5/3 && &&x_3 &&= &&0 \\ 0 &x_1 & & + 0 & & x_2 & & + \ 0 && && x_3 &&= &&1 \end{alignat*}
Below is the corresponding illustration in $\mathbb{R}^3.$
- The information sheet and the class calendar.
- We will start with Section 1.1 Systems of Linear Equations. Suggested problems are 3, 6, 7, 5, 9, 11, 12, 16, 17, 18, 21, 22, 23, 24, 25, 27, 31, 33, 34.
-
Just reading Section 1.1 you will experience that linear algebra is a subject full of new terminology:
- linear equation,
- coefficients of a linear equation,
- system of linear equations or linear system or just system,
- a solution of a linear system,
- the solution set of a linear system,
- consistent linear system,
- inconsistent linear system,
- equivalent systems,
- coefficient matrix of a system,
- augmented matrix of a system,
- elementary row operations,
- row-equivalent matrices.
- Learning the terminology of a field is crucial to getting to understand that field. This is true in general and particularly true in Linear Algebra. Therefore I started a webpage with Glossary of Linear Algebra Terms with my comments.
-
What is the oldest linear algebra problem?
-
Clay tablet VAT 8389 from the Old Babylonian period, from 2000 to 1600 BC, contains what is believed to be the earliest word problem that can be interpreted as a system of linear equations:
A translation of this word problem into a system of linear equations is as follows: \begin{alignat*}{4} &x_1 & &\ + &x_2 & = 1800 \\ \tfrac{2}{3} &x_1 & &- \tfrac{1}{2} &x_2 & = \phantom{1}500. \end{alignat*} -
Problem 40 of the Rhind papyrus which is dated to 1550 BC is:
Denote by $x_1$ the smallest number and by $x_2$ the common difference. After simplification, the above problem translates into the following system of linear equations: \begin{alignat*}{5} 5 &x_1 & & + 10 &x_2 & = 100 \\ \tfrac{11}{7} &x_1 & & - \phantom{1}\tfrac{2}{7} &x_2 & = \phantom{10}0. \end{alignat*} -
Most importantly for us, the oldest known treatment of systems of linear equations from antiquity which resembles the methods that we will use in this class is in Chapter 8 of the Chinese textbook Nine Chapters of the Mathematical Art which is at least 1800 years old.
-
Clay tablet VAT 8389 from the Old Babylonian period, from 2000 to 1600 BC, contains what is believed to be the earliest word problem that can be interpreted as a system of linear equations:
Different civilizations have created mathematical knowledge throughout history and sometimes passed that knowledge among themselves. A significant aspect of the growth of mathematical knowledge was that succeeding civilizations recognized the value of the knowledge created by preceding civilizations. Just the fact that I am writing this here is my recognition and appreciation of the contribution of the ancient civilizations to Linear Algebra.