Mathematics 270 Linear Algebra I

Study Guide :: Unit 4

Euclidean Vector Spaces: ${ℝ}^2$, ${ℝ}^3$ and ${ℝ}^n$

In geometry, Euclidean space encompasses the two-dimensional Euclidean plane, the three-dimensional space of Euclidean geometry, and certain other spaces. It is named after the ancient Greek mathematician Euclid of Alexandria. The term Euclidean distinguishes these spaces from other types of spaces considered in modern geometry. Euclidean spaces also generalize to higher dimensions. Classical Greek geometry defined the Euclidean plane and Euclidean three-dimensional space using certain postulates, while the other properties of these spaces were deduced as theorems. Geometric constructions are also used to define rational numbers. When algebra and mathematical analysis became developed enough, this relation became reversed, and now it is more common to define Euclidean space using Cartesian coordinates and the ideas of analytic geometry. This means that points of the space are specified with collections of real numbers, and geometric shapes are defined as equations and inequalities. This approach brings the tools of algebra and calculus to bear on questions of geometry and has the advantage that it generalizes easily to Euclidean spaces of more than three dimensions.

In many physical applications one wishes to restrict attention to vectors having only real components; in such cases it is common to refer to the space as $R^n$.

Objectives

After completing Unit 4, you should be able to:

1. perform geometric and algebraic operations on vectors;
2. determine whether two vectors are equivalent or collinear;
3. sketch vectors whose initial and terminal points are given;
4. find components of a vector whose initial and terminal points are given;
5. compute the norm of a vector and a unit vector;
6. normalize a nonzero vector;
7. determine the distance between two vectors;
8. compute the dot product of two vectors and the angle between two nonzero vectors;
9. determine whether two vectors are orthogonal;
10. determine whether a given set of vectors forms an orthogonal set;
11. find equations and vector form for lines and planes;
12. compute the components of a given vector;
13. find the distance between a point and a line, between two parallel planes, and between a point and a plane;
14. express the equations of lines and planes using either vector or parametric equations;
15. verify the orthogonality of the row vectors of a linear system of equations and a solution vector;
16. compute the cross product of two vectors;
17. know the geometric relationship between the given vectors;
18. know the properties of the cross product;
19. compute the scalar triple product of three vectors; know the geometric interpretation of the scalar triple product;
20. compute the areas of triangles and parallelograms determined by two vectors or three points in ${ℝ}^2$ or ${ℝ}^3$-space;
21. use the scalar triple product to determine whether three given vectors in ${ℝ}^3$-space are collinear;
22. find the eigenvalues and eigenvectors of a matrix; and
23. determine the transition matrix of the Markov chain.

4.1  Vector Operations and Properties

A vector is a mathematical object with magnitude and direction used to represent items such as the velocity, acceleration, or momentum of an object. A vector v can be represented by an n-tuple of real numbers,

$\overrightarrow{v}=\left(v_1,v_2,\dots ,v_n\right)$

considered to be elements (points) of ${ℝ}^n$, an $n$-dimensional real space. The vectors in two- and three-dimensional spaces can be visualized and described using the coordinate systems. For instance, the two-dimensional vector $\overrightarrow{v}=\left(v_1,v_2\right)=\left(3,4\right)$ has magnitude 5, the distance from the origin to the point $\left(3,4\right)$, and direction, the orientation of the arrow from the origin to $\left(3,4\right)$. Notice that the $v_1$ component is measured along the $\mathit{\text{x}}$-axis, while the $v_2$ component is measured along the $\mathit{\text{y}}$-axis.

Similarly the three-dimensional vector $\overrightarrow{v}=\left(v_1,v_2,v_3\right)=\left(2,-1,3\right)$ has magnitude $\sqrt{14}$, the distance from the origin to the point $\left(2,-1,3\right)$, and direction, the orientation of the arrow from the origin to $\left(2,-1,3\right)$. Notice that the $v_1$ component is measured along the $\mathit{\text{x}}$-axis, the $v_2$ component is measured along the $\mathit{\text{y}}$-axis, and the $v_3$ component is measured along the $\mathit{\text{z}}$-axis. Later we learn how to evaluate the magnitude of the given vectors.

In this section we review the basic properties of vectors in two and three dimensions with the goal of extending these properties to vectors in ${ℝ}^n$.

Reading Assignment

Read, and study, pages 131-140 (to “Exercise Set 3.1”) of the textbook.

Study the parallelogram and triangle rules for vector addition on page 132 of the textbook. Note that vector subtraction can be replaced by vector addition using the negative vector. Then study scalar multiplication on page 133. Observe that the negative vector can be defined using scalar multiplication.

You will be familiar with coordinate systems from high school. Observe that vectors can be defined in coordinate systems using their components. The addition, subtraction and scalar multiplication of vectors can be defined using components, as illustrated in Definition 3 on page 138 of the textbook.

Study the most important properties of vectors given by Theorems 3.1.1 and 3.1.2 on page 138.

We strongly recommend that you repeat the reading assignment as many times as necessary until you understand the material. Then, you can start working on the following exercises.

Exercises

Work on the following textbook exercises from “Exercise Set 3.1” (pp. 140-142):

• 3, 7, 9, 11, 13, 19, 21, 27.

4.2  Lengths, Distances and Dot / Inner Product

As we develop a geometry for the vector space ${ℝ}^n$, you should pay close attention to the approach we use. While the results are, of course, important, the way we arrive at the results is also very important. Magnitude, angle and distance are defined in ${ℝ}^n$ by generalizing expressions for magnitude, angle and distance from ${ℝ}^2$ and ${ℝ}^3$. This process of gradually extending familiar concepts to more general surroundings is fundamental to mathematics.

In linear algebra, along with the term magnitude, we use an equivalent word, norm. The norm or magnitude of a vector $\overrightarrow{v}$ is defined as $\parallel \overrightarrow{v}\parallel$. It is often convenient to present the direction of a vector by a so-called unit vector. The length of such vectors equals $1$. If $\overrightarrow{v}$ is any nonzero vector in ${ℝ}^n$, then

$\overrightarrow{u}={\displaystyle \frac{1}{\parallel \overrightarrow{v}\parallel }}\overrightarrow{v}$

defines a unit vector that is in the same direction as $\overrightarrow{v}$.

Reading Assignment

Read, and study, pages 142-153 (to “Exercise Set 3.2”) of the textbook.

Read the definition of the norm of vectors on page 142. Review Example 1 on page 143 and study how to evaluate the norms of vectors given by their components. Observe that a vector, whose length equals $1$ (unit vector) can be determined using its norm. Moreover, any vector can be determined by the linear combination of unit vectors.

Study Definition 2 and learn how to evaluate the distance between two points in $R^n$.

Study Definition 3 and review the related example. Then study Definition 4 and consider the dot product (also called Euclidean inner product). Review Example 7 on page 147 and learn how to determine the angle between two vectors using their dot product.

Review Theorems 3.2.2, 3.2.2, and 3.2.3 and observe that the dot product has many of the same algebraic properties as the product of real numbers. Moreover, the dot product can be determined as matrix multiplication. Review Table 1 on page 151.

After a careful reading of the assigned material start working on the following selected exercises.

Exercises

Work on the following textbook exercises from “Exercises 3.2” (pp. 153-155):

• 1, 5, 7, 11, 15, 17, 22, 25.

4.3  Orthogonality

Notice that the angles between the coordinate axes are equal to $90°$ (i.e., $\pi /2$ in radians). One uses a special term if the angle between the geometrical objects (lines, planes) is $90°$; we call them orthogonal or perpendicular. In the previous section we determined the angle between the vectors $\overrightarrow{u}$ and $\overrightarrow{v}$ as

$\theta ={\cos }^{-1}\left({\displaystyle \frac{\overrightarrow{u}\cdot \overrightarrow{v}}{\Vert \overrightarrow{u}\Vert \Vert \overrightarrow{v}\Vert }}\right)$.

For the cosine of the angle we obtain,

$\cos \theta ={\displaystyle \frac{\overrightarrow{u}\cdot \overrightarrow{v}}{\Vert \overrightarrow{u}\Vert \Vert \overrightarrow{v}\Vert }}$,

which gives us

$\cos \left(\pi /2\right)=0={\displaystyle \frac{\overrightarrow{u}\cdot \overrightarrow{v}}{\Vert \overrightarrow{u}\Vert \Vert \overrightarrow{v}\Vert }}$.

Therefore, the dot product of two orthogonal (or perpendicular) vectors equals zero. In this section we analyze some examples where perpendicular vectors play an important role—specifically for finding the general equations of the line in ${ℝ}^2$ and plane in ${ℝ}^3$.

Reading Assignment

Read, and study, pages 155-162 of the textbook (to “Exercise Set 3.3”).

Review Example 1 on page 155 and learn how to check the orthogonality of vectors. Continue with Example 2 and observe that the orthogonality can be used to determine general equations for the line in ${ℝ}^2$ and the plane in ${ℝ}^3$. These equations are called the Point-Normal Equations, since one only needs a point of the line (or the plane) and a normal/perpendicular vector to the line (or the plane) to find the equation of the line in ${ℝ}^2$ (or the plane in ${ℝ}^3$).

Study the concept of orthogonal projection on pages 158-159. Review Examples 4 and 5. Read Theorem 3.3.3 and observe that the Pythagoras Theorem can be generalized for $n$-dimensional spaces.

Read pages 161 and 162 to learn how to determine the distances between a line and a point, between a plane and a point, and between parallel planes.

We strongly recommend that you repeat the reading assignment as many times as necessary until you understand the material. Then, start working on the following exercises.

Exercises

Work on the following textbook exercises from “Exercise Set 3.3” (pp. 162-163):

• 1, 3, 7, 9, 11, 13, 19, 21, 25, 27, 31.

4.4  Vector and Parametric Equations of Lines and Planes

In the previous section, we used the concept of orthogonality to determine the point-normal equation of the line in ${ℝ}^2$ and the plane in ${ℝ}^3$ given one point of the line or the plane and a normal vector to it. In this section, we introduce the vector and parametric equations of the line in $R^2$ and ${ℝ}^3$ and of the plane in $R^3$ to keep interpreting the solutions of linear systems geometrically.

To illustrate the ideas of this section, consider the following system of equations,

$x+3z=4$
$y+2z=1$, (4.1)

and assign the arbitrary value $t$ to $z$. The general solution to the system can be written as

$x=-3t+4$
$y=-2t+1$
$z=t$. (4.2)

Now, observe, for example, that for

$t=0$,   $\left(x,y,z\right)=\left(4,1,0\right)$,
$t=1$,   $\left(x,y,z\right)=\left(1,-1,1\right)$,
$t=2$,   $\left(x,y,z\right)=\left(-2,-3,2\right)$.

One can see that the solution set of the given system of equations is determined by the variable $t$. Thinking ”geometrically,” we can consider the solutions $\left(4,1,0\right)$, $\left(1,-1,1\right)$ and $\left(-2,-3,2\right)$ as points in a three-dimensional space. Moreover, these points describe a line in ${ℝ}^3$, which is determined by the parametric equations (4.2), where $t$ is called the parameter.

Note that equations (4.2), which provide the solution of the system (4.1), can be rewritten in a vector form as

$\left(x,y,z\right)=\left(4,1,0\right)+t\left(-3,-2.1\right)$.

Thus, by interpreting $\left(x,y,z\right)$, $\left(4,1,0\right)$ and $\left(-3,-2,1\right)$ as vectors in a three-dimensional space and calling $\overrightarrow{x}=\left(x,y,z\right)$, ${\overrightarrow{x}}_0=\left(4,1,0\right)$ and $\overrightarrow{v}=\left(-3,-2,1\right)$, the solution set can be written in vector notation as

$\overrightarrow{x}={\overrightarrow{x}}_0+t\overrightarrow{v}$,

which is nothing more that a straight line that passes through the point ${\overrightarrow{x}}_0$ with direction (or parallel to) $\overrightarrow{v}$.

In this section we use both parametric and vector methods for defining lines and planes. We extend this idea to the solution sets of linear systems with $n$ unknowns as geometric objects in $R^n$.

Reading Assignment

Read, and study, pages 164-170 (to “Exercise Set 3.4”) of the textbook.

On pages 164-165 pay attention to the theorems and definitions. You will see that lines and planes can be determined by vector equations involving $parameters$. Note that the equation of the line requires one parameter, whereas the equation of the plane includes two of them.

When introducing this section, we explained how the same geometrical object can be defined by two types of equations: parametric and vector. Review Examples 1, 2 and 3 to observe that the lines and planes in ${ℝ}^2$, ${ℝ}^3$, and ${ℝ}^4$ are determined by parametric and vector equations.

In previous sections you learned that vectors can be determined by initial and terminal points. This concept can be used for the two-point vector equations of a line in ${ℝ}^n$. Review equations (9) and (10) and Example 4 on page 167 of the textbook. The example illustrates the method for defining the equation for the line segment given by Definition 3 on page 168. This definition is applied in Example 5.

Read pages 169 and 170 and observe how the concept of orthogonality is used to define the solution of a homogeneous linear system of equations. Also, study the relationship between a nonhomogeneous linear system and its corresponding (or associated) homogeneous system.

Remember, repeat the reading assignment as many times as necessary until you understand the material. Then, start working on the following exercises.

Exercises

Work on the following textbook exercises from “Exercise Set 3.4” (p. 170-172):

• 1, 7, 11, 13, 17, 19, 25.

4.5  Cross Product

There are two ways to take the product of a pair of vectors. One of these methods of multiplication is the dot or inner product, which was defined in Section 4.2. The other multiplication is the cross product. Before explaining how to compute the cross product, we should point out a major difference between dot and cross products, namely, the result of a dot product is a scalar (number), whereas the result of a cross product is a vector!

We should also note that the cross product requires both of the vectors to be three-dimensional vectors.

If $\overrightarrow{u}=\left(u_1,u_2,u_3\right)$ and $\overrightarrow{v}=\left(v_1,v_2,v_3\right)$ are vectors in a three-dimensional space, then the cross product $\overrightarrow{u}\times \overrightarrow{v}$ is the vector defined by

$\overrightarrow{u}\times \overrightarrow{v}=\left(u_2{v}_3-u_3{v}_2,u_3{v}_1-u_1{v}_3,u_1{v}_2-u_2{v}_1\right)$.

The resultant vector is perpendicular to both $\overrightarrow{u}$ and $\overrightarrow{v}$. In this section we give a geometric interpretation of $3\times 3$ determinants.

Reading Assignment

Read, and study, pages 172-179 (to “Exercise Set 3.5”) of the textbook.

Review Definition 1 and Example 1 on pages 172 and 173 and learn how to calculate the cross product of vectors. You might find it very useful to use the determinant notation for calculating the cross product.

Read Theorem 3.5.1 and its proof in order to understand certain properties involving both cross and dot products.

Notice that Example 2 illustrates how the resultant vector from the cross product $\overrightarrow{u}\times \overrightarrow{v}$ is perpendicular to both $\overrightarrow{u}$ and $\overrightarrow{v}$.

Study the properties of cross product stated in Theorem 3.5.2 on page 174.

Recall that $\widehat{i}=\left(1,0,0\right)$, $\widehat{j}=\left(0,1,0\right)$, $\widehat{k}=\left(0,0,1\right)$ are the standard unit vectors in ${ℝ}^3$, and review Example 3 on page 175. This example introduces you to the determinant form of the cross product, which allows one to determine the cross product using the determinant and standard unit vectors, as explained in detail on pages 175 and 176.

Pages 176-178 and Equation (6) will help you to familiarize yourself with the geometric interpretation of cross product. After studying Theorems 3.5.3 and 3.5.4, note that the areas of parallelograms and triangles, as well as the volume of parallelepipeds, can be determined using the cross product.

The concept of a scalar triple product is introduced in Definition 2 and Example 5 on page 177.

After a full understanding of the reading assignment, start working on the following exercises.

Exercises

Work on the following textbook exercises from “Exercises 3.5” (p. 179-181):

• 1, 3, 5, 7, 9, 11, 15, 17, 21, 25.

4.6  Eigenvalues and Eigenvectors

In this unit we introduce and discuss the concepts of eigenvalues and eigenvectors in ${ℝ}^n$. Eigenvalues and eigenvectors are special scalars and vectors, respectively, associated with matrices. They play an extremely important role in applied mathematics; they are used in many branches of engineering and the natural and social sciences. If you are exposed to differential equations you will also see that their role is greatly significant when studying the dynamics of systems of differential equations.

A scalar $\lambda$ is called an eigenvalue of an $n\times n$ matrix $A$ if there exists a nonzero vector $\overrightarrow{x}$ in $R^n$ such that

$A\overrightarrow{x}=\lambda \overrightarrow{x}$

The vector $\overrightarrow{x}$ is called an eigenvector corresponding to the eigenvalue $\lambda$.

Notice that the resulting vector from applying $A$ to an eigenvector is a multiple of the eigenvector, itself, whose magnitude is determined by the eigenvalue.

Let $A$ be an $n\times n$ matrix with eigenvalue $\lambda$ and corresponding eigenvector $\overrightarrow{x}$. Thus, $A\overrightarrow{x}=\lambda \overrightarrow{x}$, which may be written as

$\lambda \overrightarrow{x}-A\overrightarrow{x}=\overrightarrow{0}$,

or equivalently,

$\left(\lambda I_n-A\right)\overrightarrow{x}=\overrightarrow{0}$, (4.3)

where $I_n$ denotes the $n\times n$ identity matrix.

Equation (4.3) represents a system of homogeneous linear equations with a coefficient matrix $\left(\lambda I_n-A\right)$. Of course, $\overrightarrow{x}=\overrightarrow{0}$ is a trivial solution to this system. However, eigenvectors have been defined as nonzero vectors. Thus, looking for eigenvectors is equivalent to looking for non-trivial solutions to the system (4.3), which is possible only if the coefficient matrix is singular, in other words, if $\det \left(\lambda I_n-A\right)=0$. Hence, the eigenvalues of $A$ are given by the values of $\lambda$ such that

$\det \left(\lambda I_n-A\right)=0$. (4.4)

Equation (4.4) is called the characteristic equation of $A$. Notice that by expanding its left-hand side, we obtain a polynomial

$p\left(\lambda \right)=\det \left(\lambda I_n-A\right)$ (4.5)

in $\lambda$. This polynomial is called the characteristic polynomial of $A$.

Thus, the roots of the characteristic polynomial (4.5), or equivalently, the solutions of the characteristic Equation (4.4) provide the eigenvalues of $A$. These eigenvalues can then be substituted back into the system of linear equations (4.3), whose solution provides the corresponding eigenvectors $\overrightarrow{x}$.

The main focus of this section is the computation of eigenvalues and eigenvectors of matrices of different dimensions and the discussion of some of their properties.

Notice that when we talk about eigenvalues and eigenvectors we are always referring to square matrices, i.e., matrices of dimension $n\times n$.

Reading Assignment

Read, and study, pages 291-298 of the textbook (exclusive of “More on the Equivalence Theorem” ).

Review Examples 1, 2 and 3 on pages 292-293. These examples illustrate how to calculate the eigenvalues and eigenvectors of $2\times 2$ and $3\times 3$ matrices.

Observe in Example 4 on page 294 that if $A$ is an $n\times n$ triangular matrix, then the eigenvalues of $A$ are the entries on the main diagonal of $A$.

Read Theorem 5.1.3 and Examples 6 and 7 on pages 295-298. Note that the solution space for the linear system (4.3), determined by the eigenvectors, is called eigenspace. The concept of the base for the eigenspace is introduced in Example 6. However, the general concept of bases for general spaces is beyond the scope of this course.

After a proper understanding of this reading assignment, start to work on the following exercises.

Exercises

Work on the following textbook exercises from “Exercise Set 5.1” (p. 300-302):

• 1, 3, 5, 9, 13.

4.7  Applications of Linear Algebra to Dynamical Systems: Markov Chains

Before going to the formal definition of Markov Chains in the textbook, let us introduce the topic with an example from a real life situation. Consider a city with two kinds of populations: the inner city population and the suburb population. We assume that every year 40% of the inner city population moves to the suburbs, while 30% of the suburb population moves to the inner part of the city. Let $I_0$ and $S_0$ denote the initial population of the inner city and the suburban area, respectively. Thus, after one year, the population of the inner city is given by

$I_1=0.6I_0+0.3S_0$

while the population of the suburbs is given by

$S_1=0.4I_0+0.7S_0$.

After two years, the population of the inner city is given by

$I_2=0.6I_1+0.3S_1=0.6\left(0.6I_0+0.3S_0\right)+0.3\left(0.4I_0+0.7S_0\right)$

and the suburban population is given by

$S_2=0.4I_1+0.7S_1=0.4\left(0.6I_0+0.3S_0\right)+0.7\left(0.4I_0+0.7S_0\right)$

Representing these expressions in matrix notation, the populations after one year are given by

$\left(\begin{array}{c}{I}_1\\ S_1\end{array}\right)=\left(\begin{array}{cc}0.6& 0.3\\ 0.4& 0.7\end{array}\right)\left(\begin{array}{c}{I}_0\\ S_0\end{array}\right)$,

after two years by

$\left(\begin{array}{c}{I}_2\\ S_2\end{array}\right)=\left(\begin{array}{cc}0.6& 0.3\\ 0.4& 0.7\end{array}\right)\left(\begin{array}{c}{I}_1\\ S_1\end{array}\right)$

$=\left(\begin{array}{cc}0.6& 0.3\\ 0.4& 0.7\end{array}\right)\left(\begin{array}{cc}0.6& 0.3\\ 0.4& 0.7\end{array}\right)\left(\begin{array}{c}{I}_0\\ S_0\end{array}\right)$

$={\left(\begin{array}{cc}0.6& 0.3\\ 0.4& 0.7\end{array}\right)}^2\left(\begin{array}{c}{I}_0\\ S_0\end{array}\right)$,

and after $n$ years by

$\left(\begin{array}{c}{I}_n\\ S_n\end{array}\right)={\left(\begin{array}{cc}0.6& 0.3\\ 0.4& 0.7\end{array}\right)}^n\left(\begin{array}{c}{I}_0\\ S_0\end{array}\right)$.

Note that the matrix

$\left(\begin{array}{cc}0.6& 0.3\\ 0.4& 0.7\end{array}\right)$

has particular characteristics, namely, the entries of each column vector are positive and their sum equals $1$. Such vectors are called probability vectors, and a matrix for which all the column vectors are probability vectors is called transition or stochastic matrix. Andrei Markov (1856–1922), a Russian mathematician, was the first one to study these matrices. At the beginning of twentieth century he developed the fundamentals of the Markov Chain Theory. In this section we learn about some applications of this theory.

Reading Assignment

Read, and study, pages 332-340 of the textbook (to “Exercise Set 5.5”).

Read the definition of Dynamical Systems on page 332. This concept describes the state of a particular variable with respect to time. On pages 332 and 333 you will learn about dynamical systems through an example in market shares.

Familiarize yourself, through pages 334-340 and the examples therein, with the definition of Markov Chains.

After you have read and understood the reading assignment, start working on the following exercises.

Exercises

Work on the following textbook exercises from “Exercise Set 5.5” (p. 340-342):

• 1, 3, 5, 9, 11, 15 and 17.