Linear Algebra - IIT Bombay

Chapter 3

Linear Algebra

Dixit algorizmi. Or, "So said al-Khwarizmi", being the opening words of a 12th century Latin translation of a work on arithmetic by al-Khwarizmi (ca. 780?840).

3.1 Linear Equations

Elementary algebra, using the rules of completion and balancing developed by al-Khwarizmi, allows us to determine the value of an unknown variable x that satisfies an equation like the one below:

10x - 5 = 15 + 5x An equation like this that only involves an unknown (like x) and not its higher powers (x2, x3), along with additions (or subtractions) of the unknown multiplied by numbers (like 10x and 5x) is called a linear equation. We now know, of course, that the equation above can be converted to a special form ("number multiplied by unknown equals number", or ax = b, where a and b are numbers):

5x = 20 Once in this form, it becomes easy to see that x = b/a = 4. Linear algebra is, in essence, concerned with the solution of several linear equations in several unknowns. Here is a simple example of two equations and two unknowns x and y, written in a uniform way, with all unknowns (variables) to the left of the equality, and all numbers (constants) to the right:

145

146

CHAPTER 3. LINEAR ALGEBRA

Figure 3.1: Solving linear equations: the geometric view.

2x - y = 0

-x + 2y = 3

We woud like to find values of x and y for which these equations are true. School geometry tells us how to visualise this: each equation is a straight line in the xy plane, and since we want a value of x and y for which both equations are true, we are really asking for the values of x and y that lie on both lines (that is, the point of intersection of the two lines: see Fig. 3.1). Of course, if the lines do not meet at a point, then there are no values of x and y that satisfy the equations. And we can continue to solve problems like these geometrically: more unknowns means lines become higher-dimensional flat surfaces ("hyperplanes"), and more equations means we are looking for the single point of intersection of all these surfaces. Visually though, this is challenging for all but a small minority of us, geared as we are to live in a world of three spatial dimensions.

3.2. VECTORS AND MATRICES

147

Linear algebra, an extension of elementary algebra, gives us a way of looking at the solution of any number of linear equations, with any number of variables without suffering from this visual overload. In effect, equations are once again converted to the simple form we just saw, that is, Ax = b, although A and b are no longer just numbers. In fact, we will see that A is a matrix , and that x and b are vectors (and in order not to confuse them with variables and numbers, we will from now on use the bold-face notation x and b). Linear algebra, shows us that solutions, if they exist, can be obtained in three different ways:

1. A direct solution, using techniques called elimination and back substitution.

2. A solution by "inverting" the matrix A, to give the solution x = A-1b.

3. A vector space solution, by looking at notions called the column space and nullspace of A.

Understanding each of these requires a minimal understanding of vectors and matrices, which we give in a somewhat compressed form here.

3.2 Vectors and Matrices

It is easiest to think of a vector as a generalisation of a single number. A pair of numbers can be represented by a two-dimensional vector . Here is the two-dimensional vector representation of the pair (2, -1):

u= 2 -1

This kind of vector is usually called a "column" vector. Geometrically, such a vector is often visualised by an arrow in the two-dimensional plane as shown on the left in Fig. ??. Multiplying such a vector by any particular number, say 2, multiplies each component by that number. That is, 2u represents the pair (4, -2). Geometrically, we can see that multiplication by a number--sometimes called scalar multiplication--simply makes gives a vector with a "longer" arrow as shown on the right in the figure (assuming, of course, that we are not dealing with zero-length vectors). In general, multiplication of a (non-zero) vector u by different (non-zero) numbers a result in lines either in the direction of u (if a > 0) or in the opposite direction Suppose we now consider a second vector v corresponding to the pair (-1, 2), and ask: what is u + v. This simply adds the individual components. In our example:

u = 2 v = -1

-1

2

u+v= 2-1 = 1

-1 + 2

1

148

CHAPTER 3. LINEAR ALGEBRA

Geometrically, the addition of two vectors gives a third, which can visualised as the diagonal of the parallelogram formed by u and v (Fig. ??, left). It should be straightforward to visualise that any point on the plane containing the vectors u and v can be obtained by some linear combination au + bv, and that the space of all linear combinations is simply the full two-dimensional plane containing u and v (Fig. ??, right). For the two-dimensional example here, this plane is just the usual xy plane (we will see that this is the vector space 2). Although we have so far only looked at vectors with two components, linear algebra is more general. It allows us to use the same operations with vectors of any size. Suppose our vectors u and v are three-dimensional. Linear combinations now still fill a plane containing the two vectors. But, this is no longer the xy plane, since the vectors generated by the linear combinations are points in three-dimensional space (we will see later, that is some "subspace" of the vector space 3). Addition of a third vector w will also not necessarily result in a point on this plane, and the space of linear combinations au + bv + cw could fill the entire three-dimensional space.

Let us return now to the two equations that we saw in the previous section:

2x - y = 0

-x + 2y = 3

It should be easy to see how these can be written in "vector" form:

x 2 + y -1 = 0

-1

2

3

(3.1)

That is, we are asking if there is some linear combination of the column vectors [2, -1] and [-1, 2] that gives the column vector [0, 3]. And this is the point of departure with the usual geometric approach: we visualise solutions of equations not as points of intersections of surfaces, but as linear combination of vectors (of whatever dimension): see Fig. 3.2.

To get it into a form that is even more manageable, we need the concept of a "coefficient matrix". A matrix is simply a rectangular array of numbers, and the coefficient matrix for the left hand side of the linear combination above is:

A = 2 -1 -1 2

This is a 2 ? 2 ("two by two") matrix, meaning it has 2 rows and 2 columns. You can see that the columns of the matrix are simply the column vectors of the linear combination. Let:

3.2. VECTORS AND MATRICES

149

Figure 3.2: Solving linear equations: the geometric view from linear algebra.

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download