Linear algebra : working with matrices

I. Cnop
Vrije Universiteit Brussel
icnop@vub.ac.be

Multiplication of matrices

Use MatrixForm which adds brackets around TableForm

Scalar product (Dot product) of lists and vectors

v = {x, y};
b={2, 3};
v . b

Dot[v, b]

b . b

Square of the length

v . v

Square of a distance

Expand[(v - b) . (v - b)]

Expand[(v + b) . (v + b)]

Writing sums

a = Range[100];

b = Table[1, {Length[a]}];

Som van alle getallen tot 100

a . b

Som van alle kwadraten

a . a

(a^(2 )) . b

Som van alle derdemachten

(a^(2 )) . a

Dot product of matrices

Check

(({{3, 7, 2}, {2, 0, 4}, {3, 6, -5}})).(({{1, 1, 2, 9}, {2, 4, -3, 1}, {3, 6, -5, 0}}))//MatrixForm

(({{23, 43, -25, 34}, {14, 26, -16, 18}, {0, -3, 13, 33}}))

and be careful about the order:

(({{2, 0, 4}})).(({{9}, {1}, {3}}))//MatrixForm

(({{9}, {1}, {3}})).(({{2, 0, 4}}))//MatrixForm

(({{3, 7, 2}, {2, 0, 4}, {3, 6, -5}})).(({{18, 0, 36}, {2, 0, 4}, {6, 0, 12}}))//MatrixForm

(({{18, 0, 36}, {2, 0, 4}, {6, 0, 12}})).(({{3, 7, 2}, {2, 0, 4}, {3, 6, -5}}))//MatrixForm

% == %%

Matrices allow concise writing of systems.

(({{1, 1, 2}, {2, 4, -3}, {3, 6, -5}})).(({{x}, {y}, {z}}))//MatrixForm

A=(({{1, 1, 2}, {2, 4, -3}, {3, 6, -5}}));
X=(({{x}, {y}, {z}}));
A . X // MatrixForm

For a system we have to add a right hand side:

Solve[%==(({{2}, {3}, {4}})),{x, y, z}]

Solving  2x2  systems

An Example

(({{3, 5}, {1, 3}})).(({{x}, {y}}))==(({{2}, {0}}))

Solve[3 x + 5 y == 2, y]

Plot[y /. %,{x, -1, 3},AspectRatio -> Automatic,PlotRange -> {{-1, 3}, {-2, 2}}]

Solve[x + 3 y == 0, y]

Plot[y /. %,{x, -1, 3},AspectRatio -> Automatic,PlotRange->{{-1, 3}, {-2, 2}}]

Animate these graphics to see the intersection at

Solve[(({{3, 5}, {1, 3}})).(({{x}, {y}}))==(({{2}, {0}})),{x, y}]

But not all systems have solutions:

Solve[(({{2, 5}, {1, 5/2}})).(({{x}, {y}}))==(({{2}, {0}})),{x, y}]

Other systems behave differently:

Solve[(({{2, 5}, {1, 5/2}})).(({{x}, {y}}))==(({{2}, {1}})),{x, y}]

Solve[(({{2, 5}, {1, 5/2}})).(({{x}, {y}}))==(({{2}, {1}})),{x}]

Solve[(({{2, 5}, {1, 5/2}})).(({{x}, {y}}))==(({{2}, {1}})),{y}]

Exercise

Draw the corresponding lines for this system

Matrix of a system

An Example

In our system the following expression is   0 :

(({{3, 5}, {1, 3}})).(({{x}, {y}}))-(({{2}, {0}}))//MatrixForm

Elimination by row operations

Simplify[(({{-1, 3}})).%]//MatrixForm

Simplify[(({{-3, 5}})).%%]//MatrixForm

Exercise

Repeat this for another system

Solving  3x3  systems

An Example

Write the system in the following manner:

(({{1, 1, 2}, {2, 4, -3}, {3, 6, -5}})).(({{x}, {y}, {z}}))==(({{9}, {1}, {0}}))

A=(({{1, 1, 2}, {2, 4, -3}, {3, 6, -5}}));

R=(({{9}, {1}, {0}}));

A.(({{x}, {y}, {z}}))==R

Elimination by row operations

In our system the following expression is   0 :

(({{1, 1, 2}, {2, 4, -3}, {3, 6, -5}})).(({{x}, {y}, {z}}))-(({{9}, {1}, {0}}))//MatrixForm

Combining makes   x  disappear:

new1=First[Simplify[(({{-2, 1, 0}})).%]]

new2=First[Simplify[(({{-3, 0, 1}})).%%]]

We get a   2x2  system  in  y  and  z that can be solved

Solve[{new1 == 0, new2 == 0}, {y, z}]

and obtain  x  by substitution:

-9 + x + y + 2 z == 0 /. %

This is the Solve result:

Solve[(({{1, 1, 2}, {2, 4, -3}, {3, 6, -5}})).(({{x}, {y}, {z}}))==(({{9}, {1}, {0}})),{x, y, z}]

Complete  elimination by row operations on a square matrix with added column

Given B  the extended matrix:

B=(({{1, 1, 2, 9}, {2, 4, -3, 1}, {3, 6, -5, 0}}));

Corresponds to a system with all right hand sides equal zero:

B.(({{x}, {y}, {z}, {1}}))//MatrixForm

Make a line with no more  x :

(({{-2, 1, 0}})).(({{1, 1, 2, 9}, {2, 4, -3, 1}, {3, 6, -5, 0}}))//MatrixForm

Keep the first line and make two lines without  x  :

(({{1, 0, 0}, {-2, 1, 0}, {-3, 0, 1}})).(({{1, 1, 2, 9}, {2, 4, -3, 1}, {3, 6, -5, 0}}))//MatrixForm

Eliminating  y   and  z  is done by

(({{1, 0, 0}, {0, 11, -7}, {0, 3, -2}})).%//MatrixForm

Substituing  y  and  z  in the first line, gives  x

(({{1, -1, -2}, {0, 1, 0}, {0, 0, 1}})).%//MatrixForm

Total executed

(({{1, -1, -2}, {0, 1, 0}, {0, 0, 1}})).(({{1, 0, 0}, {0, 11, -7}, {0, 3, -2}})).(({{1, 0, 0}, {-2, 1, 0}, {-3, 0, 1}})).B//MatrixForm

operations=(({{1, -1, -2}, {0, 1, 0}, {0, 0, 1}})).(({{1, 0, 0}, {0, 11, -7}, {0, 3, -2}})).(({{1, 0, 0}, {-2, 1, 0}, {-3, 0, 1}}))

operations // MatrixForm

operations . A // MatrixForm

We have a solution!

operations . R // MatrixForm

{x, y, z} = Flatten[operations . R]

This method  is not uniquely determined

We start over more systematically, by placing a nonzero coefficient on the left by switching rows.

B=(({{1, 1, 2, 9}, {2, 4, -3, 1}, {3, 6, -5, 0}}));

(({{1, 0, 0}, {2, -1, 0}, {3, 0, -1}})).B//MatrixForm

(({{1, 0, 0}, {0, -1/2, 0}, {0, 0, 1}})).(({{1, 0, 0}, {2, -1, 0}, {3, 0, -1}})).B//MatrixForm

(({{1, 0, 0}, {0, 1, 0}, {0, 3, 1}})).(({{1, 0, 0}, {0, -1/2, 0}, {0, 0, 1}})).(({{1, 0, 0}, {2, -1, 0}, {3, 0, -1}})).B//MatrixForm

(({{1, 0, 0}, {0, 1, 0}, {0, 0, 2}})).(({{1, 0, 0}, {0, 1, 0}, {0, 3, 1}})).(({{1, 0, 0}, {0, -1/2, 0}, {0, 0, 1}})).(({{1, 0, 0}, {2, -1, 0}, {3, 0, -1}})).B//MatrixForm

We obtain a system that is "superdiagonal triangular" with entries all  1  on the diagonal.
We have  z , substitute to obtain  y  and finally obtain  x . 
All row  operations multiply to a "subdiagonal triangular" matrix:

operations=(({{1, 0, 0}, {0, 1, 0}, {0, 0, 2}})).(({{1, 0, 0}, {0, 1, 0}, {0, 3, 1}})).(({{1, 0, 0}, {0, -1/2, 0}, {0, 0, 1}})).(({{1, 0, 0}, {2, -1, 0}, {3, 0, -1}}))

operations // MatrixForm

Back substituting can be written using row operations. Rowoperations also modify the right hand side column.

operations . B // MatrixForm

(({{1, -1, 0}, {0, 1, 7/2}, {0, 0, 1}})).%//MatrixForm

(({{1, 0, -11/2}, {0, 1, 0}, {0, 0, 1}})).%//MatrixForm

Eventually we have multiplied by

(({{1, -1, 0}, {0, 1, 7/2}, {0, 0, 1}})).(({{1, 0, -11/2}, {0, 1, 0}, {0, 0, 1}})).operations//MatrixForm

This is the inverse matrix of  A :

% . A // MatrixForm

Inverse[A] // MatrixForm

Another example.

Repeat using

A=(({{2, -1, 1}, {1, 2, -1}, {5, 0, 1}}));

R=(({{3}, {4}, {5}}));

B=(({{2, -1, 1, 3}, {1, 2, -1, 4}, {5, 0, 1, 5}}));

(({{1/2, 0, 0}, {0, 1, 0}, {0, 0, 1}})).B//MatrixForm

(({{1, 0, 0}, {1, -1, 0}, {1, 0, -1/5}})).%//MatrixForm

(({{1, 0, 0}, {0, -2/5, 0}, {0, 0, 1}})).%//MatrixForm

(({{1, 0, 0}, {0, 1, 0}, {0, 1, 2}})).%//MatrixForm

We lost all information about z . 
Since the right hand side is not zero, we conclude there is no solution to this system.

Solve[A.(({{x}, {y}, {z}}))==R,{x, y, z}]

Another example.

Repeat this method starting with

x =. ; y =. ; z =. ;

A=(({{2, -1, 1}, {1, 2, -1}, {5, 0, 1}}));

R=(({{3}, {4}, {10}}));

B=(({{2, -1, 1, 3}, {1, 2, -1, 4}, {5, 0, 1, 10}}));

(({{1/2, 0, 0}, {0, 1, 0}, {0, 0, 1}})).B//MatrixForm

(({{1, 0, 0}, {1, -1, 0}, {1, 0, -1/5}})).%//MatrixForm

(({{1, 0, 0}, {0, -2/5, 0}, {0, 0, 1}})).%//MatrixForm

(({{1, 0, 0}, {0, 1, 0}, {0, 1, 2}})).%//MatrixForm

We lost all information on z, but since the right hand side is zero too we conclude that we have solutions with all values of   z . 
Solve gives a warning for this :

Solve[A.(({{x}, {y}, {z}}))==R,{x, y, z}]

Another example.

Repeat this method starting with

A=(({{1, td, d}, {2, 3, 4}, {1, 4, 3}}))

R=(({{h}, {t}, {e}}))

where    td d h t e    are the 5 digits  in your student card (or five last digits (or the last digits in a phone number) . Write

B

and proceed.

Conclusions

By row operations on a matrix we can reduce it to a form in which each line stepwise  has a leading 1,
lines with zeroes are grouped below, 
a column with leading 1 has only zeroes below
Such form is called reduced echelon form [Rowreduced Echelon Form].
The product of all executed operations is the inverse of a square matrix if it exists.

If we do not require columns to have zeroes below their  leading 1 , but merely a superdiagonal matrix , this can be achieved by multiplication with a subdiagonal matrix for all row operations.

This technique is called Gauss elimination. It is a fast method for solving linear systems. Look it up in the Help:

?RowReduce

Exercises

1. Execute the above method for the following matrices   B

(({{1, b}, {a, 1}}))

(({{-10, 6, 12, 28}, {-5, 6, -5, -1}}))

(({{0, 7, 12}, {6, 12, 28}, {6, -5, -1}}))

(({{0, 0, -2, 0, 7, 12}, {2, 4, -10, 6, 12, 28}, {2, 4, -5, 6, -5, -1}}))

and control your result by RowReduce.
Compute the inverse of the matrix if it exists.

2. Count the number of additions and multiplications to execute this RowReduce method.