Linear Algebra

(directly go to documentation on : Dot, ., InProduct, CrossProduct, Outer, o, ZeroVector, BaseVector, Identity, ZeroMatrix, Diagonal, DiagonalMatrix, OrthogonalBasis, OrthonormalBasis, Normalize, Transpose, Determinant, Trace, Inverse, Minor, CoFactor, MatrixPower, SolveMatrix, CharacteristicEquation, EigenValues, EigenVectors, Sparsity, Cholesky. )

# 16. Linear Algebra

This chapter describes the commands for doing linear algebra. They can be used to manipulate vectors, represented as lists, and matrices, represented as lists of lists.

Dot, . | get dot product of tensors
InProduct | inner product of vectors (deprecated)
CrossProduct | outer product of vectors
Outer, o | get outer tensor product
ZeroVector | create a vector with all zeroes
BaseVector | base vector
Identity | make identity matrix
ZeroMatrix | make a zero matrix
Diagonal | extract the diagonal from a matrix
DiagonalMatrix | construct a diagonal matrix
OrthogonalBasis | create an orthogonal basis
OrthonormalBasis | create an orthonormal basis
Normalize | normalize a vector
Transpose | get transpose of a matrix
Determinant | determinant of a matrix
Trace | trace of a matrix
Inverse | get inverse of a matrix
Minor | get principal minor of a matrix
CoFactor | cofactor of a matrix
MatrixPower | get nth power of a square matrix
SolveMatrix | solve a linear system
CharacteristicEquation | get characteristic polynomial of a matrix
EigenValues | get eigenvalues of a matrix
EigenVectors | get eigenvectors of a matrix
Sparsity | get the sparsity of a matrix
Cholesky | find the Cholesky Decomposition

###

Dot, . -- get dot product of tensors

##### Standard library

##### Calling format:

    Dot(t1,t2)
    t1 . t2

Precedence: 30

##### Parameters:

`t1,t2` -- tensor lists (currently only vectors and matrices are supported)

##### Description:

`Dot` returns the dot (aka inner) product of two tensors t1 and t2. The last index of t1 and the first index of t2 are contracted. Currently `Dot` works only for vectors and matrices. `Dot`-multiplication of two vectors, a matrix with a vector (and vice versa) or two matrices yields either a scalar, a vector or a matrix.

##### Examples:

    In> Dot({1,2},{3,4})
    Out> 11;
    In> Dot({{1,2},{3,4}},{5,6})
    Out> {17,39};
    In> Dot({5,6},{{1,2},{3,4}})
    Out> {23,34};
    In> Dot({{1,2},{3,4}},{{5,6},{7,8}})
    Out> {{19,22},{43,50}};

    Or, using the "."-Operator:

    In> {1,2} . {3,4}
    Out> 11;
    In> {{1,2},{3,4}} . {5,6}
    Out> {17,39};
    In> {5,6} . {{1,2},{3,4}}
    Out> {23,34};
    In> {{1,2},{3,4}} . {{5,6},{7,8}}
    Out> {{19,22},{43,50}};

##### See also:

Outer , Cross , IsScalar , IsVector , IsMatrix .

###

InProduct -- inner product of vectors (deprecated)

##### Standard library

##### Calling format:

    InProduct(a,b)

##### Parameters:

`a`, `b` -- vectors of equal length

##### Description:

The inner product of the two vectors "a" and "b" is returned. The vectors need to have the same size.

This function is superceded by the `.` operator.

##### Examples:

    In> {a,b,c} . {d,e,f};
    Out> a*d+b*e+c*f;

##### See also:

Dot , CrossProduct .

###

CrossProduct -- outer product of vectors

##### Standard library

##### Calling format:

    CrossProduct(a,b)
    a X b

Precedence: 30

##### Parameters:

`a`, `b` -- three-dimensional vectors

##### Description:

The cross product of the vectors "a" and "b" is returned. The result is perpendicular to both "a" and "b" and its length is the product of the lengths of the vectors. Both "a" and "b" have to be three-dimensional.

##### Examples:

    In> {a,b,c} X {d,e,f};
    Out> {b*f-c*e,c*d-a*f,a*e-b*d};

##### See also:

InProduct .

###

Outer, o -- get outer tensor product

##### Standard library

##### Calling format:

    Outer(t1,t2)
    t1 o t2

Precedence: 30

##### Parameters:

`t1,t2` -- tensor lists (currently only vectors are supported)

##### Description:

`Outer` returns the outer product of two tensors t1 and t2. Currently `Outer` work works only for vectors, i.e. tensors of rank 1. The outer product of two vectors yields a matrix.

##### Examples:

    In> Outer({1,2},{3,4,5})
    Out> {{3,4,5},{6,8,10}};
    In> Outer({a,b},{c,d})
    Out> {{a*c,a*d},{b*c,b*d}};

    Or, using the "o"-Operator:

    In> {1,2} o {3,4,5}
    Out> {{3,4,5},{6,8,10}};
    In> {a,b} o {c,d}
    Out> {{a*c,a*d},{b*c,b*d}};

##### See also:

Dot , Cross .

###

ZeroVector -- create a vector with all zeroes

##### Standard library

##### Calling format:

    ZeroVector(n)

##### Parameters:

`n` -- length of the vector to return

##### Description:

This command returns a vector of length "n", filled with zeroes.

##### Examples:

    In> ZeroVector(4)
    Out> {0,0,0,0};

##### See also:

BaseVector , ZeroMatrix , IsZeroVector .

###

BaseVector -- base vector

##### Standard library

##### Calling format:

    BaseVector(k, n)

##### Parameters:

`k` -- index of the base vector to construct

`n` -- dimension of the vector

##### Description:

This command returns the "k"-th base vector of dimension "n". This is a vector of length "n" with all zeroes except for the "k"-th entry, which contains a 1.

##### Examples:

    In> BaseVector(2,4)
    Out> {0,1,0,0};

##### See also:

ZeroVector , Identity .

###

Identity -- make identity matrix

##### Standard library

##### Calling format:

    Identity(n)

##### Parameters:

`n` -- size of the matrix

##### Description:

This commands returns the identity matrix of size "n" by "n". This matrix has ones on the diagonal while the other entries are zero.

##### Examples:

    In> Identity(3)
    Out> {{1,0,0},{0,1,0},{0,0,1}};

##### See also:

BaseVector , ZeroMatrix , DiagonalMatrix .

###

ZeroMatrix -- make a zero matrix

##### Standard library

##### Calling format:

    ZeroMatrix(n)
    ZeroMatrix(n, m)

##### Parameters:

`n` -- number of rows

`m` -- number of columns

##### Description:

This command returns a matrix with `n` rows and `m` columns, completely filled with zeroes. If only given one parameter, it returns the square `n` by `n` zero matrix.

##### Examples:

    In> ZeroMatrix(3,4)
    Out> {{0,0,0,0},{0,0,0,0},{0,0,0,0}};
    In> ZeroMatrix(3)
    Out> {{0,0,0},{0,0,0},{0,0,0}};

##### See also:

ZeroVector , Identity .

###

Diagonal -- extract the diagonal from a matrix

##### Standard library

##### Calling format:

    Diagonal(A)

##### Parameters:

`A` -- matrix

##### Description:

This command returns a vector of the diagonal components of the matrix `A`.

##### Examples:

    In> Diagonal(5*Identity(4))
    Out> {5,5,5,5};
    In> Diagonal(HilbertMatrix(3))
    Out> {1,1/3,1/5};

##### See also:

DiagonalMatrix , IsDiagonal .

###

DiagonalMatrix -- construct a diagonal matrix

##### Standard library

##### Calling format:

    DiagonalMatrix(d)

##### Parameters:

`d` -- list of values to put on the diagonal

##### Description:

This command constructs a diagonal matrix, that is a square matrix whose off-diagonal entries are all zero. The elements of the vector "d" are put on the diagonal.

##### Examples:

    In> DiagonalMatrix(1 .. 4)
    Out> {{1,0,0,0},{0,2,0,0},{0,0,3,0},{0,0,0,4}};

##### See also:

Identity , ZeroMatrix .

###

OrthogonalBasis -- create an orthogonal basis

##### Standard library

##### Calling format:

    OrthogonalBasis(W)

##### Parameters:

`W` - A linearly independent set of row vectors (aka a matrix)

##### Description:

Given a linearly independent set `W` (constructed of rows vectors), this command returns an orthogonal basis `V` for `W`, which means that span(V) = span(W) and `InProduct(V[i],V[j]) = 0` when `i != j`. This function uses the Gram-Schmidt orthogonalization process.

##### Examples:

    In> OrthogonalBasis({{1,1,0},{2,0,1},{2,2,1}})
    Out> {{1,1,0},{1,-1,1},{-1/3,1/3,2/3}};

##### See also:

OrthonormalBasis , InProduct .

###

OrthonormalBasis -- create an orthonormal basis

##### Standard library

##### Calling format:

    OrthonormalBasis(W)

##### Parameters:

`W` - A linearly independent set of row vectors (aka a matrix)

##### Description:

Given a linearly independent set `W` (constructed of rows vectors), this command returns an orthonormal basis `V` for `W`. This is done by first using `OrthogonalBasis(W)`, then dividing each vector by its magnitude, so as the give them unit length.

##### Examples:

    In> OrthonormalBasis({{1,1,0},{2,0,1},{2,2,1}})
    Out> {{Sqrt(1/2),Sqrt(1/2),0},{Sqrt(1/3),-Sqrt(1/3),Sqrt(1/3)},
    	{-Sqrt(1/6),Sqrt(1/6),Sqrt(2/3)}};

##### See also:

OrthogonalBasis , InProduct , Normalize .

###

Normalize -- normalize a vector

##### Standard library

##### Calling format:

    Normalize(v)

##### Parameters:

`v` -- a vector

##### Description:

Return the normalized (unit) vector parallel to `v`: a vector having the same direction but with length 1.

##### Examples:

    In> v:=Normalize({3,4})
    Out> {3/5,4/5};
    In> v . v
    Out> 1;

##### See also:

InProduct , CrossProduct .

###

Transpose -- get transpose of a matrix

##### Standard library

##### Calling format:

    Transpose(M)

##### Parameters:

`M` -- a matrix

##### Description:

`Transpose` returns the transpose of a matrix M. Because matrices are just lists of lists, this is a useful operation too for lists.

##### Examples:

    In> Transpose({{a,b}})
    Out> {{a},{b}};

###

Determinant -- determinant of a matrix

##### Standard library

##### Calling format:

    Determinant(M)

##### Parameters:

`M` -- a matrix

##### Description:

Returns the determinant of a matrix M.

##### Examples:

    In> A:=DiagonalMatrix(1 .. 4)
    Out> {{1,0,0,0},{0,2,0,0},{0,0,3,0},{0,0,0,4}};
    In> Determinant(A)
    Out> 24;

###

Trace -- trace of a matrix

##### Standard library

##### Calling format:

    Trace(M)

##### Parameters:

`M` -- a matrix

##### Description:

`Trace` returns the trace of a matrix M (defined as the sum of the elements on the diagonal of the matrix).

##### Examples:

    In> A:=DiagonalMatrix(1 .. 4)
    Out> {{1,0,0,0},{0,2,0,0},{0,0,3,0},{0,0,0,4}};
    In> Trace(A)
    Out> 10;

###

Inverse -- get inverse of a matrix

##### Standard library

##### Calling format:

    Inverse(M)

##### Parameters:

`M` -- a matrix

##### Description:

Inverse returns the inverse of matrix M. The determinant of M should be non-zero. Because this function uses `Determinant` for calculating the inverse of a matrix, you can supply matrices with non-numeric (symbolic) matrix elements.

##### Examples:

    In> A:=DiagonalMatrix({a,b,c})
    Out> {{a,0,0},{0,b,0},{0,0,c}};
    In> B:=Inverse(A)
    Out> {{(b*c)/(a*b*c),0,0},{0,(a*c)/(a*b*c),0},
    {0,0,(a*b)/(a*b*c)}};
    In> Simplify(B)
    Out> {{1/a,0,0},{0,1/b,0},{0,0,1/c}};

##### See also:

Determinant .

###

Minor -- get principal minor of a matrix

##### Standard library

##### Calling format:

    Minor(M,i,j)

##### Parameters:

`M` -- a matrix

`i`, `j` - positive integers

##### Description:

Minor returns the minor of a matrix around the element ( i, j). The minor is the determinant of the matrix obtained from M by deleting the i-th row and the j-th column.

##### Examples:

    In> A := {{1,2,3}, {4,5,6}, {7,8,9}};
    Out> {{1,2,3},{4,5,6},{7,8,9}};
    In> PrettyForm(A);

    /                    \
    | ( 1 ) ( 2 ) ( 3 )  |
    |                    |
    | ( 4 ) ( 5 ) ( 6 )  |
    |                    |
    | ( 7 ) ( 8 ) ( 9 )  |
    \                    /
    Out> True;
    In> Minor(A,1,2);
    Out> -6;
    In> Determinant({{2,3}, {8,9}});
    Out> -6;

##### See also:

CoFactor , Determinant , Inverse .

###

CoFactor -- cofactor of a matrix

##### Standard library

##### Calling format:

    CoFactor(M,i,j)

##### Parameters:

`M` -- a matrix

`i`, `j` - positive integers

##### Description:

`CoFactor` returns the cofactor of a matrix around the element ( i, j). The cofactor is the minor times (-1)^(i+j).

##### Examples:

    In> A := {{1,2,3}, {4,5,6}, {7,8,9}};
    Out> {{1,2,3},{4,5,6},{7,8,9}};
    In> PrettyForm(A);

    /                    \
    | ( 1 ) ( 2 ) ( 3 )  |
    |                    |
    | ( 4 ) ( 5 ) ( 6 )  |
    |                    |
    | ( 7 ) ( 8 ) ( 9 )  |
    \                    /
    Out> True;
    In> CoFactor(A,1,2);
    Out> 6;
    In> Minor(A,1,2);
    Out> -6;
    In> Minor(A,1,2) * (-1)^(1+2);
    Out> 6;

##### See also:

Minor , Determinant , Inverse .

###

MatrixPower -- get nth power of a square matrix

##### Standard library

##### Calling format:

    MatrixPower(mat,n)

##### Parameters:

`mat` -- a square matrix

`n` -- an integer

##### Description:

`MatrixPower(mat,n)` returns the `n`th power of a square matrix `mat`. For positive `n` it evaluates dot products of `mat` with itself. For negative `n` the nth power of the inverse of `mat` is returned. For `n`=0 the identity matrix is returned.

##### Example:

    In> A:={{1,2},{3,4}}
    Out> {{1,2},{3,4}};
    In> MatrixPower(A,0)
    Out> {{1,0},{0,1}};
    In> MatrixPower(A,1)
    Out> {{1,2},{3,4}};
    In> MatrixPower(A,3)
    Out> {{37,54},{81,118}};
    In> MatrixPower(A,-3)
    Out> {{-59/4,27/4},{81/8,-37/8}};

##### See also:

IsSquareMatrix , Inverse , Dot .

###

SolveMatrix -- solve a linear system

##### Standard library

##### Calling format:

    SolveMatrix(M,v)

##### Parameters:

`M` -- a matrix

`v` -- a vector

##### Description:

`SolveMatrix` returns the vector x that satisfies the equation M*x=v. The determinant of M should be non-zero.

##### Examples:

    In> A := {{1,2}, {3,4}};
    Out> {{1,2},{3,4}};
    In> v := {5,6};
    Out> {5,6};
    In> x := SolveMatrix(A, v);
    Out> {-4,9/2};
    In> A * x;
    Out> {5,6};

##### See also:

Inverse , Solve , PSolve , Determinant .

###

CharacteristicEquation -- get characteristic polynomial of a matrix

##### Standard library

##### Calling format:

    CharacteristicEquation(matrix,var)

##### Parameters:

`matrix` -- a matrix

`var` -- a free variable

##### Description:

CharacteristicEquation returns the characteristic equation of "matrix", using "var". The zeros of this equation are the eigenvalues of the matrix, Det(matrix-I*var);

##### Examples:

    In> A:=DiagonalMatrix({a,b,c})
    Out> {{a,0,0},{0,b,0},{0,0,c}};
    In> B:=CharacteristicEquation(A,x)
    Out> (a-x)*(b-x)*(c-x);
    In> Expand(B,x)
    Out> (b+a+c)*x^2-x^3-((b+a)*c+a*b)*x+a*b*c;

##### See also:

EigenValues , EigenVectors .

###

EigenValues -- get eigenvalues of a matrix

##### Standard library

##### Calling format:

    EigenValues(matrix)

##### Parameters:

`matrix` -- a square matrix

##### Description:

EigenValues returns the eigenvalues of a matrix. The eigenvalues x of a matrix M are the numbers such that M*v=x*v for some vector.

It first determines the characteristic equation, and then factorizes this equation, returning the roots of the characteristic equation Det(matrix-x*identity).

##### Examples:

    In> M:={{1,2},{2,1}}
    Out> {{1,2},{2,1}};
    In> EigenValues(M)
    Out> {3,-1};

##### See also:

EigenVectors , CharacteristicEquation .

###

EigenVectors -- get eigenvectors of a matrix

##### Standard library

##### Calling format:

    EigenVectors(A,eigenvalues)

##### Parameters:

`matrix` -- a square matrix

`eigenvalues` -- list of eigenvalues as returned by `EigenValues`

##### Description:

`EigenVectors` returns a list of the eigenvectors of a matrix. It uses the eigenvalues and the matrix to set up n equations with n unknowns for each eigenvalue, and then calls `Solve` to determine the values of each vector.

##### Examples:

    In> M:={{1,2},{2,1}}
    Out> {{1,2},{2,1}};
    In> e:=EigenValues(M)
    Out> {3,-1};
    In> EigenVectors(M,e)
    Out> {{-ki2/ -1,ki2},{-ki2,ki2}};

##### See also:

EigenValues , CharacteristicEquation .

###

Sparsity -- get the sparsity of a matrix

##### Standard library

##### Calling format:

    Sparsity(matrix)

##### Parameters:

`matrix` -- a matrix

##### Description:

The function `Sparsity` returns a number between `0` and `1` which represents the percentage of zero entries in the matrix. Although there is no definite critical value, a sparsity of `0.75` or more is almost universally considered a "sparse" matrix. These type of matrices can be handled in a different manner than "full" matrices which speedup many calculations by orders of magnitude.

##### Examples:

    In> Sparsity(Identity(2))
    Out> 0.5;
    In> Sparsity(Identity(10))
    Out> 0.9;
    In> Sparsity(HankelMatrix(10))
    Out> 0.45;
    In> Sparsity(HankelMatrix(100))
    Out> 0.495;
    In> Sparsity(HilbertMatrix(10))
    Out> 0;
    In> Sparsity(ZeroMatrix(10,10))
    Out> 1;

###

Cholesky -- find the Cholesky Decomposition

##### Standard library

##### Calling format:

    Cholesky(A)

##### Parameters:

`A` -- a square positive definite matrix

##### Description:

`Cholesky` returns a upper triangular matrix `R` such that `Transpose(R)*R = A`. The matrix `A` must be positive definite, `Cholesky` will notify the user if the matrix is not. Some families of positive definite matrices are all symmetric matrices, diagonal matrices with positive elements and Hilbert matrices.

##### Examples:

    In> A:={{4,-2,4,2},{-2,10,-2,-7},{4,-2,8,4},{2,-7,4,7}}
    Out> {{4,-2,4,2},{-2,10,-2,-7},{4,-2,8,4},{2,-7,4,7}};
    In> R:=Cholesky(A);
    Out> {{2,-1,2,1},{0,3,0,-2},{0,0,2,1},{0,0,0,1}};
    In> Transpose(R)*R = A
    Out> True;
    In> Cholesky(4*Identity(5))
    Out> {{2,0,0,0,0},{0,2,0,0,0},{0,0,2,0,0},{0,0,0,2,0},{0,0,0,0,2}};
    In> Cholesky(HilbertMatrix(3))
    Out> {{1,1/2,1/3},{0,Sqrt(1/12),Sqrt(1/12)},{0,0,Sqrt(1/180)}};
    In> Cholesky(ToeplitzMatrix({1,2,3}))
    In function "Check" :
    CommandLine(1) : "Cholesky: Matrix is not positive definite"

##### See also:

IsSymmetric , IsDiagonal , Diagonal .