Linear Algebra Review and NumPy Basics1

Linear Algebra Review and NumPy Basics1

Chris J. Maddison

University of Toronto

1Slides adapted from Ian Goodfellow's Deep Learning textbook lectures

Intro ML (UofT)

STA314-Tut02

1 / 27

About this tutorial

Not a comprehensive survey of all of linear algebra. Focused on the subset most relevant to machine learning. Larger subset: e.g., Linear Algebra by Gilbert Strang

Intro ML (UofT)

STA314-Tut02

2 / 27

Scalars

A scalar is a single number Integers, real numbers, rational numbers, etc. Typically denoted in italic font:

a, n, x

Intro ML (UofT)

STA314-Tut02

3 / 27

Vectors

A vector is an array of d numbers xi be integer, real, binary, etc. Notation to denote type and size:

x Rd

x1

x = x...2 xd

Intro ML (UofT)

STA314-Tut02

4 / 27

Matrices

A matrix is an array of numbers with two indices

Ai,j be integer, real, binary, etc.

Notation to denote type and size:

A Rm?n

A = A1,1 A1,2 A2,1 A2,2

Intro ML (UofT)

STA314-Tut02

5 / 27

be defined, A must have the same number of columns as B has

Matrix (Dot) Product ptheMemmaattrrniixxapn(rdDodBouctt)isjuPosftrsobhydaupplecactning tpw, othoernmCoreismoaftsrihcaespetomgethepr.,

matrix

product

just

by C

placing = AB.

two

or

more

matrices

toget(h2e.4r),

Matrix product AB is the matrix such that ct operation is deCfine=d AbyB.

(2.4)

eration

is

defineCd ib,jy=

X

(AB

Ai,k Bk,j

.)i

,j

=

Xk

k

Ai,k Bk,j .

(2.5)

the

standardCpir,oj d=uct kof

Atwi,ok Bmka,tjr.ices

is

not

just

a

matrix

(2.5) containing

f the individual elements. Such an operation exists and is called the

parnodduacrtdoprrHodaduacmt aorfdtwproodmucatt,riacneds iiss dneontojtuesdtaas mAatrBix. containing

ondduicvtidbumetawl eeelnemtweontvse.ctSourschxaanndopy=eorfamtihoensaemxiestdsimanend?siionsncaaliltlyedistthhee

t xor>yH.aWdaemcarndthpirnokduocf tt,haenmdaitsrixdepnrotdeudctaCs A= ABB.as computing

tbpertowdeuecnt tbweotwveeecntororswxi oafnAd yanodf ctohleumsanmjeodfimBe. nsionality is thep

y. We can think of pthe m34atrix product Cn= AB as coMmusptuting

duct between row i of A and column j of B.

match

34

(Goodfellow 2016)

This also defines matrix-vector products Ax and x A.

Intro ML (UofT)

STA314-Tut02

6 / 27

Identity Matrix

The identity matrix for Rd is the matrix Id such that

x Rd , Id x = x

For example, I3:

100

I3 = 0 1 0

001

Intro ML (UofT)

STA314-Tut02

7 / 27

Matrix Transpose

The transpose of a matrix A is the matrix A such that (A )i,j = Aj,i .

A1,1

A = A2,1 A3,1

A1,2 A2,2 = A A3,2

=

A1,1 A1,2

A2,1 A2,2

A3,1 A3,2

The transpose of a matrix can be thought of as a mirror image across the main diagonal. The transpose switches the order of the matrix product.

(AB) = B A

Intro ML (UofT)

STA314-Tut02

8 / 27

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download