https://stanford.edu/~shervine CS 229 – Machine Learning VIP Refresher: Linear Algebra and Calculus outer product: for x ∈ Rm , y ∈ Rn , we have: Afshine Amidi and Shervine Amidi xy T =  x1 y1 xm y1 .. . x1 yn  .. ∈ Rm×n . xm yn August 8, 2018 r Matrix-vector multiplication – The product of matrix A ∈ Rm×n and vector x ∈ Rn is a vector of size Rn , such that: General notations aT n r,1 x X ..  = Ax = ac,i xi ∈ Rn . T i=1 ar,m x  r Vector – We note x ∈ Rn a vector with n entries, where xi ∈ R is the ith entry: x1 ! x2 .. x= ∈ Rn . xn where aT r,i are the vector rows and ac,j are the vector columns of A, and xi are the entries of x. r Matrix – We note A ∈ Rm×n a matrix with n rows and m, where Ai,j ∈ R is the entry located in the ith row and j th column: ! A1,1 · · · A1,n .. .. A= ∈ Rm×n . . Am,1 · · · Am,n r Matrix-matrix multiplication – The product of matrices A ∈ Rm×n and B ∈ Rn×p is a matrix of size Rn×p , such that: aT r,1 bc,1 .. AB =  . T ar,m bc,1  Remark: the vector x defined above can be viewed as a n × 1 matrix and is more particularly called a column-vector. r Identity matrix – The identity matrix I ∈ Rn×n is a square matrix with ones in its diagonal and zero everywhere else:  1 0 ··· 0  . .. ..  . ..  . I= 0  .. . . .. . . 0 . 0 ··· 0 1 aT n r,1 bc,p X n×p .. = ac,i bT r,i ∈ R . i=1 aT r,m bc,p  T where aT r,i , br,i are the vector rows and ac,j , bc,j are the vector columns of A and B respectively. r Transpose – The transpose of a matrix A ∈ Rm×n , noted AT , is such that its entries are flipped: ∀i,j, Remark: for all matrices A ∈ Rn×n , we have A × I = I × A = A. AT i,j = Aj,i Remark: for matrices A,B, we have (AB)T = B T AT . r Diagonal matrix – A diagonal matrix D ∈ Rn×n is a square matrix with nonzero values in its diagonal and zero everywhere else:  d1 0 · · · 0  .. .. ..  . . .  D= 0  . .. .. .. . . 0 0 0 dn r Inverse – The inverse of an invertible square matrix A is noted A