These pages are a collection of my personal review on Matrix Analysis, mainly about matrices and something relating to them, such like the Space, Norm, etc. They are really the things that matter in data science and almost all the machine learning algorithms. Hence, I collected them in this form for the convenience of anyone who wants a quick desktop or mobile reference.
For x,y∈Rn, we say that x is majorized by y, denoted by x≺y, if
∑j=1kxj↓∑j=1nxj↓≤=∑j=1kyj↓forkin[1:n−1]∑j=1nyj↓
2)Weak Majorization
For x,y∈Rn, we say that x is weak majorized by y, denoted by x≺y, if
∑j=1kxj↓∑j=1nxj↓≤≤∑j=1kyj↓forkin[1:n−1]∑j=1nyj↓
1.7 Supremum and Infimum
T is a subset of poset (S,⪯), a is said to be a supremum of T, denoted by sup T, if
1)2)3)a∈Sb⪯aforallb∈Ta⪯cforanyotherupperboundc
a is said to be a infimum of T, denoted by inf T, if
1)2)3)a∈Sa⪯bforallb∈Tc⪯aforanyotherlowerboundc
1.8 Lattice
Let a,b∈S, then inf{a,b} is also denoted by a∧b, called the meet of a,b ; and sup {a,b} is denoted by a∨b, called the join of a,b. Then, a poset (S,⪯) is called a lattice if a∧b and a∨b exist for all a,b∈S.
2 Linear Spaces
2.1 Linear Space
A set χ is said to be a linear space(or vector space) over a filed F, if
The characteristic polynomial of A is defined to be
CA(z)=det(zI−A)
A complex number
λ
satisfying
CA(λ)=0
is called an eigenvalue of A, and the vector
x∈Cn
such that
Ax=λx
is called the right eigenvector of A corresponding to the eigenvalue
λ
.
3.2 Spectrum
Spectrum is the set of eigenvalues of A. Spectral Radius ρ(A) is the maximum modulus of the eigenvalues of A, i.e., ρ(A)=max|λi|.
3.3 Diagonalization
cA(z)=(z−λ1)n1(z−λ2)n2...(z−λnl)nl
where
ni≥1
and
∑li=1ni=n
,
ni
is the algebraic multiplicity of
λi
.
Choosing arbitrary basis from εi˜ to form P, and tranfer A by P−1AP to get a Jordan Canonical Form. We can also get P from:
Aμ1Aμ2Aμ3⋮===λμ1λμ2+μ1λμ3+μ2
3.5 QR Factorization
An×m=QR
QR==[q1q1...qm]Q′A
3.6 Schur Factorization
Tn×n=U∗An×nU
U: unitary matrix
A: with eigenvalues
λ1,...,λn T: an upper triangular matrix
3.7 SVD Decomposition
Am×n=USm×nV∗
The left -singular vectors of A(columns of U) are a set of orthonormal eigenvectors of
AA∗
.
The right-singular vectors of A(columns of V) are a set of orthonormal eigenvectors of
A∗A
.
The diagnal entries of S are the square roots of the non-negative eigenvalues of both
A∗A
and
AA∗
, known as the singular values.
e.g. For a square matrix T
For A, let λmin=λ1≤λ2≤...≤λn=λmax, 1≤i1≤i2≤...≤ik≤n are integers, xi1,xi2,...xik are orthonormal vectors such that Axip=λipxip, S=span{xi1,xi2,...,xik},then we have
a)b)λi1≤x∗Ax≤λikforx∈Sλmin≤x∗Ax≤λmaxforx∈Cn
4.3 Hermitian Matrix
Hermitian Matrix: A=A∗ Skew-Hermitian: A=−A∗ Theorem: If A is a Hermitian Matrix, then a) x∗Ax is real for all x∈Cn. b)λ(A) are real. c) S∗AS is Hermitian.
5 Special Topics
5.1 Stochastic Matrix
A nonnegative matrix Sn×n is said to be a stochastic matrix if each of its row sums is equal to one. S satisfies Se=e, which means the eigenvalue and eigenvector of S are respectively 1 and [1...1]T.Obviously , if S and T are stochastic, so is ST.