Column Space Null Space
All linear combinations c v + d w c \textbf{v}+d\textbf{w} cv+dw are in the space.
Subspace S S S and T T T:
Intersection S ∩ T S\cap T S∩T is a subspace
Column Space
A = [ 1 1 2 2 1 3 3 1 4 4 1 5 ] A=\left[ \begin{matrix} 1&1&2 \\ 2&1&3 \\ 3&1&4 \\ 4&1&5 \end{matrix} \right] A=⎣⎢⎢⎡123411112345⎦⎥⎥⎤
The column space of A is a subspace of R 4 R^4 R4.
C ( A ) C(A) C(A) is all linear combinations of columns.
A X = b AX=b AX=b not have a solution for every b b b.
But for some right hand side, we can solve it, exactly when b b b is in the column space.
Null Space
Null space of A: contains all solutions X X X to the equation A X = 0 AX = \bf{0} AX=0
In this example, the null space is a subspace of R 3 R^3 R3.
Lecture 7: Pivot Variables-Free Variables
A = [ 1 2 2 2 2 4 6 8 3 6 8 10 ] A=\left[ \begin{matrix} 1&2&2&2 \\ 2&4&6&8 \\ 3&6&8&10 \end{matrix} \right] A=⎣⎡1232462682810⎦⎤
− > [ 1 2 2 2 0 0 2 4 0 0 0 0 ] − > e c h e l o n f o r m ->\left[ \begin{matrix} 1&2&2&2 \\ 0&0&2&4 \\ 0&0&0&0 \end{matrix} \right]->echelon\ form −>⎣⎡100200220240⎦⎤−>echelon form
2 pivot columns
2 free columns
x 1 + 2 x 2 + 2 x 3 + 2 x 4 = 0 2 x 3 + 4 x 4 = 0 \begin{aligned} x_1 + 2 x_2 + 2x_3 + 2x_4 = 0 \\ 2x_3 + 4x_4 = 0 \end{aligned} x1+2x2+2x3+2x4=02x3+4x4=0
free variables: x 2 , x 4 x_2, x_4 x2,x4
[ − 2 1 0 0 ] [ 2 0 − 2 1 ] \left[ \begin{matrix} -2 \\ 1 \\ 0 \\ 0 \end{matrix} \right] \left[ \begin{matrix} 2 \\ 0 \\ -2 \\ 1 \end{matrix} \right] ⎣⎢⎢⎡−2100⎦⎥⎥⎤⎣⎢⎢⎡20−21⎦⎥⎥⎤
So the solution is:
X = c [ − 2 1 0 0 ] + d [ 2 0 − 2 1 ] X=c\left[ \begin{matrix} -2 \\ 1 \\ 0 \\ 0 \end{matrix} \right]+ d\left[ \begin{matrix} 2 \\ 0 \\ -2 \\ 1 \end{matrix} \right] X=c⎣⎢⎢⎡−2100⎦⎥⎥⎤+d⎣⎢⎢⎡20−21⎦⎥⎥⎤
Rank: the number of the pivot variables
n − r = 4 − 2 n-r = 4-2 n−r=4−2: number of free variables
R = r e d u c e d r o w e c h e l o n f o r m R=reduced\ row\ echelon\ form R=reduced row echelon form
[ 1 2 2 2 0 0 2 4 0 0 0 0 ] − > [ 1 2 0 − 2 0 0 1 2 0 0 0 0 ] = R \left[ \begin{matrix} 1&2&2&2 \\ 0&0&2&4 \\ 0&0&0&0 \end{matrix} \right]-> \left[ \begin{matrix} 1&2&0&-2 \\ 0&0&1&2 \\ 0&0&0&0 \end{matrix} \right]=R ⎣⎡100200220240⎦⎤−>⎣⎡100200010−220⎦⎤=R
Lecture 8: AX=b
Sovability condition on b
A X = b AX = b AX=b solvable when b b b is in C(A)
If a combination of rows of A A A gives zero row, the same combination of b b b must give 0.
Find complete solution to A X = b AX=b AX=b
- x p a r t i c u l a r x_{particular} xparticular: set all free variables to zero. Solve A X = b AX=b AX=b for pivot variables
- x n u l l s p a c e x_{nullspace} xnullspace:
x = x p + x n x = x_p + x_n x=xp+xn
Full column rank
r = n r = n r=n
No free variables
N ( A ) = { z e r o v e c t o r } N(A) = \{zero \ vector\} N(A)={
zero vector}
X = x p X = x_p X=xp
unique solution if it exists
Full row rank
r = m r=m r=m
Can solve A X = b AX = b AX=b for every b b b
Left with n − r n-r n−r free variables
Conclusion
r = m = n r = m = n r=m=n:
R = I R = I R=I
1 solution for A X = b AX=b AX=b
r = n < m r = n<m r=n<m:
R = [ I 0 ] R = \left[ \begin{matrix} I\\0 \end{matrix} \right] R=[I0]
0 or 1 solution
r = m < n r = m < n r=m<n:
R = [ I F ] R = \left[ \begin{matrix} I & F \end{matrix} \right] R=[IF]
∞ \infin ∞ solutions
r < m , r < n r < m, r < n r<m,r<n:
R = [ I F 0 0 ] R = \left[ \begin{matrix} I & F \\ 0 & 0 \end{matrix} \right] R=[I0F0]
0 or ∞ \infin ∞ solutions
Lecture 9: Linear Independence
Linear independence
Vectors x 1 , x 2 , … , x n x_1, x_2, \dots, x_n x1,x2,…,xn are independent if no combination gives zero vector (except the zero combination)
c 1 x 1 + c 2 x x + ⋯ + c n x n ≠ 0 ( c i ≠ 0 ) c_1x_1+c_2x_x+\dots+c_nx_n \neq 0 (c_i\neq0) c1x1+c2xx+⋯+cnxn=0(ci=0)
Span a space
vectors: v 1 , … , v l v_1, \dots, v_l v1,…,vl span a space means the space consists of all combinations of those vectors
Basis
Basis for a space is a sequence of vectors with 2 properties:
- They are independent
- They span the space
E.g: Space is R 3 R^3 R3
One basis is :
[ 1 0 0 ] [ 0 1 0 ] [ 0 0 1 ] \left[ \begin{matrix} 1\\0\\0 \end{matrix} \right] \left[ \begin{matrix} 0\\1\\0 \end{matrix} \right]\left[ \begin{matrix} 0\\0\\1 \end{matrix} \right] ⎣⎡100⎦⎤⎣⎡010⎦⎤⎣⎡001⎦⎤
Another basis:
[ 1 1 2 ] [ 2 2 5 ] [ 0 4 π ] \left[ \begin{matrix} 1\\1\\2 \end{matrix} \right] \left[ \begin{matrix} 2\\2\\5 \end{matrix} \right]\left[ \begin{matrix} 0\\4\\ \pi \end{matrix} \right] ⎣⎡112⎦⎤⎣⎡225⎦⎤⎣⎡04π⎦⎤
Dimension
Every basis for the space has the same number of vectors. This number is called the dimension
rank( A A A) = # pivot columns = dimension of C( A A A)
d i m N ( A ) = dimN(A)= dimN(A)= # free variables
Lecture 10
4 Subspaces
column space C ( A ) C(A) C(A)
nullspace N ( A ) N(A) N(A)
rowspace = all combinations of the columns of A T A^T AT = C ( A T ) C(A^T) C(AT)
nullspace N ( A T ) N(A^T) N(AT) = the left nullspace of A A A
C ( A ) C(A) C(A) is in R m R^m Rm
N ( A ) N(A) N(A) is in R n R^n Rn
C ( A T ) C(A^T) C(AT) is in R n R^n Rn
N ( A T ) N(A^T) N(AT) is in R m R^m Rm