Linear Algebra

learning

1.1

1. vector addition
2. scalar multiplication
3. the relation and distinction among line, plane and space

1.2

1. dot product :$w_1.v_1+w_2.v_2=0$
2. length = $\left\|v\right\|=\sqrt {v.v}$
3. unit vector : $u=\frac{v}{\left\|v\right\|}$
4. the angle between two vectors
5. the dot product is $v.w=0$ when vector $v$ and $w$ are perpendicular

1.3

1. matrix times vector
2. inverse matrix and singular matrix
3. independence and dependence

2.1

1. matrix equation :$Ax=b$
2. identity matrix : $Ix=b$

2.2

1. upper triangular system
2. the pivots are on the diagonal of the triangle after elimination
3. elimination succeed if $n$ equation we get n pivots

2.3

1. $Ax=x_1$ times column $1 + ... + x_n$ times column $n$
2. elimination matrix and exchange matrix
3. augmented matrix
4. when $A$ multiplies any matrix $B$, it multiplies each column of $B$ separately

2.4

1. the entry in row $i$ and column $j$ of $AB$ is (row $i$ of $A$).(column $j$ of $B$)
2. matrix times column
3. row times matrix
4. block elimination

2.5

1. the inverse exists if and only if elimination produces $n$ pivots
2. if $Ax=0$ for a nonzero vector $x$, then $A$ has no inverse
3. the inverse of $AB$ is the reverse product $B^{-1}A^{-1}$
4. the steps of calculating $A^{-1}$

2.6

1. $A=LU$
2. $Lc=b, Ux=c$
3. the cost of elimination

2.7

1. transpose : $\left(A^T\right)_{ij}=A_{ji}$
2. • $\left(A+B\right)^T=A^T+B^T$
   • $\left(AB\right)^T=B^TA^T$
   • $\left(A^{-1}\right)^T=\left(A^T\right)^{-1}$

3. if $A^T=A,A=LDL^T$

4. a permutation matrix $P$ has a 1 in each row and column, and $P^T=P^{-1}$

3.1

1. the space $R^n$ consists of all column vectors $v$ with $n$ conponents
2. $M$(2 by 2 matrices) and $F$(functions) and $Z$(zero vector alone) are vector spaces
3. if $v$ and $w$ are vectors in the subspace abd $c$ is any scalar, then
   
   • $v+w$ is in the subspace
   • $cv$ is in the subspace
4. a subspace containing $v$ and $w$ must contain all linear combinations $cv+dw$
5. the system $Ax=b$ is solvable if and only if $b$ is in the column space of $A$

3.2

1. the nullspace $N(A)$ is a subspace of $R^n$. It contains all solutions to $Ax=0$
2. solving $Ax=0$ by elimination
3. the complete solution to $Ax=0$ is a combination of the special solutions
4. if $n>m$ then $A$ has at least one column without pivots, giving a special solution. So there are nonzero vectors $x$ in the nullspace of this rectangular $A$.

3.3

1. the rank of $A$ is the number of pivots
2. those $r$ pivot columns are not combinations of earlier columns
3. those $n-r$ free columns are combinations of earlier columns

3.4

1. the rank $r$ is the number of pivots
2. $Ax=b$ is solvable if and only if the last $m-r$ equations reduce to $0 = 0$
3. complete solution :$x=x_p+x_n$
4. 
   
   • $R=[I]:r=m=n$
   • $R=\left[ \begin{array}{ll}I & F \end{array}\right]:r=m<n$
   • $R=\left[\begin{array}{l}I\\0\end{array}\right]:r=n<m$
   • $R=\left[\begin{array}{ll}I&F\\0&0\end{array}\right]:r<m,r<n$

3.5

1. the columns of $A$ are independent if $x = 0$ is the only solution to $Ax=0$
2. a basis consists of linearly independent vectors that span the space
3. the pivot columns are one basis for the column space
4. this number of vectors in a basis is the dimension of the space

3.6

1. the column space and row space both have dimension $r$
2. the nullspaces have dimensions $n - r$
3. the left nullspaces have dimensions $m - r$
4. (dimension of column space) + (dimension of nullspace) = dimension of $R^n$
5. every rank one matrix has the special form $A=uv^T$ =column times row

4.1

1. $V$ and $W$ are orthogonal complements if $W$ contains all vectors perpendicular to $V$
2. inside $R^n$, the dimensions of complements $V$ and $W$ add to $n$
3. $N(A)$ and $C(A^T)$ are orthogonal complements; $N(A^T)$ and $C(A)$ are orthogonal complements
4. any $n$ independent vectors in $R^n$ will span $R^n$
5. $x=x_r+x_n$( a row space vector $x_r$ and nullspace vector $x_n$)

4.2

1. $p=\hat{x}a=\frac{a^Tb}{a^Ta}a=Pb$ when the matrix is $p=\frac{aa^T}{a^Ta}$
2. when $A$ has full rank $n$, the equation $A^TA\hat{x}=A^Tb$ leads to $\hat{x}$ and $p=A\hat{x}$
3. the projection matrix $P=A\left(A^TA\right)^{-1}A^T$ has $P^T=P$ and $P^2=P$
4. when A has independent columns, $A^TA$ is square, symmertic, and invertible

4.3

1. the best $\hat{x}$ comes from the normal equations $A^TA\hat{x}=A^Tb$
2. the least squares solution $\hat{x}$ makes $E=\left|\left|Ax-b\right|\right|^2$ as small as possible
3. the partily derivatives of $\left|\left|Ax-b\right|\right|^2$ are zero when $A^TA\hat{x}=A^Tb$
4. the closest line $C+Dt$ has heights $p_1,...,p_m$ with errors $e_1,...,e_m$

4.4

1. when $Q$ is quare, $Q^TQ=I$ means that $Q^T=Q^{-1}$: transpose = inverse
2. the length of $Qx$ equals the length of $x:\left|\left|Qx\right|\right|=\left|\left|Q\right|\right|$
3. $Q$ preserves dot products:$(Qx)^T(Qy)=x^TQ^TQy=x^Ty$
4. if $Q$ is square then $P=I$ and every $b=q_1(q_1^Tb)+...+q_n(q_n^Tb)$
5. the Gram-Schmidt process
6. • $R^TR\hat{x}=R^TQ^Tb$
   • $R\hat{x}=Q^Tb$
   • $\hat{x}=R^{-1}Q^Tb$

5.1

1. the determinant is zero when the matrix has no inverse
2. the determinant changes sign when two rows(or two columns) are exchanged
3. the determinant of the n by n identity matrix is 1
4. the determinant is a linear function of each row separately
5. subtracting a multiple of one row from another row leaves det $A$ unchange
6. if $A$ is singular then $det A=0$.If $A$ is invertible then $det A \ne 0$
7. • $det A=\pm det U=\pm$(product of the pivots)
   • $|AB|=|A||B|$
   • (det $A$)(det $A^{-1}$=det $I$ =1
   • $|A|=|A^T|$

5.2

1. if no row exchanges are involved, mutiply pivots to find the determinant
2. the determinant of that corner submatrix $A_k$ is $d_1d_2...d_k$
3. det $A$ = sum over all $n!$ column permutations $P=(\alpha,\beta,...,\omega)$
   =$\sum(det P)a_{1\alpha},a_{2\beta}...a_{n\omega}$
4. the cofactor expansion is det $A$ = $a_{11}C_{11}+a_{12}C_{12}+...+a_{1n}C_{1n}$
5. $D_n=2D_{n-1}-D_{n-2}$

5.3

1. $A^{-1}=\frac{C^T}{detA}$
2. the volume of a box is |det $A$|, when the box edges are the rows of $A$
3. the cross product is a vector with length $\lVert{u}\rVert \Vert{v}\rVert\lvert{\sin{\theta}}\rvert$.Its direction is perpendicular to $u$ and $v$

6.1

1. the basic equation is $Ax=\lambda x$. the number $\lambda$ is an eigenvalue of $A$
2. the number $\lambda$ is an eigenvalue of $A$ if and only if $A-\lambda I$ is singular
3. the eigenvalue of $A^2$ and $A^{-1}$ and $\lambda ^2$ and $\lambda ^{-1}$, with the same eigenvectors
4. the product of the $n$ eigenvalues equals the determinants. The sum of the $n$ eigenvalues equals the sum of the $n$ diagonal entries
5. projections $P$, reflections $R$, $90^o$ rotations $Q$ have special eigenvalues $1, 0, -1, i, -i$. Singualr matrices have $\lambda =0$.Triangular matrices have $\lambda$'s on their dialonal

6.2

1. If $A$ has $n$ independent eigenvectors $x_1,\dots,x_n$, they go into the columns of $S$. $A$ is diagonalized by $S$. $S^{-1}AS=\Lambda$ and $A=S\Lambda S^{-1}$
2. $A^k=S\Lambda ^kS^{-1}$
3. Solution for $u_{k+1}=A^ku_0=c_1(\lambda _1)^kx_1+\cdots+c_n(\lambda_n)^kx_n$
4. $A$ is diagonalizable if every eigenvalue has enough eigenvectors$(GM=AM)$

6.3

1. $n$ equations $\frac{du}{dt}=Au$ starting from the vector $u(0)$ at $t=0$
2. $u(t)=c_1e^{\lambda_1t}x_1+\cdots+c_ne^{\lambda_nt}x_n$
3. $u(t)$ approaches zero (stability) if every $\lambda$ has negative real part
4. the solution is always $u(t)=e^{At}u(0)$, with the matrix exponential $e^{At}$

6.4

1. A symmetric matrix has only real eigenvalues
2. The eigenvectors can be chosen orthonormal
3. $A=Q\Lambda Q^{-1}=Q\Lambda Q^T$ with $Q^{-1}=Q^T$
4. the number of positive eigenvalues of $A=A^T$ equals the number of positive pivots
5. Every square matrix can be "triangularized" by $A=QTQ^{-1}$

6.5

1. Symmetric matrices that have positive eigenvalues
2. The eigenvalues of A are positive is and only if $a>0$ and $ac-b^2>0$
3. The eigenvalues of $A=A^T$ are posivive if and only if the pivots are positive:$a>0$ and $\frac{ac-b^2}{a}>0$
4. A is positive definite if $x^TAx>0$ for every nonzero vector $x$:
   $x^TAx=\begin{bmatrix}x&y\end{bmatrix}=\left[\begin{array}{ll}a&b\\b&c\end{array}\right]\left[\begin{array}{l}x\\y\end{array}\right]=ax^2+2bxy+cy^2>0$
5. $A=R^TR$ is automatically definite if $R$ has independent columns
6. The ellipse $x^TAx=1$ has its axes along the eigenvectors of $A$. Lengths $1/\sqrt{\lambda}$

6.6

1. Let $M$ be any invertible matrix. Then $B=M^{-1}AM$ is similar to $A$
2. Similar matrices have the same eigenvalues. Eigenvectors are multiplied by $M^{-1}$
3. If $A$ has $n$ independent eigenvectors then $A$ is similar to $\Lambda$ (take $M=S$)

6.7

1. the numbers $\sigma_1^2,\dots,\sigma_r^2$ are the nonzero eigenvalues of $AA^T$ and $A^TA$
2. The orthonormal columns of $U$ and $V$ are eigenvectors of $AA^T$ and $A^TA$

7.1

1. • $T(v+w)=T(v)+T(w)$
   • $T(cv)=cT(v)$
2. Combinations to combinations: $T(c_1v_1+\cdots+c_nv_n)=c_1T(v_1)+\cdots+c_nT(v_n)$

7.2

1. Suppose we know $T(v_1),\dots,T(v_n)$ for the basis vectors $v_1,\dots,v_n$. Then linearity prodeces $T(v)$ for every other input vector $v$
2. The $j$th column of $A$ is found by applying $T$ to the $j$th basis vector $v_j$
   $T(v_j)$ = combination of basis vector of $W=a_{1j}w_1+\cdots+a_{mj}w_m$
3. If $A$ and $B$ represent $T$ and $S$, and the output basis for $S$ is the input basis for $T$, then the matrix $AB$ represents the transformation $T(S(u))$

7.3

1. $S^{-1}AS=\Lambda$ when the input and output bases are eigenvectors of $A$
2. $U^{-1}AV=\sum$ when those bases are eigenvectors of $A^TA$ and $AA^T$
3. The $SVD$ chooses an input basis of $v$'s and an output basis of $u$'s. Those orthonormal bases diagonalized $A$. This is $Av_i=\sigma_iu_i$, and in matrix from $A=U\Sigma V^T$
4. Every real square matrix can be factored into $A=QH$, where $Q$ is orthogonal and $H$ is symmetric positive semidefinite. If $A$ is invertible, $H$ is positive definite

归纳总结

1. 前七章主要讲述了向量、矩阵、空间、空间的投影、矩阵的秩、特征值和特征向量、线性变换。
2. 与学校所发教材相比，前四章内容大同小异，只是例题较少，但解题方法更为简单（如对零空间的求解、LU分解），措辞更为准确。
3. 第五到七章有较多内容未学过，阅读起来比较困难，对知识点理解得比较浅。