Matrices provide a compact way to represent linear mappings and data, where rows often correspond to observations and columns correspond to features. This chapter focuses on three central questions about matrices:

1. How to summarize a matrix with key numerical characteristics.

2. How to decompose a matrix into simpler, interpretable components.

3. How to use these decompositions for approximations and analysis.

<p align="center">

<img src="Figure4.1MML.png" alt=" A mind map of the concepts introduced in this chapter, along with where they are used in other parts of the book." width="400">

</p>

The **determinant** of a square matrix \( \mathbf{A} \in \mathbb{R}^{n \times n} \), denoted as \(\det(\mathbf{A})\) or \(|\mathbf{A}|\), is a scalar that characterizes several key properties of \( \mathbf{A} \).

For small matrices:

\det \begin{pmatrix} a_{11} \end{pmatrix} = a_{11}, \quad

\det \begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix} = a_{11}a_{22} - a_{12}a_{21}.

<div class="example">

Let

\mathbf{A} =

\begin{bmatrix}

3 & -1 \\

5 & 2

\end{bmatrix}.

The determinant of $\mathbf{A}$ is

\det(\mathbf{A})

= (3)(2) - (-1)(5)

= 6 + 5

= 11.

</div>

For larger matrices, we can compute determinants recursively using the **Laplace expansion**:

\det(\mathbf{A}) = \sum_{k=1}^n (-1)^{k+j} a_{kj} \det(\mathbf{A}_{k,j}),

where \( \mathbf{A}_{k,j} \) is the submatrix obtained by removing row \( k \) and column \( j \).

<div class="example">

Let

\mathbf{A} =

\begin{bmatrix}

2 & -1 & 3 \\

1 & 4 & 0 \\

-2 & 5 & 1

\end{bmatrix}.

We compute \(\det( \mathbf{A})\) using Laplace expansion along the first row.

\det( \mathbf{A})

= 2

\begin{vmatrix}

4 & 0 \\

5 & 1

\end{vmatrix}

- (-1)

\begin{vmatrix}

1 & 0 \\

-2 & 1

\end{vmatrix}

+ 3

\begin{vmatrix}

1 & 4 \\

-2 & 5

\end{vmatrix}.

Next, we compute each minor

\begin{align*}

\begin{vmatrix}

4 & 0 \\

5 & 1

\end{vmatrix}

&= (4)(1) - (0)(5) = 4\\

\begin{vmatrix}

1 & 0 \\

-2 & 1

\end{vmatrix}

&= (1)(1) - (0)(-2) = 1\\

\begin{vmatrix}

1 & 4 \\

-2 & 5

\end{vmatrix}

&= (1)(5) - (4)(-2) = 5 + 8 = 13

\end{align*}

Using these determinants in the Laplace formula gives us

\det( \mathbf{A}) = 2(4) - (-1)(1) + 3(13)

= 8 + 1 + 39

= 48.

</div>

A matrix \( \mathbf{A} \) is **invertible** if and only if \( \det(\mathbf{A}) \neq 0 \).

<div class="example">

Verify that a matrix is invertible if and only if its determinant is non-zero.

</div>

For triangular matrices, the determinant equals the product of the diagonal elements.

<div class="example">

Let

\mathbf{A} =

\begin{bmatrix}

3 & 2 & -1 \\

0 & 5 & 4 \\

0 & 0 & 7

\end{bmatrix}.

Since \( \mathbf{A}\) is upper triangular, we already know that

\det( \mathbf{A}) = 3 \cdot 5 \cdot 7 = 105.

But we will verify this using Laplace expansion.

Expand along column 1:

\begin{align*}

\det(\mathbf{A}) &=

3

\begin{vmatrix}

5 & 4 \\

0 & 7

\end{vmatrix}

- 0

\begin{vmatrix}

2 & -1 \\

0 & 7

\end{vmatrix}

+ 0

\begin{vmatrix}

2 & -1 \\

5 & 4

\end{vmatrix}\\

&= 3

\begin{vmatrix}

5 & 4 \\

0 & 7

\end{vmatrix}\\

&= 3\left[(5)(7) - (4)(0)\right]\\

&= 3(35)\\

&= 105.

\end{align*}

This matches the product of the diagonal entries, as expected for triangular matrices.

</div>

The determinant changes sign when two rows (or columns) are swapped, and scales when a row is multiplied by a scalar.

<div class="example">

Let

\mathbf{A}=\begin{bmatrix}1 & 2\\[4pt] 3 & 4\end{bmatrix}.

Compute \(\det( \mathbf{A})\):

\det( \mathbf{A})=1\cdot 4 - 2\cdot 3 = 4 - 6 = -2.

Swap row 1 and row 2 to get

\mathbf{B}=\begin{bmatrix}3 & 4\\[4pt] 1 & 2\end{bmatrix}.

Compute \(\det( \mathbf{B})\):

\det( \mathbf{B})=3\cdot 2 - 4\cdot 1 = 6 - 4 = 2.

Observation: \(\det( \mathbf{B})=2 = -(-2)= -\det( \mathbf{A})\).

Swapping two rows changed the sign of the determinant.

</div>

<div class="example">

Let

\mathbf{C}=\begin{bmatrix}2 & 1\\[4pt] 0 & 3\end{bmatrix}.

Compute \(\det( \mathbf{C})\):

\det( \mathbf{C})=2\cdot 3 - 1\cdot 0 = 6 - 0 = 6.

Multiply the first row of \( \mathbf{C}\) by \(2\) to get

\mathbf{D}=\begin{bmatrix}4 & 2\\[4pt] 0 & 3\end{bmatrix}.

Compute \(\det( \mathbf{D})\):

\det( \mathbf{D})=4\cdot 3 - 2\cdot 0 = 12 - 0 = 12.

Observation: \(\det( \mathbf{D})=12 = 2\cdot 6 = 2\det( \mathbf{C})\).

Scaling a single row by a factor \(2\) scales the determinant by \(2\).

</div>

The determinant measures the **signed volume** of the parallelepiped spanned by the columns of \( \mathbf{A} \):

- In \( \mathbb{R}^2 \): \( |\det(\mathbf{A})| \) gives the **area** of a parallelogram.

- In \( \mathbb{R}^3 \): \( |\det(\mathbf{A})| \) gives the **volume** of a parallelepiped.

If the determinant is zero, the columns are **linearly dependent** and the volume collapses to zero.

<div class="theorem">

For square matrices $\mathbf{A}$ and $\mathbf{B}$ and $\lambda \in \mathbb{R}$, the following properties hold:

\begin{aligned}

\det(\mathbf{A}\mathbf{B}) &= \det(\mathbf{A})\det(\mathbf{B}), \\

\det(\mathbf{A}^\top) &= \det(\mathbf{A}), \\

\det(\mathbf{A}^{-1}) &= \frac{1}{\det(\mathbf{A})}, \\

\det(\lambda \mathbf{A}) &= \lambda^n \det(\mathbf{A}).

\end{aligned}

</div>

<div class="example">

For $2 \times 2$ matrices,

\mathbf{A} = \begin{bmatrix} a & b \\ c & d \end{bmatrix},

\quad

\mathbf{B} = \begin{bmatrix} e & f \\ g & h \end{bmatrix},

we have

\mathbf{A}\mathbf{B} =

\begin{bmatrix}

ae + bg & af + bh \\

ce + dg & cf + dh

\end{bmatrix}.

This has determinant

\begin{align*}

\det( \mathbf{A}\mathbf{B}) &=

\begin{vmatrix}

ae + bg & af + bh \\

ce + dg & cf + dh

\end{vmatrix} \\

& = (ae + bg)(cf + dh) - (af + bh)(ce + dg)\\

& = aecf + aedh + bgcf + bgdh

- \left( afce + afdg + bhce + bhdg \right)\\

& = ac(ef - fe) + ad(eh - fg) + bc(gf - he) + bd(gh - hg)\\

& = ad(eh - fg) - bc(eh - fg)\\

& = (ad - bc)(eh - fg) \\

&= \det( \mathbf{A})\det(\mathbf{B}).

\end{align*}

</div>

<div class="theorem">

A matrix is **invertible** if and only if it is **full rank**, i.e. \( \text{rank}(\mathbf{A}) = n \).

</div>

<div class="definition">

The **trace** of a square matrix \( \mathbf{A} \in \mathbb{R}^{n \times n} \) is the sum of its diagonal elements:

\text{tr}(\mathbf{A}) = \sum_{i=1}^n a_{ii}.

</div>

<div class="theorem">

For square matrices $\mathbf{A}$ and $\mathbf{B}$ and $\alpha \in \mathbb{R}$, the following properties hold:

\begin{aligned}

\text{tr}(\mathbf{A} + \mathbf{B}) &= \text{tr}(\mathbf{A}) + \text{tr}(\mathbf{B}), \\

\text{tr}(\alpha \mathbf{A}) &= \alpha \, \text{tr}(\mathbf{A}), \\

\text{tr}(\mathbf{A}\mathbf{B}) &= \text{tr}(\mathbf{B}\mathbf{A}), \\

\text{tr}(\mathbf{I}_n) &= n.

\end{aligned}

</div>

The trace is **invariant under cyclic permutations**, meaning \(\text{tr}(\mathbf{A}\mathbf{K}\mathbf{L}) = \text{tr}(\mathbf{K}\mathbf{L}\mathbf{A})\). It is also **independent of basis**, so the trace of a linear map \( \Phi \) is the same in all matrix representations.

<div class="example">

Let

\mathbf{A} = \begin{bmatrix}

3 & -1 & 4 \\

0 & 2 & 5 \\

7 & 1 & -6

\end{bmatrix}.

The trace is

\text{tr}(\mathbf{A}) = 3 + 2 + (-6) = -1.

</div>

<div class="definition">

The **characteristic polynomial** of a square matrix \( \mathbf{A} \) is defined as:

p_ \mathbf{A}(\lambda) = \det(\mathbf{A} - \lambda \mathbf{I}) = c_0 + c_1 \lambda + \cdots + c_{n-1} \lambda^{n-1} + (-1)^n \lambda^n.

</div>

The characteristic polynomial for $\mathbf{A}$ encodes key properties of \( \mathbf{A} \):

c_0 = \det(\mathbf{A}), \quad

c_{n-1} = (-1)^{n-1} \text{tr}(\mathbf{A}).

The roots of this polynomial are the **eigenvalues** of \( \mathbf{A} \), which will be explored in the next section.

<div class="example">

Let

\mathbf{A} = \begin{bmatrix}

3 & -1 & 4 \\

0 & 2 & 5 \\

7 & 1 & -6

\end{bmatrix}.

The characteristic polynomial is

\begin{align*}

p(\lambda) &= \det(A - \lambda I) \\

&= \det\left(

\begin{bmatrix}

3 - \lambda & -1 & 4 \\

0 & 2 - \lambda & 5 \\

7 & 1 & -6 - \lambda

\end{bmatrix} \right) \\

&= (3 - \lambda)

\begin{vmatrix}

2 - \lambda & 5 \\

1 & -6 - \lambda

\end{vmatrix}

+ 7

\begin{vmatrix}

-1 & 4 \\

2 - \lambda & 5

\end{vmatrix}\\

&= (3 - \lambda)(\lambda^2 + 4\lambda - 17)

+ 7(4\lambda - 13) \\

&= -\lambda^3 - \lambda^2 + 57\lambda - 142.

\end{align*}

</div>

<div class="exercise">

Find $\det\left(\begin{bmatrix} 2 & 3 \\ 4 & 5 \end{bmatrix} \right)$ using the formula.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Find $\det\left(\begin{bmatrix} 2 & 3 &4\\ 5 & 6 &7\\ 8 & 9 & 1 \end{bmatrix} \right)$ using the formula.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Find $\det\left(\begin{bmatrix} 2 & 3 &4\\ 5 & 6 &7\\ 8 & 9 & 1 \end{bmatrix} \right)$ using the Laplace method.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Prove that if $A$ is a square matrix with a row or column of 0's, then $\det(A) = 0$.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Find $\det\left(\begin{bmatrix} 0& 2 & 3 &4\\ 5 & 6 &7&0 \\1& 8 & 9 & 1\\0&2&3&0 \end{bmatrix} \right)$ using the Laplace method.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Prove that if $A$ is a square matrix with 2 identical rows or columns, then $\det(A) = 0$.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Find $\det\left(\begin{bmatrix} 2 & 0 &0\\ 5 & 6 &0\\ 8 & 9 & 1 \end{bmatrix} \right)$.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Find $\text{tr}\left(\begin{bmatrix} 2 & 0 &0\\ 5 & 6 &0\\ 8 & 9 & 1 \end{bmatrix} \right)$.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Find the characteristic polynomial for $\begin{bmatrix} 2 & 0 &0\\ 5 & 6 &0\\ 8 & 9 & 1 \end{bmatrix}$.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Prove the following properties of the trace:

* $\text{tr}(A+B) = \text{tr}(A) + \text{tr}(B)$

* $\text{tr}(\alpha A) = \alpha \text{tr}(A)$

* $\text{tr}(I_n) = n$

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

<div style="text-align: right;">

[Solution]( )

</div>

</div>

Eigenvalues and eigenvectors provide a way to characterize a matrix \(\mathbf{A} \in \mathbb{R}^{n \times n}\) and its associated linear mapping.

<div class="definition">

**Eigenvalue & Eigenvector**:

A scalar \(\lambda \in \mathbb{R}\) is an eigenvalue of \(\mathbf{A}\) and a nonzero vector \(\mathbf{x} \in \mathbb{R}^n\) is a corresponding eigenvector if

\[

\mathbf{A}\mathbf{x} = \lambda \mathbf{x}.

\]

This is called the **eigenvalue equation**.

</div>

<div class="definition">

The span of the set of eigenvectors associated with \(\lambda\) spans a subspace \(E_\lambda \subset \mathbb{R}^n\) known as the **eigenspace**.

</div>

<div class="definition">

The set of all eigenvalues of \(\mathbf{A}\) is called the **eigenspectrum**.

</div>

<div class="definition">

Number of times \(\lambda\) appears as a root of the characteristic polynomial \(p_ \mathbf{A}(\lambda) = \det(\mathbf{A} - \lambda I)\) is called the **algebraic multiplicity** of the eigenvalue.

</div>

<div class="definition">

The dimension of the eigenspace associated with \(\lambda\) is called the **geometric multiplicity**.

</div>

<div class="example">

Find the eigenspace(s) of the matrix

\mathbf{A} = \begin{pmatrix}

2 & 1\\

0 & 2

\end{pmatrix}.

**Step 1: Find the eigenvalues**

Compute the characteristic polynomial:

\det(\mathbf{A} - \lambda \mathbf{I})

= \det \begin{pmatrix}

2-\lambda & 1\\

0 & 2-\lambda

\end{pmatrix}

= (2-\lambda)^2.

So the only eigenvalue is

\lambda = 2

\quad \text{(with algebraic multiplicity 2).}

**Step 2: Find the eigenspace for \(\lambda = 2\)**

Form the matrix:

\mathbf{A} - 2\mathbf{I} =

\begin{pmatrix}

2-2 & 1\\

0 & 2-2

\end{pmatrix}

\begin{pmatrix}

0 & 1\\

0 & 0

\end{pmatrix}.

Solve:

(\mathbf{A} - 2\mathbf{I})\mathbf{x} = 0.

This gives:

x_2 = 0.

So \(x_1\) is free, and the solution vectors have the form:

\mathbf{x} = \begin{pmatrix}x_1\\ 0\end{pmatrix}

= x_1 \begin{pmatrix}1\\ 0\end{pmatrix}.

**Step 3: Write the eigenspace**

The eigenspace is:

E_2 = \text{span}\left\{\begin{pmatrix}1\\ 0\end{pmatrix}\right\}.

So the eigenspace is **one-dimensional**, even though the eigenvalue has algebraic multiplicity 2.

</div>

Eigenvalues and eigenvectors have several key properties:

- \(\mathbf{A}\) and \(\mathbf{A}^\top\) have the same eigenvalues, not necessarily the same eigenvectors.

- Similar matrices have identical eigenvalues. Matrices \( \mathbf{A} \) and \( \mathbf{D} \) are similar if \(\mathbf{A} = \mathbf{P} \mathbf{D} \mathbf{P}^{-1} \) for some matrix \( \mathbf{P} \).

- Symmetric, positive definite matrices have real, positive eigenvalues.

- A matrix with \(n\) distinct eigenvalues has linearly independent eigenvectors forming a basis of \(\mathbb{R}^n\).

- **Defective matrices**: Have fewer than \(n\) linearly independent eigenvectors.

<div class="example">

Consider the matrix

\mathbf{A} =

\begin{bmatrix}

2 & 1 \\

0 & 3

\end{bmatrix}

and the invertible matrix

\mathbf{P} =

\begin{bmatrix}

1 & 1 \\

0 & 1

\end{bmatrix}.

We can compute a matrix \(\mathbf{B}\) that is **similar** to \(\mathbf{A}\) using the formula:

\mathbf{B} = \mathbf{P}^{-1} \mathbf{A} \mathbf{P}

First, find \(\mathbf{P}^{-1}\):

\mathbf{P}^{-1} =

\begin{bmatrix}

1 & -1 \\

0 & 1

\end{bmatrix}.

Then,

\mathbf{B} =

\begin{bmatrix} 1 & -1 \\ 0 & 1 \end{bmatrix}

\begin{bmatrix} 2 & 1 \\ 0 & 3 \end{bmatrix}

\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}

\begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}.

Thus, \(\mathbf{A}\) and \(\mathbf{B}\) are **similar matrices**, since there exists an invertible matrix \(\mathbf{P}\) such that \(\mathbf{B} = \mathbf{P}^{-1} \mathbf{A} \mathbf{P}\). One can verify that matrices $\mathbf{A}$ and $\mathbf{B}$ have the same eigenvalues ($\lambda = 2$ and 3).

</div>

<div class="theorem">

**Spectral Theorem:** If \(\mathbf{A}\) is symmetric, there exists an orthonormal basis of $\mathbb{R}^n$ consisting of eigenvectors of $\mathbf{A}$ (and all eigenvalues are real). Furthermore, $\mathbf{A}$ can be decomposed as

\[

\mathbf{A} = \mathbf{P}\mathbf{D}\mathbf{P}^\top

\]

where \(\mathbf{P}\) contains eigenvectors and \(\mathbf{D}\) is diagonal with eigenvalues.

</div>

<div class="example">

We illustrate the decomposition of

\mathbf{A} =

\begin{pmatrix}

2 & 1\\

1 & 2

\end{pmatrix}.

First, we need to find the eigenvalues/ eigenvectors for $\mathbf{A}$.

\det(\mathbf{A} - \lambda \mathbf{I}) =

\det\begin{pmatrix}

2-\lambda & 1\\

1 & 2-\lambda

\end{pmatrix}

= (2-\lambda)^2 - 1.

So the eigenvalues are:

\lambda_1 = 3, \qquad \lambda_2 = 1.

For \(\lambda_1 = 3\),

(\mathbf{A} - 3\mathbf{I})\mathbf{v}_1

\begin{pmatrix}

-1 & 1\\

1 & -1

\end{pmatrix}

\begin{pmatrix}x\\y\end{pmatrix}

= 0

\Rightarrow x = y.

Choose:

\mathbf{v}_1 = \frac{1}{\sqrt{2}}

\begin{pmatrix}1\\1\end{pmatrix}.

For \(\lambda_2 = 1\),

(\mathbf{A}-\mathbf{I})\mathbf{v}_2

\begin{pmatrix}

1 & 1\\

1 & 1

\end{pmatrix}

\begin{pmatrix}x\\y\end{pmatrix}

= 0

\Rightarrow x = -y.

Choose:

\mathbf{v}_2 = \frac{1}{\sqrt{2}}

\begin{pmatrix}1\\-1\end{pmatrix}.

To form $\mathbf{P}$ and $\mathbf{D}$, take the normalized eigenvectors to form the columns of $\mathbf{P}$ and the eigenvalues to form the main diagonal of $\mathbf{D}$,

\mathbf{P}

\begin{pmatrix}

\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\\[4pt]

\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}}

\end{pmatrix},

\qquad

\mathbf{D}

\begin{pmatrix}

3 & 0\\

0 & 1

\end{pmatrix}.

To check,

\mathbf{PDP}^\top

\begin{pmatrix}

\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\\

\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}}

\end{pmatrix}

\begin{pmatrix}

3 & 0\\

0 & 1

\end{pmatrix}

\begin{pmatrix}

\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\\

\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}}

\end{pmatrix}

\begin{pmatrix}

2 & 1\\

1 & 2

\end{pmatrix}

= \mathbf{A}.

</div>

Eigenvalues and eigenvectors are related to the determinant and trace of a matrix. For example, \[\det(\mathbf{A}) = \prod_{i=1}^n \lambda_i.\] Furthermore, \[\text{tr}(\mathbf{A}) = \sum_{i=1}^n \lambda_i.\] Geometrically, eigenvectors are directions stretched by \(\lambda_i\); determinant gives volume scaling, trace gives scaling of perimeter.

<div class="example">

Let $\mathbf{A} = \begin{pmatrix}4 & 2 \\ 1 & 3\end{pmatrix}$. Then

- Eigenvalues: $\lambda_1 = 2, \;\;\; \lambda_2 = 5$

- Eigenspaces: $E_2 = \text{span}\{[1, -1]^\top\}, \;\;\; E_5 = \text{span}\{[2,1]^\top\}$

- Trace is $\text{sum of diagonals of the matrix} = 4 + 3 = 7 = 5 + 2 = \text{sum of the eigenvalues}$

- Determinant is $\text{by the formula} = 4(3) - 2(1) = 10 = 5 \times 2 = \text{product of the eigenvalues}$.

</div>

**Google's PageRank Algorithm** uses the eigenvector of the maximal eigenvalue (\(\lambda = 1\)) of the web connectivity matrix to rank web pages.

<div class="exercise">

Find the characteristic equation, eigenvalues, eigenspace, determinant, and trace corresponding to \[\mathbf{A} = \begin{bmatrix} 1 & 4\\3 & 2 \end{bmatrix}.\]

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Find the characteristic equation, eigenvalues, eigenspace, determinant, and trace corresponding to \[\mathbf{A} = \begin{bmatrix} 2 & 2\\1 & 3 \end{bmatrix}.\]

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Find the characteristic equation, eigenvalues, eigenspace, determinant, and trace corresponding to \[\mathbf{A} = \begin{bmatrix}1&0&0\\0&1&2\\0&0&0\end{bmatrix}.\]

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Find the characteristic equation, eigenvalues, eigenspace, determinant, and trace corresponding to \[\mathbf{A} = \begin{bmatrix}2&0&4\\0&3&0\\0&1&2\end{bmatrix}.\]

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Given a matrix $\mathbf{A} \in \mathbb{R}^{m \times n}$, we can always obtain a symmetric positive semidefinite matrix $\mathbf{S} \in \mathbb{R}^{n \times n}$ by defining $\mathbf{S} = \mathbf{A}^T\mathbf{A}$. Prove this statement for $2 \times 2$, $3 \times 2$ and $2 \times 3$ matrices.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

Explain why geometrically, an eigenvector is a vector whose direction is unchanged by multiplying by matrix $\mathbf{A}$.

<div style="text-align: right;">

[Solution]( )

</div>

</div>

<div class="exercise">

<div style="text-align: right;">

[Solution]( )

</div>

</div>

The **Cholesky decomposition** is a square-root-like factorization for symmetric, positive definite matrices. It generalizes the concept of a square root from numbers to matrices.

<div class="theorem">

**Cholesky Decomposition:** A symmetric, positive definite matrix \(\mathbf{A} \in \mathbb{R}^{n \times n}\) can be factorized as:

\mathbf{A} = L L^\top,

where \(L\) is a **lower-triangular matrix** with positive diagonal entries. The matrix \(L\) is called the **Cholesky factor** of \(\mathbf{A}\) and is unique.

</div>

<div class="example">

For

\mathbf{A} = \begin{bmatrix}

a_{11} & a_{21} & a_{31} \\

a_{21} & a_{22} & a_{32} \\

a_{31} & a_{32} & a_{33}

\end{bmatrix},

\quad

L = \begin{bmatrix}

l_{11} & 0 & 0 \\

l_{21} & l_{22} & 0 \\

l_{31} & l_{32} & l_{33}

\end{bmatrix},

the components of \(L\) are computed as:

\begin{aligned}