Orthogonal Complement
Definition 1. Let $(V, \langle \cdot, \cdot \rangle)$ be an inner product space, and let $\emptyset \neq S \subseteq V$. We define $S^{\perp}$ to be $S^{\perp} = \{x \in V \,|\, \langle x, y\rangle = 0, \forall y \in S\}$. The set $S^{\perp}$ is called the orthogonal complement of $S$.
$S$의 벡터들에 직교하는 벡터들을 모두 모아놓은 집합을 $S$의 orthogonal complement, 직교여공간이라고 부른다. 자명하게 $S^{\perp} \leq V$이고, $\{\mathbf{0}\}^{\perp} = V, V^{\perp} = \{\mathbf{0}\}$임을 알 수 있다.
Theoerm 1
Theorem 1. Let $(V, \langle \cdot, \cdot \rangle)$ be an inner product space, and let $W \leq V$ be finite-dimensional. Let $y \in V$. Then $! \exists u \in W, z \in W^{\perp}$ such that $y = u + z$. Furthermore, if $\{v_1, ..., v_k\}$ is an orthonoraml basis for $W$, then $$u = \sum_{i=1}^k \langle y, v_i \rangle v_i.$$
Proof. Let $u = \sum_{i=1}^k \langle y, v_i\rangle v_i$. Then $\langle y, u \rangle = \sum_{i=1}^k \overline{\langle y, v_i \rangle} \langle y, v_i\rangle = \sum_{i=1}^k |\langle y, v_i \rangle|^2$.
Note that $||u||^2 = \sum_{i=1}^k \langle y, v_i \rangle \overline{\langle y, v_i \rangle} ||v_i||^2 = \sum_{i=1}^k |\langle y, v_i \rangle|^2$. Thus we have $\langle y, u \rangle = ||u||^2$, and $\langle y - u, u \rangle = 0$. If we denote $z = y - u$, then $z \in W^{\perp}$. Thus $y = u + z$.
Suppose that $y = u + z = u' + z'$ where $u, u' \in W$ and $z, z' \in W^{\perp}$. Then we denote $v = u - u' = z' - z$. Note that $v \in W \cap W^{\perp}$. So $||v||^2 = \langle v, v \rangle = 0 \Longrightarrow v = \mathbf{0}$. Thus $u = u'$ and $z = z'$. $\blacksquare$
Corollary
Corollary. In the notation of Theorem 1, $u$ is the unique vector in $W$ that is closest to $y$; that is , $\forall x \in W$, $||y - x|| \geq ||y - u||$, and this inequality is an equality $\iff$ $x = u$.
Proof. Note that $$||y - x||^2 = \langle y-x, y-x \rangle = ||y||^2 - \langle y, x\rangle - \langle x, y\rangle + ||x||^2 \\ =||y||^2 - \langle u, x \rangle - \langle x, u \rangle + ||x||^2$$ and $$||y - u||^2 = ||y||^2 - \langle y, u\rangle - \langle u, y\rangle + ||u||^2 \\ = ||y||^2 - ||u||^2 - ||u||^2 + ||u||^2 = ||y||^2 - ||u||^2.$$ Then $$||y -x||^2 - ||y - u||^2 = ||u||^2 - \langle u, x\rangle - \langle x, u \rangle + ||x||^2 \\ = \langle u - x, u- x\rangle = ||y - x||^2 \geq 0 \\ \Longrightarrow ||y - x|| \geq ||y - u||.$$ If $x = u$, then clearly $||y - x|| = ||y - u||$ and
if $||y - x|| = ||y - u||$, then clearly $u = x$ because $||x|| = 0 \iff x = \mathbf{0}$. $\blacksquare$
간단히 말해서 $V = W \bigoplus W^{\perp}$로 정리할 수 있다. corollary는 꽤 중요한데, $V$의 벡터 $y$를 $V$의 부분공간 $W$로 적절히 근사시키는 상황을 고려하자. 이때 $y$와 가장 비슷하면서, 즉 차이가 작으면서 $W$에 속하는 벡터를 구하는 것이 자연스럽다. 위 정리는 그러한 벡터가 $y$를 $W$와 $W^{\perp}$로 분해했을 때 나오는 벡터 $u$임을 말해준다. 이때 $u$를 the orthogonal projection of $y$ on $W$, 즉 $W$로의 $y$의 정사영이라고 부른다.
Remark
Remark. Let $W \leq V$ be finite-dimensional. Then it is immediate that $V = W \bigoplus W^{\perp}$ from Theorem 1 and Theorem 1.
If $\beta = \{v_1, ..., v_k\}$ is an orthonormal basis for $W$, then $\beta$ can be extended to a basis $\{v_1, ..., v_k, w_{k+1}, ..., w_n\}$ for $V$. By applying the Gram-Schmidt Process to this basis, we have an orthonormal basis $\gamma = \{v_1, ..., v_k, v_{k+1}, ..., v_n\}$ for $V$. Thus an orthonormal basis $\beta$ for $W$ can be extended to an orthonormal basis $\gamma$ for $V$.
Furthermore, since $\dim(W^{\perp}) = n - k$, $\{v_{k+1}, ..., v_n\}$ is an orthonormal basis for $W^{\perp}$.
Theorem 2
Theorem 2. Let $S, S_0 \subseteq V$, and let $W \leq V$ be finite-dimensional where $V$ is an inner product space. Then
(a) $S_0 \subseteq S \Longrightarrow S^{\perp} \subseteq S_0^{\perp}$.
(b) $S \subseteq (S^{\perp})^{\perp}$
(c) $W = (W^{\perp})^{\perp}$.
Proof. (a) Let $x \in S^{\perp}$. Then $\langle x, y \rangle = 0, \forall y \in S$, so $\langle x, z \rangle = 0, \forall z \in S_0$. Thus $x \in S_0^{\perp}$.
(b) Let $x \in S$. Then $\langle x, y \rangle = 0, \forall y \in S^{\perp}$. Thus $x \in (S^{\perp})^{\perp}$.
(c) Let $y \in (W^{\perp})^{\perp}$. Then $\langle x, y \rangle = 0, \forall x \in W^{\perp}$. Note that $\langle x, z \rangle = 0, \forall z \in W$. Then $\exists z_0 \in W$ such that $z_0 = y$. Thus $y \in W$. $\blacksquare$