Cauchy-Bunyakovsky-Schwarz inequality

Branko Ćurgus

Abstract. On this page we prove the Cauchy-Bunyakovsky-Schwarz Inequality for nonnegative hermitian sesquilinear forms on vector spaces over an arbitrary scalar subfield of $\mathbb C$. For such sesquilinear forms, we also prove a characterization of the best approximation in terms of orthogonality. All three of these statements are commonly proved for inner products, that is for positive definite hermitian sesquilinear forms.

By $\mathbb C$ we denote the field of all complex numbers. For $\alpha \in {\mathbb C}$, the complex conjugate of $\alpha$ is denoted by $\overline{\alpha}$.

Definition. A subfield $\mathbb F$ of $\mathbb C$ is called a scalar field if it is closed with respect to complex conjugation. That is, $\mathbb F$ is a scalar field if it has the following properties:

Definition. Let ${\mathcal V}$ be a vector space over a scalar field $\mathbb F$. A sesquilinear form on ${\mathcal V}$ is a mapping $[\cdot,\cdot] : {\mathcal V}\times {\mathcal V} \to {\mathbb F}$ such that

Definition. Let ${\mathcal V}$ be a vector space over a scalar field $\mathbb F$. A mapping $[\cdot,\cdot] : {\mathcal V}\times {\mathcal V} \to {\mathbb F}$ is said to be hermitian sesquilinear form on $\mathcal V$ if it has the following properties

Definition. Let ${\mathcal V}$ be a vector space over a scalar field $\mathbb F$. A hermitian sesquilinear form $\langle\cdot,\cdot\rangle : {\mathcal V}\times {\mathcal V} \to {\mathbb F}$ on $\mathcal V$ is said to be a nonnegative hermitian sesquilinear form on $\mathcal V$ if the following condition is satisfied

High School Theorem. Let $a, b, c$ be real numbers. Assume $a \geq 0$. Then the following implication holds: \begin{equation} \label{eq-impl} \forall \, x \in {\mathbb Q} \quad a x^2 + b x + c \geq 0 \qquad \Rightarrow \qquad b^2-4ac \leq 0. \end{equation}

Proof.
Let $a, b, c$ be real numbers. We consider two cases: $a \gt 0$ and $a = 0$. Assume that $a \gt 0$ and \begin{equation} \label{eq-qQ} \forall \, x \in {\mathbb Q} \quad a x^2 + b x + c \geq 0. \end{equation} Since the function $x \mapsto a x^2 + b x + c, \ x \in {\mathbb R}$, is continuous, \eqref{eq-qQ} implies \begin{equation} \label{eq-qR} \forall \, x \in {\mathbb R} \quad a x^2 + b x + c \geq 0. \end{equation} Since $a \gt 0$ and $b \in {\mathbb R}$ we have $x_0 = -b/(2a) \in {\mathbb R}$. Using this $x_0$ in \eqref{eq-qR} we get: \begin{align*} 0 & \leq a \Bigl(-\frac{b}{2a} \Bigr)^2 - b \frac{b}{2a} + c \\ & = \frac{b^2}{4a} - \frac{b^2}{2a} + c \\ & = - \frac{b^2}{4a} + c \\ & = - \frac{b^2-4ac}{4a}. \end{align*} Since $- 4 a \lt 0$ the inequality \[ - \frac{b^2-4ac}{4a} \geq 0 \] implies $b^2 - 4ac \leq 0$. Thus, implication \eqref{eq-impl} is proved for $a \gt 0$.
Now assume that $a = 0$. In this case implication \eqref{eq-impl} reads \begin{equation} \label{eq-impls} \forall \, x \in {\mathbb Q} \quad b x + c \geq 0 \qquad \Rightarrow \qquad b = 0. \end{equation} Again, since the function $x \mapsto b x + c, \ x \in {\mathbb R}$, is continuous we have \[ \forall \, x \in {\mathbb Q} \quad b x + c \geq 0 \qquad \Leftrightarrow \qquad \forall \, x \in {\mathbb R} \quad b x + c \geq 0. \] Therefore \eqref{eq-impls} is equivalent to \begin{equation} \label{eq-implsR} \forall \, x \in {\mathbb R} \quad b x + c \geq 0 \qquad \Rightarrow \qquad b = 0. \end{equation} It is easier to prove the contrapositive of \eqref{eq-implsR}: \begin{equation} \label{eq-implsc} b \neq 0 \qquad \Rightarrow \qquad \exists \, x \in {\mathbb R} \quad b x + c \lt 0. \end{equation} To prove \eqref{eq-implsc}, assume $b \neq 0$ and set \[ x = -\frac{c+1}{b} \in {\mathbb R}. \] Then \[ b x + c = - b \frac{c+1}{b} + c = -c-1+c = -1 \lt 0. \] This proves \eqref{eq-implsc}. Thus \eqref{eq-implsR} is proved, and thus \eqref{eq-impls} is proved. This completes the proof.
$\blacksquare$

Cauchy-Bunyakovsky-Schwarz Inequality. Let ${\mathcal V}$ be a vector space over a scalar field $\mathbb F$ and let $\langle\cdot,\cdot\rangle$ be a nonnegative hermitian sesquilinear form on $\mathcal V$. Then \begin{equation} \label{eq-CSBi} \forall \, u, v \in {\mathcal V} \quad |\langle u, v \rangle |^2 \leq \langle u,u \rangle \langle v,v \rangle. \end{equation}

Proof.
Let $u, v \in {\mathcal V}$ be arbitrary. Since $\langle\cdot,\cdot\rangle$ is nonnegative we have \begin{equation} \label{eq-CSBst1} \forall \, t \in {\mathbb Q} \qquad \bigl\langle u + t \langle u,v\rangle v, u + t \langle u,v\rangle v \bigr\rangle \geq 0. \end{equation} Since $\langle\cdot,\cdot\rangle$ is a sesquilinear hermitian form on $\mathcal V$, \eqref{eq-CSBst1} is equivalent to \begin{equation} \label{eq-CSBst2} \forall \, t \in {\mathbb Q} \qquad \langle u, u \rangle + 2 t |\langle u,v\rangle|^2 + t^2 |\langle u,v\rangle|^2 \langle v,v\rangle \geq 0. \end{equation} As $\langle v,v \rangle \geq 0$, the High School Theorem applies and \eqref{eq-CSBst2} implies \begin{equation} \label{eq-CSBst3} 4 |\langle u,v\rangle|^4 - 4 |\langle u,v\rangle|^2 \langle u, u \rangle \langle v,v\rangle \leq 0. \end{equation} Again, since $\langle u, u \rangle \geq 0$ and $\langle v,v\rangle \geq 0$, \eqref{eq-CSBst3} is equivalent to \begin{equation*} |\langle u,v\rangle|^2 \leq \langle u, u \rangle \langle v,v\rangle. \end{equation*} Since $u, v \in {\mathcal V}$ were arbitrary, \eqref{eq-CSBi} is proved.
$\blacksquare$

Cauchy-Bunyakovsky-Schwarz Equality. Let ${\mathcal V}$ be a vector space over a scalar field $\mathbb F$ and let $\langle\cdot,\cdot\rangle$ be a nonnegative hermitian sesquilinear form on $\mathcal V$. Let $u, v \in {\mathcal V}$. Then \begin{equation} \label{eq-CSBeq} | \langle u, v \rangle |^2 = \langle u,u \rangle \, \langle v,v \rangle \end{equation} if and only if there exists $\alpha, \beta \in {\mathbb F}$ not both $0$ such that \begin{equation} \label{eq-CSBeq2} \langle \alpha u + \beta v, \alpha u + \beta v \rangle = 0. \end{equation}

Proof.
We prove the "if" part first. Assume that $u, v \in \mathcal V$ and $\alpha, \beta \in \mathbb F$ are such that $|\alpha|^2 + |\beta|^2 > 0$ and \eqref{eq-CSBeq2} holds. We need to prove \eqref{eq-CSBeq}. Since $|\alpha|^2 + |\beta|^2 > 0$, we have $\alpha \neq 0$ or $\beta \neq 0$. We consider the case $\alpha \neq 0$. The case $\beta \neq 0$ is similar. Set $w = \alpha u + \beta v$. Then $\langle w,w \rangle = 0$ and $u = \gamma v + \delta w$ where $\gamma = -\beta/\alpha$ and $\delta = 1/\alpha$. Notice that the Cauchy-Bunyakovsky-Schwarz inequality and $\langle w, w \rangle = 0$ imply that $\langle w, x \rangle = 0$ for all $x \in \mathcal V$. Using this, we calculate \[ | \langle u, v \rangle | = | \langle \gamma v + \delta w , v \rangle | = | \gamma \langle v, v \rangle + \delta \langle w , v \rangle | = | \gamma \langle v, v \rangle | = | \gamma | \langle v, v \rangle \] and \[ \langle u, u \rangle = \langle \gamma v + \delta w ,\gamma v + \delta w \rangle = \langle \gamma v, \gamma v \rangle = | \gamma |^2 \langle v, v \rangle. \] Thus, \[ | \langle u, v \rangle |^2 = | \gamma |^2 \langle v, v \rangle^2 = \langle u, u \rangle \langle v, v \rangle. \] This completes the proof of the "if" part.
To prove the converse assume $|\langle{u},{v} \rangle |^2 = \langle{u},{u} \rangle \langle{v},{v} \rangle$. If $\langle v, v \rangle = 0$, then with $\alpha = 0$ and $\beta = 1$ we have \[ \langle{\alpha u + \beta v},{\alpha u + \beta v} \rangle = \langle v, v \rangle = 0. \] If $\langle v, v \rangle \neq 0$, then with $\alpha = \langle v, v \rangle$ and $\beta = - \langle u, v \rangle$ we have \[ \langle {\alpha u + \beta v},{\alpha u + \beta v} \rangle = \langle v, v \rangle\bigl( \langle v, v \rangle\langle u, u \rangle - |\langle u, v \rangle|^2 - |\langle u, v \rangle|^2 + |\langle u, v \rangle|^2\bigr) = 0. \] This completes the proof of the "only if" part.
$\blacksquare$

Best Approximation-Orthogonality Theorem. Let ${\mathcal V}$ be a vector space over a scalar field $\mathbb F$ and let $\langle\cdot,\cdot\rangle$ be a nonnegative hermitian sesquilinear form on $\mathcal V$. Let ${\mathcal U}$ be a subspace of ${\mathcal V}$. Let $v \in {\mathcal V}$ and $u_0 \in {\mathcal U}$ be arbitrary. Then \begin{equation} \label{eq-mind} \forall \, u \in {\mathcal U} \qquad \langle v - u_0, v -u_0 \rangle \leq \langle v - u, v -u \rangle \end{equation} if and only if \begin{equation} \label{eq-orth} \forall \, u \in {\mathcal U} \qquad \langle v - u_0, u \rangle = 0. \end{equation}

Proof. First we prove the "only if" part.
Assume \eqref{eq-mind}. Let $u \in {\mathcal U}$ be arbitrary. Set $\alpha = \langle v - u_0, u \rangle$. Clearly $\alpha \in {\mathbb F}$. Let $t \in {\mathbb Q} \subseteq {\mathbb F}$ be arbitrary. Since $u_0- t \alpha u \in {\mathcal U}$, \eqref{eq-mind} implies \begin{equation} \label{eq-mind1} \forall \, t \in {\mathbb Q} \qquad \langle v - u_0, v -u_0 \rangle \leq \langle v - u_0 + t \alpha u, v - u_0 + t \alpha u \rangle. \end{equation} Now recall that $\alpha = \langle v - u_0, u \rangle$ and expand the right-hand side of \eqref{eq-mind1}: \begin{align*} \langle v - u_0 + t \alpha u, v - u_0 + t \alpha u \rangle & = \langle v - u_0, v - u_0 \rangle + \langle v - u_0, t \alpha u \rangle + \langle t \alpha u, v - u_0 \rangle + \langle t \alpha u,t \alpha u \rangle \\ & = \langle v - u_0, v - u_0 \rangle + t \overline{\alpha} \langle v - u_0, u \rangle + t \alpha \langle u, v - u_0 \rangle + t^2 |\alpha|^2 \langle u, u \rangle \\ & = \langle v - u_0, v - u_0 \rangle + 2 t |\alpha |^2 + t^2 |\alpha|^2 \langle u, u \rangle. \end{align*} Thus \eqref{eq-mind1} is equivalent to \begin{equation} \label{eq-mind2} \forall \, t \in {\mathbb Q} \qquad 0 \leq 2 t |\alpha|^2 + t^2 |\alpha|^2 \langle u, u \rangle. \end{equation} By the High School Theorem, \eqref{eq-mind2} implies \[ 4 |\alpha |^4 - 4 |\alpha|^2 \langle u, u \rangle \, 0 = 4 |\alpha |^4 \leq 0. \] Consequently $\alpha = \langle v - u_0, u \rangle = 0$. Since $u \in {\mathcal U}$ was arbitrary, \eqref{eq-orth} is proved.
Alternatively one can prove the contrapositive of the "only if" part.
Assume the negation of \eqref{eq-orth}. That is, assume that there exists $w \in \mathcal U$ such that $\langle v-u_0,w\rangle \neq 0$. By the Cauchy-Bunyakovsky-Schwarz inequality $\langle w,w \rangle \gt 0$. Set \[ \alpha = \frac{\langle v-u_0,w\rangle}{\langle w,w \rangle} \quad \text{and} \quad u = u_0 + \alpha \, w. \] Then $u - u_0 = \alpha \, w$, \[ \bigl\langle (u - u_0), (u - u_0) \bigr\rangle = \bigl\langle \alpha \, w, \alpha \, w \bigr\rangle = |\alpha|^2 \langle w,w \rangle \gt 0 \] and \begin{align*} \langle v- u, u - u_0 \rangle & = \left\langle v - u_0 - \alpha \, w, \alpha\, w \right\rangle \\ & = \overline{\alpha} \langle v - u_0, w \rangle - | \alpha |^2 \, \langle w,w \rangle \\ & = 0. \end{align*} Therefore \begin{align*} \langle v- u_0, v - u_0 \rangle & = \bigl\langle (v - u) + (u - u_0), (v - u) + (u - u_0) \bigr\rangle \\ & = \bigl\langle (v - u) , (v - u)\bigr\rangle + \bigl\langle(u - u_0), (u - u_0) \bigr\rangle \\ & \gt \bigl\langle (v - u) , (v - u)\bigr\rangle \end{align*} This proves the negation of \eqref{eq-ind}.
A proof of the converse is left as an exercise. $\blacksquare$

Definition. Let ${\mathcal V}$ be a vector space over a scalar field $\mathbb F$. A nonnegative hermitian sesquilinear form $\langle\cdot,\cdot\rangle : {\mathcal V}\times {\mathcal V} \to {\mathbb F}$ on $\mathcal V$ is said to be a positive definite hermitian sesquilinear form on $\mathcal V$ if the following condition is satisfied

A synonym for "positive definite hermitian sesquilinear form" is inner product.