Xrfgtjtk

How do you show that the column space of a matrix A is orthogonal to its nullspace?

What you have written is only correct if you are referring to the left nullspace (it is more standard to use the term "nullspace" to refer to the right nullspace).

The row space (not the column space) is orthogonal to the right null space.

Showing that row space is orthogonal to the right null space follows directly from the definition of right null space.

Let the matrix $A in mathbb{R}^{m times n}$. The right null space is defined as $$mathcal{N}(A) = {z in mathbb{R}^{n times 1} : Az = 0 }$$

Let $
A =
left[ {begin{array}{c}
a_1^T \
a_2^T \
ldots \
ldots \
a_m^T
end{array} } right]$. The row space of $A$ is defined as
$$mathcal{R}(A) = {y in mathbb{R}^{n times 1}: y = sum_{i=1}^m a_i x_i text{ , where }x_i in mathbb{R} text{ and }a_i in mathbb{R}^{n times 1} }$$
Now from the definition of right null space we have $a_i^T z = 0$.

So if we take a $y in mathcal{R}(A)$, then $y = displaystyle sum_{k=1}^m a_i x_i text{ , where }x_i in mathbb{R}$. Hence, $$y^Tz = (sum_{k=1}^m a_i x_i)^T z = (sum_{k=1}^m x_i a_i^T) z = sum_{k=1}^m x_i (a_i^T z) = 0$$

This proves that row space is orthogonal to the right null space. A similar analysis proves that column space of $A$ is orthogonal to the left null space of $A$.

Note: The left null space is defined as ${z in mathbb{R}^{m times 1}: z^TA = 0}$

The OP's question would have been well-phrased had he specified that the matrix $A$ is symmetric i.e. $A=A^top$, in which case ${rm colspan}(A)={rm rowspan}(A^top)={rm rowspan}(A)$. Now, consider the definition of ${rm null}(A)$ as the space of all vectors $mathbf{v}$ such that $Amathbf{v}=mathbf{0}$. Letting $mathbf{a}_1,ldots,mathbf{a}_n$ be the rows (and columns) of $A$, matrix multiplication tells us that $mathbf{a}_icdotmathbf{v}=0$ for each $i=1,ldots,dim(A)$. Thus any vector $mathbf{v}in{rm null}(A)$ is orthogonal to ${rm colspan}(A)$. It follows that ${rm null}(A)perp{rm colspan}(A)$.$square$

You can't show this: it is false. Take

$$begin{bmatrix} 0 & 1 \ 0 & 0 end{bmatrix};$$

both its column space and null space are the subspace ${(x, 0) mid x in mathbb{R}}$ (the $x$-axis, so to speak). I think you need self-adjointness (symmetry in the real case) to get this.

EDIT: Sivaram anticipated me.

21

What you have written is only correct if you are referring to the left nullspace (it is more standard to use the term "nullspace" to refer to the right nullspace).

The row space (not the column space) is orthogonal to the right null space.

Showing that row space is orthogonal to the right null space follows directly from the definition of right null space.

Let the matrix A∈Rm×n. The right null space is defined as
N(A)={z∈Rn×1:Az=0}
Let A=⎡⎣⎢⎢⎢⎢⎢⎢aT1aT2……aTm⎤⎦⎥⎥⎥⎥⎥⎥. The row space of A is defined as
R(A)={y∈Rn×1:y=∑i=1maixi , where xi∈R and ai∈Rn×1}
Now from the definition of right null space we have aTiz=0.

So if we take a y∈R(A), then y=∑k=1maixi , where xi∈R. Hence,
yTz=(∑k=1maixi)Tz=(∑k=1mxiaTi)z=∑k=1mxi(aTiz)=0
This proves that row space is orthogonal to the right null space. A similar analysis proves that column space of A is orthogonal to the left null space of A.

Note: The left null space is defined as {z∈Rm×1:zTA=0}