Four fundamental subspaces

First, keep the input and output spaces distinct.

Original vector · input

x=\begin{pmatrix}1\\3\end{pmatrix}\in\mathbb R^2

→

Linear transformation

A=\begin{pmatrix}1&2\\2&4\end{pmatrix}

2 × 2

→

Transformed vector · output

b=\begin{pmatrix}7\\14\end{pmatrix}\in\mathbb R^2

An m × n matrix accepts n-component vectors and produces m-component vectors: A : ℝⁿ → ℝᵐ. These are its domain and codomain. A vector (a,b) belongs to ℝ²; (a,b,c) belongs to ℝ³, not ℝ² as written. The number of entries tells us which ambient space to use. The question “does Ax=b have a solution?” fixes A and b and asks whether any permitted input x reaches b.

01Which outputs can the columns build?

Start with the identity matrix. Every vector (x₁,x₂) is x₁(1,0)+x₂(0,1). Now replace those two standard basis vectors by the columns of A. Matrix multiplication uses exactly the same coefficients: scale each column, then add the resulting vectors tip to tail.

INPUT · ℝ²

x₁ and x₂ are numbers: they tell us how much of each column to use.

OUTPUT · ℝ²

The two independent columns reach the whole plane.

x₁ · component1

x₂ · component1

Ix=1\begin{pmatrix}1\\0\end{pmatrix}+1\begin{pmatrix}0\\1\end{pmatrix}=\begin{pmatrix}1\\1\end{pmatrix}

Animation progress0%

1 · Scale the first column by x₁. A negative coefficient reverses its direction; zero contributes no displacement.

Drag x or use its sliders. Teal = first column contribution; amber = second. Faint dots sample possible outputs. A span contains every real linear combination and continues beyond the drawing.

A column is a vector. A coefficient is a number.

A=[a_1\ a_2\ \cdots\ a_n],\qquad Ax=\sum_{i=1}^{n}x_i a_i

All aᵢ belong to ℝᵐ. Their span is the set of every vector built by scaling and adding them. These columns are the images Aeᵢ of the standard input basis vectors. They form a basis of their span only when they are linearly independent; dependent columns are redundant.

The column space is precisely the reachable set.

\operatorname{Col}(A)=\operatorname{span}\{a_1,\ldots,a_n\}

=\{Ax:x\in\mathbb R^n\}\subseteq\mathbb R^m

This definition explicitly allows every input x in ℝⁿ. The column space is the image (or range), which can be smaller than the codomain ℝᵐ. A fixed target b can be reached exactly when b belongs to Col(A). For the zero matrix, the only possible output is 0.

Ax=b\text{ is solvable}\iff b\in\operatorname{Col}(A)

02Read the same multiplication by rows.

Each output component is the dot product of one entire row with the entire input vector x. The rows have n entries, so x must have n entries too. This establishes the input space ℝⁿ; it does not require x to lie in the span of the rows.

A=\begin{pmatrix}1&2\\2&4\end{pmatrix},\qquad r_1=(1,2),\quad r_2=(2,4)

INPUT · ℝ², with its row space

The amber line is Row(A). Inputs elsewhere in this plane are still allowed.

OUTPUT · each row gives one number

The two numbers together form the output vector b.

x₁ · component1

x₂ · component3

r_1\cdot x=1(1)+2(3)=7

r_2\cdot x=2(1)+4(3)=14

This input is outside Row(A), but Ax is still perfectly well-defined. The full input space is the whole plane.

Ax=\begin{pmatrix}r_1\cdot x\\r_2\cdot x\\\vdots\\r_m\cdot x\end{pmatrix}

Here rᵢ means row i, regarded as a vector in ℝⁿ. The same x appears in every dot product; xᵢ on its own would be just one scalar component, not the vector needed for a dot product.

\operatorname{Row}(A)=\operatorname{span}\{r_1,\ldots,r_m\}=\operatorname{Col}(A^T)

Transposing swaps rows and columns, so Aᵀ has the rows of A as its columns. Row(A) is a subspace of the full domain ℝⁿ. We will soon see why it is the part of the input that determines the output.

03Which inputs disappear entirely?

For A = [1 2; 2 4], call the nonzero vector (2,−1) xₙ. Its output is (0,0). Scaling that input does not change this fact. The line through the origin and xₙ is the null space, also called the kernel: every point on it is sent to the output origin.

INPUT · move along Null(A)

Purple is the null-space line. Every real value of c is allowed.

OUTPUT · always zero

Changing the purple input produces no displacement here.

Scale c1

Animation progress0%

A\begin{pmatrix}2\\-1\end{pmatrix}=\begin{pmatrix}2-2\\4-4\end{pmatrix}=0

A(cx_n)=c(Ax_n)=c0=0

The current input is (2, -1). Its two components cancel within each row calculation. This is why it disappears, rather than because it has zero length.

\operatorname{Null}(A)=\{x\in\mathbb R^n:Ax=0\}

An inverse does not become ambiguous at zero. If a square matrix is invertible, A⁻¹0 = 0, uniquely. Our example has dependent columns and no inverse A⁻¹ at all. Independent columns give Null(A) = {0}, even for a rectangular matrix; a two-sided inverse additionally requires a square matrix. A nontrivial null space can make solutions non-unique for nonzero b too, as we will see below.

04The output side has a perpendicular space too.

Apply the null-space idea to Aᵀ. Its inputs live in ℝᵐ, the original output ambient space. A vector y is in Null(Aᵀ) when it is perpendicular to every column of A. This is the left null space: Aᵀy = 0 is equivalent to yᵀA = 0ᵀ.

OUTPUT SPACE · where is the target b?

Teal = Col(A); rose = Null(Aᵀ). Drag the target or use the sliders.

Test the perpendicular direction

y = (2,−1). This same test checks every target b for this example.

A^Ty=\begin{pmatrix}1&2\\2&4\end{pmatrix}\begin{pmatrix}2\\-1\end{pmatrix}=0

y^Tb=2b_1-b_2=3

Nonzero: b has a component perpendicular to the column space. No input can create that component, so Ax=b has no solution.

$b_c$ is the component in Col(A); $b_\ell$ is the perpendicular component in Null(Aᵀ). The target is reachable exactly when this perpendicular component is zero.

b=b_c+b_\ell=\begin{pmatrix}0.8\\1.6\end{pmatrix}+\begin{pmatrix}1.2\\-0.6\end{pmatrix}

b₁ · component2

b₂ · component1

Why does this test work?

b=Ax\ \Longrightarrow\ y^Tb=y^TAx=0

A solution would force yᵀb to be zero for every y in the left null space. Conversely, if b is orthogonal to the entire left null space, it belongs to the column space. One nonzero result is enough to prove there is no solution.

\operatorname{Null}(A^T)=\{y\in\mathbb R^m:A^Ty=0\}=\{y\in\mathbb R^m:y^TA=0^T\}

The left null space is the kernel of Aᵀ, not the set of output vectors that A sends to zero. In a rectangular problem those vectors might not even have the right number of entries to be inputs to A.

05Perpendicular parts, adding up to the whole space.

A typical input does not belong exclusively to the row space or to the null space. It has one component in each. Drag x: the amber and purple displacements together reconstruct it, like perpendicular components on ordinary coordinate axes.

INPUT · two perpendicular components

xᵣ lies in Row(A); xₙ lies in Null(A). Dashed lines complete their rectangle.

The decomposition is unique

The nearest point to x on the row-space line is xᵣ. The perpendicular remainder is xₙ.

x=\begin{pmatrix}1\\3\end{pmatrix}=\begin{pmatrix}1.4\\2.8\end{pmatrix}+\begin{pmatrix}-0.4\\0.2\end{pmatrix}

Ax_n=0,\qquad x_r\cdot x_n=0

This x is on neither coloured line. Nevertheless, it is the sum of one vector from each line. This is why a union is not enough.

x₁ · component1

x₂ · component3

Row(A)Null(A)Filled black dot: x · drag or use the sliders

1 · Why are the two input subspaces orthogonal?

Take any z in Null(A). Because Az = 0, every output component rᵢ·z is zero. Thus z is orthogonal to every row. Any row-space vector w is a linear combination of those rows; distribute the dot product to see that w·z is zero too. For nonzero vectors this means a right angle. The zero vector is also orthogonal to every vector by the dot-product definition, although it has no direction.

w=\sum_i c_i r_i\quad\Longrightarrow\quad w\cdot z=\sum_i c_i(r_i\cdot z)=0

Conversely, a vector perpendicular to every row makes every component of Az zero, so it belongs to Null(A). That proves equality with the entire orthogonal complement, not just perpendicularity. Applying the same argument to Aᵀ gives the output-side result.

\operatorname{Null}(A)=\operatorname{Row}(A)^\perp,\qquad\operatorname{Null}(A^T)=\operatorname{Col}(A)^\perp

2 · Why is zero their only common vector?

If w belongs to both perpendicular subspaces, it must be perpendicular to itself. Its dot product with itself is its squared length. A sum of squared real components is zero only if every component is zero. Apply the same argument on the output side.

w\cdot w=\|w\|^2=0\quad\Longrightarrow\quad w=0

\operatorname{Row}(A)\cap\operatorname{Null}(A)=\{0\},\qquad\operatorname{Col}(A)\cap\operatorname{Null}(A^T)=\{0\}

3 · Use a direct sum, not a union.

Project any input onto the row space. The remainder is perpendicular to every row, so it belongs to the null space. This gives every x as xᵣ+xₙ. If two such decompositions existed, subtracting them would produce a vector in both subspaces; their intersection is {0}, so the two decompositions must be identical. A union merely collects vectors on either subspace and misses vectors with both components.

\mathbb R^n=\operatorname{Row}(A)\oplus\operatorname{Null}(A)

\mathbb R^m=\operatorname{Col}(A)\oplus\operatorname{Null}(A^T)

The symbol ⊕ means every vector is a unique sum of one vector from each subspace. Here the two parts are also orthogonal. On the output side, split any candidate b into its column-space and left-null-space components. The equation is solvable exactly when bₗ = 0.

06Different inputs. Exactly the same output.

Consider xₚ = (1,3), xₙ = (2,−1), and A(xₚ+xₙ) = A(3,2) = (7,14) = Axₚ. Here xₚ is a particular solution, not a row-space vector. Its unique row-space component is xᵣ = (7/5,14/5). Use that common component as the starting point for every solution.

INPUT · a whole line of solutions

Every pale dot has the same amber row-space component. Purple displacements change only the null part.

OUTPUT · locked at (7,14)

A erases every purple contribution; the amber contribution always produces the same b.

Null-space coefficient c-0.2

Animation progress0%

x=x_r+c\begin{pmatrix}2\\-1\end{pmatrix}\approx\begin{pmatrix}1\\3\end{pmatrix}

Ax=Ax_r+cA\begin{pmatrix}2\\-1\end{pmatrix}=\begin{pmatrix}7\\14\end{pmatrix}

The drawn solution line is a shifted copy of Null(A). Because b is nonzero, this line does not contain the origin and is not a vector subspace. The c = 0 input is the only solution in Row(A).

Why adding a null vector changes nothing

A(x_r+x_n)=Ax_r+Ax_n=Ax_r

Linearity lets us distribute A across the sum. The null-space term vanishes. In fact this works starting from any particular solution xₚ, whether or not xₚ lies in Row(A). Conversely, if Ax = Axₚ, then A(x−xₚ)=0, so every solution differs from xₚ by a null-space vector.

\{x:Ax=b\}=x_p+\operatorname{Null}(A)\quad(b\in\operatorname{Col}(A))

Row(A) maps one-to-one onto Col(A)

Every reachable b has a solution x. Decompose it: xᵣ produces that same b, so restricting A to Row(A) still reaches all of Col(A). If two row-space vectors had the same output, their difference would be in both Row(A) and Null(A), hence would be zero. They must be the same vector.

A|_{\operatorname{Row}(A)}:\operatorname{Row}(A)\ \longleftrightarrow\ \operatorname{Col}(A)

This restricted map is invertible between these two subspaces. That does not give a two-sided inverse matrix for the original singular A on the full ambient spaces.

07The four subspaces, in one diagram.

Read each large box as a different ambient space. The left box accepts inputs; the right box contains candidate outputs. The top arrow carries the row-space part to the column space. The lower arrow shows the null-space part disappearing at the output zero. Regions are schematic: read their dimensions rather than their drawn area.

A=\begin{pmatrix}1&2\\2&4\end{pmatrix}

Animation progress0%

Rank r counts the independent column directions and also the independent row directions. Dimension counts the number of vectors in a basis. Losing n−r input directions leaves r independent output directions.

\underbrace{r+(n-r)}_{\text{input}}=1+1=2,\quad\underbrace{r+(m-r)}_{\text{output}}=1+1=2

Subspace	Definition	Ambient space	Dimension
Row(A)	$\operatorname{span}\{r_1,\ldots,r_m\}$	$\mathbb R^n$	r = 1
Null(A)	$\{x\in\mathbb R^n:Ax=0\}$	$\mathbb R^n$	n − r = 1
Col(A)	$\{Ax:x\in\mathbb R^n\}$	$\mathbb R^m$	r = 1
Null(Aᵀ)	$\{y\in\mathbb R^m:A^Ty=0\}$	$\mathbb R^m$	m − r = 1

Dimension 0 means the subspace {0}, not an empty set. Orthogonality is asserted within each ambient space; Row(A) and Col(A) need not be perpendicular or even have the same number of coordinates. They have equal dimension r, not necessarily equal vectors.