Subspaces of Rn and Linear Independence

Subspaces

For any vector space, a subspace is a subset that is itself a vector space, under the inherited operations.

Lemma 1

For a nonempty subset $S$ of a vector space, under the inherited operations, the following are equivalent statements.

1. $S$ is a subspace of that vector space
2. $S$ is closed under linear combinations of pairs of vectors: for any vectors ${\vec {s}}_{1},{\vec {s}}_{2}\in S$ and scalars $r_{1},r_{2}$ the vector $r_{1}{\vec {s}}_{1}+r_{2}{\vec {s}}_{2}$ is in $S$
3. $S$ is closed under linear combinations of any number of vectors: for any vectors ${\vec {s}}_{1},\ldots ,{\vec {s}}_{n}\in S$ and scalars $r_{1},\ldots ,r_{n}$ the vector $r_{1}{\vec {s}}_{1}+\cdots +r_{n}{\vec {s}}_{n}$ is in $S$.

Briefly, the way that a subset gets to be a subspace is by being closed under linear combinations.

Proof:

"The following are equivalent" means that each pair of statements are equivalent.

$(1)\!\iff \!(2)\qquad (2)\!\iff \!(3)\qquad (3)\!\iff \!(1)$

We will show this equivalence by establishing that $(1)\implies (3)\implies (2)\implies (1)$. This strategy is suggested by noticing that $(1)\implies (3)$ and $(3)\implies (2)$ are easy and so we need only argue the single implication $(2)\implies (1)$.

For that argument, assume that $S$ is a nonempty subset of a vector space $V$ and that $S$ is closed under combinations of pairs of vectors. We will show that $S$ is a vector space by checking the conditions.

The first item in the vector space definition has five conditions. First, for closure under addition, if ${\vec {s}}_{1},{\vec {s}}_{2}\in S$ then ${\vec {s}}_{1}+{\vec {s}}_{2}\in S$, as ${\vec {s}}_{1}+{\vec {s}}_{2}=1\cdot {\vec {s}}_{1}+1\cdot {\vec {s}}_{2}$.

Second, for any ${\vec {s}}_{1},{\vec {s}}_{2}\in S$, because addition is inherited from $V$, the sum ${\vec {s}}_{1}+{\vec {s}}_{2}$ in $S$ equals the sum ${\vec {s}}_{1}+{\vec {s}}_{2}$ in $V$, and that equals the sum ${\vec {s}}_{2}+{\vec {s}}_{1}$ in $V$ (because $V$ is a vector space, its addition is commutative), and that in turn equals the sum ${\vec {s}}_{2}+{\vec {s}}_{1}$ in $S$. The argument for the third condition is similar to that for the second.

For the fourth, consider the zero vector of $V$ and note that closure of $S$ under linear combinations of pairs of vectors gives that (where ${\vec {s}}$ is any member of the nonempty set $S$) $0\cdot {\vec {s}}+0\cdot {\vec {s}}={\vec {0}}$ is in $S$; showing that ${\vec {0}}$ acts under the inherited operations as the additive identity of $S$ is easy.

The fifth condition is satisfied because for any ${\vec {s}}\in S$, closure under linear combinations shows that the vector $0\cdot {\vec {0}}+(-1)\cdot {\vec {s}}$ is in $S$; showing that it is the additive inverse of ${\vec {s}}$ under the inherited operations is routine.

We usually show that a subset is a subspace with $(2)\implies (1)$.

Example 1

The plane $P=\{{\begin{pmatrix}x\\y\\z\end{pmatrix}}\,{\big |}\,x+y+z=0\}$ is a subspace of $\mathbb {R} ^{3}$. As specified in the definition, the operations are the ones inherited from the larger space, that is, vectors add in $P$ as they add in $\mathbb {R} ^{3}$

${\begin{pmatrix}x_{1}\\y_{1}\\z_{1}\end{pmatrix}}+{\begin{pmatrix}x_{2}\\y_{2}\\z_{2}\end{pmatrix}}={\begin{pmatrix}x_{1}+x_{2}\\y_{1}+y_{2}\\z_{1}+z_{2}\end{pmatrix}}$

and scalar multiplication is also the same as it is in $\mathbb {R} ^{3}$. To show that $P$ is a subspace, we need only note that it is a subset and then verify that it is a space. Checking that $P$ satisfies the conditions in the definition of a vector space is routine. For instance, for closure under addition, just note that if the summands satisfy that $x_{1}+y_{1}+z_{1}=0$ and $x_{2}+y_{2}+z_{2}=0$ then the sum satisfies that $(x_{1}+x_{2})+(y_{1}+y_{2})+(z_{1}+z_{2})=(x_{1}+y_{1}+z_{1})+(x_{2}+y_{2}+z_{2})=0$.

Example 2

The $x$-axis in $\mathbb {R} ^{2}$ is a subspace where the addition and scalar multiplication operations are the inherited ones.

${\begin{pmatrix}x_{1}\\0\end{pmatrix}}+{\begin{pmatrix}x_{2}\\0\end{pmatrix}}={\begin{pmatrix}x_{1}+x_{2}\\0\end{pmatrix}}\qquad r\cdot {\begin{pmatrix}x\\0\end{pmatrix}}={\begin{pmatrix}rx\\0\end{pmatrix}}$

As above, to verify that this is a subspace, we simply note that it is a subset and then check that it satisfies the conditions in definition of a vector space. For instance, the two closure conditions are satisfied: (1) adding two vectors with a second component of zero results in a vector with a second component of zero, and (2) multiplying a scalar times a vector with a second component of zero results in a vector with a second component of zero.

Example 3

Another subspace of $\mathbb {R} ^{2}$ is

$\{{\begin{pmatrix}0\\0\end{pmatrix}}\}$

which is its trivial subspace.

Any vector space has a trivial subspace $\{{\vec {0}}\,\}$.

At the opposite extreme, any vector space has itself for a subspace. Template:AnchorThese two are the improper subspaces. Template:AnchorOther subspaces are proper.

Example 4

The condition in the definition requiring that the addition and scalar multiplication operations must be the ones inherited from the larger space is important. Consider the subset $\{1\}$ of the vector space $\mathbb {R} ^{1}$. Under the operations $1+1=1$ and $r\cdot 1=1$ that set is a vector space, specifically, a trivial space. But it is not a subspace of $\mathbb {R} ^{1}$ because those aren't the inherited operations, since of course $\mathbb {R} ^{1}$ has $1+1=2$.

Example 5

All kinds of vector spaces, not just $\mathbb {R} ^{n}$'s, have subspaces. The vector space of cubic polynomials $\{a+bx+cx^{2}+dx^{3}\,{\big |}\,a,b,c,d\in \mathbb {R} \}$ has a subspace comprised of all linear polynomials $\{m+nx\,{\big |}\,m,n\in \mathbb {R} \}$.

Example 6

This is a subspace of the $2\!\times \!2$ matrices

$L=\{{\begin{pmatrix}a&0\\b&c\end{pmatrix}}\,{\big |}\,a+b+c=0\}$

(checking that it is nonempty and closed under linear combinations is easy).

To parametrize, express the condition as $a=-b-c$.

$L=\{{\begin{pmatrix}-b-c&0\\b&c\end{pmatrix}}\,{\big |}\,b,c\in \mathbb {R} \}=\{b{\begin{pmatrix}-1&0\\1&0\end{pmatrix}}+c{\begin{pmatrix}-1&0\\0&1\end{pmatrix}}\,{\big |}\,b,c\in \mathbb {R} \}$

As above, we've described the subspace as a collection of unrestricted linear combinations (by coincidence, also of two elements).

Span

The span(or linear closure) of a nonempty subset $S$ of a vector space is the set of all linear combinations of vectors from $S$.

$[S]=\{c_{1}{\vec {s}}_{1}+\cdots +c_{n}{\vec {s}}_{n}\,{\big |}\,c_{1},\ldots ,c_{n}\in \mathbb {R} {\text{ and }}{\vec {s}}_{1},\ldots ,{\vec {s}}_{n}\in S\}$

The span of the empty subset of a vector space is the trivial subspace. No notation for the span is completely standard. The square brackets used here are common, but so are "${\mbox{span}}(S)$" and "${\mbox{sp}}(S)$".

Lemma 2

In a vector space, the span of any subset is a subspace.

Proof:

Call the subset $S$. If $S$ is empty then by definition its span is the trivial subspace. If $S$ is not empty, then we need only check that the span $[S]$ is closed under linear combinations. For a pair of vectors from that span, ${\vec {v}}=c_{1}{\vec {s}}_{1}+\cdots +c_{n}{\vec {s}}_{n}$ and ${\vec {w}}=c_{n+1}{\vec {s}}_{n+1}+\cdots +c_{m}{\vec {s}}_{m}$, a linear combination

$p\cdot (c_{1}{\vec {s}}_{1}+\cdots +c_{n}{\vec {s}}_{n})+r\cdot (c_{n+1}{\vec {s}}_{n+1}+\cdots +c_{m}{\vec {s}}_{m})$

$=pc_{1}{\vec {s}}_{1}+\cdots +pc_{n}{\vec {s}}_{n}+rc_{n+1}{\vec {s}}_{n+1}+\cdots +rc_{m}{\vec {s}}_{m}$

($p$, $r$ scalars) is a linear combination of elements of $S$ and so is in $[S]$ (possibly some of the ${\vec {s}}_{i}$'s forming ${\vec {v}}$ equal some of the ${\vec {s}}_{j}$'s from ${\vec {w}}$, but it does not matter).

Example 7

The span of this set is all of $\mathbb {R} ^{2}$.

$\{{\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\-1\end{pmatrix}}\}$

To check this we must show that any member of $\mathbb {R} ^{2}$ is a linear combination of these two vectors. So we ask: for which vectors (with real components $x$ and $y$) are there scalars $c_{1}$ and $c_{2}$ such that this holds?

$c_{1}{\begin{pmatrix}1\\1\end{pmatrix}}+c_{2}{\begin{pmatrix}1\\-1\end{pmatrix}}={\begin{pmatrix}x\\y\end{pmatrix}}$

Gauss' method

${\begin{array}{rcl}{\begin{array}{*{2}{rc}r}c_{1}&+&c_{2}&=&x\\c_{1}&-&c_{2}&=&y\end{array}}&{\xrightarrow[{}]{-\rho _{1}+\rho _{2}}}&{\begin{array}{*{2}{rc}r}c_{1}&+&c_{2}&=&x\\&&-2c_{2}&=&-x+y\end{array}}\end{array}}$

with back substitution gives $c_{2}=(x-y)/2$ and $c_{1}=(x+y)/2$. These two equations show that for any $x$ and $y$ that we start with, there are appropriate coefficients $c_{1}$ and $c_{2}$ making the above vector equation true. For instance, for $x=1$ and $y=2$ the coefficients $c_{2}=-1/2$ and $c_{1}=3/2$ will do. That is, any vector in $\mathbb {R} ^{2}$ can be written as a linear combination of the two given vectors.

Linear Independence

We first characterize when a vector can be removed from a set without changing the span of that set.

Lemma 3

Where $S$ is a subset of a vector space $V$,

$[S]=[S\cup \{{\vec {v}}\}]\quad {\text{if and only if}}\quad {\vec {v}}\in [S]$

for any ${\vec {v}}\in V$.

Proof: The left to right implication is easy. If $[S]=[S\cup \{{\vec {v}}\}]$ then, since ${\vec {v}}\in [S\cup \{{\vec {v}}\}]$, the equality of the two sets gives that ${\vec {v}}\in [S]$.

For the right to left implication assume that ${\vec {v}}\in [S]$ to show that $[S]=[S\cup \{{\vec {v}}\}]$ by mutual inclusion. The inclusion $isobvious.Fortheotherinclusion<math>[S]\supseteq [S\cup \{{\vec {v}}\}]$, write an element of $[S\cup \{{\vec {v}}\}]$ as $d_{0}{\vec {v}}+d_{1}{\vec {s}}_{1}+\dots +d_{m}{\vec {s}}_{m}$ and substitute ${\vec {v}}$'s expansion as a linear combination of members of the same set $d_{0}(c_{0}{\vec {t}}_{0}+\dots +c_{k}{\vec {t}}_{k})+d_{1}{\vec {s}}_{1}+\dots +d_{m}{\vec {s}}_{m}$. This is a linear combination of linear combinations and so distributing $d_{0}$ results in a linear combination of vectors from $S$. Hence each member of $[S\cup \{{\vec {v}}\}]$ is also a member of $[S]$.

Example: In $\mathbb {R} ^{3}$, where

${\vec {v}}_{1}={\begin{pmatrix}1\\0\\0\end{pmatrix}}\quad {\vec {v}}_{2}={\begin{pmatrix}0\\1\\0\end{pmatrix}}\quad {\vec {v}}_{3}={\begin{pmatrix}2\\1\\0\end{pmatrix}}$

the spans $[\{{\vec {v}}_{1},{\vec {v}}_{2}\}]$ and $[\{{\vec {v}}_{1},{\vec {v}}_{2},{\vec {v}}_{3}\}]$ are equal since ${\vec {v}}_{3}$ is in the span $[\{{\vec {v}}_{1},{\vec {v}}_{2}\}]$.

Lemma 2 says that if we have a spanning set then we can remove a ${\vec {v}}$ to get a new set $S$ with the same span if and only if ${\vec {v}}$ is a linear combination of vectors from $S$. Thus, under the second sense described above, a spanning set is minimal if and only if it contains no vectors that are linear combinations of the others in that set. We have a term for this important property.

Definition of Linear Independence

A subset of a vector space is linearly independent if none of its elements is a linear combination of the others. Otherwise it is linearly dependent. }}

Here is an important observation:

${\vec {s}}_{0}=c_{1}{\vec {s}}_{1}+c_{2}{\vec {s}}_{2}+\cdots +c_{n}{\vec {s}}_{n}$

although this way of writing one vector as a combination of the others visually sets ${\vec {s}}_{0}$ off from the other vectors, algebraically there is nothing special in that equation about ${\vec {s}}_{0}$. For any ${\vec {s}}_{i}$ with a coefficient $c_{i}$ that is nonzero, we can rewrite the relationship to set off ${\vec {s}}_{i}$.

${\vec {s}}_{i}=(1/c_{i}){\vec {s}}_{0}+(-c_{1}/c_{i}){\vec {s}}_{1}+\dots +(-c_{n}/c_{i}){\vec {s}}_{n}$

When we don't want to single out any vector by writing it alone on one side of the equation, we will instead say that ${\vec {s}}_{0},{\vec {s}}_{1},\dots ,{\vec {s}}_{n}$ are in a linear relationship and write the relationship with all of the vectors on the same side. The next result rephrases the linear independence definition in this style. It gives what is usually the easiest way to compute whether a finite set is dependent or independent.

Lemma 4

A subset $S$ of a vector space is linearly independent if and only if for any distinct ${\vec {s}}_{1},\dots ,{\vec {s}}_{n}\in S$ the only linear relationship among those vectors

$c_{1}{\vec {s}}_{1}+\dots +c_{n}{\vec {s}}_{n}={\vec {0}}\qquad c_{1},\dots ,c_{n}\in \mathbb {R}$

is the trivial one: $c_{1}=0,\dots ,\,c_{n}=0$. Proof: This is a direct consequence of the observation above.

If the set $S$ is linearly independent then no vector ${\vec {s}}_{i}$ can be written as a linear combination of the other vectors from $S$ so there is no linear relationship where some of the ${\vec {s}}\,$'s have nonzero coefficients. If $S$ is not linearly independent then some ${\vec {s}}_{i}$ is a linear combination ${\vec {s}}_{i}=c_{1}{\vec {s}}_{1}+\dots +c_{i-1}{\vec {s}}_{i-1}+c_{i+1}{\vec {s}}_{i+1}+\dots +c_{n}{\vec {s}}_{n}$ of other vectors from $S$, and subtracting ${\vec {s}}_{i}$ from both sides of that equation gives a linear relationship involving a nonzero coefficient, namely the $-1$ in front of ${\vec {s}}_{i}$.

Example 8

In the vector space of two-wide row vectors, the two-element set $\{{\begin{pmatrix}40&15\end{pmatrix}},{\begin{pmatrix}-50&25\end{pmatrix}}\}$ is linearly independent. To check this, set

$c_{1}\cdot {\begin{pmatrix}40&15\end{pmatrix}}+c_{2}\cdot {\begin{pmatrix}-50&25\end{pmatrix}}={\begin{pmatrix}0&0\end{pmatrix}}$

and solving the resulting system

${\begin{array}{*{2}{rc}r}40c_{1}&-&50c_{2}&=&0\\15c_{1}&+&25c_{2}&=&0\end{array}}\;{\xrightarrow[{}]{-(15/40)\rho _{1}+\rho _{2}}}\;{\begin{array}{*{2}{rc}r}40c_{1}&-&50c_{2}&=&0\\&&(175/4)c_{2}&=&0\end{array}}$

shows that both $c_{1}$ and $c_{2}$ are zero. So the only linear relationship between the two given row vectors is the trivial relationship.

In the same vector space, $\{{\begin{pmatrix}40&15\end{pmatrix}},{\begin{pmatrix}20&7.5\end{pmatrix}}\}$ is linearly dependent since we can satisfy

$c_{1}{\begin{pmatrix}40&15\end{pmatrix}}+c_{2}\cdot {\begin{pmatrix}20&7.5\end{pmatrix}}={\begin{pmatrix}0&0\end{pmatrix}}$

with $c_{1}=1$ and $c_{2}=-2$.

Example 9

The set $\{1+x,1-x\}$ is linearly independent in ${\mathcal {P}}_{2}$, the space of quadratic polynomials with real coefficients, because

$0+0x+0x^{2}=c_{1}(1+x)+c_{2}(1-x)=(c_{1}+c_{2})+(c_{1}-c_{2})x+0x^{2}$

gives

${\begin{array}{rcl}{\begin{array}{*{2}{rc}r}c_{1}&+&c_{2}&=&0\\c_{1}&-&c_{2}&=&0\end{array}}&{\xrightarrow[{}]{-\rho _{1}+\rho _{2}}}&{\begin{array}{*{2}{rc}r}c_{1}&+&c_{2}&=&0\\&&2c_{2}&=&0\end{array}}\end{array}}$

since polynomials are equal only if their coefficients are equal. Thus, the only linear relationship between these two members of ${\mathcal {P}}_{2}$ is the trivial one.

Example 10

In $\mathbb {R} ^{3}$, where

${\vec {v}}_{1}={\begin{pmatrix}3\\4\\5\end{pmatrix}}\quad {\vec {v}}_{2}={\begin{pmatrix}2\\9\\2\end{pmatrix}}\quad {\vec {v}}_{3}={\begin{pmatrix}4\\18\\4\end{pmatrix}}$

the set $S=\{{\vec {v}}_{1},{\vec {v}}_{2},{\vec {v}}_{3}\}$ is linearly dependent because this is a relationship

$0\cdot {\vec {v}}_{1}+2\cdot {\vec {v}}_{2}-1\cdot {\vec {v}}_{3}={\vec {0}}$

where not all of the scalars are zero (the fact that some of the scalars are zero doesn't matter).

Resources

• Subspaces, Interactive Linear Algebra from Georgia Tech

Licensing

Content obtained and/or adapted from:

• Definition and Examples of Linear Independence, WikiBooks Linear Algebra under a CC BY-SA license
• Subspaces and Spanning Sets, WikiBooks Linear Algebra under a CC BY-SA license
• Subspaces, WikiBooks Linear Algebra under a CC BY-SA license