Linear Dependence of Vectors

Linear Independence and Dependence

Definition: A set of vectors $V=\{\mathbf {x_{1}} ,\mathbf {x_{2}} ,...,\mathbf {x_{n}} \}$ and the scalars $k_{1},k_{2},...,k_{n}$ has a solution for the vector equation $k_{1}\mathbf {x_{1}} +k_{2}\mathbf {x_{2}} +...+k_{n}\mathbf {x_{n}} =0$, namely when $k_{1}=k_{2}=...=k_{n}=0$. If this is the only solution to the vector equation, then the set $V$ is said to be Linearly Independent. If there exists other solutions where not all $k_{i}=0$, then $V$ is said to be Linearly Dependent.

We will now look at some examples of vector sets which are either linearly independent or linearly dependent.

Note: Another way to write that a set of vectors $\{v_{1},v_{2},...,v_{n}\}$ is linearly independent is by saying that $\mathrm {span} (v_{1},v_{2},...,v_{n})=\bigoplus _{i=1}^{n}\mathbb {F} v_{i}$, which says that any vector that is a linear combination of the set of vectors $v_{1},v_{2},...,v_{n}$ is a unique linear combination.

Example 1

Determine whether or not the vector set $V=\{(1,2),(-5,-3)\}$ is linearly independent or linearly dependent.

We first must see if there exists only one set of scalars $c_{1}=c_{2}=0$ (the trivial solution) as a solution to the vector equation $c_{1}(1,2)+c_{2}(-5,-3)=0$ or if there exists more solutions.

From this vector equation we get the following system of linear equations:

${\begin{aligned}c_{1}-5c_{2}=0\\2c_{1}-3c_{2}=0\end{aligned}}$

When we reduce this system to RREF, we get that:

${\begin{aligned}c_{1}+0c_{2}=0\\0c_{1}+c_{2}=0\end{aligned}}$

Therefore our only set of scalars $c_{1},c_{2}$ are both equal to zero. Therefore, the vector set $V$ is linearly independent.

Example 2

Determine whether the vector set $V=\{(3,6),(4,8)\}$ is a linearly independent or a linearly dependent set.

Once again, for $V$ to be a linearly independent set then the vector equation $c_{1}(3,6)+c_{2}(4,8)=0$ must containing only one set of scalars $c_{1}=c_{2}=0$. From our vector equation be obtain the following system of linear equations:

${\begin{aligned}3c_{1}+4c_{2}=0\\6c_{1}+8c_{2}=0\end{aligned}}$

When we solve this system of equations we obtain that:

${\begin{aligned}c_{1}+{\frac {4}{3}}c_{2}=0\\0c_{1}+0c_{2}=0\end{aligned}}$

Suppose that $c_{2}=s$ so that $c_{1}=-{\frac {4}{3}}s$. We see that for any $s\in \mathbb {R}$, the scalars $c_{1}$ and $c_{2}$ satisfy the vector equation and thus there are infinitely many scalars sets. Therefore $V$ is a linearly dependent set.

Example 3

Show that if $A$ is an $n\times n$ invertible matrix, then the column vectors of $A$ denoted $C_{1}$, $C_{2}$, …, $C_{n}$ are linearly independent.

Suppose that $A$ is an invertible matrix and suppose that we have the linear system $Ax=0$. We note that the only solution to this linear system is the trivial solution, namely $x=0$. This system corresponds to the following vector equations though:

$x_{1}C_{1}+x_{2}C_{2}+...+x_{n}C_{n}=0$

And since $x=(x_{1},x_{2},...,x_{n})=(0,0,...,0)$, we see that this vector equation only have the scalars $x_{1}=x_{2}=...=x_{n}=0$, so the column vectors of $A$ are linearly independent.

Example 4

Consider the set of vectors $\{1,1-x,1-x^{2}\}$ from the vector space $\wp _{2}(\mathbb {F} )$. Determine if this set of vectors is linearly independent or linearly dependent.

When we expand the vector equation $a(1)+b(1-x)+c(1-x^{2})=0$ as follows, notice:

${\begin{aligned}a(1)+b(1-x)+c(1-x^{2})=0\\a+b-bx+c-cx^{2}=0\\(a+b+c)+(-b)x+(-c)x^{2}=0\end{aligned}}$

We can clearly see that this equation holds only if $a=b=c=0$. We can also verify this with the following matrix:

${\begin{aligned}A={\begin{bmatrix}1&1&1\\0&1&0\\0&0&1\end{bmatrix}}\end{aligned}}$

Since this is an upper triangular matrix, its determinant is the product of the entries down the main diagonal and so $\mathrm {det} (A)=1\cdot (-1)\cdot (-1)=1$ which implies $A$ is invertible, and so the homogenous system ${\begin{bmatrix}1&1&1\\0&-1&0\\0&0&-1\end{bmatrix}}{\begin{bmatrix}a\\b\\c\end{bmatrix}}={\begin{bmatrix}0\\0\\0\end{bmatrix}}$ has only the trivial solution $a=b=c=0$.

Example 5

Show that the set of vectors $\{m,m-x,m-x^{2}\}$ from $\wp _{2}(\mathbb {F} )$ is a linearly independent set for all nonzero $m$.

We should note that this example is a more general version of example 4. First, let's expand the vector equation $a_{1}m+a_{2}(m-x)+a_{3}(m-x^{2})=0$ as follows:

${\begin{aligned}a_{1}m+a_{2}(m-x)+a_{3}(m-x^{2})=0\\a_{1}m+a_{2}m-a_{2}x+a_{3}m-a_{3}x^{2}=0\\(a_{1}+a_{2}+a_{3})m+(-a_{2})x+(-a_{3})x^{2}=0\end{aligned}}$

Once again, this vector equation is only true if $a_{1}=a_{2}=a_{3}=0$. The same matrix from example 4 can be used to provide a more extensive argument.

Linear Dependence Lemma

We will now look at a very important lemma known as the linear dependence lemma.

Lemma (Linear Dependence Lemma): Let $\{v_{1},v_{2},...,v_{m}\}$ be a set of linearly dependent vectors in the vector space $V$ and $v_{1}\neq 0$. Then there exists $j\in \{2,3,...,m\}$ such that:

a) $v_{j}\in \mathrm {span} \{v_{1},...,v_{j-1}\}$.

b) $\mathrm {span} \{v_{1},...,v_{j-1},v_{j+1},...,v_{m}\}=\mathrm {span} \{v_{1},...,v_{m}\}$.

The linear dependence lemma tells us that given a linear dependent set of vectors where the first vector is nonzero, then there exists a vector in the set $\{v_{1},v_{2},...,v_{m}\}$ such that $v_{j}$ can be written as a linear combination of $\{v_{1},v_{2},...,v_{j-1}$ (that is $v_{j}\in \mathrm {span} \{v_{1},v_{2},...,v_{j-1}\}$), and that the set of linear combinations of the set of vectors $\{v_{1},...,v_{j-1},v_{j+1},...,v_{m}\}$ is the same set as the combination of the set of vectors $\{v_{1},v_{2},...,v_{m}\}$.

• Proof: Let $\{v_{1},v_{2},...,v_{m}\}$ be a set of vectors that are linearly dependent where $v_{1}\neq 0$, and let $a_{1},a_{2},...,a_{m}\in \mathbb {F}$. Since this set of vectors is not linearly independent, then:

$a_{1}v_{1}+a_{2}v_{2}+...+a_{m}v_{m}=0$

• Where not all $a_{i}$ are zero (otherwise the set of vectors would be linearly independent). Now since $v_{1}\neq 0$, it follows that not all $a_{2},a_{3},...,a_{m}\in \mathbb {F}$ are equal to zero, and so there exists an $a_{j}\in \mathbb {F}$ such that $a_{j}\neq 0$ where $j$ is the largest index such that $a_{j}\neq 0$, in other words, $a_{j+1}=0$, $a_{j+2}=0$, etc…, and so so:

${\begin{aligned}\quad a_{j}v_{j}=-a_{1}v_{1}-a_{2}v_{2}-...-a_{j-1}v_{j-1}-\underbrace {a_{j+1}v_{j+1}} _{=0}+...+\underbrace {a_{m}v_{m}} _{=0}\\\quad a_{j}v_{j}=-a_{1}v_{1}-a_{2}v_{2}-...-a_{j-1}v_{j-1}\\\quad v_{j}=-{\frac {a_{1}}{a_{j}}}v_{1}-{\frac {a_{2}}{a_{j}}}v_{2}-...-{\frac {a_{j-1}}{a_{j}}}v_{j-1}\end{aligned}}$

• Therefore $v_{j}$ can be written as a linear combination of the set of vectors $\{v_{1},v_{2},...,v_{j-1}\}$ so $v_{j}\in \mathrm {span} \{v_{1},v_{2},...,v_{j-1}\}$ which proves A.

• Now let $u\in \mathrm {span} \{v_{1},v_{2},...,v_{m}\}$, and so $u$ can be written as a linear combination of the vectors in this set, and so there exists $b_{1},b_{2},...,b_{m}\in \mathbb {F}$ such that:

$u=b_{1}v_{1}+b_{2}v_{2}+...+b_{m}v_{m}$

• Now we substitute that $\quad v_{j}=-{\frac {a_{1}}{a_{j}}}v_{1}-{\frac {a_{2}}{a_{j}}}v_{2}-...-{\frac {a_{j-1}}{a_{j}}}v_{j-1}$ and so:

${\begin{aligned}\quad u=b_{1}v_{1}+b_{2}v_{2}+...+b_{j-1}v_{j-1}+b_{j}v_{j}+b_{j+1}v_{j+1}+...+b_{m}v_{m}\\\quad u=b_{1}v_{1}+b_{2}v_{2}+...+b_{j-1}v_{j-1}+b_{j}[-{\frac {a_{1}}{a_{j}}}v_{1}-{\frac {a_{2}}{a_{j}}}v_{2}-...-{\frac {a_{j-1}}{a_{j}}}v_{j-1}]+b_{j+1}v_{j+1}+...+b_{m}v_{m}\\\quad u=(b_{1}-a_{1}b_{j})v_{1}+(b_{2}-a_{2}b_{j})v_{2}+...+(b_{j-1}-a_{j-1}b_{j})v_{j-1}+(b_{j+1}-a_{j+1}b_{j})v_{j+1}+...+(b_{m}-a_{m}b_{j})v_{m}\end{aligned}}$

• Therefore $u$ can be written as a linear combination of the vectors $\{v_{1},...,v_{j-1},v_{j+1},...,v_{m}$ and so $u\in \mathrm {span} \{v_{1},...,v_{j-1},v_{j+1},...,v_{m}\}$. Therefore $\mathrm {span} \{v_{1},v_{2},...,v_{m}\}=\mathrm {span} \{v_{1},...,v_{j-1},v_{j+1},...,v_{m}\}$. $\blacksquare$

Licensing

Content obtained and/or adapted from:

• Linear Independence and Dependence, mathonline.wikidot.com under a CC BY-SA license
• Linear Dependence Lemma, mathonline.wikidot.com under a CC BY-SA license