Why Some Vectors Matter More

Recap

In this article, we introduce the concept of how scaling and combining vectors can create new vectors in space. This becomes intuitive once you realize a vector is nothing but data — numerical information that a computer understands. A linear combination gives us the power to twist and transform that data to create new data points.

We also briefly touched on the concept of span: if you have two non-parallel vectors, their span fills the entire 2D plane. Similarly, in any N-dimensional space, you can reach any point using a combination of the right number of linearly independent vectors.

Let’s now dive deeper to understand what this really means and why it’s important.

Non-parallel Vectors & Linear (In)Dependence

The word span literally means “the amount of space that something covers.” In linear algebra, the span of vectors refers to all the possible vectors you can create by taking linear combinations of those vectors.

Let’s try to prove this in 2D space.

Suppose we have two vectors:

We want to prove that for any arbitrary vector u,

there exist scalars α and β such that:

This leads to the system of equations:

This is a system of two equations with two unknowns and hence, this system has a unique solution. It means, we can reach any point (x, y)in the 2D plane by substituting different values for α and β.

Let’s try with a simple example.

Suppose:

and we want to see if a new vector u (given below), can be written as a linear combination of v1 and v2. That means solving below,

After solving this system, we find:

Yeah!! vector u does lie in the span of vector v1 and v2​. Similarly, any other vector in 2D space can be represented as a linear combination of these two — this is what linear independence ensures.

Linear Dependence

But what happens if two vectors are parallel?

In that case, both vectors lie along the same line through the origin. One is a scalar multiple of the other:

So, when we try to express any new vector, it becomes:

This is just another scalar multiple of vector v1​. So, the span of v1​ and v2​ is simply the line through the origin in the direction of v1​. You cannot reach any point outside that line — the span is one-dimensional.

This is a technique to reduce the dimensionality of data, which is used to reduce redundancy of data, improve efficiency of model as lower dimensions mean faster computations, lesser storage.

Formal Definition

A set of vectors is said to be linearly dependent if there exist scalars, not all zero, such that:

This means at least one vector in the set can be expressed as a combination of the others.

If the only solution to that equation is all scalars being zero, the vectors are linearly independent.

What does it matter in AI?

In machine learning and AI, your data is represented as vectors. When building models, especially in high-dimensional spaces, you want your data features (vectors) to be linearly independent. Why?

Because:

• Redundant (dependent) vectors don’t add new information.
• Linear independence ensures that your features are diverse and informative.

· Many algorithms (like PCA, regression, etc.) assume or benefit from independence for stability and accuracy.

For example, if your feature space is linearly dependent, your model might not be able to find a unique solution or might overfit due to redundant information.

Conclusion

Linear independence and span aren’t just abstract math — they’re practical concepts that show up all over machine learning and data science. When you understand how vectors relate through linear combinations, you understand how data can be transformed, compressed, or reconstructed — which lies at the heart of many AI systems.