I will first try to understand what a matrix is, and how it can be interpreted as a set of transforms that can be applied to a point in space. Then I’ll look at different types of matrices and what makes them unique. Finally, I’ll make an attempt at understanding the SVD on the next chapter.
Note: All the images used in the page are from the playlist by Visual Kernel[1]. Highly recommend that you check out his videos.
I like to think of a matrix as a number box. It’s like saying that a parrot is a green bird. Sure it tells me what it looks like, but it does not tell me what it does. Instead, one can think of a matrix as an operation that can be applied to a vector.
When you move a point from one position to another, is there a function that mathematically defines that movement?
A vector can be thought of as a point in N dimensional space. It can be represented by the co-ordinates of the point like p = [1, 3, -4] is a 3 dimensional vector such that it represents the point x=1, y=3, z=-4 in 3D space.
We can multiply a matrix to a vector, and that gives us another vector. For a better intuition, we will be visualizing a set of points and what a single matrix (read: transform) does to it. In this case, the set of points would be a rectangle.
It’s basically the do nothing transform. Kind of like multiplying something by 1 or adding 0 to it. Does nothing.
Strech or squeeze uniformly along all axes.
{{< math.inline >}}
Now if (k > 1), then the rectangle gets bigger by a factor of (k). In a similar way, it also gets smaller if (0 < k < 1). Interestingly enough, the rectangle would flip both horizontally and vertically if (k = -1)
This is what happens when (k = 2), the rectangle just becomes 2x bigger.
{{</ math.inline >}}
Strech or squeeze along only one axis.
Notice how only one of the diagonal elements are not 1. That is the axis along which we’ll scale the vectors. Given below is an example of a where we strech the y axis by a factor of 3.
If we scale only one axis by a factor of -1, then we basically get the reflection of that object along the corresponding axis.
Strech or squeeze differently on different axes
Every diagonal matrix can be broken down into a series of off-one matrices. Given below is an example where we break down a single diagonal matrix (to the left) into a series of matrices which when sequentially applied would give us the same result.
Slant along one or more axes
For example, a rectangle would slant into a parallelogram.
In 2D, there are 4 directions in which we can apply the shear transform. You can see them below.
Pristine Rotation
It has 3 important properties: 1. It is a square matrix: same size along all axes 2. All column vectors are unit vectors: If we take each column as a vector, then all of their distance from the origin should be 1. 3. All column vectors are orthogonal: The vector represented by column 1 is perpendicular to all other columns, which in this case is column 2.
This is the kind of a transform we would apply to rotate our rectangle clockwise:
{{<math.inline >}}
If I remember correctly, these strange numbers have something to do with (()).
Another interesting thing to note is that when we’re rotating something in 2D space ((x, y)), we’re actually rotating along a third (z) axis. {{</ math.inline >}}
Notice that this is the same matrix as shown above, just that we add another z axis as an identity.
And here’s some more example for you to develop an intuition:
In order to understand this, we first need to understand the concept of a subspace.
For such subspaces, there exists a transformation (read: matrix) which would move every point outside the subspace onto the subspace. Every vector always moves to it’s closest point on the subspace.
Reverses an existing trasformation
Note that a zero matrix and a projection matrix does not have an inverse. This is because when we squash a vector from a higher dimension to a lower dimension, there is a loss of information.
Resources