Signals and Systems Part 2: Fourier Series
Lesson 2: Orthogonal Functions
Wait, functions can be orthogonal?
Yes! In the last lesson on the inner product, we saw that we can extend the idea of the Dot product to the inner product, which lets us ask questions like: how much of one function is contained within another? But just as vectors can be orthogonal to each other, so too can functions. For vectors, recall orthogonality just means they point in completely different (orthogonal) directions, like these vectors \(\overrightarrow{a}\) and \(\overrightarrow{b}\) below:
We can test whether two vectors are orthogonal by taking their dot product - if that dot product is zero, then the vectors are said to be orthogonal. Same for functions, but replace 'dot product' with 'inner product'. We can get some intuition for what it means for signals to be 'orthogonal' with a couple examples:
What is the inner product of the two functions above (one plotted in orange, the other blue)?
Solution
We can use the definition of the inner product, which is just the integral of the two functions multiplied together. But since the function in blue is zero when the function in orange is 1, and vice versa, multiplying the two together is zero everywhere, so the inner product must be zero.
Trading-off being Zero
From this example we can see one condition that lets functions be orthogonal - if one is zero when the other is not, and vice versa. This kinda make sesnse, because each function never 'sees' the other one. They never have any common ground, and so they are orthogonal. But there's other cases when functions can be orthogonal, too, even if they have some overlap:
Find the inner product of the two functinos plotted above (one in orange, the other blue). What is its value?
Solution
If we multiply the two functions together, we just get a scaled version of the original squarewave. But the positive parts are exactly canceled out by the negative parts, so the inner product is zero. This shows us another way two functions can be orthogonal, if the positive parts of the inner product cancel the negative ones. One interesting inner product, which was an example in the previous lesson, is sinewaves of different frequencies, for example, \(sin(x)\) and \(sin(2x)\). It turns out that these functions are orthogonal, too. The reason for this is exactly the same as in the example above - when you multiply the two functions and integrate, the positive parts are exactly canceled out by the negative ones.
In fact, it turns out that the inner product of \(sin(x)\) with any other sinewave with an integer multiple of the save frequency (so \(sin(2x), sin(3x)\), etc.) is orthogonal to \(sin(x)\). This will turn out to be very useful, because it means you can build very interesting things out of sinewaves. In fact, it turns out you can build anything.
If you found this content helpful, it would mean a lot to me if you would support me on Patreon. Help keep this content ad-free, get access to my Discord server, exclusive content, and receive my personal thanks for as little as $2. :)
Become a Patron!