At the end of these types of popsci explanations you still can't grok the math any better and you end up with a hand-wave-y misleading understanding of what's going on.
It starts off by saying that the Fourier Transform "measures every possible cycle". This is an wrong approach to thinking of the Fourier Transform. If you think in these terms you will quickly get lost. The Fourier basis is a set of sinusoids that are mutually orthogonal (ie. none can be represented as a combination of the others). They just happen to generally cover the frequency ranges you're interested in (though this depends on your problem). If you choose some random sinusoidal signal that isn't a harmonic of your sampling period, then the Fourier Transform gives you back a mess that's hard to interpret (how a off-harmonic sinusoid gets "smeared" across the basis is actually something I frankly still don't quite understand https://ccrma.stanford.edu/~jos/mdft/Frequencies_Cracks.html )
This is in contrast to the phase information that you get that is continuous. Each frequency has two associated basis vectors and they can give you any phase offset. The frequencies are only set harmonics, but the phase can be any value. This is the result of how the basis is setup. If anything, the phase information you get from the Fourier transform is more interesting than the frequency information you can glean.
The "Circular Paths" visualization is also highly misleading. The complex plane is a crutch that gives you pretty pictures. It pretends to be like a 2D plane but it's actually fundamentally distinct from a Cartesian/"normal" X-Y plane b/c complex numbers can be multiplied while normal [x,y] pairs can not (b/c the operation is simply undefined - you can't multiply two points on a map). The product of two complex numbers is giving you another complex number.. but good luck visualizing where that point ends up! You'll notice all complex plane examples stay on the unit circle b/c you get a "rotation" but it's an edge case. The whole 2D analogy is setting you up to be very confused
Why is there always this pushback against "popsci explanations" in these threads? Do people not realize every brain is unique and people have different learning styles? A few hours of 3blue1brown videos gave me a useful intuition about FT, FFT, quaternions, and other mathematical things that years of courses with equations and homework failed to do.
I'm talking real, useful intuition that improved my work in signals processing, using wavelets, filters, convolutions etc.
And yes I can visualize 3.5D pretty easily in my head: a 3D volume with layers and colors that overlap. Though usually it's easier to picture two 2D frames side-by-side.
I don't fault people when they grok words/equations but visual diagrams do nothing for them (blueprints/CAD are an enigma to my partner). Don't fault people for needing visualization to grok equations.
I didn't mean to imply that there is anything wrong with visualization. I feel they're easier to understand than equations for the vast majority of people. But as a concrete example, to understand the Fourier transform it probably took me a couple of weeks of reading different textbooks in my spare time to really grok how the math works here.. it took some effort and I'm not some autistic savant that sees things in equations - so it has nothing to do with learning styles.
Looking at some diagrams and videos for certain topics is a lazy and harmful approach to learning because some problems are just not conducive to visualization. Efforts to force a visualization ends up being actively bad and give viewers the wrong idea. My comment was a point by point explanation of how this ends up happening. At least in this particular case, with the Fourier Transform, the visualizations in effect made it harder for me personally to understand how the math works b/c they are so sloppy and emphasis all the wrong points.
In the grand scheme of things, having a half-assed popsci understanding is generally worse than not know anything. You can get a sense of overconfidence and then go on to apply these techniques incorrectly and make broken/fragile systems that make no mathematical sense (or publish insane papers)
If you really work with signal processing and don't understand the mathematics of the fourier transform then that's frankly a little scary. I'd really recommend taking the time to understand it b/c I think it really pays off - but yes, it takes a bit more effort than watching a youtube video..
>But as a concrete example, to understand the Fourier transform it probably took me a couple of weeks of reading different textbooks in my spare time to really grok how the math works here.. it took some effort and I'm not some autistic savant that sees things in equations - so it has nothing to do with learning styles.
So you're still claiming your experience covers all.
When did I say anything about not understanding the math of the FT? I took a whole semester of classes in signals, control theory, comms, etc. Derived FT, FFT, and friends, from scratch. I understand the math. But I have limited intuition about what I can do with the math. I can look it up in a book, or plug and chug, but it's not very fluid. The amount of times I've needed to manipulate equations since that class is zero.
>Looking at some diagrams and videos for certain topics is a lazy and harmful approach to learning
god my eyes can't roll farther back in my head. your point would be less absurd if it weren't for the fact that algebraic geometry is an enormously powerful and productive area of math whose whole ass founding principle is that geometric interpretations of algebraic objects are extremely useful.
AND
spectral theory (what fourier analysis is called today) is an extremely geometric area owing to the dependence of convergence on the metric.
>In the grand scheme of things, having a half-assed popsci understanding is generally worse than not know anything
yes because this is a resource for rocket scientists and not interested people (probably high school and college students)?
>If you really work with signal processing and don't understand the mathematics of the fourier transform then that's frankly a little scary.
bro this isn't oppenheim. it's just a nice intro for people that might be curious or are learning for the first time.
>Why is there always this pushback against "popsci explanations" in these threads?
because underneath all of the rhetoric about entrepreneurialism and "disrupting" thing tech (and by extension HN) is one of the most conservative and elitest communities that has ever existed. think about what people used to say about dynamic languages vs statically typed, about frontend vs backend, about sql vs nosql, etc etc etc. stodgy old engineers have always (and will always) resist the bar being lowered and newcomers being admitted.
edit:
what's funny is that the opposite response shows up, invariably, on posts that are very precise and technical. i.e. something like "this is too technical i wish mathematicians would accommodate us lay folk".
> stodgy old engineers have always (and will always) resist the bar being lowered and newcomers being admitted.
I try not be elitist or unwelcoming to newcomers, but yeah uh why should we lower the bar? Are the newcomers somehow incapable of learning engineering at the same level that the old ones did? Its no fun to work with people who aren't as motivated and cry for a lower bar instead of rising to the challenge. Its easy to work with someone more motivated than you but it sucks to work with someone less motivated.
> The complex plane is a crutch that gives you pretty pictures.
That’s a very poor outlook on an incredibly useful mathematical tool. Maths is done essentially by finding new ways of viewing an old thing, and restricting the set of tools available seems like a poor idea in general. As for teaching mathematics, finding more ways of looking at something, however imprecise, is a huge aid to understanding. Never be afraid to show someone a picture that’s only 80% true but could potentially aid their understanding in how something works!
I also disagree that the “circular paths” visualisation is misleading - what is misleading about it? This is a completely accurate representation of how a Fourier series is added up to get a real part which varies over time. It even makes it intuitive that multiplying the series by a fixed complex number (corresponding to rotating and scaling the “orbiting” points on the left) will have the effect of translating the resulting signal to the right or the left.
Also for what it’s worth, the product of complex numbers is easy to visualise? Add their angles, multiply their magnitudes.
B/c it actually adding a layer of complexity that's not necessary for the sake of getting a pretty picture. I'll try to illustrate..
The roots of unity are just trivially observed mathematically
Through Euler's identity: e^i2π = cos(2π) + isin(2π) = 1
then obviously (e^i2π/b)^b = 1
So then e^i2π/b is a b'th root of 1. Done!
You don't need a complex plane.. You don't need to explain why you're multiplying 2D points.. and then explain that now you actually have this new operation defined and it's not really a 2D point oh and btw this multiplication is giving you a rotation.. which rotates your point back to 1+i0 (which is maybe the crummiest part b/c 2D rotations are not intuitive and visualizations should leverage our "gut feeling" and intuition). You just skip all of that! The equations are easy. Everyone knows (A^b)^c = A^bc
Then you show that (e^i2π/b)^(b+1) = (e^i2π/b) - so it's "cyclical", the equations starts returning previous values at higher integer inputs.
And then you show orthogonality between basis vector by just taking two basis vectors and doing a dot product. If you write out an example it's really easy to see how the values pair off and cancel out. There is no easy visual equivalent. You can't see how a real/imaginary sinusoid pair is orthogonal to another.. If you try to force a visualization it's only going to be more confusing.
And actually, b/c it's hard to visualize this last step is usually sorta glossed over in intro material. But I'd argue it's the most important step b/c it's the whole reason why are we choosing these extra-challenging e to the i things to build a basis... it's for this orthogonality. If we just wanted sinusoids that are cyclical then we'd be doing everything with real numbers and keeping it simple. You'd cross correlate a sinusoid at different frequencies for instance or something to that effect... but the fourier transform is giving you something much better than that. And as a bonus you see why the chosen frequencies are discrete and not arbitrary.
All operations have to be defined, and learned first. That's like saying that you can't multiply two matrices... (Though it does annoy me when a symmetric symbol is used for non-commutative operations !)
Oh woah, I've never seen that visualization - very cool. I only meant to really highlight that often things are thrown on the complex plane and presented in introductory material as making the problem in 2D. The few times I've seen this (physics, math and signal processing classes) I've never actually seen it emphasized anywhere that the complex plane is fundamentally different from a cartesian 2D plane. It's the crux of the whole spinning unit circle stuff and then why you get orthogonality
I still find the visualization completely unnecessary b/c it doesn't help illustrate or give any kind of intuition for why the basis vectors are orthogonal
Is what bothers you that the complex plane has this very specific feature that purely imaginary x purely imaginary = real, so you're "mixing" orthogonal dimensions ?
However, I still don't understand your reticence, since imaginary numbers are fundamentally about cycles and rotations (via exponentiation), and rotations fundamentally require having 2 dimensions to represent them : hence a plane !
(Like you say, you also need 2 dimensions to be able to have orthogonality...)
And I'm getting a bit rusty here, but I guess that imaginary being orthogonal to real might be why these Fourier basis vectors are orthogonal ?
I mean, it's literally Euler's formula :
exp(𝛕iθ) = cos(𝛕θ) + i sin(𝛕θ)
I don't remember, aren't those 2 orthogonal basis vectors "cos" and "sin" ?
I guess what bothers me is that visualizations are supposed to leverage our intuition and gut feeling. Like in Geometry, the teacher tell you that when the lines are parallel, the angels on on a bisecting line are the same. You "see" it on the image and it immediately makes sense.
When you have a weird new operation that doesn't behave in any relatable way then the visualization is effectively useless. I explained a bit further here how you can completely skip this pseduo-2D stuff and things remain very clear:
Maybe the 2D plane is useful in other contexts and it's an analogy that's worth internalizing - I don't wanna knock it entirely - but in the case of Fourier Transform it's a high cognitive load for something that's actually not that important and easy demonstrated from the roots of unity.
And the net effect is that the stuff that can't be visualized, and that is arguably more important, is glossed over b/c it's not pretty enough
It starts off by saying that the Fourier Transform "measures every possible cycle". This is an wrong approach to thinking of the Fourier Transform. If you think in these terms you will quickly get lost. The Fourier basis is a set of sinusoids that are mutually orthogonal (ie. none can be represented as a combination of the others). They just happen to generally cover the frequency ranges you're interested in (though this depends on your problem). If you choose some random sinusoidal signal that isn't a harmonic of your sampling period, then the Fourier Transform gives you back a mess that's hard to interpret (how a off-harmonic sinusoid gets "smeared" across the basis is actually something I frankly still don't quite understand https://ccrma.stanford.edu/~jos/mdft/Frequencies_Cracks.html )
This is in contrast to the phase information that you get that is continuous. Each frequency has two associated basis vectors and they can give you any phase offset. The frequencies are only set harmonics, but the phase can be any value. This is the result of how the basis is setup. If anything, the phase information you get from the Fourier transform is more interesting than the frequency information you can glean.
The "Circular Paths" visualization is also highly misleading. The complex plane is a crutch that gives you pretty pictures. It pretends to be like a 2D plane but it's actually fundamentally distinct from a Cartesian/"normal" X-Y plane b/c complex numbers can be multiplied while normal [x,y] pairs can not (b/c the operation is simply undefined - you can't multiply two points on a map). The product of two complex numbers is giving you another complex number.. but good luck visualizing where that point ends up! You'll notice all complex plane examples stay on the unit circle b/c you get a "rotation" but it's an edge case. The whole 2D analogy is setting you up to be very confused