Lot physics here. Actually pure physics here. This is my layman understanding. But a person who deals with speaker design will explain it better. You need to understand resonance. Resonance occurs widely in nature, and is exploited in many devices. It is the mechanism by which virtually all
sinusoidal waves and vibrations are generated. Many sounds we hear, such as when hard objects of
metal,
glass, or
wood are struck, are caused by brief resonant vibrations in the object. By varying the size of the speaker cone, we are actually exploiting resonance naturally.
1. Everything that can vibrate has something known as
resonant frequency. Large objects have low resonant frequency. Small objects have higher resonant frequency. Take an example of a very long rope with both ends tied to a wall. Hit the rope with your finger and let go and it will vibrate slowly and will vibrate for very long. Compare this with a rope of very small length tied at two ends. Hit it with a finger and let go. It will vibrate fast and die fast. Now if you got this, look at any guitar or string instrument and pluck the string. The longest wire will make a sound of lower frequency and will produce sound for longer compared to the shortest string. Take example of a large drum (which gives bass) or cello. The sound from these will be at lower frequency and will last longer. Take example of percussion instruments, sound of keys jingling, the frequency will be high but will die very fast. Take a large gong and it it with a hammer, the sound will be at lower frequency and will last long. Now we just discussed things which produce sound.
2. Now speakers come with various sizes. If you see the woofer has much larger diameter. They are meant to produce sound of lower frequency. That's the reason we have to make it large. They are not meant to be fast and they cannot move fast. That makes them excellent to reproduce low frequency sound and since they move slowly, they will reproduce accurately the decay of low frequency producing objects like the drum, bass guitar, etc. If you have a fast moving woofer (which is physically impossible because of large size), they will ruin the sound and decay. So in reality you don't actually require fast bass. It is a misnomer actually. Similarly you don't need a tweeter to move slowly (which is again physically impossible because of small size). A slow moving tweeter will ruin reproduction of high frequency sound.
3. Coming back to point 1, it is physically impossible to make a long rope move up and down fast enough and also impossibel to make a short rope move up and down slowly. If you want to suddenly stop the larger rope fast, you will require lot of power to do that. You will litterally have to use both your hands and hold the rope with your fists to stop that. This is also important for large suspension bridge where you need to prevent it from vibrating. If it starts vibrating up and down it is physically impossible to stop it and the bridge will collapse.
4. From point 3, we can conclude that woofer cannot reproduce high frequency and tweeter cannot reproduce low frequency. But the output of the amplifer has all frequencies mixed in the two wires. So what do we do. We use a cross over to separate the low frequency and feed that to the woofer and high frequency to the tweeter. You can in fact make it better by make a speaker 3 way or 4 way with drivers of different sizes. The largest for low frequency, the middle ones for mid and the smallest diameter driver for high frequency.
5. Now sometimes you want the low frequency decay to be controlled. Like you hit a drum with the stick once and place your stretched palm on the drum skin to stop the decay. Now if this is fed to your loud speaker, the speaker will keep vibrating and will not reproduce the sudden stop of the decay. Just like you used the palm on the drum skin to stop the bass suddenly, the amplifer should be able to damp the sound suddenly when there is no low frequency component coming in the signal. Here is where the damping factor comes in.
6. In
loudspeaker systems, the value of the damping factor between a particular loudspeaker and a particular
amplifier describes the ability of the amplifier
to control undesirable movement of the speaker cone near the
resonant frequency of the speaker system. It is usually used in the context of low-frequency driver behavior, and especially so in the case of electrodynamic drivers, which use a magnetic
motor to generate the forces which move the
diaphragm. So when we talk about fast bass, it is the amplifier that is responsible. It should be powerful enough and with a good damping factor. It is not the job of the woofer to be fast, that is the job of the amp. A woofer that is fast will be very very very bad emasculated woofer.
7. Another important thing is that the sound is moving at the same speed whether it is bass or non-bass. They are all moving at the speed of around 343 m/s. So nothing is moving slower or faster than the song.
It is the same reason with voices. Men have longer vocal chords and hence they cannot sing like Lata. Women have shorter vocal chords and it is impossible for them to sing like this
https://www.classicfm.com/artists/tim-storms/lowest-vocal-note-world-record/
Lions cannot chirp or tweet and birds cannot growl. A woofer shouldn't be fast enough to tweet and a tweeter shouldn't be slow enough to woof.