I wonder why it was necessary for consumer modems to make these sounds. It's not really useful for debugging, so in the end it was just an annoying sound. Yes, I know you could often turn it off, but as far as I know the sound was always turned on by default.
Modem sounds were totally useful for troubleshooting. You'd know if your modem dialed someone's voice line, or answering machine, or a fax machine, or a busy signal, or "number not in service", or if it just rang with nothing answering, or if someone was already using the line for a voice call, or blessedly if it hooked up with an actual modem on the other end. You could also hear interruptions from someone else picking up a phone on the line or from call waiting; not all modem hardware would do it, but the relay on mine would audibly pulse.
The sounds of the actual connection weren't all that useful, but did at least indicate the negotiated speed between the modems (28800 and up does the kabing-kabing sound, 14400 and lower doesn't). So you'd know quickly if you hit a slow node on your BBS or AOL.
It was extremely useful for debugging. You could tell if you dialed the wrong number. You could tell if it was having trouble handshaking due to noise on the line. You could tell if the modem on the other side was slower than you expected.
Even non-technical users would get used to a certain sequence and could tell if things changed suddenly.
For what its worth, you definitely could identify the data rate before windows would report it. I used to listen and when I heard it connect at 26400, Id disconnect and retry until I heard and saw 33.6 or better. Not all of those trunklines were created equal back then.
I swear my practical peripherals 14.4kbps external modem was faster than my 28.8 and 33.6 at actual throughput (error correction maybe?). Never did the 56k analog.. went 128k ISDN. Having bonded channels was awesome. No cool analog sounds on that though...
It was totally useful for debugging. In addition to connection speed, you could also tell if your desired protocol was accepted by the computer you called. If it didn't support ZMODEM you could tell immediately by the sound of the test packets, and you wouldn't waste your time on a slow-protocol connection. That mattered when you had precious few hours in the middle of the night to download important stuff.