As a web developer, learning music and audio programming makes my mind melt.
We often say "real time" when we mean "fast." But in audio real time means "really fast,
all the time" and somewhat deterministically.
If your tempo drifts, then you're not going to hear the rhythm correctly. If you have
a bit of latency on your instrument, it's like turning on a delay pedal where the only
signal coming through is the delay.
One might assume if you just follow audio programming guides then you can do all this, but
you still need to have your system setup to handle real time audio, in addition to your program.
> We often say "real time" when we mean "fast." But in audio real time means "really fast, all the time" and somewhat deterministically.
As a former developer of real time software, the usage of "real time" to mean "fast" makes me cringe a bit whenever I read it. If there's a TCP/IP stack in the middle of something, it's probably not "real time."
"real time" means there's a deadline. Soft real time means missing the deadline is a problem, possibly a bug, and quite bad. Hard real time means the "dead" part of "deadline" could be literal, either in terms of your program (a missed deadline is an irrecoverable error) or the humans that need the program to make the deadline are no longer alive.
And to demonstrate that hard realtime is not about speed, there is a whole hard-real time JVM implementation with GC and everything used in military contexts.
Modern computers are ridiculously fast, relatively speaking you don’t need much resources to calculate a missile trajectory, so “simply” 100% sure doing some calculations at a fixed rate, with even a GC cycle that has a deterministic higher bound (e.g. it will go throw the whole, non-resizable heap, but it will surely always take n seconds), you can pass the requirements. Though a desktop computer pretty much already begets the hard part of hard realtime, due to all the stuff that makes it fast - memory caching, CPU pipelining, branch prediction, normal OSs scheduling, etc.
What is it like working with hard real time? As in, is there some tooling to determine
how long a given function will take to run to meet a deadline? It starts to sound like
an NP problem, but we have other kinds of correctness and proof testing.
I suppose it's hard to make guarantees with different environments and hardware, but
I realized when we (non-realtime people) ship software we don't really have guarantees
for when our functions run.
It's just a very different programming model. You don't write potentially infinite loops, you allocate memory ahead of time, anything that might be potentially unbounded is handled asynchronously, etc. Things like I/O run on precise timers and so on.
Keep in mind you don't use the same operating systems (or often even the same hardware) in hard real time applications. You'll use a real time operating system (FreeRTOS, VxWorks, etc) with a very different task scheduler than you're probably used to with processes or threads in Unix-like platforms. That said, while multitasking exists for RTOSes in practice you're not going to be running nearly as many tasks on a device as say a web server.
You can get worst case performance of a section of code by guaranteeing that it has a fixed maximum number of operations (no infinite loops, basically). Of course the halting problem applies, but you're never concerned with solving it in the general case, just for critical sections. It gets tricky with OOO architectures but you can usually figure out a worst case performance.
Absolutely you can. With WebAsm SIMD you have near-native DSP performance. Downsides from my experience [1]:
- You are at the mercy of the browser. If browser engineers mess up the audio thread or garbage collection, even the most resilient web audio app breaks. It happens.
- Security mitigations prevent or restrict use of some useful APIs. For example, SharedArrayBuffer and high resolution clocks.
If your tempo drifts, then you're not going to hear the rhythm correctly. If you have a bit of latency on your instrument, it's like turning on a delay pedal where the only signal coming through is the delay.
One might assume if you just follow audio programming guides then you can do all this, but you still need to have your system setup to handle real time audio, in addition to your program.
It's all noticeable.