Electronic music - notes on audio APIs: Difference between revisions
mNo edit summary |
|||
(17 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
{{#addbodyclass:tag_tech}} | |||
{{#addbodyclass:tag_media}} | |||
{{avnotes}} | {{avnotes}} | ||
Line 483: | Line 486: | ||
===Windows APIs=== | ===Windows APIs=== | ||
====APIs in terms of latency==== | |||
<!-- | <!-- | ||
tl;dr: | |||
* 1 to 3ms of overall latency comes from hardware necessity ( | Latency tl;dr: | ||
: and after you bought | * what you can get | ||
:: not thinking about this gives you 30-100ms of overall latency | |||
:: with some specific tweaking you can expect 3-10ms out of a lot of modern hardware | |||
:: 1-3ms, without issues, is relatively rare | |||
* 1 to 3ms of overall latency comes from hardware necessity (a smaller part of which may ''also'' be hardcoded choices in the driver) | |||
: and after you bought hardware, you have ''no'' control over what it ''needs'' due to its design, and ''little'' over the driver | |||
:: hardware is what you bought | :: hardware is what you bought | ||
:: driver comes with your hardware | :: driver comes with your hardware | ||
Line 495: | Line 507: | ||
:: some parts of that are configurable | :: some parts of that are configurable | ||
:: though unless driver and app specifically expose these choices as settings, they are still effectively fixed | :: though unless driver and app specifically expose these choices as settings, they are still effectively fixed | ||
* newer sound subsystems have put more thought into design regarding lower latency, so allow lower latencies with less trouble | * newer sound subsystems have put more thought into design regarding lower latency, so allow lower latencies with less trouble | ||
Line 508: | Line 519: | ||
:: originally it was by far the easiest way to get low latency | :: originally it was by far the easiest way to get low latency | ||
:: though these days the difference may be smaller, sometimes almost nonexistent - but ASIO is still more of a known quantity | :: though these days the difference may be smaller, sometimes almost nonexistent - but ASIO is still more of a known quantity | ||
* if the sound API has a mixer, you can often get lower latency | * if the sound API has a mixer, you can often get lower latency | ||
Line 514: | Line 526: | ||
: how much better varies (e.g. the newer WASAPI can be pretty decent even in shared mode) | : how much better varies (e.g. the newer WASAPI can be pretty decent even in shared mode) | ||
* the next bit of | * the next bit of latency improvement is in lowering any buffers, particularly the client application's | ||
: at some point that'll start breaking up due to programs not always having samples soon enough. | : at some point that'll start breaking up due to programs not always having samples soon enough. | ||
:: this usually happens somewhere in the 2-5ms range | :: this usually happens somewhere in the 2-5ms range - you should assume apps increase to ~6ms even if the hardware can do 3ms | ||
--> | |||
====APIs that exist==== | |||
<!-- | |||
* '''Drivers and APIs are a source of a much of confusion, and more misinformation.''' | |||
: Which is quite understandable, because | |||
:: in Windows there is no clear-cut distinction between driver API and sound APIs, | |||
:: implementation details to most relevant APIs have changed over time and the fact is not always well documented | |||
:: some APIs will be there for compatibility, but now actually be wrapping others, so even information that was once very precise may now be wrong | |||
:: DAWs sometimes call things by friendlier names - of which the meaning is not always clear | |||
:: I'm still not all that clear sure where precisely which buffers are in which case | |||
* roughly: there is ASIO, Core Audio/WASAPI, what we call WDM/KS and WDM, DirectSound, MME | |||
:: ...which is actually a blend of lower-level and higher-level APIs, that are not 100% separable | |||
:: but you generally want an option earlier in that list | |||
* ASIO | * ASIO | ||
:: is a do-one-thing-and-do-it-well API | :: is a do-one-thing-and-do-it-well API | ||
Line 545: | Line 555: | ||
* WDM | * WDM | ||
: ''technically'' | : ''technically'' that acryonym means "the name for the package that all windows drivers for ''all'' hardware for the last two decades come in". | ||
: {{comment|(so yes, ASIO drivers are also WDM, in a 'technically' way that doesn't matter to this context)}} | : {{comment|(so yes, ASIO drivers are also WDM, in a 'technically' way that doesn't matter to this context)}} | ||
: in this context we usually 'WDM drivers for sound cards that provide a Windows sound API' (basically anything other than ASIO) | : in this context we usually 'WDM drivers for sound cards that provide a Windows sound API' (basically anything other than ASIO) | ||
Line 578: | Line 588: | ||
--> | --> | ||
====Some history==== | =====Some API history===== | ||
<!-- | <!-- | ||
Line 677: | Line 687: | ||
''' | {{zzz|For context| | ||
ASIO is an API that is completely separate from windows sound APIs. 'Doing it entirely our own way because it's more controlled' was a good deal of the point. Speaking ASIO to an ASIO driver that directly controls its hardware might be called '''native ASIO'''.}} | |||
{{zzz|For a little extra confusion, | {{zzz|For a little extra confusion, drivers that speak ASIO might...| | ||
* choose to expose ''only'' the ASIO API, because much of the point is ignoring all the and unknowns and variation that comes from windows APIs. | * choose to expose ''only'' the ASIO API, because much of the point is ignoring all the and unknowns and variation that comes from windows APIs. | ||
* ...or, sometimes, choose to expose both ASIO and a windows API | * ...or, sometimes, choose to expose both ASIO and a windows API -- more flexible in theory ('you can just use it in windows'), though can also somewhat more confusing in use and ends up with a set of maybe-true, maybe-superstitions, such as | ||
:: "it's better only when windows or some other software isn't also trying to use it" | |||
:: or "don't set it as the windows out or it will refuse to open as ASIO" | |||
:: or "don't set it as the windows out or windows out will stop when you open it as ASIO" | |||
:: or something else. | |||
}} | }} | ||
Given that context, you would think that "I speak ASIO" means "I provide ASIO capabilities for a specific audio device". | |||
'''ASIO wrappers''' have a different goal | '''ASIO wrappers''' speak ASIO, but have a different goal. | ||
ASIO wrappers open a regular (non-ASIO) sound card via a regular Windows sound API calls (in practice typically via WDM/KS or Core Audio's WASAPI), | ASIO wrappers open a regular (non-ASIO) sound card via a regular Windows sound API calls (in practice typically via WDM/KS or Core Audio's WASAPI), force some specific settings (e.g. smaller buffers, and exclusive mode where possible), | ||
force some specific settings (e.g. smaller buffers, and exclusive mode where possible), | then present that via the ASIO API. {{comment|(I guess you could also call them something like ASIO rerouters)}}. | ||
then present that via the ASIO API. | |||
Is this counter to ASIO's shortest-path-to-the-hardware principle? Yes. | Is this counter to ASIO's shortest-path-to-the-hardware principle? Yes. | ||
Line 700: | Line 714: | ||
Is there still good reason to do it? Actually yes. | Is there still good reason to do it? Actually yes. | ||
Line 747: | Line 763: | ||
* FlexASIO [https://github.com/dechamps/FlexASIO] | * FlexASIO [https://github.com/dechamps/FlexASIO] | ||
* ASIO Link | |||
: more focused on allowing {{imagesearch|asio link pro|more complex routing}} | |||
Line 1,002: | Line 1,019: | ||
Confusingly, some Audio APIs are tied closely to the underlying driver API, and some are not. | Confusingly, some Audio APIs are tied closely to the underlying driver API, and some are not. | ||
--> | |||
===="Why are some things called loopback?"==== | |||
<!-- | |||
Loopback, in a wider sense, tends to mean 'wire an output to input, probably on the same device'. | |||
Often, loopback is done to be able to record that output again. | |||
: (...ideally without creating a bunch of feedback) | |||
'''Windows audio devices called loopback''' | |||
Windows-API devices with {{inlinecode (looppack)}} at the end of the name | |||
are windows giving you a copy of whatever is being ''played'' on them. | |||
: basically windows saying "hey I'm putting data to device, you want some too?" | |||
: and distinguishing it from the recording the inputs that the device may (or may not) provide | |||
{{comment|As loopback could be used to mean other things, even on exactly the the same device, a better name for that feature might have been 'playback tap'. Whatever.}} | |||
It's a useful distinction - e.g. the Focusrite in front of me shows up as both | |||
* Speakers (Focusrite USB Audio) (loopback) | |||
:: (I called it Speakers, because reasons) | |||
* Analogue 1+2 (Focusrite USB Audio) | |||
'''Other things called loopback''' | |||
That Focusrite's settings also has something called loopback. What's that? | |||
...same idea. You have the options | |||
* analogue - recording from it presents its analog inputs -- default, and what you would usually expect it to do | |||
* playback - recording from it presents what is being played to the device | |||
* Custom mix - a mix of the above | |||
Some effect boxes like the VT-3 also have loopback. | |||
This one is more confusing, because it's thinking from its perspective. | |||
Also, it's a sound card for your PC -- and the effects part and PC part are handled separately. | |||
It's thinking about live performance, where the being-a-soundcard is primarily for the | |||
Line 1,144: | Line 1,215: | ||
* zero latency ''does not exist'', a few milliseconds of relative offsets happens ''all over the place'' | * zero latency ''does not exist'', a few milliseconds of relative offsets happens ''all over the place'' | ||
* amounts of ''added'' latency can matter, | * amounts of ''added'' latency can matter, and can be made ''low'' | ||
* latency matters when hearing yourself live, or syncing to something live (e.g. looper pedals) | * latency matters when hearing yourself live, or syncing to something live (e.g. looper pedals) | ||
Line 1,190: | Line 1,261: | ||
--> | --> | ||
==API stuff== | ==API stuff== | ||
===Windows API tweaking=== | ===Windows API tweaking=== | ||
Line 1,337: | Line 1,409: | ||
===="Delay compensation?"==== | ===="Delay compensation?"==== | ||
<!-- | <!-- | ||
Delay compensation (or 'latency conpensation') is not advanced magic sauce that lowers your latency. | Delay compensation (or 'latency conpensation') is not advanced magic sauce that lowers your latency. | ||
: Delay compensation does ''nothing'' to lower interface latency, | |||
: Delay compensation does ''nothing'' for you live. | |||
Delay compensation is some variation on the theme of | |||
"specific music-producing/processing things have different amounts of implicit latency. When we know it for all, we can align it better ''when they play together''". | |||
As a side effect of this feature ''reporting what it's measuring'' | |||
you may learn the latency of each, you may learn that disabling specific things, or simplifying specific processing, helps. | |||
But by itself, this often means that the slowest element is sets the standard, | |||
and everything else gets played later to align with it. | |||
Usually, that comes down to | Usually, that comes down to |
Latest revision as of 23:23, 21 April 2024
Why latency exists (the long version)
Latency and physics
Latency in the real world exists because of distance and the speed of sound.
The speed of sound is 343 meters per second.
Which is roughly 3 milliseconds per meter, so purely physically:
- a mic on a guitar cab has maybe 1ms to physically get sound from speaker to mic.
- talking to someone closeby is easily 5ms
- a small-to-moderate music practice space easily has maybe 15ms of delay from one wall to the other
- opposite ends of a 15m bus would be 40ms
- halfway across a sports field is easily 100ms
So distance alone is
- why bands may well watch their drummers
- why in larger spaces you may want to use headphones instead (but not bluetooth ones)
- one of a few reasons orchestras have conductors
Some other context:
- two frames in 60fps game are 17ms apart
- two frames in 24fps movie are 42ms apart
In musical context
Latency in hardware
The nature of digital audio
Why larger buffers are generally useful
When smaller buffers are required
Input, output, round-trip, software? On reported and actual latency
Measuring latency
On drivers and APIs
Windows APIs
APIs in terms of latency
APIs that exist
Some API history
On ASIO wrappers
ASIO is an API that is completely separate from windows sound APIs. 'Doing it entirely our own way because it's more controlled' was a good deal of the point. Speaking ASIO to an ASIO driver that directly controls its hardware might be called native ASIO.
- choose to expose only the ASIO API, because much of the point is ignoring all the and unknowns and variation that comes from windows APIs.
- ...or, sometimes, choose to expose both ASIO and a windows API -- more flexible in theory ('you can just use it in windows'), though can also somewhat more confusing in use and ends up with a set of maybe-true, maybe-superstitions, such as
- "it's better only when windows or some other software isn't also trying to use it"
- or "don't set it as the windows out or it will refuse to open as ASIO"
- or "don't set it as the windows out or windows out will stop when you open it as ASIO"
- or something else.
Given that context, you would think that "I speak ASIO" means "I provide ASIO capabilities for a specific audio device".
ASIO wrappers speak ASIO, but have a different goal.
ASIO wrappers open a regular (non-ASIO) sound card via a regular Windows sound API calls (in practice typically via WDM/KS or Core Audio's WASAPI), force some specific settings (e.g. smaller buffers, and exclusive mode where possible), then present that via the ASIO API. (I guess you could also call them something like ASIO rerouters).
Is this counter to ASIO's shortest-path-to-the-hardware principle? Yes.
Will it get you latencies better than that the underlying API could give you anyway? No.
Is there still good reason to do it? Actually yes.
"If you could get these latencies from the underlying API anyway, why add a layer?"
Convenience, mostly.
- it is easier for you to figure out the precise settings (small-buffer, possibly-exclusive) once,
- in one place - the wrapper's settings (doesn't change with how you use it), rather than for every DAW-soundcard combination you have
- that maybe you can save and restore later
- there is a clean separation between "speaks ASIO" and "does various latency-lowering tricks"
- in a way that anything that speaks ASIO can benefit from equally
- using that wrapper might make explanations a lot easier.
- Particularly towards people who just care for 'make it work decently' than reading up on decades of idiosyncratic programing history. (Per specific DAW where applicable)
- In fact some DAWs speak mainly or only ASIO, because their approach is to tell you to either get ASIO hardware, or if not, figure out low latency in something external, and talk to that.
- case in point: FL Studio, Cubase, MAGIX supply ASIO wrappers
There's a few more useful reasons hiding in the details, like
- you can often force WASAPI cards down to maybe 5-10ms output latency without exclusive mode
- ...which means you don't have to dedicate a sound card to a DAW that only talks ASIO.
- Which is good enough e.g. for when playing some piano on a laptop on the go, so pretty convenient
- some ASIO wrappers can talk to on different sound cards for input and output, at the cost of slightly higher latency (will probably glitch at the lowest latencies), which DAWs talking native ASIO will typically refuse to do (for latency and glitch reasons).
As far as I can tell
- ASIO4ALLv2 is a WDM/KS wrapper
- needs to force exclusive mode
- can talk to different sound cards for input and output
- FL Studio ASIO (a.k.a. FLASIO) is a WASAPI wrapper
- Comes with FL studio (including its demo), and is perfectly usable in other DAWs
- can talk to different sound cards for input and output
- "Generic Low Latency ASIO Driver" is similar to ASIO4ALL but with different options
- Comes with Cubase
- MAGIX ASIO - (verify)
- ASIO2KS [1]
- ASIO2WASAPI [2]
- FlexASIO [3]
- ASIO Link
- more focused on allowing more complex routing
Some of them are multiclient - allow multiple ASIO clients to connect to the same ASIO device (at the cost of a little more latency than if you stay exclusive)
A few cases are only that, e.g. Steinberg's multiclient driver[4] seems nothing beyond ASIO in ASIO.
There is further software that happens to present its result as ASIO, sometimes more for options than for latency reasons
- VOICEMEETER also presents its result as ASIO
On custom ASIO drivers
In theory, you can write an ASIO driver for any hardware.
It won't do you too much good if the hardware isn't designed for low latency, It won't be one installed automatically, and these days there may be less reason to do so (in that you can get similar latency with Core Audio/WASAPI).
...but e.g. the Kx project, that made a specific series of Sound Blaster cards more capable, also included ASIO support, and would let the cards do on the order of 5ms.
...and I have a handheld recorder that, while fairly cheap, happens to also act as a good low-latency ASIO sound card - when the drivers are installed, anyway.
Some audio interfaces can speak both a regular API and ASIO.
It's more accomodating, but also potentially more confusing, because you cannot talk ASIO while the regular.
And if windows decides it's first in the list of cards, that means you need to poke settings before it does what you think it should.
"So which API is best?"
"Why are some things called loopback?"
Linux APIs
Kernel level
Higher level
OSX APIs
Because OSX was a relatively clean slate at the time, at lower levels there is primarily Core Audio (also present in iOS, slimmed down(verify)), which has been quite capable for a long while.
https://developer.apple.com/audio/
Lowering latency
In general
tl;dr
- zero latency does not exist, a few milliseconds of relative offsets happens all over the place
- amounts of added latency can matter, and can be made low
- latency matters when hearing yourself live, or syncing to something live (e.g. looper pedals)
- digital input, output, and/or processing have some latency
- In ways that are (looking at forums) usually partly misunderstood