Difference between revisions of "Electronic music - notes on audio APIs"

From Helpful
Jump to: navigation, search
m (Hardware, and the nature of digital audio)
m ("So which API is best?")
Line 518: Line 518:
  
 
That's sort of the wrong question, because in lowest latency:
 
That's sort of the wrong question, because in lowest latency:
 +
 
* hardware is a factor
 
* hardware is a factor
: which may have some design fundamentals that add an unavoidable millisecond or two
+
:: which you have little control over
 +
:: which may have some design fundamentals that add an unavoidable millisecond or two
  
 
* the driver is a factor
 
* the driver is a factor
: which you have zero control over
+
:: which you have zero control over
: which may have some design fundamentals that add an unavoidable millisecond or two
+
:: which may have some design fundamentals that add an unavoidable millisecond or two
  
 
* the layers on top
 
* the layers on top
Line 530: Line 532:
 
* the mixer  
 
* the mixer  
 
:: which fundamentally adds 20-200ms, that ''is'' avoidable (but not practical in ''all'' situations)
 
:: which fundamentally adds 20-200ms, that ''is'' avoidable (but not practical in ''all'' situations)
 +
  
  
 
The API sits on top of the driver and hardware.
 
The API sits on top of the driver and hardware.
 
It doesn't say how well they did, only how you access it.
 
It doesn't say how well they did, only how you access it.
 +
 +
Sure, seeing 'ASIO' ''suggests'' that people thought about latency in the hardware and driver,
 +
but WDM/KS and WASAPI don't ''necessarily'' do worse.
  
  
Line 571: Line 577:
  
  
Twenty-five years later, while ASIO remains the ''shortest'' general answer to the question,
+
Twenty-five years later, while ASIO remains the ''easiest and shortest'' general answer to the question,
it isn't always much better.  
+
it isn't always much better.
 +
 
 
''If'' you can talk to it both via WASAPI and ASIO, you may find that the latency difference is very small.
 
''If'' you can talk to it both via WASAPI and ASIO, you may find that the latency difference is very small.
 
: If both ASIO and a windows sound API exist for hardware, the programmer was likely thinking of low latency, so the regular sound API (if present) is also likely to be pretty good).  
 
: If both ASIO and a windows sound API exist for hardware, the programmer was likely thinking of low latency, so the regular sound API (if present) is also likely to be pretty good).  

Revision as of 11:47, 23 May 2022

The physical and human spects dealing with audio, video, and images

Vision and color perception: objectively describing color · the eyes and the brain · physics, numbers, and (non)linearity · color spaces · references, links, and unsorted stuff

Image: file formats · noise reduction · halftoning, dithering · illuminant correction · Image descriptors · Reverse image search · image feature and contour detection · OCR · Image - unsorted

Video: format notes · encoding notes · On display speed


Audio physics and physiology: Basic sound physics · Human hearing, psychoacoustics · Descriptions used for sound and music

Digital sound and processing: capture, storage, reproduction · programming and codescs · some glossary · Audio and signal processing - unsorted stuff


Electronic music: Some history, ways of making noises · Gaming synth · on APIs (and latency) ··· microphones · studio and stage notes · Effects · sync ·

Music electronics: device voltage and impedance, audio and otherwise · amps and speakers · basic audio hacks · Simple ADCs and DACs · digital audio · multichannel and surround ·

Noise stuff: Stray signals and noise · sound-related noise names · electronic non-coupled noise names · electronic coupled noise · ground loop · strategies to avoid coupled noise · Sampling, reproduction, and transmission distortions · (tape) noise reduction


Unsorted: Visuals DIY · Signal analysis, modeling, processing (some audio, some more generic) · Music fingerprinting and identification

For more, see Category:Audio, video, images


Why latency exists (the long version)

Latency and physics

Latency in the real world exists because of distance and the speed of sound.


For some context of how long a millisecond is, and what distance does with sound:

a mic shoved guitar cab has maybe 1ms to get sound from speaker to mic.
talking to someone at one or two meters is 3ms to 6ms
a smallish practice space easily has ~15ms of delay from one wall to the other
opposite ends of a 15m bus would be 40ms
two frames in 24fps movie are 42ms apart
halfway across a sports field is easily 100ms

Which is all just physical length divided by the speed of sound. Use wolfram alpha if you're lazy.


So distance alone is

why bands may well watch their drummers
why in larger spaces you may want to use headphones instead (but not bluetooth ones)
one of a few reasons orchestras have conductors


In musical context

Hardware, and the nature of digital audio

Why larger buffers are generally useful

On drivers and APIs

Windows APIs

Some history

On ASIO wrappers

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)


ASIO usually refers to a driver talking directly to sound hardware, often exposing just the ASIO API, as much of the point is ignoring windows's sound architecture altogether.


ASIO wrappers are different (and in this context, the above is often called 'native ASIO' in contrast).

ASIO wrappers open a sound card via a regular Windows sound API (in practice typically WDM/KS or WASAPI), force settings that are lower latency (small buffer, exclusive if possible), and present it via ASIO API.

Yes, this is counter to ASIO's shortest-path-to-the-hardware principle, but there's still good reason to do it.

ASIO wrappers are usually about pushing underling hardware to lowest latencies.


Though yes, you will only get latencies that were always possibly to get from that underlying sound API anyway.

So why add a layer?

Convenience, mostly.

  • It lets you have just one place to configure a small-buffer, possibly-exclusive way to these existing APIs.
You figure out the details just once, in the wrapper's settings, rather than for every DAW-soundcard combination you have, which is usually more work, and the config details may vary somewhat between DAWs which is sometimes more fiddly and/or confusing.
and some programs (basically, things that are not the most common DAWs) may not allow all that tweaking - but can still talk ASIO
also not unimportantly, using that wrapper can also be easier to explain to people who care more about music than decades of programing history.
  • There's also some DAWs/software that speak mainly or only ASIO, because their approach is to figure out low latency in something external, and talk to that.


There's a few more useful reasons hiding in the details, like

  • you can often force WASAPI cards down to maybe say, 5-10ms without exclusive mode, which means you don't have to dedicate a sound card, to a DAW that only talks ASIO.
Which is good enough e.g. for when playing some piano on a laptop on the go, so pretty convenient
  • some ASIO wrappers can talk to on different sound cards for input and output, at the cost of slightly higher latency (will probably glitch at the lowest latencies), which DAWs talking native ASIO will typically refuse to do (for latency and glitch reasons).


As far as I can tell

  • ASIO4ALLv2 is a WDM/KS wrapper.
needs to force exclusive mode
can talk to different sound cards for input and output
  • FL Studio ASIO (a.k.a. FLASIO) is a WASAPI wrapper.
Comes with FL studio (including the demo), also usable in other DAWs
can talk to different sound cards for input and output
  • "Generic Low Latency ASIO Driver" is similar to ASIO4ALL but with different options
Comes with Cubase


And there appear to also be ASIO multiclient wrappers, basically ASIO in ASIO.

ftp://ftp.steinberg.net/Download/Hardware/ASIO_multiclient_driver/


"So which API is best?"

Linux APIs

Kernel level

Higher level

OSX APIs

Lowering latency

In general

tl;dr

zero latency does not exist, a few milliseconds of relative offsets happens all over the place
amounts of added latency can matter, though
latency matters when hearing yourself live, or syncing to something live (e.g. looper pedals)
digital input, output, and/or processing have some latency
In ways that are (looking at forums) usually partly misunderstood


Decide how low is necessary

The basic steps

API stuff

Windows API tweaking

Use a sound API designed for lower latency

Use sound API settings that lower the latency: exclusive mode

Use sound API settings that lower the latency: smaller buffer sizes

Linux API tweaking

OSX API tweaking

DAW considerations

Considering effects

Higher sample rates?

"Delay compensation?"

End-to-end latency

Further considerations

Hardware bus?

On network latency

Unsorted