Color notes - physics, numbers, and (non)linearity

From Helpful
Jump to navigation Jump to search
The physical and human spects dealing with audio, video, and images

Vision and color perception: objectively describing color · the eyes and the brain · physics, numbers, and (non)linearity · color spaces · references, links, and unsorted stuff

Image: file formats · noise reduction · halftoning, dithering · illuminant correction · Image descriptors · Reverse image search · image feature and contour detection · OCR · Image - unsorted

Video: format notes · encoding notes · On display speed · Screen tearing and vsync


Audio physics and physiology: Sound physics and some human psychoacoustics · Descriptions used for sound and music

Noise stuff: Stray signals and noise · sound-related noise names · electronic non-coupled noise names · electronic coupled noise · ground loop · strategies to avoid coupled noise · Sampling, reproduction, and transmission distortions · (tape) noise reduction


Digital sound and processing: capture, storage, reproduction · on APIs (and latency) · programming and codecs · some glossary · Audio and signal processing - unsorted stuff

Music electronics: device voltage and impedance, audio and otherwise · amps and speakers · basic audio hacks · Simple ADCs and DACs · digital audio · multichannel and surround
On the stage side: microphones · studio and stage notes · Effects · sync


Electronic music:

Electronic music - musical terms
MIDI · Some history, ways of making noises · Gaming synth · microcontroller synth
Modular synth (eurorack, mostly):
sync · power supply · formats (physical, interconnects)
DAW: Ableton notes · MuLab notes · Mainstage notes


Unsorted: Visuals DIY · Signal analysis, modeling, processing (some audio, some more generic) · Music fingerprinting and identification

For more, see Category:Audio, video, images

Numbers and systems; color management

Digital color data consists of two things - numbers, and their colorful meaning. A color space maps between the two.


Each color space has different ideas of what part of XYZ to cover. This particularly makes sense for different media: while an image may look similar on screen and on paper, that's your eyes correcting via context - absolute-color-wise they are vastly different. (And you also need color management to translate them decently)

A color space may also care about the non-linear nature of our eyes. And should if it wants to blend colors in a natural way. A lot of software, even a lot of common and pricy image processing software, is surprisingly lazy about this.


When spending limited bits per pixel, there is also a tradeoff between range and resolution: a limited number of steps in a larger space means the steps are larger. The resolution is usually enough for this not to be noticeable, though.




Clarifications and the Meaning of Numbers

Luminance vs. Brightness

Luminance refers to the amount of energy that arrives per area. (often the relevant energy, e.g. photocameras will have filters to just accept visible light) (for a fuller story, see a sectionbelow)



How humans percieve luminant energy isn't linear at all, and we call this brightness.

You can say that ((delta-Intensity) / Intensity) is roughly constant, which is similar to the observations that

the brighter something is, the more absolute brightness we need to add to see a noticable brightness difference.
we are better at seeing the difference between dark parts of a scene than the light parts

One good approximation is that that perceived_brightness = log(energy). Square root is a decent estimation in a pinch.


(Accurate modeling is rather more interesting than that - to be more accurate than 'roughly proportional to log' you have to put people in front of colors and test them. This is where various measures of Just Noticeable Differences (JND) come from. People also use the term 'Difference Threshold'), to test the difference in intensity that means noticeable variation to an observer.)


The difference between luminance and brightness is an important practicality to, for example, cameras. Camera sensors are roughly linear with energy. In theory, creating an imaging device for human consumption might be done by just recording, storing, and displaying proportional to that energy we saw. In practice, various steps inbetween involve various conventions and corrections that make this a lot more interesting (but we can worry about that later).


Related physics

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

Going from physics to human perception involves a few targeted steps.


Starting from the physics end, there's

  • radiant flux: an energy source's strength (not its observation), measuring total EM radiation in Watt (W)
  • luminous intensity: radiant intensity in a frequency range of human interest (via a standardized model of the human eye). The unit is candela (cd)
  • luminance: luminant intensity per area (cd/m2)
  • lightness / brightness - the human response. This has varied definitions. Often it is defined based on luminance. This is various points of pragmatism combined.


There are more (with similar names), which seems confusing at first, but is mostly just different combinations of per angle, per second, per area per second; power at all, power filtered for humans, etc., which all happen to be more practical to different situations.



Physics and eyes

Regardless of the exact definition of lightness/brightness, one has to wonder: "Okay, so we took a frequency range and named it visible light. But what do our eyes actually do?"

Our eyes's rods and cones have sensitivities to varied ranges within the visible range, which only together cover the part of the electromagnetic spectrum that we have named visible light.

Which introduces color.


On the physics side, the objective way to record color is to sample how much energy there is at every frequency: a Spectrum Power Distribution (SPD) for the visible range.

This lets you reproduce without having human perception enter into it at all -- and this is useful in spectroscopy[1].

It's also a lot harder to implement, very redundant, and complete overkill when you only care about human viewing. (So cameras and reproduction cheat in human-specific ways, to be much easier and cheaper)


Also, the sensitivity to different frequencies varies quite a bit. We say we see wavelengths between 400-700nm, but near those ends our eye's response is almost insignificant. See the CIE luminosity function[2] for an idea of how much this varies.


Linear and nonlinear coding

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

One of areas a lot of people are confused about is the (non)linearity of signals.

REWRITING

Transfer functions

Gamma

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

REWRITING








See also:

Luminance and Chrominance versus Luma and Chroma

Luminance refers to the black and white part of a color image, chrominance to that which has to be added to get the intended colored image. The terms are used when an image is split this way, or to suggest thinking about it this way.


Luminance, broadly speaking, does not strongly indicate whether the coding is linear or non-linear.

Because many people, documents, and even standards use luminance in the linear, non-linear, or even both senses, without mention of possible confusion, terms like Luma and Chroma were introduced to carry the meaning of being nonlinear counterparts of luminance and chrominance.

They are not always used correctly either, so always pay attention.


Similarly, a prime generally indicates nonlinearity - e.g. Y indicatng linear luminance and Y' for luma.

However, many people are lazy and omit or forget it, so this is a convention at best and not a consistent way of recognizing what sort of channel you're dealing with. Read the standard to be sure.

See also

Gamma