Difference between revisions of "Color notes - physics, numbers, and (non)linearity"

From Helpful
Jump to: navigation, search
m (==)
Line 278: Line 278:
In particular, compressed images typically store [[brightness]] rather than [[luminescence]],
In particular, compressed images typically store [[brightness]] rather than [[luminance]],
which are related via a power law.
which are related via a power law.

Revision as of 12:25, 21 May 2022

The physical and human spects dealing with audio, video, and images

Vision and color perception: objectively describing color · the eyes and the brain · physics, numbers, and (non)linearity · color spaces · references, links, and unsorted stuff

Image: file formats · noise reduction · halftoning, dithering · illuminant correction · Image descriptors · Reverse image search · image feature and contour detection · OCR · Image - unsorted

Video: format notes · encoding notes · On display speed

Audio physics and physiology: Basic sound physics · Human hearing, psychoacoustics · Descriptions used for sound and music

Digital sound and processing: capture, storage, reproduction · programming and codescs · some glossary · Audio and signal processing - unsorted stuff

Electronic music: Some history, ways of making noises · Gaming synth · on APIs (and latency) ··· microphones · studio and stage notes · Effects · sync ·

Music electronics: device voltage and impedance, audio and otherwise · amps and speakers · basic audio hacks · Simple ADCs and DACs · digital audio · multichannel and surround ·

Noise stuff: Stray signals and noise · sound-related noise names · electronic non-coupled noise names · electronic coupled noise · ground loop · strategies to avoid coupled noise · Sampling, reproduction, and transmission distortions · (tape) noise reduction

Unsorted: Visuals DIY · Signal analysis, modeling, processing (some audio, some more generic) · Music fingerprinting and identification

For more, see Category:Audio, video, images

Numbers and systems; color management

Digital color data consists of two things - numbers, and their colorful meaning. A color space maps between the two.

Each color space has different ideas of what part of XYZ to cover. This particularly makes sense for different media: while an image may look similar on screen and on paper, that's your eyes correcting via context - absolute-color-wise they are vastly different. (And you also need color management to translate them decently)

A color space may also care about the non-linear nature of our eyes. And should if it wants to blend colors in a natural way. A lot of software, even a lot of common and pricy image processing software, is surprisingly lazy about this.

When spending limited bits per pixel, there is also a tradeoff between range and resolution: a limited number of steps in a larger space means the steps are larger. The resolution is usually enough for this not to be noticeable, though.

Clarifications and the Meaning of Numbers

Luminance vs. Brightness

Luminance refers to the amount of relevant energy that arrives (photons for photocameras) per area. (for a fuller story, see a sectionbelow)

Human-perceived brightness isn't linear at all.

You can say that ((delta-Intensity) / Intensity) is roughly constant, which is similar to the observations that

the brighter something is, the more absolute brightness we need to add to see a noticable brightness difference.
we are better at seeing the difference between dark parts of a scene than the light parts

One good approximation is that that perceived_brightness = log(energy). Square root is a decent estimation in a pinch.

(Accurate modeling is rather more interesting than that - to be more accurate than 'roughly proportional to log' you have to put people in front of colors and test them. This is where various measures of Just Noticeable Differences (JND) come from. People also use the term 'Difference Threshold'), to test the difference in intensity that means noticeable variation to an observer.)

e.g. Camera sensors are roughly linear. If we care to e.g. display photos, then monitor output has to, on the actual display, output light proportional to how much luminosity we originally saw (the various steps inbetween involve various conventions and corrections, but we can worry about that later).

Related physics

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

Going from physics to human perception involves a few targeted steps.

Starting from the physics end, there's

  • radiant flux: an energy source's strength (not its observation), measuring total EM radiation in Watt (W)
  • luminous intensity: radiant intensity in a frequency range of human interest (via a standardized model of the human eye). The unit is candela (cd)
  • luminance: luminant intensity per area (cd/m2)
  • lightness / brightness - the human response. This has varied definitions. Often it is defined based on luminance. This is various points of pragmatism combined.

Yes, there are more, which is a bit confusing, until you realize it's just different combinations of per angle, per second, per area per second; power at all, power filtered for humans, etc.

Regardless of the exact definition of lightness/brightness, the concept behind it is: "Okay, so we took a frequency range and named it visible light. But what do our eyes actually do?". Our eyes's rods and cones have varied sensitivities, which only together cover the part of the electromagnetic spectrum that we have named visible light.

Which introduces color.

On the physics side, the objective way to record color is to sample how much energy there is at each frequency: a Spectrum Power Distribution (SPD) in the visible light frequencies. This lets you reproduce without having human perception enter into it at all -- and this is useful in spectroscopy[1].

Yet when the target is only human eyes, then SPDs are rather redundant, rather bulky information, and the fact that they are much harder to implement in a camera doesn't help.

Also, the sensitivity to different frequencies varies quite a bit. We say we see wavelengths between 400-700nm, but near those ends our eye's response is almost insignificant. See the CIE luminosity function[2] for an idea of how much.

Linear and nonlinear coding

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

One of areas a lot of people are confused about is the (non)linearity of signals.


Transfer functions


This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)


See also:

Luminance and Chrominance versus Luma and Chroma

Luminance refers to the black and white part of a color image, chrominance to that which has to be added to get the intended colored image. The terms are used when an image is split this way, or to suggest thinking about it this way.

Luminance, broadly speaking, does not strongly indicate whether the coding is linear or non-linear.

Because many people, documents, and even standards use luminance in the linear, non-linear, or even both senses, without mention of possible confusion, terms like Luma and Chroma were introduced to carry the meaning of being nonlinear counterparts of luminance and chrominance.

They are not always used correctly either, so always pay attention.

Similarly, a prime generally indicates nonlinearity - e.g. Y indicatng linear luminance and Y' for luma.

However, many people are lazy and omit or forget it, so this is a convention at best and not a consistent way of recognizing what sort of channel you're dealing with. Read the standard to be sure.

See also