Illuminant correction

From Helpful
(Redirected from Retinex)
Jump to navigation Jump to search
The physical and human spects dealing with audio, video, and images

Vision and color perception: objectively describing color · the eyes and the brain · physics, numbers, and (non)linearity · color spaces · references, links, and unsorted stuff

Image: file formats · noise reduction · halftoning, dithering · illuminant correction · Image descriptors · Reverse image search · image feature and contour detection · OCR · Image - unsorted

Video: format notes · encoding notes · On display speed · Screen tearing and vsync


Audio physics and physiology: Sound physics and some human psychoacoustics · Descriptions used for sound and music

Noise stuff: Stray signals and noise · sound-related noise names · electronic non-coupled noise names · electronic coupled noise · ground loop · strategies to avoid coupled noise · Sampling, reproduction, and transmission distortions · (tape) noise reduction


Digital sound and processing: capture, storage, reproduction · on APIs (and latency) · programming and codecs · some glossary · Audio and signal processing - unsorted stuff

Music electronics: device voltage and impedance, audio and otherwise · amps and speakers · basic audio hacks · Simple ADCs and DACs · digital audio · multichannel and surround
On the stage side: microphones · studio and stage notes · Effects · sync


Electronic music:

Electronic music - musical terms
MIDI · Some history, ways of making noises · Gaming synth · microcontroller synth
Modular synth (eurorack, mostly):
sync · power supply · formats (physical, interconnects)
DAW: Ableton notes · MuLab notes · Mainstage notes


Unsorted: Visuals DIY · Signal analysis, modeling, processing (some audio, some more generic) · Music fingerprinting and identification

For more, see Category:Audio, video, images

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)


Automatic illuminant correction

The intent is usually to take out any mild tint that the illuminant has, or to correct a camera's mis-estimation of the illiminant.

In other words, mainly white balance correction, a.k.a. gray balance.


Color correction often comes down to

  • estimating what the illuminant probably was
photographers who cares about color accuracy tend to use a gray card to have a known-absolute reference.
without one, it's based on assumptions
and incorrect assumptions can introduce unnatural tinting in the process
  • apply chromatic adaptation so that that illuminant effectively becomes a given illuminant, such as D65 (mid-day sun), or just numerically equalize channels.


There are some more specific cases you could focus on, such as cases where you know color filters were used, or e.g. faded photographs could consider the dyes in use - their relative fading is usually documented.


Gray world

The idea: in a well balanced image, the average color is a neutral gray.


So the correction scales channels to makes the average become gray, typically implemented by making the average of the red, green, and blue the same, often by a basic linear gain.


The results vary a little with in which color space you do this.

Gray world makes decent sense when large patches are expected to be neutral, which is e.g. true in darker photographs.


Pro:

  • Simple
  • works well on removing the illuminant's tint on images with a lot of white, a lot of dark areas, and/or a lot of each color, or a photograph which had such a neutrality but had a tint applied to it


Con:

  • Only true for images with a roughly white illuminant
if tinted, assumes that you want to fully negate that tint. Consider e.g. an image intentionally taken with monochrome red illuminant - we will effectively just ramp up green and blue - of which there was very little (possibly only negligible sensor-noise levels), and since we handle the channels independently so it likely comes out quite unnatural
that specific issue can be tempered by limiting that gain
  • effectively equally uses all of the image for measurement
an assumption that is flawed to varying degrees



Auto levels

The idea: The brightest color should be white, the darkest color should be black.

So: independently rescale each channel's histogram to span the full range, accepting that some (e.g. 1%) will lie outside and be truncated.


Pro:

  • brightest color becomes white, which on outside photos is often good enough
  • somewhat less sensitive to monochrome-illuminants than gray world

Con:

  • suffers from similar problems to gray world
  • not sensitive to how much small the brightest pixel-area is. For example, having one tiny bright window in the background means a huge white wall will be ignored.

Retinex

Retinex is in itself a wider theory dealing with various color constancy effects, also dealing with local context and some human interpretation.


In the context of whole-image illuminant/color correction, it usually points to the fact that perceived white tends towards the strongest cone signal.

This is largely a gentler form of gray world, referring to the overall effects rather than a single spot.


It roughly means that the maximum within each channel should be the same.


While image-pixel-RGB doesn't correspond precisely to color perception, that implementation is close enough to work pretty well.

The correction could be implemented as just (linear) gain on each channel to make the maxima the same, though in practice using a very-high-percentile point can be more robust, by ignoring a small amount of potential outlier pixels.


Pro:

  • still a fairly simple idea and simple code
  • better behaved than gray world, in that it avoids many larger color shifts


Con:

  • sometimes too cautious
e.g. does less on overexposed images
  • because it still has the underlying assumption the illuminant must be white-ish, it breaks
    • on scenes that had such an illuminant but no near-illuminant color in it
    • where the illuminant is colored, e.g. underwater photography
can be tempered by limiting the difference between the different gains applied - because usually significant different gains makes no sense
...though that in combination with the percentile logic can have some odd side effects

Gray world plus retinex

"Combining Gray World and Retinex Theory for Automatic White Balance in Digital Photography" argues that combining the two makes sense.

Which requires a little trickery, as linear correction alone cannot satisfy both criteria at once.

Robust Automatic White Balance

Essentially a variant of gray world that is selective about the areas it uses, primarily looking for nearly-white parts, so e.g isn't distracted by the average of the colored parts.


Pro:

  • Doesn't make as many mistakes as plain gray world
  • photos taken outside usually have such near-whites, so this makes sense for them

Con:

  • images may not have representative near-whites
  • selection of areas to use turns out to be harder than it sounds, depending on how robust you want it to be.


J Huo et al., "Robust Automatic White Balance Algorithm using Gray Color Points in Images"

More reading

http://ipg.fer.hr/ipg/resources/color_constancy

D Nikitenko et al., "Applicability Of White-Balancing Algorithms to Restoring Faded Colour Slides: An Empirical Evaluation"

A Rizzi et al., "A new algorithm for unsupervised global and local color correction"

D Cheng et al., "Illuminant Estimation for Color Constancy: Why spatial domain methods work and the role of the color distribution"