Illuminant correction

✎ This article/section is a stub — some half-sorted notes, not necessarily checked, not necessarily correct. Feel free to ignore, or tell me about it.

The intent is usually to take out any mild tint that the illuminant has, or to correct a camera's mis-estimation of the illiminant.

In other words, mainly white balance correction, a.k.a. gray balance.

Color correction often comes down to

estimating what the illuminant probably was

photographers who cares about color accuracy tend to use a gray card to have a known-absolute reference.

without one, it's based on assumptions

and incorrect assumptions can introduce unnatural tinting in the process

apply chromatic adaptation so that that illuminant effectively becomes a given illuminant, such as D65 (mid-day sun), or just numerically equalize channels.

There are some more specific cases you could focus on, such as cases where you know color filters were used, or e.g. faded photographs could consider the dyes in use - their relative fading is usually documented.

Gray world

The idea: in a well balanced image, the average color is a neutral gray.

So the correction scales channels to makes the average become gray, typically implemented by making the average of the red, green, and blue the same.

Often by a calculating those averages, and applying a basic linear gain to make them the same.

The results vary a little with in which color space you do this.

Gray world makes decent sense when large patches are expected to be neutral, which is e.g. true in darker photographs.

Pro:

Simple

works well on removing the illuminant's tint on images with a lot of white, a lot of dark areas, and/or a lot of each color, or a photograph which had such a neutrality but had a tint applied to it

Con:

effectively equally uses all of the image for measurement

an assumption that is flawed to varying degrees

Only true for images with both a roughly white illuminant, and with either mostly neutral colors or equal color use

if tinted, assumes that you want to fully negate that tint.

which may be good at neutralizing white balance

but consider e.g. an image intentionally taken with monochrome illuminant

we will effectively just ramp up green and blue tens of times more (maybe hundreds, if the only contibution of those colors was negligible sensor-noise levels), because there was so little of it

and since we handle the channels independently so it likely comes out quite unnatural

that specific issue can be tempered by limiting that gain (recognizing that most off-whites are only somewhat off-white so the gain difference should never be large), but that mostly avoid weird extremes. Sometimes you might want to detect this might happen and do nothing rather than a not-great half-fix

The combination of these assumptions means it actually looks unnatural on a lot of images.

There are a number of variations on gray world that are better behaved

Auto levels

The idea: The brightest color should be white, the darkest color should be black.

So: independently rescale each channel's histogram to span the full range, accepting that some (e.g. 1%) will lie outside and be truncated.

Pro:

brightest color becomes white, which on outside photos is often good enough
somewhat less sensitive to monochrome-illuminants than gray world

Con:

suffers from similar problems to gray world
not sensitive to how much small the brightest pixel-area is. For example, having one tiny bright window in the background means a huge white wall will be ignored.

Retinex

Retinex is in itself a wider theory dealing with various color constancy effects, also dealing with local context and some human interpretation.

While retinex turned out to not be a great model of human perception, it was a good start that served as inspiration for further methods.

Even in image correction it can refer to multiple things, so the following is specifically about retinex in the context of whole-image illuminant correction.

This application points to the fact that perceived white tends towards the strongest cone signal.

This is largely a gentler form of gray world, but using only the brightest parts to inform our overall correction.

A crude but decent implementation would be to make the maximum within each channel be the same.

The correction could be implemented as just (linear) gain on each channel to make those maxima the same, though in practice using a very-high-percentile point can be more robust, ignoring a small amount of (potential-)outlier pixels.

While image-pixel-RGB doesn't correspond precisely to color perception, that implementation is close enough to work pretty well.

Pro:

still a fairly simple idea and simple code

better behaved than gray world, in that it avoids many larger color shifts

Con:

sometimes too cautious

e.g. does less on overexposed images

because it still has the underlying assumption the illuminant must be white-ish, it breaks whenever that is not true, e.g.
- on scenes that had such an illuminant but no near-illuminant color in it
- where the illuminant is naturally colored, e.g. underwater photography

can be tempered by limiting the difference between the different gains applied, because in many cases applying significantly different gains per channel makes no sense

...though that in combination with the percentile logic can have some odd side effects

Gray world plus retinex

"Combining Gray World and Retinex Theory for Automatic White Balance in Digital Photography" argues that combining the two makes sense.

Which requires a little trickery, as linear correction alone cannot satisfy both criteria at once.

Robust Automatic White Balance

Essentially a variant of gray world that is selective about the areas it uses, primarily looking for nearly-white parts, so e.g isn't distracted by the average of the colored parts.

Pro:

Doesn't make as many mistakes as plain gray world

photos taken outside usually have such near-whites, so this makes sense for them

Con:

images may not have representative near-whites

selection of areas to use turns out to be harder than it sounds, depending on how robust you want it to be.

J Huo et al., "Robust Automatic White Balance Algorithm using Gray Color Points in Images"

Illuminant correction

Contents

Gray world

Auto levels

Retinex

Gray world plus retinex

Robust Automatic White Balance

More reading

Navigation menu