Chapter 5The Day Color Drained Away

Visual Intelligence: How We Create What We See

Donald D. Hoffman

Inability to See Color Story of Jonathan I., artist who worked with Georgia O’Keeffe Was in a car accident, suffered a concussion Saw no colors, only shades of gray. Diagnosed by Oliver Sacks to have cerebral achromatopsia Colorblindness caused by damage to the cerebral cortex rather than retinal

abnormalities Most common cause: damage to the occipital lobe, in the lingual and

fusiform gyri

Cross-Dependence If you damage the lingual and fusiform gyri of just the left hemisphere, you

can no longer construct color in the right visual field; you can only construct shades of gray.

Damage to the right hemisphere results in inability to construct color in the left visual field, only grays.

The result: a strange half-colored/half-gray world of hemiachromatopsia.

We Construct Color

On the left: sequence of short blue lines On the right: Same sequence, with black lines added to the ends. On the right, we construct something more than lines: a glowing blue worm with

clear subjective boundaries. The lines are blue, as is the space between the blue lines. If you measure the color between the lines with a photometer, you would not find any

blue.

Another Example: The Redies-Spillmann Figure

Color Construction Not simply light or pigment at a point that governs the color we construct. When we construct color, we do not construct just color. We construct several visual properties at once and try to make them all

mutually consistent: Organize our visual world into objects

Endow these objects with three-dimensional shapes

Place light sources that illuminate those objects

Assign color to both the light sources and the objects.

Of course, images are infinitely ambiguous.

Jan Koenderink’s Color Shuffle

Notice: On the left you see not only colored squares, but colored lights illuminating from those squares. A yellow light shines from one corner, red from another, blue and green from the remaining.

On the right you no longer see different-colored illuminants. Instead you see a single uncolored illuminant shining uniformly over all the squares. Why?

Rules 21 and 22: Rule 21: Interpret gradual changes of hue, saturation,

and brightness in an image as changes in illumination. Rule 22: Interpret abrupt changes of hue, saturation,

and brightness in an image as changes in surfaces.

Back to the Color Shuffle…

On the left: the squares are arranged as gradually as possible. These gradual transitions are interpreted as illumination changes. There are also abrupt changes at the edges between squares. These you interpret as changes in surface color. Were there no edges, just a smooth transition, you would see no changes in surface color, just lights shining on a surface.

On the right: there are no smooth transitions of colors. You attribute none of the image changes to illumination changes: you see a single uniform illumination. Instead, you attribute all of the image changes to changes in surface color.

Rules 23 and 24: Rule 23: Construct as few light sources as possible. Rule 24: Put light sources overhead. The “Muffin Pan:” Demonstrates the influence of light

and shading on our perception of images

Michael White’s Gray Bars See horizontal black bars separated by horizontal white bars. Two vertical gray bars, the right one appearing lighter than the left one. A photometer would detect no difference between the right and left. We don’t yet know the rules we use to construct grays or group these gray

structures.

Ted Adelson Figure; Filters The diamond on the left looks lighter than the diamond on the right. Almost like transparent filters of differing darknesses, through which you

view the underlying pattern.

Filters, cont’d.; Rule 25:

In this figure, it looks as though C and D are filters overlapping regions of A and B, respectively. Notice that B is darker than A. You require then that D be darker than C; if it’s not, then you refuse to construct a filter.

Rule 25: Filters don’t invert lightness.

Filters, cont’d.; Rule 26:

Filters decrease light differences. Notice the difference in lightness between B and A. You require that the difference between D and C be smaller; if it’s not, then again you refuse to construct a filter.

Rule 26: Filters decrease lightness differences.

Spatial Relations

In this figure, regions A, B, C, and D all have the same lightnesses as before, but now you no longer create transparency.

Our construction of transparency is affected by the boundaries of construction.

The meeting of cusps implies part boundaries.

Construction of Grays To construct gray at one point you don’t just use the luminance of the image

at that point. You don’t generally just use nearby luminances either. You use large portions of the image and sophisticated rules of grouping that

we have yet to articulate . You don’t just construct colors or grays in isolation. Instead you construct

them as part of a coordinated construction of surface shapes, surface colors, light sources, and transparent filters.

Bill Freeman Image

Probably see a cylinder of uniform gray metal, lit by one surface from above. Could see numerous other images, with different lighting scenarios or in

different dimensions. We prefer one light source, and we only see one possibility (at least at one

time). We must use some rule to narrow our choices…

Fair PickThat rule is the rule of generic views. For example…

Pick any combination of shape, color, and source that gives an image. Call it a fair pick.

Randomly perturb the image. It’s now different than it would be if you took a picture of the original.

Fair Pick, cont’d.

Rule 27: Choose the fair pick that’s most stable.

This is a powerful rule because it cuts down the possibilities to one image.

Relative Size-DependenceQuestion: If you construct grays from image luminance, how do you decide what is white, what is black, and what the total range of grays will be?

Answer: We don’t know. The gray you assign to a region depends on its size: the larger the region (in relative size), the lighter the gray.

Rule 28: Interpret the highest luminance in the visual field as white, fluorent, or self-luminous.

Color Construction in Aperture Displays According to a photometer, aperture

displays can differ in many, many different ways.

According to humans, one aperture display can differ from another in three ways: hue, saturation, and brightness.

This image shows their relationship: Hue: How much red, yellow, green, blue,

etc. are in a color. Saturation: The purity of hue. Varies

from neutral grays to highly pure hues. Brightness: varies from barely visible to

dazzling.

Photometer and Energy Detection Photometer does report change in one factor that humans do

not detect: roughly, the energy of light at each of many frequencies.

Light is a confusing phenomenon that has lead to the study of quantum mechanics.

Light can be considered a particle, a wave, or sometimes both!

Photometer/Energy, cont’d.Simple Explanation of Light: As a particle (not as a wave), light comes in discrete “packets”

called quanta. Each quantum of light has a specific frequency. Photometers count how many quanta of light there are at each

specific frequency. There are many possible frequencies, so light coming from an

aperture display can vary in many dimensions, one dimension for each possible frequency of light.

Photometer/Energy, cont’d.However, this explanation is a bit too simple. Why? Light has no quanta and no specific frequencies until these

properties are measured. Only by measurement can one assert that light has specific frequencies.

The photometer constructs the properties of light that it reports.

Hoffmann gets off the topic of light and promises to return later…

Color Dimension Construction The three color dimensions (hue, saturation, and brightness)

are constructed beginning at the retina. In the retina, there are two types of photoreceptors: rods and

cones. Rods mediate vision in low light settings, cones in bright light. Three different types of cones, each of which corresponds

differently to light. These three different types of cones are marked “S,” “M,” and

“L” (short, medium, and long wavelengths) in the following diagram:

Young-Helmholtz Trichromatic Theory

Cones Differ because they have different pigment molecules that

respond differently to light. Each pigment molecule has two parts: a large protein called

opsin and a derivative of vitamin A called retinal. The particular sequence of amino acids in the opsin determines

the response of the pigment to light. Specific genes code for these opsin molecules, and their

nucleotide sequences have been found. Should one of these genes be missing, one would be unable to

see the corresponding pigment(s), causing colorblindness.

Lacking Specific Genes Lacking the L pigment results in protanopia, where you cannot

see differences between red and green. Lacking the M pigment results in deuteranopia, where you

(again) cannot see differences between red and green. Lacking the S pigment results in tritanopia, where you cannot

see differences between blue and yellow.

Recent Findings: L Pigment Recently found that the gene for the L pigment is polymorphous: it

comes from two different varieties in the normal population. These two varieties lead to two different versions of the L pigment,

which differ in position 180 of the amino acid sequence of the opsin protein.

One version has alanine at position 180, and a peak sensitivity to light at 552.4 nm. About 38% of males with normal color vision have this version.

The other version has serine at position 180, and a peak sensitivity to light at 556.7 nm. About 62% of males with normal color vision have this version.

Why is this Interesting? Interesting for two reasons:

1. Underlines again that you construct the colors you experience.

2. It is the first case where we have traced a difference in visual construction to a difference at a single site of a gene.

“The significance of these discoveries for psychologists cannot be exaggerated. Here is a case where a differenece of a single nucleotide places people in distinct phenomenal worlds…” –John Mollon

Color Opponency Opponent colors are something that photometers cannot detect. We do

because we construct them. The photometer makes no distinction between reds and greens, nor

between blues and yellows. Here is a simple model of how we construct opponent colors:

Color Opponency, cont’d. The boxes labeled L, M, and S represent cone responses. On the left, a difference between L and M cone responses creates red-green

opponency, labeled R-G. On the right, a sum of L and M cone responses creates a dimension of

luminance. A difference between this luminance and the S cone response creates blue-yellow opponency, labeled B-Y.

Approximate Color Constancy We have a talent for creating roughly the same colors despite

changes in illumination, called approximate color constancy. Researchers have yet to figure out why we can do this. Linear models is one promising proposal. They assume that there are only a handful of distinct, true

illuminants, and that all other illuminants are really simple combinations of the true illuminants.

They also assume that there are only a handful of truly distinct ways a surface can reflect light, and that all surface reflectances are really simple combinations of this basic handful.

Summary We construct the color that we see. We don’t need light or even eyes to see color; just stimulation of the lingual

and fusiform gyri. In most respects, we are more advanced color detectors than photometers. Lighting and shading affect our perception of images (think of the muffin

pan). An image could be interpreted in many, many different ways! Your visual

intelligence tries to find a lowest-cost solution.