Pictures at an Exhibition:The role of visual displays in an

evidence-based science

Howard WainerNational Board of Medical Examiners

We typically date the beginning of empirical science with

Aristotle (384 BC - 322 BC)

Both before and after Aristotle there was strong opposition to an empirical approach, for data did not always support popular opinion. Aristotle got away

with it because he had a Great friend who protected him.

The next big step forward took more than 1500 years and is generally credited to the work of

the fabulous Bacon boys.

Modern evidence-based science probably begins with Roger Bacon (1214-1294)

QuickTime™ and a decompressor

are needed to see this picture.

Although it is more often credited to Francis Bacon (1561 -1626)

Before ways to look at data could be developed we needed the epistemology that confirmed that

the path to knowing required data.

It was surely not an accident that breakthroughs in looking at data appeared after the British empiricists

John Locke (1632 –1704) and George Berkeley (1685-1753) and the Scot David Hume (1711 – 1776)

expanded and amplified Bacon’s work .

Language developed before science

and so was not ideally suited to it. A more visual communication medium was

needed to allow us to look at scientific evidence.

One started to appear in the 17th century, but achieved most of its modern day strength with

the publication of Playfair’s Atlas in 1786.By the 19th century scientific presentations

were so laconic that the words almost disappeared entirely.

Much of modern science involves, to some extent, the atheoretical plotting of points

and a search for suggestive patterns.

“The greatest value of a graph is when is forces us to see what were not expecting.”

J. W. Tukey, 1977

The balance of this presentation is a single illustration of the marvelous breadth of

possibilities and the lessons it provides.

Designer Will Burtin, whose 1951 graph of the efficacy of 3 antibiotics on 16 bacteria, forms

the core of this presentation.

AntibioticBacteria Penicillin Streptomycin Neomycin Gram StainingAerobacter aerogenes 870 1 1.6 negativeBrucella abortus 1 2 0.02 negativeBrucella anthracis 0.001 0.01 0.007 positiveDiplococcus pneumoniae 0.005 11 10 positiveEscherichia coli 100 0.4 0.1 negativeKlebsiella pneumoniae 850 1.2 1 negativeMycobacterium tuberculosis 800 5 2 negativeProteus vulgaris 3 0.1 0.1 negativePseudomonas aeruginosa 850 2 0.4 negativeSalmonella (Eberthella) typhosa 1 0.4 0.008 negativeSalmonella schottmuelleri 10 0.8 0.09 negativeStaphylococcus albus 0.007 0.1 0.001 positiveStaphylococcus aureus 0.03 0.03 0.001 positiveStreptococcus fecalis 1 1 0.1 positiveStreptococcus hemolyticus 0.001 14 10 positiveStreptococcus viridans 0.005 10 40 positive

The entries of the table are the minimum inhibitory concentration (MIC) that represents the concentration of antibiotic required to prevent growth in vitro.

The covariate “Gram staining” describes the reaction of the bacteria to Gram staining. Gram-positive bacteria are those that are stained dark blue or violet; Gram-negative

bacteria do not react that way.

The cause of evidence-based science requires looking at evidence to advance the science.

How is this cause helped by presenting data in tables?

“Getting information from a table is like extracting sunbeams from a cucumber”

Farquahr & Farquahr, 1891

Despite this warning, the table dominates the medical literature as the preferred conveyor of quantitative

evidence.

0 .60 .50 .40 .30 .20 .10 .0S c a tte r P lo tE v e ry th in g E ls eO rg a n iz a tio n a l D ia g ra mF lo w C h a rtL in eB a rTa b leD is p la y F o rm a t F re q u e n c y fo r th e 2 0 0 8N e w E n g la n d Jo u rn a l o f M e d ic in eP ro p o rtio n o f D is p la y sD is p la y Ty p e

0.60.50.40.30.20.10.0

Scatter Plot

Everything Else

Organizational Diagram

Flow Chart

Line

Bar

Table

Display Format Frequency for the 2008

New England Journal of Medicine

Proportion of Displays

Display Type

0.80.60.40.20.0

Scatter Plot

Everything else

Organizational Diagram

Flow Chart

Line

Bar

Table

Frequency of Occurence in 2008

Journal of the American Medical Association

Proportion of Displays

Display Type

0 .60 .50 .40 .30 .20 .10 .0S c a tte r P lo tE v e ry th in g E ls eO rg a n iz a tio n a l D ia g ra mF lo w C h a rtL in eB a rTa b leD is p la y F o rm a t F re q u e n c y fo r th e 2 0 0 8N e w E n g la n d Jo u rn a l o f M e d ic in eP ro p o rtio n o f D is p la y sD is p la y Ty p e0 .60 .50 .40 .30 .20 .10 .0S c a tte r P lo tE v e ry th in g E ls eO rg a n iz a tio n a l D ia g ra mF lo w C h a rtL in eB a rTa b leD is p la y F o rm a t F re q u e n c y fo r th e 2 0 0 8N e w E n g la n d Jo u rn a l o f M e d ic in eP ro p o rtio n o f D is p la y sD is p la y Ty p e

Everything else

Organizational Diagram

Scatter Plot

Flow Charts

Line Graphs

Bars

Tables

JAMA Data as a Pie

Note, how much more eloquent dot plots are than the more usual pie

AntibioticBacteria Penicillin Streptomycin Neomycin Gram StainingAerobacter aerogenes 870 1 1.6 negativeBrucella abortus 1 2 0.02 negativeBrucella anthracis 0.001 0.01 0.007 positiveDiplococcus pneumoniae 0.005 11 10 positiveEscherichia coli 100 0.4 0.1 negativeKlebsiella pneumoniae 850 1.2 1 negativeMycobacterium tuberculosis 800 5 2 negativeProteus vulgaris 3 0.1 0.1 negativePseudomonas aeruginosa 850 2 0.4 negativeSalmonella (Eberthella) typhosa 1 0.4 0.008 negativeSalmonella schottmuelleri 10 0.8 0.09 negativeStaphylococcus albus 0.007 0.1 0.001 positiveStaphylococcus aureus 0.03 0.03 0.001 positiveStreptococcus fecalis 1 1 0.1 positiveStreptococcus hemolyticus 0.001 14 10 positiveStreptococcus viridans 0.005 10 40 positive

The entries of the table are the minimum inhibitory concentration (MIC) that represents the concentration of antibiotic required to prevent growth in vitro.

The covariate “Gram staining” describes the reaction of the bacteria to Gram staining. Gram-positive bacteria are those that are stained dark blue or violet; Gram-negative

bacteria do not react that way. It was named after its inventor, the Danish scientist Hans Christian Gram (1853 -1938), who developed the technique in 1884.

Display Challenges

1. Scale - the MICs vary over six orders of magnitude, a display in the MIC metric will improperly lump together all data less than 100.

2. Metaphor - a graph is a visual metaphor. Which one chosen to represent the data is critical to understanding and to memory.

3. Adaptability - a powerful display should allow us to generalize beyond these data by accommodating to additional drugs and additional bacteria.

4. Allow Comparisons - allow us to compare the efficacy of the three drugs, and allow comparisons of similar behavior among bacteria.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Antibacterial ranges of Neomycin, Penicillin and Streptomycin

The chart compares the in vitro sensitivities to neomycin of some of the common pathogens

(Gram+ in red and Gram- in blue) with their sensitivities to penicillin and streptomycin. The effectiveness of the antibiotics is expressed as the highest dilution in g/ml, which inhibits the test organism. High concentrations are inward from the periphery; consequently the length of the colored bar is proportional to the effectiveness.

• Good displays allow us to answer the questions of interest.

• Better displays force us to see what we were never expecting.

• Great displays also form a coherent image that stays in our memory.

The Greatest Statistical Graph Ever Drawn

The four purposes of graphic display

• Exploration - there is a message in the data and the display helps us to learn what it is.

• Communication - we know something and the display helps us to effectively tell others.

• Calculation - a nomograph, the display automatically calculates for us.

• Decoration - the graph is pretty and enlivens the presentation.

The first display is an unusual combination of exploration and decoration.

No one could doubt that this display would make a gorgeous poster -- but careful study

reveals much more.

More about this later.

Gram negative pill is 74 times larger than gram positive pill

Drop volumes not as evocative as number of pills, but provides separate estimates by bacteria type -- this is

a hybrid display that also includes the data table.

Escaping Flatland

Burtin’s data can be thought of as three-dimensional in which each antibiotic is a

dimension and each bacterium as a point in that three-space.

This approach allows us to immediately address questions about the

relationships between antibiotics.

Are there less arcane alternatives?

Often simpler is better.

If a display is not going to be repeated (as in an atlas) we should avoid asking the viewer to work hard to learn a new display format

unless that format offers remarkable benefits unavailable elsewhere.

In cooking, any dish can be improved with either garlic or chocolate chips (never both).

By the same token, almost any display can be improved by adding information.

The bars take up a lot of space, but there is still room.This version has extra interpretive information and also corrects

some errors (more about this later).

Note that the dependent variable is 1/MIC

So that bigger is better.

Bars are nice, but all of the information in the bar is contained in its terminal

line.

Why use up all of the space with a bar (decreasing the data-ink ratio)?

We’re almost never interested in ‘Alabama First.’

Or, in this case, ‘Aerobacter aerogenes’

first.

Data displays should almost always be ordered by some aspect of the data, not

the alphabet.

Ordered by Gram character and Penicillin efficacy

On LegendsUsing a legend requires us to use two

moments of perception to understand the graph.

We must first read and memorize the legend and then look at the graph.

It changes the task to reading the graph rather than seeing the graph.

The latter is easier, faster, and less error prone.

Avoid legends whenever possible.

Changing the plotting symbol to something more evocative allows us to eliminate the legend

Dots rarely form memorable images

We can often make a dot plot more memorable by connecting the dots.

Some will carp that connecting dots from categorical variables is misleading;

phooey!

All that paying attention to such complaints will get you is to miss a memorable picture that you

might have made.

There are two kinds of good displays.

A Strongly Good Display - that tells you everything you want to know just by looking at it.

A Weakly Good Display - that tells you everything you want to know just by looking at it, once you know what to look for.

You can change a weakly good display into a strongly good one through the inclusion of

informative labels.

Escaping Flatland through the use of multivariate icons

One way to show multivariate data on a two dimensional surface is to invent an multivariate icon in which each feature represents one dimension of the data

(remember Minard’s 6-dimensional display of Napoleon’s March).

But never waste the two dimensions of the display

plane

Spatial representation is the most powerful perceptual tool.

It allows us to see information instead of reading it.

“That’s funny...”

The most exciting phrase to hear in science, the one that heralds new discoveries, is

not ‘Eureka’ but ‘That’s funny....’

Isaac Asimov (1920 - 1992)

Why is Streptococcus fecalis so different?

It would seem that its credentials as a member of the Strep family are impeccable; as

Sherman. Mauer & Porter (1937) described it: In some respects Streptococcus fecalis (Andrewes &

Horder, 1906) might be considered one of the better established species of the streptococci, and certainly

some of the rather unique characteristics of this organism, or the general group to which it belongs, are

commonly known by bacteriologists.

Yet, in 1984, its genus was changed and its name became Enterococcus faecalis.

Perhaps had these data been plotted in a way that allowed us to

compare the profile of responses of these bacteria to antibiotics the

classification of these two bacteria would have come under scrutiny

sooner.

Now that we know what to look for, it is hard to miss.

Even a table, if well-prepared, would’ve shown the phenomenon

This provides a vivid reminder of how important it is to understand the data and

not proceed blindly.

At the very least the data analyst must work closely with the person who chose to

gather the data - ideally they should be the same person.

I believe that this has serious implications for the way that data mining is currently

done.

A great display simplifies the complex

Taking a maze of information and turning it into evidence for action with an image

that can be remembered and communicated.

It is not always easy to describe the precise character of a great display, but

you know one when you see it.

Two graphical factors

Data tables can usually be described by the labels of their rows (X) and columns (Y).

And so there are two natural questions that suggest themselves:

1. What are the groupings of the Xs?

2. What are the groupings of the Ys?

Usually each of these questions requires a different graphical construction.

Obiter dictum

A graph that does everything is wonderful, but rare.

A graph that does one thing well may still be valuable

In 1951 antibiotics were new and so a question that was still of clinical interest

was, “What is this drug good for?”

What have we learned from this exhibition of pictures?

1. There are many paths to salvation.For any data set there are many possible good displays, although the same rules for construction underlay them all.

2. Any display can be improved. Good writing means we rewrite our prose many times. Because images are more memorable than words, it is more important to revise our displays than our words. In the past this was difficult and expensive. Now it isn’t.

3. A display is never done -- genius is the infinite capacity for taking pains.

Accurate interpretation of results often needs more information

For example, how does our interpretation of these results change when we learn that kidney damage

sometimes accompanies the use of Neomycin?

And certainly we would not have seen the misclassification of the two bacteria had we not

cared about what the labels signified.

This exhibition owes an enormous amount to the many scientists and designers who lent

their genius to solving the problem first posed by Will Burtin half a century ago.

Sine quibus non

Display Sources

Will Burtin (1951)

Charles Joseph Minard

(1865)Jana Asher

Carnegie MellonTroy Brandt

Stanford

Benjamin Lauderdale, PrincetonKatherine Lauderdale, Harvard

Lawrence B. FinerChristian C. Ryan

New YorkPierre Dangauthier

London Donald SchopflocherAlberta

Massimiliano MarchiBologna, Italy

Phil Price Lawrence Berkeley National Laboratory

Charlotte Wickham Berkeley

Emil Friedman Danbury, CT

Georgette AschermanFt. Lee, NJ

Dibyojyoti HaldarBangalore, India

Brian SchmotzerEmory

Mark NicolichLambertville, NJ Christine Schmotzer

EmoryJacques Bertin

Paris