General Methods

Relations among Color, Music, and Emotion

Background

Color, Music, and Emotion Stephen E. Palmer1, Thomas Langlois1, Tawny Tsang1, Karen B. Schloss1, & Daniel J. Levitin2

1 University of California, Berkeley, Department of Psychology 2 McGill University, Department of Psychology

Emotions Mediate Color-Music Associations

Conclusions

References and Acknowledgements

Color and Music: Stimuli

SATURATED

MUTED

LIGHT

DARK

8 Hues: Red, Yellow, Green, Blue, Orange, Chartreuse, Cyan, Purple4 Saturation/Lightness levels (”cuts”): Saturated, Light, Muted, Dark

+ 5 Achromatic Colors: White, Black, Light, Medium, & Dark Gray

Berkeley Color Project (BCP) 37 Colors 64 Mozartian Melodies

Systematic associations between classical orchestral music and color dimensions: (Schloss, Lawler & Palmer, VSS-08)

The Emotional Mediation Hypothesis: When choosing colors to “go best with” an emotional stimulus in a di�erent modality, people choose the colors that best match the emotional content of that stimulus.

The original color-music experiment used 18 recordings of orchestral selections as musical stimuli. 1. Will the color-music associations replicate with more tightly-controlled musical stimuli? 2. Which musical dimensions are most in�uential in the relation between colors and music?

The present study used single-line unaccompanied piano theme-and-variation melodies to vary: mode (major vs. minor key) note-density (quarter-notes vs. eighth-notes), tempo (fast vs. slow speed) register (high vs. low)

while controlling for (i.e., no variation in): dynamics (soft vs. loud) orchestration (multiple instruments playing together) timbre (tone color) harmonic mode (harmony of simultaneous notes)

timbretimbre

Color-Music Associations

37 colors presented simultaneously with each melody. 1) Select the 3 colors that are most consistent with the melody in order: 1st, 2nd, 3rd most consistent.

2) Select the 3 colors that are most inconsistent with the melody in order: 1st, 2nd, 3rd most inconsistent.

Music-Emotion Associations

HappySad

Each of 37 colors were rated on four bipolar emotional scales, one at a time: Agitated-Calm Happy-Sad Strong-Weak Angry-Not Angry

HappySad

Color-Music Association (CMAD) was calculated for each melody for each color appearance dimension (D) as the average D-value of the 3 colors judged to be most consistent with the melody (CD) minus the average D-value of the 3 colors judged to be most inconsistent with the melody (ID):

CD = (3c1,D+ 2c2,D+ c3,D )/3, ID = (3i1,D+2i2,D+ i3,D )/3, CMAD = CD - ID

most consistent color

4 single-line melodies adapted from Mozart piano themes

16 variations for each melody varied along four dimensions:

1) major vs. minor mode 3) sparse vs. dense notes 2) slow vs. fast tempo 4) high vs. low register

Slow-Sparse Slow-DenseFast-Sparse

Fast-Dense-150

-100

-50

0

50

100

150

200

250

CM

A (S

H)

Note Rate

Slow-Sparse Slow-DenseFast-Sparse

Fast-Dense-150

-100

-50

0

50

100

150

200

250

CM

A (A

)

Note Rate

Slow-Sparse Slow-Dense

Fast-SparseFast-Dense

-150

-100

-50

0

50

100

150

200

250

CM

A (W

S)

Note Rate

Slow-Sparse Slow-DenseFast-Sparse

Fast-Dense-150

-100

-50

0

50

100

150

200

250

CM

A (C

A)

Note Rate

Happier emotion Sadder Emotion Major (42%) Minor Faster (33%) Slower Higher (7%) LowerTotal variance 82%

Strong emotion Weak emotion Faster (60%) Slower High (5%) Low Major (2%) MinorTotal variance 67%

Agitated emotion Calm emotion Faster (69%) Slower Minor (4%) MajorTotal variance 73%

Angry emotion Not Angry Minor (73%) MajorTotal variance 73%

Yellower colors Bluer colors Faster (41%) Slower Major (22%) Minor Higher (11%) LowerTotal variance 74%

Redder colors Greener colors Fast (45%) SlowerTotal variance 45%

Lighter colors Darker colors Major (53%) Minor Higher (17%) Lower Faster (8%) SlowerTotal variance 78%

Higher saturation Lower saturation Faster (67%) Slower Major (20%) MinorTotal variance 87%

As in the �rst color, music, and emotion study (Schloss, Lawler & Palmer, VSS-08), both color and music were strongly associated with speci�c emotions: Happy-Sad, Agitated-Calm, Strong-Weak, Angry-Not Angry.

Better-controlled musical stimuli revealed the signi�cant in�uence of distinct musical dimensions (major/minor mode, slow/medium/fast note-rate, and low/high register) on color-associations and ratings of emotional content (independent of timbre, dynamics, orchestration, and harmonic mode).

When making color-music associations, subjects seem to have evaluated the emotional content of the melodies and to have chosen the colors that had the same emotional content. This �nding supports the Emotional Mediation Hypothesis.

Listen to the music

Evaluate the emotional content of the music, based on its mode, note-rate, and register.

Choose the colors whose emotional content best

matches that of the music

Schloss, K. B., Lawler, P. & Palmer, S. E. (VSS-2008). “The Color of Music.” Presented at the 8th Annual Meeting of the Vision Sciences Society, Naples, FL, May 2008.Xu, Schloss, K. B. & Palmer, S. E. (VSS-2010). “The Color of Emotionally Expressive Faces.” Presented at the 10th Annual Meeting of the Vision Sciences Society, Naples, FL, May 2010.

AcknowledgementsWe thank Rosa Poggesi, Arielle Younger, Madison Zeller, and Mathilde Heinemann (at UC Berkeley) for collecting data. We also thank Eli Strauss (also at UC Berkeley) for help with programming in Presentation, and Karle-Philip Zamor (at McGill) for generating the �nal musical stimuli from the original wave �les. We further thank the National Science Foundation (#BCS-0745820) and Google for �nancial support.

most inconsistent color

There are high correlations between the emotional ratings of each melody and the emotional ratings of colors consistent with that melody:

CE = (3c1,E + 2c2,E + c3,E )/3, IE = (3i1,E+2i2,E+ i3,E )/3, CMAE = CE - IE

most consistent color

most inconsistent color

Emotional Color-Music Association (CMAE) was calculated for each melody for each emotion dimension (E) as the average E-value of the 3 colors judged to be most consistent with the melody (CE) minus the average E-value of the 3 colors judged most inconsistent with the melody (IE):

Choose the color whose emotional content matches that of the stimulus

Evaluate emotional content of the object

Experience the stimulus to be associated with the color

Xu, Schloss & Palmer (VSS-10)

Schloss, Lawler & Palmer (VSS-08)

-100 -50 0 50 100-150

-100

-50

0

50

100

150

Happy vs. Sad

-100 -50 0 50 100-150

-100

-50

0

50

100

150

Angry vs. Not Angry

Music-Emotion Association

-100 -50 0 50 100-150

-100

-50

0

50

100

150

Strong vs. Weak

Music-Emotion Association

-100 -50 0 50 100-150

-100

-50

0

50

100

150

Agitated-Calm

Music-Emotion Association

CM

AE

HA

PPY

SAD

0

HAPPYSAD 0

Music-Emotion Association

0 ANGRYNOT-ANGRY

CM

AE

AN

GRY

NO

T-A

NG

RY

0

STR

ON

GW

EAK

0

CM

AE

STRONGWEAK 0

AG

ITAT

EDC

ALM

0

CM

AE

AGITATEDCALM 0

r = 0.92*** r = 0.82*** r = 0.63*** r = 0.88***

Slow, sparse melody

Slow, dense melody

Fast, sparse melody

Fast, dense melody

A B C D

A A’ B’ C’B C D D’

A B DC

A A’ B’ C’ D’B DC

Medium

Slow

Fast

TimeNote RateMelody

250 250

Emotional correlations between color and ...

Happy-Sad Strong-Weak music r = .97 r = .96

Happy-Sad Strong-Weak faces r = .97 r = .91

Happy-Sad Angry-NotAngry gestures r = .95 r = .94

D = Yellow/Blue D = Red/Green D = Light/Dark D = Saturation

Minor

Major

High

HighLow

Low

Minor

MajorHigh

High

Low

Low

Slow-Sparse Slow-DenseFast-Sparse

Fast-Dense-150

-100

-50

0

50

100

150

200

250

CM

A (Y

B)

Happy-Sad Strong-Weak

Note Rate

Slow-Sparse Slow-Dense

Fast-SparseFast-Dense

-150

-100

-50

0

50

100

150

200

250

CM

A (R

G)

Note Rate

Slow-Sparse Slow-DenseFast-Sparse

Fast-Dense-150

-100

-50

0

50

100

150

200

250

CM

A (L

D)

Note Rate

Slow-Sparse Slow-DenseFast-Sparse

Fast-Dense-150

-100

-50

0

50

100

150

200

250

CM

A (S

at)

Note Rate

Minor

Major

Slow Medium Fast

-200

0

200

Saturation

Tempo

Col

or-M

usic

Ass

ocia

tion Major

Minor

Slow Medium Fast

-200

0

200

Light/Dark

TempoSlow Medium Fast

-200

0

200

Red/Green

TempoSlow Medium Fast

-200

0

200

Yellow/Blue

Tempo

Satu

rate

dD

esat

ura

ted

Lig

htn

ess

Dar

knes

s

Red

nes

sG

reen

nes

s

Yello

wn

ess

Blu

enes

s

Minor

Major

High

High

Low

Low

Minor

Major

These data provide strong support for the emotional mediation hypothesis

Color-Emotion Associations

High

Low

Minor

Major

Minor

Major

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Dimension 1

Dim

ensi

on 2

Major Medium Low

Major Fast High

Major Fast Low

Major Slow High

Major Slow Low

Major Medium Low

Minor Medium High

Minor Medium Low

Minor Fast High

Minor Fast Low

Minor Slow High

Minor Slow Low

Minor Medium High

Minor Medium Low

Happy

Sad

Emotional MDS of Melodies

Slow

Fast

Minor Major

Strong

Weak

Major Medium HighMajor Medium High

Happy

Sad

Strong

Weak

Dimension 1

Dim

ensi

on 2

-1.5

-1

-0.5

0

0.5

1

1.5

-1.5 -1 -0.5 0 0.5 1D

esat

urat

edSa

tura

ted

Dark Light

Emotional MDS of ColorsMDS of Emotional Spaces for Melodies and Colors

Melodies: A 16x16 proximity matrix was derived from the correlations of the average emotional ratings for each of the 16 melody types (major/minor, fast/slow, high/low, dense/sparse).

This matrix was input to ALSCALE, a multidimensional scaling (MDS) program, and the best-�tting 2-D solution was found. The primary emotional axes appear to be happy-sad and strong-weak.

Colors: A 37x37 proximity matrix was derived from the correlations of the average emotional ratings for each of the 37 colors (8 hues at each of 4 lightness-saturation levels, plus 5 grays).

This matrix was input to ALSCALE, a multidimensional scaling (MDS) program, and the best-�tting 2-D solution was found. The primary emotional axes also appear to be happy-sad and strong-weak.

Each of 64 melodies were rated on four bipolar emotional scales, one at a time: Agitated-Calm Happy-Sad Strong-Weak Angry-Not Angry

E = Happy/Sad E = Angry/Not Angry E = Agitated/CalmE = Strong/Weak