encouraging students towards meaningful subjective comparisons

encouraging students towards meaningful subjective comparisonsBradford SwansonJuly 27, 2013 - Murfreesboro, TNAES 50th Conference on Audio Education

bradfordswanson.com

encouraging students TO:use their earsbeware of outside stimuliunderstand research methodsincorporate into daily practice

bradfordswanson.com

outlineaudio research methodspractices in other disciplinessample comparison experienceaudio specific recommendations

bradfordswanson.com

subjectivity research in audiocodecsup/downmixingmultichannel techniquesHRTF & spatial perception

bradfordswanson.com

subjectivity research in audiographical representationscomparisons ratings

bradfordswanson.com

sound qualityspeech intelligibility

localization & envelopment

audio research methods2:15 PM - CoE 164 - Paper #21“From Practice to Research and Back Again: Research Skills in Audio Engineering Education” Drs. Leffords & Berg

bradfordswanson.com

audio research methodsITU-R BS.1116-1ITU-R BS.1534-1 ITU-R BS.1284-1semantic differential

bradfordswanson.com

ITU-R BS.1116-1 (ABC/HR)

bradfordswanson.com

Methods for the subjective assessment of small impairments in audio systems

Three stimuliRated by impairmentSingle listen to eachReference Reference = one of the stimuli

ITU-R BS.1534-1 (MUSHRA)

bradfordswanson.com

Method for the subjective assessment of intermediate quality level of coding systems

MUltiple Stimulus with Hidden Reference and Anchor

Three or more stimuliRated by sound qualityMultiple listensReference Reference = one of the stimuliAnchor = one of the stimuli

ITU-R BS.1534-1 (MUSHRA)

bradfordswanson.com

Method for the subjective assessment of intermediate quality level of coding systems

MUltipleStimulus

and

Anchor

HiddenReference

Source: mushraJS

ITU-R BS.1284-1 (general)

General Methods for the Subjective Assessment of Sound Quality

bradfordswanson.com

Single stimuli

ITU-R BS.1284-1 (general)

General Methods for the Subjective Assessment of Sound Quality

bradfordswanson.com

Multiple stimuli

semantic differential

bradfordswanson.com

Source: Marui, A., Martens, W. - “Timbre of nonlinear distortion effects: Perception beyond aptitude beyond “sharpness”?”

ratings along bipolar adjective scales

semantic differential

bradfordswanson.com

9 stimuli randomized through all 12 adjective pairs

Stimuli One

Dark ---------------------- Bright

ratings along bipolar adjective scales

semantic differential

bradfordswanson.com

Stimuli Eight

Dark ---------------------- Bright

9 stimuli randomized through all 12 adjective pairs

ratings along bipolar adjective scales

semantic differential

bradfordswanson.com

Stimuli Eight

Sharp ---------------------- Dull

9 stimuli randomized through all 12 adjective pairs

ratings along bipolar adjective scales

semantic differential

bradfordswanson.com

Source: Marui, A., Martens, W. - “Timbre of nonlinear distortion effects: Perception beyond aptitude beyond “sharpness”?”

9 stimuli assessed using all 12

adjective pairings

ratings along bipolar adjective scales

practices in other disciplinesFormal Sensory Evaluation Psychological TendenciesCultural ConsiderationsCasual/Social Experiences

bradfordswanson.com

sensory evaluation methodsAffective Tests

bradfordswanson.com

“Untrained” ParticipantsQuantative Suvery QuestionairreQualitative Interview or focus group Free response description

sensory evaluation methodsDescriptive Analysis orAttribute Difference Tests

bradfordswanson.com

Highly trained participantsSensory lexicon of defined termsRating intensity or quality on a scale

sensory evaluation methodsDiscrimination or Overall Difference Tests

bradfordswanson.com

Highly trained participantsThree Stimuli (two similar)Whether there is a difference and/or

How much of a difference there is

sensory evaluation methodsDiscrimination or Overall Difference Tests

bradfordswanson.com

A - B - Awhich is the odd one out?

sensory evaluation methodsDiscrimination or Overall Difference Tests

bradfordswanson.com

A (ref.) - A - B or

B (ref.) - A - B

which is most/least like the reference?

sensory evaluation methodsDiscrimination or Overall Difference Tests

bradfordswanson.com

A (ref.) - A - B or

B (ref.) - A - B

how much difference is there between these samples?

psychological tendenciesExpectation ErrorsSuggestion EffectsDistraction ErrorsStimulus/Logical ErrorsHalo Effects/Proximity ErrorsCarry-over, Order or Contrast Effects

bradfordswanson.com

cultural considerationsPersonal PreferencesBackground ExperienceAccessibility

bradfordswanson.com

sensory evaluation methodsExperimental Design

bradfordswanson.com

Identify test objectivesDetermine statistical analysis methodDefine the terminology usedProvide a controlConsider screening order

subjective listening experience15 examples Varying timbral qualitiesMark your perception

bradfordswanson.com

SUBJECTIVE LISTENING EXPERIENCETest Objective

bradfordswanson.com

develop critical listening skillscompare timbral perception of equipment

SUBJECTIVE LISTENING EXPERIENCEStatistical Analysis

bradfordswanson.com

Not collected - for personal use

SUBJECTIVE LISTENING EXPERIENCEDefine the terminology

bradfordswanson.com

Timbe:Content = ContentPresentation = PresentationPitch = PitchLoudness = LoudnessSomething = Something

SUBJECTIVE LISTENING EXPERIENCEDefine the terminology

bradfordswanson.com

Timbe:Harmonic Content or Frequency Spectrum

SUBJECTIVE LISTENING EXPERIENCEDefine the terminology

bradfordswanson.com

Timbe:Content = ContentPresentation = PresentationPitch = PitchLoudness = LoudnessSomething = Something

SUBJECTIVE LISTENING EXPERIENCEConsider Screening Order

bradfordswanson.com

?

subjective listening experience15 examples Varying timbral qualitiesMark your perception

bradfordswanson.com

subjective listening experience

bradfordswanson.com

“uncolored”or

natural reproduction of the harmonic content of this voice

subjective listening experience

bradfordswanson.com

“colored”or

significant deviations from the natural harmonic content of this voice

subjective listening experience

bradfordswanson.com

colored uncolored

l------l------l------l------l------l

subjective listening experience

bradfordswanson.com

what the heck was that?

SUBJECTIVE LISTENING EXPERIENCEConsider Screening Order

bradfordswanson.com

RandomisationBalanced Test Design

*5 stimuli*each presented 3 times*in different orders

subjective listening experience

bradfordswanson.com

What the heck was that?Shakespeare’s 29th SonnetPerformed onceBob Kuhn (bkvoice.com)5 microphones - 3 times eachReferences included as stimuliSimilar signal flow

subjective listening experience

bradfordswanson.com

Colored referenceExamples 4, 8, and 15

Shure 55sw

subjective listening experience

bradfordswanson.com

Examples 3, 9, and 14

Beyer M-88

subjective listening experience

bradfordswanson.com

Examples 1, 7, and 10

Sennheiser MD-409

subjective listening experience

bradfordswanson.com

Examples 2, 6, and 12

AKG C-535

subjective listening experience

bradfordswanson.com

Uncolored referenceExamples 5, 11, and 13

DPA 4006

subjective listening experience

bradfordswanson.com

1) MD-4092) C-5353) M-884) 55sw5) 40066) C-5357) MD-4098) 55sw

9) M-8810) MD-40911) 400612) C-53513) 400614) M-8815) 55sw

Casual/Social experiences

“it’s a wine-tasting,

bradfordswanson.com

not a shootout” Tom Bates

recommendationsExperimental Design

bradfordswanson.com

Define GoalsDefine VariablesMinimize Distractors

recommendationsMatching non-variables

bradfordswanson.com

PerformancePresentationSignal PathGain

recommendationsListening Experience Design

bradfordswanson.com

Listening EnvironmentsDouble-Blind TestingConduct a pilot sessionListen past the technology

bradfordswanson.com/mic