Sensory Evaluation of Sound

Edited by

Nick Zacharov

(M&\ C R C Press

\Cf^ J Taylor & Francis Group Boca Raton London New York

CRC Press is an imprint of the Taylor & Francis Group, an informa business

Contents

Foreword xix

Preface xxiii

Acknowledgements xxix

Nomenclature and Abbreviations xxxi

Contributors xli

S E C T I O N I Background

C H A P T E R 1 • Introduction 3

Nick Zacharov

1.1 WHAT IS SENSORY EVALUATION? 4 1.2 WHY DO WE NEED SENSORY EVALUATION? 4 1.3 WHEN TO APPLY SENSORY EVALUATION? 5 1.4 WHAT CAN WE LEARN FROM SENSORY EVALUATION? 7 1.5 CONCEPTS AND TERMINOLOGY 8

1.5.1 Percepts, attributes, and attribute names 8 1.5.2 The relationship between attributes and dimensionality 9 1.5.3 The number of response variables and dimensions 9

1.6 BEFORE MOVING ON 10

C H A P T E R 2 • Why Sound Matters 11

Julian Treasure

C H A P T E R 3 « Sound, Hearing and Perception 21 Ville Pulkki

3.1 BASICS OF SOUND AND AUDIO 23 3.1.1 Vibrations as source of sound 23 3.1.2 Sound sources 23 3.1.3 Sound waves and sound pressure 24 3.1.4 Sound pressure level 25 3.1.5 Frequency-weighted measurements for sound level 27 3.1.6 Capturing sound signals 28

vi i

viii • Contents

3.1.7 Digital presentation of signals 30 3.1.8 Further reading 32

3.2 HEARING - MECHANISMS AND BASIC PROPERTIES 32 3.2.1 Anatomy of the ear 32 3.2.2 Structure of the cochlea 33 3.2.3 Passive frequency selectivity of the cochlea 34 3.2.4 Active feedback loop in the cochlea 34 3.2.5 Mechanical to neural transduction 35 3.2.6 Non-linearities in the cochlea 36 3.2.7 Temporal response of the cochlea 37 3.2.8 Frequency selectivity of hearing 37 3.2.9 Filterbank model of the cochlea 39 3.2.10 Masking effects 39

3.3 PSYCHOACOUSTIC ATTRIBUTES 41 3.3.1 Loudness 41 3.3.2 Pitch 43 3.3.3 Other psychoacoustic quantities 44

3.4 SPATIAL HEARING 45 3.4.1 Head-related acoustics 45 3.4.2 Localisation cues 46 3.4.3 Binaural cues 47 3.4.4 Monaural cues 49 3.4.5 Dynamic cues 50

3.5 PSYCHOPHYSICS AND SOUND QUALITY 50 3.6 MUSIC AND SPEECH 51 3.7 HEARING IMPAIRMENT 53 3.8 SUMMARY 56

SECTION II Theory and Practice

CHAPTER 4 • Sensory Evaluation in Practice 59 Torben Holm Pedersen and Nick Zacharov

4.1 INTRODUCTION 61 4.1.1 Can sensory evaluation be scientific? 61

4.1.1.1 Perceptual/descriptive scaling 62 4.1.1.2 Affective/subjective scaling 62 4.1.1.3 The filter model 63

4.2 DESIGNING A LISTENING TEST 65 4.2.1 Defining the purpose of a test 65 4.2.2 Test method selection 65

4.2.2.1 Quantification of impression 68

Contents • ix

4.2.3 Experimental variables 68 4.2.3.1 Input variables 69 4.2.3.2 Response variables 69 4.2.3.3 Other experimental variables 69 4.2.3.4 Design of experiment 70

4.2.4 Test duration 71 4.2.5 Test samples 71 4.2.6 Usage scenarios 72 4.2.7 Perceptual constancy 73 LISTENING TEST INGREDIENTS 73 4.3.1 Stimuli 73

4.3.1.1 Product testing 74 4.3.1.2 Product testing - By recording 75 4.3.1.3 Algorithm testing 76 4.3.1.4 Brand sound and music 76

4.3.2 Attributes 76 4.3.2.1 Requirements for good consensus attributes 77 4.3.2.2 Lexicons and sound wheels 78 4.3.2.3 Attribute development for novel applications 81 4.3.2.4 Attribute validation 84 4.3.2.5 Language issues 85

4.3.3 Assessors and panels 87 4.3.3.1 Assessor types 88 4.3.3.2 Panel types 90 4.3.3.3 Assessor selection 92 4.3.3.4 Assessor performance evaluation 93 4.3.3.5 Panel training 95 4.3.3.6 Panel maintenance 96 4.3.3.7 Ethics 96

4.3.4 Listening facilities 96 4.3.4.1 Listening rooms 97 4.3.4.2 Listening booths 98 4.3.4.3 Field testing 98

4.3.5 Equipment 99 4.3.5.1 Electroacoustic equipment 99 4.3.5.2 Systems for test administration 99

TEST ADMINISTRATION 100 4.4.1 Calibration and equalisation 100 4.4.2 Instructions 101 4.4.3 Familiarisation 101 4.4.4 Controlling bias 101

x • Contents

4.4.5 Anchoring 102 4.5 GOOD PRACTICES IN REPORTING LISTENING TESTS 102

4.5.1 Front page 102 4.5.2 Introduction 103 4.5.3 Method 103 4.5.4 Measuring objects 103 4.5.5 Equipment 103 4.5.6 Location 103 4.5.7 Assessors 103 4.5.8 Physical measurements 104 4.5.9 Test administration 104 4.5.10 Analysis and discussion 104 4.5.11 Results 104

4.6 SUMMARY CHECKLIST 105

CHAPTER 5 • Sensory Evaluation Methods for Sound 107 Jesper Ramsgaard, Thierry Worch, and Nick Zacharov

5.1 TEST METHOD FAMILIES 109 5.1.1 Discrimination methods 109 5.1.2 Integrative methods 109 5.1.3 Descriptive analysis methods 112 5.1.4 Mixed methods 114 5.1.5 Temporal methods 115 5.1.6 Performance methods 115

5.2 DISCRIMINATION METHODS 117 5.2.1 Comparing pair(s) of systems discrimination tests 117 5.2.2 Paired comparison of multiple systems based on a sensory

attribute 121 5.3 INTEGRATIVE METHODS 124

5.3.1 Preference and affective methods 124 5.3.2 Recommendation ITU-T P.800 (ACR, CCR, DCR) 126 5.3.3 Recommendation ITU-R BS. 1534 (MUSHRA) 128 5.3.4 Recommendation ITU-R BS. 1116 130

5.4 CONSENSUS VOCABULARY METHODS 132 5.4.1 Quantitative descriptive analysis 134 5.4.2 Semantic differential 136 5.4.3 Recommendations ITU-T P.806 and ITU-T P.835 138

5.5 INDIVIDUAL VOCABULARY METHODS 139 5.6 MIXED METHODS - EXPLAINING PREFERENCE 142

5.6.1 Audio descriptive analysis and mapping 143 5.6.2 Multiple stimulus - Ideal profile method 144

Contents • xi

5.7 INDIRECT ELICITATION METHODS 146 5.7.1 Multi-dimensional scaling 147 5.7.2 Free sorting 148 5.7.3 Projective mapping and napping 150

5.8 ASSOCIATIVE METHODS 151 5.8.1 Free association 152 5.8.2 Check-all-that-apply (CATA) 153 5.8.3 Rate-all-that-apply (RATA) 154

5.9 CLOSING WORDS 155

CHAPTER 6 • Applied Univariate Statistics 157 Per Bruun Brockhoff and Federica Belmonte

6.1 INTRODUCTION 158 6.2 SOME BASICS OF STATISTICAL REASONING 159 6.3 THE GENERAL EXPERIMENTAL AND ANALYSIS

APPROACHES 161 6.3.1 Simple versus elaborate analysis approaches 162 6.3.2 Multiple comparison corrections 163

6.4 MIXED MODELS FOR SENSORY AND CONSUMER DATA 164 6.4.1 Headphone data - Structure and exploratory analysis 164 6.4.2 Overview of mixed model tools 168 6.4.3 From t-tests to multi-factorials and multi-blockings 171

6.4.3.1 Extension 1: K > 2 levels of the "treatment" factor 173

6.4.3.2 Extension 2: Replications - Two blocking factors 173

6.4.3.3 Extension 3: More than a single treatment factor 176

6.4.3.4 Extension 4: Even more complex structures 178 6.4.4 Headphone data analysis, modelling by mixed models 178

6.4.4.1 Post hoc analysis 180 6.4.5 Analysis of and correction for scaling effects 182 6.4.6 SensMixed-package: Multi-attribute automated analysis 184

6.5 MODEL VALIDATION AND PERSPECTIVES: A VIEW TOWARDS NON-NORMAL AND NON-LINEAR MIXED MODELS WITHIN SENSORY EVALUATION 186 6.5.1 Model validation 187 6.5.2 Non-normal data - What to do generally? 187

6.5.2.1 Ignore it! 188 6.5.2.2 Use non-parametric (rank-based) methods 188 6.5.2.3 Use transformation to obtain (near) normality 189

xii • Contents

6.5.2.4 Identify the "correct" distribution and do statistics based on that 189

6.5.2.5 Use simulation-based methods: 189 6.5.2.6 sensR - A brief on discrimination and similarity

testing 190 6.5.2.7 Ordinal - A brief on ordinal data analysis 191

6.6 RELATION TO MULTIVARIATE ANALYSIS 192

CHAPTER 7 • Applied Multivariate Statistics 195 Sebastien Le

7.1 A MAIEUTIC DEFINITION OF MULTIDIMENSIONAL ANALYSIS 195

7.2 WHEN A BIVARIATE QUESTION TURNS INTO A MULTIDIMENSIONAL ISSUE 197 7.2.1 Does occupation impact the music genre people listen to? 197 7.2.2 To what extent do farmers compare to executives and

pensioners? 198 7.2.3 In summary 206

7.3 ONE OF THE MAIN ISSUES IN MULTIDIMENSIONAL ANALYSIS: CHOOSING THE PROPER DISTANCE 206 7.3.1 How would you describe a company with such a sound

logo? ' 208 7.3.2 Do you think this logo is too serious compared to the

brand's values? 209 7.3.3 In summary 210

7.4 SOME CONSIDERATIONS DEPENDING ON THE DATA TYPE 212 7.4.1 Some considerations for understanding MCA 213 7.4.2 Some considerations to better understand PCA 217 7.4.3 In summary 223

SECTION III Application

CHAPTER 8 • Telecommunications Applications 227 Alexander Raake, Janto Skowronek, and Micha! Soloducha

8.1 INTRODUCTION 228 8.2 PRINCIPLES OF SPEECH QUALITY 233

8.2.1 Historical background 233 8.2.2 Quality, elements and features 235 8.2.3 From traditional test methods to new developments 238

8.3 SPEECH QUALITY VS. INTELLIGIBILITY 241 8.3.1 General considerations 241 8.3.2 Quality and intelligibility for packet loss degradations in

VoIP 242

Contents • xiii

8.4 ASSESSING SPEECH QUALITY WITH TERMINAL DEVICES 244 8.5 SPEECH QUALITY DIMENSIONS 246

8.5.1 Listening tests 248 8.5.1.1 Test methods 249 8.5.1.2 Insights 252

8.5.2 New trends: Conversation tests 252 8.6 SPEECH QUALITY AND DELAY 254 8.7 MULTIPARTY QUALITY TESTS 258

8.7.1 Standardised method for telemeeting assessment 259 8.7.2 Spatial audio meeting assessment 260 8.7.3 Assessment of asymmetric conditions 262

8.8 SUMMARY AND OUTLOOK 265

CHAPTER 9 • Hearing Aids 269 Lars Bramsl0w and S0ren Vase Legarth

9.1 INTRODUCTION 270 9.2 HEARING AID FUNCTION 270 9.3 TECHNOLOGY AND HISTORY 272 9.4 FACTORS AFFECTING SOUND QUALITY 275 9.5 HEARING IMPAIRED TEST PANELS 277

9.5.1 Hearing loss 278 9.5.2 Experts versus consumers 280 9.5.3 Maintaining test panels 281

9.6 LISTENING SCENARIOS 281 9.7 OUTCOME MEASURES 282

9.7.1 Speech intelligibility 282 9.7.2 Sound quality 284 9.7.3 New types of outcome measurements 286

9.8 HEARING AID ALGORITHM ASSESSMENT 287 9.9 ASSESSMENT OF A NOISE REDUCTION ALGORITHM 287

9.9.1 Test method and design 288 9.9.2 Summary 289

9.10 QUALIFYING A HEARING IMPAIRED PANEL 290 9.10.1 Audiometrie screening tests 291 9.10.2 Sensory screening tests 292 9.10.3 Brand preference test 293 9.10.4 Results 293 9.10.5 Summary 294

9.11 THE MULTIPLE STIMULUS IDEAL PROFILE METHOD FOR OPTIMISATION OF HEARING AID SOUND QUALITY 296 9.11.1 Selection of samples and recordings of systems 296

x i v • Contents

9.11.2 9.11.3 9.11.4 9.11.5

Identifying attributes for the test Data collection Results Summary

297 298 299 300

9.12 TESTING SMALL DIFFERENCES USING PAIRED COMPARISONS 302 9.12.1 Perceptual effects of delay 303 9.12.2 Test setup, listeners, hearing aids and signals 303 9.12.3 Paired comparison of delay and high-pass cut-off 307 9.12.4 Results 308 9.12.5 Discussion 312 9.12.6 Summary 314

9.13 CONCLUSION 315

CHAPTER 10 • Car Audio 317 Patrick Hegarty and Neo Kaplanis

10.1 DESCRIPTIVE SENSORY ANALYSIS OF REPRODUCED SOUND IN CARS 318 10.1.1 Case study - A listening test system for car audio 318 10.1.2 Methods 319

10.1.2.1 The test car and associated equipment 319 10.1.2.2 Assessors selection 320 10.1.2.3 Independent variables 320 10.1.2.4 Dependent variables 322 10.1.2.5 Experimental design 322

10.1.3 Data analysis 322 10.1.4 Results 324 10.1.5 Conclusion 326

10.2 FLASH PROFILE FOR AUTOMOTIVE AUDIO ASSESSMENT 328 10.2.1 Flash profile - A brief 328 10.2.2 Why does flash profile suit automotive audio assessment? 329

10.2.2.1 Flash profile and the evaluation of acoustical fields 329

10.2.3 Feasibility study 330 10.2.4 Case study: The influence of a car's interior on the perceived

sound quality 331 10.2.4.1 Background 331

10.2.5 Methods 331 10.2.6 Materials and apparatus 332

10.2.6.1 Acquisition of sound fields 332 10.2.6.2 Presentation of sound fields 334

10.2.7 Assessors 334

Contents • xv

10.2.8 Experimental procedure 335 10.2.9 Data analysis 337

10.2.9.1 Ordination using multiple factor analysis 337 10.2.9.2 Influence of programme and acoustical

conditions 339 10.2.9.3 Generalising results 340 10.2.9.4 Clustering of individually elicited attributes 341 10.2.9.5 Clustering validation 342 10.2.9.6 Constructing the sensory profile 342

10.2.10 Results 343 10.2.10.1 The effect of equalisation - Validation 343 10.2.10.2 Ceiling effect 344 10.2.10.3 Absorption effect 345 10.2.10.4 Front side window effect 345 10.2.10.5 Windscreen effect 345

10.2.11 Discussion 345 10.2.12 Considerations of using flash profile in audio evaluation 346

10.2.12.1 Future work 348

CHAPTER 11 • Binaural Spatial Reproduction 349 Brian F.G. Katz and Rozenn Nicol

11.1 INTRODUCTION 350 11.2 SPATIAL HEARING: BINAURAL VERSUS LOUDSPEAKERS 350 11.3 PERCEPTUAL DIMENSIONS 353

11.3.1 Multidimensional perception 353 11.3.2 Quality metrics 355

11.4 ASSESSMENT METHODS 357 11.5 CASE STUDIES 358

11.5.1 Source localisation 358 11.5.1.1 Egocentric vs. exocentric pointing 359 11.5.1.2 Egocentric pointing variations 360 11.5.1.3 Spatial coordinates and front/back confusions 361 11.5.1.4 Lateralisation reporting 364 11.5.1.5 Paired comparison of compression degradations 366

11.5.2 Scene description 367 11.5.2.1 Scene drawing 368 11.5.2.2 Scene reconstruction 370 11.5.2.3 Environment reconstruction 372 11.5.2.4 Scene stability 373

11.5.3 Overall quality 374 11.5.3.1 Subjective global quality ranking 374

xv i • Contents

11.5.3.2 Repeatability of subjective rankings 377 11.5.3.3 Preference judgement 378

11.5.4 Perceptual attributes 380 11.5.4.1 Multiple comparisons with reference and anchor 381 11.5.4.2 Multi-dimensional scaling & HRTF differences 382 11.5.4.3 Verbal elicitation k HRTF differences 385

11.6 CONCLUSION 386

CHAPTER 12 • Concert Hall Acoustics 389 Antti Kuusinen and Tapio Lokki

12.1 INTRODUCTION 389 12.2 CHARACTERISTICS OF SOUND FIELD IN CONCERT HALLS 390 12.3 PERCEPTION OF CONCERT HALL ACOUSTICS - A BRIEF

HISTORY 391 12.4 DESCRIPTIVE PROFILING OF CONCERT HALL ACOUSTICS

WITH VIRTUAL ACOUSTICS 396 12.4.1 Auralising concert hall acoustics with the loudspeaker

orchestra 397 12.4.2 Selection of stimuli: Concert halls and receiver positions 399 12.4.3 Three studies of individual vocabulary profiling 400 12.4.4 Analysing IVP data 402 12.4.5 Summary of main findings 407

12.5 FUTURE DIRECTIONS 407

CHAPTER 13 • Emotions, Associations, and Sound 411 Jesper Ramsgaard

13.1 WHAT ARE EMOTIONS? 412 13.1.1 Categorical models of emotions 414 13.1.2 Dimensional models of emotions 415 13.1.3 Direct assessment and self-report of emotions 416 13.1.4 Physiological measurements of emotions 419 13.1.5 Neuroimaging techniques 420

13.2 EMOTIONAL RESPONSE TO SOUND AND MUSIC 421 13.2.1 Basic emotions and music 422 13.2.2 Music emotion schemas 422 13.2.3 Dimensional models of emotions and sound/music 425

13.3 ASSOCIATIONS AND MEANING 429 13.3.1 Semantic distance - Sentences and music 429 13.3.2 Free associations 431

13.4 CONCLUSION 434

Contents • xv i i

CHAPTER 14 • Audiovisual Interaction 435 Dominik Strohmeier and Satu Jumisko-Pyykkö

14.1 INTRODUCTION 436 14.2 MULTIMODAL PERCEPTION OF QUALITY 436 14.3 OPEN PROFILING OF QUALITY (OPQ): A STRUCTURED,

MIXED METHODS APPROACH TO STUDY AUDIOVISUAL QUALITY 437

14.4 STUDYING AUDIOVISUAL QUALITY FOR MOBILE 3D VIDEO IN LABORATORY AND FIELD CONDITIONS 440 14.4.1 Introduction 440 14.4.2 Research method 440

14.4.2.1 Test participants 440 14.4.2.2 Test setup 441 14.4.2.3 Test procedure 441 14.4.2.4 Method of analysis 442

14.4.3 Results 443 14.4.3.1 Preference evaluation 443 14.4.3.2 Sensory evaluation and external preference

mapping 444 14.4.3.3 Comparison of results for laboratory and the

actual context of use 446 14.4.4 Discussion and conclusion 447

14.5 PERCEPTUAL DIFFERENCES BETWEEN ONLY-GOOD AND ONLY-BAD QUALITY STIMULI 450 14.5.1 Theories of liking and disliking 450 14.5.2 Research method 451

14.5.2.1 Test participants 451 14.5.2.2 Test setup 451 14.5.2.3 Test procedure 452 14.5.2.4 Method of analysis 453

14.5.3 Results 453 14.5.3.1 Items of only-bad experienced quality 454 14.5.3.2 Items of only-good experienced quality 454 14.5.3.3 Interviews 455

14.5.4 Discussion and conclusion 456 14.6 EXPERIENCED QUALITY OF AUDIOVISUAL DEPTH:

PROFILING AUDIOVISUAL QUALITY PERCEPTION ACROSS ASSESSORS 457 14.6.1 Research method 458

14.6.1.1 Test participants 458 14.6.1.2 Test setup 458

14.6.2 Test procedure 458

xviii • Contents

14.6.3 Sensory evaluation results 458 14.6.4 Interpretation of perceptual spaces between assessors 461 14.6.5 Discussion and conclusion 461

14.7 SUMMARY AND CONCLUSIONS 462

SECTION IV Annexes

APPENDIX A • Description of Data Sets 467

A.l HEADPHONE DATA SETS 467 A. 1.1 Recorded stimuli data sets 467

A.2 CONCERT HALL DATA SETS 469 A.2.1 Data set summary 469

A.3 SONIC BRANDING DATA SET 470 A.4 TRAM NOISE DATA SET 470

Bibliography 471

Index 529