I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
1
ICA 2004, Kyoto, April 4-9, 2004 / Session: HRS43 Paper No. 00413 (Tu.5.X2.5)
Evaluation of Speech Processing Schemes Using Binaural Dichotic
Presentation to Reduce the Effect of Masking in Hearing-Impaired Listeners
By
A.N. CheeranP.C. Pandey
IIT Bombay, Indiahttp://www.ee.iitb.ac.in
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
2
Outline of the presentation
● Introduction ● Processing schemes ● Evaluation ● Test results ● Conclusion
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
3
Introduction (1/9)
Outer ear Inner earMiddle ear
External canal
Pinna
Nasal cavity
Cochlea
Cochlear nerve
Vestibular nerve
Oval windowRound windowEustachian tube
Eardrum (Tympanicmembrane)
Vestibular apparatus with semicircular canals
MalleusIncusStapes
Hearing impairments• Conductive • Sensorineural • Central • Functional
Peripheral auditory system
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
4
Introduction (2/9)
Causes of Sensorineural Loss----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
• Damage to hair cells in the cochlea
• Degeneration of auditory nerve fibers
Sensorineural Loss Characteristics-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
• Increased hearing thresholds
• Reduced dynamic range, loudness recruitment
• Poor freq. selectivity, increased spectral masking
• Poor time resolution, increased temporal masking
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
5
Introduction (3/9)
Effects of Increased Spectral Masking----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-
Broader auditory filters Smearing of spectral peaks & valleys Reduction in internal spectral contrast. Reduced discrimination of consonantal place feature.
Effects of Increased Temporal Masking----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-
Forward & backward masking of weak segments by strong ones. Reduced ability to discriminate sub-phonemic segments
like noise bursts, voice-onset time, formant transitions. Reduced discrimination of duration & other
consonantal features.
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
6
Introduction (4/9)
Dichotic Presentation for Bilateral Residual Hearing
Masking at the peripheral level.
Information integration from the two ears at higher levels.
Binaural dichotic presentation for bilateral residual hearing: speech signal split into a complementary form.
Signal components likely to mask each other presented to different ears.
Improved speech recognition due to reduction in the effects of masking.
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
7
Introduction (5/9)
Binaural Presentation Schemes
Spectral SplittingFiltering by two complementary comb filters: Sensory cells corresponding to alternate filter bands on the basilar membrane not stimulated. Reduction in spectral masking. Better consonantal place reception.
Temporal SplittingGating by two complementary fading functions: Cyclic stimulation and relaxation of all the sensory cells Reduction in temporal masking Better duration and place reception
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
8
Introduction (6/9)
Binaural Presentation Schemes (contd.)
Combined Splitting Processing by two time-varying complementary comb filters
Sensory cells in alternate bands in a given ear stimulated at a time. All the sensory cells on the basilar membrane get periodic relaxation from stimulation.
Reduction in spectral and temporal masking
Better reception of consonantal duration, place, and other features.
Better overall speech recognition & reduction in load on perception process
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
9
Introduction (7/9)
Earlier Work
Lunner et al. [Linkoping, 1993]. • Comb filters using filter bank with constant BW (700 Hz) filters & band gain adjustment : Improvement of 2 dB in SNR. • Combined splitting by symmetrical switching (at 50 Hz) of comb filter bands: No further improvement. Poor sound quality due to switching.
Chaudhari & Pandey [IIT Bombay, 1998]. • Comb filters with auditory critical bands and sharp transitions. Improvement in reception of primarily the consonantal place feature. Reduction in response time. • Individually adjustable frequency response further improvement, but not distinctly for place.
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
10
Introduction (8/9)
Earlier Work (contd.)Jangamashetti & Pandey [IIT Bombay, 2003]
Temporal splitting using symmetrical inter-aural switching of 20 ms along with overlap and various fading functions.
Improvement mainly in the reception of the duration feature
Jangamashetti, Cheeran & Pandey [IIT Bombay, 2003]
Combined splitting using time-varying comb filters with constant sweep cycle of 20 ms and with 2, 4, 8, & 16 shiftings
Improvement in reception of both duration & place features
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
11
Introduction (9/9)
Objective of the present study
Implementation of binaural dichotic presentation schemes & evaluation through listening tests
Spectral splitting with perceptually balanced comb filters
Temporal splitting with trapezoidal fading functions of inter-aural switching periods of 20, 40, 80 ms
Combined splitting with cyclically swept comb filters with sweep cycle of 20, 40, 80 ms and 4, 8, and 16 shiftings
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
12
Parameters:
Transition width: 78 Hz – 117 Hz, Inter-band crossover gains: -4 to -6 dB,
Pass band ripple < 1 dB.
Processing schemes (1/3)
Spectral Splitting with Perceptually Balanced Comb Filters
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
13
Processing schemes (2/3)
Temporal Splitting with Trapezoidal Fading
Parameters:
Inter-aural switching period (N ) : 20, 40, 80 ms
Duty cycle (L) : 70 % of N , Transition duration (M ) : 3 ms
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
14
Processing schemes (3/3)
Combined Splitting with Time-varying Comb Filters
Parameters: Sweep cycle duration : 20, 40, 80, 120, 160 ms No of shiftings (m) : 4, 8, 16
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
15
Evaluation
Listening Tests for Evaluation
Type of evaluation: Open-set
Test material: Phonetically balanced set of monosyllables (used at AYJNIHH, Mumbai) in the first language of the subjects.
Subjects & listening condition: • Seven normal hearing S’s with loss simulated by adding Gaussian noise with short-time SNRs of, 3, 0, -3, -6, -9 dB. MCL (70-75 dB SPL).
Performance measurement: • Response time (indication of load on perceptual processing) • Recognition scores
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
16
Results (1/4) Listening test results
Averaged scores for normal hearing S’s (7) with simulated loss
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
17
Results (2/4)
Averaged Recogn. Scores vs SNR
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
18
Processing scheme Rel. imp. in rec. score (%)
-9 dB SNR -6 dB SNR -3 dB SNR
SS
TS-20
TS-40
TS-80
CS-20/4
CS-20/8
CS-20/16
CS-40/4
142.8 93.2 30.1
31.8 26.9 8.1
23.4 25.4 4.6
-30.4 -21.3 -9.5
82.2 54.5 10.5
83.8 61.4 12.5
90.3 54.1 17.4
97.5 69.3 17.4
Unprocessed (% rec. score) = 23.9 at –9dB, 39.9 dB at –6 dB & 66.3 at –3 dB
Results (3/4)
Relative improvements in recogn. scores
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
19
Processing scheme Rel. imp. in rec. score (%)
-9 dB SNR -6 dB SNR -3 dB SNR
CS-40/8
CS-40/16
CS-80/4
CS-80/8
CS-80/16
CS-120/8
CS-120/16
CS-160/8
CS-160/16
88.6 64.0 16.4
110.6 68.1 21.6
106.8 71.6 27.1
130.2 75.4 26.3
118.2 81.0 27.4
100.9 70.1 22.6
108.5 65.0 24.6
104.3 79.0 24.8
103.4 73.1 22.8
Unprocessed (% rec. score) = 23.9 at –9dB, 39.9 dB at –6 dB & 66.3 at –3 dB
Results (4/4)
Relative improvements in recogn. Scores (contd.)
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
20
Conclusion (1/2)
Summary of results
• Spectral splitting showed highest relative improvement at all SNR condition• For 60 % recognition score spectral splitting gave an improvement of 5 dB SNR
• Combined splitting at sweep cycle 80 ms with 8 and 16 shiftings showed improvements close to spectral splitting• Results similar for 8 and 16 shiftings
• Temporal splitting was best with inter-aural switching of 20 ms• Inter-aural switching of 80 ms showed degradation
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
21
Conclusion (2/2)
Further work
Evaluation by conducting listening tests on moderate bilateral sensorineural hearing-impaired persons.
Combining audiogram dependant filtering and multi-band compression with dichotic presentation.
Implementation of dichotic presentation schemes in wearable hearing aids, with personalized parameter setting.
I
IT B
omba
y{p
cpan
dey ,
alic
enc@
ee.ii
t b.a
c.in
}
• Intro. • Proc. Schemes • Evaluation • Results • Conclusion
ICA
200
4 , K
yoto
, Jap
an, A
pril
4 - 9
, 200
4
22