a new approach for the adaptation of hmms to reverberation and...

Available online at www.sciencedirect.com

www.elsevier.com/locate/specom

Speech Communication 50 (2008) 244–263

A new approach for the adaptation of HMMs to reverberationand background noise

Hans-Gunter Hirsch *, Harald Finster

Niederrhein University of Applied Sciences, Department of Electrical Engineering and Computer Science, Reinarzstr. 49, 47805 Krefeld, Germany

Received 27 October 2006; received in revised form 6 September 2007; accepted 15 September 2007

Abstract

Looking at practical application scenarios of speech recognition systems several distortion effects exist that have a major influence onthe speech signal and can considerably deteriorate the recognition performance. So far, mainly the influence of stationary backgroundnoise and of unknown frequency characteristics has been studied. A further distortion effect is the hands-free speech input in a reverber-ant room environment.

A new approach is presented to adapt the energy and spectral parameters of HMMs as well as their time derivatives to the modifi-cations by the speech input in a reverberant environment. The only parameter, needed for the adaptation, is an estimate of the rever-beration time. The usability of this adaptation technique is shown by presenting the improvements for a series of recognitionexperiments on reverberant speech data. The approach for adapting the time derivatives of the acoustic parameters can be applied ingeneral for all different types of distortions and is not restricted to the case of a hands-free input.

The use of a hands-free speech input comes along with the recording of any background noise that is present in the room. Thus thereexists the need of combining the adaptation to reverberant conditions with the adaptation to background noise and unknown frequencycharacteristics. A combined adaptation scheme for all mentioned effects is presented in this paper. The adaptation is based on an esti-mation of the noise characteristics before the beginning of speech is detected. The estimation of the distortion parameters is based onsignal processing techniques. The applicability is demonstrated by showing the improvements on artificially distorted data as well ason real recordings in rooms.� 2007 Elsevier B.V. All rights reserved.

Keywords: Robust speech recognition; HMM adaptation; Hands-free speech input; Reverberation

1. Introduction

The use of speech recognition as alternative input deviceis especially of interest where the user has not his handsavailable for controlling a keyboard or mouse. Quite oftenthis input mode comes along with the need of a hands-freespeech input. For reasons of practical usage and personalcomfort it is especially of interest without the need of wear-ing a close-talking microphone.

0167-6393/$ - see front matter � 2007 Elsevier B.V. All rights reserved.

doi:10.1016/j.specom.2007.09.004

* Corresponding author. Tel.: +49 2403 702596; fax: +49 2403 702597.E-mail address: [email protected] (H.-G.

Hirsch).

The drawback of a hands-free speech input is a modifi-cation of the speech by the acoustic environment when theinput takes place in a room. The influence of transmittingspeech in a room can be modeled as superposition of thesound on the direct path from the talker’s mouth to therecording microphone and multiple reflections of the soundat the walls and any equipment inside the room. Forstationary conditions the transmission can be modeled asconvolution of the speech with a room impulse response(RIR). But the RIR changes as soon as the talker movesin the room or room conditions change like in case of open-ing a door or a window or other people moving in theroom. Thus the adaptive estimation of the RIR is a quitedifficult and complex task.

mailto:[email protected]

H.-G. Hirsch, H. Finster / Speech Communication 50 (2008) 244–263 245

Several approaches exist for enhancing speech that hasbeen recorded in hands-free mode inside rooms. Some ofthese approaches have also been applied for the improve-ment of speech recognition. The methods can be separatedin single and multi-channel processing techniques. Most ofthe multi-channel approaches (e.g. Omologo et al., 1998;Bitzer et al., 1999; Liu and Malvar, 2001; Seltzer et al.,2004), are based on a beamforming technique to reducethe influence of the reflected sound or on a correlationbased multi-channel processing. A variety of different tech-niques are the basis for the single channel approaches (e.g.Avendano and Hermansky, 1996; Kingsbury, 1998;Yegnanarayana and Murthy, 2000; Gelbart and Morgan,2002; Tashev and Allred, 2005; Wu and Wang, 2005;Kinshita et al., 2005). Some are based on modifying theenvelope contours of subband energies.

Only a few approaches exist that try to improve speechrecognition by modifying the pattern matching process(Couvreur et al., 2001; Palomaki et al., 2002). One method(Raut et al., 2005) is also based on an adaptation and mod-ification of HMM parameters as in the approach presentedhere.

Looking at HMMs that are used for modeling speech inrecognition systems, the detailed knowledge about thetransmission as it is given by a RIR is not needed. Eachstate of a HMM represents a short speech segment withseveral tenth of milliseconds duration. The state containsinformation about the distribution of some spectral param-eters within the segment. Usually a type of MEL filterbankis applied. In general HMMs describe speech with a quitelow resolution with respect to time and frequency in com-parison to the detailed description with a RIR. Thus theestimation of the RIR is not really needed to include themodifications of the HMM parameters that are caused bythe hands-free speech input.

A new approach is presented in the next section foradapting the parameters of HMMs to the speech transmis-sion inside a room. The method is based on the descriptionof a room transmission by an impulse response with anexponentially decaying envelope as approximation for areal RIR. This approximation is applied to the fairly roughmodeling of speech as a sequence of HMM states. As con-sequence of the exponentially decaying shape of theimpulse response, the acoustic excitation at a certain pointin time will also be seen at later time segments. The effect ofreverberation is an temporal extension of an acoustic exci-tation. This extension is modeled by adding contributionsof earlier states with respect to energy and spectral param-eters. The only parameter, needed for the adaptation, is anestimate of the reverberation time T60 that defines the con-tour of the exponentially decaying RIR. Several recogni-tion experiments have been performed to proof theusability of the new approach.

In most hands-free speech input situations backgroundnoise is present in the room. Thus, the HMM adaptationfor the effects of a hands-free speech input is only usefulwhen it can be combined with a technique for compensat-

ing the influence of stationary background noise and of anunknown frequency characteristic. During the recent yearsa lot of investigations have been carried out to reduce thedeterioration of the recognition performance due to addi-tive background noise and unknown frequency characteris-tics. The approaches are either based on the extraction ofrobust features in the front-end (e.g. Macho et al., 2002;Gadrudadri et al., 2002; ETSI, 2003), or on the adaptationof the HMM parameters to the noise conditions (e.g. Gau-vain and Lee, 1994; Minami and Furui, 1996; Sankar andLee, 1996; Gales and Young, 1996; Gales, 1997; Wood-land, 2001). Most of the adaptation techniques try to esti-mate some kind of HMM parameter mapping with amaximization of the likelihood score as optimization crite-rion. This work uses signal processing approaches for esti-mating the distortion effects. The estimated distortionparameters are taken for the adaptation on the basis of asignal processing model that describes the spectral modifi-cations due to the distortions. A frequency weighting canbe caused, e.g. by the special characteristics of the record-ing microphone. In case of a hands-free speech input afrequency weighting might also occur due to the frequency-dependency of T60. This is not covered by the newapproach which assumes one frequency independent valuefor T60 so far. But it can be compensated with an addi-tional adaptation to unknown frequency characteristics.

This paper shows how the new adaptation technique canbe combined with the adaptation of HMMs to additivenoise and unknown frequency characteristics (Hirsch,2001a). This approach is based on the well known PMCmethod (Gales, 1995). The results of several recognitionexperiments are presented in the last chapter that proofthe usability of the combined adaptation to all distortioneffects as they can occur in real application scenarios ofspeech recognition systems. The achieved results arecompared to the application of the well-known MLLR(maximum likelihood linear regression) approach (Leggeterand Woodland, 1995) as an alternative adaptationtechnique.

2. Adaptation to hands-free speech input

The new approach for adapting the energy and spectralparameters of HMMs will be derived in this section. It isbased on the approach of modeling the transmission in areverberant room by an impulse response with an ideal,exponentially decaying shape. This will be presented inSection 2.1. Section 2.2 describes the adaptation of the sta-tic energy and spectral parameters of HMMs derived fromthe ideal modeling of the RIR. Finally Section 2.6 presentsa new technique to adapt also the time derivatives of theenergy and spectral HMM parameters.

2.1. Modeling the influence of a hands-free speech input

The multiple reflections of sound in a room can be ide-ally described by an exponential decay of the acoustic

246 H.-G. Hirsch, H. Finster / Speech Communication 50 (2008) 244–263

energy which has been the result of early investigations inroom acoustics (Kuttruff, 2000). This leads to a roomimpulse response h(t) with an exponentially decaying shape

EðtÞ ¼ E0 � e�6�lnð10Þ

T 60�t

with

E0 � energy of the acoustic excitation

) h2ðtÞ � e�6�lnð10Þ

T 60�t: ð1Þ

The only parameter for the description of the exponen-tial shape is the reverberation time T60 that takes approx-imately values in the range of about 0.2–0.4 s for smallerrooms and of about 0.4–0.8 s for larger rooms. It can takevalues above 1 s for very large rooms like churches. Thereverberation time depends on the interior in the roomand the individual absorption characteristics of the walls.

The RIR can be transformed to the room transfer func-tion by means of a Fourier transform. The room transferfunction has a contour that changes very fast along fre-quency. Usually only the envelope of the room transferfunction is of interest when looking at the filterbankapproaches that are applied for extracting acoustic featuresin speech recognition. This effect can be covered by anadaptation to an unknown frequency characteristic.

More important for the frame based analysis in speechrecognition is the influence on the contour of the short-term energy along time. The energy contours of a speechsignal are shown in Fig. 1 before and after the transmissionin a room. The energy is usually estimated as short-termenergy in frames of about 20 ms duration. It can be seenthat the reverberation leads to an artificial extension ofeach sound contribution. This extension occurs as so calledreverberation tail with the exponentially decaying shape ofthe RIR. The same effect will also be seen when looking atthe energy contours in single subbands of a MEL based filt-erbank that is usually applied in the front-end of a speechrecognition system.

0 200 400 600 800 1000 1200 1400time/ms

ener

gy

clean

reverberated

Fig. 1. Energy contours of a speech signal in clean and reverberantcondition.

Transforming such energy contours to the so calledmodulation spectrum by means of a Fourier transformleads to the estimation of the modulation transfer functionm(F) (Houtgast et al., 1980) which can be mathematicallydescribed as

mðF Þ ¼ 1ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1þ 2 � p � F � T 60

6�ln 10ð Þ

� �2r : ð2Þ

The low pass characteristic of the modulation transferfunction is shown in Fig. 2 for different values of T60.The cut-off frequency of the low pass characteristic is shift-ing to lower values of the modulation frequency forincreasing values of T60. This corresponds to longerreverberation tails for higher values of T60. The artificialextension of sound contributions can lead to masking theacoustic parameters of low energy sounds by the parame-ters of a preceding sound with higher energy.

2.2. Adaptation of static parameters

Looking at a sequence of HMM states the acoustic exci-tation described by the parameters of a single state will alsooccur in succeeding states at a certain attenuation. This isbased on the assumption that the HMMs have been trainedon clean speech, recorded with a close talking microphone.Fig. 3 tries to visualize this effect.

Each state Si of a HMM describes a speech segmentwith an average duration dur(Si) that can be derived fromthe transition probability p(SijSi) to remain in this state.

durðSiÞ ¼1

1� p SijSið Þ � tshift

for all states Si with 1 6 i 6 NR states; ð3Þ

where tshift is the time for shifting the analysis window inthe feature extraction and NR_states is the number of

0 5 10 15 20 250

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

modulation frequency/Hz

mod

ulat

ion

tran

sfer

fun

ctio

n

T60 = 0.3 sT60 = 0.6 sT60 = 1.2 s

Fig. 2. Modulation transfer functions for different values of T60.

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.40

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

time/s

h(t)

te(S

3)t

s(S

3)

S1 S

3S

2S

4

Fig. 3. The distribution of energy at state S1 due to reverberation.


states of an individual HMM. tshift takes a value of 10 msfor all analysis schemes applied in these investigations. Thiscorresponds to a frame rate of 100 Hz.

The energy contribution of the acoustic excitation in thefirst state to a succeeding state Si can be estimated by inte-grating the squared RIR as defined by Eq. (1) over the timesegment of the succeeding state

ai;1 ¼Z te Sið Þ

ts Sið Þh2ðtÞot where tsðSiÞ ¼

Xi�1

j¼1

durðSjÞ and teðSiÞ

¼ tsðSiÞ þ durðSiÞ and

Z 1

0

h2ðtÞot ¼ 1: ð4Þ

Given an estimate of the reverberation time T60 the con-tribution factors can be individually calculated for all statesof all HMMs. The approach for estimating T60 will bedescribed later.

Usually each state of a HMM is defined by a set of spec-tral parameters like the MEL frequency cepstral coeffi-cients (MFCCs) and an energy parameter. Theseparameters are the means of Gaussian distributions. Thecorresponding variances are needed as further parametersto completely define the shape of the Gaussians.

The mean of an energy parameter at an individual stateSi can be adapted by adding the energy contributions of thestate itself and the preceding states

~EðSiÞ ¼ ai;i � EðSiÞ þ ai;i�1 � EðSi�1Þ þ ai;i�2 � EðSi�2Þ þ � � �

¼Xi

j¼1

ai;j � EðSjÞ: ð5Þ

In the same way the means of the power density spectracan be adapted. In case of using MFCCs, the cepstral coef-ficients have to be transformed back to the spectral domainfirst.

C0;C1;C2; . . . ;CNR cep

� �!IDCT

log jX 1jð Þ; log jX 2jð Þ; . . . ;f

log jX NR meljð Þg !EXP jX 1j; jX 2j; . . . ; jX NR meljf g; ð6Þ

where NR_cep is the highest index of the cepstral coeffi-cients and NR_mel is the number of bands in the MEL fre-quency range. For NR_cep a value of 12 is chosen andNR_mel takes a value of 24 in our realization. jXkj repre-sents the value of the magnitude spectrum in the MELband with index k.

Then the power density spectra can be adapted in thesame way as the energy parameter

j~X kðSiÞj2 ¼ ai;i � jX kðSiÞj2 þ ai;i�1 � jX kðSi�1Þj2

þ ai;i�2 � jX kðSi�2Þj2 þ � � �

¼Xi

j¼1

ai;j � jX kðSjÞj2 for 1 6 k 6 NR mel: ð7Þ

The adapted spectra ~X have to be transformed to theMFCCs again. In practice, mainly 2–3 preceding HMMstates have an influence on the current state. This dependson the reverberation time and on the average durations ofthe HMM states.

The variances are not adapted. It turned out in earlierinvestigations (Gales, 1995; Hirsch, 2001a) that the modifi-cation of the variances has only a minor influence on theimprovement of the recognition performance.

The effects of this adaptation approach are visualized inFig. 4 in the spectral domain by comparing the spectro-grams as representations of different HMMs. Spectrogramsare shown in a three dimensional visualization mode.

The spectrogram of a HMM is estimated by transform-ing the MFCCs back to the linear spectral domain for allstates. The transition probabilities p(SijSi) to remain in astate are taken to model the average duration of this stateas defined by Eq. (3). In Fig. 4 three spectrograms of differ-ent HMM versions are shown for the word ‘‘six’’. EachHMM consists of 16 states where a single state is describedby a set of cepstral coefficients including the zeroth cepstralcoefficient C0. The spectrum of an individual HMM state ispositioned with respect to its point in time tSi in the middleof the segment that is described by this state.

tSi ¼Xi�1

j¼1

dur Sj

� �þ dur Sið Þ

2: ð8Þ

Furthermore a Spline interpolation is applied to thecontour of the magnitude spectral values in each MELband.

jX kðtS1Þj; jX kðtS2

Þj; jX kðtS3Þj; . . .

� �!Spline jX kð0Þj; jX kð10 msÞj; jX kð20 msÞj; . . .f g: ð9Þ

Thus the spectrum can be recreated at a frame rate of100 Hz as it is also defined by the window shift of 10 msin the feature extraction.

0

200

400

600 0

1000

2000

3000

trained on clean

t/ms

f/Hz

0200

400600

0

1000

2000

3000

trained on reverberation

t/ms

f/Hz

0

200

400

600 0

1000

2000

3000

with adaptation

t/ms

f/Hz

a b

c

Fig. 4. Different spectrograms of HMMs for the word ‘‘six’’.


The spectrogram of the clean HMM is shown in graph(a) of Fig. 4 as it has been trained with the utterances ofthe TIDigits data base (Leonard, 1984). Only the utter-ances containing a single digit have been taken for thetraining. The contributions of the fricatives at the endcan be clearly seen in the high frequency region wherethe formants of the vowel are visible in the middle of theword.

The spectrogram in graph (b) represents the HMM thathas been trained on the TIDigits data after applying anartificial reverberation to the training utterances. Thereverberation tails can be clearly seen when looking atthe contours in individual MEL bands.

The spectrogram in graph (c) of Fig. 4 represents theHMM after adapting the clean HMM with the newapproach. A fixed value is chosen for the reverberationtime T60. The reverberation tails can also be seen in thisfigure. Comparing it with the spectrogram trained on rever-berated data, a lot of similarities are visible. This showsthat the new approach allows an adaptation of the staticparameters that is comparable with training the HMMon data that have been recorded under reverberantconditions.

In practice each HMM state tries to model a certainspeech segment with a mixture of Gaussian distributionsfor each feature component. In this case the energy param-eter at state Si and for the mixture component with theindex mixj is adapted by calculating

~EðSi;mixjÞ ¼ ai;i � EðSi;mixjÞ þ ai;i�1 � EðSi�1Þþ ai;i�2 � EðSi�2Þ þ � � �

¼ ai;i � EðSi;mixjÞ þXi�1

‘¼1

ai;‘ � EðS‘Þ

for all Si with 1 6 i 6 NR states

and all mixj with 1 6 j 6 NR mix; ð10Þ

where EðS‘Þ is the average energy at state S‘ by weightingthe energies of the different mixture components with thecorresponding mixture weighting factors

EðS‘Þ ¼XNR mix

j¼1

wðmixjÞ � EðS‘;mixjÞ with

XNR mix

j¼1

wðmixjÞ ¼ 1: ð11Þ

NR_mix is the number of Gaussian distributions for mod-eling this acoustic parameter in each individual HMMstate.

Thus, the influence of an earlier state is considered byusing the average energy at this earlier state.

In the same way the power density spectra of individualmixture components are adapted by transforming back theset of average cepstral coefficients to the spectral domainfirst and adapting the subband energy in each Mel bandin the same way as the energy parameter.


Cm ¼XNR mix

j¼1

wðmixjÞ � CmðmixjÞ for 1 6 m 6 NR cep;

ð12ÞC0;C1;C2; . . . ;CNR cep

� �!IDCT

log jX 1j� �

; log jX 2j� �

; . . . ; log jX NR melj� ��

!EXP jX 1j; jX 2j; . . . ; jX NR melj� �

; ð13Þ

j~X kðSi;mixjÞj2 ¼ ai;i � jX kðSi;mixjÞj2 þXi�1

‘¼1

ai;‘ � jX kðS‘Þj2:

ð14Þ

The adapted power density spectrum is transformed tothe cepstral domain again.

2.3. Estimation of T60

The estimated reverberation time T60 is the only param-eter that is needed for the adaptation as it has beendescribed in the previous section. The recognition of anutterance is done with a set of adapted HMMs where theapplied value of T60 has been estimated from the recogni-tion of the previous utterance. T60 is estimated after therecognition of an utterance by a search for this set ofadapted HMMs that leads to a maximum likelihood foranother forced recognition of the already recognizedsequence of HMMs. The restriction to the forced recogni-tion of the already recognized HMM sequence is intro-duced to limit the computational costs. This iterativeprocess is visualized in Fig. 5.

The sequence of buffered feature vectors is used to per-form the match with the previously recognized HMMsequence. At the beginning this match is performed withadapted HMMs for the previous estimate of T60 and forvalues of T60 that differ by ±20 ms. The values for theprobability, that the feature vectors match with thesequence of adapted HMMs, are taken as input for thesearch of this T60 value that leads to a maximum likeli-hood. Dependent on the achieved probabilities the esti-mated value of T60 is lowered or increased by another20 ms or the search process is stopped in case the previousestimate of T60 leads to the maximum likelihood. In casethe search process is continued, another set of adapted

Fig. 5. Estimation of T60 by an iterative

HMMs is derived from the clean HMMs where the adapta-tion can be restricted to the previously recognized HMMs.This newly adapted HMMs are used for another forcedmatch and the search for the maximum likelihood. Becausethe hands-free conditions will usually alter only slowly inpractical applications, the modification of T60 is restrictedto the range of ±40 ms from the previous estimate. Thusonly a few matches are needed, so that the computationalcosts are fairly low.

It turns out that the estimated value of T60 varies fordifferent speakers even though the hands-free condition isthe same. This seems to be dependent on the speaking rate.Thus, this adaptation technique includes also a kind ofduration modeling to some extent.

2.4. Adaptation of delta parameters

Comparing the contours of the clean and the reverber-ant HMM at individual Mel bins it becomes obvious thatalso the Delta and Delta–Delta parameters as time deriva-tives of the static parameters are modified by the influenceof the hands-free speech input. This can be seen for exam-ple in Fig. 4 where a ‘‘valley’’ is visible between the voweland the succeeding phoneme for the clean HMM. This‘‘valley’’ is filled by the reverberation tails for the reverber-ant HMM versions. This indicates that also the time deriv-atives will be different in this region.

The Delta parameters are calculated in the featureextraction for the frame at time ti as sum of weighted dif-ferences between the static parameters of preceding andsucceeding frames (Young et al., 2005). For example thecalculation of the Delta logarithmic energy DlogE(ti) isdone as

D log EðtiÞ ¼P3

j¼1j � log EðtiþjÞ � log Eðti�jÞ

norm

with norm ¼ 2 �X3

j¼1

j2; ð15Þ

where . . . , ti�1, ti, ti+1, . . . describe the window shift by10 ms.

The Delta parameters of the reverberant speech are esti-mated as described below by looking at the adapted staticparameters of all HMM states. The average logarithmic

search of the maximum likelihood.


frame energies of all states are considered for a singleHMM. A Spline interpolation is applied to recreate theaverage energy contour at a frame rate of 100 Hz.

log EðtS1Þ; log E tS2

ð Þ; log E tS3

� �; . . .

� �!Spline

log Eð0Þ; log E 10 msð Þ; . . . ; log E tið Þ; . . .� �

: ð16Þ

In the same way an interpolated version of the averageenergy contour can be calculated for the average frameenergies of the clean HMM.

The procedure for calculating the Deltas in the featureextraction, as described by Eq. (15), can be applied to theinterpolated energy contours of the clean and the adaptedHMM. Thus the average logarithmic Delta energiesD log EcleanðtiÞ and D log EadaptedðtiÞ become available forthe clean and for the adapted HMM version at the framerate of 100 Hz (ti = 0,10 ms, 20 ms, . . .). As the number offrames is equal for the clean and the adapted HMM ofan individual word class the difference between the cleanand the adapted Delta energies can be calculated

D log EdiffðtiÞ ¼ D log EadaptedðtiÞ � D log EcleanðtiÞfor all ti ¼ 0; 10 ms;20 ms; . . . ð17Þ

These values describe the average differences betweenthe Delta logarithmic energies of the adapted and the cleanHMM at each frame. By means of a Spline interpolationthe average differences are calculated for all HMM states.

D log Ediffð0Þ;D log Ediffð10 msÞ; . . . ;D log EdiffðtiÞ; . . .� �!Spline

D log EdiffðtS1Þ;D log EdiffðtS2

Þ;D log EdiffðtS3Þ; . . .

� �:

ð18Þ

A weighted version of these average differences is addedto the corresponding Delta parameters of the clean HMMto create a set of adapted Delta parameters.

D log ~EðtSi ;mixjÞ ¼ D log EcleanðtSi ;mixjÞ þ b � D log EdiffðtSiÞfor all Si with 1 6 i 6 NR states and all mixj with

1 6 j 6 NR mix: ð19Þ

This is done individually for each state and for each mix-ture component. A factor b is introduced for the weightedsummation of the differences. During recognition experi-ments we found a value of 0.7 for b to achieve highestperformance.

The Delta cepstral parameters can be adapted in thesame way. The average logarithmic Mel spectral valuesare taken as basis as they can be calculated by Eqs. (12)and (13) from the average cepstral coefficients for eachHMM state. A Spline interpolation can be applied to recre-ate the contour of the logarithmic Mel magnitude in eachMel band at the frame rate of 100 Hz.

log jX kðtS1Þj; log jX kðtS2

Þj; log jX kðtS3Þj; . . .

� �!Spline

log jX kð0Þj; log jX kð10 msÞj; . . . ; log jX kðtiÞj; . . .� �

for k ¼ 1; 2; . . . ;NR mel: ð20Þ

The logarithmic spectral domain seems to be the rightdomain for applying the Spline interpolation even thoughthe interpolation could also be applied to the averagecepstral parameters. The interpolated average logarithmicspectra are transformed to the cepstral domain

log jX 1ðtiÞj� �

; log jX 2ðtiÞj� �

; . . . ; log jX NR melðtiÞj� ��

!DCTC0ðtiÞ;C1ðtiÞ; . . . ;CNR cepðtiÞ� �

for ti ¼ 0; 10 ms; . . .

ð21Þ

The Delta coefficients can be calculated for the contourof each individual average cepstral coefficient. This can bedone again for the clean as well as for the adapted HMMso that the difference between these two versions can beestimated

DCmdiffðtiÞ ¼ DCmadapted

ðtiÞ � DCmcleanðtiÞ

for all ti ¼ 0; 10 ms;20 ms; . . .

and for 1 6 m 6 NR cep: ð22Þ

These values describe the average differences betweenthe Delta cepstral coefficients of the adapted and the cleanHMM at each frame. By means of a Spline interpolationthe average differences are calculated individually for eachcepstral coefficient for all HMM states

DCmdiffð0Þ;DCmdiff

ð10 msÞ; . . . ;DCmdiffðtiÞ; . . .

� �!Spline

DCmdiffðtS1Þ;DCmdiff

ðtS2Þ;DCmdiff

ðtS3Þ; . . .

� �for 1 6 m 6 NR cep: ð23Þ

The adapted cepstral coefficients can be calculated byadding a weighted version of the average differences tothe Delta coefficients of the clean HMM

D~CmðSi;mixjÞ ¼ DCmcleanðSi;mixjÞ þ b � DCmdiff

ðSiÞfor all Si with 1 6 i 6 NR states

and all mixj with 1 6 j 6 NR mix

and for 1 6 m 6 NR cep: ð24Þ

The adaptation of each cepstral coefficient is done indi-vidually for each state and for each mixture component.The value of b is the same as applied in Eq. (19) for theenergy.

Fig. 6 summarizes and visualizes this new technique forcalculating the differences between the Delta coefficientsthat can be derived from the static parameters of the cleanand the adapted HMMs. It is not only applicable in case ofadapting HMMs to the conditions of a hands-free speechinput. This method can be applied in any case of adaptingthe static parameters of HMMs to the influence of distor-tion effects.

The Delta–Delta parameters can be adapted in the sameway as the Delta parameters. The Delta–Delta parameters

Apply DCT

Clean cepstra

Average difference between Delta cepstra of clean and adapted HMMs

Clean Delta cepstra

Calculate Deltas

Adapted Delta cepstra

Adapted cepstra

+

+-

Apply DCT

Calculate Deltas

Interpolated spectraframe rate = 100 Hz

Clean average logarithmic spectra Adapted average logarithmic spectra

Calculate Deltas

Fig. 6. Scheme for estimating the differences between the Delta parameters of clean and adapted HMMs.


are calculated from the Delta parameters in the same waythe Delta parameters are determined from the staticparameters. Looking at Eq. (15) the only difference is avalue of 2 for the higher summation index. This results ina calculation of the Delta–Delta parameters over five setsof Delta parameters. Otherwise the adaptation of theDelta–Delta parameters is done as described by Eqs.(16)–(24).

2.5. Restrictions in case of a connected word recognition

The complete adaptation scheme as described aboveworks well for whole word HMMs in case of an isolatedword recognition. For the recognition of connected wordsthe adaptation will not be perfect when uttering a sequenceof words without even short pauses between the words. Thebeginning of a word is modified by the reverberation tail ofthe acoustic excitation at the ending of the preceding word.These effects can be taken into account only in a type ofonline adaptation. To get knowledge about the precedingword, the log likelihood is observed at the final states ofall HMMs during the frame-wise recognition with theViterbi algorithm. The adaptation of the first frames ofall HMMs can be done when a high log likelihood is com-puted for the final states of a HMM so that is very likelythat the corresponding word was spoken. Thus the energyand spectral parameters, that are contained in the final

states of the HMM with the high likelihood, can be usedto adapt the first states of all HMMs.

The implementation of this online technique is quitecomplex because it is based on a frame-wise decision pro-cess whether and which HMM creates a high likelihoodat its final states. The authors implemented this approachbut could find only small improvements for a connecteddigit recognition with respect to word accuracy. Alsobecause of the high computational effort the approachwas not investigated further.

2.6. Adaptation of triphone HMMs

Most often triphone HMMs are used in case of a pho-neme based recognition which is used for the recognitionof large vocabularies. A triphone HMM is applied tomodel the acoustic characteristics of a phoneme in thecontext of a specified preceding and a specified succeedingphoneme. Thinking about the adaptation of this triphoneHMM, the knowledge about the preceding phoneme andthe succeeding phoneme can be taken to apply the adapta-tion approach as it was presented for the application towhole word HMMs in the previous sections. Looking ata single triphone HMM as it is done for the triphone‘‘s-I-k’’ in Fig. 7, the number of possible preceding andsucceeding triphone HMMs is restricted.

The triphone HMM ‘‘s-I-k’’ for the vowel ‘‘i’’ in thecontext of a preceding ‘‘s’’ and a succeeding ‘‘k’’ will be

Fig. 7. Possible preceding and succeeding triphone HMMs for one selected triphone HMM.


preceded by a triphone HMM ‘‘unknown-s-I’’ for the fric-ative ‘‘s’’ with the vowel ‘‘i’’ following. Only the precedingphoneme of ‘‘s’’ is not defined. In the same way a triphoneHMM ‘‘I-k-unknown’’ for the ‘‘k’’ will succeed thetriphone HMM for the ‘‘i’’. This knowledge about thesequence of HMMs and hence also about the completesequence of states enables the applicability of the newapproach for the adaptation to reverberation. Looking atthe sequence of three triphones as shown in Fig. 7, thecomplete sequence of nine HMM states for all threemodels is used for the adaptation. It is done in the sameway as it has been described for the whole-word HMMsbefore.

Thus, the problem of being unable to adapt the firststates of a whole word HMM, does not exist in case of tri-phone models where a certain knowledge about the preced-ing model is available.

This preceding model ‘‘unknown-s-I’’ is chosen from allavailable triphone HMMs by looking at the spectral simi-larity between the last state of each triphone HMM‘‘unknown-s-I’’ and the first state of the HMM ‘‘s-I-k’’.The spectral similarity is estimated by calculating theEuclidean distance between the average cepstral coefficientsof the first state of ‘‘s-I-k’’ and the corresponding coeffi-cients of the last state of the preceding phoneme. The tri-phone HMM with the smallest spectral distance is selected

MINunknown2 all phonemesf g

XNR cep

m¼1

Cm S1ð\s-I-k"Þf g

�Cm Slast \unknown-s-I"ð Þf g2: ð25Þ

In case of modeling with a mixture of Gaussian distribu-tions the average cepstral coefficients are calculated bytaking into account the mixture weights

Cm ¼XNR mix

j¼1

w mixj

� �� CmðmixjÞ for m ¼ 1; 2; . . . ;NR cep:

ð26Þ

In the same way the preceding triphone is chosen bycomparing the last state of ‘‘s-I-k’’ with the first state ofall triphones ‘‘I-k-unknown’’

MINunknown2 all phonemesf g

XNR cep

m¼1

Cm Slast \s-I-k"ð Þf g

�Cm S1 \I-k-unknown"ð Þf g2: ð27Þ

The static and the Delta parameters are adapted for thestates of the HMM ‘‘s-I-k’’ by looking at the whole statesequence of all three consecutive triphone HMMs andapplying the adaptation technique as described for thewhole-word HMMs. By transforming back the nine setsof cepstral coefficients to the spectral domain and applyinga Spline interpolation the spectrogram for the completesegment of the three triphones is available. The knowledgeof the preceding phoneme has the advantage that theenergy and spectral information of these states can be usedto adapt the succeeding states of the HMM ‘‘s-I-k’’ withrespect to reverberation. Hence, the drawback of notknowing the preceding HMM for the adaptation of the firststates does not exist as it occurs in case of whole-wordHMMs. The knowledge of the succeeding triphone is onlyused for the better estimation of the adapted Deltaparameters.

This adaptation technique can be applied to all triphoneHMMs that are used for the recognition. In case of usingstate tying a further tying can be applied after the adapta-tion. This has not been investigated here. The authors onlyintended to demonstrate the principal applicability of thenew adaptation method to the modeling with triphoneHMMs.

3. Recognition experiments on hands-free speech input

Recognition experiments have been run to proof theapplicability of the new adaptation approach and to quan-tify the improvements that can be achieved.

Some details about the applied feature extraction andthe recognition are presented at the beginning of this chap-ter. After this the results are shown for a series of experi-ments with the intention to demonstrate the applicabilityto a word recognition based on whole word HMMs.Finally the improvements are presented separately for therecognition of a large vocabulary based on the use of tri-phone HMMs.

3.1. Feature extraction

The acoustic features are extracted from the speech sig-nal by a cepstral analysis scheme that is similar to manyrealizations in this field. A pre-emphasis is applied to thespeech signal by means of a first order FIR filter wherethe value of the preceding sample is weighted by a factorof 0.95 before subtracting it from the value of each sample.Short segments of speech are extracted with a 25 ms


Hamming window. The window is shifted by 10 ms whichcorresponds to a frame rate of 100 Hz. Each speech frameis transformed to the spectral domain by means of a 256point DFT (Discrete Fourier Transform). The so calledMEL spectrum is estimated by weighting the values ofthe DFT magnitude spectrum with triangular shaped func-tions and summing up the spectral magnitudes for each tri-angular. Thus, a MEL spectrum is computed for 24nonlinearly distributed frequency bands in the range from200 Hz up to 4000 Hz. The 24 logarithmic MEL spectralvalues are transformed to the cepstral domain by meansof a DCT (Discrete Cosine Transform). Thirteen cepstralcoefficients C0 to C12 are calculated. Thus, C0 is availableas acoustic parameter in each state of all HMMs. C0 is onlyneeded to transform back the cepstral coefficients to thespectral domain as part of the adaptation process. ButC0 is not used for the recognition. Instead of C0 an energyparameter is estimated from the preemphasised and Ham-ming weighted speech samples. A preemphasis factor of �1is applied here which leads to a slightly higher attenuationof the low frequency components. This is of advantage inthe presence of background noise with its main energy atlow frequencies. The short-term energy is calculated bysumming up the squared values of all samples in each25 ms frame.

The described analysis technique is applied to speechdata sampled at 8 kHz. In case of data sampled at16 kHz the same MEL filterbank is applied in the fre-quency range up to 4000 Hz. All filter characteristics likee.g. the preemphasis filtering or the MEL filters and allother individual settings like e.g. the frame length or theFFT length are chosen in an appropriate way so that thefinal cepstral coefficients are almost identical to the coeffi-cients which result from an analysis of the same utterancesampled at 8 kHz. This approach has been investigated inearlier work (Hirsch et al., 2001b). It allows for examplethe recognition of speech data sampled at 8 kHzwith HMMs that have been trained on data sampled at16 kHz.

Besides the 13 cepstral coefficients and the energyparameter in the range up to 4 kHz two further parametersare calculated in case of data sampled at 16 kHz. These aretwo energy parameters that describe the energy in the fre-quency range from 4 to 5.5 kHz respectively in the rangefrom 5.5 to 8 kHz. This is realized by summing up the cor-responding components of the FFT power density spec-trum. These additional coefficients can help to increasethe recognition performance a bit in comparison to the caseof recognizing data sampled at 8 kHz.

Twelve cepstral coefficients C1–C12 and the logarithmof the energy parameter are used as acoustic parametersfor the recognition. Furthermore Delta and Delta–Deltacoefficients are added as additional features where theDelta parameters are calculated as described by Eq. (15).The Delta calculation corresponds to the way of estimatingDelta parameters in the HTK software package (Younget al., 2005).

Thus, finally a feature vector consists of 39 componentsin case of speech data sampled at 8 kHz and it consists of45 components in case of data sampled at 16 kHz.

The parameters of the HMMs are determined by apply-ing the available training tools of the HTK software pack-age. The recognition is done either with an own Cimplementation of a Viterbi recognizer or with the corre-sponding tool of the HTK package. It has been verified thatthe own implementation leads to the same recognitionresults as the HTK recognizer. Furthermore the Viterbi rec-ognizer has also been implemented as Matlab module. Thiswas helpful during the development process of the adapta-tion algorithms due to the easier software development withMatlab and its graphical visualization properties.

The adaptation techniques have been implemented asMatlab and as C modules. The adaptation is individuallyapplied to each speech utterance when detecting the begin-ning of speech. The applied voice activity detector takes theMEL magnitude spectrum as input. It will be described abit more in detail later. The C modules for the analysisand the recognition are designed for an application in areal-time recognition and dialogue system (Hirsch, 1999).This means that the Viterbi match can be started and runin parallel to the feature extraction after the detection ofspeech. Techniques like a cepstral mean normalization onthe whole utterance are not considered here because theywould delay the beginning of the Viterbi match till thedetection of the end of speech.

3.2. Recognition with whole-word HMMs

The TIDigits data base is taken as basis for the experi-ments on isolated and connected word recognition. A ver-sion of the TIDigits is used that has been downsampled at8 kHz. All utterances from the adult speakers designatedfor training are taken to determine two gender dependentHMMs for each word. Each HMM consists of 16 stateswhere each state is described by the mixture of two Gauss-ian distributions for each of the 39 acoustic features. A sin-gle state HMM with a mixture of eight Gaussians is usedfor modeling the pauses. The HMMs are defined as left-to-right models without skips over states. The recognizeris set up to recognize any sequence of digits with the restric-tion that a sequence contains only models from the samegender.

3.2.1. Recognition of single digitsA first series of experiments focused on these test utter-

ances that contain only a single digit. These are about 2500utterances in total. This is done to avoid the inter-wordeffects between fluently spoken words without pausesbetween the words. As already mentioned before, the rever-beration will influence the beginning of a word by the end-ing of the preceding word.

The word error rates are shown in Fig. 8 where therecognition of connected words is still enabled so that alsoinsertion errors can occur. Results are presented for three

0

1

2

3

4

5

6

7

8

wor

d er

ror

rate

/%

Recognition of single TIDigits

Clean

Office

Living RoomRobust ETSI frontendcepstral analysis without adaptationadaptation to reverberationadapt. to reverberation & Deltas

Fig. 8. Word error rates for these TIDigits utterances that contain onlysingle digits.

Table 1Word error rates for the recognition of single TIDigits applying thecepstral analysis

Condition

Clean(%)

Office room(%)

Living room(%)

Without adaptation 0.44 3.49 6.94Adaptation to reverberation &

Deltas0.44 1.57 2.05

HMMs trained on living room 10.98 1.93 1.73


different conditions. Besides the clean data the TIDigitshave been processed with a tool for simulating the influenceof a hands-free speech input (Hirsch and Finster, 2005).This tool creates fairly natural sounding reverberation. AWeb interface exists to experience this tool (Finster,2005). Test data have been created for the simulation oftwo different rooms, an office room with a reverberationtime of about 0.4 s and a living room with a reverberationtime of about 0.6 s. As expected the error rates are higherfor the longer reverberation time.

For each condition four different processing methodsare compared. The first one is based on the advancedfront-end as it has been standardized by ETSI (2003).Robust acoustic features are extracted with this front-end. The term robustness refers to the presence of back-ground noise and unknown frequency characteristics. Thisis realized by extending a cepstral analysis scheme by twofurther processing steps. The first one contains a Wiener fil-tering based on an estimation of the noise spectrum toreduce the influence of stationary background noise. Ablind estimation and equalization of unknown frequencycharacteristics has been integrated as second processingblock. Each feature vector consists of 39 parameters. Theseare 12 Mel frequency cepstral coefficients and an energyparameter as well as the corresponding Delta and Delta–Delta parameters. Feature vectors are computed at a rateof 100 Hz. This front-end is considered as a representativefor a robust feature extraction and is taken as reference forcomparing the results with the HMM adaptation.

The cepstral analysis scheme as it has been described inthe previous section is investigated as second method.Word error rates are presented for the three cases wherethe recognition is done

– without any adaptation or– with adaptation of the static parameters only or– with adaptation of the static and the Delta and Delta–

Delta parameters.

The error rates for the clean data are in the range of 0.4–0.5%. It can be seen that the influence of the reverberationleads to a considerable deterioration of the recognitionrates for both feature extraction schemes. The high errorrate of the cepstral analysis scheme in comparison to theETSI front-end in case of the living room condition isdue to a high number of insertion errors. The number ofsubstitutions is even lower for the cepstral analysis schemein comparison to the ETSI scheme.

Error rates can be reduced by applying the adaptationmethods. Adapting also the Delta and Delta–Delta param-eters leads to an additional gain in both reverberantsituations.

The efficiency of the adaptation scheme is investigatedby comparing the obtained results to the case of trainingthe HMMs on reverberated data. Therefore a set of HMMsis trained with all TIDigits training utterances after apply-ing the simulation of a reverberation in the living room.

Results are listed in the first two lines of Table 1 for thecases without adaptation and with adaptation of the staticand Delta parameters and taking HMMs trained on cleandata only. These are the error rates as already shown inFig. 8. The third line of Table 1 contains the error ratesfor the case of applying the HMMs trained on reverberateddata and without any adaptation.

The error rate decreases from about 7% to 1.7% for theliving room condition when moving from the training onclean data and applying no adaptation to the training onreverberant data. Applying the adaptation scheme to theclean HMMs leads to an error rate quite close to the caseof training on reverberated data. This can be taken as fur-ther proof for the usefulness of the applied adaptationmethod.

The drawback of training the HMMs on reverberantdata is a considerable increase of the error rate to about11% for the recognition of clean data. This indicates thatthe training has to be done on a mixture of conditionsfor a practical application. And this can only be done ifthe whole range of conditions is known in advance.

3.2.2. Connected word recognition

The word error rates are shown in Fig. 9 for the recog-nition of all TIDigits utterances that have been designatedfor recognition. These are 8700 utterances containingabout 28000 digits in total. The results are presented for

0

2

4

6

8

10

12

14

16

18

20

wor

d er

ror

rate

/%

Recognition of TIDigits

Clean

Office

Living Room

Robust ETSI frontendcepstral analysis without adaptationadaptation to reverberationadapt. to reverberation & Deltas

Fig. 9. Word error rates for the recognition of all TIDigits.

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

5

10

15

20

25

30

35

40

reverberation time/s

wor

d er

ror

rate

/%

Recognition of TIDigits

robust ETSI frontendcepstral analysis without adaptationadapt. to reverberation & Deltas

Fig. 10. Word error rates for a variation of the reverberation time in theliving room.


the four conditions that have also been shown in the previ-ous figure.

First of all the error rates for the robust ETSI front-endare higher in comparison to applying the cepstral analysisscheme without any adaptation. For the condition of ahands-free speech input in reverberant environments itlooks like the ETSI front-end does not work as efficientas it does in the presence of background noise.

The application of the adaptation method leads also to areduction of the error rates for the case of recognizingsequences of connected words. The relative improvementis not as high as in case of recognizing single digits. Theauthors mainly regard the superposition effect at the begin-ning of words as responsible for this. The acoustic informa-tion at the beginning of a word is modified by the acousticinformation of the preceding word due to the reverbera-tion. These ‘‘inter-word’’ modifications occur especiallywhen sequences of words are spoken fluently with coartic-ulation effects. In general the speaking rate considerablyvaries between speakers when uttering a sequence of digits.This effect can only be approximately covered by modelingwith HMMs with multiple mixture components. Analyzingthe errors a bit more in detail, it turns out that about halfof the errors are due to deletions in case of recognizing theliving room data with adaptation. It seems to be difficult torecognize especially the ‘‘fast’’ speakers which create theseco-articulation effects. The ‘‘inter-word’’ modifications arenot compensated by this adaptation technique.

We observe again that the additional adaptation of theDelta and Delta–Delta parameters causes a further gainin recognition performance.

Further recognition results are presented in Fig. 10 forvarying the reverberation time in the living room condi-tion. The tool for simulating the hands-free speech inputin noisy environments allows the variation of the reverber-ation time in a certain range. The RIR for the living roomsimulation is modified inside the tool so that it reflects thedesired reverberation time.

As expected the error rate increases for higher reverber-ation time. The error rates for a reverberation time of 0.6 sare the ones shown in the previous figure. Again the recog-nition performance is slightly worse for applying the robustETSI front-end in comparison to the cepstral analysisscheme. The improvement due to the adaptation is visibleover the whole range of the reverberation time.

3.3. Recognition with triphone HMMs

Triphone HMMs are used for the recognition of a largevocabulary. The ‘‘Wall Street Journal’’ data base (WSJ0)(LDC, 1993) is taken as basis for these investigations asit has also been used for the evaluations inside the ETSIworking group Aurora (Picone et al., 2004). Approxi-mately 7200 utterances that have been recorded at a sam-pling rate of 16 kHz with a high quality microphone aretaken for the training of triphone models. The triphonesare modeled as HMMs with three states where each acous-tic parameter of each state is described by a mixture of 4Gaussian distributions. Training and recognition are donewith HTK as it has been defined in (Au Yeung and Siu,2004). The training procedure includes state tying to modelthe triphones with a total of about 3200 different states.The recognition is based on the usage of a dictionary con-taining the phoneme description of about 5000 words. Thepossible sequences of words are defined by a ‘‘bigram’’model. The recognition process is speeded up by a statebased pruning.

The cepstral analysis scheme is applied as it has beendescribed before for data sampled at 16 kHz. We achievea word error rate of 11.21% for the recognition of the330 clean utterances that have been designated for testing.The word error rate can be reduced by applying HMMsthat model acoustic parameters with a higher number ofdistributions. Because of the high computational costs,

Table 2Word error rates for the recognition of the ‘‘Wall Street Journal’’ largevocabulary

Clean Office without adaptation Office with fixed adaptation

11.2% 48.8% 39.8%


only experiments have been run with HMMs modelingwith a mixture of four distributions.

All 330 test utterances have been processed with the sim-ulation tool to investigate the recording in an office roomwith a reverberation time of about 0.4 s. The word errorrates for recognizing these data are listed in Table 2.

The word error rate increases considerably to a value of48.8% in the hands-free mode.

The adaptation of the triphone HMMs is applied asdescribed before. As the own implementation of the Viterbirecognizer does not support the use of complex languagemodels, HTK is employed for the recognition. Thus, theestimation of the reverberation time T60 as well as the indi-vidual adaptation for each utterance is not applicable. Thewhole set of triphone HMMs is adapted once at the begin-ning with a fixed value for the estimated reverberation timeT60 instead. The adaptation is implemented as Matlabfunctions. The intention of the authors is only the proofthat the new adaptation approach can be applied to tri-phone HMMs in principal. The word error rate decreasesby about 10% when applying the set of adapted triphoneHMMs.

4. Combined adaptation to different distortion effects

The hands-free speech input in a room comes along withthe recording of background noise as it is present in almostall applications of speech recognition systems. Further-more the spectrum of the speech is modified by thefrequency characteristics of the microphone and of anadditional transmission channel, e.g. in case of transmittingthe speech via telephone to a remote recognition system.This creates the need to compensate also these distortioneffects.

In earlier work (Hirsch, 2001a) the authors developed anadaptation scheme based on the well-known PMC (parallelmodel combination) approach. This scheme consists of anadaptation of the static Mel frequency cepstral coefficients.The cepstral coefficients are transformed back to the Melspectral domain where the adaptation can be realized bya multiplication with a frequency weighting function asestimate for the frequency characteristics and by addingthe estimated noise spectrum. The cepstral coefficients ofall HMMs are individually adapted for each speech utter-ance when the beginning of speech is detected. Further-more the energy parameter can be adapted with anestimate of the noise energy.

We present a short overview about the techniques forestimating the spectrum of the background noise and the

frequency weighting function in the next section. Havingthese estimates as well as an estimation of T60, it will beshown that the earlier adaptation approach (Hirsch,2001a) can be combined with the new method of adaptingthe spectra to a hands-free speech input.

4.1. Estimation of distortion parameters

The spectrum of the background noise is estimated bylooking at a smoothed version of the Mel magnitude spec-trum as it is calculated in the feature extraction. The con-tour of the spectral magnitude values is smoothed in eachMel subband by applying a first order recursive filtering

XsmoothkðtiÞ ¼ ð1� aÞ � jXikðtiÞj þ a � Xsmoothkðti�1Þfor 1 6 k 6 NR mel and ti ¼ 0; 10 ms;20 ms; . . . ð28Þ

where Xik(ti) is the Mel spectrum of the analysis frame attime ti as calculated in the feature extraction.

A VAD (voice activity detector) is applied that takes theMel spectra as input. A speech onset is detected when theestimated signal-to-noise ratios exceed an adaptive thresh-old in several subbands for a certain number of frames. TheVAD was developed for earlier investigations. More detailscan be found in (Hirsch and Ehrlicher, 1995; Hirsch,2001a).

When the beginning of speech is detected the estimatednoise spectrum is set to the smoothed spectrum of the lastanalysis frame that is marked as pause frame

Nk ¼ Xsmoothkðlast pause frameÞ for 1 6 k 6 NR mel:

ð29Þ

Furthermore the energy of the noise is estimated asenergy of the last pause frame

Enoise ¼ Eiðlast pause frameÞ; ð30Þ

where Ei(ti) is the energy of the analysis frame at time ti ascalculated in the feature extraction.

The detection of speech begin is also taken as triggerpoint to perform the adaptation of all HMMs. For the sim-ulation experiments we take the acoustic parameters of allframes from a recorded utterance as input for the Viterbirecognition. For the real-time version of the recognizer asit is applied in a speech dialogue system, we start the recog-nition process five frames earlier than the first framedetected as speech. Thus, the Viterbi calculation can berun almost in parallel with the feature extraction.

The frequency weighting function is estimated after therecognition of an utterance. It is applied for the recognitionof the next utterance. This is based on the assumption thatthe frequency characteristics of the whole speech transmis-sion will not change rapidly. Usually the microphone andthe other transmission conditions do not change during arecognition session. The weighting function is estimatedby comparing the long-term spectra of the noisy inputspeech and of the clean speech. The ‘‘best’’ sequence of


HMM states is considered as it is available after the Viterbimatch by backtracking the path with the highest likeli-hood. The long-term spectrum of the noisy input speechis calculated for all analysis frames that are mapped onspeech HMMs excluding the frames that are mapped onthe pause model

Xlongk ¼1

NR speech�

Xspeech frames

jXikðtiÞj

for ti 2 feature vectors mapped on speech HMMsf g;ð31Þ

where NR_speech is the total number of vectors mappedon speech HMMs.

In a similar way the long-term spectrum of the cleanspeech is estimated by looking at the spectral informationcontained in the HMM states at the path with highest like-lihood. A set of adapted HMMs is used for the recognition.But for the estimation of the clean spectrum the spectralinformation is extracted from the corresponding cleanHMMs. The cepstral coefficients of the correspondingclean HMM states are transformed back to the Mel spec-tral domain. In case of HMMs with multiple mixture com-ponents, the spectrum of this mixture component with thesmallest spectral distance to the corresponding spectrum ofthe input signal is taken. Therefore, the estimated noisespectrum is subtracted from the spectrum of the input sig-nal to compare it with the spectrum contained in a cleanHMM. The spectral similarity is calculated as City blockdistance. So, the long-term spectrum of the clean speechcan be estimated

Slongk ¼1

NR speech

�X

speech frames

jX k McleanðtiÞ;SðtiÞ;mixðtiÞ½ �j � jNsilkj

for ti 2 feature vectors mapped on speech HMMsf gð32Þ

with Mclean(ti) and S(ti) as recognized model and state onthe best path and mix(ti) as mixture component with small-est spectral distance.

Nsil is the Mel spectrum that can be derived from thesingle state pause model. Calculating the average cepstralvalues of the pause state according to Eq. (12), the spec-trum can be determined by transforming the cepstral valuesto the spectral domain (Eq. (13)). The pause model con-tains the spectral information of the background noise thatwas present during the recording of the training data. Incase of ‘‘clean’’ training data, Nsil takes only small values.It is subtracted here to compensate its presence in the spec-tral parameters of all HMMs. In the rare case of getting anegative value after the subtraction the result is set to afixed small positive value.

Subtracting the estimated noise spectrum from the long-term spectrum of the noisy input speech, the frequencyweighting function can be estimated

W k ¼Xlongk � Nk

Slongkfor 1 6 k 6 NR mel: ð33Þ

It turned out in earlier investigations that this type ofestimating the spectral difference between the input signaland the clean HMMs works well. Because of comparingthe spectral information from the input signal and the cleanHMMs, the weighting function does not only contain thespectral characteristics of the recording equipment andthe transmission line but also the frequency characteristicsof the individual speaker to some extent.

In the same way the difference between the energy con-tours of the input speech and the best HMM sequence canbe calculated. The energy values of the input signal areaccumulated for those frames mapped on speech HMMs

Einput¼ 1

NR speech�

Xspeech frames

EiðtiÞ

for ti 2 feature vectors mapped on speech HMMsf g:ð34Þ

The energy parameters contained in the clean HMMs onthe best path are accumulated as estimate for the cleanenergy. Models, states and mixture components areselected as described before (Eq. (32))

Eclean¼ 1

NR speech

�X

speech frames

jE McleanðtiÞ;SðtiÞ;mixðtiÞ½ �j� jEsilj

for ti 2 feature vectors mapped on speech HMMsf g:ð35Þ

The average energy Esil of the single state pause modelis subtracted to compensate the presence of backgroundnoise in the training data.

A weighting factor can be calculated that describes theaverage energy difference between the input signal andthe energies contained in the sequence of HMM states onthe best path

we ¼ Einput � EnoiseEclean

: ð36Þ

This factor contains information about the loudness ofthe individual speaker.

4.2. Combined adaptation

Having estimates for the noise spectrum, the frequencyweighting function and the reverberation time, the Melspectra of the clean HMMs are adapted as shown inFig. 11.

The cepstral coefficients of each state and mixture com-ponent are transformed back to the linear Mel spectrumfor all clean HMMs. The Mel spectra are adapted to theestimated reverberation condition as described by Eq.(14). The estimated weighting function and the estimatednoise spectrum are applied for the further adaptation

0

200

400

600 0

1000

2000

3000

f/Hz

t/ms

clean

0

200

400

600 0

1000

2000

3000

with adaptation

t/ms

f/Hz

a b

Fig. 12. Spectrograms of the clean and the adapted HMMs for the word ‘‘six’’.

Clean HMMs

Adapted HMMs

IDCT DCT

exp

exp

log

log

Adaptation toreverberation

Estimated T60

Noise spectrum Nk

Freq. weighting Wk

++

+

*

*+

Fig. 11. Scheme for adapting HMMs to all distortion effects.


bX kðSi;mixjÞ ¼ W k � ~X kðSi;mixjÞ þNk for 16 k 6NR mel:

ð37ÞThe adapted Mel spectra are transformed to the cepstral

domain again.In the same way the energy parameter that has been

adapted to the reverberation as stated in Eq. (10), is usedas input for the further adaptation

bEkðSi;mixjÞ ¼ we � ~EðSi;mixjÞ þ Enoise: ð38ÞThe adaptation to reverberation and noise is visualized

by the spectrograms in Fig. 12. The spectrogram is shownin graph (a) as it can be calculated from the HMM of theword six trained on clean data. In graph (b) the adaptedversion of this HMM is visualized. The adapted HMMhas been extracted during the recognition of artificially dis-torted TIDigits data. These data have been created from asimulation of the hands-free recording in a noisy livingroom environment. The noise spectrum as it is estimatedfor the individual input utterance, becomes visible as shiftof the complete spectrogram. The reverberation tails canalso be seen when looking at the contours along time inindividual subbands.

5. Recognition experiments on hands-free speech input in

noisy environments

Again the recognition of connected digits is employed toproof the applicability of the combined adaptation to sev-eral distortion effects. A data base called Aurora-2 (Hirschand Pearce, 2000) exists that consists of noisy versions ofthe TIDigits. Noise signals have been artificially added atdifferent SNRs. Furthermore a few test sets exist wherethe frequency characteristics have been modified to simu-late the recording with audio devices in the telecommunica-tion area. But Aurora-2 does not cover the effect of ahands-free speech input in noisy environments.

Thus new sets of distorted versions have been artificiallycreated from the clean TIDigits by applying the alreadymentioned simulation tool (Finster, 2005). To make thesedata available for the research community, they have beenput together as data base and have been combined withHTK based recognition experiments. They will becomeavailable under the title Aurora-5 (Aurora, 2006).

A few details about the new data base will be listed inthe next section before presenting the recognition resultsfor these data. Finally results will be shown for the recog-


nition of data that have been recorded in a reverberantmeeting room in hands-free mode.

5.1. Distorted data of the Aurora-5 experiment

The new data base focuses on two application scenariosfor speech recognition. The first one is the applicationinside the noisy interior of a car, the second one thehands-free speech input in an office or a living room, tocontrol e.g. electronic devices like a telephone or audio/video equipment. In comparison to Aurora-2 where only1000 utterances were selected, each test set contains all8700 utterances here.

The usage of telephone like devices is assumed in generalfor the car scenario by filtering all data with a G.712 fre-quency characteristic first (Campos-Neto, 1999). G.712 isa characteristic that attenuates all frequency componentsoutside the range from about 300 to 3400 Hz. Car noise isadded to the filtered data at different SNRs in the rangefrom 0 to 15 dB. The noise segment for distorting a singleutterance is randomly selected out of eight recordings thatwere made in different cars and under different conditionslike e.g. windows open or closed. Three different versionsexist for the car noise condition. The first version containsadditive noise only according to the recording with a closetalking microphone. The second one considers the record-ing with a hands-free microphone. The third version is likethe second one but containing a further transmission over aGSM cellular telephone network. This reflects the usage ofan information retrieval system located at a remote positionin the telephone network. The GSM transmission is simu-lated by randomly selecting an AMR (adaptive multi-rate)speech coding mode and the channel conditions of the cel-lular channel. These options exist as part of the simulationtool. In total 15 test sets exist for the five different SNR con-ditions including the clean case and the three versions.

For the second scenario randomly selected noise seg-ments are added from five recordings inside different roomslike e.g. an office room or a restaurant. The same range ofSNRs is considered. Three different versions exist. The firstone looks at additive noise only simulating the recordingwith a close talking microphone. The second one considersthe recording in an office room where the reverberation timeis randomly varied in the range from 0.3 to 0.4 s. In the thirdversion the recording in a living room is simulated where thereverberation time randomly varies in the range from 0.5 to0.6 s. This comes up again to 15 test sets in total.

In general the Aurora-5 data contain a bigger varianceof the distortion conditions inside each test set in compar-ison to Aurora-2. For example only a single noise record-ing has been taken for Aurora-2 to create one test set.

5.2. Recognition of artificially distorted digits

Cepstral parameters are extracted again as acousticfeatures, as described and applied before for the experi-ments with reverberation as the single distortion effect.

Also the same gender dependent HMMs are used thathave been created with a training on the clean TIDigits.The word error rates are presented in Fig. 13 for the threedifferent versions containing car noise.

Looking at the condition with additive noise only,shown in graph (a), the expected improvement can be seenwhen comparing the results for the robust ETSI front-endagainst the results for a cepstral analysis. Further smallimprovements are achieved when adapting the HMMs toall distortion effects as described in the previous chapter.Furthermore the error rates are shown for the unsupervisedHMM adaptation with MLLR as it is available as part ofthe HTK Viterbi recognizer. An incremental MLLR is per-formed after each utterance. We observed a worse recogni-tion performance when applying MLLR every two or moreutterances. The adaptation is performed on the HMMscontaining the features of the cepstral analysis so that theresults can be immediately compared to the new adaptationapproach. The error rates for MLLR are only a little bitworse in this case.

The improvement, comparing the new adaptationapproach against the ETSI front-end, is higher when look-ing at the condition of a hands-free speech input in thenoisy car environment. This is shown in the graph (b).The reverberation time is fairly small in a car in compari-son to rooms. The major impact of the hands-free record-ing inside a car is a modification of the frequencycharacteristics. The adaptation seems to compensate theseeffects to a higher extent than the robust feature extractionexcept for the low SNR of 0 dB. The error rates for MLLRare again slightly worse.

The adaptation scheme shows its usability also for thecase of an additional transmission over the GSM cellularnetwork as shown by the results in graph (c). In this casethe speech is further modified by the encoding and decod-ing and the transmission errors on the cellular channel. Theadaptation technique seems to cover this type of distortionconsiderably better than the robust ETSI front-end. Theperformance of MLLR is extremely low for the SNR of0 dB. This has been observed in several experiments wherethe performance without adaptation was already quite low.MLLR seems to be unable to find the right feature map-ping in such cases and seems to adapt the features in thewrong direction.

The cases with car noise do not include the major effectsof a hands-free speech input in a reverberant room environ-ment. The word error rates presented in Fig. 14 do includesuch effects. These experiments investigate recordings ofspeech inside a noisy room environment.

In the case of additive noise only, shown in graph (a),the new adaptation scheme leads to similar error rates likethe robust front-end. In general the recognition perfor-mance is lower in comparison to the case with car noisebecause the interior noise signals contain more non station-ary segments.

A considerable improvement is observed when compar-ing the new adaptation technique against the robust

0 5 10 15 Clean0

10

20

30

40

50

60

70

80

SNR/dB

wor

d er

ror

rate

/%

G.712 & car–noise

robust ETSI frontendcepstral analysis without adaptationadaptation to ALLMLLR adaptation

0 5 10 15 Clean0

10

20

30

40

50

60

70

80

SNR/dB

wor

d er

ror

rate

/%

G.712 & handsfree & car–noise & GSM


0 5 10 15 Clean0

10

20

30

40

50

60

70

80

SNR/dB

G.712 & handsfree & car–noise


a b

c

Fig. 13. Word error rates for different recording conditions inside a car.


front-end for the cases of a hands-free speech input in anoffice or a living room as shown in graphs (b) and (c).The additional adaptation to reverberation causes thisimprove- ment.

MLLR adaptation leads to worse results especially forSNRs below 15 dB. It looks like the mapping on the basisof a linear regression is not able to completely compensatethe sum of spectral modifications caused by backgroundnoise and reverberation. While additive noise and a fre-quency weighting can be modeled as a stationary modifica-tion of each frame, reverberation includes modificationsalong the time axis. In sum this can not be completely com-pensated with a linear mapping. As already observed forthe car noise conditions, MLLR seems to adapt into thewrong direction in case we obtain a low performance with-out adaptation.

5.3. Recognition of digits recorded in application scenarios

All results presented so far have been derived from a rec-ognition of artificially distorted speech data. The authors

believe that their simulation of recording conditions repre-sents the situation of applying a recognition system in areal scenario quite well.

Thus the improvements on artificially distorted datashould also be visible in real application scenarios. Thisis proofed by recognizing speech data that have beenrecorded in different situations.

The first experiment is run on the so called Bellcore dig-its. These are speech data that have been recorded overtelephone lines. There exist the recordings of 220 Americanspeakers that have spoken the 10 English digits as isolatedwords. The recordings contain the usual distortions thatoccur in case of transmitting speech over the telephone.This includes the usage of telephone devices with differentfrequency characteristics and the presence of some back-ground noise. Word error rates are shown in Fig. 15 foran isolated word recognition using the set of HMMs thathave been trained on the clean TIDigits.

For the cepstral analysis and the HMMs trained onclean TIDigits the recognition performance is low withan error rate of about 30%. The error rate can considerably

0 5 10 15 Clean0

10

20

30

40

50

60

70

80

SNR/dB

wor

d er

ror

rate

/%interior noise


0 5 10 15 Clean0

10

20

30

40

50

60

70

80

SNR/dB

handsfree–office & interior noise


0 5 10 15 Clean0

10

20

30

40

50

60

70

80

SNR/dB

wor

d er

ror

rate

/%

handsfree–living & interior noise


a b

c

Fig. 14. Word error rates for different recording conditions inside rooms.

0

5

10

15

20

25

30

wor

d er

ror

rate

/%

Recognition of 2200 Bellcore digits

Robust ETSI frontendcepstral analysis without adaptationadaptation to ALL

Fig. 15. Word error rates for the Bellcore single digits with HMMstrained on the TIDigits.


be reduced with the robust ETSI front-end to a rate lessthan 10%. Applying the adaptation technique leads to a

further relative error rate reduction by about 70% in com-parison to the error rate for the robust front-end.

MLLR adaptation is applied on the HMMs for the caseof cepstral analysis, where these results are not shown inthe figure. We obtain the highest performance with an errorrate of 12% when performing the adaptation every fiveutterances. An interesting result is achieved when applyingthe MLLR adaptation on the HMMs trained on the fea-tures of the ETSI front-end. We obtain an error rate of2.3% when performing the adaptation for each utterance.This is even slightly better than the new adaptation tech-nique. It might indicate that it is possible to combine arobust feature extraction with an additional adaptation.

Another experiment is run on some recordings of themeeting recorder project (Janin et al., 2003). Speech datahave been recorded during meetings in a meeting roomwhere the microphones were placed in the middle of atable. Thus these data contain reverberation besides alow amount of background noise. The speakers utteredalso sequences of English digits which are used for this

0

5

10

15

20

25

wor

d er

ror

rate

/%

Recognition of 7700 meeting recorder digits

Robust ETSI frontendcepstral analysis without adaptationadaptation to ALLMLLR adaptation

Fig. 16. Word error rates for the meeting recorder digits with HMMstrained on TIDigits.


experiment. Only the recordings of native Englishspeakers are used resulting in about 2350 utterances withabout 7700 digits in total. Word error rates are shown inFig. 16.

In this case the performance of the robust front-end isslightly worse in comparison to a standard cepstral analy-sis. This effect has already been observed in most of theexperiments on reverberant data presented in the precedingsections. Applying the adaptation scheme, the error ratecan be reduced by about 70% in relation to the cepstralanalysis without adaptation. We achieve almost the sameperformance when performing MLLR adaptation for eachutterance.

6. Conclusion

We present a new technique for adapting the acousticparameters of HMMs to the condition of hands-free speechinput in reverberant rooms. This approach can becombined with existing techniques for the adaptation onnoise and unknown frequency characteristics. Furthermorewe introduce a new method for adapting the Delta param-eters based on a preceding adaptation of the staticparameters.

Applying the new adaptation approach on artificiallydistorted data or on real recordings in noisy conditions,we achieve a considerably higher recognition performancein comparison to the case without adaptation. The errorrates are also lower than the ones that are achieved withthe robust ETSI front-end that can be considered as repre-sentative for a robust feature extraction.

Especially in conditions where additive noise and rever-beration distort the speech signal, we obtain a higher recog-nition performance with the new adaptation technique incomparison to MLLR adaptation. The linear regressionseems to compensate the nonlinear distortion effects worsethan the new approach.

In the future we will investigate whether and how robustfeature extraction schemes can be combined with HMMadaptation techniques.

Acknowledgments

The investigations have been partly carried out during aresearch stay at the International Computer Science Insti-tute in Berkeley, CA 94704, USA. H.G. Hirsch would liketo thank Nelson Morgan and the whole speech group forfruitful discussions and their support.

This work is supported by the German research fundingorganization DFG (Deutsche Forschungsgemeinschaft).

References

Aurora project, 2006. <http://aurora.hsnr.de>.Au Yeung, S.-K., Siu, M.-H., 2004. Improved performance of Aurora-

4 using HTK and unsupervised MLLR adaptation. In: Proc.ICSLP.

Avendano, C., Hermansky, H., 1996. Study on the dereverberationof speech based on temporal filtering. In: Proc. ICSLP, pp. 889–892.

Bitzer, J., Simmer, K.U., Kammeyer, K.D., 1999. Multi microphone noisereduction techniques for hands-free speech recognition – a compara-tive study. In: Proc. Internat. Workshop on Robust Methods forSpeech Recognition in Adverse Conditions, Tampere, Finland, pp.171–175.

Campos-Neto, S., 1999. The ITU-T software library. Internat. J. SpeechTechnol., 259–272.

Couvreur, L., Dupont, S., Ris, C., Boite, J.M., Couvreur, C., 2001. Fastadaptation for robust speech recognition in reverberant environments.In: Proc. Internat. Workshop on Adaptation Methods for SpeechRecognition, Sophia Antipolis, France.

ETSI Standard Document, 2003. Speech Processing, Transmission andQuality aspects (STQ); Distributed speech recognition; AdvancedFront-end feature extraction algorithm; Compression algorithm. ETSIdocument ES 202 050 v1.1.3 (2003-11).

Finster, H., 2005. Web interface to experience the simulation of acousticscenarios. <http://dnt.kr.hsnr.de/sireac.html>.

Gadrudadri, H., Hermansky, H., Morgan, N., et al., 2002. Qualcomm-ICSI-OGI features for ASR. In: Proc. ICSLP, pp. 21–24.

Gales, M.J.F., 1995. Model based techniques for noise robust speechrecognition. Dissertation at the University of Cambridge, GreatBritain.

Gales, M.J.F., 1997. Nice model-based compensation schemes for robustspeech recognition. In: Proc. ESCA Workshop on Robust SpeechRecognition for Unknown Communication Channels, Pont-a-Mous-son, France, pp. 55–64.

Gales, M.J.F., Young, S.J., 1996. Robust continuous speech recognitionusing parallel model combination. IEEE Trans. Speech Audio Proc. 4,352–359.

Gauvain, J.L., Lee, C.H., 1994. Maximum a posteriori estimation formultivariate Gaussian mixture observations of Markov chains. IEEETrans. Speech Audio Proc. 2, 291–298.

Gelbart, D., Morgan, N., 2002. Double the trouble: handling noise andreverberation in far-field automatic speech recognition. In: Proc.ICSLP, pp. 2185–2188.

Hirsch, H.G., 1999. HMM adaptation for telephone applications. In:Proc. European Conf. on Speech Communication and Technology,Vol. 1, pp. 9–12.

Hirsch, H.G., 2001a. HMM adaptation for applications in telecommuni-cation. Speech Comm. 34, 127–139.

http://aurora.hsnr.de

http://dnt.kr.hsnr.de/sireac.html


Hirsch, H.G., Ehrlicher, C., 1995. Noise estimation techniques for robustspeech recognition. In: Proc. ICASSP, pp. 153–156.

Hirsch, H.G., Finster, H., 2005. The simulation of realistic acoustic inputscenarios for speech recognition systems. In: Proc. Interspeech Conf.,pp. 2697–2700.

Hirsch, H.G., Pearce, D., 2000. The Aurora experimental framework forthe performance evaluation of speech recognition systems under noisyconditions. In: Proc. ISCA Workshop ASR2000, Paris, France.

Hirsch, H.G., Hellwig, K., Dobler, S., 2001b. Speech recognition atmultiple sampling rates. In: Proc. European Conf. on Speech Com-munication and Technology, pp. 1837–1840.

Houtgast, T., Steeneken, H.J.M., Plomp, R., 1980. Predicting speechintelligibility in rooms from the modulation transfer function, I.General room acoustics. Acustica 46, 60–72.

Janin, A. et al., 2003. The ICSI meeting corpus. In: Proc. ICASSP.Kingsbury, B., 1998. Perceptually inspired signal processing strategies for

robust speech recognition in reverberant environments. Dissertation atUC Berkeley, USA.

Kinshita, K., Nakatani, T., Miyoshi, M., 2005. Efficient blind derever-beration framework for automatic speech recognition. In: Proc.Interspeech Conf., Lisbon, Portugal, pp. 3145–3148.

Kuttruff, H., 2000. Room Acoustics. Spon Press.LDC, 1993. Speech Data Base CSR-I (WSJ0). Wall Street Journal, http://

www.ldc.upenn.edu.Leggeter, C.J., Woodland, P.C., 1995. Maximum likelihood linear

regression for speaker adaptation of continuous density HiddenMarkov Models. Comput. Speech Lang. 9, 171–185.

Leonard, R.G., 1984. A database for speaker-independent digit recogni-tion. In: Proc. ICASSP, Vol. 3, p. 42.11.

Liu, J., Malvar, H.S., 2001. Blind deconvolution of reverberated signals.In: Proc. ICASSP, Vol. 5, pp. 3037–3040.

Macho, D., Mauuary, L., Pearce, D., et al., 2002. Evaluation of a noiserobust DSR front-end on Aurora databases. In: Proc. ICSLP, pp. 17–20.

Minami, Y., Furui, S., 1996. Adaptation method based on HMMcomposition and EM algorithm. In: Proc. ICASSP, pp. 327–330.

Omologo, M., Svaizer, P., Matassoni, M., 1998. Environmental conditionsand acoustic transduction in hands-free speech recognition. SpeechComm. 25, 75–95.

Palomaki, K.J., Brown, G.J., Barker, J., 2002. Missing data speechrecognition in reverberant conditions. In: Proc. ICASSP, pp. 65–68.

Picone, J., Parihar, N., Hirsch, H.G., Pearce, D., 2004. Performanceanalysis of the Aurora large vocabulary experiment. In: Proc.European Signal Processing Conference, Vienna, Austria.

Raut, C.K., Nishimoto, T., Sagayama, S., 2005. Model adaptation bystate splitting of HMM for long reverberation. In: Proc. InterspeechConf., Lisbon, Portugal, pp. 277–280.

Sankar, A., Lee, C.H., 1996. A maximum-likelihood approach tostochastic matching for robust speech recognition. IEEE Trans.Speech Audio Proc., 190–201.

Seltzer, M.L., Raj, B., Stern, R.M., 2004. Likelihood-maximizing beam-forming for robust hands-free speech recognition. IEEE Trans. SpeechAudio Proc. 12 (5), 489–498.

Tashev, I., Allred, D., 2005. Reverberation reduction for improved speechrecognition. In: Proc. Workshop on Hands-free Speech Communica-tion, Rutgers, USA.

Woodland, P.C., 2001. Speaker adaptation for continuous densityHMMs: a review. In: Proc. Internat. Workshop on AdaptationMethods for Speech Recognition, Sophia Antipolis, France.

Wu, M., Wang, D., 2005. A two-stage algorithm for enhancement ofreverberant speech. In: Proc. ICASSP, Vol. I, pp. 1085–1088.

Yegnanarayana, B., Murthy, P.S., 2000. Enhancement of reverberantspeech using LP residual signals. IEEE Trans. Speech Audio Proc. 8,267–281.

Young, S. et al., 2005. The HTK Book (version 3.3). CambridgeUniversity Engineering Department, http://htk.eng.cam.ac.uk.

http://www.ldc.upenn.edu

http://www.ldc.upenn.edu

http://htk.eng.cam.ac.uk

a new approach for the adaptation of hmms to reverberation and...

Documents