mandarin tone recognition using affine-invariant prosodic features and tone posteriorgram
DESCRIPTION
Yow-Bang Wang, Lin-Shan Lee INTERSPEECH 2010. Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram. Speaker: Hsiao- Tsung Hung. 1.Introduction. Introduction. Tone recognition are definitely influenced by as least the following: Speaker - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/1.jpg)
MANDARIN TONE RECOGNITION USING AFFINE-INVARIANT PROSODIC FEATURES AND TONE POSTERIORGRAM
Yow-Bang Wang, Lin-Shan LeeINTERSPEECH 2010
Speaker: Hsiao-Tsung Hung
![Page 2: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/2.jpg)
1.INTRODUCTION
![Page 3: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/3.jpg)
Introduction
Tone recognition are definitely influenced by as least the following:1. Speaker2. The “prosodic state”3. Co-articulation effect
![Page 4: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/4.jpg)
Introduction
Although the tones depend heavily on many intra-syllabic and prosodic behaviors which are definitely speaker dependent, the native speaker of Mandarin can easily recognize the tones
This implies the tones should be classified by some “robust” prosodic cues, which remain useful across many different conditions.
![Page 5: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/5.jpg)
Introduction
in this paper we try to introduce robustness into prosodic features by different feature normalization schemes, based on the concept of affine invariance property proposed in recent years
We also incorporate the prosodic features with the context information by tone posteriorgram analogous to the TANDEM system for speech recognition.
![Page 6: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/6.jpg)
2.PROPOSED APPROACH
![Page 7: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/7.jpg)
Prosodic feature set
NumPitch Mean and slop
(3 segments)6
Mean and slop (Preceding and following syllable)
4
First frame, last frame, minimal, maximal pitch value
4
The last voiced frame pitch of preceding syllable
1
The first voiced frame pitch of following syllable
1
1Duration
Duration ratio with two adjacent syllables 2
Energy Log-energy difference with two adjacent syllables
2
![Page 8: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/8.jpg)
Affine Invariance property Consider an n-dimensional feature
vector sequence along the time axis. If a certain change of condition over these feature vectors is stationary within some period of time, and can be represented as an affine translation:
![Page 9: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/9.jpg)
Affine Invariance property There may exist some features
obtained from which remain invariant under such change of conditions:
,where is the feature function.
![Page 10: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/10.jpg)
Affine invariance for normalized pitch features Assume the transformation between
the pitch contours for the same syllable for two speakers, and , can be approximated by an affine transform:
(assume here)
![Page 11: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/11.jpg)
Affine invariance for normalized pitch features relationship between the utterance-
level means and standard deviation:
![Page 12: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/12.jpg)
Affine invariance for normalized pitch features
Any feature function M() applied to this normalized pitch contour is automatically affine-invariant.
![Page 13: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/13.jpg)
Invariance of duration and energy features Duration
Energy difference for two adjacent syllables
![Page 14: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/14.jpg)
Pitch contour normalization schemes
![Page 15: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/15.jpg)
Tone recognition
21-dimensional prosodic feature vector
SVM
Enh1 : current syllableEnh2 : current, preceding and following syllable
![Page 16: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/16.jpg)
EXPERIMENTS
![Page 17: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/17.jpg)
Corpus and experiment setup Sinica Continuous Speech Prosody
Corpora (COSPRO) Contained 4672 utterances (more
than 60,000 syllables), produced by 38 male and 40 female native speakers.
SVM tone recognizers.
![Page 18: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/18.jpg)
Experimental results
![Page 19: Mandarin Tone Recognition using Affine-Invariant Prosodic Features and Tone Posteriorgram](https://reader036.vdocuments.site/reader036/viewer/2022062310/5681619a550346895dd14fb4/html5/thumbnails/19.jpg)
Experimental results