chinese-speaking 3d talking head
DESCRIPTION
Chinese-Speaking 3D Talking Head. Project No: H08040 Sang Siew Hoon Supervisor: Dr Ng Teck Khim. System Objectives. create a realistic Chinese-speaking 3D talking model -automatic -accurate and realistic -portable. Problems. Lack of references on Chinese Involves various fields - PowerPoint PPT PresentationTRANSCRIPT
1
Chinese-Chinese-SpeakingSpeaking
3D 3D Talking HeadTalking Head
Project No: H08040Project No: H08040
Sang Siew HoonSang Siew Hoon
Supervisor: Dr Ng Teck KhimSupervisor: Dr Ng Teck Khim
2
System ObjectivesSystem Objectives create a realistic Chinese-create a realistic Chinese-
speaking 3D talking model speaking 3D talking model
-- automaticautomatic
-- accurate and realisticaccurate and realistic
-- portableportable
3
ProblemsProblems Lack of references on Lack of references on
ChineseChinese Involves various fields Involves various fields
-- Face modelingFace modeling
-- Text-to-speechText-to-speech
-- Face Animation (Focus)Face Animation (Focus)
4
System OverviewSystem Overview5 modules5 modules
1.1. Face Model PreparationFace Model Preparation
2.2. Chinese VisemesChinese Visemes
3.3. Text-to-Speech (TTS)Text-to-Speech (TTS)
4.4. AnimationAnimation
5.5. RenderingRendering
5
6
Face Model Face Model PreparationPreparation
7
Face Model Face Model PreparationPreparation
Face Modeling 3DS MaxFace Modeling 3DS Max
8
Face Model Face Model PreparationPreparation
Defining MPEG-4 feature pointsDefining MPEG-4 feature points
9
Face Model Face Model PreparationPreparation
Assigning lip verticesAssigning lip vertices
10
Face Model Face Model PreparationPreparation
Cloning morph targetsCloning morph targets
FAP#3 open_jaw
FAP#4 lower_t_midlip
11
Chinese VisemesChinese VisemesVisemesVisemes- the visual equivalent of a the visual equivalent of a
phonemephoneme
1.1. Classification of Chinese Classification of Chinese SyllablesSyllables
2.2. Definition of Chinese VisemesDefinition of Chinese Visemes
3.3. Refining Dynamic VisemesRefining Dynamic Visemes
12
Chinese VisemesChinese Visemes Definition of Chinese VisemesDefinition of Chinese Visemes
13
Chinese VisemesChinese Visemes Definition of Chinese VisemesDefinition of Chinese Visemes
14
Chinese VisemesChinese Visemes Definition of Chinese VisemesDefinition of Chinese Visemes
15
Chinese VisemesChinese Visemes Refining Dynamic VisemesRefining Dynamic Visemes1.1. <i> +<i> + e only e only replace <f> erreplace <f> er2.2. zh, z, y + I zh, z, y + I drop i drop i3.3. d, l, g, j, zh, z, y, w + u | ü d, l, g, j, zh, z, y, w + u | ü drop <i> drop <i>4.4. j, y + <f> ending with an j, y + <f> ending with an replace the an by en replace the an by en5.5. j + complex finals headed by i and followed by more than one <f> j + complex finals headed by i and followed by more than one <f> drop i in complex finals drop i in complex finals
16
Text-to-SpeechText-to-Speech
17
Text-to-SpeechText-to-Speech
18
AnimationAnimation1.1. CoarticulationCoarticulation
2.2. Automatic Generation of Automatic Generation of Face AnimationFace Animation
3.3. Enhancing RealismEnhancing Realism
19
AnimationAnimation Enhancing realismEnhancing realism
- Eye blinks- Eye blinks
- Eyebrow raising- Eyebrow raising
- Gaze- Gaze
- Head Rotation- Head Rotation
20
ResultsResults Accurate and realisticAccurate and realistic No discrepancies for new No discrepancies for new
face modelsface models Automated results need Automated results need
slight manual interventionslight manual intervention
http://www.slamstudio.com/HYP/videoshttp://www.slamstudio.com/HYP/videos
21
ConclusionConclusion ContributionsContributions1.1. own system to define Chinese visemes own system to define Chinese visemes
- adds to the current research works of- adds to the current research works of
Chinese talking heads (currently Chinese talking heads (currently limited). limited).
2.2. an automatic Chinese lip-an automatic Chinese lip-synchronization systemsynchronization system
- saves an animator much time and - saves an animator much time and efforteffort
22
ConclusionConclusion System LimitationsSystem Limitations
1.1. TTS limitationTTS limitation
2.2. FAPs conflictsFAPs conflicts
3.3. Not integrated Not integrated
– – inconvenient for untrained inconvenient for untrained usersusers
23
ConclusionConclusion Future WorksFuture Works1.1. Automatic audio signal processing Automatic audio signal processing
2.2. Resolve FAPs values conflicts Resolve FAPs values conflicts
3.3. Automatic feature points Automatic feature points assignment system assignment system
4.4. Integration of systemIntegration of system