Improving language learning for non-native speakers

Xinyu Tang, Allen Parish, Steven Chang

Introduction Speech processing and language learning Learning a second language is difficult Complicated by biases introduced by native

language Exaggeration may help to recognize the subtle

difference between sounds

Na La

Background

LPC analysis can capture core sound components

Exaggeration can actuate differences between sounds

Goals/Objectives

Overarching GoalHelp non-native speakers distinguish confusing

sounds Project Objectives

Implement existing work as a feasibility testUse LPC-based analysis to separate a cluster of

similar wordsUse LPC-based analysis to separate adjacent

phonemes

Related Work

Concept of LPC Introduced by [Makhoul 1978] [Kahn 1998] used LPC Exaggeration [Protopapas 1998] used LPC extrapolation for

non-native speakers Tools for phoneme separation

SFS Sphinx

Proposed solution

1. Standard Pairwise Exaggeration [Protopapas 1998] Knock and Lock Cop and Cup

2. Split difficult clusters of words Thud, dog, god, sod, thought, cod,

thus Exaggerate from mean Exaggerate from K - nearest

neighbors (KNN) 3. Sequential Exaggeration

Artificial, Probability

Split sounds, normalize

LPC Analysis

Exaggeration/Interpolation

LPC Synthesis

Re-assemble sounds

Standard Pairwise Results Exaggerated sounds helps Xinyu to distinguish “La” and “Na” Interpolation make sounds hard to distinguish Extreme exaggeration make sounds distorted and hard to distinguish

Na La Na La

Cla

ssifi

catio

n ra

te

Exaggerate from cluster center

Those exaggerated sounds are further from each other

LaNa

Ra

Euclidean distances of sounds Distributions of sounds in LDA projection

Original

Exaggerated

Original Exaggerated

Exaggerate from Cluster Center

Thud, dog, god, sod, thought, cod, thus Exaggerate sounds from the middle A little bit crowded for big cluster

Euclidean distances of sounds Distributions of sounds in LDA projection

KNN Exaggeration From Closest Neighbors-2NN Nearer sounds are more exaggerated

Euclidean distances of sounds Distributions of sounds in LDA projection

Sequential Exaggeration

Artificial

Probability

Original

Exaggerated

Original

Exaggerated

Analysis of Results

LPC-based exaggeration succeeds in exaggerating similar sounds

Exaggeration can help/hinder people distinguish ambiguous sounds

Difficulties The feeling of sounds are subjective Hard to exaggerated sounds “Natural” It is hard to find subjects who can’t distinguish two

original sounds Phoneme separation is a difficult and inexact task

Future Work

Better testing of subjects that have difficulties with particular sounds

Work in conjunction with linguists to apply approaches to known difficult phonemes

Automate entire process as a training tool for non-native speakers

Conclusions

LPC-based exaggeration can help people differentiate tough phonemes

Our results demonstrate feasibility of a variety of approaches

Which technique to use is still a trial and error process

References

Source CodeInterval Toolbox –

http://rvl4.ecn.purdue.edu/~malcolm/interval/1998-010/

Speech Filing System - http://www.phon.ucl.ac.uk/resource/sfs/

Original Paper Modified LPC resynthesis for controlling speech stimulus discriminability.

136th Annual Meeting of the Acoustical Society of America. Norfolk, VA, 13-16 October. [In Journal of the Acoustical Society of America 104 (3 Pt. 2): 1855]