Controlling Computer Using Speech Recognition (CCSR)

Creative Masters Group

Supervisor : Dr: Mounira Taileb

Outline

• Introduction.• Problem definition.• Suggested solutions.• CCSR architecture.• Implementation tools.• Testing : performance testing. Usability testing.• Implementation difficulties.• Future work.

Computer is considered as an important tool

in different fields and for different age

categories. As we know, the number of

computer users is increasing year after

another. For this increasing many

technologies are provided to facilitate the use

of this device

Introduction

Problem Definition

A computer user may need to control his device while he is busy with other job:

-Having a meal.

-Presenting a lecture.

-Physically challenged.

Suggested Solutions

1-Using hardware attached to the wheelchair of the physically challenged person.

2-Using glasses with attached video camera.

3-Using speech recognition software.

CCSR is an application that enable normal

and physically challenged people to

control their computer using speech

recognition instead of using mouse and

key board. CCSR is implemented in Arabic

language.

Proposed Application

CCSR Architecture

Figure1: CCSR Architecture

Implementation Tools

1- HTK toolkit for building recognition

model.

2- C# for GUI and API functions to deal with

system files.

3- Audacity for sample recording.

Implementation of CCSR

- Explain CCSR interface and our

commands.

- Start application running.

- First scenario is: Desktop–Right- open.

- Explain batch file and the comparison

with training result.

- Second scenario is: favorite.

- Third scenario close the application.

Testing

1-Unit testing .

2-Integration testing.

3-Performance testing.

4-Usability testing.

Performance Testing (1)

Figure2: Recognition rate, 7 samples in the

training

Figure3: Recognition rate with 30 samples in the

training

Performance Testing (2)

Figure5: Performance testing of all

commands, 12 commands, 51 samples in the

training, tested by 6 users

Performance Testing (3)

Figure 6: Performance testing for 3 consecutive commands, 8 users

Performance Testing (4)

Figure8: Performance testing of 4 consecutive commands, 8 users

Usability Testing

• CCSR GUI tested on 6 users and it gives an accepted

response time.

• Because there is only one way to execute each

function (only one command to activate each function

as CCSR tutorial) so the users are interact easily with

CCSR interface.

• When user requests the command by his speech, the

result (for example computer shutdown) will show on

the screen immediately.

• All tabs and buttons in CCSR GUI are designed based

on HCI guidelines.

Implementation Difficulties (1)

1 -In data preparation (Grammar and Dictionary building)

Figure 9: Grammar file

Figure 10: Dictionary file

Implementation Difficulties (2)

2 -Item selection function and getting item path.

Implementation Difficulties (3)

3 -Noise cancelation in Visual Studio

environment.

Tried solutions:

1-Change default visual studio

recorder.

2- C# audacity component.

3- Using Sox in command prompt.

Future Work

In the future work developers will improve

the application by:

1- Solve noise cancellation problem in visual

studio.

2- Add voice print feature that enables user

to log in to his computer account using his

voice.

3- Start CCSR application when Windows is

booted.