Microsoft Speech Server

Charlene Volker – Solutions Sales Specialist

Lynn Olson – Technical Specialist

2

Government Self-Service Applications

Permits and Inspections

Jury Management Citation Management Elections Tax Information Utilities Password Reset

Auto Attendant Outbound Notification License Application License Renewal Motor Vehicle

Renewal Mobil Workforce Speech Enabled

Portals

3

State and Local Governments are looking for IT Solutions That…

Provide a solution for increased services demands by constituents

Demonstrate a rapid return on investment Are easy to install and maintain Address key operational challenges

Microsoft Speech Technologies provide an easy, economical way to extend government services to a broader range of constituents, across a wide array of devices.

4

Benefits of Speech Recognition Reduces Labor Costs in the Enterprise

Call Center, IT and Operations Automates repetitive high volume, low value tasks

Improves Automation Simplifies complex transactions Adds convenient access to existing systems

Improves the User Experience Speech provides advantages over other methods

including touch-tone, email and web 80% believe speech is as easy or better to use than

internet

5

WebWeb

SpeechSpeechSpeechSpeech

Where Speech Fits

Accessibility

User Friendliness

Customers Reached

Touch-ToneTouch-Tone

6

7

8

9

New York Department of Education 1.2 million students, 80,000 teachers and 1,200

schools Highly Acclaimed Website Low computer literacy and lack of internet access –

Digital Divide Speech-enabled ASP.NET website Extends NYDOE website to phone Provides attendance, grades, lunch menu and other

key information to parents

NY DOE Chose Speech Server

“We have a large number of parents who are not comfortable with a computer. But telephones are accessible and easy to use…using Microsoft Speech Technologies, parents will have access to information 24/7.”

Richard Langford, Deputy CIO NYC DOE

10

11

12

Microsoft Speech Technology Vision

Microsoft Developer tools for speech

Contact Center

IVRMessaging

Enterprise& LOB

Applications

Desktop&

Embedded

Microsoft Speech Server Speech Technologies

Making Speech Mainstream

“Speech and natural language understanding are the key technologies that will have the most impact in the next 15

years.”-- Bill Gates

13

Permits and Inspections

Schedule Inspections Cancel/Reschedule Inspections Post Inspection Results Obtain Inspection Results Fax Results Outbound Notification

14

Municipal Court Systems

Check the status of case or citation Check the status of an account

Amount Owed Due Date Payment History

Schedule and/or Confirm Court Date Administrative Review Hearing

Credit card payment processing Jury Notification Outbound Court Appearance reminder call

15

Tax Assessment & Collections

Status Checks Current year tax payments 1st half 2nd half

Prior Year Tax Payments By year Delinquent or lien status

Credit Card Payments of Taxes Fax Options

Parcel Detail Property and Ownership detail Payment History Billing History

16

Utility Department

Account Inquiry Current balance Adjusted balance Last payment amount Pending payment Current billing date Delinquent due date Last payment date Delinquent balance

Start/Stop Service Payment Site Locator Outbound Notification Meter Reading Outage Reporting

and Status

17

Back End Integration

Microsoft CRM Accela Advantage

(Tidemark) Accela Automation Accela Permits Plus Black Bear Computronix CRW CSDC Amanda Echelon3 Technologies

Eden Hansen Harris Computer

Systems KIVA Municipal CityView Pentamation Springbrook Software SunGard HTE

18

Microsoft’s Auto Attendant Automated operator 55,000+ names in Microsoft directory Results

75% first-try task completion rate 50% reduction in agent headcount ROI (over 4 years) = 1400% Payback period = 4 months Annual savings = $270,000

What Is Microsoft Speech Server?

20

Web-based, speech-enabled IVR & multimodal server DTMF and speech

Supports open and industry standards Integrated ASR/TTS

U.S. English, U.S. Spanish, Canadian French High performance, scalable, reliable, secure Flexible deployment options Comprehensive and robust development

using VS .NET and Speech Application SDK Reporting, analysis, and tuning tools Integrated Windows management Leverages existing IT, Web, telephony, and

data assets and skills

R2

21

More Self-Service, Less Cost

Voice User

Interface

ApplicationLogic and Business

Rules

Back-end Data

Graphical User

Interface

Same development tools

Same web and web services infrastructure

Same application logic and back-end data

Same management and monitoring tools

22

Microsoft Speech Server

Media GatewayPSTNPSTN

Microsoft® Speech Application SDK

+ Visual Studio® .NET

Microsoft® Speech Application SDK

+ Visual Studio® .NET

Development WorkstationDevelopment Workstation

Telephony Card

TelephonyInterface

Manager (TIM)

TelephonyInterface

Manager (TIM)

Speech Engine Services (SES)

Telephony Application

Services (TAS)

ASP.NET Web Server

ASP.NET ControlsASP.NET Controls

GrammarsGrammars

PromptsPrompts

Web PagesWeb Pages

Speech Server Topology

TelephonyTelephony

HTML + SALT + JScript

PBX

SIP/RTP

SIP/RTP

TDM

TDM

TDM

SQL – Logging & Reporting

JH

DB

Back-end/DB ConnectivityBack-end/DB Connectivity

ADO.NET,

ODBC, Web

Services,

SOAP

App Deployed

23

MSS Configuration Options

o All-in-one - manageabilityo Distributed - fail-overo Universal Port: ASR, TTS, and Languageso Licensed per Proc

PBX (optional)

Microsoft Speech Server and Web Server

Standard Edition24 ports or less

Microsoft Speech Server (TAS service enabled)

Web Server

and DB(s)

PBX

Microsoft Speech Server(SES service enabled)

Load balancer

Enterprise Edition

AIO or Distributed

SO

AP

or

or +1

24

Advantages of Speech

Improve Constituent Satisfaction

Extend Services Easily

Leverage Existing Technology

25

For More Information…

My contact Information cvolker@microsoft.com

http://www.microsoft.com/speech/solutions/government/default.mspx

Case Studies Data Sheet Sample Application Wav Files

26

© 2003-2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

27

Speech Application SDK Overview VS.NET integrated tools

Dialog Grammar Prompts

ASP.NET speech controls Grammar libraries Debugging tools

Debugger Telephony simulator Log player

Analysis tools Call analyzer Reports

Samples, tutorials, docs

28

Design the VUI interaction – Call Flow Create grammars for expected answers Add speech controls to a Web Form Extract the meaning (answers) of the

recognition results and bind them to controls

Create the prompts Debug before deploying

Building a Speech Application

GrammarEditor

Dialog Editor

PromptEditor

Debugging Tools

29

Grammars

30

A structured list of rules that identify the words or phrases that can be used for speech input and subsequent automatic recognition

Maintained in .grxml files – adhere to the SRGS format

I would like aI would like a

I’ll have a aI’ll have a a

Give me aGive me a

smallsmall

mediummedium

largelarge

plainplain

cheesecheese

mushroommushroom

hamham

pizzapizza

pizza pleasepizza please

Preamble PostambleSemantic Items

What are Grammars?

31

Grammar Elements

Phrase (words) List (alternatives) Rule reference Group (rolls Lists up) Advanced:

Wildcard Halt Skip

Script Tag (assigns variable)

32

Creating Grammars and Semantic Mark-up

33

Adding Semantic Mark-up

Add Script Tags to your grammar rules After the phrase where a semantic value is

required This creates <tag> elements in the

underlying .grxml file Use the Semantic Script Editor to create script

Create semantic elements and attributes Use free format script editing for complex

requirements

34

Speech Controls

35

Basic Speech Controls

Prompt

Speech Controls

Listen

DialogSpeech Controls

ApplicationSpeech Controls

QA

SemanticMap

SemanticItem

Grammar

Command

AnswerCall

CreditCardNumber

Phone

DataTableNavigator

Custom ControlsCompareValidator /CustomValidator

Speech Controls

36

What are Application Speech Controls?

Composite controls, to collect common information

Examples include date, phone number and credit card controls

Application ControlApplication Control

Dialog Speech ControlsDialog Speech Controls

GrammarsGrammars

Default PromptsDefault Prompts

37

What are QA Controls?

A QA manages a question-answer interaction with a user Asks a question (prompts the user) Accepts voice input (using a

grammar)

qaPSizeqaPSize

qaPToppingqaPTopping

Which size Pizza would you like to

have?

Which size Pizza would you like to

have?

What toppings would you like on

your pizza?

What toppings would you like on

your pizza?

Question-Answer Interactions Application

38

Semantic items store elements of contextual information gathered from the user

A semantic map is a container for multiple semantic items

SemanticMapQA Control(s)

Answers Collection

Answers Collection

siPSizesiPSize

Semantic Item StateSemantic Item State

EmptyEmpty

Needs ConfirmationNeeds Confirmation

siPToppingsiPTopping ConfirmedConfirmed

Semantic Items and Semantic Map

39

Dialog Control Editor

40

Prompts

41

MSS Prompts

Supports Pre-Recorded and Text-to-Speech

Stored in a Prompt Database Easy to Manage

Prompt DB and Editor enable you to: Add Transcriptions Create Extractions Record Prompts Edit Wave Files Validate Prompt Coverage

42

The Prompt Editor

43

Speech Debugging

44

The Telephony Application Simulator

45

Edit SML ResultsEdit SML Results Trace OutputTrace OutputSpeech Control StateSpeech Control State

Simulated Exception Conditions

Simulated Exception Conditions

Break on Listen Start

Break on Listen Start

Enable SML EditingEnable SML Editing

Text or Voice InputText or Voice Input

Play PromptsPlay Prompts

The Speech Debugging Console

46

Managing your MSS Deployment

Microsoft Management Console (MMC) SES, TAS, TIM Application deployment Centralized settings and replication

Windows Management Instrumentation (WMI) All settings can be configured programmatically Uniform access for mgmt data from multiple sources Sharing mgmt data across multiple mgmt applications

App Provisioning Tool

47

Monitoring your MSS Deployment

Real-time monitoring facilities System monitor (disk, memory, proc, net, etc) Performance monitor (counters, traces, alerts)

Centralized Monitoring via MMC SNMP

Events can be configured to generate traps MOM Management Pack Log Analysis Tools

48

Data Analysis Framework

Application

49

Reporting and Tuning

Scenarios Business Manager

how many calls? which services? how many successful transactions?

Application Developer how good are my grammars? where are people

hanging up/transferring? System administrator

where are the system latencies? what is causing trace warning messages?

MSS solutions Speech Application Reports Call Viewer

50

Call Viewer

Analyst selects problematic calls queries over times, events, particular

properties quickly retrieves all calls symptomatic of the

problem Drills down into the call

flexible views, according to event filters selected turn by turn

including user audio component event by component event

detailed information even finer-grained configuration

conditional event filtering

51

Call Viewer

52

Reports

Call Aggregates

Provides total calls aggregated by various time periods

Call Volume Displays call volume across a single span of time, plus providing comparative data on disconnection types, transferred calls, and call duration.

Dialog Overview

For the selected time period, displays data on QA use and QA recognition confidence scores, plus command use.

Errors Displays summary data on call completion, a count of tasks missing TaskStart and TaskComplete events, and occurrence counts for error event types.

Messages Provides counts of messages sent by the SALT LogMessage function, and a breakdown by message name and content.

Prompts Provides counts for bargein and user-perceived latencies, prompt ending status, individual prompt totals, and comparison of text-to-speech versus recorded prompts.

Server Statistics

Provides data on latencies, channel and port usage, and answering time.

Turn Analysis Displays data on QA usage per call, sums and percentages for QA History values, compares speech versus DTMF input, and reports on time spent on pages.

Tasks Reports on task completion, duration and progress, and the count of QAs used for each task.

53

© 2003-2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.