common measureme nts the tool guy gordon assistant deputy minister service manitoba manitoba lois...

Common Measurements

the

Tool

Guy GordonAssistant Deputy Minister

Service Manitoba

Manitoba

Lois BainDirector, Quality Service

OPS Restructuring Secretariat,

Cabinet Office

Customer Feedback - CMT 2

Today’s Objectives

The importance of understanding customer needs

What is the Common Measurements Tool (CMT) What is it Who is using it Tools to help you to collect customer data Lessons Learned

New generations of the CMT Come join us on the journey!


Public OpinionHow Canadians View Customer Service (general)

DELIVERYTURN-OFF

PEOPLE TURN-OFF

PEOPLE EXCELLENCE

DELIVERYEXCELLENCE

Banks 70%

Large Retail 66%

Gov’t 40%

Small Retail 86%

Pharmacies 92%

*National Quality Institute, Nov. 1997

Source: National Quality Institute Survey 1997


Citizens Rate Recent and Specific Services Higher

Serv

ice

Qu

alit

y (0

-100

)

64

62

60

64

62

61

58

59

60

61

62

63

64

6553

47 47

57

50 51

Municipal Federal

0

10

20

40

50

60

1998

2000

Municipal Provincial Federal

0

Recent and Specific Experience

General

Provincial

Citizens First 2000


Redrawing Maps- Citizens First 2000 Citizen Satisfaction - Canada

Fire departmentsPublic librariesSupermarketsPoliceProvincial/Territorial parks, campsDepartment storesTelephone companiesCanada PostTaxisRevenue CanadaHospitalsBanksCable television companiesRoad maintenance

797772 6665 616159565452514940

SERVICES SATISFACTION4 of the top 5 services are Public Services

>>

>>


Citizen needs and expectations: (EKOS led Study)

Do citizens want to talk to government about what they want? YES: Citizens perceive an influence gap:

citizens have too little and big business and the media have too much.

Do citizens want to talk about service delivery? YES: Citizens believe it is important for the

government to engage citizens about: service delivery (82%), policy development (75%), governance (74%), and program design (73%).


The Cost of Quality...

“The Good”

““The Bad”The Bad” “The Ugly”

•people•material•overhead•etc.

•training•systems improvements

•gathering & using client feedback

Fixing errors BEFORE they reach your

clients

Fixing errors AFTER they reach your

clients

COST = Production + Prevention + Inspection & + Field Problem Costs Costs Correction Costs

Costs

(1X) (10X)(10X) (100X)

30 - 50% of costs intypical service organizations


Citizen-Centred Service Network

Citizens FirstErin Research Inc

1998 IPACGold Award

2000 CAPAMSilver Medal

Common Measurements Tool

FayeSchmit

A network of Federal/Provincial/Municipal civil servants dedicated to improving the quality of public service in Canada


Objectives of the CMT:

provide a ready-made tool for public service organizations

enable like-organizations to compare results

facilitate the sharing of information gained and lessons earned between organizations

enable organizations to build internal benchmarks


How high is up? The need for common measures

0 6Very Dissatisfied Very SatisfiedSatisfiedDissatisfied

1 4Poor ExcellentGood Very good

1 10Poor Fair Good

90% Satisfied and Very Satisfied

90% Satisfied and Very Satisfied

=

What does the CMT Measure?

Central core of standard questions and standard 5-pt measurement scales

Client feedback (not citizen level)

Five elements of the service experience in the five dimensions or areas of the service experience

Additional questions customized by the organization to met their needs


Elements of the Service Experience:

Client expectations

Client perceptions of service experience

Level of satisfaction

Level of importance

Priorities for Improvements


Understanding Service Gaps:

Definition of Service Gap:

The difference between client expectations and the client’s perception ofthe service experience.*

*Source: Zeithaml, Valerie. et al., 1990

Expect-ations

Service

Gap


BC Office Products Centre

Low High

Hig

h

Areas where priorities should be focussed

•Products arrived on time

•Prompt handling of customer complaints

Organizational strengths

•Courteous and helpful staff

•Same day delivery service

Lo

w

Low Priority

•Personal visits by customer service reps

Unnecessary strengths - possible overkill

•On-line electronic ordering

•Electronic billing

IMP

OR

TA

NC

E

SATISFACTION


Service Dimensions:

Responsiveness

Reliability

Access & Facilities

Communications

Costs


Service Dimensions:

Responsiveness

Reliability

Access & Facilities

Communications

Costs

• timeliness

• # of contacts to get service

• courtesy

• flexibility

• fairness

• competence

• etc.


Service Dimensions:

Responsiveness

Reliability

Access & Facilities

Communications

Costs

• outcome

• minimal error rate• adherence to policy & standards• etc.


Service Dimensions:

Responsiveness

Reliability

Access & Facilities

Communications

Costs

• convenient location

• telephone access

• hours of service

• variety of access modes

• etc.


Service Dimensions:

Responsiveness

Reliability

Access & Facilities

Communications

Costs

• availability of info

• official languages

• questions answered

• etc.


Service Dimensions:

Responsiveness

Reliability

Access & Facilities

Communications

Costs

• ease of payment

• reasonable cost


How Does the CMT Work?

Offers a consistent measurement system

Offers comprehensive item bank for selection

Offers descriptive and Likert types of questions

Requires customization by organization

Allows addition of questions by organizations


e-CMT Overview


e-CMT - How do I use it?

Step 1 The CMT Intranet web site Step 2 Design survey on-line Step 3 Administer survey Step 4 Load data Step 5 Analyze results


e-CMT - Two Parts

Survey Design&

Database(Internal Server)

Survey Analysis&

Reporting(Desktop Software)

Data and Surveys

Web

bas

ed s

up

po

rt


e-CMT on Manitoba Intranet


Design your survey on-line


CMT DEMO

Run DEMO here.


Customer Service Support

User Manual FAQ Help Files CD ROM Tutorial EDS Helpdesk


1. Surveying Guidebook:

Takes a surveyor through: planning constructing conducting analyzing and acting on surveys.

Customized for the CMT by the Province of Ontario with permission from the USA’s Environmental Protection Agency (EPA).


3. CMT Publications:

Client Satisfaction Surveying: Common Measurements Tool

Client Satisfaction Surveying: A Manager’s Guide

These (and other useful service publications) are available online at:

www.ccmd-ccg.gc.ca/publica/publi.html

(located under the service quality theme)

CommonMeasurementsTool

Faye Schmidt withfor the

ANAGEMENT DEVELOPMENT

Teresa Strickland


Training

Applied Research Methods Delivered by U of W 2-Day Training

Course Co-developed MB-

Canada Experimenting with

on-line Training


Who’s been using thee-CMT in Manitoba

Department Natural Resources Education & Training

STEP, Stevenson, Career Start

Family Services Regional Op’s

Labour Agriculture French Language Services SOA’s Auditor General

Department Natural Resources Education & Training

STEP, Stevenson, Career Start

Family Services Regional Op’s

Labour Agriculture French Language Services SOA’s Auditor General

Type of Survey paper

paper paper

paper & electronic

paper, telephone, in-person electronic electronic

kiosk

Type of Survey paper

paper paper

paper & electronic

paper, telephone, in-person electronic electronic

kiosk


e-CMT examples: Manitoba

Student Temporary Employment Services

• initial CMT survey identified “information” as largest value/satisfaction gap 2000

• focus groups confirmed application process as major difficulty for students (access, confirmation)

• STEP application put on-line 2001

• 94% of applications now received on line

• e-CMT telephone survey indicates 97% satisfied or very satisfied

• 2001 focus groups identified managers want requests via Intranet.


e-CMT examples (2)

Manitoba Child and Family Services• Regional operations

• Round 1 : CMT administered to direct clients via telephone interviews

• Survey confirmed clients generally satisfied but find access and program information as priorities for improvement

• Round 2 CMT delivered to “partner” agencies

• Survey revealed “partners” generally satisfied

• Managers indicate major benefit was to demonstrate and reinforce positive performance to staff


Other Jurisdictions

Government of Canada• Service Improvement

Initiative

Ontario B.C. PEI Quebec City of Winnipeg City of Toronto


CMT Implementation

Approximately 50 CMT Surveys completed Federal Government , Manitoba, Ontario, B.C., PEI

(Social Services to Internal Travel) Mail, Telephone, In-Person Interview, Internet Macro - Organizational - Business Planning Micro - Client Segments focusing on specific operational

issues.

Innovation/Custom Applications Employee Attitudes Salary Surveys/Competencies


Ministry of Labour Ontario

60

65

70

75

80

85

90

95

100

60 65 70 75 80 85 90 95 100

Percentage - Important/Very Important

Per

cen

tag

e -

Sat

isfie

d/V

ery

Sat

isfie

d

Time

Courtesy

Competence

Fair and Equitable

Informed

MEAN (S)

Opportunities – AREA FOR FOCUS

Exceed

Other Opportunities

Maintain


Lessons Learned

Project Management Jurisdictions have under estimated resources needed to support CMT/e-

CMT Slow roll-out based on pilot projects more successful Training is essential to ensure organizational readiness

Technical Customized solutions may be necessary “Firewall/Security” issues must be

dealt with immediately - internal servers Distribution of software should be centralized

Providing Customer Support is Crucial Survey design,administration, analysis e-CMT utilization


Lessons Learned (2)

Based on evaluation of CMT for use for

Internal Services (conducted for CCMD) CMT Questions need to be modified for

Internal Services CMT works best where dept’s have access to

technical experts re: survey design and administration

e-CMT correct way to go, but needs to be more flexible


CMT - Ongoing development

CMT Standards Board coordinating development of expanded CMT on-going revision of CMT based on feedback

received to date expand CMT question bank to include other

service delivery channels (telephone, mail, internet, kiosk)


CMT by Channel

Federal TBS and Manitoba testing new CMT questions designed to evaluate channel specific services: Telephone Service On-Line Services Internal Service Delivery

What drives ratings of government web sites?National results

1.Easy to find what I'm looking for

2 . Outcome: I got what I wanted

3. Visually appealing

4. Has the information I need

5. Pages load quickly

These five drivers account for 41% of the variance in citizens’ ratings of government sites.


Interest in ESD Evaluation is Growing

Post 9/11 fiscal environment ESD proponents are being challenged

to demonstrate “benefits” ESD project sponsors looking to

quantify benefits and success stories ESD project managers need

information re: success, maintenance, and future development of applications


ESD Performance Framework

CustomerSatisfactionMeasures

Application &Service RelevanceMeasures

ActivityMeasures

Custom orApplication-specificMeasures

Value Measures


Tools & Techniques

Focus Groups Interviews On-Line Survey The Common

Measurements Tool


Future of CMT

The International Institute for Citizen-Centred Service Affiliated with IPAC Lines of Business

• Establishing a CMT benchmarking centre • The development and marketing of the electronic

CMT• Intergovernmental research projects and learning

events• Establishing a best practices database re:service

delivery


CMT

Questions and Answers

…Thank you

common measureme nts the tool guy gordon assistant deputy minister service manitoba manitoba lois...

Documents

customer feedback cmtwhat

quality of public service

service delivery

customer datalessons

centred service network

studydo citizens

citizen level

client feedbackfixing