pb 6-5 customer satisfaction survey and analysis

8/7/2019 PB 6-5 Customer Satisfaction Survey and Analysis

1/13

Power Finance Corporation Ltd.(A Govt. of India Undertaking)

Space forInstitution Logo

Performance Benchmarking and Quality of Supply and ServiceDistribution Reform, Upgrades and Management (DRUM) Training Program

Customer Satisfaction Survey and AnalysisDr. S. N. Nandi, CORE International, Inc.Neena Singh, ERM India Private Limited

Introduction

Customer satisfaction measurement is a systematic process for collecting customerdata (e.g. surveys, audits, etc.), analyzing these data to make it into actionableinformation, driving the results throughout an organization, and implementingsatisfaction improvement plan. Thus, customer satisfaction measurement (CSM) is amanagement information system that continuously captures the voice of the customerthrough the assessment of performance from the customers point of view

Such measurement could me made by using any of the following methods:

Surveys Formal and informal feed back from customers

Use of customer account data

Complaints

However, survey is the most common method.

CSM will enable you to

Understand how customers perceive your organization and whether yourperformance meets their expectation.

Identify PFIs (priorities for improvement) areas where improvement inperformance will produce the greatest gain in customer satisfaction

Set goals for service improvement and monitor progress against a customersatisfaction index

Benchmark your performance against other organization

Increase profits through improved customer loyalty and goodwill

The organizations process for requesting, measuring and monitoring feedback ofcustomer satisfaction and dissatisfaction should provide information on continual basis.It should address conformance to requirements, meeting needs and expectations of

customers, as well as price and delivery of product.

. A typical customer satisfactions survey process includes following steps

1


2/13




Obtain Top Management Buy In

Determine Survey Objectives

Exploratory Research

Plan Sampling and Data collection

Design of Questionnaire

Actual Survey

Data Refining and Analysis

Recommendation and Implementation Plan

Plan timing of next Survey

Fig. 1: Survey Methodology

One must begin with essential preliminaries, such as obtaining top management buy in

and conducting exploratory research. From there, the sampling and data collectiondetails should be worked out and the key business processes identified.Essential Preliminaries

The first this to figure out is why the customer satisfaction survey is being conducted inthe first place. Answers such as to listen to the customer or because everybody has

2


3/13




one indicate a lack of clear motivation. Good reasons for doing a customer satisfactionsurvey include:

1. To find out where to focus process improvement efforts;

2. To determine whether previous improvement efforts have worked; or3. To see whether strategic advantages or disadvantages exist.

It is imperative that the reason for doing the survey be clearly stated and understoodahead of time.

Little can be done with customer satisfaction survey results without the approval of topmanagement. Thus, it is essential that the survey project have to management buy-in.top management should agree with the purpose of the research and understand whatactions will result from the survey. One way of ensuring management buy-in is toinvolve management in the design and construction of the survey.

It is also worth remembering that in any service, it is the front-line employees who havethe bulk of the customer contact and thus the greatest opportunity to implement processimprovements that directly affect the customer. Involving both management and front-line employees is crucial, because it lays the groundwork for effective implementation ofthe surveys findings.

During the exploratory research, it is essential that customer expectations are capturedin terms of factors/dimensions. These factors are preferred to be linked with differentprocesses through which services are delivered. This approach helps in identification ofprocesses requiring improvements.

Sampling

The importance of sampling

The purpose of sampling is to select a small number of units from the targetpopulation in such a way that the sample is truly representative of the total populationbeing surveyed.

Sampling Reliability

The reliability of a sample is judged on repeatability. If you did exactly the same surveyagain but with a different randomly selected sample of customers, would you get thesame results? This is primarily determined by sample size. Three specific aspects ofsample reliability determine the latter: precision, confidence level and variance.

Precision

3


4/13




The precision of a survey result can be defined in general terms as its accuracy. Thatmeans that if you conducted the same survey with a different sample the result for theyes vote could be anywhere between 33 and 43 per cent. With a much larger samplee.g. 1000 you would achieve much better precision, say +/- 1 per cent.

Confidence level

You can calculate the precision of your sample at various confidence levels. Absolutelycritical research such as medical research is typically conducted at 99 per centconfidence level. In case of distribution companies, it may be sufficient that samplingthe based on 95 per cent confidence levels, meaning that you can be confident that theresult would fall within your stated precision at least 19 out of every 20 times that thesurvey was conducted.

Variance

The precision that you can expect from a specific sample size at a given confidencelevel can still differ considerably depending on the variance of views held by therespondents. So the lower the variance (the range of views expressed), the morereliable any sample size will be at any confidence level.

Determining the sample size

The table shows required sample size against various levels of standard deviation andprecision

Table 1: Sample size determination

Required precision(+/-%) Standard deviation assumed

0.5 0.75 0.1 1.25 1.5 1.75 2.0

5 4 9 15 24 35 47 61

4.5 5 11 19 30 43 58 76

4 6 14 24 38 54 74 96

3.5 8 18 31 49 71 96 125

3 11 24 43 67 96 131 171

2.5 15 35 61 96 138 188 246

2 24 54 96 150 216 294 384

1.5 43 96 171 267 384 523 683

1 96 216 384 600 864 1176 1537

0.75 171 384 683 1067 1537 2092 2732

0.5 384 864 1537 2401 3457 4706 6147

0.25 1537 3457 6147 9604 13830 18824 24586

4


5/13




. For example: if variation in the sample is small (low S.D. of 0.5) and required precisionis within 1%, a sample of 96 will suffice. If variation in the sample is large (high S.D. of2.0) and required precision is within 1%, a sample of 1537 will be needed.

Most samples for customer satisfaction surveys would fall in the range of 200-500

respondents. Based on our experience this will yield precision of around +/- 1 per centat the 95 per cent confidence level for the vast majority of CSM surveys. Increasing thesample size would improve the accuracy but at a very high cost. Typically, to double theprecision (for example from +/-2 per cent to +/- 1 per cent) you would have to increasethe sample size by four times, quadrupling the cost of your survey.

Sampling options

There are several types of sampling methods that could be adopted for the survey.Some of those are outlined below:

Simple random sampling

To draw a simple random sample, the researcher starts with a complete list of thecustomer groups to be surveyed. The size of the sample is determined and chosen fromthe complete list on a random basis, which means that each individual has the samelikelihood of ending up in the sample.

Stratified random sampling

Random sampling can sometimes distort results in markets where some customers aremore important than others. In this case stratified random sampling is used. The method

involves weighting the sample on the basis of the importance of various segmentsmaking up the market.

Cluster sampling

The third and less costly method of producing a random sample, is cluster sampling.Random samples are therefore often drawn from a small number of tightly definedlocations (clusters) deemed to be typical of the target market in question. This method isconsidered to be of sufficient statistical accuracy for most commercial market research.

Non- random samples:

Convenience samples

This would involve gathering data from any convenient group, for example, passers-byin a street interview situation.

Judgmental samples

5


6/13




Here respondents would be selected on the basis of the researchers (or interviewers)belief that they were representative of the target market being studied. The moreknowledgeable the person who selects the respondents the more accurate it is likely tobe.

Quota samples

Quota controlled samples are frequently used by commercial market research agenciesto minimize the cost of fieldwork. The research agency, using as a basis publishedstatistics, decides on controlled quotas (or groups) of respondents for each interviewerin the field. For example, the interviewer might be told to question 20 females agedbetween 20 and 35 years, 15 females aged between 36 and 50 years and 25 femalesaged 51 years and over.

Design of Questionnaire

The basic principles of asking questions

When judging any question on a questionnaire you need to apply the following tests toit:

Will the respondent have the information / knowledge to answer it?

Will the respondent understand the question?

Is the respondent likely to give a true answer?

Will the question bias the response?

It is very easy for a question to fail by any of these tests, so lets examine each of themcarefully.

Necessary Information / Knowledge

As far as customer surveys are concerned, setting very clear objectives at the outset ofthe project will minimize this pitfall., You may want to measure the satisfaction andexplore the attitudes only of knowledgeable people with direct experience of theproduct.

Understanding the question

There are three problem areas in understanding.

Firstly, there is the loose or ambiguous meaning of many common words. A goodexample is regularly. A regular habit to one person may seem a very occasional one to

6


7/13




someone else. Do you eat out regularly? would therefore mean different things todifferent people. Rather than try to define such words it is better to avoid them andreword the question: When did you last eat out?, or, How many times have you eatenout in the last month?

The second problem is the double question. Does Ariel Future get clothes clean withoutdamaging the fabric? This one question should be split into two separate questions toaccommodate the full range of possible opinions. Sometimes questions use two verysimilar adjectives: Do you find the atmosphere in this restaurant relaxing and informal?If the researcher sees only one meaning to this question the closer of the two adjectivesshould be used and the other eliminated.

The third problem is the long rambling question, or definition, Used in a self-completionquestionnaire, it results in many respondents not taking the trouble to figure it out, eitheromitting it altogether or, even worse, ticking any available answer. In telephone surveys,too, the long question is difficult for both respondent and interviewer.

Giving a true answer

First, respondents often find it difficult to articulate their views, especially whereattitudes are concerned. They should therefore be given plenty of time to formulate theirides into words.

The Second potential pitfall is respondents memory. People may believe they aregiving a truthful answer but in reality, due to a defective memory, they are giving aninaccurate response.

The third pitfall is more difficult to identify and negotiate. Some respondents may beunwilling to answer truthfully to some questions. This is not malicious. There is noevidence that respondents try to deliberately mislead researchers in customer surveysor any other kind of research but there is evidence that respondents will give what theyconsider to be an acceptable answer which does not really reflect their views.?

A fourth reason why you may generate inaccurate data n a customer survey is that thequestion itself may suggest a form of answer to the respondent.

The most likely form of bias in a customer survey is to influence the customer towardssatisfaction rather than dissatisfaction. To avoid this it is better to use phrases like Rate

our performance rather than How satisfied are you with

Piloting

As well as studying the questionnaire an applying to it the tests of knowledge,understanding, truthfulness and absence of bias, you must pilot it to test just how well itworks in practice.

7


8/13




Types of question

The main distinction between types of question is between the closed and the openquestion. In the survey itself, where you may have time constraints and where you need

to produce quantitative answers for statistical analysis, you will use mainly closedquestions.

Closed questions

Closed questions give respondents a fixed selection of answers to choose from. Theyare quick and easy to administer and analyse, offer the least scope for interviewer orrespondent error and produce quantifiable data. The interviewer (or the respondent in aself-completion questionnaire) need tick only the relevant box.

Closed questions could be dichotomous with only two alternative answers possible.They can be multiple choices, enabling respondents to choose from any number ofpossible answers. Some questions demand that the respondent selects only oneanswer (single response), The table shows required sample size against various levelsof standard deviation and precision other questions allow selection of all relevantanswers (multi-response).

Closed questions can also be linked with a rating scale to qualify the strength of theresponse.

Open questions

Open questions are used when the researcher does not want to lead the respondent inany way. These questions are more commonly used at the exploratory stage rather thanat the survey stage.

Purpose of questions

Whether the questions are open or closed, they can be used for different purposes,usually to find out about respondents behavior, to understand their attitudes, or togather details about them for classification purposes.

Behavioral questions

Behavioral questions are about what the respondent does or has done in the past. Theyare both factual and tangible which usually make them easier to answer so they tend tobe placed at the beginning of questionnaires.

8


9/13




Attitudinal questions

Although behavioral questions can ask the respondent to dig deeply into his or hermemory, attitudinal questions usually require more thought and more decision makingand should therefore follow rater than precede behavioral questions. This gives the

respondent some time to think about the issues involved. It also provides an opportunityfor the interviewer to build rapport very useful later when the interviewer wants therespondent to make the effort to give proper consideration to any difficult attitudinalquestions.

Customer satisfaction measurement is primarily concerned with attitudinal questions.Attitudes towards the product and services and selecting the supplier, attitudes aboutthe performance of the product and supplier and, most difficult of all, attitudes about theimage of the supplier, will all come within this category.

Rating scales

Since customer satisfaction measurement is all about measuring attitudes, the methodyou use to do the measuring is an extremely important aspect of the exercise. Marketresearchers have developed several different types of rating scale to measure thestrength of peoples attitudes and her ewe examine those scales with most relevance tocustomer satisfaction measurement: Likert scales; verbal scales; semantic differentialscales; un-graded scales; numerical rating scales; ordinal scales; SIMALTO scales.

Likert scales

Likert scales are designed to measure degrees of agreement with a statement, the

wording shown in the example reflecting the Market Research Societys recommendedapproach. However, some people maintain that it is better to offer only four possibleresponses, eliminating the middle ground, easy option.

Verbal scales

Similar in principle to Likert scales, verbal scales use words to describe degrees of theattitude being measured. In the case of customer satisfaction measurement, of course,the concepts involved are importance and satisfaction.

Semantic differential scales

Semantic differential scales display an attitude battery between two opposing words.Typically you would allow more options than the four or five given in the Likert scalesbut you would not label the options in any way. The respondent is left to define thestrength of his or her attitude by selecting the proximity to the appropriate adjective.

9


10/13




Numerical rating scales

A numerical rating scale requires respondents to ascribe a mark, out of 10 for example,to indicate the strength of their attitude. There is no firm rule on the size of the scale.Some companies use a scale of 1 to 5. Federal Express uses a 101-point scale (0 to

100) to measure micro-shifts in customer satisfaction. In the UK, a ten-point scaleworks very well. It is understood by respondents who have often grown up with marksout of ten at school and it is wide enough to accommodate a good range of customeropinion.

Ordinal scales

Ordinal scales require the respondent to indicate the relative strength of his or herattitude to the different criteria by ranking them in order of importance, preference andso on. No further qualification is demanded.

Actual Survey

There are two main methods of gathering information from customers: qualitative andquantitative:

Quantitative Research(Hard issues and facts)

Qualitative Research(Soft issues and opinions)

Telephone surveys Focus groups /user groups / customer panelsSelf-completion questionnaires(Comment cards)

Face-to-face interviews

Postal surveysThird party surveysOnline surveys

Qualitative methods provide feedback from customers on soft issues and opinions andgive the organization an opportunity to probe customers feelings and attitudes on an in-depth basis. Typical qualitative techniques include user panels, customer groups, andpersonal interviews.

Quantitative methods allow the organization to quantify customer opinions in anumerical fashion. Typical quantitative techniques include telephone surveys, self

completion questionnaires, the monitoring of compliments and complaints, and mysteryshopping.

Care is required in the choice of the best methodology. The best plan is to adopt avariety of these techniques an organization is wise not to restrict itself to just onemethod. Forte Hotels, for example, uses mystery guest reports, quarterly telephonesurveys and biannual guest-satisfaction postal audits to measure customer retention. It

10


11/13




is normally possible to gain a greater degree and depth of information from qualitativemethods, although quantitative routes reach a wider audience and often prove morecost-effective. Best practice shows that organizations need to use qualitative researchto first gain insight into customers attitudes and behavior before using quantitativemethods.

Data Refining and AnalysisData collected through the questainnaire are first checked for incompleteness,inconsistencies, and inaccuracies.

Structuring the questionnaire according to business processes, makes it possible tostructure the data analysis in a managerially meaningful way. The analysis determineslinks between the following components:

Fig. 2: Hierarchy of Dimensions

Customers satisfaction level with the dimensions (subprocesses) of a process affectstheir satisfaction level with that process. Likewise, customers satisfaction level with thevarious processes influences their satisfaction level with the firm overall.

Using the statistical models for predicting satisfaction, it is then possible to estimate therelative importance of the various processes in predicting satisfaction with the firm, andthe relative importance of the dimensions in predicting process satisfaction. Theseimportance weights can then be related to performance, as measured by the percentsatisfied. Charting the importance and performance information can help providepreliminary ideas about where resources should be allocated.

Given that a complete data set has been constructed, there are both simple (bivariate)and more complex (multivariate) ways to relate the process satisfaction levels to theoverall satisfaction level. The easiest way is to use a procedure called regression.Regression is a statistical method that allows you to quantify the relationship betweenvariables. Further, once that relationship has been established, the relationship can beused to predict various outcomes.

11

Dimension (Subprocess)

Process

Overall Satisfaction


12/13




Dependent Variable = OVSAT

IndependentVariable

BetaCoefficient

StandardError

t p

Billing .50 .15 3.33 .00R2 = .25

Fig. 3: Predicting Overall Satisfaction - Example Bivariate Results

The t-statistic tells you how certain you can be that the beta coefficient is an accuratepredictor. The farther the t-statistic is from 0, either positively or negatively, the moreconfident you can be that the beta coefficient is significant.

Predicting Process Satisfaction

You have seen how the process satisfaction scores can be related to the overallsatisfaction score. Ultimately, though, you will need to know what drives the overall

process satisfaction. Thus you must relate the process dimension scores to the processsatisfaction level. Again, you may do this through the simple (bivariate regression)

Importance Performance Mapping

Another data presentation device that many managers find appealing is the importance-performance map. The idea is derived from standard quadrant analysis, as has beenused for many years in business strategy. In general, this approach argues, we shouldbe most concerned about those issues for which importance is high and performance(typically measured by average satisfaction) is poor. These yield the greatest potentialfor gain.

Recommendation and implementation plan

Based on analysis of data regarding sub-processes (dimensions./factors),processes,and overall, one can formulate important recommendations which could be consideredfor implementation. For example, one could work out process wise satisfaction levels,as shown in Fig.4. Take the case of metering process in a distribution company, say,which has been found to have a low score in satisfaction, the concerned departmentmay look for possible causes and then draw up an implementation plan.

Another set of valuable recommendations could be drawn up from the analysis of

importance-performance matrix. As shown in Fig.5,the matrix has four divisions inwhich values of different dimensions/factors will fall in. Depending upon which division afactor is located, certain actions could be suggested. For example, if a factor/sub-process/process is placed in a division-Low importance and Highperformance/satisfaction level, cost reduction program may be undertaken for the samefactor/ process. As this step will not affect the satisfaction level, higher profitability mayresult in.

12


13/13




ACTION CONSOLIDATE

LOW PRIORITY COST CUTTING

Conclusion

Monitoring of Customer Satisfaction is now very important for a distribution company,as it will lead to better customer experience and thereby goodwill. A quality survey isrequired to be undertaken periodically. The proper methodology for the survey as brieflydescribed here will go a long way.

13

System / Process / Factors

Satisfacti

onLevel(%

age)

A B C D

Fig 4: System / Process-wise Satisfaction Level

High

Low

Importanc

e

Low Performance High

Fig. 5: Areas for Actions

pb 6-5 customer satisfaction survey and analysis

Documents