The hotel management wishes to find out whether the air conditioning in the lounge is comfortable.

The supermarket wants to know whether the queuing time at the counter is a deterrent to repeat customer visits.

The car maker wants to know whether the dashboard visibility problems are solved with the new design.

The software interface designer wants to determine whether the user experience is better with the new screen design.

The professor wants to determine whether students are able to understand a complex concept better with the new interactive game he has devised.

The tax authorities want to determine whether the new e-filing system is found easier by customers than the old paper based one.

Why measure satisfaction?

A satisfied customer will bring more. It is vital to keep measuring customer satisfaction and implement measures to improve it as you go. We explore here, the essentials of customer satisfaction measurement.

What to measure and how? Methods and scales of measurement

Customer satisfaction has been popularly measured with the use of Satisfaction surveys. However, there are two glitches in the survey process. These result in a low response rate.

Problem 1- Lengthy surveys:

A survey that attempts to explore into many parameters of satisfaction may become rather lengthy. Respondents usually shun long questionnaires and do not readily respond to such. They would more likely put away the questionnaire for another day. The ‘another day’ doesn’t come up too soon. The low response rate to satisfaction surveys that we commonly see is a clear indicator of this. ‘Death by Questionnaires’ is the light-hearted phrase that often goes around in this context.

Problem 2-Scales of satisfaction measurement:

Satisfaction is measured for multiple parameters like delivery, timely availability, packaging and promptness of support. Often the respondent is asked to assign rankings for each parameter on the scale of 1 to 5 or 1 to 10 based on the level of satisfaction. Filling out a satisfaction survey form on a flight, I have often struggled whether I should assign a ranking of 4 or a 3 to the quality of boarding process and clarity of announcement. Unable to make up my mind, I choose to sit on the fence and most often assign a 3 on the scale of 1 to 5. I do believe that others too have faced this dilemma. The human mind is not comfortable with fine resolution of grading, especially for subjective concerns like the level of satisfaction.

The solution? A direct yes/ no type question. The satisfaction survey can be thus split up, and individual parameters can be surveyed at a time with questions eliciting yes/no responses.

For example, if the hotel management puts one quick question on whether I am comfortable with the air conditioning in the lounge, I am neither overloaded with a questionnaire, nor do I have a great difficulty in deciding whether the ambience is in fact comfortable. With a sufficient number of guests answering the question, a hotel manager may draw conclusions about what percentage of the guests feel comfortable and judge the level of satisfaction so far as comfort air conditioning goes.

A number of quick surveys as this for each parameter, makes satisfaction measurement a continuous process.

An important question that comes up at this point is about how many customers you need to survey in order to make a judgment about the satisfaction level. Since surveys are conducted on a few customers to make a judgement about the satisfaction levels of the whole community, determining the sample size requires a bit of statistical evaluation.

How many customers to survey? The question of sample size

For the sake of argument, let’s take a case where I ask 2 customers whether they were comfortable with the air conditioning. One says Yes and one No, could you confidently deduce that there is a 50% satisfaction level? The obvious answer would be no.

Taking the same argument ahead I survey 100 customers and 50 say they are happy with the air conditioning I come up with the same 50% level of satisfaction. But I can be more confident about what I deduce in the second case.

Putting it numerically, there are two factors that decide how many customers I should survey in order to judge satisfaction levels.

Confidence Level: This is the amount of confidence I need in judging the satisfaction level of the customer community based on the sample I survey. 95% is usually a standard confidence level used in statistical evaluation. Which means that with the sample size I determine, I would make the correct judgement about the satisfaction levels 95 times out of 100 times that I conduct the survey.

Margin of Error tolerable: I find as a result of the sample survey that the satisfaction levels are predicted at 80%. The 80% estimate is subject to a certain amount of error. This error can be limited to a predetermined level with a suitable sample size. In this case the margin of error band would be 4% (5% of 80%). This would mean that my estimate of satisfaction is a percentage in the band of 78% - 82%.
Understanding the level of customer satisfaction within a 5% margin of error is usually workable for most of us. If you need a better accuracy, say limiting the margin for error to 1%, your sample size will go up significantly without really giving you a substantial added benefit.

For example:

Goal: I need to determine whether customers visiting my website find it easy to navigate around. I ask them for their opinion yes/no terms…easy or not easy.

Number of visitors in a month: 20,000

Confidence Level: I choose to work with a 95% confidence. Meaning that I want to be right about my judgement 95 times out of 100.

Margin of error I can tolerate: 5%

The calculator tells me that need to survey 377 visitors.

On enquiry, 300 out of 377 people I survey say that the navigation on my website was easy. I can deduce as follows:

Parameter: Easy Navigation

Satisfaction level = (300/377)*100= 80% with a margin of error = 5%.
The error band has a total width of 4% (+/- 2%) Leading me to an estimate interval of 78% to 82%

Describing in words, so far as easy navigation goes, satisfaction levels are between 78% to 82%. My estimate of satisfaction levels will be correct 95 times out of 100 surveys that I conduct.

The sample size calculator given here helps determine sample sizes for satisfaction surveys.

Formulas for statistical Sample Size calculations can be found in standard statistics texts including the one referred in the bibliography^{1}.

Enter the total number of customers or persons you want to conduct the satisfaction survey on. For example, in a monthly satisfaction survey, if 10000 customers visit your book store, key-in 10000 here.

If the satisfaction survey is to be conducted on a continuous flow of customers and the number of customers is large, leave this box blank.

Confidence Level:

%

Enter the Confidence Level you desire. This should be number larger than 0 and smaller than 100. Use the default of 95% if you are not sure what value to use.

Margin of Error Tolerable:

%

Enter the Margin of Error Tolerable, as a % . This should be number larger than 0 and smaller than 100. Use the default of 5% if you are not sure what value to use.

Survey Sample Size:

The number of customers you should survey is calculated here. This survey will enable you get satisfaction level result at the desired statistical confidence and within the specified margin of error.

References:

Cochran, W. G. (1977). Sampling techniques (3rd ed.).New York: John Wiley & Sons.

Author: Ravindra Khare
Symphony Technologies Pvt. Ltd.,
B-4, Saket, Vidnyan Nagar, Bawdhan,
Pune 411 021, INDIA

Published: October 2013

Ravindra Khare is a Founder and Director of Symphony Technologies.
He is a qualified Mechanical and Industrial Engineer and a keen student of Quality & Productivity Technology for the past 25 years.