Online Survey Sites – Theoretical Example

A magazine company creates brand new cover concept designs and

changes its subscription fees and wants to test interest and appeal in those.

Survey Script A Survey Script B

To save money in the short-term, the magazine company decides to use do-it-yourself survey to create

a survey. The in-house design team designs a 15 question script which asks questions to get feedback on

cover designs and new subscription price points.

Understanding the implications and impact of this venture, the magazine company decides to hire an outside third-party consultant to manage the market

research including writing the script, managing the data collection sampling, and analyzing the results.

Using Do-It-Yourself Software Using a Professional Consultant

Created By: George Kuhn

Website: www.RMSresults.com Blog: www.RMSresults.com/blog

Unknowingly, the inexperienced survey writing team used biased language such as “how excited are you

about this slick new design”, worded the pricing questions as a sales pitch such as “for only 7 cents

more per day you will have access to…”, and used a weighted scale: extremely likely, very likely, likely, not at all likely (3 positives and 1 negative). Undoubtedly, the questions biased responses and misguided results for

both cover designs and fees.

Sampling A Sampling B

Results A Results B

The experienced survey writing team from the market research firm worked with the magazine company to

understand its objectives of the market research. It was able to standardize the questioning to evaluate each

cover concept objectively, route questions according to specific responses, and use a conjoint analysis series of

questions to evaluate the best fee structure for the magazine company. This resulted is a much more well-

rounded and better survey taking experience.

Knowing that the company receives a lot of traffic to its Facebook page, the magazine research team posted a generic survey link there for people to take the survey.

As a result, only users of Facebook were taking the survey which tends to be a more tech-savvy and

younger audience. Unfortunately, even non-subscribers who visited the Facebook page were also partaking in

the survey and this audience was very unknowledgeable about current subscription fees and

thought the new fee structure was extremely “too expensive”. In addition, 90% of respondents stated the e-reader or iPad version was more appealing than the

print versions.

The market research firm worked with the client to obtain customer email addresses for the survey so the

consultant could select a random sample of subscribers (to ensure results could be within +/- 3% points of their

stated totals). In addition to current subscribers, the firm purchased a random panel of non-subscribers to the magazine to test interest and appeal in the concepts

and asked specific questions about whether or not this would make them more likely to subscribe. Separate pricing questions were also asked to this audience

because of their unfamiliarity with current fees. The firm also emailed past subscribers to test their interest. This sampling resulted in a more accurate and well-rounded

study for the magazine company.

The internal team decided to choose the lowest pricing fee structure and focus more on e-reader and iPad editions. As a result, the magazine lost 36% of its share with users over

45 years of age and profit margins slimmed because the survey failed to ask about what features subscribers would

pay for if they were to pay the higher fee structure. If the magazine had found out its most popular feature was the

“weight loss tips” section, subscribers would pay more and if those tips were expanded.

Although using an outside consultant may have incurred thousands of dollars more for the client up-front, ultimately the market research provided the most

representative and most profitable cover design concept and fee structure to increase its subscriber base by 12%. Revenues were also increased by 28%. The additional

expenses in market research were covered with the addition of just a 2% increase in subscriber fees.

However, the market research provided 6 times that amount.