Appropriate Forms of

Opinion Polls Reporting

Brice Rambaud, 6 February 2013

Objectives

1) Determine whether a poll has been conducted according to accepted professional standards

2) Determine whether a poll’s findings have legitimate news value

3) Determine an appropriate way to publish/broadcast newsy polls findings

INTRODUCTION

Polls are an inseparable part of news coverage. There is no more accurate way to gauge the sentiments of the public than through a carefully designed and executed opinion poll:

Because well done polls are reliable sources of information, journalists must pay attention to them;

Because not all polls are well done, journalists must recognize which polls are valid and which polls are meaningless. Polls results can be influenced by many factors (wording, ordering of questions, sample etc.) so be careful not to be influenced…

1) WHEN DOES A POLL MEET ACCEPTED PROFESSIONAL

STANDARDS?

• Only report on “scientific” polls

The major difference between scientific and unscientific polls is who picks the respondents for the survey.

In a scientific one, the pollster identifies and seeks out the people while in an unscientific poll the respondents usually volunteer their opinions, selecting themselves for the poll – for instance Internet polls and SMS polls.

• Ask yourself a number of questions to decide if the poll is scientific or not (read Chapter 14 of Media Council of Kenya’s Guidelines for Election Coverage for more on this). Key Questions Every Journalist Should Ask About Poll Results include:– Who did the poll? – Who paid for the poll and why was it done? – How many people were interviewed for the survey? – How were the respondents chosen? – What is the sampling error for the poll results? – When was the poll done? – What area (nation, state, or region) or what group (teachers, lawyers,

Democratic voters, etc.) were these people chosen from?

– Are the results based on the answers of all the people interviewed? – Who should have been interviewed and was not? Or do response rates

matter? – How were the interviews conducted? What about polls on the Internet or

World Wide Web? – Who’s on first? – What questions were asked? – In what order were the questions asked? – What about "push polls?" – What other polls have been done on this topic? Do they say the same

thing? If they are different, why are they different? – What about exit polls?

Answers:- If the answers sound good, the poll was conducted properly. Then

you can decide to report or not, depending on information / news you can gather from the poll.

- If a poll appears to have been conducted using dubious methodology: 1) do not publish: its outcome should receive no mention in the media, 2) there may be another story behind the poll…

Key concerns:– Validity. Who commissioned has an influence on results.– Capacity of the pollster (reliability and truthfulness of results)– Representativity.– Margin of error.– Context.

2) WHEN IS A POLL NEWSWORTHY?

Use the opinion poll as a source of information.

Information must be timely and newsy.

Examples – Newsy or not?

-Dec 2012: Raila Odinga leads voting intentions having 42% voting intentions. The second favorite candidate for Kenyans is Uhuru Kenyatta with 40% voting intentions.

-January 2012: Kalonzo Musyoka is the most popular leader in Eastern province - 80% of voters saying they would vote for him in the next election.

-February 2012: Voters are particularly concerned about two issues in the country: inflation and security. When asked “what is your main preoccupation?” 25% responded “inflation” and 23% “insecurity’’

Be careful:

= Voting intentions are not always newsy

= Voting intentions are not the only thing that matters

= What are the real issues contained in the opinion poll that matter for your audience/readership?

= Follow up on poll results by investigations. Opinion polls are not a story on their own but a source of information.

Advice to journalists

1)Beware of press releases that suggest a “slight” or “modest” trend 2) Distinguish between poll findings and a pollster’s interpretation of poll findings.

Example 1: X leads with 42% while Y has 39%.Example 2: This has an impact on that – but there is no evidence of impact.

There may be a huge difference disguised in a press release. So have your own interpretation!3) Investigate alternative explanations for poll results that are surprising.

Is it a one day instant poll? Is it an online poll claiming to represent voters? Is there a low response rate?

Advice to journalists

4) Beware of analysts who seek to predict the election based on a poll. A poll’s ability to predict is limited because:

- at least 10% of all voters make up their mind on the voting day .

- contemporary campaigns are designed to move voters late in the campaign (most advertising is spent the weekend before the election).

- Polls often underestimate the third-party candidate (le Pen in France in 2002; Ross Perot in the US in 1992).

5) Lighten up on the horse race. Opinion polls and media releases of major candidates focus on the horse race especially the final two months of the campaign. Don’t be fixated on a story! Do not neglect to give voters what they want/need to help them decide for whom they vote.

Media studies show that voters are much more interested in stories about the candidates issue positions and stories about how the election might affect them than they are in stories about the horse race. Many voters say the media pay too much attention to the horse race.

Advice to journalists

6) Don’t use polls to decide what issues to emphasize in election coverage.

An opinion poll can’t set the campaign agenda – that would be an abdication of your leadership responsibility. Opinion poll is data. You should use poll to enhance your leadership not substitute it.

7) Don’t use polls to ask the public for its expert opinion in matters where it is not expert.

Q: “Do you think that the Kenyan foreign policy should reconsider the partnership with Burma?”

8) Don’t consider the public is monolithic with uniform knowledge and concern. It is just the sum of particular groups…

3) APPROPRIATE WAY TO PUBLISH/BROADCAST

POLLS FINDINGS

When any public opinion findings are published in print media, these should be accompanied by a clear statement of:

-a) the name of the research organization carrying out the survey-b) the universe effectively represented (i.e. who was interviewed)-c) the achieved sample size and its geographical coverage-d) the dates of field work-e) the sampling method used-f) the method by which the information was collected (personal, telephone interview etc.)-g) the relevant question asked. To avoid possible ambiguity the actual wording of the question should be given.

In case of broadcast media:

it may not be possible to give information on all these points so cover as a minimum points a) through d) above.

TV: use visuals to cover these points!

Recommendation for print journalists: use a ‘data box’

The survey was carried out by ABC Research on behalf of Internews using a national quota sample of 1111 adults of voting age personally interviewed in 102 locations between 1st-5th March 2012.

Survey carried out by XYZ Research on behalf of News Inc. National survey of 1234 adults of 18 and above, interviewed by telephone between 25th and 28th February 2012 using random digit dialing and a quota sample.

It is even simpler for on line journalists as they can provide information at the bottom of the article and a link to the whole study!

Example: http://www.cbsnews.com/8301-503544_162-57395703-503544/poll-obamas-approval-rating-sinks-to-new-low/?tag=cbsnewsLeadStoriesArea

Advice to journalists

1) Make clear which group is of voters is being measured: voters, registered voters, probable voters etc.

2) The percentage of respondents who give “I don’t know answers” and those who say they are not going to vote must always be given. They can significantly affect the findings!

3) In the case of voting intentions surveys, it must be always be made clear if voting percentages include any of these respondents who answered “don’t know” or “will not vote”.

4) Always report polls with context. Example: precise if the poll was conducted three days after the ICC confirmation of hearings, or one week before the Kenyan military incursion into Somalia etc.

5) Understand and give the margin of error in the case of voting intentions surveys, especially when there is a horse race (see New York Times policy).

Question: How could journalists report on the findings of the 2007 Synovate poll saying that Kibaki had 42% intention votes and Raila 39% bearing in mind the 3% margin of error?

Advice to journalists6) When reporting the comparison between polls, the margin of sampling error of the difference “between” polls must be determined.

7) Avoid using polls conducted by parties or individual campaigns. Prefer the polls commissioned by your media house.

THANK YOU!