research now conducting successful online focus groups november 2007

Research Now Conducting Successful Online Focus Groups

November 2007

London

Paris

Hamburg

Frankfurt

Athens

Toronto

New York

San Francisco

Chicago

Sydney

Melbourne

• Introduction

• Recruitment

• Focus Group examples

• Case Study

• Conclusions

Contents

Introduction

Background – Research Now

• Research Now is a leading international provider of online fieldwork to the market research industry, and our focus is on quality across all elements of our business

• We are fieldwork only – we are therefore not in competition with market research agencies – we will therefore focus this presentation on the enabling side of online focus groups today

• We conduct an average of 550 online projects globally per month

• However we have only conducted a handful of online focus groups for clients, and our own pilot studies and have gathered some valuable learning.

• We have also conducted qual intercept and bulletin board techniques to a greater extent, these are not covered today.

Specific Aims Today

Aims of this session….

• Today we are going to walk through the ‘enabling’ of an online focus group

• Less theoretical, you are the methodologists and will make the ultimate decision on appropriate application, though we will make some suggestions

• More practical – what are the nuts and bolts? Stages? How do you effectively execute an online focus group (including case study)?

• Then we will revisit the question….

Do online focus groups provide a good context for engaging with

Participants online?

Online Qualitative

Context

Background – Online Qualitative

• Some of the earliest online qualitative methods in the UK occurred around 1998• NOP with Bulletin Boards• Virtual Surveys with Moderated Email Groups (MEGS)

• One of the pioneers of online qualitative groups was Michael Herbert of Michael Herbert Associates, using a modified ‘chat room’ format – CyberQual

• Developed his own qualitative recruitment panel (offline and online interaction)

• However recruitment was almost exclusively conducted by phone.• Only very internet savvy / early technology adopters were able to

participate• Broadband required, and penetration very low!

• Different dial up speeds were an issue, meaning that Participants with greater speed access had an advantage in putting their views across

• Lack of familiarity with the internet meant this method was very niche and only appropriate for technology savvy audiences (very niche consumer groups or IT Managers)

• Acceptance of online research as viable was very low, and it has only been in the last 2-3 years that clients are comfortable with online

Background – The Online Audience

• Around 10% quantitative fieldwork is now conducted online in the UK – what about qualitative?

• Over two-thirds of the population are online, and over 40% have broadband access

• Researchers are increasingly looking to the internet to power their qualitative solutions

• Consumers are more and more internet-savvy and are more used to engaging socially online thanks to the proliferation of online communities and networking sites such as Facebook, Friends Reunited, MySpace and YouTube

Do online focus groups, therefore, provide a good context for engaging with


Background –Why an online qualitative solution?

- Traditionally, running a qualitative project with face-to-face

focus groups can be stressful and time-consuming:

- Need to travel to a variety of cities and venues

- Need to organise for clients to attend and view the group

- Need to find a mutual date / time that Participants, the

moderator and the client (s) can make

- Need to interpret information and produce summaries of

research findings from multiple groups quicklyConducting focus groups online can address

most, if not all of these issues

Background – Online Qual and The International

Perspective

- Face-to-face groups are a logistical

challenge for researchers looking to

conduct multi-country research

- Online focus groups allow you to talk to

Participants from multiple

geographical locations for one

single group or conduct multiple

groups in a single day

- You can also invite clients to log-in

and tune in to discussions wherever

they are in the world

Reach Participants across the world from one single touchpoint

Overview of the Online

Focus Group Process

Steps in the focus group process

• Constructs are very similar to offline qualitative. As you probably anticipate it follows the same steps with some notable differences….

• Recruitment: Can be offline and traditional for an online group, alternatively online access panels or the online environment can be suitable• Of course, a broad geographical dispersion is possible using

online focus groups – for very low incidence samples this maybe the only context that you can engage in a qualitative exercise – for example, apple iphone purchasers

• The Focus Group : Online Groups mirror all of the facets of a traditional focus group:• Moderator with discussion guide• Stimulus• Participants• Two way mirror / client area• Yellow post it’s from client to researcher

• Analysis: Transcripts available, and ‘recording’ of focus groups

Online vs Offline – Summary of The Benefits

Offline

Online

Group Feature

x No travel required

Simultaneous client viewing

x Include Participants from different cities / countries in the same group

x Conduct multiple groups across multiple locations in one day

? Recruit niche Participant sample in a matter of hours

Use your own moderator

? Seamlessly play audio / video clips and web content

x Engage with the ipod generation in their own “environment”

x Instant transcripts

Recruitment

Recruitment

- Traditional offline recruitment methods still act as a very powerful tool for

engaging participants for online focus groups

- Beyond the practical benefits of offline recruitment, panel recruitment (online)

can also add value to your research by:

- Providing the more elusive Participants quickly and easily – e.g. find

male owners of Audi TT’s in Manchester and London, or female users of a

brand of deodorant under 30 in a matter of hours rather than days

- Engaging with a recruitment source that are already used to giving their

opinions in the online modality. Certainly they are more likely to be found

IMing, Blogging and interacting socially online than in the local shopping mall

- One of the major strengths of online panel is it’s broad geographical

dispersion. This means that for focus groups we can bring people together

where previously it would have been impossible

It is possible to fuse qual and quant data collected through panel to

create very powerful insight into behaviour

Online Focus Groups

Types of Focus Groups

Fall into two broad types….

More Basic ‘Chat Room’ style online focus groups

•Moderator and Participant with basic identification

•Standardised transcript output

‘Rolls Royce’ Applications (becoming the norm).

• Usually include the ability to have a discussion guide pre-loaded into the group

• A client area• The ability to show stimulus and potentially video• Emoticons and sounds to simulate emotion, for example applause,

gasps etc.

•Standardised transcript output

Example – Participants view

1. Participant portal

Participant sees

moderator

questions

displayed on

screen and can

type responses

Participant can

also see other

participants

comments in an

open screen

Example – Moderators View

2. Moderator portal

Moderator types in

live questions or

selects a question

to ask from the

pre-loaded

discussion guide

Example – Client Viewpoint

Client participation via a

split screen –

Can view Participant

discussion as well as

communicate with

colleagues

Because the moderator is

able to communicate in

both rooms, instant

client-moderator

communications are

achievable

3. Client portal

Example - Stimulus

Moderator can

display visuals

to the group

Example- Stimulus

Moderator can

show group

websites or

other live online

content within

the focus group

environment

TRANSCRIPTS:

Instantly

available

Case Study

Case Study – Background

Research Now conducted 2 pilot groups in August 2007 amongst our own

panelists to get some diagnostic feedback on the ‘panel experience’, and

to fully test the process of conducting a focus group

The Pilot was designed to give us valuable feedback in three main areas:

1. Recruitment – what is the most effective way of recruiting to an online focus

group?

2. To test the effectiveness and technical stability of our online focus group package

3. To examine Participant engagement once panelists were focus group participants

Case Study - Schedule

Pilot Group 1 Pilot Group 2

Recruitment commenced Friday 3rd Sunday 12th

Participants confirmed Monday 6th Monday 13th

Reminder sent Tuesday 7th Tuesday 14th

Date Wednesday 8th Wednesday 15th

Time of Group 6.30pm UK time 6.30pm UK time

Total no. RN panellists contacted 500 410

Total no. RN panellists recruited 12 20

Total no. group participants 2 7

Length of group 1.5 hours

Incentive structure £20.00, if you do not qualify for the survey you will receive a free entry into our quarterly prize draw for £250

Recruitment Criteria Recruits to be UK panellists – mix of ages / gender between 18-24 –

all regular survey takers

Valued Opinions Panellists – random selection – qualitative research

Project managers Rahul Krishna, Lucy Swain and Dominic Jarville (RN)

Moderator Research Now (John Caldwell)

Group Subject The Valued Opinions Panel Experience

Case Study - Recruitment

Case Study – Recruitment

For both pilot groups, recruitment was conducted in 2 stages:

The first stage was a general screening survey, which was sent to 400-500 selected

RSN panelists to establish eligibility and interest……(more in a minute)

The second stage was used to select and invite eligible panelists to the specific

group events

Case Study – Recruitment Stage 1

ScreenerWe have an innovative new survey available for you to take. As a valued member of our panel we would like to invite you to an online session next week, where you can discuss your experience on our panel and the new technologies we are introducing to improve this experience.

If you complete the survey and undertake a one hour online session with ourselves you will receive £20.00, if you do not qualify for the survey you will receive a free entry into our quarterly prize draw for £250. These next few questions will help us determine what group to put you in. Please be advised that none of this information will be passed to any third parties.

Are you willing to take part in this Valued Opinions study taking place on Wednesday, August 8th…? Yes No


Do you or any member of your family work in any of the following industries? Please select THOSE THAT APPLY

Advertising (screened out) Public RelationsMarketing and sales Civil ServiceIT Professional (screened out) Market research (screened out) Catering/Restaurant None of the above

Are you… Please select one answer belowMaleFemale

Which of the following ranges includes your age? Please select one answer belowUnder 18 45-5418-24 55-6525-34 66+35-44 Prefer not to say

Which of the following bests describes the region of the UK you live in? PLEASE SELECT ONE ANSWER BELOW

North East (Tyne Tees) East of England (Anglia TV)North West (Granada TV) South East (Meridian TV)Yorkshire & The Humber (Yorkshire) South West (West country TV)East Midlands (Central TV) London (Carlton TV/LWT)West Midlands (Central TV) Wales (Wales/HTV)Scotland (Grampian TV/STV/Border) None of these


Your answers so far indicate your profile fits the needs of this research study. We are undertaking this study twice on the Wednesday, August 8th.

There is a limit to the number of people taking part in each session to ensure that your opinion and views are properly heard. Please indicate which of the following time you would prefer to help us determine which chat sessions are available to you

-3:30 pm – 4:30 pm Wednesday, August 8th-6:30 pm – 7: 30 pm Wednesday, August 8th

Please enter you mobile phone number…

This will only be used to send an SMS to remind you of your session time.


For both Pilots, around three quarters of those screened where eligible

and interested in taking part in this group

As with offline groups, it was important to ensure that the screener was just a

screener, and not a mini survey in its own right. We advise that you should only use

10-12 simple, closed-end questions for a recruitment screener

In our experience, the screener often needs adapting (many come from face-to-face

recruitment templates) or they get very vague screening briefs e,g “Mums with kids

under 2”.

In both cases, the screener needs some adaptation for an online audience and

clarification with the client in terms of any quotas or demographic splits required .


The second stage was to select and invite a selection of eligible Participants to

the group. Having a pool of eligible Participants to choose from allowed us to

balance the group demographically

Stage 2 focus group invitees were given more detailed information on what to

expect in the group and how to access the focus group room. Key information

communicated at this point included:

•Dates and times for the focus group

•A request to arrive a little in advance of the start time to allow time for

everyone to get into the group

•Details on the incentive offered (and rules about how the Participant

would qualify for the incentive)

•Information on how to log on to the group

•Note on limited space – designed to encourage the Participant to

turn up

•Information on who to contact should there be any technical problems

Case Study – Recruitment Stage 2 – Over-recruitment

• We over-recruited by 160% per group (with some adjustments for under and

over responsive demographic sub segments – e.g 15-24 males are typically the

worst responders!)

• In terms of timing, we would advise that screeners are sent out prior to a

weekend for a group scheduled to happen mid week

• For both of these focus groups we scheduled both pilots mid week, with the

stage 1 screener happening before the previous weekend, and stage 2

recruitment on the Monday, with a reminder on the Tuesday for a group on the

Wednesday

• For both case studies, no-one dropped out between the initial recruitment to

the reminder stage

Case Study – Incentive and Timing of Focus Group

• We pitched our incentive at £20 for 90 minutes and this seems to have worked

with our panellists

• It is important to get the timing of the focus group right so that the group is

scheduled at a time that is convenient to panellists.

• In particular, if the Participant demographic is full-time workers, it may be

more appropriate to conduct a group in the evening. Conversely, if the target

is students or an older demographic, a time during the day may well be more

appropriate.

• This also needs to be co-ordinated with a time that the client and moderator is

available and calculated to avoid any major national or sporting events that

may tempt Participants away from the group

• A good practice that we adopted for the first pilot was to ask the panellists we

screened what their preferred time was. Interestingly, feedback did vary by

gender demographic, highlighting that this is an important consideration

when agreeing times for the group

Case Study – Timing of Focus Group

Preferred TimeFemal

eMale Total

3:30 pm – 4:30 pm Wednesday, August 8th 5 14 19

6:30 pm – 7: 30 pm Wednesday, August 8th 13 13 26

Grand Total 18 27 45


FINDINGS

• Attendance for focus group pilot 1 was poor (only 2 people turned up from a

recruited pool of 12)

• Attendance for focus group pilot 2 - 7 took part in the group, indicating that

recruitment rate for the second group had improved on the first.


FINDINGSWhen examining reasons for poor attendance in pilot 1, two factors

seemed to have had an impact on success rates:

1. The group demographic was unreliable – we targeted young 18-24 year olds

– despite lining up 12 people to take part in the group at their preferred time

and reminding them by SMS the day before, it seems that this group were

unreliable. We think that perhaps this age group is less committed to taking

part in groups and may be swayed by last-minute changes of plan. By contrast,

when we mixed up the age groups for pilot 2, we saw much better attendance

rates.

2. The wording in the recruitment invitation about over-recruitment: “There is

limited space availability, so only the first to arrive will be able to

participate” seems to have put some Participants off turning up for the

group.

We amended the invitation for the second pilot, and response seems to

have been better.

Recruitment Recommendations (1)

1. Be clear in your instructions – make sure password and link information that you

send is correct!

2. Don’t put off Participants from turning up to the group by indicating that they

may be rejected (i.e ‘please ensure you turn up at least 20 minutes before the start.

Places will be given on a first come first serve)

3. Make sure that Participants are told that clients may be watching the group – this

is a MRS code of conduct requirement for all focus groups

4. Try and set the scene and give as much detail as possible about what the

experience will be like to manage expectations

Recruitment Recommendations (2)

5. Vary your recruitment strategy to suit your target demographic e.g over-recruit

young people by up to 3x

6. Time the group so that is appropriate for your target audience

7. Consider making the process more personal with a quick phone call reminder

rather than an e-mail / SMS to engender more loyalty to turn up to the group

8. Ensure technical support is available at the moment the group is about to start –

this is the most crucial moment and the moment you are most likely to lose

Participants!

Case Study – Technical

Case Study – Technical FINDINGS

Troubleshooting

• We also set up a helpdesk for participants at: [email protected]

for contacting us more than 30 minutes prior to the groups. We also provided

support via e–mail 30 minutes before the group and during the group. This

worked well in practice, but we would recommend a phone number, available

to panelists in case they have any online difficulties and need to chat to

someone

.

Setting up for the group

• We set up the room for the group a couple of days before the group was due to

take place. This enabled the moderator to go in and familiarise themselves

with the group settings and pre-load the discussion guide. In a live client

situation, we’d suggest taking advantage of this as a way of reassuring clients

and giving moderators some pre-group training

mailto:[email protected]

Case Study – Technical FINDINGS

Starting the groups

• We recommended that the moderator and a “hostess” are present in the focus

group room prior to the start time to welcome in Participants and clients who

turned up early.

• In both groups we piloted, we had people turn up ahead of schedule, and it

was good to have someone there to welcome and encourage them into the

group and keep them talking while others arrived.

• It is advisable to have some general topics lined up to engage Participants who

have turned up early or on time while you wait for the stragglers to log in.

• The hostess also provided a few “rules” of engagement for the group and a

general encouragement to speak up and express opinions. Topics covered

included:

1. Introducing the moderator

2. Telling Participants where the emoticons were

3. Re-assuring Participants that typos didn’t matter

Technical Recommendations

1. Make it as easy as possible for Participants to log in – use existing names and

passwords if you are recruiting from a panel source where possible

2. Set up a helpline for technical issues (preferably a phone and e-mail option in

case the Participant is experiencing internet connection issues)

3. We would encourage the moderator to visit a practice room ahead of the live

group

4. Ensure there is a “hostess” on-hand to welcome participants into the group,

explain the rules and turn away any “extra” Participants or late-comers

5. Have a project manager on-hand to trouble-shoot and call up non-showing

Participants

Case Study – Participant

Engagement

Case Study – Participant Engagement FINDINGS

Moderating

•The software we use allows moderators to get ahead of the game by pre-loading

questions, links to visuals and websites and Participant polls that could be activated

during the course of the group. This means that the moderator does not have to

cause any delays to the group by typing out each question and can be more reactive

to what people are saying.

•Whilst moderating online is different to a face-to-face environment, the key

principles are still the same. As with offline groups, the moderator needs to check

that everyone is fully engaged and responding (it is easier for a Participant to “leave

the room” online that it is offline!) and encourage everyone to give their feedback.

•The moderator therefore needs to keep track of who has said what (having the

transcript in front of you definitely helps here) and allow sufficient time for everyone

to respond (this can take a bit longer online to allow time for typing). However, the

advantage is that Participants can respond at once.

•As with offline groups, the moderator needs to keep an eye on time (in both of our

pilots, time went really quickly and we had to really keep an eye on the pace) and

update Participants on where they are at / how much longer there is to go etc.


Participant Engagement

• In practice, we found that Participants remained well-engaged in the groups

and in most cases, were happy to go beyond the original time set for the group

(we had trouble getting rid of some of them!).

• You can tell when someone logs out of the group (either self-imposed or

through a connection problem) make sure that you have someone on hand to

diagnose connection problems.

• Some Participants can feel intimidated by a face-to-face group full of strangers

and can feel pressurized to respond “in the right way” to the moderator’s

questions. Online, there is more of a sense of anonymity, which can make

some Participants feel less inhibited about expressing their viewpoints. .

Here is an example of a transcript output from our pilot groups….


…………Moderator: Have you ever done any Market research In-person or over the Telephone with an interviewer?

Resp 1: No

Resp 4: I’m always a bit scared of phone interview....not entirely sure why

Resp 3: In person a few times

Resp 5: I have not been interviewed over the phone but have done it when I worked. I think it would good to do one.

Moderator: Do you feel that you are able to give more complete or direct responses online compared to other ways of conducting Market research?

Resp 4: I think some people like the fact that they can take their time over a survey, and that they aren’t being judged by their response, unlike in a telephone situation… where people might think they are

Resp 5: I think it’s good to do surveys online because you have more time to think about what you want to say

Resp 3: yes, I actually prefer doing them online


Moderator: Would you like to take part in these types of online research methods in the future?

Resp 4: blogs?

Resp 1: I like this type of Focus group. It's more interesting and more interactive.

Resp 4: Focus group is interesting…I think you can squeeze more out of people in this sort of social situation

Resp 5: I have never done a focus group before but it is good to talk to other people about research

Moderator: Do you ever feel that you would like to talk directly with the Market Research companies that commission the surveys?

Resp 1: Not really. Online surveys are quite isolated…yes...and ask them why it takes them 10 minutes to tell me I'm not what they want.

Resp 4: Sometimes, I get curious why they ask some of the questions they ask, and I feel like I want to ask them.

Resp 3: Yes, I would

……………………………………….

Participant Engagement Recommendations

1 Where possible, pre-load the discussion guide – this gives you more time to

manage and engage with the responses of the group.

2. As with offline, make sure everybody is engaged!

3. At the start of the group, it can be a bit chaotic! However it does settle down in to

a nice pattern once everyone gets to know each other

4. Keep an eye on the time – it really does go quickly.

5. The moderator should have an opportunity to familiarise themselves with the

software they are using prior to the start of the group

6. Think about how you will want to analyse output from the groups and build in

time if the transcript needs translating or analysing in non-word document format

Overall Considerations in

Applying this method

Types of Application..

Qual Only

• Business Travellers / Time poor individuals who are geographically dispersed.

• Young and Technology Savvy audiences that are already used to conversing in this

way through social networking sites.

• As part of customised communities developed by yourselves and your clients.

Qual and Quant

• Arguably the most powerful application for online focus groups and other online qual

techniques…

• As part of a diary study, where you want to convene participants and glean

richness of insight around their behaviour from the previous week

• Concept development, where qual gives you the depth of understanding and

information, whilst quant gives you the breadth of information and context to

deliver very rounded findings and conclusions

Conclusions

Conclusions

Yes they do, and it is possible for you to use the flexibility of the internet to create some quite cutting edge approaches, leveraging:…

Broad Geographical recruitment – National and International.

Possibility to ‘reconvene’ groups quite cost effectively using online focus

groups

Possible to link in quantitative metrics captured alongside the qualitative

exercise

Are online focus groups any better than traditional offline focus groups?

NO! They are not a substitute, and it is difficult to make the comparison as

they are different. Clearly non verbal communication is of vital importance and something that cannot be captured online well / yet.

However there are some undeniable benefits of using online to capture

insight, and in some instances it is the only available option within cost / time parameters to gather insight into consumer and B2B sub segments.

Do online focus groups provide a good context for engaging with


Any Questions?

Chris DubreuilClient Development Director

Research Now PlcElizabeth House

39, York Road, London SE1 7NQ

T : + 44 (0) 207 7921 2400M : + 44 (0) 7974 170011

E : [email protected]

www.researchnow.co.uk

mailto:[email protected]

http://www.researchnow.co.uk/

research now conducting successful online focus groups november 2007

Documents