appendix b- assessment of robustness of...

Rapid evidence assessment of animal welfare communications | A report for Defra Appendix B - Methodology

1

Appendix B- Assessment of robustness of sources

Part of the Rapid Evidence Assessment methodology involved an assessment of the

robustness of evidence sources against certain key criteria. This appendix presents the

research team’s notes on methodology assessment.


2

Table 1 – Assessment of robustness of sources

Reference Methodology

type/category

Summary of

methodology

Sample details (where applicable) Method quality (including quality

of description and limitations as

recognised by authors)

Neutrality / reflexivity (How clear are

the assumptions / theoretical

perspectives / values that have

shaped the form and output?)

ID: 4, lucid (2010) 'CIWF Food

Business Qualitative Research:

Exploring Quality in relation to

Welfare', Powerpoint

presentation from lucid

Focus group Unspecified qualitative

method. Suspect focus

groups.

No details given. Possibly more in

appendices (not supplied with

source document).

Moderate No contextual factors are discussed,

but generally information not

available, possibly on account of it

being a summary presentation.

ID: 6, Northen, J. R. (2000)

'Quality attributes and quality

cues Effective communication in

the UK meat supply chain', British

Food Journal, Vol. 102, Iss. 3, pp.

230-245

Other: Development

of conceptual

framework

Develops a framework

drawing on previous

research

N/A - reference list provided. High Sources of evidence clearly set out.

ID: 12, Brook Lyndhurst (2010)

'Are labels the answer? Barriers

to buying higher animal welfare

products: A report for Defra',

Defra

Mixed: Literature

review, focus groups,

multi-scalograms

Rapid evidence review,

two rounds of focus

groups, and

stakeholder workshop.

Evidence review covered over 60

sources. Total of 10 focus groups,

in two rounds, with total of 96

participants. First 4 focus groups

recruited to two Defra segments

(PGs and CCs), next 6 recruited by

reported buying behaviour of

welfare foods; both rounds split

between urban and rural locations

and both had loose minimum SEG

quotas. Workshop with

High: triangulated approach with

mixed methods; focus group

recruitment designed to not prime

participants about animal welfare

issues so as to explore views about

animal welfare unprompted

These are not directly discussed, but

the study considered previous

evidence in research design.


3

stakeholders from policy, NGO,

industry, consumer and retail

sectors.

ID: 17, Williams, A., Zraik, D.,

Schiavo, R. & Hatz, D. (2008)

'Raising awareness of Sustainable

Food Issues and Building

Community via the Integrated

Use of New Media with Other

Communication Approaches',

Cases in Public Health

Communication & Marketing,

Vol. 2, pp. 159-177

Case study Reports on the

Sustainable Table

campaign - different

methods used for

evaluating different

elements

N/A Moderate The organisation does have an agenda

which is evident in the background

information provided, but evaluation

data is reported in unbiased terms.

Lead author is an academic.

ID: 20, Reger, B., Wootan, M.B. &

Booth-Butterfield, S. (1999)

'Using Mass Media to Promote

Healthy Eating: A Community-

Based Demonstration Project',

Preventive Medicine, Vol. 29, Iss.

5, Nov 1999, pp. 414-421

Trial Milk sales data from

supermarkets (not all

were able or willing to

share data), in

intervention and

control city. Pre- and

post-intervention

telephone surveys.

Intervention ran for 6

weeks and post-

intervention data was

collected 6 months

later by re-contacting

the baseline

respondents.

Intervention city had a population

of 35,000. Control city had a

population of 34,000. They are 110

miles apart and have distinct media

markets (so no spillover from

media campaign). 366 respondents

in the intervention city and 374 in

the control city completed the

baselins survey. 285 respondents in

intervention city and 258 in control

city completed the follow-up

survey (73% of baseline

respondents) - only these were

analysed. Non milk-drinkers were

excluded from analysis. Unclear

what the timescale was between

High: Appears robust and

triangulated, with data limitations

highlighted. Unclear when the

follow-up survey took place, and

survey questions not provided.

The tone of the article is objective, but

the authors do regularly highlight that

they were involved in the design and

delivery of the campaign. They

explored the characteristics of those

who completed the baseline survey

but not the follow-up: these were

discovered to be significantly different

on three counts - more likely to live in

households using high-fat milk, lower

educational attainment, and more

lower-income and fewer high-income

households. The article outlines issues

that limit the generalisability of the

study: only one intervention

community and one control


4

baseline and follow-up. community; difference in self-reported

milk-drinking behaviour in the two

communities.

ID: 21, Beaudoin, C.E.,

Fernandez, C., Wall, J.L. & Farley,

T.A. (2007) 'Promoting Health

Eating and Physical Activity:

Short-Term Effects of a Mass

Media Campaign', American

Journal of Preventive Medicine,

Vol. 32, Iss. 3, pp. 217-223

Case study The data-collection

instruments came from

the 2004 and 2005

versions of the

Behavioral Risk Factor

Surveillance System

(BRFSS) survey

designed by the CDC,

with locally added

questions specifically

addressing the media

campaign and its

intended effects.

The 2004 surveys,

which served as the

baseline, were

conducted from April to

November, 2004

(n=3137). The 2005

surveys were

conducted during the

campaign from

February to July 2005

(n=1500).

In comparison to the New Orleans

population at the time of the

telephone survey interviews, the

current sample was more likely to

be female (67%, as compared to

53%), white (35%, as compared to

29%), and educated (with 87%

having a high school degree, as

compared to 75%).

High High.


5

ID: 22, Viswanath, K. & Bond, K.

(2007) 'Social Determinants and

Nutrition: Reflections on the Role

of Communication', Journal of

Nutrition Education and

Behaviour, Vol. 39, Iss. 2,

Supplement, Mar-Apr 2007, pp.

S20-S24

Review Literature review -

methodology not

explicitly explained.

62 references High: Large number of references,

but method (e.g. search strategy)

not described.

Hard to assess neutrality - see

comments on quality.

ID: 24, Schimitt, B.H. (1999)

'Experiential Marketing', Journal

of Marketing Management, Vol.

15, Nos 1-3, Apr 1999, pp. 53-67

Review Schmitt puts forth a

conceptual paper

describing the idea of

'Experiential Marketing'

and its component

parts, illustrating this

with examples from a

variety of consumer

brands.

N/A High Angle clearly set out.

ID: 25, Antronette, K.Y. et al.

(2004) 'Population-based

Interventions Engaging

Communities of Color in Healthy

Eating and Active Living: A

Review', Preventing Chronic

Disease, Vol. 1, Iss. 1

Review Qualitative meta-

analysis of the available

literature on

engagement

interventions for

healthy eating and

active living among

ethnic minority

populations. Broad

search for documents

which are then

compared and

23 studies between 1972 and 2000

were considered to meet the

inclusion criteria.

High Very clear.


6

reviewed.

ID: 26, Wardle, J., Rapoport, L.,

Miles, A., Afuape, T. & Duman,

M. (2001) 'Mass education for

obestity prevention: the

penetration of the BBC's 'Fighting

Fat, Fighting Fit' campaign',

Health Education Research,

Theory & Practice, Vol. 16, No. 3,

pp. 343-355

Survey Part of ONS Omnibus

survey, March 1999.

Also worth mentioning

details of original FFFF

campaign - the subject

of this paper. BBC run,

7 week span, via peak

and day-time (BBC1 and

2 TV, local BBC radio),

also website, ceefax,

book, video, Radio

Times and phone lines.

Registration (£2,

included feedback

cards, booklet etc.).

Some campaign testing

beforehand.

Nationally representative Bristish

sample. Stratified probability

sample. n=1894, CAPI. FFFF

targetted at those with higher

prevalence of obesity - the main

relevant groups didn't correspond

with BBC's media reach, but there

was cross-over, namely social

classes 3NM & 3M aged 21-45

(skilled non-manual and manual

groups).

High Genuine, balanced attempt to assess

the FFFF campaign.

ID: 28, Abrahamse, W., Steg, L.,

Vlek, C. & Rothengatter, T. (2005)

'A review of intervention studies

aimed at household energy

conservation', Journal of

Environmental Psychology, Vol.

25, Iss. 3, pp. 273-291

Review Search of various

databases, following up

reference lists.

Inclusion criteria: must

have either a baseline

or a control group, and

focus on households.

38 peer-reviewed studies identified

between 1977 and 2004. Mostly

field experiments using quasi-

experimental designs. Considered

antecedent interventions (which

influence one or more

determinants prior to performance

of behaviour) and consequence

interventions (which take place

after the behaviour e.g. reward).

High They provide a long critique of the

referenced studies' approach (but not

their own). E.g. the authors point out

that where a number of interventions

have been used together, where the

experimental design does not separate

these out between groups, it's difficult

to determine the effect of any single

element.


7

ID: 41, Mosler, H. et al. (2008)

'Deriving Interventions on the

Basis of Factors Influencing

Behavioral Intentions for Waste

Recycling, Composting, and

Reuse in Cuba', Environment and

Behavior, Vol. 40, No. 4, pp. 522-

544

Survey Used a structural

equation model (SEM)

to investigate factors

which influence

behavioural intentions,

based on the TPB, using

household level survey

data on reported

behavioural intentions.

The surveys were

conducted by trained

interviews, took 1 hour.

Assumed representative selection

of households in Santiago de Cuba

for face to face

interview/surveying. Interviewers

were distributed randomly about

the city at intersections, and then

selected households on a fixed

plan. 935 households were

included in the final analysis.

High High.

ID: 42, Ochoa, J.B. et al. (2000)

'Long-Term Reduction in the Cost

of Nutritional Intervention

Achieved by a Nutrition Support

Service', Nutrition in Clinical

Practice, Vol. 15, No. 4, pp. 174-

180

Other: Patient

monitoring

Monitoring provision

and uptake of

recommendations from

Nutrition Support

Service.

Patient numbers: declined from

616 in 1991 to 124 in 1999.

Hard to assess - see comments on

neutrality/reflexivity.

The paper appears to be strongly in

support of the Nutrition Support

Service.

ID: 46, European Commission,

Directorate General for Health

and Consumer Protection (2009)

'Feasibility study on animal

welfare labelling and establishing

a Community Reference Centre

for Animal Protection and

Welfare (Part I: Animal Welfare

Labelling)', European

Commission

Mixed: Review,

working gp, 3 x

survey, 4 x case study

Literature review,

participation at a

working group, 26

interviews, three

surveys, and four case

studies of existing

schemes around the

EU.

Surveys were of general

stakeholders (110), institutions,

and stakeholders of existing

welfare labelling schemes.

Medium to high Appears to be from a neutral

perspective, presenting mainly the

evidence from different strands of the

research in a systematic way and

without an obvious agenda.


8

ID: 47, Government

Communication Network (GCN)

& Central Office of Information

(COI) (2011) 'Evaluating the

financial impact of public sector

marketing communication', COI

Publications,

Other: guide N/A - guide to assessing

impacts of public sector

communications.

N/A N/A N/A

ID: 48, IGD (2007) 'Consumer

Attitudes to Animal Welfare: A

Report for Freedom Food by IGD'

Mixed: survey,

discussion groups,

accompanied shops

Survey of 1,000 British

adults via in-home

interviews. 4 focus

groups. 4 accompanied

shops.

Survey sample 1,000, based on 210

sampling points randomly selected

each week and ACORN used to set

quota controls specific to each

location to ensure all population

sub-sectors are represented at

national and regional level. 4 focus

groups with 8 participants in each,

with shoppers at different life

stages and in different regions.

Not entirely clear where all the

data in the report has come from.

E.g. it contains a segmentation

model but not clear whether this is

based on the survey data. Focus

group quotes are used to describe

the segments, but it doesn't appear

that the focus groups were

recruited to segments.

Not explicitly discussed. The tone of

the report is very positive about higher

welfare foods but there are some

contradictions . The chapter on

Morrisons efforts on welfare is written

in 'marketing speak'.

ID: 49, Commission to the

European parliament (2009)

'Options for animal welfare

labelling and the establishment

of a European Network of

Reference Centres for the

protection and welfare of

animals', Commission of the

European communities

Other: Feasibility

study

Stakeholder

consultation.

Not given; described as "wide

stakeholder consultation"

collecting "the largest number of

contributions on this issue from

stakeholders in the EU and

outside".

High Independent research

ID: 50, Ross, G. et al. (2006) 'The

effectiveness of social marketing

interventions for health

Review Alcohol, tobacco and

drugs: Sample drawn

from reference lists of

54 studies in total. Clearly set out criteria to define

what counts as a social marketing

intervention, and assess the quality

They highlight a number of

methodological limitations.

Conclusions appear rather circular -


9

improvement: what's the

evidence?', Public Health, Vol.

120, Iss. 12, pp. 1133-1139

previous good quality

systematic reviews. 35

reviews identified and

310 individual studies

retrieved. 35 studies

met the inclusion

criteria (see right).

Physical activity:

Searched for systematic

and non-systematic

reviews (17 found) as

well as individual

intervention studies (48

articles found). From

these a further 62

articles were

generated, totalling 110

for assessment - 22 met

inclusion criteria (see

right).

of included studies as reasonable.

The comparisons made in the text

appear rather non-robust, highly

qualitative and very general -

would have liked to see a more

rigorous comparison e.g. QCA. They

do note that comparisons are

difficult, given different

methodologies of studies.

e.g. consumer orientation is identified

as a success factor, but it was also an

inclusion criterion; they state they

didn't compare amount/type of

formative research with outcomes, but

suggest that more extensive research

led to greater impacts.

ID: 51, Toma, L. et al. (2011)

'Consumer and animal welfare. A

comparison between European

Union countries', Appetite, In

press

(Re)analysis A literature review of

work on factors

influencing behavioural

willingness of

consumers to purchase

higher animal welfare

products; the use of

Eurobarometer survey

data from 9 European

countries (consumer

The data were extracted from the

Eurobarometer 66.1 Dataset of

European citizens (of the 25

Member states and 4 candidate

states). The Eurobarometer survey

was carried out by TNS Opinion and

Social, interviewing face-to-

face 29,152 citizens between 6

September and 10 October 2006.

Datasets for nine countries (Great

High High.


10

survey) to create

structural equation

models for each

country expressing the

relative importance of

each observed or latent

variable.

Britain, Finland, Ireland, Lithuania,

Malta, The Netherlands, Poland,

Portugal and Spain).

ID: 52, Passantino, A., Conte, F. &

Russo, M. (2008) 'Animal Welfare

Labelling and Approach of the

European Union: An Overview on

the Current Situation', Journal of

Consumer Protection and Food

Safety, Vol. 3, No. 4, pp. 386-399

Review More a think piece

really, but contains

references to other

research.

19 references. Moderate: Think piece. Five of the references are EC. This is

more a think piece than a review,

promoting a particular approach to

labelling.

ID: 53, Kilbride, A.L. et al. (2011)

'Associations between

membership of farm assurance

and organic certification schemes

and compliance with animal

welfare legislation', Veterinary

Record

(Re)analysis All major schemes

active in 2009 were

invited to participate:

12 farm assurance and

6 organic schemes.

Data on past and

present members was

requested from them,

and matched with

inspection data from

Animal Health which

provided data between

2001-2008, data before

2003 were excluded

due to being

15 schemes provided data for

analysis, not always complete.

Records were provided for 40,939

inspections. Sample of complete

records for analysis was 38,659.

Although the data is incomplete, it

is the most complete data set the

researchers were able to obtain

(and the one that regulators work

with), and of a significant size.

Compliance is measured in the

model as a binary variable, and the

findings probably appear simplistic

when the real picture is likely to be

more complex.

This isn't really discussed. The paper

includes a declaration that they have

no conflicts of interest. It would have

been interesting to see a more detailed

analysis by compliance score or a

rationale for the binary scale used.


11

incomplete. Enterprises

were inspected for

compliance with animal

welfare legislation and

codes in up to 12 areas,

and categorised on a 4-

point scale. For

analysis, the most

severe non-compliance

score was selected and

the outcome variable

was binary with the 4

points split into 2

categories.

ID: 58, Bernués, A., Olaizola, A. &

Corcoran, K. (2003) 'Extrinsic

attributes of red meat as

indictors of quality in Europe: an

application for market

segmentation', Food Quality and

Preference, Vol. 14, Iss. 4, pp.

265-276

Survey Exploratory research

was carried out in all

regions studied using

qualitative focus group

investigation (Corcoran

et al., 2001) and expert

meetings with meat

industry

representatives. These

fed in to the

development of the

quantitative

questionnaire.

Information from a sample

proportionally stratified by

geographical area, place of

residence (town size) and type of

outlet was collected in five

European regions located in

England, Italy, France, Scotland and

Spain. The survey was carried out

between October 1999 and January

2000. Respondents were selected

through judgmental sampling.

High High.

ID: 62, Guilkey, D.K., Hutchinson,

P. & Lance, P. (2006) 'Cost-

Review Outlines statistical

approaches for cost-

N/A High States a strong preference for

experimental designs.


12

Effectiveness Analysis for Health

Communication Programs',

Journal of Health

Communication: International

Perspectives, Vol. 11,

Supplement 2, pp. 47-67

effectiveness analysis.

ID: 76, Scandelius, C. & Cohen, G.

(2011) 'A Life Cycle Stakeholder

Management Framework for

Enhanced Collaboration Between

Stakeholders with Competing

Interests', in Finkbeiner, M. ed.

'Towards Life Cycle Sustainability

Management', Part I, pp. 15-26

Other: Framework Conceptual paper

which uses a

combination of a

literature review with a

case study about Kenco

coffee.

Case study is based on 2 depth

interviews of up to 2 hours with

managers with over 30 years

experience in the packaging

industry.

Medium Medium.

ID: 77, van der Ploeg, J. et al.

(2011) 'Assessing the

effectiveness of environmental

education: mobilizing public

support for Philippine crocodile

conservation', Conversation

Letters, Vol. 4, Iss: 4, pp. 313-

323, Aug/Sep

Trial Questionnaire survey

administered by

students in intervention

and control areas, data

on crocodile mortality

from field observations

obtained on a quarterly

basis. Informal

interviews with

residents and officials

for qualitative insight.

Surveys in 14 villages, split into 4

areas: core area intensively

subjected to project materials,

peripheral area where contact with

materials is less intensive, urban

area with less exposure to

campaign, and control area.

Population over 3,000 in each.

Aimed for 40 respondents per

village, above age 7. Total of 549

respondents.

Measures taken for quality control

include surveys carried out by

students that respondents would

not associate with the government;

simple questions.

The article states that the funding for

the campaign was spent between

2002-08, but contains a graph which

suggests crocodile mortality decreased

significantly between 1998-2000.

There is no before-and-after attitudinal

data, only post-intervention data from

both the intervention and control

areas. Attribution question not

addressed.

ID: 82, Timlett, R.E. & Williams,

I.D. (2008) 'Public participation

and recycling performance in

England: A comparison of tools

Trial Literature reviewing

and case studies of

three behaviour change

projects in Portsmouth

Each project was conducted in a

target area of approximately

10,000 households. Large blocks of

flats were excluded.

High High.


13

for behaviour change',

Resources, Conservation and

Recycling, Vol. 52, Iss. 4, Feb, pp.

622-634

between June 2005 and

December 2006 using

three different

approaches -

doorstepping,

incentives and delivery

of personalised

feedback to residents -

testing the

effectiveness of each

technique in isolation.

ID: 83, Cotter, T. et al. (2010)

'The case for recycling and

adapting anti-tobacco mass

media campaigns', Tobacco

Control, Vol. 19, Iss: 6, pp. 514-

517

Survey Telephone survey of

smokers and recent

quitters in New South

Wales aged 18+, using

list-assigned random

digit dialling, during the

time that two revised

adverts were broadcast

in this area and the two

weeks following.

453 for one advert, 456 for the

other; samples weighted to reflect

the Australian population.

Good sample sizes; unclear why

samples have been weighted to

reflect Australian population when

both the campaign and survey

were done regionally, and are

targeted at smokers rather than

general population. Also, survey

question wording not available for

review.

Makes the case for 'recycling'

advertisements, but based on strong

evidence.

ID: 85, Perl, R. et al. (2010) 'Mass

media campaigns within reach:

effective efforts with limited

resources in Russia's capital city',

Tobacco Control, Vol. 20, Iss. 6,

pp. 439-441

Survey Post-campaign

household survey of

smokers aged 18-45

(primary target

audience).

990 respondents. Moderate: Good sample size, lacks

baseline. Attributes impacts to

campaign by comparing

respondents who were and were

not aware of the campaign.

See comments on impact attribution

under 'quality'.

ID: 86, Otegbeye, M. et al. (2009)

'On achieving the state's

Other: modelling,

scenarios

Model development

and scenarios

Focuses on Northern New Jersey High High.


14

household recycling target: A

case study of Northern New

Jersey, USA', Waste

Management, Vol. 29, Iss. 2, Feb,

pp. 647-654

ID: 87, Mee, N. et al. (2004)

'Effective implementation of a

marketing communications

strategy for kerbside recycling: a

case study from Rushcliffe, UK',

Resources, Conservation and

Recycling, Vol. 42, Iss. 1, Aug, pp.

1-26

Case study Mix of quantitative and

qualitative methods,

including participation

rate measurements,

kerbside tonnages for

recycling, mini recycling

site tonnages, number

of home composter

units sold, monitoring

of the media coverage,

response rates to

promotional tools,

comments/complaints

to the Customer

Services Centre via

customer tracking

software, focus groups

and Citizens’ Panel

surveys, postal and

online questionnaires,

street surveys, and

semi-structured

interviews.

A Citizens Panel of some 1000

residents had been recruited by

RBS in 1999. Quotas were used to

ensure that the panel reflected the

profile of the local population in

terms of age, gender, economic

status and ethnicity. In February

2002, RBC commissioned a survey

of the Citizens Panel concerning a

range of local issues, including

recycling and the role of

communication. Some 926

questionnaires were mailed and a

response rate of 54%was achieved

High High: recognised limitations.


15

ID: 89, Deemer, D.R. & Lobao,

L.M. (2011) 'Public Concern with

Farm-Animal Welfare: Religion,

Politics, and Human

Disadvantage in the Food Sector',

Rural Sociology, Vol. 76, Iss. 2,

June, pp. 167-196

Survey Two mail surveys: Ohio

and nationwide, in

2007. Also includes

literature review.

The Ohio sample was stratified by

two strata: metropolitan counties

and fringe or non-metropolitan

counties, each sample drawn with

1,500 households. Of 2,808

deliverable surveys the response

rate was 35% (therefore about

982). The national sample was a

random sample of 1,677

deliverable mailings, response rate

28.5% (therefore about 478). For

both samples, surveys were

addressed to individuals to ensure

both male and female responses.

Both samples' demographic

characteristics are similar to those

of the state and national adult

population.

Reasonable - subject to usual

assumptions about what is

represented by an answer to a

statement on a Likert scale, but

they have done an extensive

literature reviews and looked at

the kinds of questions used in the

past. The relationships have been

tested repeatedly, in a number of

models.

They state that their approach draws

on the sociological stratification

approach. They also note that the Ohio

survey was conducted "as concern

about state government regulation

was growing among livestock

producers".

ID: 97, Buller, H. & Roe, E. eds

(2010) 'Certifying Quality:

Negotiating and Integrating

Animal Welfare into Food

Assurance', Welfare Quality

Reports, No. 15

Mixed: interviews,

work shadowing,

discussion panels

Cross-comparative

study of UK and France,

studying assurance

schemes for improving

animal welfare,

investigating mechanics

and practices of the

schemes. Interviews

with food chain actors;

work shadowing;

discussion panels.

Key figures interviewed for each of

the three certification schemes in

the two countries; the UK research

team shadowed an on-farm audit

for each of the schemes.

High High level of neutrality.


16

ID: 98, Kauppinen, T., Vesala,

K.M. & Valros, A. (2011) 'Farmer

attitude toward improvement of

animal welfare is correlated with

piglet production parameters',

Livestock Science, In Press

(Re)analysis Using data from a

previous postal

questionnaire in 2010,

combined with data

from the Finnish litter

recording scheme.

Original questionnaire sent to 342

pig farmers in Finland, 137

responses received. When

combined with productivity data,

this resulted in a sample of 124

farmers.

Relatively small sample, but they

acknowledge that this may be

missing the less interested farmers

and interpret the results

accordingly. Language unclear at

times.

The research hypothesis is based on

the TPB framework.

ID: 113, Vanhonacker, F., Van

Poucke, E., Tuyttens, F. &

Verbeke, W. (2010) 'Citizens’

Views on Farm Animal Welfare

and Related Information

Provision: Exploratory Insights

from Flanders, Belgium', Journal

of Agricultural and

Environmental Ethics, Vol. 23,

No. 6, pp. 551–569

Mixed: Survey, focus

groups

Quantitative (survey)

and qualitative (focus

groups) research.

Quantitative: 3 x survey (2000,

2001 & 2002), n=251 (Flanders,

convenience sample, age as only

quota controlled variable).

Qualitative: 2006, n=29 (selection

based on meat consumption profile

and on gender).

High Clear assumptions, likely to be

impartial, objective argument.

ID: 160, Kilchsperger, R., Schmid,

O. and Hecht, J. (2010) 'Animal

welfare initiatives in Europe:

Technical report on grouping

method for animal welfare

standards and initiatives', Econ

Welfare research project,

Research Institute of Organic

Agriculture (FiBL)

Survey Online survey,

conducted by partners

in 8 European countries

- assessed selected

initatives in on-line

questionnaire.

Secondly, used

clustering methodology

to carry out more in-

depth analysis

First part used 33 regulatory

initiatives and 29 non-regulatory

initiatives (e.g. campaigns).

High High.

ID: 176, Zander, K. & Hamm, U.

(2010) 'Consumer preferences

for additional ethical attributes

Survey IDM or Information

Display Matrix was used

- a computer-based

Test persons were acquired at

random and were subsequently

interviewed in front of

High Emphasise the benefits of the IDM

method.


17

of organic food', Food Quality

and Preference, Vol. 21, Iss. 5

method which monitors

the processes of

information acquisiton

preceding choice.

Decisions were made

based on price and on a

series of 'ethical'

attributes, chosen

based on a literature

review, organic farm

surveys, and expert

interviews.

supermarkets or organic food

shops. Only people who buy

organic milk at least occasionally

were asked to participate. 240

interviews in each of the five

countries were conducted - total

1192 observations.

ID: 177, Vanhonacker, F. &

Verbeke, W. (2009) 'Buying

higher welfare poultry products?

Profiling Flemish consumers who

do and do not', Poultry Science,

Vol. 88, Iss. 12, pp. 2702-2711

Survey Cross-sectional survey

data using self-

administered web-

based questionnaires.

Flemish consumers from Northern

Belgium, n = 469, selected using a

nonprobability snowball sampling

procedure, to obtain a sample that

matches with the age distribution

of the adult population.

High Clearly sets out the theory and

assumptions.

ID: 178, Verbeke, W. (2009)

'Stakeholder, citizen and

consumer interests in farm

animal welfare', Animal Welfare,

Vol. 18, Iss. 4, Special Issue, pp.

325-333

Review Literature review. Includes reference list High - would have been good to

see search strategy for literature

review however

High.

ID: 186, Howard, P.H. & Allen, P.

(2010) 'Beyond Organic and Fair

Trade? An Analysis of Ecolabel

Preferences in the United States',

Rural Sociology, Vol. 75, Iss.

Survey Mail survey; examined

ecolabels covering

humane, living wave,

locally grown, small-

scale, and US-grown;

Piloted in California in 2004, 48.3%

response rate. In 2006 it was

mailed to 1,000 randomly selected

households in all 50 states, with

instructions for it to be filled out by

Good sample size and clearly set

out method. Lacks pictures of

actual labels. No analysis of the

oddities in %s flagged earlier.

Question order unlikely to have

There is a section assessing the

limitations of the study, however these

focus mainly on additional aspects e.g.

WTP that could have been studied.

They note that 'family farm' may have


18

2, pp. 244-269 using a forced-choice

paired comparisons

format with 10 pairs of

items to provide all

possible combinations.

Also looked at potential

information sources of

which respondents

were asked to choose

up to 4 (not ranked)

and eight food-system

topics which they were

asked to rate their

interest about.

primary food purchaser in

household, response rate was

50.7% with 276 completed. The

sample was more likely to be older,

white ethnicity, have higher

educational attainment and

income, female (but primary food

purchaser was asked to complete

the survey). Mean age was little

over 50, income was fairly evenly

divided, and rural residents were a

small minority.

been randomised, as this was a

mail survey.

been a better term to use than 'small

scale'.

ID: 187, Onozaka, Y. &

McFadden, D.T. (2011) 'Does

Local Labeling Complement or

Compete with Other Sustainable

Labels? A Conjoint Analysis of

Direct and Joint Values for Fresh

Produce Claim', American Journal

of Agricultural Economics, Vol.

93, Iss. 3, pp. 689-702

Survey National web-based

survey, via a consumer

panel, in 2008.

Questionnaire

development used

focus groups and field

testing. Purchase

intentions elicited

through a conjoint

choice experiment,

looking at claim-origin

combinations; six

blocks with 8 paired

comparisons,

respondents randomly

assigned to a block. The

1,889 primary shopper panelists

invited to participate, of whom

1,268 responded. Eligible sample of

1,052. Sample looks different from

national population, but the target

audience were primary shoppers.

Good. Even includes pictures of

mock-up labels.

High.


19

claims are organic (real

label), fair trade, and

carbon footprint (mock-

up labels), and the

origins are local (within

300 miles), domestic

(specific states) and

imported (specific

counries). Apples and

tomatoes used as

products and

respondents randomly

assigned one of these.

Data analysed within a

random utility

framework.

ID: 190, Low, W. & Davenport, E.

(2009) 'Organizational

Leadership, Ethics and the

Challenges of Marketing Fair and

Ethical Trade', Journal of

Business Ethics, Vol. 86,

Supplement 1, pp. 97-108

Review Critical evaluation of

current trends in ethical

consumerism

particularly as it relates

to fair trade and 'no

sweat' products

N/A Medium-high Medium-high.

ID: 191, Hinrichs, C.C. & Allen, P.

(2008) 'Selective patronage and

social justice: Local food

consumer campaigns in historical

context', Journal of Agricultural

and Environmental Ethics, Vol.

Review Comparison of local

food movements with

other selective

patronage campaigns in

US history around food

purchase.

N/A Medium-high Strong social justice focus/angle.


20

21, Iss. 4, pp. 329-352

ID: 201, Datta, A. (2011) 'Public

engagement in international

animal welfare: Reflections and

cases', ODI, Working Paper 339

Case study 19 projects evaluated

(mostly in sub-saharan

Africa) using reviews of

project literature, two

rounds of telephone

interviews with project

leaders and other team

members. They then

report 7 projects as

case studies.

Looked at 19 projects High High.

ID: 205, Compassion in World

Farming Trust (2003) 'Educating

humane citizens: farm animal

welfare and the curriculum',

Education and Research

Department

(Re)analysis The paper mainly

analyses the national

curriculum (in 2004) for

KS1-4 for the inclusion

of ethical issues around

farming and farm

animals, particularly

welfare.

No research 'sample' of

participants; curriculum and

related textbooks only.

Moderate Relatively low neutrality - goes from

the starting point that improved

education should happen and will lead

to improvements in consumer

awareness and behaviour around

animal welfare.

ID: 206, Farm Animal Welfare

Committee (2011) 'Education,

communication and knowledge

application in relation to farm

animal welfare', Defra

Mixed: working

group, consultation

Report from working

group (est. 2007);

public consultation

(2008/9); written

evidence from 15

organisations and

indivisuals; oral

evidence from industry,

academia, vet groups,

retailers, consumers,

15 organisations/individuals in

public consultation section - list

supplied.

High Report appears balanced and relatively

free from bias, albeit from the

standpoint of higher animal welfare

being desirable.


21

animal welfare

protection

organisations.

ID: 219, Chang, C. (2009)

'Enhancing the Effectiveness of

Antismoking Messages via Self-

Congruent Appeals', Health

Communication , Vol. 24, Iss. 1,

pp. 33-40

Survey Survey (on lifestyle and

personality - to set

context); Responses to

different magazine

layouts; Read a

segment of magazine

with ads; Scale rating

for smoking attitudes

(used mocked up ads).

n=97 High Appears neutral.

ID: 220, Geeroms, N., Verbeke,

W. & Van Kenhove, P. (2008)

'Health advertising to promote

fruit and vegetable intake:

Application of health-related

motive segmentation', Food

Quality and Preference, Vol. 19,

Iss. 5, pp. 481-497

Survey Large-scale online

consumer survey in the

Dutch-speaking part of

Belgium. A link to the

questionnaire was

placed on the website

of a widespread

national newspaper.

Respondents rated 45

items on a 7 point

Likert-scale. Questions

based on desk research

and previous qualitative

research (18 in-depth

interviews and three

group discussions in

which participants were

615 consumers between the ages

of 17 and 77. Women were slightly

overrepresented (59.0%) as were

people with a higher education

(59.4%).

Medium-high Clear.


22

encouraged to talk

about their subjective

perceptions of the

meaning of health.)

ID: 224, Schmid, K.L. et al. (2008)

'Targeting or Tailoring?

Maximizing Resources to Create

Effective Health

Communications', Mark Health

Serv., 28(1): pp. 32–37

Review Appears to be a

literature review, but

no method set out.

Not stated. No reference list

available.

Unclear. Not stated.

ID: 225, Snyder, L.B. (2007)

'Health Communication

Campaigns and Their Impact on

Behavior', Journal of Nutrition

Education and Behavior, Vol. 39,

Iss. 2, Supplement, pp. S32-S40

Review Reviews the

conclusions of meta-

analyses and research

syntheses that examine

the effectiveness of

communications

campaigns and the

conclusions of

evaluations of "recent

prominent national [US]

campaigns", 2 studies

of original data

comparing campaigns,

and scientific literature

on health campaigns.

Contains 64 references. Search strategy not described. Strong focus on informing nutrition

campaigns, so many of the

recommendations are focused on

nutrition specifically.

ID: N1, Friend, K. & Levy, T.

(2002) 'Reductions in smoking

prevalence and cigarette

consumption associated with

(Re)analysis A qualitative review of

media campaigns on

how effective they

were at changes

The analysis was primarily limited

to studies conducted in the US

since Flay's review in 1987.

High: includes search strategy. High.


23

mass-media campaigns', Health

Education Research, Vol. 17, No.

1, pp. 85-98

smoking rates.

ID: N2, IGD for Freedom Food

(2011) 'Shopper attitudes to

animal welfare'

Mixed: Survey,

accompanied shops

Accompanied shops

and online survey, both

carried out in 2010.

18 accompanied shops with people

with some interest in animal

welfare, in variety of retailers in

different parts of the country.

Online survey of representative

sample of 1,000 British meat

shoppers aged 16+.

Survey - note closed questions with

set answer options. Accompanied

shops - relatively small sample and

only covers those with some

interest in the issue.

Report has been produced for

Freedom Food and has generally a

positive tone. Questionnaire given in

full in appendix.

ID: N3, Ratcliffe, J. Et al (1997)

'Cost Effectiveness of a mass

media-led anti-smoking

campaign in Scotland ', Tobacco

Control, Vol. 6, pp. 104-110

Survey Panel survey of 1 in 10

callers to the anti-

smoking telephone

helpline.

970 adults were spoken to, with

data on smoking behaviour one

year later available for 587 panel

members.

High High.

ID: N4, Dentoni et al., (2009)

'The direct and indirect effects of

locally grown on consumers

attitudes to agri-food products',

Agricultural and Resource

Economics Review, Vol. 38, Iss. 3,

Dec 2009, pp. 384–396

Other: Online

experiment

Online experiment in

2008. Respondents

were first asked to

choose up to three

experience attributes

that they inferred when

evaluating a “locally

grown” apple, from a

list of eight suggested

attributes. Second, they

were asked to choose

up to three credence

attributes that they

inferred when

Convenience sample of 60

undergraduate and graduate

students enrolled at Michigan State

University.

Medium Recognises limitations e.g.

convenience sample.


24

evaluating a “locally

grown” product, from a

list of twelve suggested

attributes.

appendix b- assessment of robustness of...

Documents