Research and Development

Projecttitle

Incorporation of conventional animal welfare assessment techniques into organic certification and farming

DEFRAproject code

OF 0314

Department for Environment, Food and Rural AffairsCSG 15

Research and Development

Final Project Report

(Not to be used for LINK projects)

Two hard copies of this form should be returned to:

Research Policy and International Division, Final Reports Unit

DEFRA, Area 301

Cromwell House, Dean Stanley Street, London, SW1P 3JH.

An electronic version should be e-mailed to resreports@defra.gsi.gov.uk

Project title

Incorporation of conventional animal welfare assessment techniques into organic certification and farming

     

DEFRA project code

OF 0314

Contractor organisation and location

Department of Clinical Veterinary ScienceUniversity of Bristol

LangfordBristo

     

Total DEFRA project costs

£ 78,782

Project start date

03/02/03

Project end date

31/03/04

Executive summary (maximum 2 sides A4)To tab in this section press the tab key and the Control key togetherPress the DOWN arrow once to move to the next question.

Providing assurances to consumers on the adherence to certain animal welfare-related standards is an important element of organic and farm assurance schemes. This project has ensured that preliminary welfare assessment protocols developed in a conventional farm assurance system (RSPCA Freedom Food scheme) are available for incorporation into organic (& conventional) certification schemes. The final system (available at www.vetschool.bris.ac.uk/animalwelfare) is an assessment tool that can provide credible (repeatable, valid & feasible) evidence for assessment of compliance with welfare standards in organic and conventional farming systems. For issues identified as causing potential concern the assessor is encouraged to conduct further investigations. This promotes a consistent thorough assessment of relevant resource standards and, where appropriate, management requirements concerning appropriate preventive and corrective action that should be contained within written health plans. Furthermore the assessment tool should enable certification bodies and relevant third parties to monitor the ability of schemes to deliver good welfare outcomes, which is useful for policymakers and consumers wishing to assess the welfare assurance associated with membership of a scheme. Finally it should provide a mechanism for assessing the farm’s own management of health and welfare parameters with their health planning systems which is now a requirement or recommendation of many welfare standards. This should enable farms to both identify their own strengths and weaknesses with respect to welfare and then to monitor any improvements resulting from husbandry changes. This is important as many of the welfare observations also have a significant influence on a farm’s profitability.

Scientific report (maximum 20 sides A4)To tab in this section press the tab key and the Control key togetherPress the DOWN arrow once to move to the next question.

1Introduction 1.1. Aim and objectives

This project aims to consolidate existing knowledge in both the conventional and organic sector. It provides a welfare assessment tool and a framework for inclusion within advisory and certification activities in the organic sector. The aims of this project are:

· To formulate welfare assessment protocols for use in organic dairy and beef cattle, pig and laying hen systems. The protocol builds on existing experience within the conventional livestock sector identified through a large research project conducted at the University of Bristol. The protocol is practical and time efficient, this is achieved by focusing on the aspects of welfare concern identified in recent consultation exercises in the organic sector (Hovi, personal communication).

· To develop a web-based database system that will produce a welfare benchmarking report for organic farmers based on information from on-farm visits by inspectors, advisors or researchers. This report is a vital advisory and training tool for motivating farmers to maintain and improve welfare standards.

· To ensure organic inspectors are appropriately trained to conduct welfare assessments

· To produce an information pack that will allow advisors and veterinary surgeons to disseminate information to farmers on the interpretation of benchmarking reports and management of the aspects causing concern. The information will include best practice knowledge currently being developed in associated studies in both the conventional and organic sector.

· To collaborate with organic sector bodies on exploring the feasibility of incorporation of welfare assessment into certification requirements of organic health and welfare programmes.

1.2Milestones

Milestone

Target date

Title

achieved

03/01

31/3/03

Phase 1 training (welfare assessment update) for inspectors completed

25/3/03

02/01

31/3/03

Basic web database infrastructure completed

31/3/03

01/01

1/5/03

Steering group formed

26/2/03

01/02

1/7/03

Initial protocols developed

1/7/03

01/03

1/9/03

Pilot testing of protocol completed

1/9/03

01/04

1/10/03

Final protocol & guidance notes produced

1/10/03

03/02

1/11/03

Training course for inspectors / advisors produced

1/11/03

02/02

1/12/03

Final version of web database completed

1/12/03

03/03

1/1/04

Phase 2 training (on-farm aspect) for inspectors completed

12/12/03

04/01

1/2/04

Presentation material farmers meeting produced

1/2/04

04/02

1/2/04

Leaflet for farmers & review article for veterinary surgeons produced

1/2/04

04/03

30/4/04

Regional meetings for advisors / veterinary surgeons completed

2/4/04

05/01

30/4/04

Final report (including agreed action with Sector bodies) completed

1.3Background

Assessing farms against compliance with certain welfare criteria is a critical component of certification schemes such as organic certification schemes that provide assurances to consumers. Welfare standards within certification schemes (or legislation) usually attempt to specify what should be provided to the animal. However, evaluation of provisions or resources is a less direct evaluation of welfare than outcomes such as direct observation of the behaviour and physical condition of the animal (Webster et al. 2004). For some husbandry provisions such as provision of a certain space allowance, welfare requirements can be very specific and directly measurable without reference to any further welfare outcomes. These “engineering-based” requirements (Mench, 2003) can be assessed objectively and are used as a statutory control measure in organic livestock production systems in Austria (Barkema, 1999). However, many requirements in organic, farm assurance and legal standards can only be assessed with reference to the animal outcomes. These have been called “performance-based” as they imply that resources provided to the animals should only be considered compliant if they result in certain minimum outcomes. For example, the provision of “adequate nutrition” requires assessment of the body condition of animals in addition to an assessment of the diet.

In addition to their use as a certification (or legislation) assessment tool, animal based assessment techniques can used for research assessments of housing systems and as a management tool to maximise productivity (see reviews by Main et al., 2003 and Johnsen et al, 2001). There has, therefore, been much scientific endeavour in developing and ensuring repeatability of animal-based assessments. For example, a frequently used technique in dairy cattle is the identification and scoring of lameness (Whay, 2002). Leeb et al. (2001) used skin lesions as welfare measures in pigs at specific points within the production system. There has been recent interest into the development of welfare assessment techniques at the group level with two international conferences (Copenhagen in 1999 & Bristol in 2002). The EU is also funding a large-scale project, Welfare Quality (Blokhuis et al, 2003) that aims to produce a European standard for welfare assessment. This research group has developed an animal-based welfare assessment protocol of conventional systems for dairy cattle, pigs and laying hens in a research study assessing the welfare impact of the RSPCA Freedom Food scheme (Whay et al., 2002, Whay et al., 2003, Main et al., 2003). This protocol as also be used for organic dairy cattle by Huxley et al. (2004). The assessment protocols used in these studies forms the basis of the Bristol Welfare Assurance Programme (BWAP).

The organic certification system is based on assuring a certain production system (Hovi et al., 2003). Whilst the system has been carefully chosen and designed over the years to deliver various outcomes, such as minimal residues, minimal environmental impact and good welfare, the certification system does not take any legal responsibility over the outcomes. The desired outcomes are, however, the reason why governments implement policies that support organic farming and consumers buy organic produce. This was highlighted in the European Action Plan for Organic Food and Farming” (2004) which stated that “Organic land management is known to deliver public goods, primarily environmental, but also rural development benefits and in certain respects may also result in improved animal welfare.” The importance of animal welfare to organic farming was recognised in the recent Soil Association-commissioned report on animal welfare, “Batteries not included” (Pye-Smith, 2003). The Soil Association response to the report proposes that certification bodies should take a more proactive role in guaranteeing good animal welfare outcomes on the farm. The response states that: “”Inspections and their reporting will be extended to incorporate a more qualitative animal welfare assessment involving careful observation of temperament and physical condition…”

2Development of the welfare assessment system

2.1Definition of parameters

The animal–based parameters used in the welfare assessment protocol were developed through evaluating and, where appropriate, incorporating protocols that had been used in previous studies (Whay et al., 2002). Protocols were developed for dairy and beef cattle, pigs and laying hens. Most of the parameters were animal-based and could either be assessed on individual animals, observation of groups of animals or from records or estimations of the farmer. The animal-based parameters used in the final assessment protocol were identified and developed under the guidance of a steering group.

The steering group was established during the final meeting of the SEERAD study (26th February 2003, Edinburgh) and further invitations were sent to all UK certification bodies. The steering group consisted of the following: The recommendations of the steering committee are given in section 8 of this report.

Chris Atkinson, SOPA

Stephen Clarkson, OF&G

Sarah Hardy, Soil Association

Marilyn James, Quality Welsh Food Certification

Katy Owen, Organic Food Federation

Roger Unwin, DEFRA/UKROFS

Malla Hovi, University of Reading

Christine Leeb, University of Bristol

David C. J. Main, University of Bristol

Helen R. Whay, University of Bristol

1) First steering group meeting on 2nd July 2003

The aim of this first meeting was to introduce the project, describe other relevant work (FF results & lameness initiative, Duchy Organic study) and give an overview on achieved progress (initial training, prototype web-based database system and evaluation of potential welfare parameters).

2) Second steering group meeting on 23rd September 2003

The specific aim for this steering group meeting was to review the welfare assessment protocols and potential risk assessment procedures (animal based, health plan, qualitative assessment) including their validation and discuss the training requirements for inspectors (on farm assessment).

3) Third steering group meeting on 2nd March 2004

During this final steering group meeting the final version of ‘BWAP’ including figures for intervention guidelines was discussed. Potential mechanisms for incorporating welfare assessment into the certification systems were evaluated and short and long term review and action were planned. The recommendations of the group were formulated and agreed.

In order to ensure appropriate measures were included in the final protocol the steering group was shown a summary evaluation for each potential parameter against a variety of criteria. The steering group were asked to consider which parameters should remain in the protocol after considering the criteria discussed below. As can be seen from the example for dairy cattle (Table 1), the group used various reasons for including or excluding parameters. Even though this decision was based, where possible, on factual information, the decisions of the group should not be considered as definitive and an on-going evaluation of existing and novel parameters is important.

a) Relevance to organic standards or legal requirements. The relevant “performance-based” standards were identified within the Compendium of UK Organic Standards (DEFRA, 2004) and the Welfare of Farmed Animals (England) Regulations 2000 (HMSO, 2000). These were standards or requirements that should be assessed by evaluating animal-based measures. For example, a requirement to provide “appropriate nutrition” would require an assessment of the body condition of the animals within a group in addition to the assessment of the diet provided. Obviously some “engineering” standards (e.g. space allowance) that define resource requirements more precisely are important for achieving desirable outcomes (animal-based measures). However, these were not considered as part of this evaluation as the assessment of these standards requires examination of the resource only.

Some “performance-based” standards were not specific and could apply to several potential animal welfare relevant parameters. For example, the Welfare of Farmed Animals (England) Regulations 2000 (HMSO, 2000) requires that “Owners and keepers of animals shall take all reasonable steps (a) to ensure the welfare of the animals under their care; and (b) to ensure that the animals are not caused any unnecessary pain, suffering or injury”. The Compendium of UK Standards (DEFRA, 2004) also requires that “8.1.1 Housing conditions for livestock must meet the livestock’s biological and ethological needs”. Hence, if there is no specific standard designed to address a particular welfare concern such as mastitis and lameness then the occurrence of these conditions is relevant for assessing these general requirements.

b) Prevalence of the condition. The relative prevalence of various welfare concerns was considered to be a relevant criteria as a relatively uncommon problem should have less priority than a more common condition. Although of course severity of the condition should also be considered. For dairy cows the prevalence of most parameters was known from previous work (Whay et al, 2003), the equivalent data for pigs and laying hens was not available fully at the time of the study.

c) Reliability. The evaluation of reliability of potential parameters was based on scientific studies, where available, and previous practical experience of the measures. Various studies have examined the repeatability of specific animal-based measures. These have included lameness in dairy cattle (Winkler et al, 2000) and pigs (Main et al., 2000). In a review of several animal-based measures, Winckler et al. (2003) used a subjective assessment of reliability to compare different parameters. The evaluations used in our study have been presented as both within observer (e.g. different times of the day/year) and between observers. Repeatability was assessed on a scale of 0-3, where 0 was considered not repeatable, 1 was poor repeatability (i.e. intensive training/instructions necessary or no scientific evidence for repeatability), 2 was good repeatability (i.e. some training/instructions necessary) and 3 was excellent repeatability (i.e. established/published method)

d) Feasibility. The feasibility in terms of time necessary to assess per animal/group of animals was known from previous work.

e) Importance. The importance of a parameter to a farm’s profitability and the animal were subjectively assessed on a scale from 0-10 (0: not important at all – 10: extremely important). Profitability was based on evaluations of knowledge of conditions and published costs of clinical disease (Kossaibati and Esslemont, 1997 and Bennett et al., 1999). Importance to the animal was based on previous knowledge and published work, such as consultation exercises (Whay et al, 2002). The estimation of importance for the animal was influenced by the likely subjective experience of the animal. For example, reduced milk yield is of minimal importance to the cow as she is unlikely to perceive such an event, even though reduced milk yield may be as a result of another welfare concern.

Table 1 Evaluation of the relevance to standards, occurrence of the condition, repeatability, time taken and importance to the farmer and the animal of 7 potential animal-based parameters prior to inclusion within the final protocol.

Examples of potential dairy cow parameters

Specific “performance” based standards that could be assessed by quantification of the relevant parameter

Average UK occurrence

(Whay, et al, 2003)

Subjective assessment of reliability

(Minimum 0 – Maximum 3)

Estimate of time taken

Subjective assessment of importance

(Minimum 0 – Maximum 10)

Inclusion within final protocol plus reasons

Compendium of UK organic standards (2004)

Welfare of Farmed Animals (England) Schedule 1 (2000)

Within observer

Between observers

Importance for farm profitability

Importance for the animal

Thin cows

(% of cows with less than Body condition score 2

“ 4.1 Feed is intended to ensure quality production rather than maximising production, while

meeting the nutritional requirements of the livestock at various stages of their

development. “

“ 22. Animals shall be fed a wholesome diet which is appropriate to their age and species and which is fed to them in sufficient quantity to maintain them in good health, to satisfy their nutritional needs and to promote a positive state of well-being. “

17 %

2.5

2.5

6 minutes / sample of 20 cows

8

8.5

Yes – important for specific standards

Swollen hocks

(% of cows with any swollen hocks)

“ 8.3.6 The housing must be provided with a comfortable, clean and dry laying/rest area ….  “

“ 11. Materials used for the construction of accommodation ….. shall not be harmful to them… “

33 %

2.5

2.5

6 minutes / sample of 20 cows

7.5

8.5

Yes – important for specific standards

Dirty udder

(% of cows with any evidence of dirty udders)

“ 8.2.5 Housing, pens, equipment and utensils must be properly cleaned and disinfected to

prevent cross-infection and the build-up of disease-carrying organisms. “

“ 4. Where any animals …. are kept in a building they shall be kept on, or have access at all times to, a lying area which either has well-maintained dry bedding or is well-drained. “

21 %

2.5

2.5

4 minutes / sample of 20 cows

7.5

4

Yes – important for specific standards

Mastitis

(Number of recorded cases / 100 cows / year)

 

 

No specific standards but general requirements, such as ensuring “welfare of animals under their care”, meeting “biological and ethological needs”, recording medicinal treatments and providing appropriate care if ill or injured, may be relevant.

29 cases/100 cows/year

2.5

3

10 minutes to examine records

9

5

Yes – important for farm & quick to assess

Milk yield

(Average annual milk production in litres)

7300 l

2.5

3

0.5 minutes for verbal question

9.5

3

No – minimal direct importance to animal

Locomotion scoring

(% of cows with a detectable lameness)

22 %

1.5

1.5

3 hours / 100 cow herd

9

9.5

Yes – but may not be sufficient time to estimate whole herd prevalence

Dull coat

(% of cows with a dull / non-shiny coat)

7 %

2.5

2

4 minutes / sample of 20 cows

4

4.5

No – minimal importance to farm and animal

2. 2Development of assessment system

The outcome of this evaluation process was that the steering group selected and approved a list of parameters for each species (Table 2).

Table 2 List of the animal-based parameters that were included in the final version of the Bristol Welfare Assessment Protocol

Cattle

Pigs

Laying hens

Thin cows

Fat cows

Dirty side

Dirty hind limb

Dirty udder

Skin lesions

Swollen hocks

Claw overgrowth

Flight distance

Rising restriction

Animal appears obviously sick/dull

Rumen bloated

Eye abnormalities/nasal discharge

Coughing

Skin irritation (hairloss, scratching,)

Abraded/ulcerated hock

Lameness

Thin animal

Fat sow

Dirty side/hindquarter/

Ocular/nasal discharge

Head/neck/side lesion

Genital and hindquarter lesions

Tail lesion

Bursas on limb

Time to return to observer

Animal appears obviously sick/dull

Signs of scouring

Coughing/sneezing/dyspnoea

Lameness

Sunburn

Skin irritation (mange, lice, scratching)

Abnormal oral behaviour (other than straw, eat, drink)

Playing (running/toy/straw)

Other (e.g. abscesses, udder, shoulder sore, haematoma, rectal prolepses)

Poor comb colour

Poor beak condition

Thin birds

Fractures

Red mites/lice

Soiling of feathers

Feather damage

Feather loss

Trauma/injury

Limb problems/toes

Flight distance

Animal appears obviously sick/dull

Thermal discomfort

Respiratory problems

Poor quality of litter

Uneven grass wear in range

Evidence of calcium deposits in eggs

The format of the assessment system was carefully designed so that it was easy to use on a farm (e.g. include paper versions), could be consistently applied (e.g. full guidance notes) and that the assessment could integrate with various certification systems. Each parameter was included on either the animal observation or record data collection forms. The forms were formatted so that once printed they printed onto single A4 pages that could be used on a farm. An assessor manual was produced for each species (cattle, pigs and laying hens). The manual contained an appropriate definition of the parameter with pictures as required and a description of how to assess the parameter. Guidance notes also gave an indication of the appropriate number of animals that should be examined. The assessment forms included conversion of the observations and records into standard units, i.e. prevalence % observed and incidence no of cases / 100 animals / year. A separate “farmer significance” report gave a short explanation of the significance (both welfare and profitability) of the parameters.

All the protocols, assessment forms and examples of completed forms are available from the following website : http://www.vetschool.bris.ac.uk/animalwelfare.

2. 3Intervention guidelines

Initial intervention guidelines were also proposed for each parameter. These guidelines, which could be modified by a scheme, were designed to guide further investigations by the assessor. If a welfare parameter would exceed the pre-defined guideline, the assessor would be encouraged to further investigate the issue. Exceeding an intervention guideline would not necessarily mean that the farm was non-compliant with organic, farm assurance or legal standards. However, it would provide evidence that certain resources provided to the animal might not be adequate. Hence, further investigations would include both a detailed assessment of compliance with the relevant “performance” based standard and an assessment of the management action being taken by the farm.

The guidelines used in the provisional cattle protocols were derived from another study (Whay et al, 2003). In this study dairy cattle welfare experts and veterinary surgeons were asked to consider for each parameter at what herd incidence level “action should be taken to improve the situation”. An initial arbitrary herd incidence level at which 75% of experts agreed was defined as the intervention guideline. These intervention guidelines were slightly adapted for dairy cows and beef animals and were discussed and agreed during the steering group meetings. Intervention guidelines for pigs and laying hens were based on a series of visits to farms (Whay, personal observations) and relevant literature. It is important to note that different schemes might choose to set their own intervention guidelines in accordance with their own scheme goals. The authors would recommend that the guidelines are periodically reviewed in the light of assessment results.

2. 4Further investigations: Compliance checklist

For measures above the intervention guideline, the assessor would be encouraged to re-examine specific resources defined within a “compliance checklist”. This compliance checklist was generated from the Compendium of UK Organic Standards (DEFRA, 2004) and the Welfare of Farmed Animals (England) Regulations 2000 (HMSO, 2000). Compliance with specific standards is particularly important for those welfare measures that are associated with specific outcomes (e.g. body condition with diet or injuries with damaged fittings). A checklist gives a short series of relevant questions, such as “Does feed meet legal requirements ?” for each parameter that relate to a specific organic or legal requirement. If the assessor then identifies non-compliance with the specific standard, the assessor would be able to report a non-compliance in line with the certifier’s existing procedures. The results of the animal-based assessment would be included as part of the certification body’s normal reporting systems.

2. 5Further investigations: Health plan assessment

An assessment of action taken by the farmer in response to a particular concern is detailed on the “health plan form”. Again, for measures that are above the intervention guideline, the assessor would be asked to conduct further investigations. In particular, the assessor would examine the farmer’s awareness of the issue and whether appropriate investigations and actions have been taken. The health plan form also allows the assessor to record information on the normal management system for common welfare concerns. For instance, for issues like lameness in dairy cows, the assessor would examine the prevention and treatment protocols present on the farm and examine the farm’s monitoring / recording systems. The assessor would also be encouraged to examine general aspects of farm management, such as use of advice and the regular review of management practices. If the assessor identified concerns with the routine management or a farm’s response to a particular welfare concern, the evidence collated in the health plan form would also be used to generate appropriate non-compliance. As with the standards compliance, this evidence would be included with the assessors normal reporting systems. An example of a completed health plan is shown in BWAP web site.

2.6Reporting system

An optional, but potentially valuable, component of the system is the capability to benchmark results between farms. Providing information to the farmer of their own performance with respect to their peers is a powerful motivational tool that is an important benefit from welfare assessment (see later). The bench-marking system operates via a web-linked database of assessment results enabling establishment of “norms” for systems being examined. For each farm assessed a report would be generated in printable, web page or email version. The report would give the summary of the results and provide information on their performance with respect to their peers.

A model web-based database was developed for the first steering group meeting. This was further refined in line with the developed protocols. The web design was subcontracted to Dr. Neil Ambrose of Smart Tuna.

2.7Pilot testing of the system

The parameters were pilot tested for each species on a total of 25 farms (6 beef, 13, dairy, 2 pig, 4 laying hen) in England, Scotland and Wales in order to assess the repeatability and feasibility of the assessments during normal inspections or independently from that. At the same time photographs were taken for defining the parameters in the assessment manual.

Table 3 : Summary of pilot testing

Species

Number of farms visited in :

England Wales Scotland

During organic CB inspections

Including data collection

Total

Dairy cows

4

2

0

3

4

6

Beef cattle

5

1

7

3

8

13

Pigs

2

0

0

0

1

2

Laying hens

2

1

1

0

2

4

Total

13

4

8

6

15

25

2.8Repeatability exercise

A model for evaluation of the repeatability of parameters and/or the consistency of individual assessors was developed. After initial training (theoretical and practical) of 10 assessors in the morning and afternoon, all of them assessed the same 10 animals (dairy cows and finishing pigs) independently. The repeatability results were used to identify poor repeatability of some parameters and poor performance of individual assessors. Data were described as percentage of correct assessments compared to most common finding (mode) (Figure 1 and 2). Since the parameters are assessed as being either present or absent, a 50% agreement would be expected by random chance. For those parameters showing lower agreement, the definition and illustration of parameters within the guidance notes was improved.

Figure 1 The proportion of assessments agreeing with the mode (i.e. most common) response for 8 parameters assessed in dairy cattle conducted on 10 cattle by 10 assessors (due to some missing data the number of assessments ranged between 94 and 100).

50%

60%

70%

80%

90%

100%

Thin

Fat

Dirty side

Dirty hindlimb

Dirty udder

Skin lesion

Swollen hocks

Overgrown claws

Figure 2 The proportion of assessments agreeing with the mode (i.e. most common) response for 7 parameters assessed in finishing pigs conducted on 10 pigs by 10 assessors (due to some missing data the number of assessments ranged between 96 and 100).

50%

60%

70%

80%

90%

100%

Thin pigs

Dirty side

Nasal discharge

Head/neck lesions

Hindquarter lesions

Tail lesion

Swollen hock / bursae

Further analysis of kappa values for individual assessors was used as an indicator of reduced repeatability by individual assessors. An example of this process is shown in table 4 for the parameter “Dirty Sides”. For this parameter, which had relatively low overall repeatability (78%), certain assessors 1, 5, 2, 6 and 7 demonstrated significant (p < 0.05 & kappa > 0.6) agreement with the mode response. However the poor repeatability within the remaining assessors indicated a need for further training. The authors recommend that this procedure is used as an ongoing system for the training and monitoring of assessors.

Table 4 : Repeatability for the parameter “Dirty side” observed on 10 dairy cattle by 10 assessors.

Assessor identification

Agreement with mode response

Kappa value (measure of repeatability)

P value

1

100%

1.000

0.001

5

91%

0.621

0.026

2

91%

0.621

0.026

6

90%

0.615

0.035

7

89%

0.609

0.047

4

78%

0.526

0.073

8

70%

0.400

0.114

9

64%

0.313

0.154

3

60%

0.200

0.292

10

50%

0.138

0.389

Total (± s.d.)

78 %(± 17%)

0.504 (± 0.252)

3Dissemination

3.1Training

Training was conducted in 2 phases as planned in the original proposal and offered to all organic certification bodies.

Workshop 1: The objective of this workshop was to discuss the animal welfare assessment requirements within current legislation and codes of practice and introduce the principles of animal-based welfare assessment within certification systems. These workshops were held on three dates : 20th, 24th, and 25th March 2003. There were 27 participants (18 SA, 3 DEFRA, 2 DARD, 2 Biodynamic, 1 OF&G, 1 Independent).

Workshop 2: The second phase (one 5 day course) covered the detailed aspects of animal-based assessment and its application to a dairy farm, laying hen and pig unit). This workshop was held over 5 days (8th -12th December) and was delivered to 9 participants (4 SA, 2 OF&G, 1 SOPA, 1 QWFC, 1 OFF).

3.2Regional meetings

Five regional meetings were organised to disseminate information about the project to organic farmers advisors and policy makers. In particular the meetings were designed to explain animal based assessment techniques and discuss its application to organic certification. The meetings were held at the following locations : Dorchester 25th February, Aberystwyth 10th March, Dumfries and Galloway 30th March, Perth 31st March and Langford 2nd April

3.3 Web-site

A web-site (http://www.vetschool.bris.ac.uk/animalwelfare) was constructed that contained the assessment protocol, guidance notes and assessment forms. It includes all the supporting information such as explanations for farmers, examples of completed forms and example of compliance checklists. This information is freely available to anybody. A registration system enables the University of Bristol to record the people and organisations that wish to use the protocols. This is slightly different format to the original intended dissemination mechanism (leaflets) but this was seen as a much more powerful tool.

3.4Other communicated outputs

· Final Meeting of SEERAD study (26th February 2003, Edinburgh)

· ADAS Workshop on Health and welfare in organic beef and sheep production. (10th April 2003 Redesdale) “Frameworks for improving the welfare of organic stocks”

· ‘The good life of calves in organic dairy herds’ Workshop (8. -9th May, 2003 Foulum, Denmark)

· Organic farming magazine

· SAFO conference, 25th-27th March 2003, Witzenhausen, Germany

4Discussion

4.1 Credibility of the assessment system

The development of the welfare assessment system was guided by the goal of producing a valid, reliable and feasible assessment tool that could be used within existing organic or conventional certification schemes. It is important to consider the extent to which the system achieves this objective.

Validity

This system aimed to be valid in as much as it should include welfare-relevant parameters. Since welfare can be considered to be influenced by many components (Webster et al., 2004), it would be inappropriate to consider each parameter as an “indicator” of an animal’s overall welfare state. No single parameter or “indicator” is ever likely to reflect all of these components (Mason & Mendl, 1993). Furthermore the BWAP does not integrate different parameters into a single score. This avoids the inevitable value judgements in weighting different components (Spoolder et al. 2003). For assessment of compliance with either certification or legislative requirements, the critical concept is one of compliance or not. The authors advocate that the level of a certain parameter is not used directly to determine compliance or not. Assessors using the BWAP would be encouraged to further investigate compliance with either the relevant resource standards or the management / health plan requirements once the measure exceeds the intervention guideline.

The parameters were based on those that experts considered to be valid welfare-relevant parameters (Whay et al., 2002). This approach is obviously likely to be limited as it relies upon a subjective interpretation using existing (and often limited) knowledge. However, it is important to recognise that this need not be the definitive list of parameters. The measures should be added to or amended as new knowledge becomes available. Indeed research work into assessing the welfare significance of these parameters should be encouraged. In particular, the parameters tend to concentrate on welfare in terms of the lack of a certain condition such as presence of injuries or dirty coat condition rather than directly observing positive aspects. The facility to record subjective general impressions was considered by the steering group as a mechanism for assessors to include (and report back to farmers) comments on more general positive aspects such as integrity and mood. Further work in defining positive parameters should be encouraged.

It is important to recognise that animal-based parameters should not replace more precisely defined and valid resource standards. For example, this system does not directly assess water provision as more obvious methodology is to assess the availability of water rather than to construct an animal-based parameter of thirst. Similarly enabling behavioural choice may be better assessed by the diversity of the environment. However, the actual use of the elements of the diverse environment is an animal-based parameter that could be used to verify that the animals have been provided with an environment that they value.

Reliability

The reliability of parameters is an important consideration as most measures inevitably involve some degree of subjectivity. The most critical aspect of assessment is the degree of consistency of assessment between and within assessors. Some degree of assessment error is inevitable. The critical issues are how much error is acceptable for certification purposes and the extent to which the repeatability of such assessments is monitored throughout its application. One should consider the consequences of an assessment error. As previously discussed the authors would not advocate that a certain level of a parameter should constitute a non-compliance in itself rather it would instigate further investigation. Since further steps are required to demonstrate a non-compliance it seems reasonable to allow a certain amount of assessment error. Indeed one could argue that the risk of some farmers having to institute corrective action even though their animal-based result could be an overestimate is a price worth paying to ensure those farms that are underestimated also take corrective action. However, it is important to strive for consistency for the credibility of the system so the authors would advocate some sort of monitoring exercise along the lines of the exercise presented in this paper (table 4). This should be instigated at training (until a minimum is achieved) and at regular (e.g. annual) intervals.

Feasibility

As already stated the system was designed to be feasible within existing certification system. So for example the system needs to operate both as paper-based on a farm and electronically for inclusion within a database. The duration of current certification visits varies between schemes but is usually between 2 and 8 hours. The authors believe that the system could replace some elements of existing assessments and extend others so it is difficult to predict the additional time required but it is likely to be between a 30 minute to 2 hour extension for most UK beef, dairy, pig or laying hen systems. Obviously research exercises (e.g. evaluating novel husbandry systems) would not be limited by these constraints, as the assessments would be able to assess the animals in greater detail.

4.2 Incorporation of the assessment system into certification systems

This system could generate three key benefits (certification of individual farms, monitoring the effectiveness of certification bodies and to provide benchmarking information to individual farmers) when incorporated into part of a certification (or enforcement) system:

a) Certification benefits. The assessment system provides credible objective evidence for certification bodies or enforcement agencies wishing to assess compliance with animal welfare related standards or legislation. A formal assessment of outcomes is relevant when assessing “goal orientated” or “performance based” standards that define the provision such as diet or housing in terms of what is “adequate”, “necessary”, “sufficient” or “appropriate”.

Inspectors for organic or conventional certification schemes can be very experienced and skilful stockpeople that have a significant knowledge of appropriate welfare standards. This system formalises this assessment and encourages them to report animal-based observations. Seppanen and Helenius, 2004 argued that organic inspectors should go beyond assessing compliance and had a valuable role to play in providing advice. However, it is important to recognise that assessors working within an accredited certification body that complies with the European standard, EN 45011, must not provide “prescriptive advice or consultancy as part of an evaluation” (EA, 1999). The author’s believe that a BWAP assessment does not constitute advice as it aims to identify problems (e.g. failure to comply with standards) rather than provide advice on potential farm specific husbandry solutions. This could and should be provided by existing advisory systems such as the attending veterinary surgeon. Furthermore failure to promote the recording and reporting of perceived/observed welfare problems to the certification bodies, may actually encourage advisory activities during inspection.

b) Monitoring / Surveillance tool benefits. The assessment tool can also be used on a sampling basis to monitor the effectiveness of the assurance process . For example third party / monitoring organisations could assess the overall effectiveness of the certification system in delivering an assurance on welfare to consumers. It could also be used as a management tool by certification bodies to improve their own performance by identifying strengths and weaknesses in existing procedures. The results from welfare assessment could also enable schemes to identify particular areas of concern, this would enables the standard setting bodies to modify and generate new standards for dealing with specific problems areas.

c) Benchmarking / Management benefits. By providing information on animal health and welfare performance with respect their peers, farmers and their advisors would be able to identify farm specific strengths and weaknesses. The attitudes and motivations of stockpersons and farmers have a critical influence on animal welfare. The benchmarking report is a mechanism for educating and encouraging staff with respect to welfare performance and it is a powerful motivation for improved welfare. For example, a recent assessment of cattle welfare (Whay et al., 2003) has shown that a mean of 22% of dairy cattle are lame at any one time. However, when asked how many cattle are lame the stockmen were only aware of 6%. Informing stockpersons on the actual levels and showing them how to assess lameness accurately is an essential first step for herd control measures. It is also established that benchmarking of welfare assessment results can have a very positive encouragement effect. (Huxley et al., 2003).

However, the extent to which each potential benefit is achieved depends upon the number and type of farms assessed (see table 5). For example if all farms are assessed by the certification body at initial and surveillance visits then the full benefits would be received. If members were assessed on their initial visit prior to joining a scheme then the results would not be useful for monitoring the “welfare performance” of members within a scheme. However, the certification and benchmarking benefits would still be of benefit to the new members. If farms were assessed after some form of risk assessment (e.g. after a history of certification problems) then the assessment would be useful for certification and benchmarking but of limited value for monitoring performance of the scheme. However, a random sample would generate information about the scheme “welfare performance” but it may not be equitable for it to be used as a certification tool as only sampled farms would be monitored in this way. Hence the 3 benefits (certification, monitoring and benchmarking) that would be generated would depend on which farms are assessed.

Table 5. : Potential benefits of Bristol Welfare Assurance Programme associated with different types of visit.

Visit type

Potential benefits ( - minimal, + some, ++ medium, +++ maximum )

 

Certification tool

Monitoring tool

Benchmarking tool

Results used to inform certification decisions

Results used to monitor (internally or externally) performance of certification body

Report identifying strengths and weaknesses given to farmer

All farms at initial & surveillance visit

+++

Complement existing assessment

+++

Complete assessment of performance

+++

All farms receive report

All farms at initial visit only

++

Additional threshold for new members

-

Results do not reflect farms in the scheme

+

Limited to first visit only

Farms identified as higher risk

e.g. history of previous problems

++

Useful for higher risk population

-

Only data from high risk farms

++

Useful for higher risk population

Sample of farms in scheme

-

Not usable as only sample

++

Useful assessment of performance

+

Only sampled farms

5Conclusions

The aims of the Bristol Welfare Assurance Programme are to deliver :

· Standardised assessment of welfare outcomes that is valid (relevant to welfare), feasible (can be incorporated into existing assessments) and repeatable (generates consistent results).

Standardised assessment of health and welfare planning provides evidence of farm specific preventive and corrective action

This system achieves this using the following elements :

· Assessor manual providing guidance on assessment

· Data collection system that can be used on farm

· Optional web-based data entry system that produces a benchmark report

· Farmer significance manual providing an interpretation of the results for the farmer

· Compliance checklist outlining the links with welfare standards and legislation

This system can generate three key benefits when incorporated into a certification system:

· Certification benefits. The assessment system provides credible objective evidence for certification bodies or enforcement agencies wishing to assess compliance with animal welfare related standards or legislation. A formal assessment of outcomes is relevant when assessing “goal orientated” or “performance based” standards that define the provision such as diet or housing in terms of what is “adequate”, “necessary”, “sufficient” or “appropriate”.

· Monitoring / Surveillance tool benefits. The assessment tool can also be used on a sampling basis to monitor the effectiveness of the assurance process. For example it can be used as a management tool by certification bodies to improve their own performance by identifying strengths and weaknesses in existing procedures. The data collected could also guide future development of the organic standards by quantifying the impact of organic standards & inspection on animal welfare.

· Benchmarking / Management benefits. By providing information on animal health and welfare performance with respect their peers, farmers and their advisors would be able to identify farm specific strengths and weaknesses.

The extent to which each potential benefit is achieved depends upon the level of application (i.e. the number and type of farms assessed. For example if all farms are assessed by the certification body at initial and surveillance visits then all 3 benefits would be received. Conversely if the assessment were used on a sampling basis by the certification body to monitor their own performance then the benchmarking would be limited to the sampled farms.

The certification and monitoring benefits of this system are the most important issues for ACOS, DEFRA & certification bodies. The benchmarking / management benefits are important for promoting the system to farmers. The system as proposed helps certification bodies and DEFRA collect objective evidence of compliance with existing organic standards and welfare legislation. This is line with DEFRA responsibilities as outlined in the Compendium of UK Organic standards Provision 9 paragraph 6 : “…DEFRA shall (a) ensure that the inspections carried out by the inspection body are objective [and] (b) verify the effectiveness of its inspection”(DEFRA, 2004). Furthermore, collation of evidence during a farm visit is an important element of compliance within EN 45011.

Most inspectors do already make some evaluation of the animals during visits to livestock units. For example a good assessor assessing compliance with adequacy of an organic diet would observe the animals and record a sample of body condition scores. This system, however, allows the assessor to also use other standardised assessments of welfare-relevant conditions such as injuries or lameness. The goal, however, of this system is to achieve a more repeatable and transparent assessment. In essence this system reflects how animal welfare standards could (and arguably should) have been evaluated in the past.

The system has been fully developed and is freely available on the web (www.vetschool.bris.ac.uk/animalwelfare) . Several organic certification bodies were positively involved in the development phase (see Steering committee recommendations) and Soil Association have starting using the system. After a meeting between University of Bristol, RSPCA & DEFRA it was decided to investigate the establishment of a Bristol Welfare Assessment User Club. This club would maintain the system and provide training and technical resources to users. Although the methodology and the forms will be freely available on the web potential users would be encouraged to contribute to & receive certain benefits from the Club.

6Recommendations of the Steering Committee

These recommendations were formulated by the following individuals on the project Steering Committee (they do not necessarily reflect the views of their relevant organisations) :

Sarah Hardy

Soil Association

Chris Atkinson

SOPA

Christine Leeb

University of Bristol

Barbara Messenger

Soil Association

Katie Owen

Organic Food Federation

Becky Whay

University of Bristol

Stephen Clarkson

OF&G

Roger Unwin

DEFRA

David Main

University of Bristol

Marilyn James

QWFC

Malla Hovi

University of Reading

Background

1. The Steering Committee (SC) recognises that an important goal of organic farming is to deliver high animal welfare standards. “The development and management of organic livestock systems requires special care in nurturing positive health and vitality, ensuring the proper control of disease and the encouragement of positive animal welfare.” (ACOS, 2003). A transparent system that could demonstrate that these goals have been achieved would be highly desirable.

2. The Bristol Welfare Assurance Programme (BWAP), which has been funded by DEFRA & RSPCA, has built upon the animal based welfare assessment protocols used for the evaluation of the RSPCA Freedom Food scheme (Whay et al, 2003). It has addressed a desire for quantifiable animal welfare assessment in organic certification and farming as highlighted by the recent SEERAD funded project (Hovi et al, 2003).

3. Animal welfare assessment schemes are also being developed in other European countries. For example, the “Animal Needs Index” is an additional legal requirement for organic farms in Austria. The importance of animal-based assessment schemes has been recognised by the recent approval of a €14 million EU funded integrated project, which will further develop such systems in Europe.

4. The critical components of the BWAP have been fully developed and pilot tested but the SC believe that some additional resources will be required to fully implement the system. In addition to the basic infrastructure (i.e. a user group, training resources and web database) there are resource (time) implications for certification bodies.

Relevance to Organic Certification Bodies

5. The SC believes that the BWAP is a credible, repeatable and valid assessment system that can be used by certification bodies to assess compliance with both animal welfare-related and health / management plan-related organic standards. The SC would like to see the development of protocols for sheep, broilers and calves.

6. Several organic CBs have demonstrated a significant commitment to the goals of this project by investing their time into the project by attending meetings, training courses and organising regional meetings for farmers.

Implementation of BWAP

7. The SC commends the approach to ACOS and recommends they seek to ensure the application of the system to organic certification and the long term goal that the assessment system be fully implemented by all certification bodies for both initial and surveillance inspections.

8. The SC recognise, however, that implementation of this system should be in a staged approach that neither reduces the competitiveness of individual certification bodies nor provides a regulatory burden for UK organic producers. The staged implementation of the assessment system should ensure that increasing levels of application (i.e. types of farms assessed) are applied by all certification bodies at the same time.

Maximising benefits from the system

9. ACOS, DEFRA & the organic farming bodies have a responsibility to advocate the health and productivity benefits of welfare assessment and benchmarking to farmers. In particular the assessment system should be incorporated into advisory visits during the conversion process.

10. The SC recognise that this evidence based system can be used to generate marketing claims concerning high animal welfare standards on UK organic farms which ACOS and other organic bodies should use to promote UK organic farming once the system has been successfully implemented.

11. In addition to certification and management benefits the SC believe that DEFRA could also use the system as a measurable outcome of a public good that might be relevant for future aid / single farm payment schemes.

Acknowledgement

This project would not have been possible without the funding by DEFRA and the previous study funded by RSPCA. Furthermore we are particularly grateful for all contributions by the certification bodies (SA, SOPA, OF&G, QWFC, OFF), farmers

References

Bartussek, H. (1999) A review of the animal needs index ANI for the assessment of animals’ well-being in the housing systems for Austrian proprietary products and legislation. Livestock Production Science, 61(2-3) 179-192.

Bennett RM, Christiansen K, Clifton-Hadley RS (1999) Direct costs of endemic diseases of farm animals in Great Britain VETERINARY RECORD 145 (13): 376-377

Blokhuis HJ, Jones RB, Geers R, Miele M, Veissier I (2003) Measuring and monitoring animal welfare: Transparency in the food product quality chain ANIMAL WELFARE 12 (4): 445-455

COMMISSION OF THE EUROPEAN COMMUNITIES

(2004) European Action Plan for Organic Food and Farming

{SEC(2004)739} COMMUNICATION FROM THE COMMISSION TO THE COUNCIL AND THE EUROPEAN PARLIAMENT

DEFRA (2004) COMPENDIUM OF UK ORGANIC

STANDARDS

EA, European co-operation for accreditation(1999) Guidelines on the Application of EN 45011, EA-6/01 (rev.00)

HMSO (2000) The Welfare of Farmed Animals (England) Regulations 2000 Statutory Instrument 2000 No. 1870

Hovi M, Sundrum A, Thamsborg SM (2003) Animal health and welfare in organic livestock production in Europe: current state and future challenges LIVESTOCK PRODUCTION SCIENCE

80 (1-2): 41-53

Huxley JN, Burke J, Roderick S, Main DCJ, Whay HR (2003) Herd health and welfare benchmarking on organic dairy farms in South-West England CATTLE PRACTICE 11: 331-333

Huxley JN, Burke J, Roderick S, Main DCJ, Whay HR (2004) Animal welfare assessment benchmarking as a tool for health and welfare planning in organic dairy herds Veterinary Record 155 237-239

Johnsen, P. F., Johannesson, T., and Sandøe, P. (2001) Assessment of Farm Animal Welfare at Herd Level: Many Goals , Many Methods. Acta Agriculturæ Scandinavica. Sect A, Abinal Science: Suppl. 30, 26-33

Kossaibati MA, Esslemont RJ (1997) The costs of production diseases in dairy herds in England VETERINARY JOURNAL

154 (1): 41-51

Leeb B, Leeb C, Troxler J, Schuh M Skin lesions and callosities in group-housed pregnant sows: Animal-related welfare indicators (2001) ACTA AGRICULTURAE SCANDINAVICA SECTION A-ANIMAL SCIENCE

: 82-87 Suppl. 30

Main D.C.J., J. Clegg, A. Spatz and L.E. Green (2000) The repeatability of a lameness Scoring System for Finishing Pigs Veterinary Record, 147: 574-576

Main D.C.J., J. Kent, F. Wemelsfelder, E. Ofner, F. Tuyttens Applications for methods of on-farm welfare assessment (2003) Animal Welfare 12: 523 - 529

Main D C J, H R Whay, L E Green and A J F Webster (2003) Effect of the RSPCA Freedom Food scheme on Dairy Cattle welfare Veterinary Record 153 : 227-231

Mason GJ & Mendl M (1993) Why is there no simple way of measuring welfare ? Animal Welfare 2 : 301-319

Mench JA (2003) Assessing animal welfare at the farm and group level: A United States perspective ANIMAL WELFARE 12 (4): 493-503

Pye-Smith, C. (2003) “Batteries not included :organic farming and animal welfare” Soil Association

Seppanen L, Helenius J (2004) Do inspection practices in organic agriculture serve organic values? - A case study from Finland AGRICULTURE AND HUMAN VALUES

21 (1): 1-13

Spoolder H, De Rosa G, Horning B, Waiblinger S, Wemelsfelder F (2003) Integrating parameters to assess on-farm welfare ANIMAL WELFARE 12 (4): 529-534

Webster AJF, Main DCJ, Whay HR (2004) Welfare assessment: indices from clinical observation ANIMAL WELFARE 13: S93-S98

Whay H R (2002) Locomotion scoring and lameness detection in dairy cattle IN PRACTICE 24 (8): 444-449

Whay H R, D C J Main, L E Green and A J F Webster (2002)Animal Based measures for the assessment of welfare state of dairy cattle, pigs and laying hens: consensus of expert opinion Animal Welfare 12: 205-219

Whay H R, D C J Main, L E Green and A J F Webster (2003) Assessment of dairy cattle welfare using animal-based measurements Veterinary Record 153 : 197-202

Winckler C, Capdeville J, Gebresenbet G, Horning B, Roiha U, Tosi M, Waiblinger S (2003) Selection of parameters for on-farm welfare-assessment protocols in cattle and buffaloANIMAL WELFARE 12 (4): 619-624

Winckler C, Willen S The reliability and repeatability of a lameness scoring system for use as an indicator of welfare in dairy cattle (2001) ACTA AGRICULTURAE SCANDINAVICA SECTION A-ANIMAL SCIENCE : 103-107 Suppl. 30

To move from one fill-in location (field) to another, press TAB, RETURN, UP or DOWN arrow keys unless instructed to do otherwise. Locations may also be selected by clicking on them.

�Discussion/conclusions?

�I would shift this into discussion and conclusions

�Perhaps this should go in the discussion/conclusions as well.

�Discussion, conclusions, recommendatiosn?

CSG 15 (Rev. 6/02)1

CSG 15 (Rev. 6/02)2

_1156857531.xls

Chart1

Thin

Fat

Dirty side

Dirty hindlimb

Dirty udder

Skin lesion

Swollen hocks

Overgrown claws

% of assessments agreeing with mode results

0.93

0.96

0.78

1

0.94

0.77

0.87

0.87

rawdatasortedandcomplete

55209245375383397417678786796797

tcil00000010000

fcil00000000000

dsil11111011011

dhil11111111111

duil11111111111

slil10100111010

swil10111111010

clil11111111011

55209245375383379417678786796797

tckn

fckn

dskn0001111000

dhkn1111111111

dukn1111111011

slkn1000101111

swkn111111010

clkn111111111

55209245375383397417678786796797

tcir0000000000

fcir00000000000

dsir11111111011

dhir11111111111

duir11110111111

slir110011101

swir111011010

clir11111111011

55209245375383397417678786796797

tcds0000010000

fcds0000100000

dsds1111111011

dhds1111111111

duds1111111111

slds100101110

swds111111010

clds10101000

55209245375383397417678786796797

tcsh000001000

fcsh000010000

dssh111111101

dhsh111111111

dush111111111

slsh010001110

swsh110101111

clsh011011101

55209245375383397417678786796797

tcmj000001001

fcmj0000000000

dsmj1011001001

dhmj1111111111

dumj1111111111

slmj1100111

swmj1111010

clmj111011111

55209245375383397417678786796797

tcko0000001000

fcko0000010000

dsko0111001100

dhko1111111111

duko1111101111

slko10011111

swko11011101

clko1111111101

55209245375383397417678786796797

tcca00000010000

fcca00000000000

dsca01110011000

dhca11111111111

duca11111111011

slca100111110

swca11111111110

clca1111111011

55209245375383397417678786796797

tcsc0000001100

fcsc00000000000

dssc11111111011

dhsc11111111111

dusc11111111111

slsc11101111110

swsc11111111110

clsc11110111101

55209245375383397417678786796797

tcbm0000000000

fcbm0000100000

dsbm011101100

dhbm1111111111

dubm1111111001

slbm101010100

swbm111000111

clbm111110101

Sheet1

Thin cowskappa ValueAppr.Sign.Fat cowskappa ValueDirty sidekappa ValueAppr.Sign.Dirty hindlimbkappa ValueDirty udderkappa ValueSkin lesionskappa ValueAppr.Sign.Swollen hockskappa ValueAppr.Sign.Overgrown clawskappa ValueAppr.Sign.total correcttotal assessed%kappa Value

il11.001.0000.001100%na1.001.0000.001100%na100%na91%0.8140.00691%0.7440.011100%1.0000.001868898%0.9116

ir30.90100%na0.910.6210.026100%na91%na89%0.7270.02389%0.7270.023100%1.0000.001788394%0.76875

ds40.901.0000.00290%na0.900.6150.035100%na100%na67%0.3080.343100%1.0000.00350%0.1580.408667687%0.6165

sh51.001.0000.00389%na0.890.6090.047100%na100%na78%0.5710.05867%-0.1740.57178%0.4000.134637288%0.4812

mj60.78100%na0.600.2000.292100%na100%na71%0.3000.427100%1.0000.00878%-0.1250.708627286%0.34375

ko71.001.0000.00290%na0.700.4000.114100%na90%na75%0.3330.34688%0.6000.064100%1.0000.002687689%0.6666

ca81.001.0000.001100%na0.640.3130.154100%na91%na89%0.7270.02391%0.6210.026100%1.0000.002788592%0.7322

sc90.900.6150.035100%na0.910.6210.026100%na100%na82%0.5600.03991%0.6210.02673%-0.1380.621798791%0.4558

bm100.9090%na0.780.5260.073100%na80%na67%0.3720.15156%-0.2000.45389%0.6090.047627682%0.32675

0.930.940.81

%kappa

thin

fat

dirty side

dirty hindlimb

dirty udder

skin lesion

swollen hocks

overgrown

Sheet1

thinthin

fatfat

dirty sidedirty side

dirty hindlimbdirty hindlimb

dirty udderdirty udder

skin lesionskin lesion

swollen hocksswollen hocks

overgrownovergrown

%

kappa

0.93

0.936

0.96

0.78

0.504

1

0.94

0.77

0.488

0.87

0.594

0.87

0.545

resultsfinal

Thin cowskappa ValueAppr.Sign.Fat cowskappa ValueDirty sidekappa ValueAppr.Sign.Dirty hindlimbkappa ValueDirty udderkappa ValueSkin lesionskappa ValueAppr.Sign.Swollen hockskappa ValueAppr.Sign.Overgrown clawskappa ValueAppr.Sign.total correcttotal assessed%kappa Value

il1100%1.0000.001100%na100%1.0000.001100%na100%na91%0.8140.00691%0.7440.011100%1.0000.001868898%0.9116

kn291%100%na50%0.1380.389100%na90%na60%0.1670.598100%1.0000.00389%688085%0.435

ir390%100%na91%0.6210.026100%na91%na89%0.7270.02389%0.7270.023100%1.0000.001788394%0.76875

ds490%1.0000.00290%na90%0.6150.035100%na100%na67%0.3080.343100%1.0000.00350%0.1580.408667687%0.6165

sh5100%1.0000.00389%na89%0.6090.047100%na100%na78%0.5710.05867%-0.1740.57178%0.4000.134637288%0.4812

mj678%100%na60%0.2000.292100%na100%na71%0.3000.427100%1.0000.00878%-0.1250.708627286%0.34375

ko7100%1.0000.00290%na70%0.4000.114100%na90%na75%0.3330.34688%0.6000.064100%1.0000.002687689%0.6666

ca8100%1.0000.001100%na64%0.3130.154100%na91%na89%0.7270.02391%0.6210.026100%1.0000.002788592%0.7322

sc990%0.6150.035100%na91%0.6210.026100%na100%na82%0.5600.03991%0.6210.02673%-0.1380.621798791%0.4558

bm1090%90%na78%0.5260.073100%na80%na67%0.3720.15156%-0.2000.45389%0.6090.047627682%0.32675

correct940.936100na800.504103na97na710.488810.594840.54571079589%

assessed101104102103103929397

total93%96%78%100%94%77%87%87%

%Kappa value

Thin93.000%0.936

Fat96.000%

Dirty side78.000%0.504

Dirty hindlimb100.000%

Dirty udder94.000%

Skin lesion77.000%0.488

Swollen hocks87.000%0.594

Overgrown claws87.000%0.545

resultsfinal

ThinThin

FatFat

Dirty sideDirty side

Dirty hindlimbDirty hindlimb

Dirty udderDirty udder

Skin lesionSkin lesion

Swollen hocksSwollen hocks

Overgrown clawsOvergrown claws

%

Kappa value

0.93

0.936

0.96

0.78

0.504

1

0.94

0.77

0.488

0.87

0.594

0.87

0.545

alldatafinal

55209245375383397417678786796797total assessed5520924537538339741767878679679755209245375383397417678786796797tot corr% corrtot corrtot asstot corrtot ass

tcil0000001000011000000000001111111111111100%il868898%il868898%

tckn000000000001100000000000111111011111091%kn688085%ir788394%

tcir000000000010000000010001111110-9111990%ir788394%ca788592%

tcds000001000010000010000001111-9101111990%ds667687%sc798791%

tcsh000001000910000000001-9111111111-99100%sh637288%ko687689%

tcmj00000100190000011000011111-9-100110778%mj627286%sh637288%

tcko000000100010000000000011111111111-910100%ko687689%ds667687%

tcca0000001000011000000000001111111111111100%ca788592%mj627286%

tcsc000000110010000000000011111111011-9990%sc798791%kn688085%

tcbm000000000010000010000001111-9101111990%bm627682%bm627682%

tot 09101010893710106910101089571010694

tot1000000620019101010899910107101

tot ass9101010899910107

mode00000010000100%100%100%100%100%100%56%78%100%100%86%93%

correct9101010896710106

% correct100%100%100%100%100%100%67%78%100%100%86%94%

fcil0000000000011000000000001111111111111100%

fckn0000000000011000000000001111111111111100%

fcir0000000000011000000000001111111111111100%

fcds000010000010000010000001111-9011111990%

fcsh000010000910000000001-9111101111-9889%

fcmj0000000000100000010000011111-91111110100%

fcko000001000010000000000011111101111-9990%

fcca0000000000011000000000001111111111111100%

fcsc0000000000011000000000001111111111111100%

fcbm000010000010000010000001111-9011111990%

tot 0910101085101010108910101085101010108100

tot100000400000910101089101010108104

tot ass910101089101010108

mode00000000000100%100%100%100%100%56%100%100%100%100%100%96%

correct910101085101010108

% correct100%100%100%100%100%56%100%100%100%100%100%96%

dsil1111101101111000000000001111111111111100%

dskn0001111000100000010000000011-911100550%

dsir111111110111100000000000111110111111091%

dsds111111101110000010000001111-10011111990%

dssh111111101910000000001-10111101111-10889%

dsmj1011001001100000010000010110-901101660%

dsko011100110010000000000010111011110-10770%

dsca01110011000110000000000001110111100764%

dssc111111110111100000000000111110111111091%

dsbm0111011009000010000010111-1011110-10778%

tot 04210341010525891054910105580

tot15891054910055910101088101010107102

tot ass910101088101010107

mode1111101101156%80%90%100%63%50%90%100%100%50%71%78%

correct58910549101055

% correct56%80%90%100%63%50%90%100%100%50%71%78%

dhil1111111111111000000000001111111111111100%

dhkn1111111111100000010000011111-101111110100%

dhir1111111111111000000000001111111111111100%

dhds111111111110000010000001111-1011111110100%

dhsh111111111910000000001-10111111111-109100%

dhmj1111111111100000010000011111-101111110100%

dhko111111111110000000000011111111111-1010100%

dhca1111111111111000000000001111111111111100%

dhsc1111111111111000000000001111111111111100%

dhbm111111111110000010000001111-1011111110100%

tot 000000000000910101088101010108103

tot1910101088101010108910101088101010108103

tot ass910101088101010108

mode11111111111100%100%100%100%100%100%100%100%100%100%100%100%

correct910101088101010108

% correct100%100%100%100%100%100%100%100%100%100%100%100%

duil1111111111111000000000001111111111111100%

dukn1111111011100000010000011111-1011011990%

duir111101111111100000000000111101111111091%

duds111111111110000010000001111-1011111110100%

dush111111111910000000001-10111111111-109100%

dumj1111111111100000010000011111-101111110100%

duko111110111110000000000011111101111-10990%

duca111111110111100000000000111111110111091%

dusc1111111111111000000000001111111111111100%

dubm111111100110000010000001111-10111001880%

tot 000001100310910101077101079897

tot19101010771010798910101088101010108103

tot ass910101088101010108

mode11111111111100%100%100%100%88%88%100%100%70%90%100%94%

correct9101010771010798

% correct100%100%100%100%88%88%100%100%70%90%100%94%

slil101001110101100000000000111111110111091%

slkn1000101111100000010000011010-1001110660%

slir1100111019010000000011-911111101-9889%

slds100101110910001000000-10001-9101111667%

slsh010001110910000000001-10111101110-9778%

slmj11001117010011001001-911-9-1001-1010571%

slko100111118010000100011-90101-10111-9675%

slca1001111109010100000001-90-91111111889%

slsc11101111110110000000000010110111111982%

slbm1010101009000010000011111-910100-9667%

tot 00449414032484694751068471

tot18260375106828610978910910692

tot ass86109789109106

mode10100111110100%67%60%100%57%88%56%100%67%80%67%77%

correct846947510684

% correct100%67%60%100%57%88%56%100%67%80%67%77%

swil101111110101100000000000101111111111091%

swkn11111101090000010100011111-101-101119100%

swir1110110109000010010001110-1011-10111889%

swds1111110109000010010001111-1011-101119100%

swsh110101111910000000001-10110101101-9667%

swmj11110107010101010001-101-101-101-101117100%

swko11011101810000001001-10110111-1011-9788%

swca111111111101100000000000111111110111091%

swsc111111111101100000000000111111110111091%

swbm1110001119000010000011110-1000101-9556%

tot 001040210607881057695610781

tot1881057695410089109781051010793

tot ass891097810510107

mode11111111010100%89%100%56%100%75%90%100%60%100%100%87%

correct8810576956107

% correct100%89%100%56%100%75%90%100%60%100%100%87%

clil1111111101111000000000001111111111111100%

clkn11111111190000010000111111-101101-10889%

clir1111111101111000000000001111111111111100%

clds101010008010010010001-1001-1001-10100450%

clsh011011101910000000001-10011011111-10778%

clmj11101111190000010001011101-10110-101778%

clko111111110110000000000011111111111-1010100%

clca111111101110010000000001-1011111111110100%

clsc11110111101110000000000011110111001873%

clbm1111101019000010000011111-1010111-10889%

tot 0011121107219799679977584

tot19799679937598101088109109697

tot ass981010881091096

mode11111111011100%88%90%90%75%88%90%100%70%78%83%87%

correct97996799775

% correct100%88%90%90%75%88%90%100%70%78%83%87%

thincow

ilknirdsshmjkocascbmtot 0tot1tot assmodecorrect% correct

55000000000909091

2090000000000100100101

2450000000000100100101

3750000000000100100101

38300000000808081

397000000000909091

417100111110369160.6666666667

678000010010729070.7777777778

7860000000000100100101

7960000000000100100101

7970000100617060.8571428571

1111101099101110100.9405940594

fat cow

ilknirdsshmjkocascbmtot 0tot1tot assmodecorrect% correct

000000000909091

0000000000100100101

0000000000100100101

0000000000100100101

00000000808081

000111001549050.5555555556

0000000000100100101

0000000000100100101

0000000000100100101

0000000000100100101

00000000808081

0.9615384615

dirty side

ilknirdsshmjkocascbmtot 0tot1tot assmodecorrect% correct

101110010459150.5555555556

10111011112810180.8

10111111111910190.9

1111111111010101101

11110001358150.625

01110010448040.5

11111011111910190.9

1111111111010101101

0000000000100100101

10111000105510150.5

1011101257150.7142857143

1110111091010111190.7843137255

dirty hindlimb

ilknirdsshmjkocascbmtot 0tot1tot assmodecorrect% correct

111111111099191

1111111111010101101

1111111111010101101

1111111111010101101

11111111088181

11111111088181

1111111111010101101

1111111111010101101

1111111111010101101

1111111111010101101

11111111088181

11101110910101111101

dirty udder

ilknirdsshmjkocascbmtot 0tot1tot assmodecorrect% correct

111111111099191

1111111111010101101

1111111111010101101

1111111111010101101

11011111178170.875

11110111178170.875

1111111111010101101

1111111111010101101

10111110103710170.7

11111111101910190.9

11111111088181

11101110910101111100.9417475728

skin lesion

ilknirdsshmjkocascbmtot 0tot1tot assmodecorrect% correct

11111111088181

001010426040.6666666667

10101100114610160.6

000000000909091

0100101437040.5714285714

11101111178170.875

101010110459150.5555555556

1111111111010101101

010111110369160.6666666667

11110111102810180.8

010100426040.6666666667

11109997891190.7717391304

swollen hock

ilknirdsshmjkocascbmtot 0tot1tot assmodecorrect% correct

11111111088181

011111111189180.8888888889

1111111111010101101

110100110459150.5555555556

1111111077171

11101110268160.75

11111111101910190.9

11111055151

00001001116410060.6

1111111111010101101

0000000707071

11999978111190.8709677419

claw overgrown

ilknirdsshmjkocascbmtot 0tot1tot assmodecorrect% correct

111111111099191

11101111178170.875

11101111111910190.9

11111011111910190.9

11101110268160.75

11011111178170.875

11111111101910190.9

111111111099191

01000100107310070.7

111011101279170.7777777778

110111156150.8333333333

1191189910101190.8659793814

raw dataDAvid

79739779638355678209417375245786

tcil00000001000

fcil00000000000

dsil10111111110

dhil11111111111

duil11111111111

slil01101101010

swil01111101110

clil11111111110

79738319624578637541755678209

tckn

fckn

dskn0100011010

dhkn1111111111

dukn1111011111

slkn1110100110

swkn011101111

clkn111111111

38379779639737541720967855245786

tcir0000000000

fcir00000000000

dsir11111111110

dhir11111111111

duir01111111111

slir011011110

swir011011110

clir11111111110

78624555678209417375196797397

tcds0000010000

fcds0000000001

dsds0111111111

dhds1111111111

duds1111111111

slds101100101

swds011111101

clds00111000

245678375417209786397383196

tcsh000100000

fcsh000000100

dssh111110111

dhsh111111111

dush111111111

slsh110101000

swsh110111011

clsh111100101

79738379679767820955417537786245

tcmj100100000

fcmj00000000000

dsmj10011010101

dhmj11111111111

dumj11111111111

slmj11111001

swmj01101101

clmj1111111011

24555678209417375786383796387

tcko0000100000

fcko0000000001

dsko1011110000

dhko1111111111

duko1111111110

slko01101111

swko11100111

clko1111110111

89479679773938341753720967855786245

tcca000001000000

fcca000000000000

dsca100001111001

dhca111111111111

duca111111111101

slca1101011110

swca010111111111

clca01111111101

79778641720967837524555397383196

tcsc0101000000

fcsc00000000000

dssc10111111111

dhsc11111111111

dusc11111111111

slsc01111011111

swsc01111111111

clsc11111111100

41767820413724555786796797397

tcbm0000000000

fcbm0000000001

dsbm111110000

dhbm1111111111

dubm1111110011

slbm010011001

swbm011011110

clbm011111011

Sheet2DAVID

clawdirt hinddirt sidedirt udderfat cowskin lesionswollen hockthin cowtotal correcttotal assess

ca1100%100%70%90%100%89%90%100%727892%

ds256%100%100%100%100%67%100%91%738190%

il3100%100%82%100%91%91%91%100%838894%

ir4100%100%92%92%92%90%80%91%849192%

kn578%100%50%90%60%100%465879%

ko6100%100%73%91%100%78%89%100%778492%

mj788%100%67%100%100%67%100%75%576588%

sc883%100%100%100%92%83%83%91%879592%

sh967%100%100%100%100%78%67%100%647289%

correct7894789082677777

assessed9094959485858782

total87%100%82%96%96%79%89%94%64371290%

all dataDAVID

55196209245375383397417678786796797tot asses5519620924537538339741767878679679755196209245375383397417678786796797tot corr% corrtot corrtot ass

cacl1111101180110101000001-9-101-101-10111118100%ca647091%

bscl

dscl10010100800100100101011-1001-1001-101-100563%ds707692%

ilcl11111111011110100000000001-9111111111111100%il838894%

ircl11111111011110100000000001-9111111111111100%ir778393%

kncl1111111119000000100011101111-10110-10-10778%kn465879%

kocl1111111101100100000000011-9111111111-1010100%ko697691%

mjcl1111111180100101000101-911-101-10110-101788%mj576588%

sccl10111011111110000000000101111101110-101982%sc808792%

shcl1011011109100000000011-10001101111-10-10667%sh647289%

tot 0cl02110210060161067590%

tot1cl82687659834582687659864574

tot ass84797869894684797869894685

modecl101111111011

correctcl826876598645

% correct100%50%86%89%100%75%83%100%100%67%100%83%90%100%50%86%89%100%75%83%100%100%67%100%83%87%

cadh11111111190100101000001-1011-101-10111119100%

dsdh11111111111000000100001011111-101111-10110100%

ildh11111111111110100000000001-10111111111111100%

irdh11111111111110100000000001-10111111111111100%

kndh111111111110000000100010111111-10111-10110100%

kodh1111111111100100000000011-10111111111-1010100%

mjdh11111111190100101000001-1011-101-10111119100%

scdh11111111111110000000000101111111111-10111100%

shdh1111111119100000000011-10111111111-10-109100%

tot 0cl000000000000

tot1cl84997869995784997869995790

tot ass84997869995784997869995790

modecl111111111111

correctcl849978699957

% correct100%100%100%100%100%100%100%100%100%100%100%100%91%100%100%100%100%100%100%100%100%100%100%100%100%100%

cads01101100090100101000000-1011-100-1011110667%

dsds11111111011000000100001011111-101111-9110100%

ilds11111011011110100000000001-101111011101982%

irds11111111011110100000000001-1011111111011091%

knds000011110010000000100010000011-10111-90550%

kods0111001100100100000000010-10111001111-10770%

mjds10100100190100101000001-1001-100-1001111667%

scds11111111101110000000000101111111111-9111100%

shds1111111109100000000011-10111111111-9-109100%

tot 0cl312103210932

tot1cl53787548902553787548993573

tot ass84997869995784997869995790

modecl111111111001

correctcl537875489935

% correct63%75%78%89%100%63%67%89%100%100%60%71%74%63%75%78%89%100%63%67%89%100%100%60%71%81%

cadu11111101190100101000001-1011-101-1011011889%

dsdu11111111111000000100001011111-101111-10110100%

ildu11111111111110100000000001-10111111111111100%

irdu11110111111110100000000001-1011101111111091%

kndu111111110110000000100010111111-10110-101990%

kodu1111101111100100000000011-10111101111-10990%

mjdu11111111190100101000001-1011-101-10111119100%

scdu11111111111110000000000101111111111-10111100%

shdu1111111119100000000011-10111111111-10-109100%

tot 0cl000001100200

tot1cl84997759975784997759975786

tot ass84997869995784997869995790

modecl111111111111

correctcl849977599757

% correct100%100%100%100%100%88%83%100%100%78%100%100%87%100%100%100%100%100%88%83%100%100%78%100%100%96%

cafc00000000090100101000001-911-91-10111119100%

dsfc00000100001000000100001011111-91111-9110100%

ilfc00000000000110100000000001-911110111111091%

irfc00000000000110100000000001-911110111111091%

knfc0111111111111-9-9-9-9-9-9-10-9-9-9-9-900%

kofc0000010000100100000000011-9111111111-910100%

mjfc00000000090100101000001-911-91-10111119100%

scfc00000000000110000000000101111110111-911091%

shfc0000010009100000000011-9111111111-9-99100%

tot 0cl738867388856

tot1cl00000030000073886738885677

tot ass73886768885673886768885680

modecl000000100000

correctcl738867388856

% correct100%100%100%100%100%100%50%100%100%100%100%100%87%100%100%100%100%100%100%50%100%100%100%100%100%96%

casl1001111080110101000001-10-90-91-1011111788%

dssl1100101109100001000010-101001-91011-101667%

ilsl10100111010110100000000001-1011111110111091%

irsl11001110190110000000011-10-911111101-9889%

knsl110001011110000000100010111010-10011-100660%

kosl1001111180110000100011-10-90101-10111-9675%

mjsl11011160110111001001-10-91-9-9-1001-1010467%

scsl11110111110110000000000101101101111-101982%

shsl0010001119100000000011-10011110111-10-9778%

tot 0cl013474130204

tot1cl73250355965273357455965463

tot ass74597768985674597768985681

modecl110100111110

correctcl733574559654

% correct100%75%60%56%100%57%83%63%100%75%100%67%70%100%75%60%56%100%57%83%63%100%75%100%67%78%

casw11111111090100101000001-1011-101-1011011889%

dssw111111100900000100101011111-1011-101-1019100%

ilsw10111111010110100000000001-1001111111111091%

irsw11101101090100010010001-10110-1011-10111889%

knsw1111111009000000101010111111-101-101-1019100%

kosw110111018110000001001-10-10110111-1011-9788%

mjsw111101070110101010001-10-101-101-101-101117100%

scsw11111111110110000000000101111111110-1011091%

shsw1110101119100000000011-10111010110-10-9667%

tot 0cl001030100607

tot1cl74794759435074794759465774

tot ass74897769495774897769495782

modecl111111111010

correctcl747947594657

% correct100%100%88%100%57%100%83%100%100%67%100%100%81%100%100%88%100%57%100%83%100%100%67%100%100%90%

catc00001000090100101000001-911-91-9111119100%

dstc00000010001000000100001011111-91111-9110100%

iltc00000010000110100000000001-9111111111111100%

irtc0000000000100100000010001-9111110-9111990%

kntc0111111111111-9-9-9-9-9-9-9-10-9-9-9-90

kotc0000001000100100000000011-9111111111-910100%

mjtc0000100180100101100001-911-91-9-100110675%

sctc0000000110100000000000111111111101-9-9990%

shtc0000001009100000000011-9111111111-9-99100%

tot 0cl738867615854

tot1cl00000006200173886766585473

tot ass73886767785573886767785577

modecl000000010000

correctcl738867665854

% correct100%100%100%100%100%100%100%86%71%100%100%80%85%100%100%100%100%100%100%100%86%71%100%100%80%95%

_1156857598.xls

Chart1

Thin pigs

Dirty side

Nasal discharge

Head/neck lesions

Hindquarter lesions

Tail lesion

Swollen hock / bursae

%

1

0.77

0.72

0.68

0.91

0.92

0.73

rawdatasortedandcomplete

12345678910

tpca0000000000

dsca0100000000

onca1111100010

hnfca0101001000

ghca0000000000

tlca0000010000

blca0010000001

tpkn0000000000

dskn0101011111

onkn110101010

hnfkn0100000000

ghkn0000000001

tlkn0000000000

blkn1110011111

tpko0000000000

dsko1000111101

onko0101111010

hnfko0000111000

ghko0000010000

tlko0000010000

blko0000000000

tpsh0000000000

dssh0100110111

onsh1111111011

hnfsh0001111000

ghsh0000010000

tlsh0000010000

blsh0000011000

tpir0000000000

dsir1110111111

onir1111111000

hnfir1100011011

ghir0000010000

tlir0000000000

blir0000000000

tpsc0000000000

dssc1111111111

onsc0001000000

hnfsc1101011100

ghsc0000000000

tlsc0000000000

blsc110011101

tpds0000000000

dsds1100111111

onds0000001010

hnfds1010010010

ghds0000010000

tlds0000010000

blds0110001101

tpbm0000000000

dsbm0100011111

onbm0111111011

hnfbm1110111010

ghbm0010011010

tlbm0000010000

blbm0010000001

tpmj0000000000

dsmj0100011101

onmj100111011

hnfmj011101010

ghmj000001

tlmj0000111

blmj1110001001

tpil0000000000

dsil0110011111

onil1111111011

hnfil0001110000

g