stakeholders’ perspective on ecological modeling in environmental risk assessment of pesticides:...

Risk Analysis DOI: 10.1111/j.1539-6924.2012.01835.x

Stakeholders’ Perspective on Ecological Modeling inEnvironmental Risk Assessment of Pesticides: Challengesand Opportunities

Agnieszka D. Hunka,1∗ Mattia Meli1 Amalie Thit,1 Annemette Palmqvist,1

Pernille Thorbek,2 and Valery E. Forbes3

The article closely examines the role of mechanistic effect models (e.g., population models) inthe European environmental risk assessment (ERA) of pesticides. We studied perspectivesof three stakeholder groups on population modeling in ERA of pesticides. Forty-three in-depth, semi-structured interviews were conducted with stakeholders from regulatory author-ities, industry, and academia all over Europe. The key informant approach was employed inrecruiting our participants. They were first identified as key stakeholders in the field and thensampled by means of a purposive sampling, where each stakeholder identified as importantby others was interviewed and asked to suggest another potential participant for our study.Our results show that participants, although having different institutional backgrounds oftenpresented similar perspectives and concerns about modeling. Analysis of repeating ideas andkeywords revealed that all stakeholders had very high and often contradicting expectationsfrom models. Still, all three groups expected effect models to become integrated in futureERA of pesticides. Main hopes associated with effect models were to reduce the amount ofexpensive and complex testing and field monitoring, both at the product development stage,and as an aid to develop mitigation measures. Our analysis suggests that, although the needsof stakeholders often overlapped, subtle differences and lack of trust hinder the process ofintroducing mechanistic effect models into ERA.

KEY WORDS: Ecological models; population models; risk assessment; stakeholder views

1. INTRODUCTION

The registration and use of pesticides in Europerequires that substances undergo a thorough risk as-

1Department of Environmental, Social and Spatial Change,Roskilde University, Universitetsvej 1, 4000 Roskilde, Denmark.

2Syngenta, Jealott’s Hill International Research Centre, Brack-nell, Berkshire, RG42 6EY, UK.

3School of Biological Sciences, University of Nebraska Lincoln,348 Manter Hall,

4Lincoln, NE 68588-0118.∗Address correspondence to Agnieszka D. Hunka, Department of

Environmental, Social and Spatial Change, Roskilde University,Universitetsvej 1, 4000 Roskilde, Denmark; tel: 004546742089;[email protected].

sessment in which the environmental fate, humanhealth effects, and ecological effects of the pesticidesare estimated. The overall process is carried out usinga tiered approach that starts with worst-case assess-ments of exposure and effects at lower tiers, and pro-ceeds to more realistic assessments at higher tiers forthose substances that fail to pass critical thresholdsfor exposure and/or effects. For environmental riskassessment (ERA), environmental fate and exposureare typically estimated using models,(1,2) whereas ef-fects are usually estimated from the results of labora-tory toxicity tests or (occasionally) from mesocosmor field studies. For most taxonomic groups, ERAaims to prevent unacceptable effects of pesticides at

1 0272-4332/12/0100-0001$22.00/1 C© 2012 Society for Risk Analysis

2 Hunka et al.

the population level,(3,4) with the exception of birdsand mammals for which effects on individuals are ofconcern.

In recognition of the difficulties in extrapolat-ing effects of standard toxicity tests and mesocosmsto likely impacts of pesticides on populations in thefield, mechanistic effect models (e.g., ecosystem orpopulation models) are seen as a way to bridge thegap between test endpoints and the ecological en-tities that the current risk assessment schemes aimto protect.(5,4) In our article we defined mechanis-tic effect models after Grimm et al.,(6) as “modelsthat mechanistically represent key ecological processes(. . .) and individual-level models quantifying adverseeffects of chemicals on organisms based on mechanis-tic understanding.” (p. 615)

The number of suitable models grows every year,as do initiatives to come up with a unified and stan-dardized approach to models and modeling docu-mentation.(7−−9) Schmolke et al.(8) conducted a com-prehensive literature search for models that deal withthe effects of pesticides on populations or commu-nities. From this review, it is clearly evident that abroad range of effect models have been applied toaddress ecotoxicological problems. Schmolke et al.grouped the models used to assess chemical riskinto three broad classes (see also Refs. 10,11,5):differential equations, matrix- and individual-basedmodels. Their review points out that the first two,more traditional modeling approaches, are the mostcommonly used, whereas relatively new tools, rep-resented by individual-based models, are still notwidespread, although their popularity is slowly in-creasing. Still, stakeholder groups involved in ERAof pesticides have different views on the applicabil-ity of effect models in real-life decision-making pro-cesses. Whereas predictive modeling is used widelyin many areas in our daily life, such as weather fore-casting, and in other fields of natural science (e.g.,conservation ecology),(12) models have not been aswidely used to predict effects of pesticides or othertoxic chemicals.

At the same time, fate models are used in ERAof pesticides for predicting concentrations of activesubstances in ground-and surface water.(13) The ma-jority of national regulatory bodies employ FOCUSmodels for that purpose.(14) The FOCUS group, es-tablished as an initiative of the European Commis-sion in 1993 was based on cooperation of scientistsand modelers from industry, academia, and regula-tory authorities. For both ground water and surfacewater the group developed several different scenar-

ios representing typical agricultural conditions in theEuropean Union (EU), and a set of models simulat-ing the distribution of pesticides in surface water andground water. There are detailed regulatory guide-lines about how to apply FOCUS modeling outputsto risk assessments. The legal status for use of ef-fect models is very different, however. For instance,European Food Safety Authority (EFSA) guidanceon risk assessment for birds and mammals states thatpopulation models may be used as refinements, on acase-by-case basis(15) Still, there is currently no guid-ance on requirements for effect model developmentor how to apply them to risk assessments.

The aim of this study was to explore the roleand the potential applicability of mechanistic ef-fect models in ERA of pesticides under the Euro-pean regulation No 1107/2009, as perceived by poten-tial model users, reviewers, creators, and evaluators.Partially employing the policy arrangements (PA)approach, we studied perspectives of different stake-holder groups (actors) on effect models and model-ing. PA provide a framework to analyze shifts andchanges in environmental policy. The framework,coined by Arts et al.,(16) has been used as a conceptlinking long-term policy changes (e.g., the shift to-ward more environmentally concerned and “green”policy and legislation in recent years), with particu-lar decision-making processes (see e.g., Ref. 17). Theconcept allows focusing on both institutional (i.e., in-volved stakeholders and their interactions) and dis-cursive (i.e., views and opinions of involved parties)characteristics of a policy change.(18)

Because mechanistic effect models are both gain-ing popularity in ecotoxicology, and also slowly get-ting on the political agenda,(19,4) we employed thePA as a framework able to grasp long-term policychange. We decided to study both stakeholders’ in-teractions and discourse in ERA toward the integra-tion of effect models into the current ERA of pes-ticides scheme. The PA framework comprises fourtightly interwoven dimensions:

• Actors (stakeholders) and their coalitions;• The division of power between actors and in-

fluence over policy outcomes;• Rules of the game, which are forms of inter-

action, both formal and informal in pursuit ofdecision-making;

• Discourse, which is the narrative and views ofinvolved parties “in terms of norms and val-ues, definitions of problems and approaches tosolutions.”

Stakeholders’ Perspective on Ecological Modeling in ERA of Pesticides 3

We identified the key actors in the EuropeanERA for registration of pesticides and interrelationsamong them. We also followed closely the currentstate of effect modeling in ERA of pesticides andthe general organization of the decision-making pro-cesses behind pesticide authorization. Moreover, wesought answers for two main questions: (1) what cri-teria does an effect model need to fulfill to be usedin ERA? and (2) what prevents models from beingused to predict effects of pesticides?

Although similar issues are emerging in otherERAs within the European legislation, such as underREACH and the Water Framework Directive, wefocused on pesticide regulation as a case study, be-cause exposure scenarios and focal species are betterdeveloped than in other regulatory frameworks forchemicals.

2. METHODS

Our aim was to reach a wide variety of stakehold-ers directly or indirectly involved with ERA for reg-istration of pesticides across Europe, and we foundthe key informants approach best suited for the pur-pose. The key informants technique originates in an-thropology and ethnographic studies, where a keyinformant is a source of detailed information basedon the expert knowledge of a particular subject.(20)

Hence, our sampling methods were strictly purpo-sive, rather than random, because we sought partic-ipants with predefined characteristics—in this caseexpertise in ERA of pesticides. First, we identi-fied key actors and continuously sampled by meansof snowball sampling,(21) where each stakeholderidentified as important by others was interviewedand asked to suggest other key stakeholders to beinterviewed.

We interviewed participants from the pesticideindustry, academia, and national and Pan-Europeanregulatory authorities involved in the registration ofpesticides. In addition, we had invited a number oflarge, international nongovernmental organizations,actively involved in agrochemical campaigns; how-ever, none of these accepted our invitation to partici-pate. We recruited not only risk assessors and mech-anistic effect modelers, but also risk managers, policymakers, fate modelers and some contract researchersfrom 10 different European countries and the UnitedStates. In total, more than 60 prospective participantswere invited, out of which 43 were interviewed: 15participants from industry, 14 from regulatory bod-ies, and 14 from academic institutions.

To gather a variety of responses, we preparedan interview guide comprising 15 open topics. Onlya part of the interview guide was directly related tomodels in ERA (see Appendix for a complete in-terview guide). The focus was on the use of modelsand recent changes in guidance documents and pesti-cides legislation, however, participants were encour-aged to share their own views and stories. We alsodefined “models” very broadly, asking our partici-pants whether they used/came across models at all,and whether they used or developed models to assessany potential effects of pesticides.

All interviews were confidential and conductedby the same person, as different interviewers couldset a different tone and influence the answers. Theinterviewer was not professionally involved with ei-ther pesticide registration or the development ofpopulation models, but trained in gathering in-terviews and survey data. The interviews lasted40 minutes on average and they were recordedwith the participants’ permission, after which thetexts were transcribed verbatim and submitted tothe participants for approval. We divided the in-terviewees into the three above-mentioned groupsand analyzed the transcripts accordingly, assuring thefull anonymity of our participants. Two researchersworked on coding and organizing the anonymoustranscripts, grouping and cross referencing similarpieces of text to identify main themes and key-words related to modeling. Working separately al-lowed controlling for potential biases in a single re-searcher, but the coders discussed and comparedtheir separate results afterwards. Transcripts werefirst divided into the smallest logical units (couplesof sentences forming single ideas), which were thengrouped together. We cross-referenced them bothwithin a single interview and against other inter-views. The method we used here is commonly em-ployed in qualitative data analysis in social, psycho-logical, and ethnological studies, and it is knownas the “grounded theory approach,” because of theway text data are treated.(22,23) The approach is onlyused to answer broad, open-ended research prob-lems, instead of testing any predefined hypothe-ses. The authors, Glaser and Strauss,(23) proposedthis method to uncover any common, repeatingthemes “grounded” in qualitative data. The tech-nique is not meant to achieve the best possiblestatistical representativeness, but to obtain a widerange of responses and in-depth perspectives.(23) Wealso based our methods on discourse analysis forthe parts of interviews revolving around interactionsamong stakeholders. Discourse analysis is, to a great

4 Hunka et al.

extent, similar to the grounded theory approach as amethod (grouping raw text data into larger, meaning-ful units), but the stress is on social interactions.(24)

A researcher also looks for emerging themes (calleddiscourses), but the main focus is on inconsisten-cies, repetitions, citing others to support one’s ownviews, etc. The purpose is to study, for instance,power relations between different stakeholders (so-cial groups).(25)

We continued to invite new participants up to thepoint of theoretical saturation in the data. Theoreti-cal saturation is the point at which further interview-ing does not add to the findings, or repeats alreadycollected information. It is commonly used in thegrounded theory approach(22) and helps researchersto decide when to stop recruiting new participants.

Our qualitative text analysis is accompanied byquantitative data, as we also calculated frequenciesof keywords that we identified during the codingphase with TextSTAT 2.8 software (TextStat is opensource software, created by Matthias Huning, FreieUniversitat Berlin, Berlin, Germany). We countedthe keywords—in this case the key criteria for ac-cepting models, first automatically, then manually.Keywords and their equivalents (e.g., validation, val-idated, validate) were counted together. As we pri-marily used the keywords count to get a better under-standing of criteria a model needs to fulfill, accordingto our respondents, to be used/accepted we excludedsome keywords from the count due to their con-text, for example the ones that occurred in phraseslike “models cannot be validated,” without contend-ing whether such phrases were true/false.

3. RESULTS

We first focus on general patterns, common forall participants, before moving to a detailed analy-sis of common themes and differences between thethree groups (coded as academia, industry, and regu-lators). To protect anonymity of the participants, wepresent the results in a narrative form, without anydirect quotations. However, all themes, trends, andpatterns described in the next sections were directlyderived from and supported by text (transcribed in-terviews) data.

We asked the respondents about modeling inrisk assessment first, and then in very broad termsabout “the use of models to predict effects.” Our aimwas to keep the interviews open and let the partic-ipants decide for themselves what they understoodby an “effect model.” Therefore, we obtained vari-

Table I. Fate and Effect Modeling in ERA of PesticideRegistration

I Hardly I Am anI Use I Use Ever Come Active ModelerFate Effect across Effect in the Effect

Models Models Models Area

True 29 15 9 8False 14 28 34 35Breakdown of “True” by each group

Regulators 14 4 5 0Industry 11 3 1 1Academia 4 8 3 7

First columns show numbers of respondents expressing agreement(true) or disagreement (false) with the Heading Statements. Sec-ond section shows how the agreement (true) with each headingstatement is distributed in each group.

ous reactions, although only four respondents fromour sample did not associate “effect model” withsome kind of mechanistic effect modeling approach.The best-known approach for the participants turnedout to be individual-based population models (seeRef. 8). This type of ecological model was also theonly one that respondents thought tended to have ahistory of use in pesticide ERA.

3.1. General Patterns

In our sample, there were only eight active mod-elers, working with different kinds of effect mod-els: individual- or agent-based, matrix, energy bud-get, or meta-population models. On the other hand,our interviewees were familiar with environmen-tal fate modeling, and a common pattern duringthe course of an interview was to assume that by“models” the interviewer meant “fate models” only.Table I presents a summary of this pattern among ourparticipants.

Effect models were not as familiar to our partic-ipants as were fate models. The regulatory author-ities working with risk management were the leastlikely to work with or use models at all, followed bythose academics exclusively devoted to experimentalwork and consultants from contract research com-panies. Skepticism about models among the partic-ipants did not follow exactly the same pattern. Themost positive group turned out to be regulators—both risk assessors and risk managers, while someskepticism about effect models was present amongthe academics involved in experimental research. Insome cases modeling was mentioned as a threat tothe experimental approach in terms of funding pri-orities. Still, models seemed to be rather warmly


expected, however, as shown in Table I they are yetto be widely used. In the whole group, eight partic-ipants were skeptical about the usefulness of effectmodels in ERA: four academics, three industry rep-resentatives and one regulator.

The last result is contrasted with the popular-ity of fate models. Not only were such models fa-miliar to the majority of our participants, but theywere often set as an example for effect models,mainly of robustness, user-friendliness, and as hav-ing well-documented version control. Participants,particularly from industry and regulatory authorities,stressed that, provided population models were de-veloped in a similar manner as fate models, theywould quickly become more commonly used. At thesame time, skepticism of models estimating pesticideeffects was matched by skepticism of experimentalresults estimating pesticide fate. Many participantsnoticed that the outcome of environmental fate mod-els to predict exposure to pesticides (e.g., FOCUSGroundwater models) was generally more trustedand preferred over results of field experiments.

3.2. Discourse on Modeling: Common Themes

There were three main themes present in ourdata: expectations associated with models, obstaclesin their implementation, and criteria that popula-tion models had to meet to be accepted in ERA ofpesticides.

3.2.1. Expectations.

Both industry and regulators put a lot of hope inpopulation models as a tool to reduce the costs andincrease cost-effectiveness of risk assessment andmanagement measures. Industry representatives sawmodels as a way to reduce the amount of requiredtesting, which would have benefits for animal welfareand reduce costs, especially for expensive field exper-iments performed for higher tier assessments. Regu-lators, on the other hand, expected modeling to helpthem reduce the costs of expensive monitoring pro-grams.

Yet, our analysis showed that the expectationsthat models should meet were contradictory. First,participants in every group expected models to besimple—easy to understand, create, re create, andcontrol. They should be transparent in terms of al-lowing much control over the input parameters andrepresenting processes that should be easy to fol-low. At the same time, every group put much hope

in modeling as a tool to tackle problems that werecomplex and/or not yet addressed in ERA of pesti-cides. Therefore models were expected to be com-plex enough to address several problems mentionedby the respondents: the issue of mixture toxicity (reg-ulators), allowing extrapolations to different condi-tions, time spans and species (all three groups), aid-ing understanding of real populations (academia),and accommodating landscape changes (all threegroups). Second, ecological models were expected tobe specific, in terms of addressing precisely some ex-plicit questions of higher tier assessments (regulators,industry), but also generic enough to be applicableinternationally, represent average situations, land-scape and climatic conditions, even generic species(all three groups). Third, models were supposed tosolve the current problems of risk management byproviding different scenarios and a whole range ofprobability estimates (all three groups). At the sametime, some stakeholders also expected the models toproduce a binary output, in terms of allowing the de-cision maker to answer yes/no or below/above safetythreshold questions (regulators).

3.2.2. Obstacles.

The three groups were surprisingly unanimouson obstacles to population models becoming ac-cepted for regulatory use. The main problems men-tioned by respondents were: lack of trust in modelingaccompanied by lack of models suited for decision-making purposes, and the problem of uncertaintypresent in modeling output.

3.2.2.1. Vicious circle. The main obstacle that ev-ery group mentioned preventing the use of ecologicalmodels in ERA was a lack of trust in them. Modelswere rarely submitted as part of the risk assessment,as participants from industry stressed, because theywere not trusted and tended to be rejected by the reg-ulatory authorities. As familiarity breeds trust, andworking with models breeds familiarity, at the mo-ment ecological modeling is in a Catch-22—not usedbecause of the lack of trust, and not trusted becauseof the lack of use.

3.2.2.2. Regulatory question. Another obstacle,particularly apparent to our respondents from regu-latory authorities was that models very often failedto sufficiently address the issues regulators expectedthem to address. Participants mentioned several

6 Hunka et al.

instances of such a situation. First, an ecologicalmodel could use an inappropriate species to showpotential effects, for example, a species that did notrepresent a worst-case scenario. Second, a modelcould fail to provide a clear answer about meetingthe cut-off criteria, for example, by providing aprobability, instead of a yes/no result. Finally, manyparticipants from all three groups mentioned thatmodels would not be able to answer regulatoryquestions due to procedural factors. Regulatory re-spondents, for instance, said that the protection goalswere not clear, as the current pesticide legislationstates these goals in very general terms (e.g., . . .shallhave no unacceptable effects on the environment;Regulation (EC) 1107/2009).(3) Broad legal bound-aries made it challenging to adapt modeling outputsto regulatory problems.

3.2.2.3. Black box. All three groups stressed thatunderstanding what happened inside a model waskey, and that the lack of transparency in modelingwas a serious obstacle. Two main reasons for such asituation were mentioned: the lack of necessary doc-umentation, which was stressed by modelers them-selves, and the lack of control over the input param-eters, which was especially important for regulators.Interestingly, an understandable model did not nec-essarily have to be simple. Regulators actually pre-ferred transparency in terms of clear documentationand control of modeling input over simplicity.

3.2.2.4. Uncertainty. Uncertainty of a future out-come is built into the definition of the word “risk.”Still, it turned out that uncertainty that is built intoa model’s outcome could be a serious obstacle inaccepting effect models. Uncertainty and probabil-ity inherent in models were identified as an issueby all three groups. Regulators would ideally like tosee modeling that reduces uncertainty as much aspossible, or otherwise, they preferred the most con-servative model. Both industry representatives andacademics stressed that probability distributions pro-duced by models needed to be communicated prop-erly, but a model’s output could not replace a fi-nal decision on management and protection goals,which had to be made by risk managers them-selves. Several participants from regulatory author-ities stressed that the emphasis on reducing uncer-tainty is strongly interwoven with social and psy-chological factors accompanying management deci-sions. Respondents mentioned that registration of

pesticides was “about life,” so the final decision-making process could not be based on scientific crite-ria alone—for instance, they stressed that decisions,risk managers had to make were strongly connectedwith valuing the environment (from an anthropocen-tric or ecocentric perspective), so the ERA schemeshould accommodate that fact.

3.2.2.5. ERA is flawed. Many participants men-tioned that mechanistic effect modeling was the fu-ture of ERA of pesticides. Still, the problems of un-certainty and acceptance of models were reinforcedby the current risk assessment scheme, namely bythe lack of clear protection and management goals.Both modelers and model reviewers found it prob-lematic to translate some exceptionally vague legaltexts into modeling input and output. Illustrating theproblem, participants gave two particular examplesof “protecting ongoing behaviour of (. . .) species”and “no unacceptable effects on the environment”(Reg. (EC) 1107/2009, Ref. 3). Participants stressedthat the future acceptance of models had to be pre-ceded by having clear and specific protection goals.

3.2.3. Criteria to Meet.

We asked the participants what formal criteria amodel had to fulfill to be accepted. All three groupsmentioned validation among the main criteria. A re-alistic output turned out to be important as well, andwas mentioned more frequently than, for instance,transparency. Models also needed to be calibratedand accurate, in terms of being able to repeatedly re-turn similar output results from the same input pa-rameters. Stakeholders from industry also stressedthat a model needed to be scientifically robust.

Model validation was one of the most fre-quent issues addressed by our interviewees. Allthree groups expected models to be validated some-how, but the detailed discourse on validation var-ied greatly. First, modelers themselves presented arigid definition of validation, and at the same time,they were the most skeptical about it. Some mod-elers mentioned that it might not be possible at allto thoroughly validate a model against experimentaldata. If a model was expected to be realistic, extrap-olate to different conditions, and project results overlong time spans (e.g., 100 years), such extensive ex-perimental designs were either not available at themoment or virtually impossible to conduct.

Modelers also stressed that a model needed tobe verified and tested in the first place, and parts


Table II. What Criteria Does a Model Need to Fulfill to be Used?

Academia/ Industry/ Regulators/Number of Number of Number of

Keywords Respondents Respondents Respondents

Calibrated 3/3 2/1 1/1Realistic 18/13 3/2 6/5Robust 2/2 4/3 0/0Simple 14/12 10/10 4/3Tested 10/10 2/2 0/0Transparent 12/10 5/5 4/3Understandable 42/13 42/15 19/14Validated 15/9 15/14 22/14Verified 9/7 2/2 0/0

Numbers are frequencies of keywords present in raw text data,taking into account the context the keywords appear in. Inter-views were divided into three groups (regulators, academia, in-dustry) before the keywords were counted for each group sepa-rately. We also counted the number of interviews where keywordswere present, separately for each group. Keywords are sortedalphabetically.

of it should be validated wherever possible, but theend users had to be aware that there were limita-tions to validation possibilities. In contrast, the ma-jority of the nonmodelers expected that effect modelswould be thoroughly validated against experimen-tal data. Validation was the most important criterionfor our regulatory respondents, followed closely byacademics and industry. Yet, “validation” had dif-ferent meanings for different participants—some ofthem defined it in very general terms, as “testing themodel.” In Table II we compared the most impor-tant criteria mentioned by our respondents across thethree groups.

Not taking into account the obvious between-group differences in the absolute number of key-words, it was apparent that key criteria were differentin each group. Academics put understanding of mod-els in first place, followed by realism, validation, andsimplicity. For industry, the need to have their mod-els understood was the most important factor just be-fore validation. Models also needed to be simple andscientifically robust. For regulators, the most impor-tant criteria were validation, understandable inter-face, and realistic output.

3.3. “Rules of the Game”: Who Can Change theStatus Quo?

According to our participants, communicationof models and modeling was key. There were, how-ever, two things that needed to be accounted for be-

forehand. First, although informal opportunities forcommunication were, according to our interviewees,sufficient, communication flow between the threegroups was perceived as flawed. Respondents fromindustry pointed out that they did not get enough op-portunities to discuss and exchange feedback on theirsubmissions. Regulatory representatives mentionedthat the risk assessment scheme was getting more andmore complicated and overly scientific, while the re-ality of management decisions was not taken into ac-count. Respondents from academia told us that eventhough their models were used in regulatory sub-missions, they hardly ever received any feedback onthem. Second, although the discourse on ecologicalmodels was in principle similar in all three groups,there were some important differences, which mighthinder communication.

3.3.1. Shared Views and Different Priorities.

After a closer look at Table II it is apparent that,although in general our participants talked aboutsimilar things and shared the same views, they pri-oritized different issues. For example, modelers—both from academia and industry—put much efforton making models user-friendly, that is, simple, un-derstandable, and transparent. But for the users fromthe regulatory community, simple models were not atop priority, as long as they were reliable to run—thoroughly validated, understandable, and allowingflexibility of input parameters.

The call for realistic output of ecological mod-els was another shared view. Again, every groupwanted an increased realism in ERA, but it turnedout that “realistic” meant different things to differ-ent stakeholders. Academics wanted realistic mod-els that would be ecologically relevant, that is tosay, showed what happened in real ecosystems as ac-curately, as possible. Regulators, in their need forrealism stressed that a realistic model was as con-servative as possible, and, especially in case of manyuncertainties allowed to run a worst-case scenario.For industry, “realistic” was something that was notoverly simplistic, scientifically robust, and most im-portantly aiding the decision-making process in thebest possible way.

Yet, another issue important for every partic-ipant was the need for better communication ofmodels. The majority of stakeholders believed thatimproved communication is key to widespread ac-ceptance of population models. However, each grouphad their own take on communication, and perceived

8 Hunka et al.

the role of the two other groups according to theirown communication preferences. Academics pre-sented a slightly top-down approach—they wanted toeducate others about modeling and to “make them”understand the principles of effect models. Partici-pants from industry expected communication to takea form of training in models. At the same time regu-lators looked forward to having an open dialogue onmodels and modeling—they not only wanted to un-derstand models but also to take a more active partas a communication partner.

3.3.2. What/Who Prevents the Use of Effect Modelsin ERA?

Who has the power to change the current situ-ation? Is mechanistic effect modeling the future ofERA? The majority of our participants agreed thatmodeling was the way forward. Regulators were es-pecially optimistic and believed that sooner or laterecological models would be used on the effect sideof ERA, the same way as fate models are used onthe exposure side. Our participants mentioned sev-eral ways to change the status quo.

Academics perceived themselves as a groupthat slowed down models’ acceptance. They stressedthat first, there was no agreement on modeling ap-proaches. In addition, the way research was fundedand organized naturally encouraged diversity—it waseasier to get funding for innovations than for ap-plying established methods. Some participants fromacademia pointed out that they perceived their roleas constantly coming up with new modeling meth-ods for ERA. On the other hand, respondents saidthat if some models were a part of widely acceptedmainstream science, they would be easily accepted byother stakeholders that were generally expected tokeep up with scientific developments. On the otherhand, industry participants pointed out that effectmodels suffered from lack of trust—they believedthat models could be perceived as a way of “massag-ing” data in higher tier risk assessment, and that mod-eling approaches were not accepted because theywere always submitted by industrial applicants.

All three groups agreed that the best way to in-troduce ecological models into ERA of pesticides forgood would be to work on guidance documents forthem. Although models could be used in higher tierassessment, their legal status was not clear to manyregulators we interviewed. Ideally, our participantswould like to see all three-stakeholder groups work-ing together on guidance for ecological models sim-

ilar to the process under which the FOCUS modelswere developed.

4. DISCUSSION

Perhaps the most important take-home messagefor modelers in our study is to be aware of expec-tations end-users and evaluators have about mecha-nistic effect models. Our results show that attitudestoward models among the majority of risk assessorsand managers are generally positive, but whether amodel is going to be successfully used in pesticideERA, depends on whether the model meets the spe-cific wants and needs of the regulatory community.

Modelers have to be aware of contradicting ex-pectations about their models as well. The need forcomplex scenarios and simplicity at the same timecould be, for instance solved by restricting the sim-plicity to a transparent and easily accessible doc-umentation, and user-friendly interface, while themodel itself could operate on a complex scenario andmany input parameters. The need for realistic andgeneric situations at the same time is more difficult toaddress. Our participants themselves proposed a wayto solve this problem by suggesting that effect modeldevelopment should follow FOCUS group footsteps,where generic landscape and climate scenarios weredeveloped for different parts of Europe. Combiningthe probabilistic output with the requirement of an-swering binary questions seems to be impossible toaddress at this moment, although there have beensome attempts to provide a framework for address-ing uncertainty in ERA.(26) Whereas the FOCUSgroup was suggested as a good example to follow inmodel development, we believe that the familiarityof FOCUS models can be partially blamed for theexpectations of simplified output from effect models.Probably an increased overall confidence and generalacceptance of effect models would partially solve theproblem, but the desire for an output similar in cer-tainty to the FOCUS-type fate models is one of themore serious obstacles in effect models’ use. How-ever, we need to keep in mind that effect models arejust a tool, and they can only aid, but never replacethe decision-making process in risk management.

The discourse on effect modeling has a sound ba-sis in a growing number of models available and cur-rent modeling developments,(6) yet our study showsthat the three stakeholder groups involved in mod-eling speak different languages: it is important tomention that some of the obstacles we found maybe triggered mostly by different understanding of


certain concepts, such as “validation.” Thus, despitethe declared enthusiasm, we found some serious ob-stacles that need to be addressed before mechanis-tic effect modeling can be implemented in ERA. In-terestingly, as the majority of our respondents werepositive about effect modeling, the obstacles we iden-tified were, in most cases, external to models andmodeling itself. Models were generally perceived assomething useful and benign, whereas what hindertheir acceptance were human and procedural factors.

Most importantly, the lack of guidance on ef-fect models and modeling, which is additionally re-inforced by the flaws in the ERA procedures, wasidentified as one of the main issues by our regula-tory respondents. Whereas we found a number ofsignificant attempts to provide documentation proto-cols on effect models for ERA,(6,27,8) our respondentsfrom regulatory authorities sometimes shunned ef-fect models only because there was no guidanceavailable on how to use and evaluate them. On theother hand, the wish for clear guidance can eas-ily lead to very prescribed solutions, which leaveno place for development of new models. More-over, stakeholders agreed that they did not wantmore complicated procedures to follow and paper-work to handle, yet they wanted more guidance doc-uments on modeling. It is then a challenge for mod-elers to provide documentation which would fulfillboth criteria—setting out an understandable frame-work, without being overly complex and bureau-cratic, whereas the actual development of regulatoryguidelines for using effect models in ERA obviouslyhas to be carried out in a forum with the relevantstakeholders (i.e., including modelers, regulatory au-thorities and other end users).

The second problem that is vital for effectmodels’ popularity is the issue of protection goals.Risk managers expect modelers to provide themwith models returning applicable, clear-cut answers,whereas modelers expect risk managers to providethem with clear management goals that can be mod-eled. Neither can be achieved as long as protectiongoals for pesticide ERA are not clearly set. TheEFSA Panel on Pesticides provided some solutionswith the ecosystem services concept.(4) Hopefullythe guidance on protection goals can help to solvethe problem for risk managers and modelers alike.The issue of protection goals is strongly linked withgeneral attitudes toward the environment and itsuse.(28) As some risk managers mentioned, decisionsthey were making were, first “about life.” Effect mod-els, especially population or individual-based models

in many cases provide an output of recovery proba-bility after a pesticide application, so the implicit as-sumption is indeed about survival (“life”) of the mod-eled species. Growing environmental concern,(29) ifcombined with uncertainty in the model outcome,is an additional obstacle that can prevent model ac-ceptance. Empirical information on pesticide effectsclearly seemed to carry more weight for our regula-tory respondents than the information provided byoutputs of effect models.

Finally, we found that although model makersand model users are concerned about similar prob-lems, their values and priorities differ. This was espe-cially visible in the validation discourse. Although allrespondents stressed that validation was at the topof the modeling agenda, the frequency of keywordsproved otherwise. It seems that the problem lies indifferent “validation discourses” each group is using.A validated model for a modeler does not have to bethe same as for a stakeholder with a regulatory per-spective. The problem clearly has to be addressed,but looking at our results it seems that, for instance,regulators expect effect models to be well tried andtested, reliable to run, based on sound scientific prin-ciples, and widely accepted by the scientific commu-nity.

Perhaps one of the most practical and applicablefindings of our study is the call for the next FOCUS-like group working on population models, wordedby many of our interviewees. The FOCUS groupworked from 1992(2) until 2009, developing fate mod-els and scenarios, which are presently used in the ma-jority (but not all) of the EU member states. The FO-CUS group was a community of academics, industryrepresentatives, risk assessors, and managers work-ing together on addressing the problems of pesticidefate in different environmental compartments. Ourrespondents wanted effect models to be developed ina similar manner. One has to be cautious, however,of the different communication requirements. In ourstudy, regulators are the group expecting somethingclosest to a round-table dialogue, whereas academicswould rather take a role of teachers, and industryprovides training. Finding a common platform forcommunication is key, and it seems that the FOCUSgroup indeed set an example for effect models inERA.

Overall, the stakeholder processes in ERA ofpesticides are no different from other decision-making areas where all interested parties are some-how involved in reaching a final consensus.(30) Themain issue present in all stakeholder-based decisions

10 Hunka et al.

is the variety of factors influencing the final out-come.(31) Our respondents wish for the ecotoxicolog-ical decisions to be based on purely scientific results,but there are many other, socioeconomic, psycho-logical, and political factors that are accounted forwhen the final consensus is reached. There are, how-ever, no systematic studies analyzing the exact shareof all components in the decision-making process. Inthe area of technological hazards, one of the moreimportant issues is social amplification of risk.(32)

Pesticides risk can be amplified, for instance, bymass-media focusing on catastrophic events. In turn,perceived risks influence the stakeholders’ take onthe ERA of pesticides.(33) Therefore, a participa-tory approach in environmental matters is sometimesquestioned for its quality and efficiency. For instance,in a study of regulatory decisions concerning haz-ardous waste, Viscusi and Hamilton(34) found thatthe efficiency of decisions decreased with the increas-ing emphasis on political power. However, Beierle(31)

conducted a large, comparative case study into thequality of stakeholder-based decisions. He found thatstakeholders’ involvement in fact results in overallimproved decisions (compared to status quo). More-over, he also found that different stakeholders haveaccess and use technical and scientific resources invarious environmental decisions.

In our study, we tried to reach as many stake-holders as possible and collect a wide variety of re-sponses. However, there are some limitations to ev-ery qualitative research design. We have to take intoaccount, for instance, that our respondents were vol-unteers and they had not received any remunerationfor their time. It may have biased the overall positiveattitude toward effect modeling in pesticide ERA,because our interviewees took part in the study dueto their own interests in mechanistic effect models.We tried to balance that effect by continuous recruit-ment during the data collection process, where westarted working with text data and kept interview-ing new participants at the same time.(22) Moreover,we tried to reach a wide variety of prospective re-spondents during ecotoxicology-related conferences,workshops, and project meetings, which allowed usto recruit not only modelers, but also respondents,who have never come across effect models in ERA.

There are many questions with regard to the useof effect models that seem worth answering. One ofthe possible research directions would be to study theconnection between the development of protectiongoals and guidance on effect modeling. The majorityof our regulatory respondents pointed out that both

the guidance on modeling and opinions on protectiongoals(4) are much awaited and it would be interestingto see how the protection goals(4) can be accommo-dated into population modeling.

The need for further guidance and more dia-logue among stakeholders, which is apparent fromour analysis, is being more and more recognized,and several initiatives to solve these issues are tak-ing place, for instance within the Society of En-vironmental Toxicology and Chemistry (SETAC).SETAC workshops have already been the arenafor discussions among stakeholders within the fieldsof ecotoxicology and ecological risk assessment ofchemicals, with outcomes that directly led to guid-ance documents currently being used in ERA of pes-ticides (see for instance Ref. 35). The SETAC Eu-rope advisory group MEMORISK(36) is now active asa forum for communication through the organizationof group meetings and workshops as well as a realadvisory group for regulators willing to use models inrisk assessment. Their latest effort is the organizationof a workshop to provide a transparent overview ofthe current state of science related to ecological mod-eling and developing experimental/regulatory guid-ance for when and how to apply ecological models toregulatory risk assessment of pesticides.

In summary, effect modeling can play an im-portant role in ERA of pesticides. However, it maytake years before the models are widely used and ac-cepted. The process can be accelerated by a closercooperation between the most important stakeholdergroups. Our analysis suggests that the needs of differ-ent stakeholders often overlap and thus that there is agood chance that consensus on the role and require-ments of ecological modeling for risk assessment canbe reached, but at the same time modelers have to re-vise their own priorities to meet the expectations ofmodel users. Although we have used ERA for pes-ticide registration as a case study, most, if not all,of the issues raised by stakeholders will apply to ef-fect models used in ERA for other legislative pur-poses (e.g., REACH, Water Framework Directive,etc.), and we believe that our findings have broadrelevance.

ACKNOWLEDGMENT

This research has been financially supported bythe European Union under the 7th Framework Pro-gramme (project acronym CREAM, contract num-ber PITN-GA-2009–238148).


APPENDIX: INTERVIEW GUIDE

All interviews were semi-structured and kept asopen as possible. The guide was used as a frameworkfor each conversation.

1. Introduction/opening questions

� What is your involvement in risk assess-ment/management?

� What are your responsibilities?

2. Networking/relationship with other stake-holders

� Who are your main partners?

3. Risk assessment/management

� What do you see as the biggest strength andthe biggest weakness of the current risk man-agement practice?

� What are the most important consequences ofoverestimating and underestimating the pes-ticide risk?

4. Protection goals

� What is it that risk assessors are trying to pro-tect?

� What are the protection goals?

5. Ecosystem services1

� Are you familiar with the ecosystem servicesconcept?

� Is it useful for the development of protectiongoals?

6. Models’ familiarity

� Do you use models in your work?

� Do you use models to predict effects?

1 This theme has been added later during the course of the study,hence used only in 30 interviews.

7. Attitude toward models and modeling

� What do you think of model’s use in risk as-sessment?

8. Criteria—expectations

� What criteria does a model need to fulfill to beused?

� What prevents people from using ecologicalmodels?

9. Policy/regulation change

� What changes in pesticide risk policies haveyou observed in recent (10) years?

10. Societal trends behind policy changes

� What is, in your opinion, a general drive forthese changes?

� What values, in your opinion, influence pesti-cide risk policy?

11. Relevance/sufficiency of regulations

� Are you satisfied with current pesticide riskregulations?

� Had you an opportunity, what would youchange in risk regulations?

12. Stakeholder involvement

� Who, in your opinion, should be given avoice?

13. Risk communication

� Any room for improvements?

14. Transparency of risk assessment

� Do you think there is anything in the cur-rent risk assessment that is not transparentenough?

15. Risk perception: public versus experts

� What is your perception of pesticide risk?� What does “general public” think of pesti-

cides in your opinion?

12 Hunka et al.

REFERENCES

1. Boesten JT. Influence of dispersion length on leaching calcu-lated with PEARL, PELMO and PRZM for FOCUS ground-water scenarios. Pest Management Science, 2004; 60(10):971–980.

2. Boesten JT, Businelli M, Delmas A, Edwards V, HelwegA, Jones R, Klein M, Kloskowski R, Layton R, MarcherS, Schaefer H, Smeets L, Styczen M, Russell M, Travis K,Walker A, Yon D. Leaching models and EU registration.1694/VI/95: Commission of the European Communities, 1995.

3. European Commission. Regulation (EC) No 1107/2009 ofthe European Parliament and of the Council of 21 October2009 concerning the placing of plant protection products onthe market and repealing Council Directives 79/117/EEC and91/414/EEC. In: L309/1. Series Regulation (EC) No 1107/2009of the European Parliament and of the Council of 21 Octo-ber 2009 concerning the placing of plant protection productson the market and repealing Council Directives 79/117/EECand 91/414/EEC. Official Journal of the European Union,2009.

4. EFSA Panel on Pesticides into European Food Safety Au-thority. Scientific opinion on the development of specific pro-tection goal options for environmental risk assessment of pes-ticides, in particular in relation to the revision of the Guid-ance Documents on Aquatic and Terrestrial Ecotoxicology(SANCO/3268/2001 and SANCO/10329/2002). EFSA Jour-nal, 2010; 8(10):1821–1876.

5. Forbes VE, Calow P, Sibly RM. The extrapolation problemand how population modeling can help. Environmental Toxi-cology and Chemistry, 2008; 27(10):1987–1994.

6. Grimm V, Ashauer R, Forbes V, Hommen U, Preuss T,Schmidt A, van den Brink P, Wogram J, Thorbek P. CREAM:A European project on mechanistic effect models for ecolog-ical risk assessment of chemicals. Environmental Science andPollution Research, 2009; 16(6):614–617.

7. Grimm V, Berger U, Bastiansen F, Eliassen S, Ginot V,Giske J, Goss-Custard J, Grand T, Heinz SK, Huse G, HuthA, Jepsen JU, Jørgensen C, Mooij WM, Muller B, Pe’er G,Piou C, Railsback SF, Robbins AM, Robbins MM, Ross-manith E, Ruger N, Strand E, Souissi S, Stillman RA, VabøR, Visser U, DeAngelis DL. A standard protocol for de-scribing individual-based and agent-based models. EcologicalModelling, 2006; 198(1–2):115–126.

8. Schmolke A, Thorbek P, Chapman P, Grimm V. Ecolog-ical models and pesticide risk assessment: Current model-ing practice. Environmental Toxicology and Chemistry, 2010;29(4):1006–1012.

9. Schmolke A, Thorbek P, DeAngelis DL, Grimm V. Eco-logical models supporting environmental decision making: Astrategy for the future. Trends in Ecology and Evolution,2010; 25(8):479–486.

10. Bartell SM, Pastorok RA, Akcakaya HR, Regan H,Ferson S, Mackay C. Realism and relevance of ecologicalmodels used in chemical risk assessment. Human and Eco-logical Risk Assessment: An International Journal, 2003;9(4):907–938.

11. Pastorok RA, Akcakaya R, Regan H, Ferson S, Bartell SM.Role of ecological modeling in risk assessment. Human andEcological Risk Assessment: An International Journal, 2003;9(4):939–972.

12. Starfield AM. A pragmatic approach to modeling forwildlife management. Journal of Wildlife Management, 1997;61(2):261–270.

13. FOCUS Work Group. Generic Guidance for Tier 1 FO-CUS Ground Water Assessments Version: 2.0, 2011; http://focus.jrc.ec.europa.eu/gw/docs/Generic guidance FOCUSGW V2.pdf.

14. EFSA Panel on Pesticides into European Food Safety Au-thority. European Food Safety Authority. Opinion of the Sci-entific Panel on Plant protection products and their residues(PPR) on a request from EFSA on the FOCUS groundwa-ter models comparability and the consistency of this risk as-sessment of ground water contamination EFSA Journal, 2004;93:1–20.

15. European Food Safety Authority. Risk assessment for birdsand mammals. EFSA Journal, 2009; 7(12):1438–1796.

16. Arts B, Leroy P, van Tatenhove J. Political modernisation andpolicy arrangements: A framework for understanding envi-ronmental policy change. Public Organization Review, 2006;6(2):93–106.

17. Veenman S, Liefferink D, Arts B. A short history of Dutchforest policy: The ‘de-institutionalisation’ of a policy arrange-ment. Forest Policy and Economics, 2009; 11(3):202–208.

18. Wiering MA, Arts BJM. Discursive shifts in Dutch river man-agement: ‘Deep’ institutional change or adaptation strategy?Hydrobiologia, 2006; 565:327–338.

19. Forbes VE, Calow P, Grimm V, Hayashi TI, Jager T, KatholmA, Palmqvist A, Pastorok R, Salvito D, Sibly R, Spromberg J,Stark J, Stillman RA. Adding value to ecological risk assess-ment with population modeling. Human and Ecological RiskAssessment: An International Journal, 2011; 17(2):287–299.

20. Marshall M. The key informant technique. Family Practice,1996; 13(1):92–97.

21. Goodman LA. Snowball sampling. Annals of MathematicalStatistics, 1961; 32:148–170.

22. Corbin JM, Strauss AL. Basics of Qualitative Research: Tech-niques and Procedures For Developing Grounded Theory,3rd ed. Los Angeles: Sage Publications, Inc., 2008, 379 pp.

23. Glaser BG, Strauss, AL. The Discovery of Grounded Theory:Strategies for Qualitative Research. Chicago: Aldine Publish-ing Co., 1967, 271 pp.

24. Gasper D, Apthorpe R. Introduction: Discourse analysis andpolicy discourse. The European Journal of Development Re-search, 1996; 8(1):1–15.

25. Hutchby I, Wooffitt R. Conversation Analysis : Principles,Practices, and Applications. Cambridge, Malden, MA: PolityPress, 1998, 273 pp.

26. Hart A, Roelofs W, Crocker J, Mineau P. Addressing Un-certainty and Variability in Pesticide Risk Assessment forBirds and Mammals DEFRA Report, SID 5 (Rev. 3/06). 2006,York, UK.

27. Topping CJ, Høye TT, Olesen CR. Opening the black box—Development, testing and documentation of a mechanisti-cally rich agent-based model. Ecological Modelling, 2010;221(2):245–255.

28. Butler WF, Acott TG. An inquiry concerning the acceptanceof intrinsic value theories of nature. Environmental Values,2007; 16(2):149–168.

29. Stern PC, Dietz T. The value basis of environmental concern.Journal of Social Issues, 1994; 56:121–145.

30. Beierle TC, Cayford J. Democracy in practice: Public par-ticipation in environmental decisions. Washington, DC.: Re-sources For the Future, 2002, 149 pp.

31. Beierle TC. The quality of stakeholder-based decisions. RiskAnalysis, 2002; 22(4):739–749.

32. Kasperson RE, Renn O, Slovic P, Brown HS, Emel J,Goble R, Kasperson JX, Ratick S. The social amplificationof risk: A conceptual framework. Risk Analysis, 1988; 8(2):177–187.

33. Kraus N, Malmfors T, Slovic P. Intuitive toxicology: Expertand lay judgments of chemical risks. Risk Analysis, 1992;12(2):215–232.

34. Viscusi WK, Hamilton JT. Are risk regulators rational? Ev-idence from hazardous waste cleanup decisions. AmericanEconomic Review, 1999; 89(4):1010–1027.


35. Barrett KL, Grandy N, Harrison EG. Guidance document onregulatory testing procedures for pesticides and non-targetarthropods : From the ESCORT workshop (European stan-dard characteristics of beneficials regulatory testing). In TheESCORT Workshop. IAC Wageningen, the Netherlands: SE-TAC, 1994, 51 pp.

36. Preuss T, Hommen U, Alix A, Ashauer R, van den BrinkP, Chapman P, Ducrot V, Forbes V, Grimm V, Schafer D,Streissl F, Thorbek P. Mechanistic effect models for eco-logical risk assessment of chemicals (MEMoRisk)—A newSETAC-Europe Advisory Group. Environmental Scienceand Pollution Research, 2009; 16(3):250–252.