lable at ScienceDirect

Environmental Modelling & Software 55 (2014) 77e91

Contents lists avai

Environmental Modelling & Software

journal homepage: www.elsevier .com/locate/envsoft

An integrated environmental modeling framework for performingQuantitative Microbial Risk Assessments

Gene Whelan a,*, Keewook Kim a,b, Mitch A. Pelton c, Jeffrey A. Soller d, Karl J. Castleton c,Marirosa Molina a, Yakov Pachepsky e, Richard Zepp a

aU.S. Environmental Protection Agency, Office of Research and Development, Athens, GA 30605, USAbU.S. Department of Energy, Oak Ridge Institute for Science and Education, Oak Ridge, TN, USAc Pacific Northwest National Laboratory, Richland, WA, USAd Soller Environmental, LLC, Berkeley, CA, USAeU.S. Department of Agriculture, Agricultural Research Service, Beltsville, MD, USA

a r t i c l e i n f o

Article history:Received 5 April 2013Received in revised form5 December 2013Accepted 19 December 2013Available online 5 February 2014

Keywords:Integrated environmental modelingQMRARisk assessmentPathogensManureWatershed modeling

* Corresponding author. Tel.: þ1 706 355 8305; faxE-mail address: Whelan.Gene@epa.gov (G. Whelan

1364-8152/$ e see front matter Published by Elsevierhttp://dx.doi.org/10.1016/j.envsoft.2013.12.013

a b s t r a c t

Standardized methods are often used to assess the likelihood of a human-health effect from exposure toa specified hazard, and inform opinions and decisions about risk management and communication. AQuantitative Microbial Risk Assessment (QMRA) is specifically adapted to detail potential human-healthrisks from exposure to pathogens; it can include fate and transport models for various media, includingthe source zone (initial fecal release), air, soil/land surface, surface water, vadose zone and aquifer. Theanalysis step of a QMRA can be expressed as a system of computer-based data delivery and modeling thatintegrates interdisciplinary, multiple media, exposure and effects models and databases. Although QMRAdoes not preclude using source-term and fate and transport models, it is applied most commonly wherethe source-term is represented by the receptor location (i.e., exposure point), so the full extent ofexposure scenarios has not been rigorously modeled. An integrated environmental modeling infra-structure is, therefore, ideally suited to include fate and transport considerations and link the riskassessment paradigm between source and receptor seamlessly. A primary benefit of the source-to-outcome approach is that it allows an expanded view of relevant cause-and-effect relationships,which facilitate consideration of management options related to source terms and their fate andtransport pathways. The Framework for Risk Analysis in Multimedia Environmental Systems (FRAMES)provides software technology for analysts to insert appropriate models and databases that fit theproblem statement and design and construct QMRAs that are reproducible, flexible, transferable, reus-able, and transparent. A sample application using different models and databases registered withFRAMES is presented. It illustrates how models are linked to assess six different manure-basedcontaminant sources, following three pathogens (Salmonella eterica, Cryptosporidium spp., and Escher-ichia coli O157:H7) to a receptor where exposures and health risk impacts are then evaluated. Themodeling infrastructure demonstrates how analysts could use the system to discern which pathogensmight be important and when, and which sources could contribute to their importance.

Published by Elsevier Ltd.

1. Introduction

Contamination of recreational/bathing waters by excessiveamounts of fecal bacteria is known to indicate increased risk ofpathogen-induced illness (from bacteria, protozoa, and viruses) tohumans and represents a problem throughout the world. In theUnited States alone, EPA (2002) revealed that 35% of impaired rivers

: þ1 706 355 8302.).

Ltd.

and streams were polluted by fecal bacteria (generally indicated byfecal coliforms, Enterococci, or Escherichia coli) which could indicatethe presence of pathogens. Epidemiology studies have linkedswimming-associated gastrointestinal illnesses with fecal indicatorbacteria (FIB) densities in sewage-impacted recreational waters(Pruss, 1998; Wade et al., 2003; Zmirou et al., 2003); in thosestudies, elevated FIB levels correspond to possible fecal contami-nation (NRC, 2004).

The numbers of pathogenic organisms are often few and diffi-cult to identify and isolate, partly due to their highly varied incharacteristic or type (EPA, 2012a, 2001; NRC, 2004; Savichtcheva

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e9178

and Okabe, 2006). Scientists and public health officials, therefore,typically monitor non-pathogenic bacteria that are associated withfecal contamination and more easily sampled and measured, andregulators impose limits on the amount of microorganisms allowedin waters where humans could potentially be infected (e.g. EPA,2012a; WHO, 2011). The presence of pathogens in manures couldbe highly site- and season-specific and varies greatly by animalhost. E. coli O157:H7 can be found in cattle of different ages, and it ismore abundant in cattle than it is in pigs. In addition, it has wideregional variations (Zhao et al., 1995; Low et al., 2005). For instance,in a study that screened calves of different ages across the UnitedStates, E. coli O157:H7 was present in calves sampled in 11 out of 14states (Zhao et al., 1995). Presence and associated concentration ofSalmonella in beef cattle manures are generally low (<10% and <3MPN g�1 of feces, respectively), although shedding of up to 3 � 103

MPN g�1 of feces has been reported (Fegan et al., 2004). In dairycattle, shedding of Salmonella in feces seems to be a little higher,with numbers ranging from<10 to at least 20% presence dependingon calf age (Huston et al., 2002; Lailler et al., 2005; Berge et al.,2006). Cryptosporidium is typically found at higher concentrationsin calves <4 months old, and although it is also found in adultcattle, the prevalence is rather low in cattle �12 months old (Atwillet al., 1999).

Understanding and simulating how such pathogens get into,travel through, and eventually infect humans is a challengingproblem that involves many different aspects of the environment.To allow regulators to undertake quantified risk assessments, theseprocesses have to be combined flexibly in any software system.

1.1. Quantitative Microbial Risk Assessment for waters

A Quantitative Microbial Risk Assessment (QMRA) characterizespotential human health risk using four pieces of information:average pathogen densities, meanwater ingestion for the exposurescenario, doseeresponse relationships for pathogens and condi-tional probability of illness (Haas et al., 1999; Hunter et al., 2003).The risk assessment approach differs from epidemiological ap-proaches (Calderon et al., 1991; Colford et al., 2012; Haile et al.,1999) in that the latter seek to associate levels of self-reporteddisease (e.g., in a group of swimmers) with the water qualitymeasured by fecal indicator bacteria, and not the etiologicalagent(s) responsible for the disease. Epidemiology studies implic-itly characterize the source of fecal contamination, fate and trans-port kinetics of the microbes, the natural variability of the microbesin the environmental matrix, the etiological agent(s) and exposurescenario studied, while QMRA deals explicitly with these compo-nents. Transparent treatment of these components within the riskassessment framework offer considerable benefits to decision-making and to risk communication and management. QMRAscomplement epidemiological studies (Pruss, 1998; Zmirou et al.,2003) with better interpretation of ambiguous epidemiologicalresults and generating estimates of human-health risk in waterswhere it would be impractical to conduct an epidemiological study.

QMRAs have been used to assess potential health risks from (1)exposure to recreational waters (Ashbolt et al., 2010; Rose et al.,1987; Roser et al., 2006; Soller et al., 2006, 2003); (2) waters con-taining seagull excreta and primary sewage effluent (Schoen andAshbolt, 2010); (3) human enteric viruses (Soller et al., 2010a);(4) the relative contribution of FIB and pathogens when a mixtureof human sources impact a recreational waterbody (Schoen et al.,2011); and (5) fresh gull, chicken, cattle, and swine feces (Solleret al., 2010b). Although QMRAs do not preclude using source-term, fate, and transport models (Benham et al., 2006; Bradfordand Schijven, 2002; Bradford et al., 2006; Bulygina et al., 2009;Guber et al., 2009, 2006; Kim et al., 2010; Kouznetsov et al., 2007,

2004; Pachepsky et al., 2006a,b; Shelton et al., 2003), includinglinkages to exposure/risk at the exposure point (Ferguson et al.,2007a,b; Muirhead et al., 2006; Signor et al., 2007, 2005; Stoutet al., 2005), they most commonly address exposure/risk byassuming that fresh manure was deposited directly into a recrea-tional water (Soller et al., 2010b), without fully characterizing thepotential attenuation during transport of pathogens and FIB fromthe source of release to the point of exposure and impact (McBrideet al., 1998; Soller et al., 2006, 2003).

1.2. Relative and forward QMRAs

EPA (2010) describes two approaches for implementing aQMRA: relative and forward. A forward QMRA has also beenreferred to as conventional or traditional. A relative QMRA com-pares risks from exposure to animal-impacted waters to thoseassociated with human sources (Schoen and Ashbolt, 2010; Solleret al., 2010b). Each fecal source is assumed to contribute enoughcontamination that the hypothetical waterbody contains FIB equalto a predetermined reference density. By setting the referencedensity at a level associated with a known incidence of human-health effects, the risks between animal- and human-basedcontamination are compared. QMRA results can then be used todraw inferences about risks in water impacted by human and an-imal wastes.

A forward QMRA characterizes the risk of illness associated withexposure (EPA, 2010), based on pathogen densities determinedthrough monitoring activities, or by modeling microbial releasefrom sources of contamination and fate and transport to the re-ceptor location. The risk of illness is estimated using pathogendoses and doseeresponse models (Haas, 2002; Haas et al., 1999).The forward QMRA approach is implemented in this study.

1.3. Integrated environmental modeling

The complexity and uncertainty of a QMRA with its differentsources of pathogens, pathways, and receptors is highly demandingand requires an integrated approach. The nascent field of integratedenvironmental modeling (IEM) has recognized this problem andhas been developing solutions by representing and linking models,databases, and visualizations tools in various ways to providecomprehensive and flexible solutions to these complex environ-mental problems (Laniak et al., 2013).

The QMRA studies listed earlier indicate that multiple modelswith varying degrees of scale and resolution were configured withdatabases to construct IEM paradigms. Laniak et al. (2013) note thatIEM helps to solve increasingly complex, real-world problemsinvolving the environment and its relationship to human systemsand activities. The complexity and interrelatedness of real-worldproblems require higher-order systems thinking and holistic solu-tions (EPA, 2008a,b; MEA, 2005; Parker et al., 2002). IEM provides ascience-based structure or framework that develops and organizesmulti-disciplinary knowledge and applies it to explore, explain, andforecast environmental system responses to natural and human-induced stressors. The QMRA framework can be considered a mi-crobial version of the existing chemical risk paradigm (EPA, 2012b,2005a, 2000, 1989, 1986a): (1) problem formulation, includingproblem definition and data collection; (2) occurrence, fate,transport, and exposure assessment of the pathogens; (3) healtheffects assessment including doseeresponse relationships andhealth endpoints; and (4) risk characterization including sensi-tivity, variability, and uncertainty analyses, and evaluation of de-cision points.

One big difference between chemicals and microbes is thatmicrobes are living organisms, resulting in variability and

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e91 79

uncertainty. Variability refers to temporal, spatial, or inter-individual difference in the value of an input (Cullen and Frey,1999). Uncertainty determines how the estimated amount anobserved or simulated value differs from the true value (Lapedes,1978). Characterization of variability is based on knowledge,whereas characterization of uncertainty reflects the extent ofignorance about the true value of a quantity, or the true distributionthat represents the variability (i.e., measure of incomplete knowl-edge) (Cullen and Frey, 1999). Typically, the variability cannot bereduced with additional measurements (Cullen and Frey, 1999),while uncertainty can be more precisely characterized (Haas et al.,1999). Variability manifests itself in detection methods, temporaland spatial heterogeneities in pathogen densities, or variations inintake volumes across individuals (e.g., by age or gender) (Cullenand Frey, 1999; EPA, 2010; Haas et al., 1999). Sources of uncer-tainty in QMRAs include modeling pathogen densities; choice ofparameter distributions in release mechanisms from the manureand fate and transport calculations; knowledge of doseeresponserelationships; and model uncertainty (EPA, 2010).

Matott et al. (2009) cataloged 65 different model evaluationtools for applicability across several model evaluationmethods, andLaniak et al. (2013) presented a summary of methods used formodel evaluation. Model evaluation combines quantitative andqualitative information about a modeling system’s appropriatenessand effectiveness for the problem and ability to characterize theuncertainty of model predictions (Laniak et al., 2013). Uncertainties(e.g., parameter, model, numerical, experimental, scenario, inter-polation, aleatoric, epistemic) can enter mathematical models andexperimental measurements in various contexts (Kennedy andO’Hagan, 2001; Der Kiureghiana and Ditlevsen, 2009; Matthies,2007; Højberg and Refsgaard, 2005; Rojas et al., 2008). An attrac-tive attribute of an IEM infrastructure is its ability to account forvariability and uncertainty analyses; this is typically accomplished

Fig. 1. One possible rendition of QMRA from an integrated, multi-disciplinary multimedia mprocessing; integrated modeling framework with source-to-receptor environmental models,epidemiology studies and policy-related uses.

using a probabilistic framework and characterizing key model pa-rameters through statistical distributions, where parameters ofthose distributions account for variability and/or uncertainty (EPA,2010).

A QMRA’s conceptual design fits well within an integrated,multi-disciplinarymodeling perspective (illustrated in Fig.1) whichdescribes the problem statement, data access retrieval and pro-cessing [e.g., D4EM (EPA, 2013; Whelan et al., 2009; Wolfe et al.,2007)]; software frameworks for integrating models and data-bases [e.g., FRAMES (Johnston et al., 2011)]; infrastructures forperforming sensitivity, variability, and uncertainty analyses [e.g.,SuperMUSE (Babendreier and Castleton, 2005)]; and risk quantifi-cation. By coupling modeling results with epidemiology studies,policy-related issues (e.g., EPA, 2010; EPA and USDA, 2012) can beexplored (Fig. 1), such as what is the risk of illness associated withrecreation at a freshwater beach impacted by agricultural animal(cattle, swine, and chicken) sources of fecal contamination, or howdo those risks compare to risks associated with freshwater beachesimpacted by human sources of fecal contamination (EPA, 2010)?What has been lacking is a flexible software platform from whichmultiple QMRA modeling approaches and paradigms can beconsistently captured, expressed, and exercised, resulting in moretransparency and reproducibility. Reproducibility refers to the dataand computer code producing the same results when made avail-able to others (Peng, 2011), which is different from repeatabilitywhere the same results are reproduced by the same analyst (NMS,2011; Fomel and Claerbout, 2009). Particularly, a modeling infra-structure needs to facilitate users’ ability to link disparate modelsand databases of their choice to support a custom-designedassessment and structure to engage QMRA more fully beyond thepoint of exposure.

This paper presents an example of a QMRA application toaddress multiple microbial source types and organisms that impact

odeling framework perspective that links problem definition; data access, retrieval, andhoused within a sensitivity/uncertainty software structure; risk quantification linked to

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e9180

downstream receptors, using the component-based softwareinfrastructure FRAMES (Framework for Risk Analysis in MultimediaEnvironmental Systems) to link a series of disparate models anddatabases. Although hypothetical, the example is based onmeasured and typical values with references, and assumptionswhere noted. Although FRAMES supports Monte Carlo simulationswith various input values for the source, fate, transport, andexposure modeling, based on the Mersenne twister pseudorandomnumber generator (Matsumoto and Nishimura, 1998), it also allowsthe individual modules to exercise their own Monte Carlo algo-rithms. For this example, a Monte Carlo analysis is demonstratedwith the intake and risk assessment which is exercised within andbased on Mathcad (PTC, 2013) algorithms.

2. Materials and methods

FRAMES demonstrates source-to-outcome microbial exposure and riskmodeling for an agriculture-contaminated runoff scenario in a hypothetical water-shed, based on Whelan et al. (2010). The example consists of multiple fecalcontamination sources present within the same watershed. Because the problemdescription dictatesmodel and data requirements, its conceptualization is presentedprior to model descriptions.

2.1. Problem description

This problem assumes that six potential sources of manure-based microbialcontamination (Fig. 2) exist in a watershed: (1) pathogens entering the catchmentthrough tributary inflow, (2) an animal farming operation with grazing cattleconstantly shedding on an open field, (3) a concentrated animal feeding operationwaste lagoon/pond leaking to an aquifer, (4) pond (lagoon) overflow during largeprecipitation events, resulting in direct discharge to the stream, (5) periodic landapplication of pond contents, and (6) cattle shedding directly to the stream. Rainfallevents drive contamination from the two runoff-related sources (grazing cattle andland application of pond waste), and the other sources are affected by agriculturaloperations and practices. In each case, fecal contamination enters stream segmentsat different locations and flows downstream to a receptor location (denoted as

Fig. 2. Example schematic, describing the physical layout of the receptor location and siwatershed: (1) pathogens entering the catchment through tributary inflow, (2) an animaconcentrated animal feeding operation waste lagoon/pond leaking to an aquifer, (4) pond ovperiodic land application of pond contents, and (6) cattle shedding directly to the stream (

Recreator), resulting in exposure, intake, and potential health impacts. Assumptionsassociated with this assessment include:

� Ten evenly spaced (wevery five weeks) periodic rainfall events over the year,each reflecting a 1-yr return frequency, 24-hr duration, and 9.65 cm depth, usingconditions for Wilmington, NC (NOAA, 2009, 2004).

� Overland transport due to precipitation-driven runoff events from a 180-acsquare subcatchment, where contents of a single waste pond are emptied fourtimes per year.

� Overland transport due to precipitation-driven runoff events from 360 cowscontinually grazing on 180 acres, with a shedding rate of 24 kg/d/cow (Dorneret al., 2004; EPA, 2010).

� Constant leakage to the aquifer from a manure slurry storage pond, at the unithydraulic gradient, over approximately 1% of the pond’s area.

� Continuous replenishment (i.e., no die-off) of the storage pond between quar-terly applications.

� Overflow of the storage pond during storm events (ten occurrences per year).� Inflow from an upstream tributary that assumes a two-day lag time.� Consideration of three pathogens (Cryptosporidium, E. coli O157:H7, and Sal-

monella) with their microbial characteristics, presented in Table 1. Inactivation isassumed to account for all mechanisms, so solar inactivation is not singled out,although this may be particularly important on overland and in surface waters[coastal ocean (Silverman et al., 2013), wastewater (Davies-Colley et al., 1999),with one of the earliest publications provided by Downes and Blunt (1877)].

� Prevalence is the fraction of animals infected with each pathogen and excretiondensity is the number of pathogens per weight of fecal material (EPA, 2010). Themathematical product of prevalence, excretion density, shedding rate, andnumber of animals provides the number of pathogens available for transport.

� Probability of infection based on (1) an individual-based risk resulting from asingle exposure event; (2) negligible potential for secondary transmission ofinfection/disease (Soller et al., 2008, 2004) and immunity to infection frommicrobial agents; (3) the doseeresponse function as the critical healthcomponent; and (4) recurring exposure events that are independent of eachother.

Using FRAMES, the following assessment questions were considered: Whichpathogens might be important to recreational receptors, from a risk perspective?When might they be important? Which sources would contribute to their

x potential sources of manure-based microbial contamination within a hypotheticall farming operation with grazing cattle constantly shedding on an open field, (3) aerflow during large precipitation events, resulting in direct discharge to the stream, (5)after Whelan et al., 2010).

Table 1Microbial characteristics (after Whelan et al., 2010).

Parameter Units Salmonella E. coli O157:H7 Cryptosporidium Reference

Inactivation rate (in soils) 1/d 0.23 0.16 0.04 EPA (2010)Inactivation (in surface water) 1/d 1.30 0.54 10 EPA (2010)Distribution coefficient mL/g 9 9 9 Pachepsky et al. (2006a)Prevalence % 10 20 30 EPA (2010)Maximum inflow from tributary g/yr 6.00E�03 6.00E�03 1.39Eþ00 Assumed, 2-day lagExcretion density (Log10) #/g Manure 3 2 2 EPA (2010)Pathogen pond concentration mg/L 8.85E�03 1.77E�03 6.14E�01 Assumed (based on ratio of excretion

density with E. coli, after Rogers et al., 2009)

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e91 81

importance? Based on the above assumptions and problem description, FRAMESand the following software were used in this assessment.

2.2. Framework for risk analysis in multimedia environmental systems

FRAMES is an open-architecture, object-oriented software framework that in-teracts with environmental databases, helps the user construct a real-world work-flow, allows the user to choose the most appropriate models to solve simulationrequirements, and presents graphical packages for analyzing results (Whelan andLaniak, 1998). FRAMES is intended to provide a platform where various modelscan interact with each other and facilitate a “plug-and-play” atmospherewithmodelchoices in site assessments. FRAMES contains “sockets” for a collection of computercodes that simulate elements of transport, exposure, and risk assessment, includingcontaminant source and release to and through overland soils, vadose and saturatedzones, air, surface water, food supply, intake human health impacts, sensitivity/uncertainty, ecological impacts, etc. (Whelan and Laniak, 1998).

A set of FRAMES software requirements, outlining its design, have beencompiled by Buck et al. (2002), Whelan and Nicholson (2002), and Whelan et al.(1997). These requirements are similar to those of other IEM software in-frastructures, as discussed throughworkshops (EPA, 2008a, 2007; Gaber et al., 2008;iEMSs, 2012, 2010; Laniak et al., 2013; Moore et al., 2012; SOT, 2012) and reported inthe literature (Brooks, 1987; Gause and Weinberg, 1989; HarmonIT, 2002; IEEE,1998; Peckham et al., 2013; Wiegers, 1999). The requirements address issues withstandards, component connectivity, linkage protocols, system architecture andfunctionality, and web-based access, all of which facilitate the creation of plug-and-play components from stand-alone models. FRAMES was designed to

� develop and standardize consistent, repeatable protocols to pass data betweenmodels and inside and outside the system, based onmetadata input/output (I/O)and model standards.

� provide tools/editors that help the user link models to the system and registermodel I/O and associated units.

� provide tools/editors to account for unit conversions (e.g., feet to meters).� track units and metadata through the system, and address error handlings,

thereby contributing to quality assurance/quality control.� package simulations (e.g., input data and workflows) so they can be transported

and re-created by different users at different PCs.� provide a user interface to visualize workflows, which are conceptually simple

and easy to understand.� provide functionality to view results in graphical and tabular form.� provide sensitivity/uncertainty capabilities.� address any discipline, although its main use has been environmental.� handle models that are based on multiple programming languages.� assimilate models and data from any discipline.

For additional information, including tutorials for FRAMES and its tools andcomponents, refer to Babendreier and Castleton (2005), Buck et al. (2002), Castletonand Meyer (2009), FRAMES (2013, 2010a,b, 2008, 2006, 2001), Gelston et al. (2004),Hoopes et al. (2005a,b,c,d,e), Johnston et al. (2011), Pelton et al. (2005), Whelan andNicholson (2002), and Whelan et al. (2010, 1997).

2.3. Fate, transport, exposure, and risk models

To illustrate how FRAMES supports a QMRA application, a series of reduced-form models (covering several components of the QMRA paradigm includingdifferent source-term types, watershed, stream, aquifer, and human exposure/risk,designed and built by different developers) were registered to FRAMES and linkedappropriately. A reduced-form model is one that is typically based on analytical orsemi-analytical solutions, while high-fidelity models require numerical solutions.Semianalytical solutions are a combination of analytical and numerical approaches,which simplifies full numerical solutions (e.g., Serrano, 1992).

2.3.1. Release from pondIt is assumed that a disposal storage pond leaks directly to the aquifer below,

over approximately 1% of its area. A unit hydraulic gradient is assumed below the

pond (Hillel, 1971), resulting in constant outflow to the soil medium. As watercontent approaches saturation, the infiltration rate approaches the saturated hy-draulic conductivity.

2.3.2. Watershed modelOverland transport was calculated via the kinematic wave approximation

(Eagleson, 1970), accounting for runoff, soil adsorption/desorption, inactivation ordie-off, mass balance, and mass accumulation in soil over time. This modelcomputed runoff during overland grazing and land application of ponded waste. Asimplewatershedmodel was constructed for these calculations to account for runofffrom different subcatchments due to land application of pond waste and grazingcattle with daily loadings, respectively. By assuming that a single, uniform, planesubcatchment, solutions to the kinematic wave approximation for overland flow,continuity and momentum equations can be written, using of Manning’s equation(Eagleson, 1970; Whelan et al., 1987):

vh=vt þ vq=vx ¼ i (1)

q ¼ ahm ¼�aS1=2=n

�hm (2)

tc ¼�Lið1�mÞ=a

�1=m(3)

hL ¼ it for 0 � t � tc � tr (4)

hL ¼ ðLi=aÞ1=m for tc � t � tr (5)

where h is the depth of overland runoff, q is the excess discharge per unit width, t istime, x is the distance in the flow direction, i is rainfall excess rate, a coefficient thatis a function of overland roughness and slope, m is Manning’s exponent (¼ 5/3), “a”is Manning’s unit conversion (1.49 for English units), S is slope, “n” is Manning’sroughness coefficient, L is the overall length in the flow direction, tc is the time ofconcentration, tr is the duration of the rainfall excess (equaling 24 h), and hL is thedepth of flow at the end of the runoff segment. Integrating the excess runoffdischarge curve, setting it equal to the rainfall excess volume, and solving for thehydrographs runoff end time (te), assuming a linear recession limb results in thefollowing (Whelan, 1980):

te ¼ ½ð2iL� qÞtr=q� þ tc (6)

For grazing cattle, microbial mass loading is assumed to be added continuouslyto the land. On the other hand, pond waste is uniformly spread on the land surfacefour times per year. For land application and grazing cattle, microbial mass loading isaccounted for as storage in the solid phase until the next storm event, at which timeit is linearly partitioned (e.g., Teng, 2012; Sirivithayapakorn and Keller, 2003; Harveyand Harms, 2002) between solid and liquid phases:

Mcw ¼ McT=�1þ K 0

d� ¼ McT=½1þ ðBdKd=nTÞ� ¼ McT=½1þ ðMcs=McwÞ� (7)

where McT, Mcw, and Mcs are total, liquid phase (i.e., runoff), and solid phasecontaminant masses (i.e., counts), Kd

0 is the dimensionless distribution coefficient,Bd is the dry bulk density, Kd is the distribution coefficient, and nT is total porosity.Ginn et al. (2002) have noted that it is difficult to state definitive correlations be-tween bacterial properties and transport, although more sophisticated algorithmshave been proposed to account for attachment/detachment mechanisms, etc. (e.g.,Guber et al., 2013; Ginn et al., 2002).

Assuming that the shape of microbial runoff is similar to the runoff hydrograph,accounting for first-order inactivation, mass balance, and mass accumulation in soilover time and integrating the mass flux curve and setting it equal to the mass in theliquid phase adjusted for inactivation, a microbial runoff curve can be developedwith a peak runoff rate (Qcp) from tc to tr of

Qcp ¼�2Mcwe�ltc

�.ðtr þ te � tcÞ (8)

where l is the first-order inactivation rate constant. The watershed model resultswere exercised using Data Client Editors (DCEs) within FRAMES (DCE, 2010),

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e9182

modules throughwhich boundary condition data can be passed. It was assumed thatsufficient soil moisture was available prior to each major event to ensure that eachevent represented rainfall excess.

2.3.3. Groundwater modelThe aquifer receives pond leakage, and the groundwater model simulates mi-

crobial fate and transport from the bottom of the pond to the edge of the stream.This is based on the three-dimensional dispersive, one-dimensional advectiveequation with inactivation and soil-water partitioning (Laniak et al., 1997; Millset al., 1997; Whelan et al., 1999, 1992):

vC=vt þ�vp=Rf

�vC=vx ¼

h�Dxv

2C=vx2 þ Dyv2C=vy2 þ Dzv

2C=vz2�i.

Rf � lC (9)

Rf ¼ 1þ ðBdKd=nT Þ (10)

where C is concentration; vp is the pore water velocity; Rf is the retardation factor;and Dx, Dy, Dz, are dispersion coefficients in the longitudinal, lateral, and verticaldirections, respectively. As a boundary condition, it consumes the time-varyingmicrobial flux rates from pond leakage. Values for dispersivities (ax, ay, and az forx, y, and z directions, respectively), which are used to define the dispersion co-efficients (i.e., dispersivity times pore-water velocity), are not a well known, areusually treated as fitting parameters (Teng, 2012), have been found to be sensitive toscale (Gelhar and Axness, 1983), and were defined using algorithms reported byBuck et al. (1995) and Mills et al. (1985) (i.e., ax ¼ 0.1x, ay ¼ 0.033x, andaz ¼ 2.5 � 10�4x, where x is the longitudinal travel distance). In comparison, Teng(2012) referenced a large field experiment, where ax ¼ 0.83 (log10 x)2.414.

2.3.4. Surface water modelThe MEPAS surface water model transports microbes to the receptor point,

based on a vertically integrated, steady-state solution to the one-dimensionaladvective, one-dimensional (lateral) dispersive equation with inactivation (Laniaket al., 1997; Mills et al., 1997; Whelan and McDonald, 1996):

uvC=vx ¼ Dyv2C=vy2 � lC (11)

where u is the average stream velocity. As a boundary condition, the surface watermodel consumes the microbial flux rates from overland runoff. The runoff duration(24 h or greater) is nearly constant and sufficiently greater than instream travel timeto the receptor (44e88 min); therefore, much of the inflow appears nearly steady-state. With this model, we assumed that microbes remain in the water column,unaffected by sediments or more sophisticated hydraulics. For situations thatrequire additional sophistication, more robust instream algorithms (e.g., transientstorage-based in Yakirevich et al., 2013) and models should be used [e.g., HEC-RAS(Brunner, 2010), WASP (Ambrose et al., 1987; Di Toro et al., 1983), FVCOM (Chenet al., 2006), etc.].

2.3.5. Exposure/risk modelMicrobial Risk Assessment Interface Tool (MRA-IT) is MathCad-based, integrated

software tool based on the pathogen of interest, exposure, intake, and dose (Solleret al., 2008, 2004). It characterizes human-health risk resulting from ingestion ofreclaimedwater through recreational activities, consumption of crops irrigatedwithreclaimed water, or incidental/inadvertent ingestion of reclaimed water associatedwith golf course/landscape irrigation. Key components include, but are not limitedto, pathogen specification, exposure scenario identification, and doseeresponserelationships. MRA-IT includes individual (static) and population (dynamic) riskmodels (Soller and Eisenberg, 2008); the static risk model was employed foringestion through recreational activities. MRA-IT provides four doseeresponsedefault models which are summarized in Table 2: exponential, beta-Poisson, hy-pergeometric, and Gompertz-log (Conlan et al., 2011; EPA, 2010; Haas, 2002; Haaset al., 1999; Soller et al., 2008, 2004). Dose is a multiple between microbial den-sity and intake volume (Soller et al., 2004) and was used as input to the appropriatedoseeresponse relationship, resulting in a probability of infection and subsequentillness.

Pelton (2009) linked MRA-IT (as a standard FRAMES module) to a simpleexternal UI that provides a dropdown-list, allowing the user to set basic MRA-IT

Table 2Default doseeresponse functions [F(d)], available in the MRA-IT model (after Conlan et a

Model Equation Description

Exponential F(d) ¼ 1 � exp (�rd) Single parameter modeBeta-Poisson F(d) ¼ 1 � (1 þ d/b)�a Two parameter model (Hypergeometric F(d) ¼ 1 � 1F1(a, a þ b, �d) Two parameter model (

Kummer confluent hypequation of the first kin

Gompertz-log F(d) ¼ 1 � exp{�exp[�a þ b ln(d)]} Two parameter model (

d ¼ dose.

input including the type of water treatment, statistical distribution used to modelthe input data, type of exposure scenario, tolerance for error, number of realizationsassociated with the MRA-IT Monte Carlo simulation, and simulation starting andending times. MRA-IT retained its interactive functionality for this assessment,which is not unique to MathCad [e.g., Analytica (Pelton et al., 2010)]. The currentversion of MRA-IT lacks upstream fate and transport components and relies on anexternal source to provide pathogen and indicator densities (i.e., concentrations) inthe water column prior to exposure. Because MRA-IT cannot accept inputs of mi-crobial densities from multiple upstream models, FRAMES provides a system toolthat combines multiple-module upstream boundary conditions to produce a singleinput to MRA-IT (Whelan et al., 2006); this forms a time-varying input density curve(see “Sum SW Concentrations” in Fig. 3).

3. Results and discussion

FRAMES supports the simulation of a number of complicatedfate and transport problems, as illustrated by a number of disparateassessments (Johnston et al., 2011; Whelan et al., 2010). Here,FRAMES illustrates its flexibility and applicability to supportQMRAs through a problem that involves six source/managementtypes and three pathogens. The underlying complexities unfoldthrough source-allocation and risk assessment.

3.1. Conceptual site model

Fig. 3 presents a FRAMES-based workflow of the six potentialdisparate sources of manure-based pathogen contamination,routed from their sources to a receptor of concern; each icon inFig. 3 represents a separate model. For this particular problem, thefate and transport of pathogens from each source to the receptorwas simulated separately, resulting in six source models, an aquifermodel, six river-routing models, and an intake/risk model. Becausethe risk model can only accept input from one upstream model, a“Plus-operator Module” (icon titled “Sum SW Concentrations”)temporally combined multiple river outputs at the receptor loca-tion. In addition, microbial characteristics (e.g., name, unit mass)were supplied by the FRAMES Constituent Database (“PathogenDatabase”). The “GIS” module is a required by this Domain forspatial orientation, although it is not needed or used by the selectedmodels, as it is currently a place holder for future functionality. Inall, 17 separate modules were linked, their I/O defined and cross-correlated, where appropriate, to ensure seamless transfer of datathrough the system. The system is flexible enough to allow eachmodel to produce its own unevenly incremented time series anddefine its own set of units; both English and metric units are pre-sented. A summary of the data employed to implement thisworkflow is provided in Tables 1 and 3. Because each model in-cludes input in a variety of units, Tables 1 and 3 intentionally pre-sent them, since they highlight the ability of FRAMES toaccommodate models and databases with different base units.

3.2. Distribution of pathogens

Fig. 4 presents the time-varying loading and storage of patho-gens (i.e., Salmonella, E. coli O157:H7, and Cryptosporidium) on the

l., 2011; EPA, 2010; Haas, 2002; Haas et al., 1999; Soller et al., 2008, 2004).

Pathogen Default values used

l (r) Cryptosporidium r ¼ 0.04 to 0.16 (uniform distribution)a, b) Not used Not useda, b); 1F1 is theergeometricd

E. coli O157:H7 a ¼ 0.08 (point estimate)b ¼ 1.44 (point estimate)

a, b) Salmonella a ¼ 29e50 (uniform distribution)b ¼ 2.148 (point estimate)

Fig. 3. Simulation editor, depicting the conceptual site model for six source terms (overland grazing, direct cattle shedding to stream, land application from pond, known tributaryinput, direct pond release per storm, pond release to aquifer), fate and transport (surface water, groundwater, sum SW concentrations), and health impacts (MRA-IT) at a receptorlocation (after Whelan et al., 2010).

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e91 83

two subcatchments receiving manure waste (pond application tothe land and grazing cattle). As noted in the assumptions, pondcontents were emptied and applied four times a year (years 0, 0.25,0.5, and 0.75), but immediate depletion due to overland runoffoccurred only for years 0 and 0.5. Applications at years 0.25 and0.75 occurred between rainfall events, with no immediate deple-tion due to runoff. Magnitude of the applications overwhelmed anydifferences that pre-existed on the land surface, giving theappearance of four identical curves. First-order die-off followedeach peak (Fig. 4.) The inset in Fig. 4 for Salmonella at year 0.5 in-dicates that microbial runoff had minor impact (<3%) on the totalstorage, and depletion of microbes on the land surface is mainlydue to die-off. On the other hand, grazing cattle continually shed tothe land surface, creating a dynamic equilibrium between cattleshedding to the land surface and pathogen die-off (Fig. 4).

Fig. 5 presents typical time-varying pathogen densities for thefirst four rainfall events (peaks 1 through 4) associated with Cryp-tosporidium at the receptor location, accounting for contaminationfrom all six sources. The shape and timing of these results are verysimilar to those exhibited by Salmonella and E. coli O157:H7 andindicative of the entire one-year simulation. This is the densitycurve exiting from the Plus-Operator module in Fig. 3. The onlysources contributing to contamination at the receptor at all timeswere leakage from the pond (when travel time was overcome) andcattle shedding directly to the stream; all other sources contributedonly when a storm event occurred, although tributary inflow had atwo-day lag time prior to entering the stream.

Fig. 6 provides a source apportionment of these density curvesover a time duration covering the first four storm events for eachpathogen. It also includes results for tributary baseflow whichimpacts the receptor two days later, and baseflowwhich representsthe majority of the time. Using Cryptosporidium as an example(refer to Figs. 4 and 5):

� Year 0 (Storm 1) had the highest densities (see Fig. 5) becausethe rainfall event coincided with simultaneous release of path-ogens from all sources except aquifer inflow, whose contribu-tion turned out to be minor. Forty-seven percent of the peakdensity in Fig. 6 was due to pond application to the land surface,while 25% and 23% were due to pond overflow and overlandgrazing, respectively. Pond overflow assumed that wastebypassed the pond and was released directly into the stream.The other sources minimally contributed to the peak concen-tration because manure applications coincided with the stormevent.

� The density level at Year 0.1 (Storm 2) in Fig. 5 was less becausethe second storm event occurred approximately fiveweeks afterthe first pond application, meaning any land-applied waste hadundergone some degree of die-off which reduced the signifi-cance of its overland runoff. Pond overflow contributed to 45% ofthe peak, while overland grazing and pond applicationcontributed 30% and 18%, respectively (Fig. 6).

� Between the first pond application and the third storm event atYear 0.2, there was an even larger delay (10 weeks) and die-off

Table 3Source and media characteristics (after Whelan et al., 2010).

Parameter Value Units Reference

Animal characteristicsCow density 5 cow/ha Duhigg (2009),

Butler et al. (2008a)Number of cows 360 # AssumedNumber of cows

shedding to stream36 # Assumed 10%

Shedding rate of cow 24 kg/d EPA (2010);Duhigg (2009);Butler et al. (2008a)

Soil characteristicsSoil type Sandy-Loam Butler et al. (2008b)Land bulk density 1.58 g/cm3 Meyer et al. (1997)Land porosity 0.41 fraction Meyer et al. (1997)Saturated hydraulic

conductivity1.17E�03 cm/s Meyer et al. (1997)

Overland flow characteristicsMannings conversion

constant (a)1.49 Eagleson (1970),

Whelan et al. (1987)Mannings roughness

coefficient (n)0.20 Whelan (1980)

Friction slope (Sf) 0.005 AssumedPrecipitation intensity (I) 9.68 cm/d NOAA (2009)Mannings exponent (m) 1.67 Eagleson (1970),

Whelan et al. (1987)Size of overland areas 72.8 ha Assumed squarePrecipitation events

per year10 #/yr Assumed

Pond application characteristicsDepth of pond 3 m AssumedArea of pond 1.44Eþ03 m2 Assumed squareFraction of pond that leaks 0.010 fraction AssumedPond land applications

per year4 #/yr Assumed

Storage basin E. coliconcentration

3.16Eþ06 MPN E. coli/100 mL

Haines andRogers (2006),Rogers et al. (2009)

Flow rate into pond perstorm event (flow ratein ¼ flow rate out)

133 L/d/cow Duhigg (2009),Butler et al. (2008a)

Groundwater characteristicsSoil type Sandy-loam Meyer et al. (1997)Darcy velocity 1 cm/d Assumed

Surface water characteristicsDischarge 42.5 m3/s AssumedWidth 30.5 m AssumedVelocity 0.91 m/s Assumed

Tributary characteristicsLag time of inflow

to main stream2 d Assumed

Maximum tributarydischarge

6.8 m3/s Assumed

Distances: sources to receptorCows shedding into

stream2.40 km Assumed

Onsite applicationof pond waste

3.20 km Assumed

All pond discharges 4.00 km AssumedGrazing cattle 4.80 km AssumedTributary inflow 4.80 km Assumed

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e9184

(Fig. 5), resulting in only a slightly lowered peak because othersources were becoming important: 54% was due to pond over-flow and 34% due to Overland Cattle Grazing (Fig. 6).

� Peak density increased for the fourth storm event relative to thethird event (Fig. 5) because there was only a 2.6-wk (0.05-yr) lagbetween the second pond application and the fourth stormevent at Year 0.3, resulting in less die-off prior to runoff.

Contributions to the peak density were more evenly distributedbetween pond overflow, pond application, and overland cattlegrazing at 37%, 34%, and 23% respectively (Fig. 6).

� Following each storm event, there was tributary inflow (two-day lag) (see Fig. 5), while direct instream shedding andgroundwater inflow, due to the leaking pond, occurred contin-uously. During tributary inflow, 59% of peak densities werecontributed by the tributary and 41% by direct instream shed-ding (Fig. 6).

� Most of the time, base flow occurred between runoff events(Fig. 5), and instream cattle shedding was the dominant source(nearly 100%) of contamination (Fig. 6). Because the peak con-centration from groundwater inflow will not arrive until fiveyears into the simulation, contribution from the leaking pondand its importance will continue to increase gradually.

A similar discussion could be presented for results associatedwith Salmonella and E. coli O157:H7.

3.3. Exposure and risk of infection

MRA-IT is an “event-based” exposure/risk model to estimaterisks where a receptor is potentially exposed to contaminatedwater (e.g., swimming for the day at a beach when one periodicallyenters the water); hence, an event windowmust be defined. In thisexample, an exposure event was chosen following the fourth stormevent and assumed to last over a four-day weekend. The MRA-ITmodel consumes pathogen densities and allows the user to fiteither a lognormal or Weibul distribution to the data; in this case, alognormal distribution was used. The exposure analysis alsoassumed no water treatment, so computed pathogen densities atthe receptor represented the levels of exposure. Standard defaultsfor parameter distributions accompanying the MRA-IT model wereused in defining ingestion volumes; i.e., ingestion volume in mLwas lognormally distributed with a log mean of 2.92 and logstandard deviation of 1.43. Pathogen doseeresponse relationshipsare provided in Table 2, including the default values assign to re-lationships for Cryptosporidium (see EPA, 2006), Salmonella, andE. coli O157:H7 (Conlan et al., 2011; EPA, 2010; Haas, 2002; Haaset al., 1999; Soller et al., 2008, 2004).

A Monte Carlo simulation was performed with 5000 re-alizations, sampling input boundary condition densities from thesurface water simulations and default parameters. Table 4 sum-marizes the Monte-Carlo-based risk assessment for the threepathogens associated with a 1e1½ day exposure window sur-rounding Storm 4 (year 0.3). The risk for infection to Cryptospo-ridium is slightly larger than E. coli O157:H7 and significantly largerthan Salmonella. For example, there is a 50% probability ofexceeding an individual risk of 7.3 � 10�5 and 2.7 � 10�5 forCryptosporidium and E. coli O157:H7, respectively, and a 10% prob-ability (90th percentile in Table 4) of exceeding an individual risk of9.6 � 10�4 and 2.9 � 10�4 for Cryptosporidium and E. coli O157:H7,respectively. For comparison, the risk of illness of 0.03 is associatedwith the current geometric mean recreational water quality criteriabased on water impacted by human sources of contamination(USEPA, 2010, 1986b).

Fig. 7 uses data similar to Table 4 and converts and summarizesthe median risks of infections per 100,000 recreators for all threepathogens during the first four storm events. As one would expect,the highest risk of infection is associated with the first storm eventwhich had the highest densities at the receptor (see Fig. 5, forexample). Likewise, the order of importance per storm event fol-lowed the magnitude of densities associated with them: Storm 1,Storm 4, Storm 3, and Storm 2 (Fig. 5). Cryptosporidium representsthe highest risks e ranging from 5.0 to 15.0 infections per 100,000

Fig. 4. Viable pathogen counts remaining on the land surface from pond application and overland grazing over the simulation window covering 10 precipitation events for Sal-monella, E. coli O157, and Cryptosporidium.

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e91 85

for Storms 1 and 4, respectively e closely followed by E. coliO157:H7, ranging from 2.2 to 8.9 for Storms 1 and 4, respectively. Inall cases, Salmonella risks were de minimus.

Fig. 6 indicates that a single dominant source by pathogen doesnot exist, and importance of source for any given pathogen dependson many factors. Closer inspection of results indicates that thepathogen loading type (e.g., shedding, spreading, pond leakage,etc.), pathogen rate of release, timing of the loading, sequence andtype of transporting media, pathogen characteristics (e.g., preva-lence, excretion density, inactivation rate, and distribution coeffi-cient), timing of rainfall events, and landscape characteristics wereall important in determining which source contributed to thecontamination and pathogen density and degree at the receptorlocation. Although not varied in this example, duration, intensity,and spatial distribution of rainfall and antecedent moisture con-ditions also played important roles. For example, Salmonella’s

Fig. 5. Time-varying Cryptosporidium concentrations at the receptor location, illus-trating the effects during the first four rainfall events, tributary inflow, and base flow.

loading rates are 5 and 3.3 times higher than E. coli O157:H7 andCryptosporidium, respectively, but its half-life is 1.4 and 5.8 timesless than E. coli O157:H7 and Cryptosporidium, respectively. Itsprevalence is less, but its excretion density is higher than otherpathogens. Coupled with differences in doseeresponse relation-ships, it becomes increasingly difficult to assess Salmonella’simportance, relative to the other pathogens, without a modelinginfrastructure.

Likewise, by inspecting the source and media characteristics inTable 1, it is difficult to discern which sources of contaminationmight dominate the risk assessment, and when. Fig. 6 demon-strates that the importance of a given land-based disposal optionmay differ, depending upon when it occurs (relative to runoffevents), how it is applied (continual versus a single application),and how much is applied. The importance of pond application forCryptosporidium diminished from Storm 1 to Storm 3 (Fig. 6), whilethe importance of pond overflow and runoff form overland grazingincreased. Although Fig. 6 provides insight into why and whensources become important, it does not indicate which pathogensare important from a risk perspective. For example, the pathogencounts for Salmonella were higher than Cryptosporidium and E. coliO157:H7, but its risks were de minimus (Fig. 7 and Table 4).

In the inset in Fig. 4, which depicts a more detailed temporaldistribution of Salmonella on the land due to pond waste applica-tion for Storm 6 (year 0.5), modeling results demonstrate thatrunoff has a de minimus impact on the mass that remains on theland surface. Yet pathogens that ran off, as illustrated by E. coliO157:H7 and Cryptosporidium which were lower in number, havean impact on risk at the receptor, as illustrated by Storm 1 in Fig. 7.So reducing pathogen runoff e even for small pathogen mobiliza-tion fractions (EPA, 2010) e will reduce subsequent human healthimpacts, which could be achieved through better managementpractices. Although not pursued here, the modeling and softwareinfrastructure will allow analysts to explore many alternatives.

Fig. 6. Concentration contribution by source at the receptor over a time durationcovering the first four storm events for each pathogen, including results during trib-utary inflow and baseflow between storms 3 and 4.

Fig. 7. Simulated infections for each pathogen associated with a Monte Carlo analysisof the first four storm events using the median risk calculations in MRA-IT.

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e9186

4. Summary and conclusions

Laniak et al. (2013) note that the discipline of integrated envi-ronmental modeling (IEM) is inspired by the need to solve

Table 4Risk of infection associated with a 1.5-day exposure window for storm 4.

Risk level Cryptosporidium E. coli O157:H7 Salmonella

10th Percentile 5.70E�06 2.70E�06 <1.0E�9Median 7.30E�05 2.70E�05 <1.0E�9Mean 5.50E�04 1.30E�04 <1.0E�990th Percentile 9.60E�04 2.90E�04 <1.0E�9

increasingly complex problems and provides a science-basedstructure to develop and organize multi-disciplinary knowledge.A flexible infrastructure described here can support developmentand application of Quantitative Microbial Risk Assessments(QMRAs), allowing analysts to begin assessment at any point in theprocess. As Fig. 1 illustrates, QMRA can include and describe thecomplete spectrum of source/stressor and environmental charac-terizations, transport and fate, density levels, exposures (pathways,duration, intensity, and frequency), dose, and effects, which followsthe EPA (2009) source-to-outcome framework.

The concept is a standards-based modeling infrastructure foranalysts to construct appropriate QMRAs to address problemswhile utilizing their own models and databases. If indicator orpathogen densities are available at the receptor location, fate andtransport analyses do not have to be exercised, unless one wants toknow and manage the sources. QMRA has been applied mostcommonly to the receptor location without an infrastructure toextend analysis upstream of the exposure point (e.g., EPA andUSDA, 2012; Haas et al., 1999). If a broader analysis is required,Microbial Source Tacking (MST) techniques (EPA, 2005b; Rogersand Haines, 2005) can be included because MST assumes thatwith the appropriate method and source identifier, the source ofpollution can be detected. When more physics-/process-based as-sessments require continuous linkage and assessment betweensource and receptor (including fate and transport considerations),the complete source-to-outcome continuum can be addressed. IEMallows us to insert appropriate models and databases to fit theproblem statement, versus fitting the problem statement to a singlechoice of models or solution algorithms. The intent is to design andconstruct QMRAs that are reproducible, flexible, transferable,reusable, and transparent.

A modeling infrastructure described here facilitates linkingdisparate models and databases to support a custom-designedassessment and structure to engage QMRA more fully beyond thepoint of exposure. The Framework for Risk Analysis in MultimediaEnvironmental Systems is a comprehensive infrastructurecomposed of an Application Programming Interface (API), which isa protocol used to facilitate communication using a library thatincludes specifications for routines, data structures, object classes,and variables (API, 2013), that handles execution management, fileinput/output, and a series of editors (i.e., wizards) to allow users toregister and operate components inside and outside the system. Asimple assessment with a series of models and databases assessedsix potential, disparate sources of manure-based pathogencontamination, simulating the fate, transport, and health impactsfrom three pathogens (Salmonella, Cryptosporidium, and E. coliO157:H7) to a recreational receptor at a downstream point ofexposure. An uncertainty analysis was included in this exampleassessment, limited to parameter uncertainty associated with the

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e91 87

exposure, intake, and risk of infection due to pathogens. Given theimportance of different forms of uncertainty (e.g., parameter,model, numerical, experimental, scenario, interpolation), the de-gree of their importance require additional exploration in thefuture.

The model’s infrastructure demonstrates how it may be used byanalysts to discern pathogens of importance, when they may beimportant, and sources which could contribute to their importance.By combining fate and transport modeling with point-of-exposurecalculations, analysts can begin to evaluate the importance ofcomponents with a more holistic analysis that includes manureapplication method, pathogen rate of release, timing of the manureloading, sequence and type of transporting media, pathogen char-acteristics, timing of rainfall events, duration and intensity ofrainfall, antecedent moisture conditions, and landscape character-istics. The potential evaluation and environmental and public-health benefits of best management practices represent anadvantage of an IEM framework, like the one described in thispaper. This combination of factors in a usable form enables regu-lators to more easily quantify risk to human health from contami-nated recreational/bathing waters.

Supporting QMRAs through the use of standardized modelinginfrastructures, like FRAMES, requires some degree of computerknowledge to implement an interdisciplinary modeling paradigm,not to mention if one wants to construct, register, link, and executedisparate models from their basic components. These in-frastructures have made strides to avoid constructs that areperfectly valid as software products but are ugly or even useless asmodeling paradigms (i.e., ‘integronsters’, Voinov and Shugart,2013). Additional efforts are still required for QMRA-based in-frastructures to ensure that confederations of models are notinappropriately linked or modeling constructs do not misrepresentthe problem. Therefore, it is important to combine monitored datainto the assessment process to anchor simulations to some form ofreality. Future efforts are aimed at tying the results of basic researchand field sampling directly to the QMRA modeling paradigm;automating manual processes of data gathering, meeting inputmodel data requirements, and model coupling processes, as muchas possible; providing tools for discovery, access, and integration ofscience data, models, and methods to support QMRA decisionmaking; developing methods for data exchange among QMRAcomponents that resolve potential conflicts (e.g., space, time, ag-gregations of demographics, indicators versus pathogens, etc.); anddeveloping support processes for independent review and QualityAssurance/Quality Control of QMRA applications (e.g., inversemodeling).

Acknowledgments

The United States Environmental Protection Agency (EPA)through its Office of Research and Development collaborated withJohn Ravenscroft of the EPA Office ofWater. It has been subjected toAgency review and approved for publication. This research wassupported in part by an appointment to the Research ParticipationProgram at the EPA Office of Research and Development, admin-istered by the Oak Ridge Institute for Science and Educationthrough an interagency agreement between the U.S. Department ofEnergy and EPA.

Appendix. Software/data availability

Categories of free and nonfree software include the following(GNU, 2013a,b): Free (or open source) software comes withpermission for anyone to use, copy, and/or distribute, eitherverbatim or with modifications, either gratis or for a fee. Most free

software is copyrighted, and the copyright holders have legallygiven permission for everyone to use it in freedom, using a freesoftware license. Public domain software is not copyrighted. Pro-prietary (nonfree) software is licensed under exclusive legal right ofits owner.

FRAMES-2Software Developer Mitch Pelton and Karl CastletonAddress PNNL, Battelle Boulevard, Richland, WA 99354, USATel þ1 509 372 6551 and þ1 970 248 1837Fax (509) 372-4995E-mail Mitch.Pelton@pnnl.gov and Karl.Castleton@pnnl.govFirst available 2005Hardware requirements 700 MHz CPU 512 MB RAMSoftware requirements Windows 95 Or NewerAvailability Open source (no license, source code available upon

request)Cost FreeProgram language Visual Basic and CþþProgram size 700 MB on disk after installationSoftware Access (download FRAMES) http://iemhub.org/resources/

133/

FRAMES DatabaseSoftware Developer Mitch PeltonAddress PNNL, Battelle Boulevard, Richland, WA 99354 USATel þ1 509 372 6551Fax (509) 372-4995E-mail Mitch.Pelton@pnnl.govFirst available 2005Hardware requirements 700 MHz CPU 512 MB RAMSoftware requirements Windows 95 or NewerAvailability Open source (no license, source code available upon

request)Cost FreeProgram language Visual BasicProgram size 500 kb on disk after FRAMES installationForm of database repository MS-AccessSize of database 200 kbSoftware Access (download FRAMES) http://iemhub.org/resources/

133/

Data Client EditorSoftware Developer Mitch PeltonAddress PNNL, Battelle Boulevard, Richland, WA 99354 USATel þ1 509 372 6551Fax (509) 372-4995E-mail Mitch.Pelton@pnnl.govFirst available 2005Hardware requirements 700 MHz CPU 512 MB RAMSoftware requirements Windows 95 Or NewerAvailability Open source (no license, source code available upon

request)Cost FreeProgram language Visual BasicProgram size 500 kbSoftware Access http://iemhub.org/resources/133/ as part of

FRAMES installation

MEPAS Surface Water ModuleSoftware Developer Gene WhelanAddress 960 College Station Road, Athens, GA 30605 USATel þ1 706-355 8305Fax þ1 706 355 8302E-mail Whelan.Gene@epa.gov

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e9188

First available 1987Hardware requirements 700 MHz CPU 512 MB RAMSoftware requirements Windows 95 Or NewerAvailability Open source (no license, source code available upon

request)Cost FreeProgram language FORTRAN 77Program size 700 kbSoftware Access (download EARTH Domain) http://iemhub.org/

resources/133/supportingdocs

MEPAS Groundwater ModuleSoftware Developer Gene WhelanAddress 960 College Station Road, Athens, GA 30605 USATel þ1 706-355 8305Fax þ1 706 355 8302E-mail Whelan.Gene@epa.govFirst available 1987Hardware requirements 700 MHz CPU 512 MB RAMSoftware requirements Windows 95 Or NewerAvailability Open source (no license, source code available upon

request)Cost FreeProgram language FORTRAN 77Program size 550 kbSoftware Access (download EARTH Domain) http://iemhub.org/

resources/133/supportingdocs

Plus OperatorSoftware Developer Karl CastletonAddress PNNL, Battelle Boulevard, Richland, WA 99354 USATel þ1 970 248 1837Fax (509) 372-4995E-mail Karl.Castleton@pnnl.govFirst available 2003Hardware requirements 700 MHz CPU 512 MB RAMSoftware requirements Windows 95 Or NewerAvailability Open source (no license, source code available upon

request)Cost FreeProgram language CþþProgram size 220 kbSoftware Access (download EARTH Domain) http://iemhub.org/

resources/133/supportingdocs

MathCadSoftware Developer PTCAddress 140 Kendrick Street, Needham, MA 02494, USATel þ1 781 370 5000Fax þ 1 781 370 6000E-mail http://www.ptc.com/company/contacts/index.htmFirst available 2006Hardware requirements 700 MHz CPU 512 MB RAMSoftware requirements Windows 2000 or newerAvailability ProprietaryCost $150e$1200Program language MathCad 14.0Program size 665 MBSoftware Access http://www.ptc.com/products/mathcad/mathcad

14-whats-new.htm

MRA-ITSoftware Developer Jeffrey Soller (Soller Environmental, LLC)Address 3022 King Street, Berkeley, CA USATel þ1 510 847 0474

Fax e

E-mail jsoller@sollerenvironmental.comFirst available 2008Hardware requirements 700 MHz CPU 512 MB RAMSoftware requirements Windows 2000 or newerAvailability Open source (no license)Cost FreeProgram language MathCad 14.0Program size 665 MBSoftware Access e

References

Ambrose, R.B., Wool, T.A., Connolly, J.P., Schanz, R.W., 1987. WASP4, a Hydrodynamicand Water Quality Model-model Theory, Users Manual, and ProgrammersGuide. EPA/600/3-87/039. U.S. Environmental Protection Agency, Athens, GA,USA.

API, 2013. Application Programming Interface. <http://en.wikipedia.org/wiki/Application_programming_interface> (last accessed 04.04.13).

Ashbolt, N.J., Schoen, M.E., Soller, J.A., Roser, D.J., 2010. Predicting pathogen risks toaid beach management: the real value of quantitative microbial risk assessment(QMRA). Water Res. 44 (16), 4692e4703.

Atwill, E.R., Johnson, E., Klingborg, D.J., Veserat, G.M., Markegard, G., Jensen, W.A.,Pratt, D.W., Delmas, R.E., George, H.A., 1999. Age, geographic, and temporaldistribution of fecal shedding of Cryptosporidium parvum oocysts in cow-calfherds. Am. J. Vet. Res. 60 (4), 420e425.

Babendreier, J.E., Castleton, K.J., 2005. Investigating uncertainty and sensitivity inintegrated, multimedia environmental models: tools for FRAMES-3MRA. Envi-ron. Modell. Softw. 20 (8), 1043e1055.

Benham, B.L., Baffaut, C., Zeckoski, R.W., Mankin, K.R., Pachepsky, Y.A.,Sadeghi, A.M., Brannan, K.M., Soupir, M.L., Habersack, M.J., 2006. Modelingbacteria fate and transport in watersheds to support TMDLs. Trans. ASABE 49(4), 987e1002.

Berge, A.C.B., Moore, D.A., Sischo, W.M., 2006. Prevalence and antimicrobial resis-tance patterns of Salmonella enterica in preweaned calves from dairies and calfranches. Am. J. Vet. Res. 67 (9), 1580e1588.

Bradford, S.A., Schijven, J.F., 2002. Release of Cryptosporidium and Giardia from dairycalf manure: Impact of solution salinity. Environ. Sci. Technol. 36 (18), 3916e3923.

Bradford, S.A., Tadassa, Y.F., Pachepsky, Y.A., 2006. Transport of giardia and manuresuspensions in saturated porous media. J. Environ. Qual. 35, 749e757.

Brooks Jr., F.P., 1987. No silver bullet: essence and accidents of software engineering.IEEE Comput. 15 (1), 10e18.

Brunner, G.W., 2010. HEC-RAS River Analysis System User’s Manual Version 4.1. U.S.Army Corps of Engineers Hydrologic Engineering Center, Davis CA, p. 766.<http://www.hec.usace.army.mil/software/hec-ras/documentation/HEC-RAS_4.1_Users_Manual.pdf> (last accessed 27.08.13).

Buck, J.W., Whelan, G., Droppo Jr., J.G., Strenge, D.L., Castleton, K.J., McDonald, J.P.,Sato, C., Streile, G.P., 1995. Multimedia Environmental Pollutant AssessmentSystem (MEPAS): Application Guidance e Guidelines for Evaluating MEPASInput Parameters for Version 3.1. PNL-10395. Pacific Northwest Laboratory,Richland, WA, USA.

Buck, J.W., Tolle, D.A., Whelan, G., Mast, T.J., Peffers, M.S., Evers, D.P., Corley, R.A.,Eslinger, M.A., Kirk, J.L., Pelton, M.A., Townsend, C.C., Nishioka, M.G., Kogan, V.,Mahasenan, S., Dorow, K.E., Stenner, R.D., Strenge, D.L., 2002. Design of theComprehensive Chemical Exposure Framework and Identification of ResearchNeeds for American Chemistry Council. PNWD-3184. Battelle, Pacific NorthwestDivision, Richland, WA. <http://mepas.pnl.gov/ACC/title.htm> (last accessed04.04.13).

Bulygina, N., McIntyre, N., Wheater, H., 2009. Conditioning rainfall-runoff modelparameters for ungauged catchments and land management impacts analysis.Hydrol. Earth Syst. Sci. 13, 893e904.

Butler, D.M., Randells, N.N., Franklin, D.H., Poore, M.H., Green Jr., J.T., 2008a. Runoffquality from manured riparian grasslands with contrasting drainage andsimulated grazing pressure. Agri. Ecosyst. Environ. 126, 250e260.

Butler, D.M., Franklin, D.H., Cabrera, M.L., Tasistro, A.S., Xia, K., West, L.T., 2008b.Evaluating aeration techniques for decreasing phosphorus export from grass-lands. J. Environ. Qual. 37, 1279e1287.

Calderon, R., Mood, E., Dufour, A., 1991. Health effects of swimmers and nonpointsources of contaminated water. Int. J. Environ. Health Res. 1, 21e31.

Castleton, K.J., Meyer, P.D., 2009. FRAMES-2.0 Software System: FRAMES 2.0 PESTIntegration (F2PEST). PNNL-18395. Pacific Northwest National Laboratory,Richland, WA.

Chen, C., Beardsley, R.C., Cowles, G., 2006. An Unstructured Grid, Finite-volumeCoastal Ocean Model FVCOM User Manual (Draft), SMAST/UMASSD-06-0602,second ed. Woods Hole Oceanographic Institution, Woods Hole, MA USA, p. 315.<http://woodshole.er.usgs.gov/staffpages/rsignell/rps/share/FVCOM_Manual.pdf>. (last accessed 27.08.13).

Colford Jr., J.M., Schiff, K.C., Griffith, J.F., Yau, V., Arnold, B.F., Wright, C.C., Gruber, J.S.,Wade, T.J., Burns, S., Hayes, J., McGee, C., Gold, M., Cao, Y., Noble, R.T.,Haugland, R., Weisberg, S.B., 2012. Using rapid indicators for Enterococcus to

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e91 89

assess the risk of illness after exposure to urban runoff contaminated marinewater. Water Res. 46 (7), 2176e2186.

Conlan, A.J.K., Line, J.E., Hiett, K., Coward, C., Van Diemen, P.M., Stevens, M.P.,Jones, M.A., Gog, J.R., Maskell, D.J., 2011. Transmission and doseeresponse ex-periments for social animals: a reappraisal of the colonization biology ofCampylobacter jejuni in Chickens. J. R. Soc. Interface 8, 1720e1735.

Cullen, A.C., Frey, H.C., 1999. Probabilistic Techniques in Exposure Assessment: AHandbook for Dealing with Variability and Uncertainty in Models and Inputs.Plenum Press, New York.

Davies-Colley, R.J., Donnison, A.M., Speed, D.J., Ross, C.M., Nagels, J.W., 1999. Inac-tivation of faecal indicator microorganisms in waste stabilisation ponds: In-teractions of environmental factors with sunlight. Water Res. 33 (5), 1220e1230.

DCE, 2010. FRAMES Downloads, EarthDomainDocumentation.zip, Data Client Editor(DCE)_0710.ppt. <http://iemhub.org/resources/133/supportingdocs> (lastaccessed 04.04.13).

Der Kiureghiana, A., Ditlevsen, O., 2009. Aleatory or epistemic? Does it matter?Struct. Saf. 31 (2), 105e112.

Di Toro, D.M., Fitzpatrick, J., Thomann, R.V., 1983. Water quality analysis simulationprogram (WASP) and model verification program (MVP)-Documentation. EPA-600/3-81-044. Duluth, MN, USA. <http://dspace.udel.edu:8080/dspace/handle/19716/1450> (last accessed 27.08.13).

Dorner, S.M., Huck, P.M., Slawson, R.M., 2004. Estimating potential environmentalloadings of Cryptosporidium spp. and Campylobacter spp. from Livestock in theGrand River Watershed, Ontario, Canada. Environ. Sci. Technol. 38, 3370e3380.

Downes, A., Blunt, T.P., 1877. Researches on the effect of light upon bacteria andother organisms. Proc. R. Soc. Lond. 26 (179e184), 488e500.

Duhigg, C., 2009. Health Ills Abound as Farm Runoff Fouls Wells. New York Times.<http://www.nytimes.com/2009/09/18/us/18dairy.html?_r¼1&emc¼eta1>(last accessed 04.04.13).

Eagleson, P.S., 1970. Dynamic Hydrology. McGraw-Hill, New York.EPA (U.S. Environmental Protection Agency), 1986a. Guidelines for the health risk

assessment of chemical mixtures, risk assessment forum, September 24, 1986.Fed. Regist. 51 (185), 34014e34025. Washington, DC. <http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid¼20533#Download> (last accessed 04.04.13).

EPA (U.S. Environmental Protection Agency), 1986b. Ambient Water Quality Criteriafor Bacteria, 440/5-84-002. Office of Emergency and Remedial Response,Washington, DC.

EPA (U.S. Environmental Protection Agency), 1989. Risk Assessment Guidance forSuperfund. In: Human Health Evaluation Manual (Part a), vol. I. Office ofEmergency and Remedial Response, Washington, DC. Interim Final. EPA/540/1-89/002. <http://www.epa.gov/oswer/riskassessment/ragsa/pdf/rags_a.pdf>(last accessed 04.04.13).

EPA (U.S. Environmental Protection Agency), 2000. Supplementary Guidance for Con-ducting Health Risk Assessment of Chemical Mixtures. EPA/630/R-00/002,Washington, DC. <http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid¼20533#Download> (last accessed 04.04.13).

EPA (U.S. Environmental Protection Agency), 2001. Protocol for Developing Path-ogen TMDLs. EPA 841-R-00-002, first ed. Office of Water, Washington, DC.<http://www.epa.gov/owow/tmdl/techsupp.html>. (last accessed 04.04.13).

EPA (U.S. Environmental Protection Agency), 2002. National Water Quality In-ventory: 2000 Report. EPA-841-R-02-001, Washington, DC.

EPA (U.S. Environmental Protection Agency), 2005a. Guidelines for Carcinogen RiskAssessment. EPA/630/P-03/001B, Washington, DC. <http://www.epa.gov/ttnatw01/cancer_guidelines_final_3-25-05.pdf> (last accessed 04.04.13).

EPA (U.S. Environmental Protection Agency), 2005b. Microbial Source TrackingGuide Document. EPA-600/R-05/064. Office of Research and Development,Washington, DC, p. 131.

EPA (U.S. Environmental Protection Agency), January 5, 2006. National PrimaryDrinking Water Regulations: Long Term 2 Enhanced Surface Water TreatmentRule (LT2ESWTR); Final Rule. 40CFR Parts 9, 141 and 142, vol. 71. No. 654.

EPA (U.S. Environmental Protection Agency), 2007. Workshop report: integratedmodeling for integrated environmental decision making workshop. January30eFebruary 1, 2007. Research Triangle Park, NC. <http://www.epa.gov/crem/crem_integmodelwkshp.html> (last accessed 04.04.13).

EPA (U.S. Environmental Protection Agency), 2008a. Workshop report: collaborativeapproaches to integrated modeling better integration for better decision-making. December 10e12, 2008. Phoenix, AZ. http://www.epa.gov/crem/integrated-modeling-workshop2008.html (last accessed 16.10.12).

EPA (U.S. Environmental Protection Agency), 2008b. Integrated Modeling for Inte-grated Environmental Decision Making. EPA-100-R-08-010. Office of the Sci-ence Advisor, Washington, DC. <http://www.epa.gov/crem/integrated-model-paper.html> (last accessed 04.04.13).

EPA (U.S. Environmental Protection Agency), 2009. A Conceptual Framework for U.S.EPA’s National Exposure Research Laboratory. EPA/600/R-09/003,Washington, DC.

EPA (U.S. Environmental Protection Agency), 2010. Quantitative Microbial RiskAssessment to Estimate Illness in Freshwater Impacted by Agricultural AnimalSources of Fecal Contamination. EPA 822-R-10-005. Office of Water, Washing-ton DC.

EPA (U.S. Environmental Protection Agency), 2012a. Recreational Water QualityCriteria. Office of Water, Washington, DC. 820-F-12-058. <http://water.epa.gov/scitech/swguidance/standards/criteria/health/recreation/index.cfm> (lastaccessed 03.09.13).

EPA (U.S. Environmental Protection Agency), 2012b. Risk Assessment Guidance forSuperfund, Part D (RAGS D) Tables. Office of Solid Waste and Emergency

Response, Washington, DC. <http://www.epa.gov/oswer/riskassessment/sghandbook/ragsd.htm> (last accessed 04.04.13).

EPA (U.S. Environmental Protection Agency), 2013. Data for EnvironmentalModeling (D4EM). Office of Research and Development, Athens, GA. <http://www.epa.gov/athens/research/d4em.html> (last accessed 04.04.13).

EPA (U.S. Environmental Protection Agency) and USDA (U.S. Department of Agri-culture/Food Safety and Inspection Service), 2012. Microbial Risk AssessmentGuideline: Pathogenic Microorganisms with Focus on Food andWater. EPA/100/J-12/001; USDA/FSIS/2012-001. Washington, DC.

Fegan, N., Vanderlinde, P., Higgs, G., Desmarchelier, P., 2004. Quantification andprevalence of Salmonella in beef cattle presenting at slaughter. J. Appl. Micro-biol. 97 (5), 892e898.

Ferguson, C.M., Davies, C.M., Kaucner, C., Krogh, M., Rodehutskors, J., Deere, D.A.,Ashbolt, N.J., 2007a. Field scale quantification of microbial transport frombovine faeces under simulated rainfall events. J. Water Health 5 (1), 83e95.

Ferguson, C.M., Croke, B.F.W., Beatson, P.J., Ashbolt, N.J., Deere, D.A., 2007b. Devel-opment of a process-based model to predict pathogen budgets for the Sydneydrinking water catchment. J. Water Health 5 (2), 187e208.

Fomel, S., Claerbout, J., 2009. ‘Guest editors’ introduction: reproducible research.Comput. Sci. Eng. 11 (1), 5e7.

FRAMES, 2001. FRAMES API components. <http://mepas.pnnl.gov/FramesV2/Documents/Developer/index.html> (last accessed 04.04.13).

FRAMES, 2006. FRAMES 2.0, a framework system for model and data integration,analysis, and visualization: tutorial. <http://mepas.pnnl.gov/geniitutorial/index.html> (last accessed 04.04.13).

FRAMES, 2008. FRAMES: Framework for Risk Analysis in Multimedia EnvironmentalSystems (Version 2.x). <http://mepas.pnnl.gov/FramesV2/techdocs.stm> (lastaccessed 04.04.13).

FRAMES, 2010a. FRAMES downloads, supporting docs. <http://iemhub.org/resources/133/supportingdocs> (last accessed 04.04.13).

FRAMES, 2010b. FRAMES downloads. <http://iemhub.org/resources/frames> (lastaccessed 04.04.13).

FRAMES, 2013. FRAMES API classes. <http://mepas.pnnl.gov/FramesV2/Documents/index.html> (last accessed 04.04.13).

Gaber, N., Laniak, G.F., Linker, L., 2008. Integrated Modeling for Integrated Envi-ronmental Decision Making. EPA 100/R-08/010. U.S. Environmental ProtectionAgency, Washington, DC. <http://www.epa.gov/CREM/library/IM4IEDM_White_Paper_Final_(EPA100R08010).pdf> (last accessed 04.04.13).

Gause, D.C., Weinberg, G.M., 1989. Exploring Requirements Quality before Design.Dorset House Publishing, New York.

Gelhar, L.W., Axness, C.L., 1983. Three-dimensional stochastic analysis of micro-dispersion. Water Resour. Res. 19 (1), 161e180.

Gelston, G.M., Pelton, M.A., Lundgren, R.E., Castleton, K.J., Whelan, G., Hoopes, B.L.,Kirk, J.L., Pospical, A.J., Eslinger, M.A., Droppo Jr., J.G., Strenge, D.L., 2004. UsingDictionaries to Manage Data within a Modeling Framework System. PNWD-3507. Pacific Northwest Division, Battelle Memorial Institute, Richland, WA.

Ginn, T.R., Wood, B.D., Nelson, K.E., Scheibe, T.D., Murphy, E.M., Clement, T.P., 2002.Processes in microbial transport in the natural subsurface. Adv. Water Resour.25, 1017e1042.

GNU, 2013a. What is free software?. <http://www.gnu.org/philosophy/free-sw.en.html> (last accessed 28.08.13).

GNU, 2013b. Categories of free and nonfree software. <http://www.gnu.org/philosophy/categories.html> (last accessed 28.08.13).

Guber, A.K., Shelton, D.R., Pachepsky, Y.A., Sadeghi, A.M., Sikora, L.J., 2006. Rainfall-induced release of fecal coliforms and other manure constituents: comparisonand modeling. Appl. Environ. Microbiol. 72 (12), 7531e7539.

Guber, A.K., Yakirevich, A.M., Sadeghi, A.M., Pachepsky, Y.A., Shelton, D.R., 2009.Uncertainty evaluation of coliform bacteria removal from vegetated filter stripunder overland flow condition. J. Environ. Qual. 38, 1636e1644.

Guber, A.K., Pachepsky, Y.A., Dao, T.H., Shelton, D.R., Sadeghi, A.M., 2013. Evaluatingmanure release parameters for nonpoint contaminant transport model KINE-ROS2/STWIR. Ecol. Model. 263, 126e138.

Haas, C.N., 2002. Conditional doseeresponse relationships for microorganisms:development and application. Risk Anal. 22 (3), 455e463.

Haas, C.N., Rose, J.B., Gerba, C.P., 1999. Quantitative Microbial Risk Assessment. JohnWiley & Sons, Inc., New York, p. 449.

Haile, R.W., Witte, J.S., Gold, M., Cressey, R., McGee, C., Millikan, R.C., Glasser, A.,Harawa, N., Ervin, C., Harmon, P., Harper, J., Dermand, J., Alamillo, J., Barrett, K.,Nides, M., Wang, G.Y., 1999. The health effects of swimming in ocean watercontaminated by storm drain runoff. Epidemiology 10 (4), 355e363.

Haines, J.R., Rogers, S.W., 2006. Characterizing and Mitigating Pathogenic OrganismsRelated to CAFOs. LTG 3 Poster 07, BOSC Water Quality Review, January 2006.National Risk Management Research Laboratory, U.S. Environmental ProtectionAgency, Cincinnati, OH. <http://cfpub.epa.gov/si/si_public_record_Report.cfm?dirEntryId¼146424&CFID¼105239217&CFTOKEN¼20266879&jsessionid¼4e30599ac1becaa6e7d1241b4e1b2f675ce2> (last accessed 04.04.13).

HarmonIT, 2002. Requirements Analysis. IT Frameworks (HarmonIT), EC Framework 5.Energy Environment and Sustainable Development, DHI Water and Environment,Horsholm, Demark. <https://4310b1a9-a-cb397f23-s-sites.googlegroups.com/a/openmi.org/home/archives/harmonit/harmonit_wp2_final_report_v2_approved.pdf?attachauth¼ANoY7coKCtE7JCMFHVG2XtO5qMGx_LR8rVyHH3wmF9uJP_9lr958_6F9fufaROTcfhxeTy9jIMlA0PxnRmDLzfSD0W6QgtL3aCFfUAWo6Kt_xOAE-qIWB1KoBveEN1nyBB1wrK5eQORprR7KQcQ0fqqu1qOd7euUlGuobfK60z38kCeBDoQbbBAoJwEweZh1-EyXFh_lUb3fqpLhMRKC60z9eR0P0B99vj3LgdjMzjcSNqp6g5JdIDoUuq415Ks0aGK7bZJSiPV41&attredirects¼0> (last accessed 04.04.13).

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e9190

Harvey, R.W., Harms, H., 2002. Tracers in groundwater: use of microorganisms andmicrospheres. In: Bitton, G. (Ed.), Encyclopedia of Environmental Microbiology,6John Wiley & Sons, Inc., New York, pp. 3194e3202. <http://www.academia.edu/2964825/Tracers_in_groundwater_-_Use_of_microorganisms_and_microspheres> (last accessed 27.08.13).

Hillel, D., 1971. Soil and Water: Physical Principles and Processes. Academic Press,New York.

Højberg, A.L., Refsgaard, J.C., 2005. Model uncertainty-parameter uncertainty versusconceptual models. Water Sci. Technol. 52 (6), 177e186.

Hoopes, B.L., Pelton, M.A., Castleton, K.J., Gelston, G.M., Whelan, G., Taira, R.Y.,2005a. Documentation for Framework Development Environment. PNWD-3509. Pacific Northwest National Laboratory, Richland, WA. <http://mepas.pnl.gov/FramesV2/documents/PNWD-3509/index.html> (last accessed 04.04.13).

Hoopes, B.L., Pelton, M.A., Castleton, K.J., 2005b. Documentation for the DictionaryEditor of the FRAMEwork System (FRAMES). PNWD-3504. Pacific NorthwestDivision, Battelle Memorial Institute, Richland, WA. <http://mepas.pnl.gov/FramesV2/documents/PNWD-3504/index.html> (last accessed 04.04.13).

Hoopes, B.L., Pelton, M.A., Castleton, K.J., 2005c. Documentation for the ModuleEditor of the FRAMEwork System (FRAMES). PNWD-3506. Pacific NorthwestDivision, Battelle Memorial Institute, Richland, WA. <http://mepas.pnl.gov/FramesV2/documents/PNWD-3506/index.html> (last accessed 04.04.13).

Hoopes, B.L., Pelton, M.A., Castleton, K.J., Whelan, G., Gelston, G.M., 2005d. Docu-mentation for the Domain Editor of the FRAMEwork System (FRAMES). PNWD-3508. Pacific Northwest National Laboratory, Richland, WA. <http://mepas.pnl.gov/FramesV2/documents/PNWD-3508/index.html> (last accessed 04.04.13).

Hoopes, B.L., Pelton, M.A., Castleton, K.J., Gelston, G.M., Whelan, G., 2005e. Docu-mentation for the Simulation Editor of the FRAMEwork System (FRAMES).PNWD-3510. Pacific Northwest Division, Battelle Memorial Institute, Richland,WA. <http://mepas.pnl.gov/FramesV2/documents/PNWD-3510/index.html>(last accessed 04.04.13).

Hunter, P.R., Payment, P., Ashbolt, N., Bartram, J., 2003. Assessment of risk. In:Ronchi, E., Bartram, J. (Eds.), Assessing Microbial Safety of Drinking Water:Improving Approaches and Methods. OECD/WHO, Paris, pp. 79e109. OECD/WHO guidance document (Chapter 3).

Huston, C.L., Wittum, T.E., Love, B.C., Keen, J.E., 2002. Prevalence of fecal shedding ofSalmonella spp in dairy herds. J. Am. Vet. Med. Assoc. 220 (5), 645e649.

IEEE, 1998. Recommended Practice for Software Requirements SpecificationsSponsor Software Engineering Standards Committee of the IEEE ComputerSociety. IEEE Std 830-1998. IEEE-SA Standards Board. Institute of Electrical andElectronics Engineers, New York, p. 39.

iEMSs, July 2010. iEMSs Integrated Modeling Technologies Workshop: The Future ofScience and Technology of Integrated Modeling. International EnvironmentalModelling & Software Society, Ottawa, Canada. In: <http://www.iemss.org/iemss2010/proceedings.html> (last accessed 04.04.13).

iEMSs, July 2012. iEMSs Workshop D9: Community for Integrated EnvironmentalModeling (CIEM), Current Status and Future Direction. International Environ-mental Modelling & Software Society, Leipzig, Germany. <http://www.iemss.org/sites/iemss2012/> (last accessed 04.04.13).

Johnston, J.M., McGarvey, D.J., Barber, M.C., Laniak, G.F., Babendreier, J.E., Parmar, R.,Wolfe, K., Kraemer, S.R., Cyterski, M., Knightes, C., Rashleigh, B., Suarez, L.,Ambrose, R., 2011. An integrated modeling framework for performing envi-ronmental assessments: application to ecosystem services in the Albemarle-Pamlico basins (NC and VA, USA). Ecol. Model. 222 (14), 2471e2484.

Kennedy, M.C., O’Hagan, A., 2001. Bayesian calibration of computer models. J. R.Stat. Soc. Ser. B 63 (3), 425e464.

Kim, J.-W., Pachepsky, Y.A., Shelton, D.R., Coppock, C., 2010. Effect of streambedbacteria release on E. coli concentrations: monitoring and modeling with themodified SWAT. Ecol. Model. 221, 1592e1604.

Kouznetsov, M.Y., Pachepsky, Y.A., Gillerman, L., Gantzer, C.J., Oron, G., 2004. Mi-crobial transport in soil caused by surface and subsurface drip irrigation withtreated wastewater. Int. Agrophys. 18, 239e247.

Kouznetsov, M.Y., Roodsari, R., Pachepsky, Y.A., Shelton, D.R., Sadeghi, A.M.,Shirmohammadi, A., Starr, J.L., 2007. Modeling manure-borne bromide and fecalcoliform transport with runoff and infiltration at a hillslope. J. Environ. Manage.84, 336e346.

Lailler, R., Sanaa, M., Chadoeuf, J., Fontez, B., Brisabois, A., Colmin, C., Millemann, Y.,2005. Prevalence of multidrug resistant (MDR) Salmonella in bovine dairy herdsin western France. Prev. Vet. Med. 70 (3e4), 177e189.

Laniak, G.F., Droppo Jr., J.G., Faillace, E.R., Gnanapragasam, E.K., Mills, W.B.,Strenge, D.L., Whelan, G., Yu, C., 1997. An overview of a multimedia bench-marking analysis for three risk assessment models: RESRAD, MMSOILS, andMEPAS. Risk Anal. 17 (2), 203e214.

Laniak, G.F., Olchin, G., Goodall, J., Voinov, A., Hill, M., Glynn, P., Whelan, G.,Geller, G., Quinn, N., Blind, M., Peckham, S., Reaney, S., Gaber, N., Kennedy, R.,Hughes, A., 2013. Integrated environmental modeling: a vision and roadmap forthe future. Environ. Modell. Softw. 39, 3e23.

Lapedes, D.H., 1978. McGraw-Hill Dictionary of Scientific and Technical Terms.McGraw-Hill, New York.

Low, J.C., McKendrick, I.J., McKechnie, C., Fenlon, D., Naylor, S.W., Currie, C.,Smith, D.G.E., Allison, L., Gally, D.L., 2005. Rectal carriage of enter-ohemorrhagic Escherichia coli O157 in slaughtered cattle. Appl. Environ.Microb. 71 (1), 93e97.

Matott, L.S., Babendreier, J.E., Purucker, S.T., 2009. Evaluating uncertainty in inte-grated environmental models: a review of concepts and tools. Water Resour.Res. 45, W06421. http://dx.doi.org/10.1029/2008WR007301.

Matsumoto, M., Nishimura, T., 1998. Mersenne twister: a 623-dimensionally equi-distributed uniform pseudo-random number generator. ACM Trans ModelComput. Simulat. 8 (1), 3e30.

Matthies, H.G., 2007. Quantifying Uncertainty: Modern Computational Represen-tation of Probability and Applications, Extreme Man-made and Natural Hazardsin Dynamics of Structures. In: NATO Security through Science Series, vol. 105e135. http://dx.doi.org/10.1007/978-1-4020-5656-7_4.

McBride, G.B., Salmond, C.E., Bandaranayake, D.R., Turner, S.J., Lewis, G.D., Till, D.G.,1998. Health effects of marine bathing in New Zealand. Int. J. Environ. HealthRes. 8 (3), 173e189.

MEA, 2005. Ecosystems and Human Well-being: Synthesis. Millenium EcosystemAssessment. Island Press, Washington, DC.

Meyer, P.D., Rockhold, M.L., Gee, G.W., 1997. Uncertainty Analyses of Infiltration andSubsurface Flow and Transport for SDMP Sites. NUERG/CR-6565, PNNL-11705.Pacific Northwest National Laboratory, Richland, WA.

Mills, W.B., Porcella, D.B., Ungs, M.J., Gherini, S.A., Summers, K.V., Mok, L., Rup, G.L.,Bowie, G.L., 1985. Water Quality Assessment: A Screening Procedure for Toxicand Conventional Pollutants. EPA/600/6-85/002b. U.S. Environmental Protec-tion Agency, Athens, GA, USA.

Mills, W.B., Cheng, J.J., Droppo Jr., J.G., Faillace, E.R., Gnanapragasam, E.K., Johns, R.A.,Laniak, G.F., Lew, C.S., Strenge, D.L., Sutherland, J.F., Whelan, G., Yu, C., 1997.Multimedia benchmarking analysis for three risk assessment models: RESRAD,MMSOILS, and MEPAS. Risk Anal. 17 (2), 187e202.

Moore, R., Hughes, A., Gaber, N., Geller, G., Glynn, P., Laniak, G.F., Voinov, A.,Whelan, G., 2012. International Summit on Integrated Environmental Modeling.EPA/600/R/12/728jDecember 2012jwww.epa.gov/research and BGS OR/12/087.U.S. Environmental Protection Agency, British Geological Survey, and U.S.Geological Suvey, Washington, DC. <http://cfpub.epa.gov/si/si_public_record_report.cfm?dirEntryId¼248471> (last accessed 04.04.13).

Muirhead, R.W., Collins, R.P., Bremer, P.J., 2006. Numbers and transported state ofEscherichia coli in runoff direct from fresh cowpats under simulated rainfall.Lett. Appl. Microbiol. 42 (2), 83e87.

NMS (National Measurement System), 2011. What Is the Difference between Repeat-ability and Reproducibility?. Chemical and Biological Meteorology Website.<http://www.nmschembio.org.uk/GenericArticle.aspx?m¼223&amid¼905>(last accessed 28.11.13).

NOAA (National Oceanic and Atmospheric Administration), 2004. Point precipita-tion Frequency Estimates from NOAA Atlas 14. National Weather Service, SilverSpring, MD. <http://hdsc.nws.noaa.gov/cgi-bin/hdsc/buildout.perl?type¼pf&units¼us&series¼pd&statename¼NORTHþCAROLINA&stateabv¼sc&study¼orb&season¼All&intype¼1&plat¼&plon¼&liststation¼WILMINGTONþ7þNþþþþþþþþþþþNCþ%2Cþ31-9467&slat¼lat&slon¼lon&mlat¼35.235&mlon¼-75.416> (last accessed 04.04.13).

NOAA (National Oceanic and Atmospheric Administration), 2009. NOAA Atlas 14.Version 3, Ohio River Basin and Surrounding States: North and South Carolinavol. 2. <ftp://hdsc.nws.noaa.gov/pub/hdsc/data/orb/na14orbv3_nc25y24h.pdf>(last accessed 04.04.13).

NRC (National Research Council), 2004. Indicators for Waterborne Pathogens. TheNational Academies. The National Academies Press, Washington, DC.

Pachepsky, Y.A., Sadeghi, A.M., Bradford, S.A., Shelton, D.R., Guber, A.K., Dao, T.,2006a. Transport and fate of manure-borne pathogens: modeling perspective.Agr. Water Manage. 86, 81e92.

Pachepsky, Y.A., Devin, B.A., Polyanskaya, L.M., Shelton, D.R., Shein, E.V., Guber, A.K.,2006b. Limited entrapment model to simulate the breakthrough of Arthro-bacter and Aquaspirillum in soil columns. Int. Agrophys. 20, 207e218.

Parker, P., Letcher, R., Jakeman, A., Beck, M.B., Harris, G., Argent, R.M., Hare, M., Pahl-Wostl, C., Voinov, A., Janssen, M., Sullivan, P., Scoccimarro, M., Friend, A.,Sonnenshein, M., Barker, D., Matejicek, L., Odulaja, D., Deadman, P., Lim, K.,Larocque, G., Tarikhi, P., Fletcher, C., Put, A., Maxwell, T., Charles, A., Breeze, H.,Nakatani, N.,Mudgal, S., Naito,W., Osidele, O., Eriksson, I., Kautsky, U., Kautsky, E.,Naeslund, B., Kumblad, L., Park, R., Maltagliati, S., Girardin, P., Rizzoli, A.,Mauriello, D., Hoch, R., Pelletier, D., Reilly, J., Olafsdottir, R., Bin, S., 2002. Progressin integrated assessment andmodelling. Environ. Modell. Softw.17 (3), 209e217.

Peckham, S., Hutton, E., Norris, B., 2013. A component-based approach to integratedmodeling in the geosciences: the design of CSMDS. Comput. Geosci. 53, 3e12.

Pelton, M.A., 2009. Application Requirements, Design, and Specifications for theIntegration of MRA-IT in FRAMES. PNWD-4122. Pacific Northwest Division,Battelle Memorial Institute, Richland, WA.

Pelton, M.A., Castleton, K.J., Hoopes, B.L., 2005. Documentation for the ConversionEditor of the FRAMEwork System (FRAMES). PNWD-3503. Pacific NorthwestDivision, Battelle Memorial Institute, Richland, WA. <http://mepas.pnl.gov/FramesV2/documents/PNWD-3503/index.html> (last accessed 04.04.13).

Pelton, M.A., Castleton, K.J., Clopton, S.J., 2010. Application Requirements, Design,and Specifications, for the Integration of CAFO in FRAMES. PNWD-4240. PacificNorthwest Division, Battelle Memorial Institute, Richland, WA.

Peng, R.D., 2011. Reproducible research in computational science. Science 334(6060), 1226e1227.

Pruss, A., 1998. Review of epidemiological studies on health effects from exposureto recreational water. Int. J. Epidemiol. 27 (1), 1e9.

PTC, 2013. PTC Mathcad. <http://www.ptc.com/product/mathcad/> (last accessed05.12.13).

Rogers, S.W., Haines, J.R., 2005. Detecting and Mitigating the Environmental Impactof Fecal Pathogens Originating from Confined Animal Feeding Operations: Re-view. EPA/600/R-06/021. National Risk Management Research Laboratory, U.S.Environmental Protection Agency, Cincinnati, OH.

G. Whelan et al. / Environmental Modelling & Software 55 (2014) 77e91 91

Rogers, S.W., Haines, J.R., Holder, E., Donnelly, M., LaFeve, S., Olivieri, E., Hayes, G.,2009. Identifying and quantifying pathogens in agricultural runoff and evalu-ation of bmp effectiveness. In: Presentation to the National Risk ManagementResearch Laboratory, U.S. Environmental Protection Agency, Cincinnati, OH,March 4, 2009.

Rojas, R., Feyen, L., Dassargues, A., 2008. Conceptual model uncertainty ingroundwater modeling: combining generalized likelihood uncertainty estima-tion and Bayesian model averaging. Water Resour. Res. 44, W12418. http://dx.doi.org/10.1029/2008WR006908.

Rose, J.B., Mullinax, R.L., Singh, S.N., Yates, M.V., Gerba, C.P., 1987. Occurrence ofrotaviruses and enteroviruses in recreational waters of Oak Creek, Arizona.Water Res. 21 (11), 1375e1381.

Roser, D.J., Davies, C.M., Ashbolt, N.J., Morison, P., 2006. Microbial exposureassessment of an urban recreational lake: a case study of the application of newrisk-based guidelines. Water Sci. Technol. 54 (3), 245e252.

Savichtcheva, O., Okabe, S., 2006. Alternative indicators of fecal pollution: relationswith pathogens and conventional indicators, current methodologies for directpathogen monitoring and future application perspectives. Water Res. 40 (13),2463e2476.

Schoen, M.E., Ashbolt, N.J., 2010. Assessing pathogen risk to swimmers at non-sewage impacted recreational beaches. Environ. Sci. Technol. 44 (7), 2286e2291.

Schoen, M.E., Soller, J.A., Ashbolt, N.J., 2011. Evaluating the importance of faecalsources in human-impacted waters. Water Res. 45 (8), 2670e2680.

Serrano, S.E., 1992. Semianalytical methods in stochastic groundwater transport.Appl. Math. Model. 16 (4), 181e191.

Shelton, D.R., Pachepsky, Y.A., Sadeghi, A.M., Stout, W.L., Karns, J.S., Gburek, W.J.,2003. Release rates of manure-borne coliform bacteria from data on leachingthrough stony soil. Vadose Zone J. 2, 34e39.

Signor, R.S., Roser, D.J., Ashbolt, N.J., Ball, J.E., 2005. Quantifying the impact of runoffevents on microbiological contaminant concentrations entering surface drink-ing source waters. J. Water Health 3 (4), 453e468.

Signor, R.S., Ashbolt, N.J., Roser, D.J., 2007. Microbial risk implications of rainfall-induced runoff events entering a reservoir used as a drinking-water source.J. Water Supply Res. Technol. 56 (8), 515e531.

Silverman, A.I., Peterson, B.M., Boehm, A.B., McNeill, K., Nelson, K.L., 2013. Sunlightinactivation of human viruses and bacteriophages in coastal waters containingnatural photosensitizers. Environ. Sci. Technol. 47 (4), 1870e1878.

Sirivithayapakorn, S., Keller, A., 2003. Transport of colloids in saturated porousmedia:a pore-scale observation of the size exclusion effect and colloid acceleration.Water Resour. Res. 39 (4), 1109,. http://dx.doi.org/10.1029/2002WR001583.

Soller, J.A., Eisenberg, J.N.S., 2008. An evaluation of parsimony for microbial riskassessment models. Environmetrics 19 (1), 61e78.

Soller, J.A., Olivieri, A.W., Crook, J., Cooper, R.C., Tchobanoglous, G., Parkin, R.T.,Spear, R.C., Eisenberg, J.N., 2003. Risk-based approach to evaluate the publichealth benefit of additional wastewater treatment. Environ. Sci. Technol. 37 (9),1882e1891.

Soller, J.A., Olivieri, A.W., Eisenberg, J.N.S., Sakaji, R., Danielson, R., 2004. Evaluationof Microbial Risk Assessment Techniques and Applications, 00-PUM-3. WaterEnvironmental Research Foundation, Alexandria, VA.

Soller, J.A., Eisenberg, J., DeGeorge, J., Cooper, R., Tchobanoglous, G., Olivieri, A.,2006. A public health evaluation of recreational water impairment. J. WaterHealth 4 (1), 1e19.

Soller, J.A., Seto, E., Olivieri, A.W., 2008. Microbial Risk Assessment Interface Tool: UserDocumentation. Water Environmental Research Foundation, Alexandria, VA.

Soller, J.A., Bartrand, T., Ashbolt, N.J., Ravenscroft, J., Wade, T.J., 2010a. Estimating theprimary etiologic agents in recreational freshwaters impacted by humansources of faecal contamination. Water Res. 44 (16), 4736e4747.

Soller, J.A., Schoen, M.E., Bartrand, T., Ravenscroft, J., Ashbolt, N.J., 2010b. Estimatedhuman health risks from exposure to recreational waters impacted by humanand non-human sources of faecal contamination.Water Res. 44 (16), 4674e4691.

SOT, May 2012. Building for Better Decisions: Multi-scale Integration of HumanHealth and Environmental Data. Society of Toxicology, Research Triangle Park, NCUSA. <https://www.toxicology.org/ai/meet/cct_b4bd.asp,last accessed February25,2013> (last accessed 04.04.13).

Stout, W.L., Pachepsky, Y.A., Shelton, D.R., Sadeghi, A.M., Saporito, L.S.,Sharpley, A.N., 2005. Runoff transport of faecal coliforms and phosphorusreleased from manure in grass buffer conditions. Lett. Appl. Microbiol. 41 (3),230e234.

Teng, J., 2012. Microbial Risk Assessment Modeling for Exposure to Land-appliedClass B Biosolids. PhD dissertation. Drexel University. <http://idea.library.drexel.edu/bitstream/1860/3857/1/Teng_Jingjie.pdf> (last accessed 26.08.13).

Voinov, A., Shugart, H., 2013. ‘Intergronsters,’ integral and integrated modeling.Environ. Modell. Softw. 39, 149e158.

Wade, T.J., Pai, N., Eisenberg, J.N., Colford, J.M.J., 2003. Do U.S. Environmental Pro-tection Agency water quality guidelines for recreational waters preventgastrointestinal illness? A systematic review and meta-analysis. Environ. HealthPerspect. 111, 1102e1109.

Whelan, G., 1980. Distributed Model for Sediment Yield. Master’s thesis. IowaInstitute of Hydraulic Research. University of Iowa, Iowa City, IA.

Whelan, G., Laniak, G.F., 1998. A risk-based approach for a national assessment. In:Benson, C.H., Meegoda, J.N., Gilbert, R.B., Clemence, S.P. (Eds.), Risk-basedCorrective Action and Brownfields Restorations. American Society of Civil En-gineers, Reston, Virginia, pp. 55e74. Geotechnical Special Publication Number82.

Whelan, G., McDonald, J.P., 1996. Multimedia Environmental Pollutant AssessmentSystem (MEPAS): Riverine Pathway Formulations. PNNL-11176. Pacific North-west National Laboratory, Richland, WA.

Whelan, G., Nicholson, T.J. (Eds.), 2002. Proceedings of the Environmental SoftwareSystems Compatibility and Linkage Workshop. U.S. Nuclear Regulatory Com-mission, Rockville, Maryland. NUREG/CP-0177. <http://iemhub.org/site/wiki/13/Proceedings_of_the_Environmental_Software_Systems_Compatibility_and_Linkage_Workshop.pdf> (last accessed 04.04.13).

Whelan, G., Strenge, D.L., Droppo Jr., J.G., Steelman, B.L., Buck, J.W., 1987. TheRemedial Action Priority System (RAPS): Mathematical Formulations. PNL-6200. Pacific Northwest Laboratory, Richland, WA.

Whelan, G., Buck, J.W., Strenge, D.L., Droppo Jr., J.G., Hoopes, B.L., Aiken, R.J., 1992.An overview of the multimedia environmental pollutant assessment system(MEPAS). Hazard. Waste Hazard. 9 (2), 191e208.

Whelan, G., Castleton, K.J., Buck, J.W., Gelston, G.M., Hoopes, B.L., Pelton, M.A.,Strenge, D.L., Kickert, R.N., 1997. Concepts of a Framework for Risk Analysis inMultimedia Environmental Systems (FRAMES). PNNL-11748. Pacific NorthwestNational Laboratory, Richland, WA.

Whelan, G., McDonald, J.P., Gnanapragasam, E.K., Laniak, G.F., Lew, C.S., Mills, W.B.,Yu, C., 1999. Benchmarking of the saturated-zone module associated with threerisk assessment models: RESRAD, MMSOILS, and MEPAS. Environ. Eng. Sci. 16(1), 67e80.

Whelan, G., Castleton, K.J., Buck, J.W., Rodgers, L.P., Taira, R.Y., Gelston, G.M.,Strenge, D.L., 2006. Combining Multiple-module Output Boundary Condition toProduce a Single-input-module Boundary Condition in FRAMES. PNNL-16130.Pacific Northwest National Laboratory, Richland, WA.

Whelan, G., Tenney, N.A., Pelton, M.A., Coleman, A.M., Ward, D.L., Droppo Jr., J.G.,Meyer, P.D., Dorow, K.E., Taira, R.Y., 2009. Techniques to Access Databases andIntegrate Data. PNNL-18244. Pacific Northwest National Laboratory, Richland,WA. <http://www.pnl.gov/main/publications/external/technical_reports/PNNL-18244.pdf> (last accessed 04.04.13).

Whelan, G., Tryby, M.E., Pelton, M.A., Soller, J.A., Castleton, K.J., 2010. Using an in-tegrated, multi-disciplinary framework to support quantitative microbial riskassessments. In: Swayne, D.A., Yang, W., Voinov, A.A., Rizzoli, A., Filatova, T.(Eds.), Proceedings of the 2010 International Congress on EnvironmentalModelling and Software, Ottawa, Canada, vol. 2, pp. 1223e1230. July 5e8, 2010.

WHO (World Health Organization), 2011. Guidelines for Drinking-water Quality,fourth ed. Geneva, Switzerland. <http://www.who.int/water_sanitation_health/publications/2011/dwq_guidelines/en/> (last accessed 03.09.13).

Wiegers, K.E., 1999. Software Requirements. Microsoft Press, Redmond, WA.Wolfe, K.L., Parmar, R.S., Laniak, G.F., Parks, A.B., Wilson, L., Brandmeyer, J.E.,

Ames, D.P., Gray, M.H., 2007. Data for environmental modeling (D4EM): back-ground and example applications of data automation. In: Presented at Inter-national Symposium on Environmental Software Systems, Prague, CzechRepublic, May 22e25, 2007. <http://cfpub.epa.gov/si/si_public_record_report.cfm?dirEntryId¼166789&fed_org_id¼770&SIType¼PR&TIMSType¼&showCriteria¼0&address¼nerl&view¼citation&personID¼18011&role¼Author&sortBy¼pubDateYear&count¼100&dateBeginPublishedPresented¼> (last accessed04.04.13).

Yakirevich, A., Pachepsky, Y.A., Gish, T.J., Guber, A., Shelton, D.R., Cho, K., 2013.Modeling transport of Escherichia coli in a creek during and after artificial high-flow events: three year study and analysis. Water Res. 47 (8), 2676e2688.

Zhao, T., Doyle, M.P., Shere, J., Garber, L., 1995. Prevalence of enterohemorrhagicEscherichia coli O157:H7 in a survey of dairy herds. Appl. Environ. Microbiol. 61(4), 1290e1293.

Zmirou, D., Pena, L., Ledrans, M., Letertre, A., 2003. Risks associated with themicrobiological quality of bodies of fresh and marine water used for recrea-tional purposes: summary estimates based on published epidemiologicalstudies. Arch. Environ. Health 58 (11), 703e711.