PERSPECTIVE

Dealing with femtorisks ininternational relationsAaron Benjamin Franka,1, Margaret Goud Collinsb,2, Simon A. Levinc,1,2, Andrew W. Lod,2, Joshua Ramoe,2,Ulf Dieckmannb, Victor Kremenyukf, Arkady Kryazhimskiyb,g,3, JoAnne Linnerooth-Bayerb, Ben Ramalingamh,J. Stapleton Royi, Donald G. Saarij, Stefan Thurnerb,k, and Detlof von WinterfeldtlaDepartment of Computational Social Science, George Mason University, Arlington, VA 22030; bInternational Institute for Applied SystemsAnalysis, A-2361 Laxenburg, Austria; cDepartment of Ecology and Evolutionary Biology, Princeton University, Princeton, NJ 08544; dSloanSchool of Management, Massachusetts Institute of Technology, Cambridge, MA 02142; eKissinger Associates, New York, NY 10022; fInstitutefor US and Canadian Studies, Russian Academy of Sciences, Moscow 123995, Russia; gSteklov Institute of Mathematics, Russian Academy ofSciences, Moscow 119991, Russia; hOverseas Development Institute, London, United Kingdom; iKissinger Institute on China and the UnitedStates, The Wilson Center, Washington, DC 20004; jInstitute for Mathematical Behavioral Sciences, University of California, Irvine, CA 92697;kMedical University of Vienna, A-1090 Vienna, Austria; and lDaniel J. Epstein Department of Industrial & Systems Engineering, University ofSouthern California, Los Angeles, CA 90089

Edited by Kenneth W. Wachter, University of California, Berkeley, CA, and approved October 17, 2014 (received for review July 6, 2014)

The contemporary global community is increasingly interdependent and confronted with systemic risks posed by the actions and interactionsof actors existing beneath the level of formal institutions, often operating outside effective governance structures. Frequently, these actors arehuman agents, such as rogue traders or aggressive financial innovators, terrorists, groups of dissidents, or unauthorized sources of sensitive orsecret information about government or private sector activities. In other instances, influential “actors” take the form of climate change,communications technologies, or socioeconomic globalization. Although these individual forces may be small relative to state governments orinternational institutions, or may operate on long time scales, the changes they catalyze can pose significant challenges to the analysis andpractice of international relations through the operation of complex feedbacks and interactions of individual agents and interconnectedsystems. We call these challenges “femtorisks,” and emphasize their importance for two reasons. First, in isolation, they may be inconse-quential and semiautonomous; but when embedded in complex adaptive systems, characterized by individual agents able to change, learnfrom experience, and pursue their own agendas, the strategic interaction between actors can propel systems down paths of increasing, evenglobal, instability. Second, because their influence stems from complex interactions at interfaces of multiple systems (e.g., social, financial,political, technological, ecological, etc.), femtorisks challenge standard approaches to risk assessment, as higher-order consequences cascadeacross the boundaries of socially constructed complex systems. We argue that new approaches to assessing and managing systemic risk ininternational relations are required, inspired by principles of evolutionary theory and development of resilient ecological systems.

complex adaptive systems | systemic risk | risk analysis | contagion | resilience

Recent events and analyses have highlightedthe limitations of traditional approaches tomeasuring systemic risk in complex adaptivesystems (CAS) in the sphere of internationalaffairs. Many recent developments related tointernational systems—for example the 2008financial crisis; the Arab Spring of 2011–2012; the Ukrainian crisis of 2014; the rap-idly changing Arctic; Hezbollah’s increas-ing embeddedness in regional and globalpolitics; and global health risks arising fromzoonoses—reveal significant limitations tounderstanding of 21st century internationalsystems and the risks posed by financial,political, and technological developments.The destabilizing potential consequencesof the actions of small-scale actors introducea new element to the analysis of complexinternational systems, and indicate that newapproaches to analyzing and coping withthreats within CAS are required (1).These risks reveal the challenges posed by

increasing interdependence in the international

sphere as the coupling of multiple naturalsystems (e.g., climate, energy resource, andecological) with artificial, socially constructed,and purposeful systems (e.g., financial, healthcare, and technological) become tighter (2–5).We introduce the term “femtorisks” to refer tothreats that confront international decisionmakers as a result of the actions and interac-tions of actors that exist beneath the level offormal institutions or operate outside of estab-lished governance structures. We chose thatterm, using the prefix that denotes a factor of10−15, to highlight the apparent insignificanceof the individual actor that might be a sourceof such a risk, and to emphasize the status ofthe femtorisk as a semiautonomous agent,responding to and acting in its local environ-ment. The use is analogous to the deploymentof the terms “femtocell” vs. “picocell” in theworld of cellular communications (6); that is,following the use in cellular communications,“femto-” here does not mean literally some-thing 15 orders-of-magnitude smaller than

the macro scale, but just something manyorders-of-magnitude smaller.Femtorisks may be pictured as small

fissures inside nodes arrayed along networktopologies of various sorts. The primaryfeature of such risks is that they onlybecome apparent when the system topologyshifts in some fashion. The “revealing” offemtorisks can be for internal or externalreasons (i.e., inside or outside the node orsystem where the risk is present) but theeffect is the same: What once appearedsolid and risk-free becomes a locus forsystemic change, often in a catastrophic

Author contributions: A.B.F., M.G.C., S.A.L., A.W.L., J.R., U.D., V.K.,

A.K., J.L.-B., B.R., J.S.R., D.G.S., S.T., and D.v.W. wrote the paper.

The authors declare no conflict of interest.

This article is a PNAS Direct Submission.

1To whom correspondence may be addressed. Email: afrank3@gmu.edu or slevin@princeton.edu.

2M.G.C., S.A.L., A.W.L., and J.R. contributed equally to this work.

3Deceased November 3, 2014.

17356–17362 | PNAS | December 9, 2014 | vol. 111 | no. 49 www.pnas.org/cgi/doi/10.1073/pnas.1400229111

fashion with phase-transition and hysteresisas possible outcomes.In many cases, the actors that pose these

risks are human agents, such as rogue tradersor aggressive financial innovators, terrorists,groups of dissidents, or unauthorized sourcesof sensitive or secret information that revealgovernment or private sector activities. Inother instances, these “actors” may take theform of developments in climate change,communications technologies, or socioeco-nomic globalization. Any of these can alterinteractive circumstances between humanagents and produce events that exist far inthe tails of probability. These changes canpose significant challenges to the governanceand management of major elements of globalsocietal support systems through the opera-tion of complex feedbacks and interactions ofindividual agents in profoundly interdepen-dent systems (7).In isolation, femtorisks may be small and

inconsequential; but when embedded inCAS, their potential to propel systems downpaths of instability poses a significant chal-lenge to the complex network of formal andinformal arrangements that underlie moderneconomies and political systems. Because theinfluences of femtorisks stem from interac-tions at the interfaces of multiple systems(e.g., social, financial, political, technological,ecological, and so forth), complex inter-actions between actors can often producemacroscopic outcomes that cannot be de-rived from examinations of individual choicesand action in isolation (8). These complex in-teractions confound standard approaches torisk assessment by creating low-probability/high-impact events as second- and third-order consequences cascade across systems’boundaries (4, 9).Broadly speaking, innovative ways of re-

fining the estimation and management offemtorisks come in two complementaryforms. The first form requires improvementsin the quality of predictions about increas-ingly small-scale actors, low-probabilityevents, and the cross-scale implications ofrapidly evolving technologies and relation-ships. The second form involves the creationof new ways of coping with uncertainties thatdo not depend on precise forecasts of theprobability and consequences of futureevents, but rather develop approaches tonegotiation, governance, and regulation thatincorporate an understanding of CAS andthe development of new means for measur-ing and enhancing their robustness andresilience under uncertainty (10). Al-though continued scientific research ina variety of domains will make better pre-dictions possible, we argue that the latter

approach provides greater utility to riskmanagers across a variety of domains thatare responsible for the governance of ele-ments of the systems upon which global-ized society depends. By drawing uponlessons learned from evolutionary theoryand biological systems, new governingregimes can be developed, ones designedto cope with—even benefit from—the un-certainty and dynamism in CAS.

Examples of Femtorisks in InternationalCAS2008 Financial Crisis. The 2008 FinancialCrisis provides an example of a CAS inwhich investors, regulators, and policymakerswere all lulled into a false sense of security byfocusing on variables that suggested the fi-nancial system was stable and threats rela-tively unimportant, while neglecting theincreasing complexity and scale that laid thefoundations for extraordinary losses. In 1998,Commodities and Futures Trading Com-mission Chairman Brooksley Born warned ofsystemic risks resulting from credit defaultswaps (CDS) and other derivatives securities.However, at the time this market was rela-tively small, stable, and highly liquid. Theaggregate notional exposure of the CDSmarket in 1998 was $180 billion, a smallfraction of the financial industry’s total assets,leading regulators to believe that interventionwas unnecessary. Thus, Born’s warnings weredismissed by the Federal Reserve Chairman,Treasury Secretary, and other regulators.However, the rapid and unsupervised growthof the CDS market led to aggregate notionalexposures of $6 trillion in 2004 and $58trillion in 2007. Market volatility that wasregarded as unimportant just a few yearsearlier brought down AIG, one of the largestand most respected insurance companies inthe world. The nature of regulatory mecha-nisms as either “on” or “off” meant thatregulators and other stakeholders failed tonotice or adapt to the changing circum-stances of the market, the amount of capitalconcentrated among a handful of big institu-tions, and the dynamic characteristics of itsrisk exposures. Thus, systemic risk increasedwithout notice and ultimately led to the worstfinancial crisis since the Great Depression (11).

2010–2012 Arab Spring. The ArabSpring began with the self-immolation ofMohammad Bouazazi in Tunisia in De-cember 2010 and persisted into 2012.During this period, regimes in Tunisia,Egypt, and Yemen all changed under thepressure of popular protest, whereas Lib-ya’s dictatorship fell in the face of armedrebellion supported by NATO military

forces, and the Syrian state descended intocivil war that persists into the present. Inaddition to those states that experiencedregime change or civil war, Algeria, Iraq,Jordan, Kuwait, Morocco, Sudan, Maur-itania, Oman, and Saudi Arabia all ex-perienced social protests that invitedpolitical reforms, drew harsh responses bygovernment security forces, or both. The eventsof the Arab Spring demonstrated how small-scale actions could combine with distinctivelocal features of individual states and long-termhistorical, regional and global factors to pro-duce unexpected, rapid changes in composi-tion of the region’s political structure (12).The sources postulated in academic and

diplomatic analyses of the contagion of pro-test and regime change of the Arab Springare varied, with outcomes seen as theintersection between local circumstances,short-term choices, and long-term trends,and vulnerabilities established over decades.Factors identified by experts on the region’shistory and politics include demographics,unemployment, aging dictatorships, corruptand weak political institutions, nationalism,food prices, social media, external influences,the role of Mosques in social mobilization,poor decision-making by political and mili-tary leadership, and contagion dynamics (12–15). For example, agricultural policies acrossthe region rendered the states of the MiddleEast and North Africa among the world’slargest importers of cereals: the Arab statesimported 50% of their food between 2006and 2010, making regimes acutely vulnerableto rising food prices (16). The weak agricul-tural system in the region encouraged mi-gration to cities, which ensured a deepreserve of unemployed youth, primed foraction. New social media and communica-tions technologies offered that restless de-mographic the means for social mobilizationand a route to global attention; from theseconverging elements, the coordination anddiffusion of antiregime activities emerged.The many competing explanations for the

Arab Spring highlight the challenges facingrisk managers and policy analysts in the in-ternational system. Although many signals ofpolitical, economic, and societal fragility canbe identified ex post facto, no analytic orconceptual framework was in place to pro-vide policy makers with analyses that couldmitigate threats to, or ease the transition of,established regimes (17).

2013–2014 Ukraine Crisis. The local up-rising on Maidan Square in Kiev, Ukraine, inthe winter of 2013–2014 displays manycharacteristics of femtorisks, with the fullconsequences of their manifestation to be

Frank et al. PNAS | December 9, 2014 | vol. 111 | no. 49 | 17357

PERS

PECT

IVE

determined in the months and years ahead.The Maidan demonstrations began in re-sponse to the government’s decision not toestablish closer European economic integrationin November 2013. After a violent effort by thegovernment to disperse the protestors, popularmobilization against the government expanded,eventually forcing Viktor Yanukovych frompower in February 2014 (18).The collapse of the Yanukovych regime

has triggered an ongoing governmental andinternational crisis. The predominantly Rus-sian ethnic population of Crimea voted tosecede from Ukraine in February 2014, withthe aim of joining the Russian state. Con-tinuing unrest within the Russian-speakingpopulation of eastern Ukraine has resulted inincreasing Russian political and military iso-lation and provocative diplomatic and mili-tary maneuvers between Russia and NATOmembers. Moreover, regular and irregularmilitary forces within Ukraine have maneu-vered to capitalize on the vacuum of powerand legitimacy that followed the collapse ofthe old regime, catalyzing an escalation in theuse of deadly weaponry in eastern Ukraine—sometimes by poorly trained and disciplinedforces—that resulted in the downing ofMalaysian Air Flight 77 (19). As a result,what started as a series of political demon-strations against the Yanukovych govern-ment’s reluctance to deepen Ukraine’sintegration into European economic andpolitical life, and its complementary tilt to-ward closer economic and social integrationwith Russia, has provoked the largest crisisbetween great powers since the end of theCold War (20, 21).

1980s to Present Hezbollah. Hezbollah isone of the world’s more sophisticated andcapable violent nonstate actors. The organi-zation emerged during Lebanon’s 15-y civilwar, 1975–1990, and offered an official mani-festo declaring loyalty to Iran’s AyatollahRuhollah Khomeini, demanding the ex-pulsion of United States, French, and Israeliforces from Lebanese territory, and calling forthe destruction of Israel in 1985. Since thattime, it has been responsible for a variety ofterrorist attacks in the Middle East, Europe,and Latin America. Hezbollah’s attacks haveincluded the use of hijacking, car and suicidebombings, political assassination, kidnapping,and a campaign of rocket attacks launchedfrom Lebanon into Israeli territory. In addi-tion, Hezbollah has established political andmilitary autonomy over several regions ofLebanon, and maintains the ability to operatein Lebanon’s Shia communities along thesouthern border with Israel, the Eastern

border with Syria, and the southern sectorsof Beirut.Through a combination of political and

civic activities, Hezbollah has embedded itselfwithin the Lebanese state, operating a seriesof parallel governance structures that providepolitical, military, economic, and social ser-vices that have established its local legitimacyand autonomy from the official Lebanesegovernment. Over the last decade, the extentto which Hezbollah has grown capable ofautonomously affecting regional conflict andstability has become increasingly apparentthrough notable events, such as its suspectedassassination of Rafic Hariri in 2005 and the2006 conflict with Israel.

Climate Change in the Arctic. Climatewarming in the Arctic is triggering changesin the region’s environmental, economic,resource development, transportation, em-ployment, population, and cultural systems,embodying a CAS in which established actorsare adapting to changing strategic circum-stances and to new entrants into region’sphysical and political domain: for example,the 2013 addition of six observer nations intothe Arctic Council, most notably includingChina (22). The rapid increase in accessibilityhas increased interest in the region becauseof the economic opportunities representedby mining, energy resources, new shippingroutes, and increased tourism.New mining opportunities in Greenland

offer an example of these cascading impli-cations for the Arctic region. Geologistshave identified abundant deposits of rareearth elements among the island’s richmineral resources, and mining companiesfrom around the world are competing forthe opportunity to develop these critical rawmaterials for 21st century technologies. Theramifications are already apparent in Green-land’s politics and governance arrangementswith Denmark, as the mining scenarios couldtransform the icy island’s economics,and challenge its environmental regulationframework (23). However, the production ofrare earth elements could also effect profoundchange on the global supply of those crucialminerals, and the debate on environmentalstandards illuminate the challenges facingthe governments of these sparsely inhabitedregions. Moreover, the opportunities formineral resources are setting off a reexami-nation of national boundaries, with bothCanada and Russia having announced in-tentions to extend territorial claims to, andbeyond, the North Pole (24, 25).

Failure Modes of Conventional RiskAssessment ApproachesConventional risk assessment approachesrely on the ability to accurately estimate theprobability and consequences of events thatmay occur in the future (26–29). Theseestimates rely on historical events and anunderstanding of the systems characterizingthe risk. In the case of femtorisks, as theexamples noted above reveal, historical ex-perience and an understanding of the sys-tems are lacking. Risk assessment has thusfailed to provide decision-makers with themeans to manage some of the most difficultchallenges facing public and private sectorgovernance. Although they are not mutuallyexclusive, these failure modes can be placedinto three categories: the static nature of riskanalysis with the need to estimate risk be-yond the predictive limits of CAS, the im-proper orientation of risk analysts in theirtreatment of missing information and slowvariables, and inertia within decision-makingprocesses that limit the effectiveness of riskassessments for organizational reasons.

Structure and Dynamics of CAS. The firsttype of failure is grounded in conventionalrisk management and policy-making para-digms that assume that the behaviors ofsystems are predictable, either with cer-tainty or probabilistically. This approachemphasizes the design of rational, utility-maximizing choices and seeks improve-ments through the development and re-finement of predictive models. An implicitassumption in such predictive efforts is thatthe world is a stable system and historicaldata can be used to infer the probabilisticstructure of the future.However, CAS are composed of interact-

ing units that adapt and change in responseto actors buried deep within the system, aswell as to each other (30, 31). Moreover suchsystems often accelerate the propagation ofrandomness that emerges inside the system.As a result, the ability of conventional riskmodels to reflect the adaptations and feed-backs of CAS, such as natural selection,social learning, or strategic calculation, maybe limited, particularly when models in-corporate the assumption that historicalpatterns and relationships will persist into thefuture (32). In fact, several CAS have dem-onstrated a propensity for rapid restructuringor transformation as innovative behaviorsdiffuse, and result in periods where formerlysuccessful predictions fail quickly based on thepopularity of particular strategies or behaviors.In an example from the financial sector,

the employment of Sharpe’s Capital Asset

17358 | www.pnas.org/cgi/doi/10.1073/pnas.1400229111 Frank et al.

Pricing Model (33) in the 1970s led to thepractice of “passive investing” in broadly di-versified portfolios of securities that repli-cated indexes, such as the S&P 500, and hasbecome so widespread that virtually everyworker with an employee-sponsored pensionfund is now invested in such products. Al-though the multitrillion-dollar index-fundbusiness successfully provided investors withattractive returns at low fees for decades, itspopularity also implies a much more tightlycoupled system of investors in which a smalldecline in the S&P 500 can quickly turn intopanic selling and a stock-market crash, as wediscovered on October 19, 1987.Because they are generally nonstationary,

CAS do not conform to risk analysts’ char-acterizations of “known” or even “unknown”systems, whose properties are well suited fortraditional approaches to risk measurementand management. Portrayal of “known”systems denotes that the probability dis-tributions of future states are completelyspecified, as in the case of automobile- or life-insurance claims (in the absence of large-scale events, such as mass-casualty terroristattacks or natural disasters), whereas “un-known” systems occur when all possiblestates of the future system can be enumeratedbut the probabilities associated with enteringinto them cannot (27). Instead, CAS presentrisk analysts and decision-makers with ig-norance, or deep uncertainty, where all pos-sible future states and their associatedprobabilities cannot be specified (34–37).

Analytic Orientation. A second source offailure in traditional risk approaches con-cerns the analytic orientation of those re-sponsible for measuring and managing risksources and levels. These analysts may im-properly focus their attention on the collec-tion and analysis of information that cannotdetect risks posed by impending phase tran-sitions or regime changes in CAS (i.e., rapidchanges in the system’s structure and dy-namics). In these instances, the informationnecessary to provide accurate assessmentsof risk may be available to analysts anddecision-makers, but the data have notbeen collected or properly interpreted. As aresult, events that catch decision-makersunprepared might have been anticipatedthrough the use of a different analytic anddata-collection framework.The Arab Spring offers an important ex-

ample of this phenomenon. In the cases ofTunisia and Egypt, the protesters were pre-dominantly urban and secular youth, many ofwhom had no prior history of political activ-ism. They demanded democratic reforms andthe strengthening of state institutions through

the rule of law and abolition of endemic cor-ruption. Their relative size and demands sur-prised governments and observers alike, eachof which had previously focused on the tra-ditional sources of antiregime rhetoric andaction from well-organized Islamist organ-izations. Without a model for a leaderlessrevolution, the threats to these establishedregimes from spontaneously formed crowds ofsecular protestors could not be accuratelyassessed (38).Emerging concerns in areas of cyber-

security offer another set of examples.So-called “zero-day exploits” or “advancedpersistent threats” are examples of risksembedded inside existing systems that areamplified as systems become increasinglyconnected, where integration creates com-plex feedbacks between units that weredesigned to perform specialized or individualpurposes. Such risks are not visible until afterthey have been exploited, presenting a chal-lenge to traditional modeling. Indeed, thestrategies introduced for controlling andproviding stability in highly complex tech-nological systems have been known to havethe opposite effect, inadvertently increasingthe risk of cascading impacts in response tounforeseen changes (4).Ensuring that risk analysts are properly

oriented can be particularly difficult to ach-ieve in CAS because collecting informationmay alter the behavior of the system’s units,provoking undesired or unintended adaptiveresponses. For example, national intelligenceorganizations carefully guard their sourcesand methods because they implicitly recog-nize that their revelation may invite adaptivechanges in their target’s behavior, under-mining their future use, transforming theminto channels for passing deceptive infor-mation, and damaging relations with otheractors (39, 40).Another problem associated with achiev-

ing a proper analytic orientation within CASregards establishing the necessary depthand breadth of analysis. The system and itscomponents change, and actors perceive andadapt to their environment as a result of theirunique experiences. Their behavior may de-viate from the expectations of utility maxi-mization or rational choice, and analysts arefaced with a challenge of adjusting the ori-entation of their risk models to incorporatethe effects these nontraditional factors on thebroader systems.Finally, CAS contain dynamics that oper-

ate on multiple temporal scales. Thus, thepresence of slow-variables, such as cultural orenvironmental changes, may be difficult toperceive and harder to quantify becausethey are often masked by higher-frequency

dynamics. These slow variables often es-tablish boundary or threshold conditionsthat inform faster-paced changes in sys-tems. In the contemporary social systems,these variables may include demographicstructures, educational and technical skillsetswithin populations, changes in ideologi-cal worldviews, access to credit, Internetpenetration within populations and sub-populations, the diffusion of small arms, andso forth. Although none of these may bedeterminative of a femtorisk’s materiali-zation, they may enable or hinder the abilityof a CAS to transition into a new state withlittle or no warning. Traditional risk assess-ments consider only visible dynamics thatoccur on faster time scales (41–43).Improper analytic orientation enables

small systemic risks, which may be easy tomitigate early on, to grow into crises by thetime they are recognized. Moreover, devotingcontinuous attention and resources to varia-bles that possess little dynamism or cannot beaffected by individual or collective actionimposes opportunity costs by robbing atten-tion and resources from alternative inter-ventions that could be more successful inavoiding or mitigating the social costs posedby systemic risks. This aspect is particularlytrue for strategies that must interact with andshape global systems, and cannot rely ondiscrete “on/off” interactions and or use “exitstrategies” whenever contact between actorsis persistent.

Decision-Making Inertia. Asymmetriesexist in CAS. The actions of individuals orsmall groups of well-positioned actors maypush the system beyond bounds of func-tionality, thus creating femtorisks. The ef-fective prevention of systemic breakdownrequires collective action by multiple stake-holders. Therefore, effective risk managementof international CAS is often dependent onsocial processes related to the interaction ofanalysts and decision makers, even within thesame organization, with challenges com-pounded when the regulation of differentsystems in different national governancestructures require coordination. In additionto the complexities of analytic assessments ofCAS, these social processes introduce newsources of complexity as personal and orga-nizational politics of negotiation and powercome into play (44). Thus, the relationshipsbetween those who assess systemic risks andthose who make decisions about how to ad-dress them often determine how successfullyanalysis is used in decision-making processes.Even if analysts correctly assess risks withinCAS, decision-makers may be vested in pre-serving their hard-won political agreements

Frank et al. PNAS | December 9, 2014 | vol. 111 | no. 49 | 17359

PERS

PECT

IVE

or economic advantages, or may question theaccuracy of analyst’s conclusions based ontheir own expertise and experiences. As aresult, organizational decision-making oftenreveals highly confrontational relationshipsbetween analysts and decision-makers thatinfluence how information and knowledgeabout risks are used (35, 45–50).This relationship has repeatedly under-

mined the assumptions of rational actioninherent in risk management and the beliefthat states and firms are unified actors. In-stead each has distinctive roles in the as-sessment and management of risk. Analystsfocus on the substantive aspects of systems,whether financial markets, critical infra-structure, international development, cli-mate, and so forth. Decision-makers wrestlewith a wider range of concerns, but maypossess a narrower organizational or evenpersonal set of interests, and focus theirefforts on the types of actions that they andother stakeholders can agree to or imposegiven their relative power. Because decision-makers are involved in a political process, asopposed to a technical engineering or designprocess, they are often reluctant to changepolicies, particularly if the costs of doing sowould open existing commitments to re-negotiation or damage their reputation orparticipation in future negotiations. Oncecommitted, decision makers often neglectnew information that might indicate thattheir policies are failing, or ignore the analysisof specialists that contradicts their expect-ations (51, 52). This willful blindness canallow the harmful effects of small risks toproliferate, diffuse, and amplify.

Biological Models of Adaptation forCoping with UncertaintyAdaptive evolution is to large extent shapedby uncertainty, and the need to respond tothat uncertainty through averaging mecha-nisms, plasticity, or genetic change. Manyenvironmental features confronting organ-isms, like diurnal or seasonal cycles, arepredictable in a probabilistic sense, and haveshaped a range of physiological and behav-ioral responses; others, however, such aswhich pathogens will attack an organism, areunpredictable and have evolutionarily led toadaptive responses, such as the vertebrateimmune system. Rather than seek to designrobust institutions and strategies that mayfare well only against specified futures, riskmeasurement and management professionalswould benefit from imitating how evolutionhas dealt with unknowable challenges, andtake steps toward assuring those responsiblefor sustaining global stability and prosperitypossess sufficient learning capabilities and

adaptive capacity to detect and mitigate theeffects of novel risks (53, 54).In any complex system, robustness [often

termed resilience in the ecological literature(55)] depends upon the balance among threeinterrelated aspects (56): (i) the diversity ofthe units within the system, which encodes itsadaptive capacity; (ii) the extent to which thesystem contains functional redundancies, pro-viding insurance against the loss of key ele-ments; and (iii) the degree of modularity withrespect to the coupling between components.When taken as a whole, the trade-offs

among these three features enable a system tokeep functioning in the face of changes in itsenvironment, and provide opportunities forexperimentation to tolerate disruptions andproduce innovations without placing theentire system at risk (3).One of the most remarkable triumphs of

evolution, the vertebrate immune system,provides a model for how to design resilientsystems that can withstand the shocks pro-duced by femtorisks and mitigate systemicrisks. The key features of this model include:(i) the maintenance of a set of genericdefenses that can rapidly identify and re-spond to threats, (as the body does when itrecognizes a pathogen and rushes generalizedantibodies to the site of the threat); (ii) per-sistent engagements that enable rapid learn-ing through interaction with threateningactors or processes (as the body producesspecialized antibodies in response to the in-vader); (iii) translation lessons from priorexperiences into customized, localized defensesagainst previously encountered threats (thebody produces permanent, or at least long-lasting, defenses against the infection); and (iv)the maintenance of an archive of previouslyexperienced threats and the addition of suc-cessful countermeasures to the set of genericresponses to mitigate future encounters rapidly(the body’s antibody repertoire).As a result, the vertebrate immune system

is able to allow organisms to persist de-spite constantly encountering novel threatsthrough continuous learning and adaptation.The traditional rational actor model of de-cision-making emphasizes observation, cal-culation, and periodic action, all of whichlimit engagements and therefore provide fewopportunities to interact and learn. The evo-lutionary approach proposed as an alternativebasis for risk management and governanceencourages sustained engagement and feed-back to create learning opportunities thatenhance the adaptive capacity of internationalinstitutions in many domains.

Enriching PolicyMany of the aspects of the immune systemmodel have comparable features in in-ternational relations. For example, studies ofmilitary innovation often emphasize thecomplexity of peacetime developments,when uncertainties about the capabilitiesand strategies of other forces proliferatebecause of a lack of engagement. Militaryplanners have recognized the resulting needto maximize flexibility through the creationof generic capabilities that can be modifiedas new information about rivals’ goals, strat-egies, tactics, and technologies becomeavailable (57–59). Military organizationsoften study one another’s successes andfailures for insights into their own capa-bilities, as demonstrated by the United Statesanalysis of Israeli’s air campaign againstHezbollah in 2006. In addition, militaryorganizations simulate conflict scenarioswith unproven capabilities or unfamiliarrivals to generate insights into threats tomaintain an archive of possible counter-measures for use in an array of possiblefuture encounters (60, 61). Likewise, foreignpolicy professionals often argue in favorof maintaining ties with rogue states tostrengthen perspective, insights, and strate-gic context, and increase opportunities toevolve specialized defenses, and enhance thecoevolution of symbiotic systems (62).Clearly, some quarters have recognized

that the development of resilient risk-management schemes, which can cope withnovelty, surprise, and inaccuracies in pre-dictions, can have important benefits forpublic and private sector governance. Forexample, an adaptive approach based onlearning through interaction and feedbackaccords with the boundedly rational foun-dation of human cognition and organiza-tional behavior (63–65). This approach shiftsthe evaluative criteria of policy options awayfrom optimal, often brittle solutions that re-quire accurate predictions, in favor of resilientsolutions that can be adapted in response tonew information and experiences.Moreover, diminishing the role of predic-

tions in the formulation of risk-managementstrategies and policy can alter the conductof decision-making itself by reducing ten-sions between multiple stakeholders, riskanalysts, and decision-makers. As long aspredictions serve as the basis of decision-making within organizations, strong incen-tives exist to control analytic processes andfeatures, such as the terms of reference andscope of analysis, sources of data, method-ology, and so forth, to ensure that partic-ular personal or organizational perspectives

17360 | www.pnas.org/cgi/doi/10.1073/pnas.1400229111 Frank et al.

prevail in agenda setting. By moving towardan increasingly adaptive risk-managementframework, decision-making rewards thoseactors that can more rapidly gather, process,and react to new information, activities thatare enriched by searching across broadranges of possible futures generated by al-ternative, competing perspectives on thesystem (34). Thus, analytic frameworks foraddressing systemic risks may be more suc-cessful at solving collective action problemsassociated with the governance of CAS byaccepting uncertainty, allowing stakeholdersto discover shared interests and concernsabout potential futures that support the for-mation of coalitions willing to invest inlearning actions, rather than in practices thatreward bureaucratic maneuvering to see oneset of future risks thrive at the expense ofother views.Together, these benefits address the

problems associated with the limits ofprediction in CAS, the challenge of properorientation by risk analysts, and the needto reduce inertia in decision-making pro-cesses by improving relations betweenanalysts and decision-makers.The evolutionary model of risk manage-

ment in CAS provides the additional benefitof aligning with boundedly rational decision-making exemplified by individuals andgroups. In this approach, decision-makersreach satisfaction by eliminating options thatare believed to ineffectively manage risksuntil they identify an option or set of optionsthat meet their needs. This approach does notproduce optimal outcomes, but allows fordecision-makers to engage in a continuousprocess of adaptation as situations changeand new information about policy opportu-nities and threats become known.Another common feature of successful

risk-management strategies and biologicalsystems is the importance of building onsolutions or capabilities with multiple bene-fits. For example, improved border securityprovides benefits to counterterrorism, coun-ternarcotics, and illegal immigration. Sim-ilarly, improvements in public healthmonitoring and access to medical servicesprovide the foundation for the early detectionand mitigation of natural epidemics andbioterror attacks. Just as in biology, successfulrisk mitigation strategies may be generic,allowing for their reuse in multiple domainsor for many purposes. Thus, rather thanemphasize point solutions that may be opti-mized to particular risks, risk managersshould prefer capabilities that have positiveexternalities and spillover effects.Evolution provides promising models for

coping with risks because of those models’

ability to cope with uncertainty and relax therationality assumptions embedded in tradi-tional analytic approaches. However, im-portant differences remain between socialsystems and biological ones. In biologicalevolution, adaptation occurs as a result ofrandom search, building upon designs thatworked in the past. By comparison, adapta-tion in social systems is not purely random asthe search for new solutions to challenges isbased on the interests, ethics, and strategicanticipation of actors. Although we ad-vocate the need for an evolutionary modelthat emphasizes engagement, learning, andresilience as opposed to predictability andoptimality, the evolutionary dynamics ofinternational relations may differ fromthose observed in biological systems be-cause of fundamental differences in theirsearch processes.

ConclusionsThe conceptual framework provided by bi-ological models of evolution for coping withthreats to the functioning of internationalsystems calls for new approaches to mea-suring and managing risk under uncertaintyCAS (66–68). Approaches based on bi-ological evolution and resilience, which aimfor sustaining systemic functions withoutrelying on predictions about the future stateof the world, offer a promising model. In-tegrating this perspective with the game-theoretical and systems-theory approachesdeveloped by social science disciplines andcommunities of practice for managing sys-temic risk can offer new perspectives onpublic- and private-sector governance overa multitude of CAS spanning internationalsystems, from global financial networks, tocritical infrastructure, to the diffusion of so-cial movements and the rapid collapse ofestablished regimes (1). These are, however,games of a different nature than classic the-ory, emphasizing the interplay between manyindividuals in large populations in whichindividual agency is essential in explainingmacroscopic consequences. In recent years,such extensions of game theory have beentermed “mean-field games,” and hold greatpotential for addressing the problems con-sidered here (69–71).

A world of femtorisks makes risk-calcula-tion much more challenging, as it seemslikely that the probability of some femtoriskinside critical systems may hover near 1—orat least should be assumed to be at that levelfor planning purposes—although this itselfbecomes a challenging problem. For example,it has been argued by leading bankers, suchas Bank of England Chief Economist AndrewHaldane, that the problem of the 2008 fi-nancial crisis was the addition of risk to asystem that was already risk-critical becauseof competitive searches for yield by financialgroups (72). Adding in subprime risk wasone grain of sand too many by this logic.However, what if the issue was not the ad-dition of more risk, but rather the addition ofmore connectivity, which activated embed-ded femtorisks? In this sense femtorisksallow us to begin adding precision to theimportant fact that connectivity is risk, andoffer the elements of connectivity as areasfor examination.Globalization will continue to allow new

femtorisks to emerge, as different cultures,economies, and ecological imperatives createnew frictions in new configurations. Withhumility and openness, the new forum pro-posed herein will develop fresh tools andperspectives to examine approaches to evolv-ing complex problems, and might spawn theconversations and community required togather lessons from the broad experienceof this diverse network, and deploy that ex-perience to help to understand and solvenetworked problems.

ACKNOWLEDGMENTS. We thank the participants inthe International Institute for Applied Systems Analysis’s(IIASA) 2011 international conference Security in the Ageof Systemic Risk: Strategies, Tactics and Options for Deal-ing with Femtorisks and Beyond for their contributions.The perspectives offered in this paper extend ideas orig-inally expressed during conference discussions and pub-lished by IIASA in 2012. Conference participants notlisted among the coauthors of the present paper includeMichael Clegg, Zvi Shtauber, Karl Sigmund, Jonathan Tep-perman, and Wang Yiwei. The original conference reportis available at www.iiasa.ac.at/publication/more_IR-12-010.php. Financial support for the conference was pro-vided by IIASA and by the National Academy of Sciences,through National Science Foundation Grant 0738129, forthe support of National Academy of Sciences activitiesrelated to its responsibilities as the United States NationalMember Organization for IIASA.

1 Jervis R (1998) System Effects: Complexity in Political and Social

Life (Princeton Univ Press, Princeton, NJ).2 Stein A (1990) Why Nations Cooperate (Cornell Univ Press,

Ithaca, NY).3 Simon H (1996) The Sciences of the Artificial (MIT Press,

Cambridge, MA).4 Perrow C (1984) Normal Accidents: Living with High-Risk

Technologies (Princeton Univ Press, Princeton, NJ).5 Keohane R, Nye J (2011) Power and Interdependence (Longman

Classics in Political Science, Boston).

6 Chambers D (2008) What’s the difference between picocells and

femtocells? Think Small Cell. Available at www.thinksmallcell.com/

FAQs/whats-the-difference-between-picocells-and-femtocells.html.

Accessed June 1. 2014.7 Levin S, et al. (2012) Social-ecological systems as complex adaptive

systems: modeling and policy implications. Environ Dev Econ 17(6):1–22.8 Schelling T (1978) Micromotives and Macrobehavior (Norton,

New York).9 Cohen M, Kunreuther H (2007) Operations risk management:

Overview of Paul Kleindorfer’s contributions. Prod Oper Manag

16(5):525–541.

Frank et al. PNAS | December 9, 2014 | vol. 111 | no. 49 | 17361

PERS

PECT

IVE

10 Ramo J (2009) The Age of the Unthinkable: Why the Now WorldDisorder Constantly Surprises Us and What We Can Do About It(Little Brown and Company, New York).11 Khandani A, Lo A, Merton R (2013) Systemic risk and therefinancing ratchet effect. J Financ Econ 108(1):29–45.12 Pollack K, et al. (2011) The Arab Awakening: America and theTransformation of the Middle East (Brookings Institution,Washington, DC).13 Murphy C (2012) The Arab Spring: The uprising and itssignificance. Trinity Magazine. Available at www.trinitydc.edu/magazine-2012/the-arab-spring-the-uprising-and-its-significance/.Accessed June 1. 2014.14 Srinivasan R (2012) Taking power through technology in theArab Spring. Al Jazeera. Available at www.aljazeera.com/indepth/opinion/2012/09/2012919115344299848.html. Accessed June 1.2014.15 Grand S (2014) Understanding Tahrir Square: What TransitionsElsewhere Can Teach Us About the Prospects for Arab Democracy(Brookings Institution, Washington, DC).16 Anonymous (May 17, 2012) Let them eat baklava. TheEconomist Available at www.economist.com/node/21550328.Accessed June 1, 2014.17 Gause FG (2011) The Middle East Academic Community and the“Winter of Arab Discontent”: Why Did We Miss It? Seismic Shift:Understanding Change in the Middle East (Stimson Center,Washington, DC), pp 11–28.18 McMahon R, ed (2014) Ukraine in Crisis: Backgrounder. Councilon Foreign Relations. Available at www.cfr.org/ukraine/ukraine-crisis/p32540. Accessed June 14, 2014.19 Biddle S, Oelrich I (July 21, 2014), Why the Ukraine separatistsscrewed up: Badly organized insurgents can’t master complexweapons systems.Washington Post Monkey Cage Available at www.washingtonpost.com/blogs/monkey-cage/wp/2014/07/21/why-the-ukraine-separatists-screwed-up-badly-organized-insurgents-cant-master-complex-weapons-systems/. Accessed September 1, 2014.20 Pifer S (2014) Ukraine, Russia and the U.S. Policy Response.Testimony of Steven Pifer, Brookings Institution, Before the SenateForeign Relations Committee, June 5, 2014, Available at www.brookings.edu/research/testimony/2014/06/05-ukraine-russia-us-policy-response-pifer. Accessed September 1, 2014.21 Moss W (2014) Are we starting another Cold War over Russianactions in Ukraine? History News Network: George Mason University,May 4, 2014. Available at http://historynewsnetwork.org/article/155525. Accessed September 1, 2014.22 Myers SL (May 15, 2013) Artic Council adds 6 national asobserver states, including China. New York Times Available at www.nytimes.com/2013/05/16/world/europe/arctic-council-adds-six-members-including-china.html. Accessed September 1, 2014.23 Gravgaard A (2014) Greenland’s rare earths gold rush. ForeignAffairs, October 28, 2013, Available at www.foreignaffairs.com/features/letters-from/greenlands-rare-earths-gold-rush. AccessedJune 1, 2014.24 Macalister T (January 5, 2014) Greenland explores Arctic mineralriches amid fears for pristine region. The Guardian Available at www.theguardian.com/world/2014/jan/05/greenland-mines-arctic-fears-pristine-environment. Accessed June 1, 2014.25 Yenikeyeff S, Krysiek TF (2007) The battle for the next energyfrontier: The Russian polar expedition and the future of Arctichydrocarbons, Oxford Energy Comment, Oxford Institute for EnergyStudies, Available at www.oxfordenergy.org/wpcms/wp-content/uploads/2011/01/Aug2007-TheBattleforthenextenergyfrontier-ShamilYenikeyeff-and-TimothyFentonKrysiek.pdf. AccessedSeptember 9, 2014.

26 Hubbard D (2009) The Failure of Risk Management: Why It’sBroken and How to Fix It (John Wiley & Sons, Hoboken, NJ).27 Diebold F, Doherty N, Herring R, eds (2010) The Known, theUnknown, and the Unknowable in Financial Risk Management(Princeton Univ Press, Princeton, NJ).28 Vose D (2009) Risk Analysis: A Quantitative Guide (Wiley, NewYork).29 Bedford T, Cooke R (2001) Probabilistic Risk Analysis (CambridgeUniv Press, New York).30 Holland J (1996) Hidden Order: How Adaptation BuildsComplexity (Helix Books, New York).31 Miller J, Page S (2007) Complex Adaptive Systems (PrincetonUniv Press, Princeton, NJ).32 Popper K (2004) The Poverty of Historicism (Routledge, NewYork).33 Sharpe WF (1964) Capital asset prices—A theory of marketequilibrium under conditions of risk. J Finance XIX(3):425–442.34 Lempert R, Popper S, Bankes S (2003) Shaping the Next One-Hundred Years: New Methods for Quantitative, Long-Term PolicyAnalysis (RAND, Santa Monica, CA).35 Knight F (2006) Risk, Uncertainty and Profit (Dover, Mineola,NY).36 Granger C (2010) Risk: A decision maker’s perspective. TheKnown, the Unknown, and the Unknowable in Financial RiskManagement, eds Diebold F, Doherty N, Herring R (Princeton UnivPress, Princeton, NJ), pp 31–46.37 Zeckhauser R (2010) Investing in the unknown and unknowable.The Known, the Unknown, and the Unknowable in Financial RiskManagement, eds Diebold F, Doherty N, Herring R (Princeton UnivPress, Princeton, NJ), pp 304–343.38 Merlini C, Roy O, eds (2012) Arab Society in Revolt: The West’sMediterranean Challenge (Brookings Institution, Washington, DC).39 Bruce J, Bennett M (2008) Foreign denial and deception:Analytical imperatives. Analyzing Intelligence: Origins, Obstacles,and Innovations, eds George R, Bruce J (Georgetown Univ Press,Washington, DC), pp 122–137.40 Lowenthal M (2011) Intelligence: From Secrets to Policy(Congressional Quarterly Press, Washington, DC).41 Holling CS (1996) Engineering resilience versus ecologicalresilience. Engineering Within Ecological Constraints, ed Schulze P(National Academies Press, Washington, DC), pp 31–44.42 Jansson B, Jansson A (2002) The Baltic Sea: Reversibly Unstableor Irreversibly Stable? Resilience and the Behavior of Large-ScaleSystems, eds Gunderson L, Pritchard L (Island Press, Washington,DC), pp 71–109.43 Walker BH, Carpenter SR, Rockstrom J, Crépin A-S, Peterson GD(2012) Drivers, “slow” variables, “fast” variables, shocks, andresilience. Ecol Soc 17(3):30.44 George R, Rishikoff H, eds (2011) The National SecurityEnterprise: Navigating the Labyrinth (Georgetown Univ Press,Washington, DC).45 Hilsman R (1956) Strategic Intelligence and National Decisions(The Free Press, Glencoe, IL).46 Kent S (1968) Estimates and influence. Studies in Intelligence3(2):11–21.47 Meldman M (1989) Order Without Design: InformationProduction and Policy Making (Stanford Univ Press, Stanford, CA).48 Betts R (2007) Enemies of Intelligence: Knowledge & Power inAmerican National Security (Columbia Univ Press, New York).49 Terverton G (2008) Intelligence analysis: between “politicization”and irrelevance.” Analyzing Intelligence: Origins, Obstacles, andInnovations, eds George R, Bruce J (Georgetown Univ Press,Washington, DC), pp 91–106.

50 Rovner J (2011) Fixing the Facts: National Security and the

Politics of Intelligence (Cornell Univ Press, Ithaca, NY).51 Jervis R (2010) Why Intelligence Fails: Lessons from the Iranian

Revolution and the Iraq War (Cornell Univ Press, Ithaca, NY).52 Nickerson R (1998) Confirmation bias: A ubiquitous

phenomenon in many guises. Rev Gen Psychol 2(2):175–220.53 Aven T (2011) On some recent definitions and analysis

frameworks for risk, vulnerability, and resilience. Risk Anal 31(4):

515–522.54 Scholz R, Blumer T, Brand F (2012) Risk, vulnerability, robustness,

and resilience from a decision-theoretic perspective. J Risk Res 15(3):

313–330.55 Levin S, Lubchenco J (2008) Resilience, robustness, and marine

ecosystem-based management. Bioscience 58(1):27–32.56 Levin S (1999) Fragile Dominion: Complexity and the Commons

(Perseus Publishing, Cambridge, MA).57 Rosen S (1994) Winning the Next War: Innovation and the

Modern Military (Cornell Univ Press, Ithaca, NY).58 Murray W, Millett A, eds (1998) Military Innovation in the

Interwar Period (Cambridge Univ Press, New York).59 Murray W, Knox M (2001) Conclusion: The future behind us.

The Dynamics of Military Revolution, 1300–2050, eds Knox M,

Murray W (Cambridge Univ Press, New York), pp 175–194.60 Arkin W (2007) Divine victory for whom? Airpower in the 2006

Israel-Hezbollah War. Strategic Studies Quarterly 1(2):98–141.61 Herman M, Frost M, Kurz R (2009) Wargaming for Leaders:

Strategic Decision Making from the Battlefield to the Boardroom

(McGraw Hill, New York).62 Brzezinski Z, Gates R (2004) Iran: Time for a New Approach

(Council on Foreign Relations, New York).63 March J, Simon H (1993) Organizations (Blackwell, Cambridge,

MA).64 Simon H (2000) Bounded rationality in social science: Today and

tomorrow. Mind Soc 1(1):25–39.65 Kahneman D (2011) Thinking Fast and Slow (Farrar, Strauss and

Giroux, New York).66 Sagarin RD, et al. (2010) Decentralize, adapt and cooperate.

Nature 465(7296):292–293.67 Sagarin R (2012) Learning from the Octopus: How Secrets from

Nature Can Help Us Fight Terrorist Attacks, Natural Disasters, and

Disease (Basic Books, New York).68 Ramalingam B (2013) Aid on the Edge of Chaos: Rethinking

International Cooperation in a Complex World (Oxford Univ Press,

Oxford).69 Jovanovic B, Rosenthal RW (1988) Anonymous sequential

games. J Math Econ 17(1):77–87.70 Lasry JM, Lions PL (2007) Mean field games. Jpn J Math 2(1):

229–260.71 Huang MY, Malhame RP, Caines PE (2006) Large population

stochastic dynamic games: Closed-loop McKean–Vlasov systems and

the Nash certainty equivalence principle, Special issue in honor of the

65th birthday of Tyrone Duncan. Communications in Information

and Systems 6(3):221–252.72 Haldane AG (2012), Tails of the unexpected, speech given at

“The Credit Crisis Five Years On: Unpacking the Crisis” Conference,

University of Edinburgh Business School, Edinburgh, Scotland. June 8,

2012. Available at www.bankofengland.co.uk/publications/

Documents/speeches/2012/speech582.pdf. Accessed October

14, 2014.

17362 | www.pnas.org/cgi/doi/10.1073/pnas.1400229111 Frank et al.