using natural experiments to study the impact of media on the family

JOSEPH PRICE Brigham Young University

GORDON B. DAHL University of California, San Diego*

Using Natural Experiments to Study the Impact

of Media on the Family

The randomized trial is the gold standard inscientific research and is used by several fieldsto study the effects of media. Although usefulfor studying the immediate response to mediaexposure, the experimental approach is not wellsuited to studying long-term effects or behavioroutside the laboratory. The ‘‘natural experi-ment’’ approach, a quasi-experimental methodcommonly used by economists, exploits settingsin which there is naturally occurring variationthat ‘‘randomly’’ influences the amount and typeof media that individuals are exposed to. We dis-cuss the methodological issues of natural exper-iments and the ways in which findings basedon this approach can complement the findingsfrom laboratory or other experimental settings.The examples we discuss highlight the value ofinterdisciplinary work in better understandingmedia’s impact on family issues such as fertility,divorce, domestic violence, and child well-being.

The question of how media affects families isimportant from both a scientific and practicalperspective, as it can help inform the decisions ofboth practitioners and parents. Family membersare increasingly exposed to a variety of media,

Department of Economics, Brigham Young University,Provo, UT 84602. ([email protected]).

*Department of Economics, University of California, SanDiego, La Jolla, CA 92093 ([email protected]).

Key Words: family, media, natural experiments

including television, movies, print and digitaladvertisements, and the Internet (Pergrams &Zaradic, 2006). Some of these media may havenegative effects, whereas other types of mediamay have positive effects. Social scientists froma variety of disciplines have used differenttechniques to model and estimate these impacts.In this paper, we discuss what economistshave learned about these questions throughthe use of ‘‘natural experiments.’’ We viewthe economist’s approach as complementary tothose in other disciplines (and also recognizeits occasional use by other disciplines). Aprimary aim of this article is to encourage amore interdisciplinary approach to evaluatingthe impact of the media on the family.

The gold standard in scientific research isthe randomized controlled trial. In a randomizedcontrolled trial, subjects are randomly assignedto a treatment or control group. By construction,a correctly implemented randomized experimentyields a causal estimate of the effect of treatmenton the outcome of interest because it balances allof the potential confounding factors between thetwo groups. There are, however, limitations toreal randomized trials. First, many experimentswould be unethical or immoral. Second, manyimportant outcome measures are impracticalto measure in a laboratory. Third, laboratoryand especially field experiments can be veryexpensive to administer, especially for researchquestions that involve large interventions orrequire large samples. Attrition is also a majorproblem in long-term experimental studies.

Family Relations 61 (July 2012): 363 – 373 363DOI:10.1111/j.1741-3729.2012.00711.x

364 Family Relations

When laboratory or field experiments are notpossible, economists have tried to take advantageof ‘‘natural experiments.’’ At the broadest level,a natural experiment is a situation that is thoughtto approximate a randomized trial even thoughthere was no actual randomization involved(DiNardo, 2008). Economists have searched forsettings in which there is naturally occurringvariation in treatment, for example, a suddenchange in public policy (e.g., law changes),random environmental shocks (e.g., weather),or other unexpected events (such as the birth oftwins). The setting must be one where treatmentis arguably independent of all the confoundingfactors that might otherwise bias an estimate.

Natural experiments are a quasi-experimentalmethodology, and as such the key questionfor internal validity is whether the identifyingvariation (the ‘‘treatment’’) is actually random.Except in rare cases, there are no direct statisticaltests to establish internal validity. This is becauseit is impossible to test whether treatment iscorrelated with an unobserved variable. Instead,the researcher must provide arguments andindirect tests in an attempt to convince thereader that the natural experiment can be used toestimate a causal effect. To start, the researcherlays out why the natural experiment occurred andwhy it is likely to introduce random variationin treatment. The researcher then continues byempirically ruling out alternative explanationswith a variety of indirect and placebo testsas falsification exercises. Although the indirecttests will vary from application to application,some of the more common ones includeshowing (a) the treatment occurs before theoutcome, (b) the treatment cannot be predictedby lagged outcomes, (c) there are no differentialpreexisting trends in the outcome, and (d) theobserved covariates are balanced in the treatmentand control samples. Often, supporting evidencecomes from analysis of auxiliary data.

In this paper, we develop the idea of naturalexperiments as we discuss the contributions ofeconomists to the literature on media’s causaleffect on families. We begin by comparing andcontrasting natural experiments to those donein the laboratory, which, by design, estimateshort-term effects. In this first section, we focuson two examples. We start with the effect ofexposure to violent movies in the laboratoryversus the field. We outline why the twoapproaches identify different effects and whythese complementary approaches are important

for a better understanding of media’s effect onfamilies. We then provide an example of anatural experiment that cannot be well studied inthe laboratory, namely, the effect of unexpected,negative emotional cues on domestic violence.In the second half of the paper, we turn to thearguably more important question, namely, thelong-term effects of repeated media exposureon families. We explore how economists’ use ofnatural experiments adds to our understanding ofthe long-term causal effects of media exposure,especially television exposure. The variednatural experiments we discuss show that long-term exposure has both positive and negativeeffects on families. In this section, we provideseveral examples of how economists evaluate thevalidity of a natural experiment. Throughout, wepoint out the benefits and potential challenges tothe use of natural experiments.

NATURAL AND LABORATORY EXPERIMENTSIN THE SHORT RUN

Whereas natural experiments have a long historyin economics (Meyer, 1995), the use of naturalexperiments by economists to study the effectsof media is more recent. An early pioneerin the use of natural experiments to studythe effect of media is the sociologist DavidPhillips. He initially called his settings ‘‘foundexperiments,’’ although the idea is the sameas a natural experiment. In a series of papers,he documented that suicide rates increase aftermass media reporting of suicides, includingan increase in teenage suicides (Phillips &Carstensen, 1986). In similar papers, Phillipsfound that motor vehicle accident fatalitiesalso increase after newspaper stories aboutsuicide (Phillips, 1979). Phillips has also useda similar methodology to study the impact ofmass media violence on homicides, using bothfictional television stories and news reporting asnatural experiments (Phillips, 1983; Phillips &Hensley, 1984). In each case, the idea behindPhillips’s natural experiments is that, onceseasonal patterns have been controlled for, mediareports of suicides, murder, or other violentbehavior should be random events. The spikesin homicides and suicides following such eventscan be argued to be the causal result of mediaexposure. He argued that these effects operatethrough imitation or suggestion. By their design,these experiments are well suited to estimateshort-run effects but not long-run impacts.

Using Natural Experiments 365

One reason to conduct natural experimentsis that they serve as useful complements tolaboratory experiments. Over the past severaldecades, laboratory experiments have greatlyadded to our understanding of the causal effectsof media in the short run. The experimental lit-erature using the laboratory, largely conductedby psychologists, has focused heavily on study-ing the effect of media violence on aggressivebehavior, both for violent video clips and violentvideo games (see Anderson & Bushman, 2001;Anderson et al. 2003; Bushman & Anderson,2002). This experimental literature has con-vincingly demonstrated a causal link betweenviolent media on aggressiveness immediatelyafter exposure, including a sharp increase inaggression for children and young adults. Thesefindings have led many scientists, parents, andpoliticians to worry about the risk of mediaviolence and its negative impacts.

To focus on how the use of a naturalexperiment can add to what can be learnedfrom the laboratory, we make a comparisonto laboratory experiments that expose subjects(typically children or college students) to short,violent film clips. That literature provides causalevidence on the short-run effect of mediaviolence on aggressiveness but not whether thistranslates into higher levels of violent crime inthe field (through arousal or imitation). Dahland DellaVigna (2009) addressed the questionof whether violent movies increase violent crimeusing a natural experiment. They exploited thenatural experiment induced by variation in whenviolent movies are released.

The idea behind this natural experiment isthat in some years there will be strongly violentmovies on certain weekends during the yearwheres in other years those same weekends willhave nonviolent blockbuster movies. Which yearhappens to have a violent blockbuster movieshould otherwise be uncorrelated with violentcrime, after flexibly controlling for seasonality.As an example, in the second week of February2001, the strongly violent movie Hannibal wasreleased, whereas in other years, the secondweek in February was usually dominated byblockbuster romantic comedies. This provides anatural experiment, where the treatment is theavailability of a violent movie compared to thecontrol of the availability of a nonviolent movieon similar weekends.

Dahl and DellaVigna (2009) found thatviolent crime is actually lower on days when

the audience for blockbuster movies is high.Broken down by time of day, in the eveninghours when people are in the theaters, there is asignificant negative effect on crime for stronglyand mildly violent movies; violent crime fallsby around 1.2% per million violent moviegoers. In contrast, weekends with nonviolentblockbuster movies have small and insignificanteffects. The authors interpreted this first effectas ‘‘voluntary incapacitation.’’ On eveningswith high attendance at violent movies, manypotential violent criminals choose to be ina movie theater and are incapacitated fromcommitting a crime during the movie.

In the night-time hours after a violent movieis over, crime falls by an even larger percent,whereas nonviolent movies again have nodiscernable impact. This delayed effect initiallyseems at odds with the psychology experiments,which find large increases in aggressiveness dueto arousal immediately following exposure. Itis, however, important to recognize that thetwo methodologies capture different effects.The laboratory experiments estimate the impactof exposure to a violent movie compared toa nonviolent movie. The natural experimentallows individuals to choose between a movieand an array of alternative activities. Dahl andDellaVigna’s (2009) paper provides evidencethat violent movie attendance is displacingmore violent-prone alternative activities (suchas drinking at a bar).

The natural experiment emphasizes that timeuse plays an important role in how violentmedia affects crime in the short run. Violentmedia keeps violent people busy during themovie and changes an evening’s activities evenafter the movie is over, so that violent movieattendees are less prone to commit violent crime.The psychology experiments emphasize contentalone, which explains the difference in findings.To be clear, Dahl and DellaVigna (2009) werenot arguing for a catharsis effect from violentmovies; they actually found some evidence foran arousal effect, but argued that this effect(which increases violent crime) is more thanoffset by the much larger time use effect (whichlowers violent crime relative to the alternativeactivity).

It is also important to point out thatthe laboratory experiments are capturing thereaction to media violence in a representativesample of the population, whereas the naturalexperiment results are largely driven by which


types of individuals choose to sort into violentmovie attendance. As such, the laboratorysetting is not representative of the effect ofexposure in most field settings, where consumerschoose what media to consume. Rather it isrepresentative of an instance of unexpectedexposure, such as a violent trailer placed withinfamily television programming.

We believe that the causal estimates fromboth the laboratory and the natural experimentare interesting and informative for answeringdifferent questions. For example, the naturalexperiment findings are important for evaluatingpolicies that would restrict access to violentmovies, because these policies will lead to asubstitution toward alternative activities thatmay well be more violent in the short run. Wealso believe, however, that differences betweenthe laboratory and field can be narrowed bychanges in the design of laboratory experiments.For example, laboratory experiments couldincorporate sorting into a violent movie toreplicate selection in the field or can changeexposure to a full-length movie (Lazear,Malmendier, & Weber, 2012).

Although Dahl and DellaVigna (2009) foundno impact of movie violence in the weeks fol-lowing exposure, their design, like the laboratoryexperiments, cannot address the more importantquestion about the long-run effect of violentmedia consumption. In the next section, we turnto natural experiments that can get at the effectsof long-term exposure to television (althoughnot necessary violent media). Before doing so,we provide one more example of a natural exper-iment that addresses short-run effects.

The previous example illustrates how labora-tory and natural experiments complement eachother by answering different questions. Manytimes, however, laboratory or field experimentsfor important family outcomes cannot be donedue to ethical or practical considerations. Onesuch example is whether unexpected emotionalcues (or ‘‘visceral factors’’) play an importantrole in precipitating family violence. Media canhave a powerful potential to affect people’s emo-tions, as has been demonstrated in the laboratory.But linking this evidence to an effect on domes-tic violence is difficult to do, and laboratory andfield experiments attempting to measure actualdomestic violence would be both impracticaland unethical.

Is family violence linked to unexpecteddisappointments (negative emotional cues) that

are broadcast via the media? To help frame thisquestion, consider the theory of gain-loss utilitydeveloped by economists and psychologists,which says that individuals asymmetricallyframe gains and losses around a rationallyexpected reference point (Koszegi & Rabin,2006). In simple terms, this theory says twothings: (a) an individual’s happiness dependsnot on actual outcomes but on outcomes judgedrelative to expectations, and (b) unexpecteddisappointments have a larger impact onindividuals compared to pleasant surprises.

To test such a theory in the context ofdomestic violence, one would need an exper-iment where family violence outcomes couldbe measured after both unexpected positive andnegative surprises. Card and Dahl (2011) usethe natural experiment created by NFL footballgame broadcasts. The setting is motivated bythree reasons. First, a large number of fans arestrongly attached to local teams, with regular-season Sunday games averaging a televisionaudience equal to 25% of all local households.Second, there are detailed game statistics thatmake it easy to identify more or less salientgames. Perhaps most importantly, the existenceof a well-organized betting market allows one toinfer the expected outcome of the game, whichserves as the reference point for gain-loss utility.Each of these characteristics is crucial to theirability to use this setting as a natural experiment.

Does this natural experiment provide ‘‘ran-dom’’ positive and negative upset shocks? Ifbetting markets are unbiased predictors of thegame outcome, the answer is yes. With unbi-ased betting markets, after conditioning on thepregame point spread any differential effect of awin versus a loss is the causal effect of the gameoutcome by construction. So a sufficient condi-tion for the validity of this natural experimentis easy to state and verify. Indeed, empiricalevidence suggests one cannot make money bybetting against the market (i.e., there is strongevidence the market is unbiased).

Card and Dahl (2011) found that policereports of male-on-female intimate partnerviolence rise 10% in areas where the localNFL team lost a game they were favored towin by four or more points. Consistent withreference point behavior, a loss when the gamewas expected to be close or when the local teamwas expected to lose does not have a significanteffect on family violence. Also consistent withthe theory, they found little protective effect of


an unexpected win, pointing to an importantasymmetry in the reaction to unexpected gainsversus unexpected losses.

Timing is an important element of theirnatural experiment, and Card and Dahl (2011)match the hourly local police reports of violenceto the time frame of the football games. Theincreases in violence after an upset loss areconcentrated in a narrow time frame at the endof the game, as one might expect if the spikein violence is due to a transitory emotionalshock. These patterns are more pronounced forupset losses in games that are more emotionallycharged, such as games against a traditionalrival, when the local team was still in playoffcontention, or after a particularly frustratingperformance (an excessive number of sacks,turnovers, or penalties). Their paper suggeststhat a better awareness and management ofexpectations could help to reduce violencewithin families.

One drawback of many natural experiments isthat it can be difficult to identify the mechanisms,even if a causal effect can be documented.For example, in the Card and Dahl (2011)paper, although their analysis points to a causalspike in domestic violence after an unexpectedloss, the mechanism is difficult to unravelwithout further information. For example, theeffect could operate through increased alcoholconsumption in response to the unexpectedloss, but their design and data can only hintat whether this is the mechanism. Anotherinherent challenge with natural experiments isthat they are not always easy to replicate and maynot have external validity. For example, othertypes of unexpected and upsetting events mayaffect different segments of the population oroperate in different ways, and therefore have adramatically different impact on violent crime.

It is important to discuss the role of controlvariables in natural experiments versus bothlaboratory experiments and field experiments.In laboratory and field experiments, the onlyreason to include control variables is toimprove the efficiency of the estimates (i.e., todecrease the standard errors). This is becausewith a laboratory or field experiment, thetreatment is asymptotically uncorrelated withthe covariates as a result of randomization.In many natural experiments, however, it isimportant to include a set of control variables,because often the treatment is only a randomevent conditional on those controls. In the

natural experiments that rely on timing, it isimportant to control for seasonality, as in theDahl and DellaVigna (2009) example. This isalso true for natural experiments where thereis a conditioning variable that captures all theavailable information up to a point in time, suchas a natural experiment involving stock marketprices or betting odds (see Card & Dahl, 2011).

LONG-RUN EFFECTS OF MEDIA

The short-run effects of media are primarilyof interest because of the belief that theaccumulation of short-run effects is likely toresult in long-run changes in behavior. Theability to influence long-run media exposure inan experimental setting, however, is extremelylimited. As a result, most of the research on long-run effects of media is based on correlationsbetween media exposure and the outcome ofinterest (controlling for as many confoundingfactors as possible). There is often emphasisput on collecting longitudinal data so thatthe temporal ordering of media exposure andmeasured behavior can be established.

Even with the best longitudinal data, theprimary concern about nonexperimental dataremains: Individuals get to choose the amountand type of media exposure. As such, there maybe unobserved characteristics about individualsthat influence both their media choices and theiroutcomes (even if these outcomes are measuredat a later period). Many of these omitted vari-ables can be included as controls in the analysis,but there is always the possibility that importantcharacteristics are not observed or might evenbe unmeasureable. If these unobserved charac-teristics affect both media consumption and theoutcome, the resulting estimate of media expo-sure will be biased. This problem of unobservedheterogeneity affects many important areas ofresearch that involve individual choices, such aschurch attendance, marriage, and paternal andmaternal employment, all of which are likelyto influence important family outcomes but forwhich the selection problem is difficult to dealwith.

Although observational data in these settingscan provide important descriptive relationships,economists’ focus on causal estimates has forcedthem to look for natural experiments, a quasi-experimental method that tries to approximatethe gold standard of a randomized trial. Thekey to this approach is to look for a situation


in which there are two groups of individualswith similar characteristics but who experiencedifferent levels of exposure to media (due toreasons that are unrelated to their own individualattributes and choices).

Except in rare cases, there are generally notany direct statistical tests that can be usedto establish validity, because the question iswhether treatment is correlated with an unob-served variable. Rather, it is a combinationof understanding why the natural experimentoccurred, indirect and placebo tests as falsifi-cation exercises, and supplementary empiricalevidence that together can provide a convincingcase. Many of the common and convincing indi-rect tests involve the timing of the treatment andoutcomes. Such arguments and indirect tests aimto convince the reader that the source of varia-tion in media exposure is not being influenced bythe decisions and characteristics of the individ-ual (particularly those related to the outcome ofinterest). Because each application is unique andhas its own set of plausible alternatives that mustbe ruled out, we explore arguments and indirecttests for quasi-random assignment in the contextof the several examples that follow.

Recent work by Olken (2009) provided anexcellent illustration of the natural experimentapproach. Indonesia’s mountainous terrain pro-vides a situation in which two very similar andclosely located villages can experience very dif-ferent levels of television signal strength basedon which side of the mountain they happen to beon. By combining models of signal strength withdetailed geographic information system data onthe topography of Indonesia, Olken constructeda measure of the signal strength in each villageand separated the variation in the strength ofsignal across villages into the part that is dueto predictable factors (distance to major city,elevation, etc.) and the part due to the ‘‘hap-penstance of topography.’’ He found that thetopography-induced differences in exposure totelevision and radio cause individuals in the moremedia-exposed villages to become less involvedin community organizations.

Although not the focus of his research, theapproach Olken (2009) used could be applied tooutcomes that are more specifically related to thefamily, such as fertility, divorce, or child well-being. The ability to extend this type of researchhinges crucially on the availability of the relevantdata for the outcome of interest in locationsand time periods where the natural experiment

occurs. In fact, one of the primary criticisms ofthe natural experiment approach is that it limitsresearchers to looking at those settings in whicha natural experiment has occurred. For better orworse, this feature of natural experiments has ledmany researchers to use historical data to lookat interesting stages in the diffusion of differentforms of media (Gentzkow & Shapiro, 2008;Stromberg, 2004) and then provide thoughtfulanalysis of whether the results at a certainplace and time have external validity and canbe applied to other settings.

The ideal natural experiment is often unavail-able. As a result, researchers will examine situ-ations in which the variation in media exposureis not as random as the topography of Indonesia,but might still unfold in ways that are unrelated tothe characteristics of the individuals being influ-enced by the media. R. Jensen and Oster (2009)used a 3-year panel of individual-level data from180 villages in India. During this time period,21 of these villages received cable television forthe first time. Their empirical approach com-pared outcomes ‘‘between survey rounds acrossvillages based on whether (and when) they addedcable television’’ (p. 1059). They found that theintroduction of television decreased acceptanceof domestic violence, increased the amount ofautonomy experienced by women, and led tolower rates of fertility.

One of the concerns about this setting isthat trends in other variables (such as incomeor ‘‘modernity’’) may cause certain villages toboth adopt cable television and also experiencechanges in the outcomes of interest. R. Jensenand Oster (2009) used many of the traditionalmethods in this vein of research to provide sup-port for the validity of their natural experiment.First, they showed that there were no preexistingdifferential trends in these outcomes between thevillages that did or did not adopt cable television.Second, they showed that the outcomes of inter-est are not correlated with future cable access.That is, behavior this year cannot be predicted byusing information on whether you adopt cablenext year, which would often be the case if therelationship being estimated is driven by under-lying but unobserved differences between thetreatment and control groups. Third, they col-lected additional information about how cableoperators made the decision about which vil-lages to operate in. They found that this decisionis largely driven by issues related to the costof supplying the cable rather than village-level


demand issues (in particular those that mightinfluence the outcomes).

A piece of supporting evidence in the naturalexperiment approach is to provide a theoreticalexplanation for the relationship between thechange in media and the outcome of interest. R.Jensen and Oster (2009) discussed some of theways in which television may have brought aboutthe changes they document (some of which theywere able to provide supporting evidence forusing additional data they collected). Increasedtelevision access may alter fertility choices byproviding more information on family planningalternatives. It may also change the value ofwomen’s time or increase the relative value ofleisure for men, leading men to grant greaterliberty for women to do more outside of thehome. Also, television exposes rural villagesto urban culture, a culture that exhibits fewercases of acceptable wife beatings, less interestin having their next child be male, and greaterautonomy for women.

There have also been studies outside ofeconomics that use the natural experimentapproach to study this issue. Centerwall (1989)used differences across countries in the timingof television adoption to look at the effectit had on homicide rates. Joy, Kimball, andZabrack (1986) examined a town in Canadathat did not receive television until 1973 andcompared the change in physical aggressionamong children between this town and twocontrol towns. Both of these studies found thatthe introduction of television led to an increase inviolence. Subsequent criticisms of Centerwall’sresearch highlight some of the issues that wediscuss in this paper, such as the importanceof examining preexisting trends (G. Jensen,2001) and controlling for confounding factors(Felson, 1996).

In both studies, the small number of relevantgeographic units (three countries or threetowns) makes precise inference a challenge. Incontrast, the more recent research in economicsused multiple geographic units within a singlecountry. This makes it possible to control forcountry-level changes over time and providesenough distinct geographic units to providemeaningful statistical inference after properlyclustering the standard errors (Bertrand, Duflo,& Mullainathan, 2004).

A pair of recent papers using data from Brazilprovides more specific evidence that the effectof television can operate through the content

of the shows (Chong & La Ferrera, 2009; LaFerrara, Chong, & Duryea, 2008). Rather thanlooking at the diffusion of a media technology,they examined the diffusion of a particulartype of show on television, the soap opera. InBrazil, the television network Rede Globo hada virtual monopoly on the production of soapoperas, a set of shows that depict a lifestylethat involves smaller family sizes and higherlevels of divorce and infidelity. These papersexploited differences in the timing of when theGlobo network entered different areas of Braziland found that the arrival of this network led toincreased rates of divorce (Chong & La Ferrera,2009) and lower levels of fertility (La Ferraraet al., 2008).

The primary concern about this naturalexperiment is whether the Globo networkdecided to locate in those areas that were alreadystarting to experience the demographic changesthat are depicted in the soap operas. The authorsused many of the common approaches in thisliterature to establish the validity of the naturalexperiment. First, they examined changes infertility during the years before and after theentry of Globo and found no decrease in fertilityprior to entry but a sharp decline immediatelyafter the arrival of Globo. Second, they showedthat which areas received the Globo networkare not the areas that were already experiencinglarger changes in fertility or divorce. Third,they documented that the presence of Globoin neighboring areas did not influence localfertility (which would be the case if the resultswere driven by unobserved cultural or economicfactors that might be common to adjoiningareas).

La Ferrara et al. (2008) also used within-show variation to provide further evidence thatthe effects are operating through the contentbeing depicted on these shows. They foundthat children born in areas receiving Globoare much more likely to be named after themain characters of the soap operas aired thatyear. Also, the fertility effects are strongest forthe women who are closest in age to those ofthe main female characters in that particularyear. These two examples illustrate the type ofadditional evidence that can be used to supportthe main findings of a natural experiment andhelp provide insight into the specific mechanismthrough which the effect is operating.

Other examples of recent research, althoughless focused on outcomes related to the family,


highlight the type of natural experiments thatcan be used to examine the more generaleffects of media on society. Each paper followsthe same general approach: (a) documentingthe differences across similar individuals in theamount or type of media they were exposedto (the strength of the natural experiment),(b) providing evidence that the difference inexposure was not determined by other factorsthat could induce a spurious relationship(the validity of the natural experiment), and(c) documenting the changes in the outcome ofinterest that accompanied the change in mediaexposure.

DellaVigna and Kaplan (2007) exploitedvariation in the content that households receiveon television, focusing on the effects of theintroduction of Fox News (a conservative cablenews channel) that had reached about 20%of towns in the United States by November2000. They first showed that the availability ofthis channel in different towns appears to belargely idiosyncratic after conditioning on a setof basic controls. They found, however, that thetowns that received Fox News between 1996and 2000 experienced a 0.4 to 0.7 percentagepoint increase in the fraction of votes goingto the Republican candidates. They noted thatthis change in behavior might represent a‘‘temporary learning effect for rational voters,or a permanent effect for non-rational voterssubject to persuasion’’ (p. 1187).

Gentzkow and Shapiro (2008) used informa-tion on the timing of the arrival of broadcasttelevision in different counties in the UnitedStates to examine the pressing concern voicedby parents about whether ‘‘television rots thebrain.’’ They used a large nationally representa-tive data set on test scores during the 1940s and1950s (when television diffusion occurred in theUnited States) and found that each additionalyear of childhood exposure to television actu-ally increased test scores during adolescence by0.02% of a standard deviation.

They also considered whether differentgroups of children were affected differentlyby exposure to television. This part of theiranalysis highlighted the fact that, for manychildren, the effects of exposure to media aredetermined largely by the type of activities thatare being displaced by television. For childrenin the types of homes with the highest levels ofparental involvement, the effect of televisionwas negative (although this effect was not

statistically significant). For these households,the additional media time was likely replacingother activities that would have provided bettercognitive development. They found, however,much larger positive effects for children fromhouseholds where English is not the primarylanguage, for children whose mothers have lessthan a high school education, and for non-White children. These homes provide muchless language-rich environments (Dickinson &Snow, 1987), so that the alternative use of timewould have provided less cognitive developmentthan television. In addition, television may havepulled children away from harmful influencesoutside of the home or diffused situations in thehome that lead to abuse. This last point illustratesthe importance of considering both the exposureeffect and the time-use effect of media (a pointalso highlighted by the Dahl & DellaVigna,2009, paper discussed in the previous section).

DISCUSSION

The purpose of this article is to describe theeconomist’s approach to examining the effectof media on outcomes that are important to thefamily. The primary approach of economistshas been the use of natural experiments thatprovide quasi-random variation in the amount,type, or content of the media that individualshave access to. Examples of these naturalexperiments include changes in the content ofmovies released in a particular week, differencesacross areas in the strength of television signalsthey can receive, and differences in the timing ofwhen different technologies or specific contentproviders arrive in different parts of a country.Each of these create settings where otherwisesimilar individuals experience different types ofaccess to media, and these differences can beused to estimate the effect of media on families.

Table 1 provides a summary of each of thenatural experiments that we describe in thispaper. The information in the publication yearcolumn highlights the fact that economists arerecent arrivals to studying the effects of media,an area of research that has been examined byother fields for a much longer period of time. Assuch, it is easy for these new arrivals to ignore theimportant research done in other fields and theaccumulation of knowledge that has occurredover the decades in which economists playedalmost no role in this research area. It is alsoeasy for fields that have a more established


Tab

le1.

Sum

mar

yof

Nat

ural

Exp

erim

ents

byE

cono

mis

tsR

elat

edto

the

Effe

cts

ofM

edia

Pape

rY

ear

Med

iaE

xpos

ure

Nat

ural

Exp

erim

ent

Out

com

esM

ain

Find

ing

Dah

l&D

ella

Vig

na20

09V

iole

ntm

ovie

sin

thea

ters

Tem

pora

lvar

iatio

nin

the

viol

ence

leve

lof

bloc

kbus

ter

mov

ies

Vio

lent

crim

eE

ach

mill

ion

peop

lew

atch

ing

avi

olen

tmov

iede

crea

ses

viol

entc

rim

eby

1.3%

duri

ngth

eho

urs

the

mov

ieis

inth

eth

eate

ran

dby

1.9%

inth

eho

urs

imm

edia

tely

afte

rex

posu

reC

ard

&D

ahl

2011

Une

xpec

ted

emot

iona

lcu

esfr

omlo

cal

NFL

gam

es

Une

xpec

ted

win

san

dun

expe

cted

loss

espr

ovid

edby

com

pari

ngfin

alN

FLga

me

outc

omes

topr

egam

epo

int

spre

ads

Intim

ate

part

ner

viol

ence

Ups

etlo

sses

lead

toa

10%

incr

ease

inin

timat

epa

rtne

rvi

olen

ce;a

loss

whe

nth

ete

amw

asex

pect

edto

lose

orha

vea

clos

ega

me

has

smal

land

insi

gnifi

cant

effe

cts

Olk

en20

09T

elev

isio

nan

dra

dio

Dif

fere

nces

inte

levi

sion

and

radi

osi

gnal

stre

ngth

due

toth

eto

pogr

aphy

ofvi

llage

sin

Indo

nesi

a

Part

icip

atio

nin

com

mun

ityor

gani

zatio

nsan

dm

easu

res

oftr

ust

An

addi

tiona

lcha

nnel

ofte

levi

sion

rece

ptio

nle

ads

to7%

few

erso

cial

grou

ps,a

ndad

ults

atte

ndin

g11

%pe

rcen

tfew

ergr

oup

mee

tings

Gen

tzko

w&

Shap

iro

2008

Tel

evis

ion

asa

pres

choo

ler

Var

iatio

nin

the

timin

gof

whe

nte

levi

sion

was

intr

oduc

edto

diff

eren

tlo

calm

arke

tsin

the

Uni

ted

Stat

esdu

ring

the

1940

san

d19

50s

Tes

tsco

res

inm

athe

mat

ics,

spat

ial

reas

onin

g,ve

rbal

,and

read

ing

Eac

had

ditio

naly

ear

ofac

cess

tote

levi

sion

incr

ease

dav

erag

ete

stsc

ores

by0.

02%

ofa

stan

dard

devi

atio

n

R.J

ense

n&

Ost

er20

09C

able

tele

visi

onV

aria

tion

inth

etim

ing

ofca

ble

tele

visi

onin

rura

lInd

iaA

ttitu

des

abou

tdom

estic

viol

ence

,son

pref

eren

ce,w

omen

’sau

tono

my,

and

fert

ility

The

intr

oduc

tion

ofca

ble

tele

visi

onde

crea

ses

son

pref

eren

ceby

70%

,the

auto

nom

yin

dex

by41

%,t

henu

mbe

rof

acce

ptab

lebe

atin

gsi

tuat

ions

by46

%,a

ndth

epr

obab

ility

ofbe

ing

preg

nant

attim

eof

surv

eyby

53%

Cho

ng&

La

Ferr

era

2009

Soap

oper

ason

tele

visi

onV

aria

tion

inth

etim

ing

ofav

aila

bilit

yof

the

Red

eG

lobo

netw

ork,

the

maj

orpr

ovid

erof

soap

oper

asin

Bra

zil

Shar

eof

wom

enw

hoar

ese

para

ted

ordi

vorc

edT

heav

aila

bilit

yof

Red

eG

lobo

incr

ease

sth

epr

obab

ility

ofse

para

tion

ordi

vorc

eby

7%

La

Ferr

ara,

Cho

ng,

&D

urye

a20

08So

apop

eras

onte

levi

sion

Var

iatio

nin

the

timin

gof

avai

labi

lity

ofth

eR

ede

Glo

bone

twor

k,th

em

ajor

prov

ider

ofso

apop

eras

inB

razi

l

Fert

ility

rate

sT

heav

aila

bilit

yof

Red

eG

lobo

decr

ease

sth

eav

erag

enu

mbe

rof

child

ren

by1%

and

the

prob

abili

tyof

givi

ngbi

rth

inth

atye

arby

6%D

ella

Vig

na&

Kap

lan

2007

Fox

New

son

tele

visi

onV

aria

tion

inth

etim

ing

ofw

hen

Fox

New

sbe

cam

eav

aila

ble

indi

ffer

ent

part

sof

the

Uni

ted

Stat

es

Rep

ublic

anvo

tesh

are

and

vote

rtu

rnou

tT

owns

whe

reF

oxN

ews

was

intr

oduc

edha

da

0.4

to0.

7pe

rcen

tage

poin

thig

her

Rep

ublic

anvo

tesh

are

inth

e20

00pr

esid

entia

lele

ctio

nsco

mpa

red

to19

96


history of research in this area to discount theresults of economists who are new arrivals tothis research area and in the nascent stages ofcontributing to this research question.

The interdisciplinary nature of the conferencethat led to this special issue illustrates the degreeto which important advances in understandingcan occur when considering the approachesof multiple fields. Ultimately this type ofinterdisciplinary interaction can lead to a morecomprehensive view of the effects of mediaon important family outcomes. This articlehighlights some of the ways in which theeconomist’s approach can complement theimportant research findings of other fields. First,our discussion of the effects of movie violenceillustrates the importance of considering both theexposure effect and the time-use effect of media.The total effect will depend on both the contentof the media and the activities that are displacedduring the time spent using the media. Second,the natural experiments that we describe provideone way to study the long-run effects of media onimportant outcomes (such as divorce, fertility,and domestic violence) for which traditionalexperiments would be unethical or immoral andhandles the challenging issue of unobservedheterogeneity.

NOTE

This research was supported by a Mentoring EnvironmentGrant from Brigham Young University. We are gratefulfor the research assistance provided by Nathan Nazzise andNolan Pope and for the comments provide by participants atthe Media and Family Conference at BYU.

REFERENCES

Anderson, C., Berkowitz, L., Donnerstein, E., Hues-man, L. R., Johnson, J., Linz, D., Malamuth, N.,& Wartella, E. (2003). The influence of mediaviolence on youth. Psychological Science in thePublic Interest, 4, 81 – 110.

Anderson, C., & Bushman, B. (2001). Effects of vio-lent video games on aggressive behavior, aggres-sive cognition, aggressive affect, physiologicalarousal, and prosocial behavior: A meta-analyticreview of the scientific literature. PsychologicalScience, 12, 353 – 359.

Bertrand, M., Duflo, E., & Mullainathan, S. (2004).How much should we trust differences-in-differences estimates? Quarterly Journal of Eco-nomics, 119, 249 – 275.

Bushman, B. J., & Anderson, C. A. (2002). Violentvideo games and hostile expectations: A test of the

general aggression model. Personality and SocialPsychology Bulletin, 28, 1679 – 1686.

Card, D., & Dahl, G. B. (2011). Family violenceand football: The effect of unexpected emotionalcues on violent behavior. Quarterly Journal ofEconomics, 126, 103 – 143.

Centerwall, B. (1989). Exposure to television as arisk factor for violence. American Journal ofEpidemiology, 129, 643 – 652.

Chong, A., & La Ferrara, E. (2009). Televisionand divorce: Evidence from Brazilian novelas,Journal of the European Economic Association, 7,458 – 468.

Dahl, G. B., & DellaVigna, S. (2009). Does movieviolence increase violent crime? Quarterly Journalof Economics, 124, 677 – 734.

DellaVigna, S., & Kaplan, E. (2007). The Fox Newseffect: Media bias and voting. Quarterly Journalof Economics, 122, 1187 – 1234.

Dickinson, D. K., & Snow, C. E. (1987). Interrela-tionships among prereading and oral languageskills in kindergartners from two social classes.Early Childhood and Research Quarterly, 2,1 – 25.

DiNardo, J. (2008). Natural experiments and quasi-natural experiments. In S. Durlauf & L. Blume(Eds.), The new Palgrave dictionary of economics(2nd ed.). Basingstroke, UK: Palgrave Macmillan.

Felson, R. (1996). Mass media effect on violentbehavior. Annual Review of Sociology, 22,103 – 128.

Gentzkow, M., & Shapiro, J. (2008). Preschooltelevision viewing and adolescent test scores:Historical evidence from the Coleman study.Quarterly Journal of Economics, 123, 279 – 323.

Jensen, G. (2001). The invention of television as acause of homicide: The reification of a spuriousrelationship. Homicide Studies, 5, 114 – 130.

Jensen, R., & Oster, E. (2009). The power of TV:Cable television and women’s status in India.Quarterly Journal of Economics, 124, 1057 – 1094.

Joy, L., Kimball, M., & Zabrack, M. (1986). Tele-vision and children’s aggressive behavior. InT. Williams (Ed.), The impact of television:A natural experiment in three communities(pp. 303 – 360). Orlando, FL: Academic Press.

Koszegi, B., & Rabin, M. (2006). A model ofreference-dependent preferences. Quarterly Jour-nal of Economics, 121, 1133 – 1165.

La Ferrara, E., Chong, A., & Duryea, S. (2008). Soapoperas and fertility: Evidence from Brazil. CEPRDiscussion Paper #6785.

Lazear, E. P., Malmendier, U., & Weber, R. A.(2012). Sorting in experiments with application tosocial preferences. American Economic Journal:Applied Economics, 4, 809 – 811.

Meyer, B. D. (1995). Natural and quasi-experimentsin economics. Journal of Business & EconomicStatistics, 13, 151 – 161.


Olken, B. (2009). Do television and radio destroysocial capital? Evidence from Indonesian villages.American Economic Journal: Applied Economics,1(4), 1 – 33.

Pergrams, O. R. W., & Zaradic, P. A. (2006). Islove of nature in the US becoming love ofelectronic media? 16-year downtrend in nationalpark visits explained by watching movies,playing video games, Internet use, and oilprices. Journal of Environmental Management, 80,387 – 393.

Phillips, D. P. (1979). Suicide, motor vehicle fatali-ties, and the mass media: Evidence toward a theoryof suggestion. American Journal of Sociology, 84,1150 – 1174.

Phillips, D. P. (1983). The impact of mass mediaviolence on US homicides. American SociologicalReview, 48, 560 – 568.

Phillips, D. P., & Carstensen, L. L. (1986). Clusteringof teenage suicides after television news storiesabout suicide. New England Journal of Medicine,315, 685 – 689.

Phillips, D. P., & Hensley, J. E. (1984). When vio-lence is rewarded or punished: The impact of massmedia stories on homicide. Journal of Communi-cation, 34, 101 – 116.

Stromberg, D. (2004). Radio impact on publicspending. Quarterly Journal of Economics, 119,189 – 221.

using natural experiments to study the impact of media on the family

Documents