methods for evaluating natural experiments in obesity ...• natural experiments most commonly used...

435
Comparative Effectiveness Review Number 204 Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review e

Upload: others

Post on 28-May-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

Comparative Effectiveness Review Number 204

Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review

e

Page 2: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

Comparative Effectiveness Review Number 204 Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review Prepared for: Agency for Healthcare Research and Quality U.S. Department of Health and Human Services 5600 Fishers Lane Rockville, MD 20857 www.ahrq.gov Contract No. 290-2012-00007-I Prepared by: Johns Hopkins University Evidence-based Practice Center Baltimore, MD Investigators: Wendy L. Bennett, M.D., M.P.H. Lawrence J. Cheskin, M.D. Renee F. Wilson, M.S. Allen Zhang, B.S. Eva Tseng, M.D., M.P.H. Oluwaseun Shogbesan, M.D. Emily A. Knapp, M.H.S. Elizabeth A. Stuart, Ph.D. Eric B. Bass, M.D., M.P.H. Hadi Kharrazi, M.D. AHRQ Publication No. 18-EHC006-EF December 2017

Page 3: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

ii

Purpose of Review To characterize studies of programs and policies in obesity prevention and control in terms of data sources, data linkages, measures reported, study designs, and analytic approaches, and to identify needed methodological advances. Key Messages

• Relevant programs, policies, or built environment changes were evaluated in 156 natural experiments, 118 experimental studies, and 20 other studies.

• Criteria for a data system (source exists, is available for research, is sharable, and has outcomes of interest) were met by 106 data sources.

• Thirty-seven percent of U.S. data systems were linked to secondary data. • Outcome measures included dietary behavior (148 studies), physical activity (152 studies),

childhood weight (112 studies), and adult weight (32 studies). • Natural experiments most commonly used regression models comparing exposed and

unexposed groups at one time. • Natural experiments generally had moderate risk of selection bias and high risk of bias for

losses to follow-up. • Research could be advanced by more use of data dictionaries, reporting standards on data

linkage, long-term obesity-related outcomes, and study designs with multiple pre- and post-exposure time points.

Page 4: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

iii

This report is based on research conducted by the Johns Hopkins University Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2012-00007-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services. None of the investigators have any affiliations or financial involvement that conflicts with the material presented in this report. The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well-informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients. This report is made available to the public under the terms of a licensing agreement between the author and the Agency for Healthcare Research and Quality. This report may be used and reprinted without permission except those copyrighted materials that are clearly noted in the report. Further reproduction of those copyrighted materials is prohibited without the express permission of copyright holders. AHRQ or U.S. Department of Health and Human Services endorsement of any derivative products that may be developed from this report, such as clinical practice guidelines, other quality enhancement tools, or reimbursement or coverage policies, may not be stated or implied. This report may periodically be assessed for the currency of conclusions. If an assessment is done, the resulting surveillance report describing the methodology and findings will be found on the Effective Health Care Program Web site at www.effectivehealthcare.ahrq.gov. Search on the title of the report. Persons using assistive technology may not be able to fully access information in this report. For assistance contact [email protected]. Suggested citation: Bennett WL, Cheskin LJ, Wilson RF, Zhang A, Tseng E, Shogbesan O, Knapp EA, Stuart EA, Bass EB, Kharazzi H. Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review. Comparative Effectiveness Review No. 204. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. 290-2012-00007-I.) AHRQ Publication No. 18-EHC006-EF. Rockville, MD: Agency for Healthcare Research and Quality; December 2017. Posted final reports are located on the Effective Health Care Program search page. DOI: http://doi.org/10.23970/AHRQEPCCER204.

Page 5: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

iv

Preface The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-based

Practice Centers (EPCs), sponsors the development of systematic reviews to assist public- and private-sector organizations in their efforts to improve the quality of health care in the United States. These reviews provide comprehensive, science-based information on common, costly medical conditions, and new health care technologies and strategies.

Systematic reviews are the building blocks underlying evidence-based practice; they focus attention on the strength and limits of evidence from research studies about the effectiveness and safety of a clinical intervention. In the context of developing recommendations for practice, systematic reviews can help clarify whether assertions about the value of the intervention are based on strong evidence from clinical studies. For more information about AHRQ EPC systematic reviews, see www.effectivehealthcare.ahrq.gov/reference/purpose.cfm.

AHRQ expects that these systematic reviews will be helpful to health plans, providers, purchasers, government programs, and the health care system as a whole. Transparency and stakeholder input are essential to the Effective Health Care Program. Please visit the Web site (www.effectivehealthcare.ahrq.gov) to see draft research questions and reports or to join an email list to learn about new program products and opportunities for input.

If you have comments on this systematic review, they may be sent by mail to the Task Order Officer named below at: Agency for Healthcare Research and Quality, 5600 Fishers Lane, Rockville, MD 20857, or by email to [email protected]. Gopal Khanna, M.B.A. Director Agency for Healthcare Research and Quality

Stephanie Chang, M.D., M.P.H. Director Evidence-based Practice Center Program Center for Evidence and Practice Improvement Agency for Healthcare Research and Quality

Arlene S. Bierman, M.D., M.S. Director Center for Evidence and Practice Improvement Agency for Healthcare Research and Quality

Lionel L. Bañez, M.D. Task Order Officer Center for Evidence and Practice Improvement Agency for Healthcare Research and Quality

Page 6: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

v

Acknowledgments The authors gratefully acknowledge the following individuals for their contributions to this

project: Jessica Gayleard, Ritu Sharma, Catalina Suarez, and Lisa Wilson.

Technical Expert Panel In designing the study questions and methodology at the outset of this report, the EPC

consulted several technical and content experts. Broad expertise and perspectives were sought. Divergent and conflicted opinions are common and perceived as healthy scientific discourse that results in a thoughtful, relevant systematic review. Therefore, in the end, study questions, design, methodological approaches, and/or conclusions do not necessarily represent the views of individual technical and content experts.

Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

The list of Technical Experts who participated in developing this report follows: David Allison, Ph.D.* Dean and Provost Professor Indiana University School of Public Health Bloomington, IN Cheryl Anderson, Ph.D., M.P.H.* Associate Professor Department of Family Medicine and Public Health University of California, San Diego La Jolla, CA Donna Coffman, Ph.D.* Dept. of Epidemiology and Biostatistics Temple University Philadelphia, PA Trina Histon, Ph.D. Senior Principal Consultant Kaiser Permanente Oakland, CA *Provided input on Draft Report.

Lisa Sanbonmatsu, Ph.D.* National Bureau of Economic Research Cambridge, MA Brandon Simms Greater Gethsemane Missionary Baptist Church Baltimore, MD Xuemei Zhu, Ph.D.* Associate Professor University of Texas College of Architecture Texas A&M University College Station, TX

Page 7: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

vi

Peer Reviewers Prior to publication of the final evidence report, EPCs sought input from independent Peer

Reviewers without financial conflicts of interest. However, the conclusions and synthesis of the scientific literature presented in this report do not necessarily represent the views of individual reviewers.

Peer Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals with potential nonfinancial conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential nonfinancial conflicts of interest identified.

The list of Peer Reviewers follows: Shiriki Kumanyika, Ph.D., M.P.H. Professor Emerita of Epidemiology University of Pennsylvania Philadelphia, PA Marian Neuhouser, Ph.D., R.D. Affiliate Professor of Epidemiology University of Washington Seattle, WA

Page 8: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

vii

Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review Structured Abstract Objectives. Obesity is an enormous public health problem among adults and children. Our objective was to systematically review studies evaluating programs and policies addressing obesity prevention and control in terms of their population-based data sources, use of data linkages, measures reported, study designs, and analytic approaches. The overarching goal of the review was to identify methodological advances that could strengthen research that uses natural experiments to evaluate the effectiveness of policies and programs to prevent and control obesity. Data sources. We systematically searched PubMed®, CINAHL®, PsycINFO®, and EconLit from 2000 to August 21, 2017, to identify all U.S. and non-U.S. studies of programs or policies targeting obesity prevention and control in people of all ages and in any setting. Review methods. Two independent reviewers screened abstracts and full-text articles. We required articles to be in English; address a program, policy, or built environment change; include 100 or more study subjects; and have a defined comparison or unexposed group. We used the Effective Public Health Practice Project (EPHPP) tool to rate studies for their risk of bias. This tool rates studies for their ability to draw causal inferences about program effectiveness. Results. The search identified 26,316 unique citations. Of the 294 studies (reported in 312 articles) eligible for inclusion (188 U.S. and 106 non-U.S.), 156 (53%) were natural experiment studies, 118 (40%) were experimental studies (randomized or nonrandomized controlled trials), and 20 (7%) had other study designs that did not fall into either of the other categories.

Data sources: We identified 143 secondary data sources and 26 sharable primary data sources, totaling 116 sharable data sources after duplicates were removed. Criteria for a data system (data source exists, is available for research, is sharable, and contains outcomes of interest) were met by 106 data sources (71 U.S. and 35 non-U.S.). Sixty-two percent of the U.S. data systems contained at least one of the main measures for weight or body mass index in adults or children, or dietary or physical activity behaviors. Fifty-three percent of the U.S. data systems included at least one outcome related to the food environment, physical activity environment, commuting behavior, or purchasing behavior, or included information about a relevant exposure in a policy, program, or built environment change. These 71 U.S. data systems often reported more than one outcome. Thirty-seven percent of the U.S. data systems were linked with a secondary data source or system other than the primary data source. Most studies that linked their data systems with external data systems used an individual-level key or a geographic allocation.

Outcomes/measures: Of the 294 included studies, we identified 112 studies with childhood weight measures, 32 studies with adult weight measures, 152 studies with physical activity measures, and 148 studies with dietary measures. Thirty-seven of the 294 studies reported on

Page 9: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

viii

outcomes related to the food environment, physical activity environment, commuting behavior, or purchasing behavior.

Study design and methods: Natural experiment studies most commonly used cross-sectional comparisons of exposed and unexposed groups (n=55; 35%). Difference-in-differences approaches that compared exposed and unexposed groups before and after an exposure were used in 45 studies (29%), while 48 studies (31%) used pre/post designs that compared one group before and after an exposure. Most natural experiment studies were rated as having a “weak” global rating (i.e., high risk of bias), with 63 percent having a weak rating for handling of withdrawals and dropouts, 42 percent having a weak rating for study design, 40 percent having a weak rating for confounding, and 26 percent having a weak rating for data collection. Experimental studies were rated as “strong” (low risk of bias) in study design, control of confounding, and data collection methods, but were weaker in blinding and selection bias. We identified methodological and analytic advances that would help to strengthen efforts to estimate the effect of programs, policies, or built environment changes on obesity prevention and control, such as consistent use of data dictionaries, reporting standards on linkage methods of data sources, data sources with long-term public health surveillance of obesity and health behavioral outcomes, and use of study designs with multiple pre- and post-exposure time points.

Conclusions. Our systematic review identified numerous natural experiment studies (n=156) and data sources, including sharable and non-sharable data sources (n=216), that have been used to estimate the effect of programs, policies, or built environment changes on obesity prevention and control. The studies used a wide variety of outcome measures and analytic methods, often with substantial risk of bias. The findings reinforce the need for methodological and analytic advances that would strengthen efforts to improve obesity prevention and control.

Page 10: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

ix

Contents Introduction ....................................................................................................................................1

Background ................................................................................................................................1 Obesity and Its Public Health Consequences .......................................................................1 Framework for Change: Public Health Policy and Neigborhood Approaches to Obesity

Prevention and Control ........................................................................................................2 Challenges in Evaluating Programs, Policies, and Built Environment Changes Aimed at

Preventing or Controlling Obesity .......................................................................................2 Natural Experiments To Address Obesity Prevention and Control .....................................2 Approach to Review and Identification of Key Questions ........................................................3 Scope of Review and PICOTS ...................................................................................................4 Populations ...........................................................................................................................4 Interventions ........................................................................................................................4 Comparators .........................................................................................................................6 Outcomes and Measures ......................................................................................................6 Timing ..................................................................................................................................7 Setting ..................................................................................................................................7 Analytic Framework ..................................................................................................................7 Organization of Report ..............................................................................................................7

General Methods and Results .......................................................................................................9 Methods......................................................................................................................................9

Search Methods .......................................................................................................................9 Inclusion and Exclusion Criteria ...........................................................................................10 Data Abstraction and Data Management ..............................................................................11 Grading Strength of Evidence ...............................................................................................11 Assessing Applicability ........................................................................................................12 General Data Abstraction ......................................................................................................12

Results ......................................................................................................................................15 Results of Literature Search ..................................................................................................15 General Study Details ...........................................................................................................15 Programs, Policies, and Built or Environmental Changes Identified in Included Studies ..15

Key Question 1: Population-Based Data Sources .....................................................................19 Overview ..................................................................................................................................19 Key Findings ............................................................................................................................19 Methods....................................................................................................................................19

Data Abstraction ...................................................................................................................19 Data Synthesis .......................................................................................................................20

Results ......................................................................................................................................21 Identification of Data Sources and Data Systems .................................................................21

Key Question 2: Linking Data Sources ......................................................................................27 Overview ..................................................................................................................................27 Key Findings ............................................................................................................................27 Methods....................................................................................................................................27

Data Abstraction ...................................................................................................................27 Results ......................................................................................................................................27

Key Question 3: Assessment Measures ......................................................................................29

Page 11: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

x

Overview ..................................................................................................................................29 Key Findings ...........................................................................................................................29 Methods....................................................................................................................................29

Data Abstraction ...................................................................................................................29 Data Synthesis ......................................................................................................................30

Results ......................................................................................................................................30 Obesity Measures: Weight and BMI in Children and Adults ...............................................30

Measures of Dietary Behaviors in Adults and Children .......................................................32 Measures of Physical Activity in Adults and Children .........................................................34 Other Co-Outcomes ..............................................................................................................37

Key Question 4: Experimental and Non-Experimental Methods ...........................................39 Overview ..................................................................................................................................39 Key Findings ............................................................................................................................39 Methods....................................................................................................................................39

Data Abstraction ...................................................................................................................39 Data Synthesis .......................................................................................................................40

Results ......................................................................................................................................40 Description of Methods Used in Natural Experiment Studies ..............................................41 Description of Methods Used in Experimental Studies ........................................................41 Description of Methods Used in Other Study Approaches ...................................................41

Key Question 5: Risk of Bias.......................................................................................................43 Overview ..................................................................................................................................43 Key Findings ............................................................................................................................43 Methods....................................................................................................................................43

Data Abstraction ...................................................................................................................43 Data Synthesis .......................................................................................................................44

Results ......................................................................................................................................47 Risk of Bias in Natural Experiment Studies .........................................................................47 Risk of Bias in Experimental Studies ...................................................................................49 Risk of Bias in Other Study Designs ....................................................................................50 Risk of Bias Assessment Using Another Scale.....................................................................51

Key Question 6: Methodological Advances ...............................................................................52 Overview ..................................................................................................................................52 Key Finding .............................................................................................................................52 Methods....................................................................................................................................52

Data Abstraction ...................................................................................................................52 Data Synthesis .......................................................................................................................53

Results ......................................................................................................................................53 Discussion .....................................................................................................................................56

Findings in Relationship to What Is Already Known .................................................................57 Applicability ...............................................................................................................................57 Limitations of the Systematic Review Process ...........................................................................58 Limitations of the Current Evidence ...........................................................................................60 Future Research Needs and Opportunities ..................................................................................61 Conclusions .................................................................................................................................62

References .....................................................................................................................................63

Page 12: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

xi

Tables Table 1. PICOTS framework for Key Questions, with inclusions and exclusions described .........5 Table 2. Study and participant characteristics abstracted ..............................................................13 Table 3. Intervention details abstracted .........................................................................................14 Table 4. Summary of study methods; programs, policies, and built environment goals; and

targets of intervention (N=294) ..................................................................................................17 Table 5. Criteria to determine if a data source meets the criteria of a data system related to

obesity .........................................................................................................................................20 Table 6. Weight outcomes and measures in adults and children, by study design. .......................30 Table 7. Dietary outcomes and measures for children and adults, by study design. .....................35 Table 8. Physical activity outcomes and measures, by study design .............................................36 Table 9. Summary of distribution of co-outcomes among study designs (n=37 studies) ..............38 Table 10. Overview of study design or data collection structure (n=294 studies) ........................42 Table 11. Analytic methods that can be used in non-experimental studies and specific bias

concerns ......................................................................................................................................45 Table 12. Summary of Effective Public Health Practice Project quality assessment tool .............47 Table 13. Common categories of control variables accounted for in natural experiment studies

(n=156) .......................................................................................................................................49 Table 14. Analytic and methodologic advances identified by internal and external experts. .......55 Figures Figure 1. Analytic framework for obesity prevention and control ..................................................8 Figure 2. Results of literature search .............................................................................................16 Figure 3. Identification of data sources and data systems in included studies. .............................23 Figure 4. Level of information available for U.S. data systems (n=71) ........................................24 Figure 5. Original purpose for data use and function for U.S. data systems (n=71) .....................25 Figure 6. Demographic coverage of U.S. data systems (n=71) .....................................................25 Figure 7. Coverage of obesity-related outcomes and exposures in U.S. data systems ..................26 Figure 8. Risk of bias for natural experiment studies (N=156) .....................................................48 Figure 9. Risk of bias for experimental studies (N=118) ..............................................................50 Figure 10. Risk of bias for other study designs (N=20) .................................................................51

Appendixes Appendix A. Acronyms Appendix B. Glossary of Key Terms Appendix C. Detailed Search Strategies Appendix D. Abstraction Forms Appendix E. Data System Classification/Coding Schema Appendix F. Approach to Identifying Methodological/Analytic Advances That Would Help Strengthen Efforts To Estimate the Effect of Programs, Policies, or Built Environment Changes on Obesity Prevention and Control Appendix G. Included Studies Appendix H. Evidence Tables Appendix I. Programs, Policies, or Built Environment Changes in Included Studies (U.S. Only) Appendix J. Data Systems Identified by the Systematic Review

Page 13: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

xii

Appendix K. Individual Study Risk of Bias Ratings Appendix L. Comparison of Two Tools for Assessing Study Risk of Bias

Page 14: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

1

Introduction Background

Obesity and Its Public Health Consequences The prevalence of overweight and obesity among adults and children has increased dramatically in the United States in recent years.1 Seventeen percent of U.S. children and adolescents are obese2, 3 and 69 percent of adults are overweight or obese.4 Obesity is now a worldwide epidemic, according to the World Health Organization, with an estimated 1.9 billion adults overweight or obese globally.5, 6 Obesity is associated with multiple adverse health consequences, including type-2 diabetes, cardiovascular disease, and many cancers,7-10 which have, in part, led to rising health care costs.8, 11 Economic implications of obesity also include higher absenteeism and reduced work productivity.12 The drivers of the obesity epidemic are complex, and involve intertwined, multilevel, and dynamic forces that influence health behaviors related to eating and physical activity, food systems, and access to physical activity-friendly environments.13, 14 Therefore, the search for effective solutions to prevent and control obesity needs to extend beyond the focus on the individual, to address the local neighborhood context, as well the social/cultural/political context unique to regions and countries.6 Since obesity is associated with racial and socioeconomic disparities,15 to successfully address obesity, we ultimately need programs and policies that can also reduce social and health inequities.

Framework for Change: Public Health Policy and Neighborhood Approaches to Obesity Prevention and Control In 2012, the Institute of Medicine (IOM) (see Appendix A for a list of acronyms) released a report entitled “Accelerating Progress in Obesity Prevention: Solving the Weight of the Nation.”16 The goal of the report was to evaluate strategies for their potential to prevent obesity. The Report identified several broad policy and programmatic goals, including making physical activity a routine part of life, creating a food/beverage environment that includes healthy options, creating messaging that can help to improve physical activity and nutrition, and using schools as an important focus. In addition, the report presented a “systems approach,” suggesting five critical areas—or environments for change: (1) environments for physical activity, (2) food and beverage environments, (3) message environments, (4) health care and work environments, and (5) school environments. Aligned with the targets recommended by the IOM Report, non-governmental and governmental policies have been enacted and implemented. Examples include a sugar-sweetened beverage tax in Berkeley, California,17 menu or calorie-labeling regulations in New York City,18-

20 and supporting new supermarkets in food deserts.21 In addition, many school systems have developed programs aimed at increasing children’s fruit and vegetable consumption, and increasing time spent outside during recess.22, 23 Rigorous published evaluations of these policies and programs, including those whose goal is to promote increased physical activity and active transportation, are enhancing the evidence base and increasing the ability of policymakers and funders to adapt, scale and disseminate those that are effective.24

Page 15: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

2

Individual or group-based behavioral programs have been the cornerstone of effective weight loss interventions in adults.25 In their recommendations to increase physical activity and improve nutrition, the Centers for Disease Control and Prevention (CDC) included both the “systems” approach, as well as individually-focused, behaviorally-oriented programs to guide individuals in making changes in behavior.26, 27 Many behavioral interventions involve in-person or remotely delivered (phone/ mobile phone/ internet based) behavioral counseling to enhance individuals’ motivation to make and sustain decreased caloric intake and increased physical activity.28 Though often effective for short-term weight loss, the challenge with individually-focused interventions has been our ability to scale and sustain them in a community or healthcare setting. One example of a successfully scaled program is called CHAMPS (Community Healthy Activities Model Program for Seniors). After it was shown to be effective in a randomized controlled trial (RCT), CHAMPS was adapted for diverse communities and scaled for delivery in senior centers in San Francisco, California.29 The most recognizable behavioral weight loss trial, aimed at preventing type 2 diabetes in overweight adults, was the Diabetes Prevention Program (DPP).30 This intervention has been adapted as a community program and is being widely disseminated through the CDC’s National DPP program, with in-person and remotely-delivered classes in over 1,200 programs in the United States.31

Challenges in Evaluating Programs, Policies, and Built Environment Changes Aimed at Preventing or Controlling Obesity Challenges in evaluating obesity interventions stem in part from the complexity of the obesity problem, which tends to thwart interventions that focus on single and even multiple drivers of the problem, and in part from a lack of standards for designing and evaluating interventions.6 Evaluations need to take advantage of existing data sources; link policy, program or transportation data with clinical or health data, such as electronic health records (EHRs); and follow populations over time to assess impact. The National Institutes of Health, through multiple funding opportunity announcements and meetings, and the Patient Centered Outcomes Research Institute have invested in two types of studies to tackle the obesity epidemic: 1) traditional study designs involving a pragmatic clinical trial to integrate obesity care into primary care for underserved populations;32 2) enhancement of the infrastructure of EHR data linked between health systems (called the Clinical Data Research Networks), to support observational studies about obesity.33 However, RCTs of obesity-prevention and control policies and programs may not always be feasible or appropriate. Economic, urban planning, systems-modeling and legislative policy evaluation approaches are being adapted to answer questions about the health impact, as well as the environmental and unintended positive and negative consequences of programs and policies.34, 35

Natural Experiments To Address Obesity Prevention and Control Novel approaches are needed to evaluate the effect of population health policies and programs, which are not always under the control of researchers using experimental designs. An important focus of this review is the opportunity to appraise and evaluate the approach known as a “natural experiment,” as well as other non-experimental designs, designed to study the effects

Page 16: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

3

of programs, policies, and changes to the built environment and social structures. Although natural experiments and other non-experimental designs are not new to public health, more publications are using the term “natural experiment,” and non-experimental designs are growing in popularity.36 For example, researchers used natural experiment approaches to examine the effect of regulation limiting the use of serotonin reuptake inhibitors in youth on suicide rates in the United Kingdom (UK),37 and of tobacco control policies on hospitalizations for acute myocardial infarction.38-40 To summarize the methodology and provide a standard definition, The UK’s Medical Research Council (MRC) recently released guidance to assist researchers in conducting and evaluating the rigor of natural experiment designs.41, 42According to the MRC, the key features of a natural experiment are that “(1) the intervention is not undertaken for the purposes of research, and 2) the variation in exposure and outcomes is analyzed using methods that attempt to make causal inferences.”41, 42 Their natural experiment study definition was not limited to specific study designs. Applying the MRC definition to categorize existing studies is challenging, as few studies are explicitly labeled as natural experiments. In addition, the stage at which researchers and evaluators became involved with policies or programs may not always be clear, and even when it is, the degree of “control” the research team has may not be well described. Finally, although most studies using natural experiment approaches use non-experimental designs, it is also possible to embed a natural experiment within an RCT, such as the Moving to Opportunities study.43 In this study, the U.S. Department of Housing and Urban Development randomly assigned people to receive a rental subsidy in a low poverty neighborhood, with the goal of assessing social and economic outcomes. While not explicitly calling their work a natural experiment, health researchers took advantage of a randomized design to collect data about diabetes and obesity. They showed that the group with the voucher had a lower prevalence of severe obesity and diabetes.43 The goal of natural experiment designs is to obtain an approximate estimate of causality of a policy, program, or built-environment change on a public health outcome. The major challenge is in the selection of an “unexposed” comparison group. Selection bias is a common source of bias in natural experiment approaches. Many non-experimental study designs, such as use of propensity scores, interrupted time series, and regression discontinuity, are useful in identifying appropriate comparison groups.44 Most, but not all, natural experiment studies rely on existing data sources, and evaluating the strengths and limitations, and the types of measures these sources contain, is important to advancing future research.

The overarching goals of this review were to 1) improve understanding of the population-based data sources that have been used to evaluate programs, policies, and built environment changes designed to prevent or control obesity, and 2) identify methodological/analytic advances that would help strengthen future efforts to evaluate the programs, policies, and built environment changes related to obesity prevention and control. This review focuses on methods and does not evaluate the effectiveness of the programs and policies on the outcomes of interest. The review has a broad scope, including both natural experiment and experimental studies, to identify evaluations of programs, policies and built environment changes that aimed to prevent or control obesity in children and adults.

Approach to Review and Identification of Key Questions The Key Questions were identified to inform the Pathways to Prevention Workshop Methods

for Evaluating Natural Experiments in Obesity. The results of this systematic review will provide

Page 17: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

4

background on the state of the evidence for this Workshop. To assist the reader, we also developed a glossary with definitions of key terms (Appendix B). Our project sought to address the following Key Questions (KQs):

KQ1. What population-based data sources have been used in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes?

KQ2. What methods have been used to link different population-based data sources?

KQ3. What obesity measures, dietary and physical behaviors, and other outcomes have been assessed in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control?

KQ4. Which experimental and non-experimental methods have been used in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes?

KQ5. What are the risks of bias in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes?

KQ6. What methodological/analytic advances (e.g., data system features, approaches to linking data sources, or analytic methods) would help to strengthen efforts to estimate the effect of programs, policies, or built environment changes on obesity prevention and control?

Scope of Review and PICOTS We used the PICOTS typology (Populations, Interventions, Comparators, Outcomes, Timing, Setting) as a framework to define the scope of the review and inform the selection of studies of programs, policies or built environment changes targeting obesity. The PICOTS typology is a standard framework used in systematic reviews to be explicit and transparent about the inclusions/exclusions, search methods, study selection and data extraction, and is informative for discussions with stakeholders. Below we provide a rationale for the inclusion/exclusion criteria by PICOTS category,45 displayed in Table 1.

Populations Studies were included if they targeted a community, school, or worksite population, with participants of any age. We included studies focused on one sub-population, i.e., studies conducted only within overweight or obese people. We excluded studies focused within other clinical sub-populations, such as people who had bariatric surgery or children with Prader-Willi

Page 18: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

5

syndrome, a genetic disorder.

Interventions We included studies of programs, policies or built environment changes targeting population-level behavior change, such as increased physical activity or healthier diet. We included programs, policies or built environment changes that may not have been originally developed for obesity control or prevention purposes, but were evaluated for this purpose. We excluded very broad policies that had very diffuse effects, such as the impact of cigarette taxes on obesity.46 We also excluded individually-focused behavioral interventions such as individual or group-based counseling programs.

Page 19: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

6

Table 1. PICOTS framework applied to Key Questions, with inclusions and exclusions described PICOTS Inclusions and Exclusions

Population(s) Include: • All ages, general population; and sub-populations of obese and overweight individuals Exclude: • Studies within specific clinical populations only, other than obese/overweight populations (e.g.,

severe mental illness, people with genetic predispositions for obesity).

Intervention(s) Include: • U.S. and non-U.S. policies, programs, and built environment changes targeting a population. Exclude: • Programs or policies targeting individual-level intervention and change (not a system or

population-level), e.g., Weight Watchers with individual or group-based behavioral counseling. [Note: if the program or policy includes multiple levels of intervention (e.g., population-level and individual-level) it would be included.]

• School-based studies that occur in a single school in the district. • School-based studies that only involve a change to the curriculum without an environmental or

systems change (e.g., nutrition education change in the health class).

Comparator(s) Include: • Studies with a clearly defined concurrent or non-concurrent comparison group either prior to the

policy or a defined group without exposure to the policy or program • Observational or cohort studies that use regression models to assess the association of a

policy or program on an exposed versus an unexposed group. Exclude:

• Studies without a comparison or unexposed group.

Outcome(s) Include: Outcomes of interest: • Body weight • Body mass index • Individual physical activity behavior assessed using a validated questionnaire that assesses

both quantity and type of activity, or measures physical activity objectively (e.g., step counts). • Individual dietary intake assessed using a validated questionnaire or 24-hour dietary recall,

measuring one or more of the following: - Total daily caloric intake, - Specific dietary macronutrients related to obesity: vegetable, fruit, or fiber intake. - Specific eating behaviors associated with obesity: sugar sweetened beverage intake, or fast

food frequency. Co-outcomes: Co-outcomes were described among included studies which had at least one “outcome of interest.”

Co-outcomes were defined by the following categories: food-environment (e.g., perceived access to fruits and vegetables), physical activity environment (e.g., walkability, observed physical activity at a park or bike path), commuting behavior, and food purchasing behavior.

Exclude: • Studies without reference for validation or use of a validated instrument to measure diet or

physical activity. • Studies with observed behaviors as their only outcomes, such counts of people using a park or

walking path, and studies using the SOPARC/SOPLAY tool for assessing physical activity at a park.

Timing Programs and policies enacted or implemented in 2000 or later. The U.S. Surgeon General's Call To Action To Prevent and Decrease Overweight and Obesity was published in 2001 and marked a turning point to raise public health awareness about obesity.47

Page 20: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

7

PICOTS Inclusions and Exclusions Setting Include:

• U.S. and non-U.S. settings at all levels (e.g., national, state, community/neighborhood). Exclude: • Studies in specific settings that would not be generalizable to a free-living population or community (e.g., prison, nursing home) • Studies in countries listed below the 2016 “human development index” (HDI) of “very high”.48

PICOTS: populations, interventions, comparators, outcomes, timing, setting;SOPARC/SOPLAY=system for observing play and recreation in communities/system for observing play and leisure activity in youth

Comparators We required studies to have a concurrent or non-concurrent comparison or unexposed group. We did not exclude studies based on other elements of study design, as the goal was to broadly describe the obesity natural experiment field and all study design approaches ranging from the strongest designs like RCTs to non-experimental approaches that defined an unexposed group. We did not want to exclude any study designs that had a comparison group, but rather wanted to describe the strengths and limitations of all approaches identified. In addition, evaluations of community- or school-based programs, policies and built environment changes employ a wide range of study designs from experimental to non-experimental. Community-wide obesity prevention studies, like The Massachusetts Childhood Obesity Research Demonstration Study, contain multiple programmatic and policy components, making it more difficult to discern at what point researchers become involved in the design and implementation.49-51 Therefore, we sought to describe the breadth of these approaches, the data sources and outcome measures they used, and the risks of bias associated with the designs, ultimately, to inform future research to advance obesity prevention and control.

Outcomes and Measures We focused on clinically relevant outcomes in adults and children, including weight and body mass index. We also included studies that reported on two main weight-related individual health behaviors, diet and physical activity. Defining the behavioral outcomes of dietary change and physical activity was challenging because many instruments rely on self-report, resulting in varying degrees of comprehensiveness, validity, and reliability.52 We categorized measures consistent with The National Collaborative on Childhood Obesity Research (NCCOR) Measures Registry53 for children’s measures, as well as the literature for adult measures.54 The NCCOR organizes the measures into Individual Diet, Food Environment, Individual Physical Activity, and Physical Activity Environment. We applied similar categories.53 The field of obesity control and prevention has advanced to having a large number of natural experiment studies that address weight, dietary intake and physical activity behaviors,18, 43, 55-62 so we excluded studies that only reported the food or physical activity environment or intermediate outcomes, such as measures of access to healthy food (e.g., stocking of shelves), fruit/vegetable purchases, distance to a park, or “walkability” of a neighborhood. We also excluded studies that only reported observed behaviors, such as the System for Observing Play and Recreation in Communities (SOPARC/SOPLAY),63 which assesses usage of park and recreation areas by observation rather than quantitative measurement of an individual’s change in type and quantity of physical activity. These studies record numbers of individuals engaged in certain activities, and do not measure duration of any of the activities. Finally, for all included

Page 21: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

8

studies, we reported on measures of the food-environment (e.g., perceived access to fruits and vegetables), physical activity environment (e.g., walkability, observed physical activity at a park or bike path), commuting behavior, or purchasing behavior, which we considered to be relevant “co-outcomes.”

Timing We focused on programs and policies enacted or implemented in 2000 or later. The U.S. Surgeon General's Call To Action To Prevent and Decrease Overweight and Obesity was published in 2001 and marked a turning point to raise public health awareness about obesity. A previous systematic review also used 2000 as a starting date.64 We also performed preliminary searching that showed an exponential increase in studies meeting our search criteria were published each year after 2000, with an increase in natural experiment approaches after 2005.36

Setting We included studies evaluating policies, programs, and built environment changes within and outside the U.S. We excluded policies, programs, and built environment changes in countries below the 2016 “human development index” (HDI) of “very high,” to enhance consistency between programs and policies being evaluated across studies. The HDI is a composite statistic of life expectancy, education, and income per capita indicators. A country has a higher HDI when the life expectancy at birth is longer, the education period is longer, and the income per capita is higher. Fifty-one countries are listed in the “very high” HDI category.48

Analytic Framework Figure 1 depicts the KQs within the context of the PICOTS described in the previous section (Table 1). The figure illustrates the review’s focus on identifying data sources and linkages. It also shows that the review assesses the study designs and analytic methods that have been used to evaluate these programs and policies so that future research opportunities can be identified.

Organization of Report This report is organized by chapter. Each chapter represents either a main section of the report (i.e., Introduction, General Methods), or a KQ. Due to the complexity of the report, we present general methods and results in one chapter, and KQ-specific methods and results in their own chapters.

Page 22: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

9

Figure 1. Analytic framework for obesity prevention and control

KQ=Key Question; SES=socioeconomic status

Page 23: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

10

General Methods and Results Methods

Search Methods We conducted a systematic search of the published literature to identify studies focusing on

programs and policies implemented for obesity prevention and control, including changes in the built environment. The results of this phase were used to address Key Questions (KQs) 1 through 5. We searched PubMed, CINAHL, PsycINFO, and EconLit. A search strategy was developed for PubMed and used as a guide to develop search strategies for the other search engines. This search focused on identifying studies addressing obesity and behavioral changes impacting obesity. All databases were searched through August 21, 2017. Ten index articles identified by internal experts as applicable to this study were used during the search development. Terms used in the titles and abstracts, as well as relevant medical subject headings were identified in the index articles and used to develop the main search strategy. After the search strategy was developed, we tested it to ensure all index articles were captured.18, 43, 55-62 Detailed search strategies are described in Appendix C, Tables C1 and C2.

The search was supplemented with a hand search to identify references in other relevant systematic and narrative reviews. This hand search was conducted in two phases. The first phase involved searching the references of relevant systematic reviews captured in the main search strategy. Reference articles were selected through independent screening by two trained and experienced co-investigators with backgrounds in adult and child obesity prevention. These individuals identified potentially relevant articles based on the inclusion and exclusion criteria described below. Titles identified in this phase of the hand searching process were added to the main database of studies for full inclusion/exclusion review. The second phase of the hand search involved a search for systematic reviews in PubMed focusing on change in diet and physical activity outcomes without requiring any mention of obesity (detailed search strategy is described in Appendix C, Table C3). This search was developed using an additional set of index articles suggested by experts. Those index articles had not been identified by the original search strategy (which focused on finding articles that referred to obesity, overweight, or body mass index (BMI)).65-71 If we had expanded the original search by not requiring any mention of obesity, overweight or BMI, we would have had an unmanageable number of citations to review. We decided that the best way to identify eligible studies that focused on diet and physical activity outcomes was to search for relevant systematic reviews from 2013 through July 2017. The systematic reviews were evaluated for applicability to this study by two individuals who were involved in the main search and screening protocol. If a systematic review was considered applicable after the abstract and full text was reviewed, references for the included articles in each review were identified, and added to the main pool of articles for inclusion or exclusion.

We limited the search to studies published between 2000 and August 21, 2017. This time frame was selected to encompass the U.S. Surgeon General’s report, “The Surgeon General's Call To Action To Prevent and Decrease Overweight and Obesity,” released in 2001 and calling attention to the major public health problem of obesity.47 The publication of this report marked a shift toward directing public health funding and policies toward the prevention and control of

Page 24: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

11

obesity and sparked new research in this area. We gathered information on the number of articles captured using the original search strategy, and we identified the number of articles published per year. In 2000, the search captured 286 titles. This number steadily increased to 2698 in 2016.

Inclusion and Exclusion Criteria The inclusion and exclusion criteria for the systematic review were derived from the PICOTS

(populations, interventions, comparators, outcomes, timing, setting) framework (Table 1). We searched for all studies of programs, policies, or changes in the built environment targeting obesity prevention and control, change in physical activity behaviors, or change in diet behaviors, in people of any age to identify potential data sources. Abstracts and full text articles were screened by two types of people (screeners). We included screeners who are considered “junior,” meaning they are not experts in the field, are experienced research assistants working in systematic reviews, and are trained by the program manager to understand the inclusion and exclusion parameters of this project. “Senior” screeners are project leaders (principal investigators, investigators, and senior-level managers) who have either a clinical background on the topic of obesity prevention, or multiple years of experience developing and conducting systematic review screening processes. All abstracts, and all potentially relevant full text articles were reviewed by one junior screener and one senior screener. Agreement on inclusion or exclusion was required. Disagreements between screeners were discussed and, if they could not be resolved, were adjudicated by a third-party screener. Abstracts were excluded for the following reasons:

● No original data ● Study of fewer than 100 participants total ● Does not address a change in the built environment or is not a population-based program

or policy ● No comparison group, unexposed group, or pre-post comparison ● Studies in a setting not generalizable to a free-living population or community ● Study only targets a specific clinical sub-population (e.g., children with Prader Willi

syndrome or people with severe mental illness) other than obese and overweight populations. Note if a sub-population was included within the broader population, study was not excluded.

● Not relevant to the KQs ● No abstract ● No human data reported ● Not in English ● Protocol of a relevant study not reporting preliminary data

Full-text articles were excluded for the reasons above as well as the following:

● Outcomes were outside of the parameters of this review (see Table 1 for outcomes list) ● Study measured only a specific micro/macronutrient or dietary health behavior ● Meeting abstract ● Policy or program was not focused on obesity or nutrition-related structural or

environmental changes

Page 25: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

12

● Programs and policies enacted or implemented prior to 2000 ● Study conducted in a single school. ● A school-based policy or program that only included a curriculum change without

evidence of a structural or environmental component ● Study took place in a country that does not have a “very high Human Development Index

(HDI),” based on the United Nations Development Programme, Human Development Reports.72 The HDI is a composite statistic of life expectancy, education, and income per capita indicators. A country has a higher HDI when the life expectancy at birth is longer, the education period is longer, and the income per capita is higher.

We included studies that reported on at least one of the following outcomes of interest: obesity measures (either body weight or BMI in adults or BMI z-score or BMI percentile in children) or obesity-related individual health behaviors (dietary and physical activity) (Table 1 lists the outcomes of interest in detail).

Data Abstraction and Data Management We used Distiller SR software (Evidence Partners, Ottawa, Canada) for data collection and

reporting during the review process. Distiller SR is a Web-based data management program for systematic reviews and literature reviews that manages all levels of the review process. Data from applicable studies was abstracted in Distiller SR using predefined data abstraction forms (Appendix D) and will be uploaded to the Systematic Review Data RepositoryTM (SRDR), a Web-based data repository, at the completion of this project. This source serves as both an archive and a data abstraction tool. Data will be exported to SRDR in a project-specific database to serve as archived or backup copies and to create detailed evidence and summary tables.

Data Abstraction Overview and Process For each included study, two trained research assistants abstracted data about the study’s

characteristics (e.g., year of publication, country), study design, participants and populations, the policy and/or program evaluated, year of enactment, type of legislation/policy, location of policy (e.g., state, country, locality), original goal of policy/program (diet, physical activity, both, other), data sources and linkages, and analytic methods. See below for additional data abstraction elements by KQ. We also assessed how each of the outcomes was measured and whether a validated reference for instruments was provided.

After the data were abstracted, an independent data abstraction expert reviewed a random sample for quality assurance. Data abstraction experts were project leaders (principal investigators, investigators, and senior-level managers) who have either a clinical background on the topic of obesity prevention, or multiple years of experience developing and conducting systematic review screening processes. Inconsistencies in data abstraction were resolved by a consensus approach involving the research assistants and the expert reviewer. If consensus was not attainable for a specific case, it was discussed among the project leaders (principal investigators, investigators, and senior-level managers) and resolved by a majority vote. The same process was used for all data abstraction activities throughout this project.

Grading Strength of Evidence We assessed the overall risk of bias in the studies as described above. We did not evaluate the

overall strength of evidence for a particular comparison or outcome as we are not assessing the comparative effectiveness of policies or programs (interventions) in this review. Thus, we did

Page 26: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

13

not assess other domains of strength of evidence, such as consistency between studies, precision of estimated effects, or whether studies provided direct estimates of effects on the most important outcomes of interest.

Assessing Applicability We assessed applicability in terms of the PICOTS framework, as well as in terms of each of the KQs.

General Data Abstraction

Study and Participant Characteristics Study and participant characteristics were abstracted for all included studies. Table 2 lists the

study and participant characteristic categories and details abstracted. See Appendix D for the data abstraction form.

Because of the focus of this review on methods for natural experiment studies on obesity, we classified studies as either natural experiment studies, experimental studies, or other study designs. We used the definition of a natural experiment as proposed by the MRC (see Appendix B).41 Studies in which the researchers had control of the intervention and assigned participants or communities to intervention and control conditions were classified as experimental studies, which included both RCTs and controlled clinical trials. When the intervention was assigned to intervention and control groups by factors outside of the control of investigators, we classified the study as a “natural experiment study.” Studies for which there was not sufficient information on the assignment mechanism to determine whether a natural experiment or experiment approach was used were labeled as “other study design.” Study designs were further classified in the description of Methods for KQ 4.

Interventions: Policy, Program, or Built Environment Change Intervention details were abstracted for all included studies. A number of studies included

multiple interventions. Each intervention was abstracted. Table 3 lists the information abstracted about the interventions. The categories for the intervention targets were based on the 2012 Institute of Medicine (IOM) Report.16 See Appendix D for the data abstraction form.

Page 27: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

14

Table 2. Study and participant characteristics abstracted Study Characteristic Details Study Design (see below) Natural experiment study

Experimental study Other study design

Study Dates Start date End date Not reported

Community Type Urban Suburban Rural Not described

Study Setting Early childhood education or daycare (e.g., pre-school, Head Start) School: elementary (K-5); middle (6-8); high (9-12); other University After school or summer school Senior center Community center (e.g., job training, youth) Community or neighborhood (e.g., parks, farmers markets) Employer or worksite Transportation (e.g., train, bus, car, walking) Other Settings

Inclusion and Exclusion Criteria ● Define whether the criteria applied to individuals, site, or not specified ● Specific criteria: Sex Age Race/ethnicity Other

Participant Characteristics Comparison Group Identification of all comparison groups in the study Population Size Baseline N, follow-up period, and N at last follow-up for each comparison

group. Baseline Data Sex

Age category or grade category Pre-intervention weight or BMI (adult): weight; BMI; BMI category Pre-intervention BMI (child): BMI z-score; BMI percentile

BMI=body mass index; N=number

Page 28: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

15

Table 3. Intervention details abstracted Study Characteristic Details Goal of the program, policy, or built environment change

Physical activity environment (e.g., increasing opportunities for walking in low resource areas)

Food and beverage environment (e.g., food assistance programs, farmers market)

Messaging environment, about nutrition or physical activity (e.g., food labeling, food marketing)

Healthcare environment (health insurance, health system, or healthcare providers)

Work environment (e.g., active living; healthy living at work) Physical and built environment (e.g., transit, park, other outdoor spaces,

farmers’ markets, new supermarkets) Other

Was a policy evaluated? If yes: Name of the policy Governmental Federal/National (specify country) If United States, provide bill name and number Date of passage, date of enactment Is the policy still in effect? State/province/region (specify) If U.S. State, provide bill name and number Date of passage, date of enactment Is the policy still in effect? Local/community (specify) Date of passage, date of enactment Is the policy still in effect? Non-governmental Organization implementing the policy Policy start and end year

Was a program evaluated? If yes: Name of the program Governmental Federal/National (specify country) Program start and end date State/province/region (specify) Program start and end date Local/community (specify) Program start and end date Non-governmental Organization implementing the program program start and end year

Was a built environment or other change evaluated?

If yes: Name and nature of the built environment or other change Federal/National (specify country) Built environment start and end date State/province/region (specify) Built environment start and end date Local/community (specify) Built environment start and end date

Page 29: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

16

Results

Results of Literature Search The literature search identified 26,316 unique citations, of which 294 studies (reported in 312 articles) were eligible for inclusion (Figure 2). See Appendix G for a list of included studies

General Study Details We included studies that occurred in both the United States and other countries. The majority of studies were within the United States (n=188). One hundred and six were included from outside the United States, with Canada, the UK, and Italy most common after to the United States. We classified all studies by study design. The methods section describes the classification process (see Appendix B, Glossary). About half of the studies were natural experiment studies (n=156, 53%), followed by experimental studies (randomized controlled trials or non-randomized controlled trials) (n=118, 40%), and other study designs that did not fall into either of the other categories (n=20, 7%). Results that follow from each of the KQs are also described within these study design classifications, followed by a description by population and setting. We included detailed information on each study, the participant characteristics, and information on policy, program and built environment interventions in Appendix H (Evidence Tables H1 and H2).

Programs, Policies, and Built or Environmental Changes Identified in Included Studies We identified the original programmatic or policy goals of the program, policy or built environment change in reported in each study based on critical areas (or environments) for change from the IOM Report (2012).16 These included: environments for physical activity, food and beverage environments, message environments (posted flyers or information campaigns), health care environments and work environments, transportation environments, parks and recreation environments (See Methods for Data Abstraction and Appendix B for Glossary); studies often were classified as having more than one goal. In addition, studies were described by target setting, such as work, school or community. Table 4 provides information on the number of studies classified by study method, program, policy, or built environmental goals, and the target settings. Eighty-six studies out of 294 reported having multiple programmatic, policy or environmental goals. Most studies, regardless of study design, targeted schools and addressed changes in the environments for physical activity or the food and beverage environment. Nearly all of the studies addressing changes in the transportation or parks and recreation environments (n=27) followed a natural experiment method and targeted communities or neighborhoods (Table 4).

Page 30: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

17

Figure 2. Results of literature search

Page 31: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

18

Table 4. Summary of study methods; programs, policies, and built environment goals; and targets of intervention (N=294)

Study design, n Studies

Programmatic, Policy, or Built Environment Goal, n Studies*

Target Setting of the Program, Policy or Built Environment Change, n Studies

Natural experiment studies, n=156

Physical activity environment, 43 School, 34 Community or neighborhood, 8 Transportation, 1

Food and beverage environment, 78 School, 50 Food assistance program†, 9 Community or neighborhood, 19

Transportation, 17 Community or neighborhood, 17 Parks and recreation, 8 Community or neighborhood, 8 Messaging environment,‡ 9 School, 4

Community or neighborhood, 5 Physical and built environment, 33 School, 4

Community or neighborhood, 28 Employer or worksite, 1

Policy, 11 School, 8 Community or neighborhood, 3

Experimental studies, n=118

Physical activity environment, 57 School, 50 Community or neighborhood, 4 Employer or worksite, 3

Food and beverage environment, 65 School, 48 University, 1 Community or neighborhood, 10 Employer or worksite, 7

Transportation, 1 Employer of worksite 1 Parks and recreation, 1 Community or neighborhood, 1 Messaging environment, ‡ 16 School, 4

University, 2 Community or neighborhood, 6 Employer or worksite, 4

Healthcare, 1 Other (National Health Service intervention), 1 Work, 1 Community or neighborhood, 1 Physical and built environment, 16 School, 5

Community or neighborhood, 10 Employer or worksite, 1

Other Policy, 2 School, 2 Other study designs, n=20

Physical activity environment, 11 School, 8 Food assistance program†, 1 Community or neighborhood, 2

Food and beverage environment, 6 School, 1 Food assistance program†, 1 Community or neighborhood, 1 Employer or worksite, 3

Messaging environment, ‡ 4 School, 2 Community or neighborhood, 2

Transportation, 1 School, 1 Physical and built, 2 School, 1

Community or neighborhood, 1 N=total population; n=number of studies

Page 32: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

19

* Some studies include multiple program, policy, or built environmental goals; some studies include multiple intervention targets † Examples of food assistance programs include SNAP (Supplemental Nutrition Assistance Program), a federal level program; Health Bucks, as local-level farmer’s market incentive program ‡ Consistent with the 2012 IOM Report that described “Message Environments”, policies and programs were classified as having specific messages or information about nutrition and /or physical activity (e.g., posted flyers or informational campaigns), which included social marketing strategies. The majority (n=152 studies) of the 188 U.S. studies across all study designs evaluated governmental programs or policies at the local, state/regional, or federal levels. Of these, we identified 139 unique policy or program evaluations (Appendix I, Table I1). Eighty-five of these studies evaluate programs, and fifty-five are evaluations of policies. For example, polices included those addressing food policy (sugar-sweetened beverage bans;73 competitive food laws74; calorie labeling laws;19, 20 Women, Infants, and Children and Supplemental Nutrition Assistance Program,61, 75, 76 physical activity policy joint use policies;77 state policies on physical education;78, 79 physical activity in daycare centers80). Additionally, we identified 36 non-governmental programs (Appendix I, Table I2). Forty-eight (31%) of the 156 natural experiment studies took place outside of the United States, and 62 percent focused primarily on children. Sixty-three percent of the natural experiment studies taking place in the United States (n=108) focused on children. Fifty (42 %) of the 118 experimental studies took place outside of the United States, and 73 percent focused primarily on children. Sixty-nine percent of the experimental studies taking place in the United States (n=68) focused on children. Seven of the 20 studies with other study designs took place outside of the United States, and 4 of them focused primarily on children, while the rest were in the United States, of which 7 focused on children. The natural experiment studies most commonly evaluated national, state, and local policies such as the UK’s provision of free local bus passes to retirees 81 or the 2008 legislation requiring chain restaurants in New York City to provide calorie information to be posted on menus 20. Researchers also commonly capitalized on changes to local built environments such as the addition of new supermarkets in food deserts21, 55, 82 or new transportation systems.56, 66, 83 We included evaluations of programs that were not originally intended for research, such the Centers for Disease Control and Prevention’s HealthMPowers program 84 to improve physical activity and nutrition in the school setting and Jamie Oliver’s cooking skills program.85

Page 33: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

20

Key Question 1: Population-Based Data Sources Key Question (KQ) 1: What population-based data sources have been used in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes?

Overview To improve our understanding of which population-based data sources have been used in

obesity prevention and control studies, KQ1 identified all data sources and encoded them based on whether they meet criteria to be a data system (i.e., most usable for obesity researchers), their original purpose and outcomes they contained.

Key Findings • 294 studies reported one or more population-based data sources. The majority (216 of

294) of the U.S. and non-U.S. studies included some primary data collection and most of these 216 primary data sources were not sharable (e.g., did not have a public or transferable license to allow data to be used for research purposes).

• 93 U.S. and non-U.S. studies included secondary and/or sharable primary data sources. • Of these 93 studies, we identified 143 secondary data sources and 26 sharable primary

data sources (totaling 116 data sources after duplicates were removed). • Of the 116 data sources, 106 (71 U.S. and 35 non-U.S. data systems) met the four criteria

for data system (data source exists, is available for research, is sharable and contains outcomes or co-outcomes of interest).

• 96 of the 106 data systems were used in natural experiment studies, 7 were used in experimental studies, and 3 were used in studies with other study designs.

• Among the 71 U.S. data systems, 28 (39%) were originally designed for administrative purposes and 22 (31%) for public health operations. More than half of the 71 U.S. data systems had national coverage (57%).

• 44 (62%) of the 71 U.S. data systems contained at least one of the main outcomes of weight or BMI or dietary or physical activity behaviors, while 40 data systems (53%) included at least one of the co-outcomes or exposures, such as information about the policy, program, built environment, or other geographical information.

Methods

Data Abstraction We abstracted and generated a list of all data sources reported in included studies. We identified each data source (U.S. and non-U.S.), with all duplicate data sources removed, and provided a count of the number of studies that used each data source. A primary data source was defined as one primarily collected for the purpose of a study (e.g., collecting survey data from participants to test the hypothesis of a study). If the data of a primary data source was being shared with other researchers, in any form including a public use dataset, we considered it to be a sharable primary data source.

Page 34: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

21

We used this list of data sources to apply a set of criteria (Table 5) to determine if a data source is a data system that is actively maintained, can be acquired or accessed by researchers, and contains the outcomes of interest. The ‘data system’ criteria (Table 5) were developed for a previous project that our team conducted for the Pathways to Prevention program focused on data sources used to evaluate suicide prevention programs.86 A data system represents one or more organized and accessible data sources that go beyond just collecting and managing data, but also contain some degree of an information technology infrastructure to maintain and operate the system. We applied these criteria to data sources identified in KQ 1 to inform obesity researchers not only about which data sources have been used in prior studies, but also what data sources are still available and can be used by researchers (i.e., meeting the data system criteria), which are key characteristics that make these data systems useful for future research.

Table 5. Criteria to determine whether a data source meets the criteria of a data system related to obesity

1 Data source is still in existence (e.g., information about the data system can be found on the Web) 2 Data are available and accessible in digital format (e.g., datasets are downloadable from a current Web site) 3 Data are sharable and can be acquired by others for research purposes (e.g., has a public or transferable

license that allows the data to be used for research purposes) 4 Data system collects/contains at least one of the outcomes of interest (e.g., primary outcomes, such as

weight and height, or secondary outcomes, such as policy and built environment)

Data sources not meeting the data system criteria (Table 5) were not further coded as they were deemed impractical for research (e.g., the data source does not exist anymore, or it cannot be shared with other researchers). Data sources meeting the data system criteria (Table 5; referred to as ‘data systems’) were further coded based on information collected from the studies and information found on the Web. NonU.S. data systems were coded in a limited manner due to language barriers and other issues with data access. U.S. data systems were further coded according to a coding and classification schema that was adapted and modified from a framework previously developed to review and evaluate community-based data sources (Appendix E).87 These data coding schema include items such as data granularity (population- or individual-level), denominator coverage, data collection method, data scalability, data governance, data uses and functions, and data linkage mechanisms (see KQ2). To complete the data coding schema for each of the identified U.S. data systems, we located and accessed publicly available information (e.g., data dictionaries) of the data systems after we obtained as much information as possible from the studies. We did not download and analyze the data systems, as that was out of the scope of this review (e.g., conducting statistical analyses of the data systems to compute data quality measures).

Data Synthesis We encoded each of the U.S. data systems identified in the studies using the coding schema (Appendix E). This process produced a list and description of all data systems, the categories of variables they contain, and various data specifications as described in the coding schema. The coding schema included a number of coding items grouped in various categories such as: data system specification and meta-data; geographical coverage (e.g., country, state, community); demographic coverage; data granularity; variables and outcomes of interest; data system

Page 35: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

22

scalability; and data system governance. See Appendix E for the detailed list of items used to encode the U.S. data systems.

Results

Identification of Data Sources and Data Systems Of 294 included studies, all reported at least one data source (Figure 3). We identified 216

studies with at least one data source from primary data collection performed for that study. A primary data source was defined as one primarily collected for the purpose of a study (e.g., collecting survey data from participants to test the hypothesis of a study). Primary data sources were used by the researchers to conduct the project but were not made sharable for other researchers (e.g., has a public or transferable license that allows the data to be used for research purposes). Non-sharable primary data sources were excluded from the next steps of the data source abstraction, as by definition, they did not qualify as a data system (e.g., are not sharable; see Table 5). See Appendix B for definition of data system.

If the data of a primary data source was being shared with other researchers, in any form including a public use dataset, we considered it to be sharable primary data source. Ninety-three studies contained a total of 169 data sources that were sharable, of which 26 were sharable primary data collection sources and 143 were secondary data sources. We defined secondary data sources as those used in the studies but not directly collected by the study researchers (i.e., data source was external to the study). After removing the duplicates (n=53) and data sources that did not meet all data system criteria (n=10) (data source exists, is available for research, is sharable and contains outcomes or co-outcomes of interest) (Table 5), 106 unique data systems (71 U.S. and 35 non-U.S.) were identified for the final review and detailed coding (Appendix J). Appendix J lists all the data sources that met criteria for a data system (data source exists, is available for research, is sharable and contains outcomes or co-outcomes of interest) reported in the included studies, stratified by U.S. and non-U.S. data studies, and by natural experiment studies, experimental studies, and studies with other study designs (see Appendix J). These 106 unique data systems included 20 sharable primary data sources and 86 secondary data sources (Figure 3). Of the 106 data systems, 96 of the data systems were used in natural experiment studies, 7 were used in experimental studies, and 3 were used in studies with other study designs.

The 71 U.S. data systems underwent detailed coding to characterize the level of information available (see KQ 1 Methods and Appendix E) and were used as the denominator for all results reported in this section. Figure 4 shows results of the coding. Most of the 71 U.S. data systems had a dedicated Web page (73%), but most of these Web pages only included high-level summaries or limited details about the data systems (32%). Only 19 percent of the Web pages about the data systems included highly detailed information, such as data quality (e.g., completeness, accuracy, and timeliness of the data). For example, the National Health and Nutrition Examination Survey– maintained by the CDC – provides detailed information about data quality issues and potential analytical pitfalls if data are not used properly. More than half of the 71 U.S. data systems (59%) were available for download but some (about 14%) required a form of registration or permission before accessing the data. Although 70 percent of the data systems had some level of information about their data structure (e.g., survey questionnaires, summary reports), only 15 percent of them presented a formal data dictionary on their Web pages (e.g., auto generated code books reflecting the database structure, variables, ranges, and definitions). Most data systems used unconventional documentation (i.e., not a data

Page 36: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

23

dictionary) to inform the users about their data structure and variables. For example, 25 percent provided only informal documentation about their data structure (which did not qualify as a data dictionary), and 29 percent offered survey instruments and other documentation that could be used to infer data structure and variables.

Figure 3. Identification and classification of data sources and systems in included studies

Page 37: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

24

Figure 4. Level of information available for U.S. data systems (n=71)

Note: Values in each group that are depicted by the lighter color are mutually exclusive

Figure 5 shows the original purpose for data use and function for the 71 U.S. data systems. Most of the data systems were designed for administrative purposes (39%) or public health operations such as surveillance (31%). About 29 percent of the data systems identified were primarily developed and maintained for research purposes. Only about 2 percent of the data systems were originally developed and maintained for clinical care purposes.

Figure 6 displays the demographic coverage of the 63 U.S. data systems. Fifteen (21%) of the U.S. data systems have the general population as their denominator. Most of the data systems focused on schools (n=26), communities (n=16), or other specific programs (n=5). Most of the data systems included data on school-age children: 22 (31%) in elementary schools, 26 (37%) in middle schools, and 23 (32%) in high schools. Fewer than 21 percent of the U.S. data systems focused on the adult population. None of the data systems were designed to capture outcomes of interest for older adults or other special populations. Socio-economic status was captured in 20 (28%) data systems. Race and ethnicity variables were available in 24 (34%) and 22 (31%) data systems, respectively.

The geographic scope was varied for many of the data sources, or provided coverage of multiple geographical granularities. In the data sources pulled, 57 percent provided national coverage, 50 percent state coverage, and 32 percent county or city. Few or no data sources provided zip code (3%) or census block (0%) data.

0% 10% 20% 30% 40% 50% 60% 70% 80%

Dedicated Web Page

· Summary of Data System (e.g., description and purpose)

· Some Detailed Information (e.g., data elements)

· Very Detailed Information (e.g., data quality reports)

Data Accessible

· Publicly Available (e.g., public use file with download link)

· Requires Automated Registration (e.g., signing up)

· Requires Manual Confirmation (e.g., email request)

Data Dictionary (any level)

· Formal Data Dictionary (i.e., data elements defined in a document )

· Informal Data Dictionary (e.g., survey questions)

· Proxy Data Dictionary (e.g., summary reports)

Page 38: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

25

Figure 5. Original purpose for data use and function for U.S. data systems (n=71)

Figure 6. Demographic coverage of U.S. data systems (n=71)

Note: Values in each group are not mutually exclusive and do not sum up to 100 percent, as a data system may cover multiple groups.

Research, 29.6%

Clinical, 1.6%

Administrative, 39.4%

Public Health, 31.0%

0% 5% 10% 15% 20% 25% 30% 35% 40%

General Population

Community level

Schools

Programs

Households

Other

Infant

Preschool

Elementary School

Middle School

High School

Teen/Adolescent/Youth (not bound to school age)

Adult

Socio-economic Status

Ethnicity

Race

Population Denominator

Age Coverage

Other Demograhpics

Page 39: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

26

Twenty-seven (38%) of the data systems contained individual-level data. However, some studies cannot share individual-level data due to Health Insurance Portability and Accountability Act restrictions and, thus, only offered aggregated-level data for sharing. In fact, 53 (74%) of the identified U.S. data systems offered aggregated data instead of individual-level data.

Figure 7 displays the outcome and exposure variables of interest in the U.S. data systems. Among data systems reporting the primary outcomes of interest in this report, weight was captured in 34 percent of the identified data systems, while height was available in 31 percent of them. Only 23 percent of the data systems offered a calculated BMI, while 42 percent of the data systems included information about dietary behaviors and 34 percent included information about physical activity. Almost half of the data systems contained information about at least one of the exposures with details about the studied policy, program, or built environment change, or contained geographical information. More than 31 percent of the data systems contained information about policies, 15 percent contained program-level information, and 48 percent contained geographical data [either as built environment data (24%) or other geographically encoded information (24%). Around 59 percent of the data systems included an objective approach (e.g., measurement by a healthcare provider; with or without other methods) to collect some of the outcome variables (not shown in the figure).

Figure 7. Coverage of obesity-related outcomes and exposures in U.S. data systems

BMI=body mass index Note: Values in each group are not mutually exclusive and do not sum up to 100 percent, as a data system may cover multiple outcomes.

0% 10% 20% 30% 40% 50% 60% 70%

Primary Outcomes

Weight

Height

BMI

Diet

Physical Activity

Exposure

Policy

Program

Built Environment

Other Geographical Information

Page 40: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

27

Key Question 2: Linking Data Sources Key Question (KQ) 2: What methods have been used to link different population-based data sources?

Overview Policy makers have expressed increasing interest in linking population-based data sources to address questions about the effectiveness of policies and programs in obesity prevention and control. The goal of KQ2 was to assess and describe data linkages between data systems identified in KQ1.

Key Findings • 26 (37%) of the 71 U.S. data systems were linked with a secondary data source or system

other than the primary data sources for the purposes of the research study. • Studies that linked their data systems with multiple external data systems either used an

individual-level key (14%; e.g., patient identifiers) or a geographic allocation (23%; e.g., patient resides in a specific county thus mapping the county specifications extracted from other data sources for that individual).

• One study used statistical models to link primary data sources with external data sources and adjust for potential covariates.

Methods

Data Abstraction For each U.S. data system identified in KQ 1, we assessed whether and how the data systems have been linked together, using information obtained from the study and, as described above, using publicly available information. The coding schema (see Appendix E) included a series of coding items to identify how the identified studies linked data sources together.

Results For KQ 2 we qualitatively described the methods used to link these data systems (see KQ2

data abstraction). We aggregated and summarized the data across various coding schema attributes. The summary report includes various data system attributes across the entire list of data systems and is accompanied with notations about which data specifications (e.g., linkage methods) were found to be effective and which ones have faced challenges. Of the 71 U.S. data systems reported in KQ 1, 26 data systems (37%) linked their data system with another data source. These data sources were linked for the purposes of the research study and had not been previously linked. All 26 data systems occurred in natural experiment studies. Ten (14%) of the 71 U.S. data systems used an individual-level linkage. Most of the survey-based studies used existing data from other surveys to locate potential participants and import already captured data about those participants in their study. Examples of these external surveys that were used for individual-level linkage include: Women, Infants, and Children, Supplemental

Page 41: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

28

Nutrition Assistance Program, Common Core Data, Early Childhood Longitudinal Study-Kindergarten, The Military Teenagers' Environments, Exercise, and Nutrition Study, and vital records. Sixteen (23%) of the 71 U.S. data systems used a geographical-level linkage to capture additional data about their denominator of participants. These added variables were eventually used as direct, indirect, or adjustment variables for the studies. Most of the studies using a geographical-level linkage used a simple geographical name match (e.g., zip code match), but two used more advanced methods (e.g., locating an address within a geographical boundary). Examples of the external geographical data systems they linked to included: U.S. Census, Dun and Bradstreet, National Household Travel Survey, Trip Identification and Analysis System, and local, state or national maps for food retails, property and sales taxes, and crime rates. Two studies used statistical models (i.e., regression models) to link a primary data source with external geographical data sources to adjust for potential confounders in the statistical models. Regression models are considered a standard approach.88

Page 42: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

29

Key Question 3: Assessment Measures Key Question (KQ) 3: What obesity measures, dietary and physical activity behaviors, and other outcomes have been assessed in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control?

Overview The goal of KQ 3 was to describe and synthesize what obesity, dietary and physical activity measures were used to assess childhood and adult obesity outcomes, dietary and physical activity behaviors, and other co-outcomes (e.g., commuting behavior). For each measure, we described the type of measure (e.g., the name of the questionnaire used to assess fruit and vegetable intake), how the data was collected (e.g., trained study staff for height/weight vs. self-reported questionnaires) and the data source that contained the measure when it was obtained from a secondary data source. We presented the results by population (children and adults) and study design (natural experiment, experimental, other study design).

Key Findings • One hundred twelve studies out of 294 (38%) reported on childhood weight outcomes,

primarily body mass index (BMI) z-score or BMI percentile. Most studies reporting childhood weight outcomes assessed the outcomes using direct measurement from trained staff.

• Thirty-two studies reported adult weight outcomes and these were mostly conducted in community or worksite settings.

• One hundred forty-eight studies reported dietary behavioral outcomes in terms of change in the intake of fruits and vegetables (n=147), sugar-sweetened beverages (n=54), total daily caloric intake (n=17), fast food intake (n=16), and fiber (n=12).

• One hundred fifty-two studies reported physical activity measures. Most studies reporting physical activity outcomes took place in school (n=89) or community (n=40) settings.

Methods

Data Abstraction We abstracted detailed information about which of the following obesity measures, dietary or physical activity behavioral measure and other co-outcomes were reported in the studies:

• Adult body weight and BMI • Childhood BMI (z-score and percentile) • Individual dietary intake and behaviors. We included measure of total daily caloric

intake; specific macronutrients related to obesity (including vegetable, fruit, or fiber intake); sugar-sweetened beverage intake; or fast food intake.

• Individual physical activity behavior, in terms of both activity type and quantity

Page 43: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

30

• Co-outcomes: food environment, physical activity environment, other outcomes (e.g., housing, economic)

We abstracted which measures and/or questionnaires were used to assess the outcomes and which of the data sources from KQ1 contained the measures.

Data Synthesis We counted the number of studies that reported each outcome and categories of outcomes of interest. We described the types of measures or instruments used to assess these outcomes, based on categories from the National Collaboration on Childhood Obesity Research Measures Registry.53 Definitions of these categories are provided in Appendix B. The data sources and questionnaires used for these measures were also reported and described. We stratified this section by study design: natural experiment studies, experimental studies, and other study designs.

Results We identified 112 studies reporting on childhood weight outcomes, 32 studies on adult weight outcomes, 152 studies on physical activity outcomes, and 148 studies on dietary outcomes. Forty-seven studies reported both diet and physical activity outcomes. Thirty-seven studies reported on co-outcomes, such as commuting behaviors, food environment, physical environment, and food purchasing behavior (see Appendix H, Evidence Tables H3-H15).

Obesity Measures: Weight and BMI in Children and Adults Table 6 displays the obesity measures used to assess weight and BMI in adults and children. For children, 112 studies reported body weight, BMI percentile, and/or BMI-z score in children: 57 in terms of BMI z-score, 46 in terms of BMI percentile change, and 27 with other childhood weight outcomes. For adults, 32 studies reported body weight in adults: 31 in terms of BMI and 6 in terms of weight change.

Page 44: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

31

Table 6. Weight outcomes and measures in adults and children, by study design Measure, n (%)* Method and Population Outcome Measured by

Trained Staff EHR Self-

Reported Other

Natural Experiment Studies Children 50 studies

Change in BMI z-score 14 (60.9) 1 (4.3) 3 (13) 5 (21.8)

Change in BMI percentile 15 (55.6) 0 6 (22.2) 6 (22.2) Change in weight 7 (87.5) 0 0 1 (12.5) Adult 17 studies

Change in body weight 0 0 1 (100) 0

Change in BMI 6 (37.5) 0 10 (62.5) 0 Experimental Studies Children 48 studies

Change in BMI z-score 27 (84.4) 0 2 (6.3) 3 (9.4)

Change in BMI percentile 13 (76.5) 1 (5.9) 2 (11.8) 1 (5.9) Change in weight 7 (77.8) 0 0 2 (22.2) Adult 13 studies

Change in body weight 5 (100) 0 0 0

Change in BMI 7 (53.8) 0 6 (46.2) 0 Other Study Designs Children 6 studies

Change in BMI z-score 3 (100) 0 0 0

Change in BMI percentile 3 (100) 0 0 0 Change in weight 1 (100) 0 0 0 Adult 2 studies

Change in body weight 0 0 0 0

Change in BMI 0 0 1 (50) 1 (50) *Many studies included more than one type of weight measure or weight outcome. BMI=Body Mass Index; BMI-z=Body Mass Index z-score; EHR=electronic health records; n=number of measures reported;

BMI Measures in Children Of the 95 studies reporting weight or BMI in children, 46 were natural experiment studies, 44 were experimental studies (randomized controlled trials (RCTs) and controlled clinical trials), and 5 were classified as “other study designs” (see Methods for definitions of study designs). Fifty-seven studies reported BMI z-score and 46 reported on BMI percentile. Most studies reporting weight measures in children were conducted in a school setting (n=88).

Natural Experiment Studies Reporting BMI in Children Of the 46 natural experiment studies reporting data on BMI z-score or BMI percentile in children, 25 studies reported the outcome based on direct measurement from trained staff, one study used an EHR, 8 studies used self-reported data, and 8 studies used other measures (e.g., FitnessGram assessment). Out of the total reported measures, natural experiment studies commonly reported using trained staff to measure BMI in children (BMI percentile: 55.6 percent, BMI-z score: 60.9%) (Table 6, Appendix H, Evidence Tables H3-H4) Evidence Table H16 in Appendix H displays details about U.S. secondary data sources containing each of the weight measures, by study design (natural experiment, experimental, other design). Natural experiment studies used 26 different U.S. data sources for childhood weight measures, including the National Survey of Children's Health, Early Childhood Longitudinal Study-Kindergarten Cohort, and School Health Policies and Programs Study (Appendix H, Evidence Table H16).

Page 45: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

32

Experimental Studies Reporting BMI in Children Of the 39 experimental studies reporting on children’s BMI-z or BMI percentile, 33 studies reported the outcome based on direct measurement from trained staff, 1 study used an EHR, 3 studies used self-reported data, and 3 studies used other measures (i.e., body composition analyzer). Out of total reported measures, experimental studies commonly reported using trained staff to measure BMI in children (BMI percentile: 76.5%, BMI-z score: 84.4%) (Table 6; Appendix H, Evidence Tables H3-H4).

Other Weight Outcomes in Children Twenty-five studies either directly captured child weight (n=17) or reported other weight outcome measurements (n=8). Of the 17 studies reporting child weight, 8 were natural experiment studies and 9 were other experimental methods. All but three studies reported outcomes based on direct measurement by trained staff (Table 6). Eight studies reported other child weight outcomes not designated in our populations, interventions, comparators, outcomes, timing, setting framework. These included BMI (not BMI-z score or BMI percentile), rates of BMI change in participants, and probability of obesity (see Appendix H, Evidence Tables H5-H6).

Weight and BMI in Adults Thirty-two studies reported on adult body weight and BMI outcomes. Thirty-one studies reported on change in adult BMI, and 6 studies reported on change in adult weight. Most studies reporting weight measures in adults were conducted in community (n=15) or worksite (n=10) settings. Seventeen studies reporting adult weight or BMI were natural experiment studies, 13 were experimental studies, and 2 were other study designs.

Natural Experiment Studies Reporting Weight or BMI in Adults Of the 17 natural experiment studies reporting data on change in body weight or BMI, 6 studies reported this outcome based on direct measurement from trained staff and 11 studies used self-reported data. None of the studies used an electronic health record (EHR) for weight or BMI measures in adults.

Evidence Table H16 in Appendix H shows details about U.S. secondary data sources containing each of the weight measures, by study design (natural experiment, experimental, other design). Natural experiment studies used 12 different U.S. data sources for adult weight and BMI measures, including the Behavioral Risk Factor Surveillance System (BRFSS) and the L.A. County Food Retail database. (Appendix H, Evidence Table H17).

Experimental Studies Reporting Weight or BMI in Adults Of the 13 experimental studies reporting data on change in body weight or BMI, 9 studies

reported the outcome based on direct measurement from trained staff (BMI: n=7; weight: n=2), no studies used an EHR, and 6 studies used self-reported data (BMI: n=6).

Measures of Dietary Behaviors in Adults and Children One hundred forty-eight studies reported dietary behavioral outcomes in terms of change in the intake of fruits and vegetables (n=147), sugar-sweetened beverages (n=54), total daily caloric intake (n=17), fast food intake (n=16), and fiber (n=12). Most studies reporting dietary

Page 46: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

33

behavioral outcomes conducted their study within the school (n=86) or community (n=42) setting. In general, studies used measures to assess only some key aspect of these dietary behaviors rather than the total diet comprehensively, such as through repeated 24-hour recalls or food frequency questionnaires (for definitions, see Appendix B). For example, the Pittsburgh Hill/Homewood Research on Eating, Shopping and Health21 was a natural experiment that examined the effect of a full-service supermarket on diet and other outcomes using the 24-hour recall. The Norwegian School Fruit Program23 study was an experimental study that evaluated a fruit and vegetable program in schools, also using the 24-hour recall. Table 7 shows the dietary measures used in studies of children and adults by study design. Overall, 77 studies were natural experiment studies, 63 were experimental studies, and 11 were other study designs (several studies collected adult and child data). Among the 107 studies in children, 95 studies reported on intake of fruits and vegetables, 45 on sugar-sweetened beverages, 11 on total calories, 12 on fast food, and 9 on fiber. Among the 50 studies in adults, 43 studies reported on intake of fruits and vegetables, 11 on sugar-sweetened beverages, 6 on total calories, 5 on fast food, and 3 on fiber.

Dietary Behaviors in Children Table 7 shows the methods for assessing dietary behaviors in children. The most commonly used method for assessing diet was through brief dietary questionnaires (e.g., The Boston Youth Survey). Of the 52 natural experiment studies reporting data on diet in children, 6 studies used a 24-hour recall, 7 studies used a food frequency questionnaire, 34 studies used other questionnaires, and 5 used a record/log. Evidence Table H16 in Appendix H provides details about U.S. secondary data sources containing each of the dietary measures, by study design (natural experiment, experimental, other design). In U.S. studies in children, the natural experimental studies used data sources with dietary measures such as the National Survey of Children's Health and the National Youth Physical Activity and Nutrition Study (Appendix H, Evidence Table H16). Among the natural experiment studies, commonly used questionnaires that assessed dietary behaviors in children included the School Physical Activity and Nutrition (SPAN) Questionnaire, Youth Risk Behavior Survey (YRBS) Questionnaire, and the Boston Youth Survey. Of the 47 experimental studies (RCTs and non-randomized controlled trials) reporting on children’s diet, 12 reported the outcome based on 24-hour recall, 17 used a food frequency questionnaire, 15 used other questionnaires, one used a record/log, 5 used observation, and 7 used other measures (i.e., structured interviews, digital images). Experimental studies used questionnaires such as The Child and Adolescent Trial for Cardiovascular Health After-School Student Questionnaire (ASSQ) (Appendix H, Evidence Table H17).

Dietary Behaviors in Adults Table 7 shows the methods for assessing dietary behaviors in adults. The most commonly used method for assessing diet was through food frequency or brief dietary questionnaires such as the National Health and Nutrition Examination Survey, as well as internally designed questionnaires (Table 7). Of the 30 natural experiment studies reporting measures of dietary behaviors in adults, 4 used a 24-hour recall, 10 used a food frequency questionnaire, 18 used other questionnaires, 3 used a record/log, and none used observation.

Page 47: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

34

Evidence Table H16 displays details about U.S. secondary data sources containing each of the dietary measures, by study design (natural experiment, experimental, other study design). Among the U.S. studies in adults, the most frequently used data source reporting adult diet behavioral measures was the brief diet screener of the BRFSS, which was used in four natural experiment studies. Both experimental and other study designs used primary data collection as their source (Appendix H, Evidence Table H16). Examples of questionnaires used by experimental and other study designs are the Customer Impact Questionnaire and the Five-a-day Community Evaluation Tool, respectively (Table 8). Of the 17 experimental studies (RCTs and non-randomized controlled trials) reporting data on diet, one reported the outcome based on 24-hour recall, 7 used a food frequency questionnaire, 6 used other questionnaires, one used a record/log, none used observation, and 4 used other measures.

Measures of Physical Activity in Adults and Children One hundred fifty-two studies reported physical activity measures. Most studies reporting physical activity outcomes took place in school (n=89) or community (n=40) settings. All 42 of the studies with a goal of changing the parks and recreation, or transportation environment, measured physical activity. In general, studies used measures to assess only some key aspect of physical activity behavior rather than total physical activity comprehensively. An example is the use of a pedometer to count steps as a proxy measurement of physical activity. Studies that used this type of measure include the Great Fun 2 Run study,89 which is a natural experiment looking at increasing physical activity levels in schools through physical education lessons. Another example is the Healthy Schools Project,90 which uses an experimental method to observe if augmented recess sessions increase physical activity over standard sessions. Table 8 shows the physical activity measures in studies of children and adults by study design. Overall, 71 studies were natural experiment studies, 66 were experimental studies, and 15 were other study designs.

Page 48: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

35

Table 7. Dietary outcomes and measures for children and adults, by study design Method and Population

Outcome 24-Recall, n (%)

FFQ, n (%) Questionnaire, n (%)

Record/Log, n (%)

Observation, n (%)

Other, n (%)

Natural Experiment Studies Children Caloric

intake 3 (50) 1 (16.7) 0 2 (33.3) 0 0

52 studies Fast food 1 (14.3) 1 (14.3) 4 (57.1) 1 (14.3) 0 0 SSB 1 (4.8) 2 (9.5) 16 (76.2) 2 (9.5) 0 0 Fruit/veg 5 (11.6) 6 (14) 29 (67.4) 3 (7) 0 0 Fiber 1 (20) 0 2 (40) 2 (40) 0 0 Adult Caloric

intake 2 (66.6) 0 0 1 (33.3) 0 0

30 studies Fast food 0 1 (25) 3 (75) 0 0 0 SSB 1 (10) 3 (30) 5 (50) 1 (10) 0 0 Fruit/veg 5 (15.6) 10 (331.3) 14 (43.8) 3 (9.4) 0 0 Fiber 0 0 0 1 (100) 0 0 Experimental Studies Children Caloric

intake 0 1 (20) 1 (20) 1 (20) 1 (20) 1 (20)

47 studies Fast food 1 (25) 0 3 (75) 0 0 0 SSB 4 (18.2) 7 (31.8) 9 (40.9) 0 1 (4.5) 1 (4.5) Fruit/veg 12 (23.1) 15(28.8) 14 (26.9) 1 (1.9) 5 (9.6) 5 (9.6) Fiber 0 3 (75) 1 (25) 0 0 0 Adult Caloric

intake 1 (50) 0 0 0 0 1 (50)

17 studies Fast food 0 1 (100) 0 0 0 0 Sugar

sweetened beverage

0 1 (100) 0 0 0 0

Fruit/veg 0 7 (46.7) 5 (33.3) 1 (6.7) 0 2 (13.3) Fiber 0 1 (50) 1 (50) 0 0 0 Other Study Designs Children Caloric

intake 0 0 0 0 0 0

8 studies Fast food 0 0 1 (100) 0 0 0 SSB 1 (20) 2 (40) 2 (40) 0 0 0 Fruit/veg 2 (25) 2 (25) 4 (50) 0 0 0 Fiber 0 0 0 0 0 0 Adult Caloric

intake 0 0 0 0 0 0

3 studies Fast food 0 0 0 0 0 0 SSB 0 0 0 0 0 0 Fruit/veg 0 2 (66.6) 1 (33.3) 0 0 0 Fiber 0 0 0 0 0 0

Many studies included more than one type of diet measure or diet outcome. FFQ=food frequency questionnaire; Fruit/veg=fruit and vegetables; n=number of measures reported; N=number of studies; SSB=sugar sweetened beverage

Page 49: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

36

Table 8. Physical activity outcomes and measures, by study design Method and Population

Electronic Monitor, n (%)

Questionnaire, n (%)

Record/Log, n (%)

Observation, n (%)

GIS, n (%) Other, n (%)

Natural Experiment Studies Children 43 studies

17 (32.7) 27 (51.9) 1 (1.9) 4 (7.7) 0 3 (5.8)

Adult 32 studies

6 (13) 35 (76.1) 2 (4.3) 1 (2.2) 0 2 (4.3)

Experimental Studies Children 53 studies

22 (33.8) 21 (32.3) 6 (9.2) 6 (9.2) 0 10 (15.4)

Adult 13 studies

3 (20) 10 (66.7) 1 (6.7) 1 (6.7) 0 0

Other Study Designs Children 10 studies

3 (27.2) 5 (45.5) 0 0 0 3 (27.2)

Adult 5 studies

1 (12.5) 5 (62.5) 1 (12.5) 0 0 1 (12.5)

*Many studies included more than one type of physical activity measure. GIS=Geographic Information System; n=number of measures reported; N=number of studies

Measures of Physical Activity in Children In children, the most commonly used methods for assessing physical activity were questionnaires (n=46) and electronic monitoring (n=32) (Table 8). Of the 43 natural experiment studies reporting data on physical activity in children, 16 measured physical activity using electronic monitoring (accelerometer n=13, pedometer n=2, telemeter n=2, global positioning system (GPS) n=1), 25 used a questionnaire, one used a record/log, 4 used observation, and 3 used other measures. U.S. studies varied considerably in the physical activity questionnaires used with children in natural experiment studies. Examples included the SPAN Questionnaire and the National Youth Risk Behavior Survey among many others. The International Physical Activity Questionnaire was the most used questionnaire (n=3), followed by the YRBS) Questionnaire (n=2). The first publication describing the use of electronic monitoring in natural experiment studies with children was in 2005 and use of such monitoring peaked in 2014 (n=7). Evidence Table H16 in Appendix H shows details about U.S. secondary data sources containing each of the physical activity measures, by study design (natural experiment, experimental, other design). In U.S. studies conducted in children, natural experiment studies most commonly used data sources such as the Physical Education-Related State Policy Classification System, Family Activity Study (FAS) data, or the School Nutrition-Environment State Policy Classification. (Appendix H, Evidence Table H16). Of the 53 experimental studies reporting on children’s physical activity, 22 used electronic monitoring (accelerometer n=13, pedometer n=7, motion sensor n=1, telemetry n=1), 21 used a questionnaire (i.e., Test of Gross Movement Development), 6 used a record/log, 6 used observation, none used geographic information systems, and 10 used other measures (i.e., Chinese National Measurement Standards on People’s Physical Fitness). (Table 8) Experimental studies in the U.S. in children used tools such as the Self-Administered Physical Activity Checklist and the Test of Gross Movement Development checklist. No questionnaire was used in more than one study. Eight of the questionnaires were self-reported, 3 were recorded by parents or teachers, and the reporting participant in 7 questionnaires could not

Page 50: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

37

be determined (Appendix H, Evidence Table H17). The use of electronic monitoring in experimental studies involving children began in 2003 and peaked in 2015 (n=5).

Measures of Physical Activity in Adults In adults, the most commonly used method for assessing physical activity was through questionnaires (n=40) (Table 8). Of the 32 natural experiment studies reporting data on physical activity in adults, 5 reported the outcome based on electronic monitoring (accelerometer n=4, GPS n=1, unreported n=1), 24 used a questionnaire, 2 used a record/log, one used observation, and 2 used other measures (e.g., Compendium of Physical Activities metabolic equivalent of task values). Data sources with measures of physical activity in U.S. studies of adults using natural experiment studies include the FAS, National Household Travel Survey, and the Trip Identification and Analysis System (Appendix H, Evidence Table H16). The most frequently used questionnaires to assess physical activity were the International Physical Activity Questionnaire (n=3) and BRFSS (n=2) for the natural experimental studies conducted in U.S. adult populations (Table 8). Studies using electronic monitoring in natural experiments with adults were first published in 2014 (n=2), and use has remained low. Of the 12 experimental studies (RCTs and non-randomized controlled trials) reporting on physical activity in adults, 3 used electronic monitoring (accelerometer n=2, laser counter n=1), 10 used a questionnaire (i.e., International Physical Activity Questionnaire), one used a record/log, one used observation, and none used geographic information system or other measures (Table 8). Experimental studies in the United States used measures such as the Godin Leisure Time Physical Activity Questionnaire (Appendix H, Evidence Table H17). Few experimental studies in adults used electronic monitoring and the first ones were published in 2012 (n=2).

Other Co-Outcomes Thirty-seven studies reported on co-outcomes such as commuting behavior, food purchasing behavior, physical environment, and food environment (see Table 9). Overall, most (n=26) of the studies reporting co-outcomes were natural experiment studies. The most common co-outcome was food purchasing behavior (n=17) (e.g., purchasing of healthy and unhealthy food score). Four studies reported on commuting behavior (e.g., average daily commute mode of transport). 6 studies reported on the food environment (e.g., school-reported soda availability), and 8 studies evaluated the physical activity environment (e.g., neighborhood characteristics to facilitate walking).

Page 51: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

38

Table 9. Summary of the distribution of co-outcomes among study designs (N=37 studies) Co-Outcome n Natural Experiment Studies Commuting behavior 3 Food environment 3 Physical activity environment 4 Physical or built environment 0 Food purchasing behavior 12 Other 3 Experimental Studies Commuting behavior 1 Food environment 3 Physical activity environment 2 Physical or built environment 0 Food purchasing behavior 5 Other 1 Other Study Designs Commuting behavior 0 Food environment 0 Physical activity environment 1 Physical or built environment 0 Food purchasing behavior 0 Other 0

N=number of studies

Page 52: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

39

Key Question 4: Experimental and Non-Experimental Methods

Key Question (KQ) 4: Which experimental and non-experimental methods have been used in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes?

Overview While randomized controlled trials (RCTs) and other experimental approaches can provide strong evidence of the effectiveness of some interventions, not all interventions are amenable to randomization. An alternative approach involves use of natural experiment designs, which aim to approximate casual effects by capitalizing on existing variation in an exposure or intervention. Natural experiment studies use a wide range of analytic approaches, which are further described in this section. The goal of KQ 4 was to describe experimental and non-experimental methods in terms of study design and analytic approach.

Key Findings • Of the 294 studies included in this review, 156 were natural experiment studies (53%),

118 were experimental studies (40%), and 20 provided insufficient information to determine if a natural experiment took place (7%) and were classified as “other study designs.”

• Natural experiment studies evaluating policies, programs, and built environment changes most commonly used cross-sectional comparisons of exposed and unexposed groups (n=55, 35%).

• Difference-in-differences approaches that compared exposed and unexposed groups before and after an exposure were used in 45 studies (29%), and pre/post designs that compared one group before and after an exposure were used in 48 studies (31%).

• A small number of natural experiment studies in this review used other non-experimental designs including 4 instrumental variable approaches, 1 regression discontinuity approach, and 4 interrupted time series analyses with more than 2 time points pre- and post-intervention.

Methods

Data Abstraction To address KQ 4, we abstracted details of the natural experiment studies, experimental studies, and other study designs and analytic approaches. After determining whether a study met the UK’s Medical Research Council criteria for a “natural experiment study,” we further classified study designs using the Effective Public Health Practice Project Quality Assessment (EPHPP) tool:91 RCT, controlled clinical trial, cohort, case-control, interrupted time series, cross-sectional, and “other”. Definitions of these designs are provided in Appendix B.

Page 53: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

40

Given that some study designs (such as cohort studies) could be used in multiple ways to estimate effects, we further distinguished the analytic approach of each study. While the EPHPP tool collects information regarding the data collection structure, our risk of bias forms additionally focused on the analytic tools used to analyze data. For each study, we classified the analytic approach as an RCT, instrumental variables design, regression discontinuity, interrupted time series, cross-sectional comparison of exposed and unexposed groups, controlled clinical trial, pre/post comparison, difference in differences, or “other” approach.

Data Synthesis We described the types of study designs and analytic methods being used, and their frequency of use, by study design.

Results

Description of Methods Used in Natural Experiment Studies Over half of the studies (n=156) included in this review met the main criterion for natural experiment studies, based on the MRC Report,41 clearly indicating that the researchers were not in control of the exposure allocation (see Methods for KQ4). Eighty-eight of the natural experiment studies took place in community settings (56%) and 49 (31%) were in other institutional settings (e.g., schools or worksites). Nineteen studies (12%) evaluated an intervention or exposure at the level of the individual. Analysis was performed at the individual level in 90 percent of these studies. For example, Project Healthy Schools, a diet and physical activity intervention, was implemented at 23 schools in Michigan, and its success was evaluated by comparing individual changes in body mass index (BMI) and diet among 6th graders before and after the program was implemented.92 Most studies evaluating community or school-level programs used methods such as multilevel modeling or robust standard errors to account for the hierarchical structure of the data, but 20 studies (13%) did not account for nested data structure in the analysis. The most common analytic approach in natural experiment studies was cross-sectional comparisons of exposed and unexposed groups (n=55, 35%). For example, Taber and colleagues used regression models to compare dietary intake among high school students in California, one of the first states to regulate the nutrition content of competitive foods in schools, to students in states without competitive food laws.93 The second most common design was pre/post with the pre-intervention period serving as the control for the post-intervention period (n= 48, 31%). For example, one study evaluated changes in physical activity after construction of a bus line and car-free walking and cycling route in Cambridge using a pre/post analysis embedded in an existing cohort study. The study compared activity levels collected from an annual survey one year before and one year after the construction was complete.55 Forty-five studies used difference-in-difference approaches (29%) looking at changes before and after the intervention compared to an external control group. This was the third most common study design. This category includes studies such as an evaluation of a new supermarket that opened in a food desert through the Pennsylvania Fresh Food Financing Initiative.55 Researchers collected BMI and fruit and vegetable intake from residents of intervention and control neighborhoods at one point before and one point after the construction of the supermarket.

Page 54: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

41

For the 48 pre/post studies, the average number of time points pre- and post-intervention was 1 and 1.6, respectively. For the 45 difference-in-difference studies, which used changes in time and across groups, there were an average of 1 and 1.2 time points pre- and post-intervention. Among the studies included in this review, all pre/post studies measured variables at a single time point pre-intervention, and 80 percent had a single measure post-intervention. Difference-in-difference studies also all had a single measure pre-intervention, and 83 percent had only a single post-intervention measure. We use the terms pre/post and difference-in-difference for studies with multiple time points post-intervention rather than interrupted time series due to the small number of time points (maximum 6 points) and the lack of formal interrupted time series methods such as autoregressive integrated moving average.

Description of Methods Used in Experimental Studies One hundred eighteen studies (40%) included in this review met the criterion for experimental studies, defined as evaluations of researcher-controlled programs, policies, or built environment changes (see Methods for KQ 4). Experimental methods included 74 RCTs (63% of the experimental studies, 25% of all studies), and 44 controlled trials (37% of experimental studies, 15% of all studies) in which the investigator assigned the exposure by means other than randomization (Table 10). Shape Up Somerville is an example of a controlled clinical trial that took place in one intervention and two control communities in Massachusetts. Researchers selected Somerville as the intervention site due to ongoing relationships with that community, so this would not be considered a natural experiment. This community-wide, multi-level diet and physical activity program included a Walk to School Campaign, changes to school breakfast and lunch programs, school and afterschool education programming, enhanced playgrounds for recess, and a restaurant initiative.94 Of the experimental studies, the unit of intervention allocation was most often at the organization level (66%) or the community level (23%). Analyses were most often conducted at the individual level (93%).

Description of Methods Used in Other Study Approaches Twenty studies in this category (7%) did not provide sufficient detail to determine whether the research team was in control of assigning the intervention, whether the intervention was originally intended to be research, or otherwise would be included as either a natural experiment or experiment. Most often, the exposure was at the community (40%) or organizational (45%) level, but the analysis was carried out at the individual level (90% of studies). Thirty-five percent of these studies did not take the hierarchical nature of the data structure into account in the analysis. These studies were most often pre/post (45%) or difference-in–differences (40%) designs.

Page 55: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

42

Table 10. Overview of study design or data collection structure (N=294 studies) n Percent Natural experiment* studies 156 53

Cross-sectional comparison of exposed and unexposed groups

55 35

Pre/Post 48 31 Difference-in-differences 45 29 Regression Discontinuity 1 1 Interrupted Time Series 4 3 Instrumental Variables 4 3 Other 1 1

Experimental studies 118 40 RCT 74 63 CCT 44 37

Other studies 20 7 Regression model 3 15

Pre/Post 9 45 Difference-in-differences 8 40

*Studies may employ multiple methods and therefore be counted in several categories.

Page 56: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

43

Key Question 5: Risk of Bias Key Question (KQ) 5: What are the risks of bias in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes?

Overview This question aims to evaluate the risks of bias using a standard tool as well as supplemental

study design-specific items to identify methodological/analytic advances that would help strengthen efforts to evaluate the effect of future programs, policies, and built environment changes related to obesity prevention and control

Key Findings • Most natural experiment studies were rated as having a “weak” global rating (i.e., high

overall risk of bias), with 63 percent having a weak rating for handling of withdrawals and dropouts, 42 percent having a weak rating for study design, 40 percent having a weak rating for confounding, 26 percent having a weak rating for data collection.

• Among natural experiment studies, regression adjustment was the most common method to control for confounding (73%), followed by direct covariate matching or stratification (12%).

Methods

Data Abstraction To address KQ 5, we used three complementary approaches recognizing that no risk of bias tool has been specifically designed to assess natural experiment studies, and that all tools have their strengths and limitations. First, to enable comparisons across all study designs (both natural experiment studies and experimental studies), we chose to apply a single risk of bias assessment tool to be used across all studies. Second, we developed additional study design-specific risk of bias questions for the non-experimental designs commonly used in natural experiment studies, such as interrupted time series. Table 11 shows the most commonly encountered types of non-experimental study designs and specific bias concerns we considered relevant to assess. Third, we used an alternative risk of bias assessment tool for a randomly selected sample of the natural experiment studies. The overall intention was to focus on assessing the risk of bias, or internal validity of studies, rather than on applicability or external validity of studies. To assess risk of bias, our goal was to select a tool that could address many of the risk of bias concerns of natural experiment studies in a single scale. We chose the Effective Public Health Practice Project (EPHPP) tool because it was simple to apply, was interpretable across multiple study designs, and had been developed specifically for population-based programs and policies similar to studies in our systematic review. We considered a number of tools and compared The Risk Of Bias In Non-randomized Studies – of Interventions (ROBINS-I)95 against the tool from the EPHPP.96 The EPHPP tool was compared with an earlier version of the ROBINS tool and was shown to have fair inter-rater agreement, a notable challenge for the highly subjective process of risk of bias assessment.97 In its guidance about using natural experiments, the UK’s

Page 57: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

44

Medical Research Council 41, 42 suggested using the Newcastle-Ottawa Scale,98 a tool developed for observational studies. However, our review included both experimental and observational studies (see Scope of the Review in Methods) and therefore we could not apply the Newcastle-Ottawa Scale across all studies. The EPHPP tool yields individual scores of the domains and an overall classification of risk of bias, and addresses the risk of bias domains common to other tools like ROBINS and the Newcastle-Ottawa Scale. We trained our risk of bias reviewers in applying the EPHPP tool. Two reviewers independently assessed each study’s risk of bias using six domains from the EPHPP tool for all studies:96 Table 12 displays the domains from the EPHPP, summarizes the included items, and describes the criteria for receiving a “weak rating.” Appendix D lists all the items for the EPHPP tool. Studies received domain-specific ratings as “strong”, “moderate” or “weak” according to the EPHPP algorithm 96. Each study also received a global risk of bias rating: “strong” if none of the domains were rated as weak, “moderate” if only one of the domains was rated as weak, or “weak” if two or more of the domains were rated as weak. For example, to assess selection bias the EPHPP has questions on whether the selected individuals were likely to be representative of the target population, and what percentage of selected individuals agreed to participate. The study would be rated as “weak” on the selection bias domain if: “the selected individuals are not likely to be representative of the target population and there is less than 60 percent participation, or the selected individuals are not described/can't tell and the level of participation is not described/can't tell.” Any study, including one with a non-experimental study design, could receive a “strong” global rating if it was not rated as “weak” in any of the domains. While we found the EPHPP’s domains applicable to assessing the risk of bias in natural experiment studies, it was not specifically designed for these types of studies, as no tool exists specific to natural experiment studies. To supplement the EPHPP in addressing bias specific to natural experiment study designs, we additionally assessed the methods to address confounding and types of adjustment in all studies, and developed the design-specific risk of bias questions to assess specific threats to bias for each of the non-experimental designs listed in Table 10. In addition, in a subset (n=20) of randomly selected natural experiment studies, we used the Newcastle-Ottawa Scale for Cohort Studies to assess the risk of bias and compared results to those from the EPHPP tool (Appendix L).98

Data Synthesis We described the risk of bias assessments across all studies and by type of study design. As described in the KQ 5 Methods, we used the EPHPP tool for all 294 studies to rate the risk of bias in the 6 domains of selection bias, study design, confounding, blinding, data collection, and withdrawals and dropouts (Appendix C). The EPHPP also provided a global bias rating based on the results across these 6 domains (Appendix C). We summarize the risk of bias ratings across studies by study design type: natural experiment studies, experimental studies, and other studies.

Page 58: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

45

Table 11. Analytic methods that can be used in non-experimental studies and specific bias concerns Analytic Method Definition Key Assumptions Specific Bias Concerns Cross-sectional comparison of exposed and unexposed groups 42

Compares exposed and unexposed groups at a single point in time Propensity score methods or regression adjustment can be used to adjust for observed confounders within this design.

Adjustment for observed confounders No unmeasured confounding

Confounding

Difference in differences 99,

100

Compares exposed and unexposed groups before and after the exposure Takes advantage of variation across time and across groups

Unobserved differences between the two groups do not change over time; i.e., in the absence of the exposure, the trend in outcome in the two groups would be identical.

Changes over time (independent of exposure) that differentially affect exposed or unexposed group

Instrumental variable 101, 102

Involves identifying an “instrument” that influences receipt of the program or policy of actual interest but does not directly influence the outcome; the instrument also needs to be, at least hypothetically, randomized.

Instrument is associated with receipt of the program or policy of interest (testable). Exclusion restriction: no direct effect of the instrument on the outcome; i.e., Instrument associated with outcome only through exposure Instrument randomized (no common cause with outcome), or at least conditionally randomized (conditional on observed factors)

Violation of exclusion restriction Non-randomization of instrument Weak instrument (not strongly predictive of exposure itself)

Interrupted time series103,

104

Assesses change over time before and after a policy intervention Design stronger if also includes data on an untreated comparison group (in which case it is an extension of a difference-in-differences design to multiple time points).

Unobserved differences between groups are fixed No other “interruption” at the time of the policy change Groups would have identical changes in trends in absence of intervention.

Group composition changing over time Comparison group not providing accurate estimate of what would have happened in absence of intervention (e.g., if trends over time differ in unobserved ways between comparison and intervention sites) Change in measurement over time

Page 59: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

46

Analytic Method Definition Key Assumptions Specific Bias Concerns Pre/Post 42 Compares one group before and after

an intervention, with the pre-intervention period serving as the control group for the post-intervention period.

No secular trend in the outcome Changes over time unrelated to the exposure

Regression discontinuity 105-107

Uses a cutoff/ rule to assign intervention status; analysis compares those just above the cutoff to those just below the cutoff to estimate effect of the intervention

Smooth model relating the cutoff variable to the outcome in the absence of the intervention Intervention must have been assigned using the cutoff/rule

Unclear demarcation at cutoff Manipulation of cutoff variable near the cutoff Incorrect model specification above or below the cutoff

Page 60: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

47

Table 12. Summary of Effective Public Health Practice Project quality assessment tool Domain Summary of Items Criteria for Weak Rating Selection bias Representativeness of sample

Participation rate Sample not likely to be representative of target population; or participation rate less than 60%

Study design Type of study design Randomization

Not reported or not RCT, CCT, cohort study, case control, or interrupted time series

Confounders Baseline differences between groups Control for confounding

Less than 60% of identified confounders accounted for in analysis; or confounding not assessed/reported

Blinding Blinding of outcome assessor Blinding of study participants

Both outcome assessor and study participants are not blinded

Data Collection Methods

Validity of data collection tools Reliability of data collection tools

Data collection tools not shown to be valid; or validity and reliability not described

Withdrawals and dropouts

Count and reasons for dropouts Percent completing study

Follow-up rate of less than 60%; or no report of attrition

Global Bias Rating

Summary of all six domains Two or more weak ratings

CCT=controlled clinical trial; RCT=randomized controlled trial

Results

Risk of Bias in Natural Experiment Studies

Figure 8 shows EPHPP risk of bias ratings for the 156 natural experiment studies. The domains most likely to be rated as strong (i.e., low risk of bias) were data collection methods and confounding, but still only a minority of natural experiment studies were rated as strong in those domains: 74 (47%) for data collection, and 69 (44%) for confounding. EPHPP rates studies as “strong” in the domain of confounding if the researchers control for at least 80 percent of identified confounders, or if there were no important differences between exposed and unexposed groups at baseline. For example, one evaluation of the Los Angeles Fast Food Ban used the California Health Interview Survey to compare changes in diet and obesity outcomes among residents of areas affected by the ban compared to residents of areas not affected by the ban.60 In addition to selecting comparison neighborhoods, researchers controlled for both individual level and neighborhood level confounders: individual gender, age, race, household size and income, marital status and income; as well as neighborhood population density, median income, and racial composition. Studies received a “strong” rating in data collection methods if the data collection tools employed were shown to be both valid and reliable; for example, the use of the Recent Physical Activity Questionnaire to evaluate the impact of a new transit system on physical activity.83 See Appendix K for individual risk of bias ratings. EPHPP’s selection bias domain covers the likelihood that the study participants are representative of the target population as well as the percentage of individuals who agree to participate. Forty-three natural experiment studies were rated “strong” on selection bias (28%); 98 studies (63%) were rated as “very likely” that the study sample was representative of the target population, but only 40 studies (25%) reported that 80 to 100 percent of selected individuals agreed to participate. Ninety-nine natural experiment studies were rated as “weak” (i.e., high risk of bias) in the domain of withdrawals and dropouts (64%). In 30 percent of natural experiment studies, withdrawals and dropouts were either not reported or not enough information was given for a reviewer to determine how attrition was handled. For example, one evaluation of

Page 61: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

48

a new bus line and pedestrian thoroughfare collected data from 1143 commuters at baseline, but was only able to collect limited follow-up data as many participants were lost from each group and for unclear reasons83. The EPHPP risk of bias assessment tool rates only randomized controlled trials (RCTs) and controlled clinical trials as “strong” in the study design domain, therefore none of the natural experiments were rated as having “strong” study designs. EPHPP rates studies on blinding of both participants and outcome assessors to intervention status. Although blinding of outcome assessors may be possible in some natural experiment studies, blinding of participants often is not feasible. Thus, it is not surprising that few studies were rated strong in the domain of blinding.

Figure 8. Risk of bias for natural experiment studies (N=156)

*Studies are given a “strong” global rating if there are no domains given a “weak” rating, a “moderate” global rating if there is a single domain with a “weak” rating, and a “weak” global rating if there are two or more domains with a “weak” rating.

Since confounding is a common source of bias in natural experiment studies, we examined the most commonly adjusted for control variables in natural experiment studies (see Table 13). This does not include the criteria on which comparison groups were matched among the studies that used, for example, a matched comparison community. Of the 156 studies, 24 percent controlled for community-level confounders, and 17 percent controlled for school or worksite level confounders. Controlling for individual-level variables such as age, race, gender, and

0%

20%

40%

60%

80%

100%

Strong (low risk of bias)

Moderate

Weak (high risk of bias)

Page 62: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

49

Table 13. Common categories of control variables accounted for in natural experiment studies (N=156)

Category Examples n (%) Age Age, age categories, grade 82 (52) Race/ethnicity Race, Hispanic ethnicity 63 (40) Sex or gender 94 (69) Baseline measure of outcome

BMI or BMI category at baseline, physical activity at baseline

32 (20)

Household socioeconomic status

Income, parental education, food security, eligibility for free lunch, welfare or SNAP recipient

68 (43)

School characteristics Proportion of students by race, proportion of students eligible for free lunch

27 (17)

Community characteristics

Proportion by race in census tract, population density, density of food and alcohol establishments, area-level employment, education, income, urban or rural

38 (24)

BMI=Body Mass Index; n=number of studies; SNAP=Supplemental Nutrition Assistance Program

household socioeconomic status, was more common. Four studies included a sensitivity analysis to explore robustness of the results to unobserved confounders. Among the natural experiment studies, the most common method used to minimize the risk of bias associated with confounding was regression adjustment (73%), followed by direct covariate stratification or matching (12%). Propensity score methods were used in 4 studies (3%). One study used synthetic control methods to create a control group and used that group in models that adjusted for confounding.108 In 2004, the Los Angeles school district began regulating the nutritional content of all food in all its schools. Due to the lack of an appropriate comparison group, the researchers created a synthetic control group, which is a weighted combination of several control districts that more closely resembles the Los Angeles school district prior to 2004, in order to evaluate the policy. Although some studies used multiple methods to control for confounding, such as regression adjustment and matched controls, others used very limited or no methods to control for confounding. A challenge in assessing the success of the confounding control, however, is that 33 percent of studies did not include a “Table 1” with a comparison of key demographic and potentially confounding variables (age, sex, race, etc.) between the exposed and unexposed groups. Instrumental variable, regression discontinuity, and interrupted time series analyses were rarely seen in the studies included in this review. We reviewed fewer than five of each of these studies and cannot generalize risk of bias in these designs due to the small numbers (see KQ 4).

Risk of Bias in Experimental Studies As shown in Figure 9, experimental studies rated strongly in the domains of study design, confounding, and data collection methods, meaning that risk of bias in these domains is low. However, most of these studies were rated as moderate or weak in the areas of blinding, selection bias, and handling of withdrawals and dropouts. In 57 percent of RCTs, the study population was rated very likely to be representative of the target population. Studies also received weak ratings in these categories when they failed to provide the information needed to assess these domains. See Appendix K for individual study risk of bias ratings.

Page 63: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

50

Figure 9. Risk of bias for experimental studies (N= 118)

*Studies are given a “strong” global rating if there are no domains given a “weak” rating, a “moderate” global rating if there is a single domain with a “weak” rating, and a “weak” global rating if there are two or more domains with a “weak” rating.

Risk of Bias in Other Study Designs

As shown in Figure 10, the 20 studies (7%) in this category did not provide sufficient detail to determine whether the research team was in control of the intervention, whether the intervention was originally intended to be research, or otherwise would be included as either a natural experiment or experiment. Sixteen of these studies (80%) received “weak” (i.e., high risk of bias) global ratings using the EPHPP tool. The domains where they were most likely to be rated “weak” included blinding (50% rated weak) and withdrawals and dropouts (65% rated weak). Only 45 percent of the studies described how potential confounders and key demographic information differed between exposed and unexposed groups, making assessments of confounding challenging. These studies did control for some individual confounders such as age (50%) and gender (45%), but none controlled for community level confounders and only 15 percent controlled for school or site level confounders.

0%

20%

40%

60%

80%

100%Strong (low risk of bias)

Moderate

Weak (high risk of bias)

Page 64: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

51

Figure 10. Risk of bias for other study designs (N=20)

*Studies are given a “strong” global rating if there are no domains given a “weak” rating, a “moderate” global rating if there is a single domain with a “weak” rating, and a “weak” global rating if there are two or more domains with a “weak” rating.

Risk of Bias Assessment Using Another Scale Because no risk of bias tool has been developed specifically for natural experiment studies,

we compared our bias assessment from EPHPP with an assessment from the Newcastle-Ottawa Scale98 (Appendix L) in a random subsample of 20 natural experiment studies. The domains of the EPHPP and the Newcastle-Ottawa Scale are not directly comparable, and only the EPHPP has a global rating score. The Newcastle-Ottawa scale includes three domains: selection, comparability (of exposed and unexposed groups), and outcome (includes both method of outcome assessment and follow-up rates). As mentioned previously, the EPHPP tool includes the domains of selection bias, study design, confounding, blinding, data collection methods, withdrawals and dropouts, and a global rating. Overall, we showed that fewer natural experiment studies received the highest rating (lowest risk of bias) using the Newcastle-Ottawa Scale, compared with the EPHPP tool. For example, using the Newcastle-Ottawa Scale, only 3 of 20 studies received the highest rating in the “selection” domain, 9 of 20 studies in the “confounding” domain, and one of 20 studies in the “outcomes” domain. Using the EPHPP tool, 5 of 20 received the maximum rating in selection and half received the maximum score in the confounding domain (Appendix L).

0%

20%

40%

60%

80%

100%

Strong (low risk of bias)

Moderate

Weak (high risk of bias)

Page 65: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

52

Key Question 6: Methodological Advances Key Question (KQ) 6: What methodological/analytic advances (e.g., data system features, approaches to linking data sources, or analytic methods) would help to strengthen efforts to estimate the effect of programs, policies, or built environment changes on obesity prevention and control?

Overview KQ6 provided an opportunity to get feedback from experts in the field of obesity research

about suggestions for methodological/analytic advances (e.g., data system features, approaches to linking data sources, or analytic methods) to strengthen efforts to estimate the effect of programs, policies or built environment changes on obesity prevention and control.

Key Finding • The internal advisors agreed about the importance of 24 methodological/analytic

advances that the core research team suggested for strengthening efforts to estimate the effect of programs, policies, or built environment changes on obesity prevention and control.

Methods

Data Abstraction To address KQ 6 about the needs for methodological/analytic advances, we followed the

following steps to engage our research team, internal advisors, and external experts:

1. We asked each investigator on the research team to suggest answers to KQ, 6 taking into consideration the results for KQ 1-5, and the following specific questions:

a. What features or types of data sources could help researchers advance methods for conducting studies, especially natural experiments, of how programs, policies or built environment changes affect obesity prevention and control?

b. What methodological advances would help to facilitate better or more frequent linkage of population-based data sources for studies, especially natural experiments, in obesity prevention and control?

c. What methodological advances (e.g., measures and data collection procedures) would help to strengthen the assessment of obesity-related outcomes, including dietary and physical activity behaviors in studies, especially natural experiments, of how programs, policies or built environment changes affect obesity prevention and control?

d. What study design and analytic approaches would help to strengthen methods and reduce the risk of bias encountered in studies, especially natural experiments, of how programs, policies or built environment changes affect obesity prevention and control?

Page 66: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

53

2. We discussed the suggestions as a team and created a master list of unique items based on the consensus of the team.

3. We developed a form to elicit feedback about the list of methodological/analytic advances that could improve obesity prevention and control natural experiments (See Appendix F for the form).

4. We asked our internal advisors to review the form and provide input on the list. We chose the experts based on their expertise. All of the internal advisors have training and practice in health policy. Two experts have clinical expertise in obesity prevention and control (adults and children). Additionally, the internal advisors provide expertise in: economic decision making; cost-effectiveness; and housing, community planning, and urban development.

5. We obtained additional input from external technical experts about the suggested methodological/analytic advances that are needed by asking them to complete the form as part of their review of the draft of the evidence report.

Data Synthesis For this report, we prepared a master list of methodological/analytic advances that would help strengthen efforts to estimate the effect of programs, policies, or built environment changes on obesity prevention and control, as developed the study team. At the request of the Pathways to Prevention Workshop planners, we did not prioritize the items on the list.

Results Our research team developed questions to elicit feedback from internal advisors and external experts about what methodological/analytic advances (e.g., data system features, approaches to linking data sources, or analytic methods) would help to strengthen efforts to estimate the effect of programs, policies or built environment changes on obesity prevention and control. We then developed several recommendations within each of these questions aligned with our KQs: KQs 1 and 2 were fcoused on identifying methodologic or analytic advances that may impact the frequency and better use of data sources in public health studies; KQ 3 was focused on identifying methodologic and analytic advances that would lead to more consistent reporting of obesity-realted outcomes and measures; KQs 4 and 5 were focused on identifying methodologic and analytic advances that would lead to more consistent use of terms in reporting study designs, as well as lead to more time series designs, natural experiment studies, and development and use of validated tools. Table 14 summarizes these questions and the corresponding recommendations from our research team regarding methodological/analytic advances that could help to strengthen efforts to estimate the effect of programs, policies, or built environment changes on obesity prevention and control. These recommendations were then independently reviewed by our internal advisors and external experts having varied areas of relevant expertise (see Methods for details on area of expertise). The advisors agreed that 24 of the 26 items were important. The only items that were not endorsed by all advisors were the call for consistent use of a standard format for describing how data were collected and validated (including data quality control processes), and the call for requiring an explanation of how each data source was intended to be used, both addressing KQ1.

Page 67: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

54

Table 14. Analytic and methodologic advances identified by internal and external experts Key Question(s)

Form Question Answers

1 What features or types of data sources could help researchers advance methods for conducting studies, especially natural experiments, of how programs, policies or built environment changes affect obesity prevention and control?

A process that encourages and facilitates dissemination of publicly available information about the existence and location of data sources that include information about obesity-related outcomes. A process that encourages and facilitates dissemination of publicly available information about the existence and location of data sources that include information about programs, policies, or built environment changes that could affect obesity prevention and control Consistent use of standard terminology in data sources that include obesity-related information, including such concepts as: classification of interventions intended to contribute to obesity prevention and control (e.g., governmental program or policy, non-governmental program or policy, or built environment change); and units of measurement of obesity-related outcomes Consistent use of data dictionaries and codebooks that define all elements, and that are readily accessible and searchable Consistent use of a standard format for describing how data were collected and validated (including data quality control processes) Explanation of how each data source was intended to be used Description of the timing of interventions in data sources containing information about programs, policies, or built environment changes that could affect obesity prevention and control Inclusion of BMI and other health behavior data in data sources covering small geographic areas

2 What methodological advances would help to facilitate better or more frequent linkage of population-based data sources for studies, especially natural experiments, in obesity prevention and control?

Consistent use of standard procedures for linking health care data from different sources Development and use of new methods for linking data on social determinants of health with health care data Development of reporting standards by publishers for the description of how data sources are linked when studies report more than one data source in manuscripts Adoption of standard policies and procedures for exchanging/sharing data

Page 68: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

55

Key Question(s)

Form Question Answers

3 What methodological advances (e.g., measures and data collection procedures) would help to strengthen the assessment of obesity-related outcomes, including dietary and physical activity behaviors in studies, especially natural experiments, of how programs, policies or built environment changes affect obesity prevention and control?

Standardized measures and methods for obesity-related outcomes across populations and studies to better facilitate comparisons Improving data collection measures for park-based and transportation-based studies to assess changes in individual and population physical activity beyond observing park or transportation use Assessment of the validity of intermediate outcomes such as studies of food purchasing behavior correlated to behavioral and health outcomes Longer term surveillance of measures to measure the impact of an intervention on changes in obesity measures over time Inclusion of BMI and other health behavior data in data sources covering small geographic areas Establishing standards for the collection of obesity-related outcomes in adults and children Consistent use of validated measures of obesity-related outcomes across studies. Consistent use of terminology in describing measures of obesity-related outcomes used in surveys or observational studies

4 and 5 What study design and analytic approaches would help to strengthen methods and reduce the risk of bias encountered in studies, especially natural experiments, of how programs, policies or built environment changes affect obesity prevention and control?

Consistent use of standards for terms and reporting in studies of obesity prevention and control using natural experiment designs, including: study design, testing assumptions, sensitivity analyses, observed (and adjusted for) confounders, and unobserved confounders Design-specific reporting standards for observational study methods used in natural experiments related to obesity prevention and control Greater use of time-series designs instead of simple pre-post comparisons in studies of obesity prevention and control Greater use of stronger natural experiment designs such as instrumental variables and regression discontinuity Development and use of validated instruments for measuring the “exposures” in studies of how programs, policies, or built environment changes affect obesity prevention and control Development and use of a specific practical and validated tool for assessing the risk of bias in observational studies of how programs, policies, or built environment changes affect obesity prevention and control

BMI=body mass index

Page 69: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

56

Discussion We conducted a systematic review of studies that evaluate how policies, programs, and built environment changes affect obesity prevention and control outcomes. For Key Question (KQ) 1 and KQ 2, we focused on summarizing and evaluating the data sources and data linkages used in those studies. For KQ 3, we summarized how studies assessed childhood and adult obesity outcomes, dietary and physical activity behaviors, and other co-outcomes (e.g., commuting behavior). For KQ 4 and KQ 5, we evaluated the risks of bias in the experimental and non-experimental studies. For KQ 6, we identified specific methodological/analytic advances that could strengthen efforts to estimate the effectiveness of programs, policies, and built environment changes intended to improve obesity prevention and control. We identified 294 studies (including 156 natural experiment studies), a majority of which were conducted in the United States. We included a wide range of policies, programs and built environment changes, including 152 studies that evaluated governmental programs or policies at the local, state/regional, or federal levels with 139 unique policy or program evaluations. For KQs 1 and 2, we reported on 93 studies using a primary or secondary data source that were sharable with other researchers, with a total of 116 unique data sources. Among these, 106 data sources met criteria for being a data system. Data systems represent organized, accessible data sources that go behind just collecting and managing data, but also have some degree of an information technology infrastructure to maintain and operate the system. Of the 106 data systems, 96 were used in natural experiments. One third of the 71 US data systems were linked with a data source other than a primary data source, most commonly using a geographic allocation (e.g., by county or zip code) or at the individual-level. For KQ 3, we found 112 studies with childhood BMI/weight outcomes (50 were natural experiment studies), 32 studies with adult weight/BMI outcomes (17 were natural experiment studies), 148 studies with dietary behaviors (77 were natural experiment studies) and 152 studies on physical activity (71 were natural experiment studies). Thirty-seven of the studies also reported on commuting behaviors, food environment, physical environment, or food purchasing behaviors. Weight was most commonly directly measured by trained staff. Questionnaires for assessing dietary behaviors included the School Physical Activity and Nutrition and Youth Risk Behavior Survey (YRBS) Questionnaires in children, and the Behavioral Risk Factor Surveillance System (BRFSS) in adults. The most common methods for assessing physical activity were questionnaires and electronic monitoring, e.g., pedometers, for all types of study designs. For KQs 4 and 5, natural experiment studies most commonly used regression models to compare exposed and unexposed groups at a single time point (35%). The next most common analytic approach was comparing exposed and unexposed groups using difference-in-difference methods (29%), followed by pre/post designs that compared one group before and after an exposure (31%). Fewer studies used instrumental variable, regression discontinuity or interrupted time series approaches with more than 2 time points pre- and post-intervention. Most natural experiment studies were rated as having a “weak” global rating (i.e., high risk of bias) due to handling of withdrawals and dropouts, weak study design and weak handling of confounding.

Page 70: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

57

Findings in Relationship to What Is Already Known Our systematic review is the first to focus on describing and appraising the methods (data sources, study design, and analytic approaches and risks of bias), used in natural experiment studies of policies, programs, and built environment changes to prevent and control obesity. Our review was broad and comprehensive, and included both U.S. and non-U.S. studies, all types of policy, programmatic and built environment interventions and all study designs. Although our review focused on evaluating the methods, other recent systematic reviews identified many of the same articles but focused on assessing the effectiveness of certain types of policies or programs, such as on food environment policies,24, 109-111 (e.g., the review by Sisnowski et al. included 36 articles), or built environment changes (e.g., reviews by Ferdinand et al included 169 articles 109 and Mayne et al included 37 articles36), as well as specific settings such as school-based programs or policies112 (included 32 articles). Another 2013 systematic review included 147 studies focused on the effectiveness of childhood obesity prevention.113 Finally, Gudzune and colleagues’ systematic review focused on the prevention of weight gain in the worksite or college settings.114 Our systematic review has several significant and original contributions to advance the field of obesity prevention and control, particularly for informing the future design and reporting of natural experiment studies related to obesity prevention and control. Although a few prior studies have described the current state of data sources and data linkages in public health research (e.g., a review of data systems for the field of suicide prevention),86 no studies have created this list in the field of obesity, which will be useful for future obesity research. Additionally, for obesity measures, a database of measures exists for childhood obesity, called the National Collaborative on Childhood Obesity Research (NCCOR) Measures Registry,53 but no database exists for adult obesity research. The NCCOR serves to catalog the data sources, but does not assess which measures are used in research or other evaluations, which is a major contribution of our examination of KQ 3. Finally, for KQs 4 and 5, we described study designs and analytic approaches currently being used in natural experiment studies, and determined the extent to which studies were limited by specific types of bias. While there is growing interest in using natural experiments to evaluate obesity policies and programs, a major contribution of our review was to highlight areas for improvement in the approaches, and the need for stronger guidance and standards for evaluators to report their designs and findings to enhance trust in their results.

Applicability The applicability of our review and the identified studies depended on the diversity of study populations, obesity policies and programs, and outcomes assessed. We summarized applicability in terms of the populations, interventions, comparators, outcomes, timing, setting framework, as well as in terms of each of the KQs. The populations of the studies included both obese and non-obese children and adults, and these populations were generally applicable to other populations and communities. However, of the 261 studies, the majority were of children, with fewer evaluations of policies and programs in adult populations or community (non-school) settings. Many studies did not report the races and ethnicities of their populations being studies. Of those that reported race and ethnicity, few studies included Native Americans, or targeted a high proportion of Hispanics.

Page 71: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

58

We included interventions that were governmental and non-governmental population-based programs. We identified a wide range of obesity-related policies and programs, including state- (e.g., Arkansas Act 1220 focused on school nutrition and physical activity)115 and federal-level legislation (e.g., Food Stamp Program).75, 116-120 These evaluations of policies and programs are applicable and specific to the setting where the policy or program was enacted and implemented. To enhance applicability of the review, we included studies that reported health outcomes and behaviors important to policymakers, program evaluators, researchers and clinicians, namely weight, dietary behaviors and physical activity. Many studies reported on childhood weight outcomes with fewer studies reporting on adult weight or body mass index. Even more studies reported dietary or physical activity measures. These outcomes are applicable to the population of interest, and are important for assessing the impact of obesity related policies and programs, particularly given the growing epidemic of obesity in both adults and children. Although weight and behavior change may be long-term outcomes, some evaluations of programs and policies have begun to demonstrate significant impact on these outcomes.19, 56 The studies’ settings were both in the United States and other countries, and set in the workplace, school and diverse communities. We included a total of 106 studies from outside the United States. Few studies reported their setting as within a rural community. Regarding the applicability of KQ1 results, we showed that the majority of the U.S. data systems were statewide or nationally representative. Data systems with a state-level focus were mainly collected in California, New York, Massachusetts, Minnesota, Pennsylvania, Texas and Alaska. Most of the U.S. data systems had a focus on schools or communities. For KQ4 and KQ5, we described methods and risks of bias in natural experiment studies, as well as experimental designs, and highlighted the strengths, as well as risks of bias associated with each of them. Although randomized controlled trials (RCTs) are considered the gold standard to reduce risks of bias, RCTs are challenging to implement due to high cost and often randomization of participants or communities to policies or programs is not feasible. To improve the applicability and validity of their evaluations, researchers could consider innovative trial designs that would allow randomization, such as stepped wedge or waitlist control designs. However, few studies used these approaches. The questions in obesity policy research are well suited for natural experiment study designs to increase the internal and external validity of their studies when assessing causal effects.

Limitations of the Systematic Review Process We noted several limitations to our systematic review process. First, we developed several exclusions, most significantly excluding: studies without a clear comparison or unexposed group or time period; studies without a defined population-based program, policy, or built environment change; and studies without our main outcomes of weight or obesity-related health behavior. Examples of studies that we excluded are those that assessed the associations between perceived or measured home, school, or physical environment and various weight-related outcomes (e.g., Wong et al. 2016121) but without a described program, policy, or change in the environment. We sometimes excluded studies that evaluated policies because they did not include one of our main outcomes, such as the study by Chen and colleagues evaluating the effect of menu labeling on change in caloric information awareness,58 but without a measure of change in dietary intake. We also excluded park and transportation studies that reported observed number of users of the park or form of transportation, instead of the individual–level change in physical activity. For example, a study by Fitzhugh and colleagues assessed the effect of an urban greenway/trail on

Page 72: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

59

directly observed physical activity in the general neighborhood and school, but did not have measures of individual behavior change, which we required in this review.122 Second, regarding our evaluation of data sources and linkages, we limited our full assessment of data specifications to studies conducted in the United States to enable consistent access and identification of codebooks in English. However, even among these data sources, our ability to fully assess data quality was limited by the low availability of online codebooks and data dictionaries. Third, although the focus of the review was to improve our understanding of obesity-related natural experiments, the definition of what methods and designs meet a definition of a “natural experiment” continues to evolve. As described in the Methods, we classified “natural experiments” according to the MRC41 definition as those where “the intervention/control was assigned to participants/communities/schools by factors outside the control of the investigators.” In applying this definition, we identified some studies where it was not clear. For example, one study evaluated the Healthy Hawaii initiative and provided trends in percent of population eating fruits and vegetables by year, but it was neither a natural experiment study nor an experimental design.123 We presented the findings of this review by “natural experiment”, experimental designs, and other non-experimental designs, in order to compare within and across study design approaches, and also to avoid being constrained by an evolving, and often subjective, natural experiment definition. Fourth, we relied on a previously developed risk of bias assessment tool. No well-established risk of bias assessment tool exists specifically for natural experiment studies. To enable comparisons across all study designs (both experimental and non-experimental) included in the review, our goal was to use a single tool. A general tool such as the Effective Public Health Practice Project (EPHPP) is useful for gathering information on risk of bias due to a number of bias concerns, including measurement error and loss to follow-up and withdrawals. However, tools like EPHPP have been criticized for being overly rigorous and failing to take into account feasibility, implementation and future scalability, which are important issues for natural experiment studies. In addition, risk of bias tools, including the EPHPP, are designed to evaluate internal validity and not external validity, while the strength of a natural experiment study is to enhance generalizability and external validity, which these tools may not address. In its guidance, the UK’s Medical Research Council (MRC) 41 suggested using a different risk of bias assessment tool for natural experiment studies, the Newcastle-Ottawa Scale,98, which was developed for cohort studies. We additionally applied the Newcastle-Ottawa Scale for cohort studies98 to a randomly selected subset of natural experiment studies. The Newcastle-Ottawa Scale has fewer domains and no global score, making it challenging to compare between the two tools. Overall, we showed that the natural experiment studies were similarly rated as “weak” in terms of risk of bias, across both the EPHPP tool and the Newcastle-Ottawa Scale. The EPHPP and our specific study design approach had other limitations, including our inability to detect the degree of risk from type 2 error among smaller RCTs or cluster RCTs with few clusters. An additional challenge was that any non-experimental study involves some untestable assumptions, and thus we could not directly assess bias but, rather, could only assess the likely validity of the assumptions underlying each approach. In addition, few of the studies did formal assessments of the robustness of results to violations of those assumptions.

Page 73: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

60

Limitations of the Current Evidence We identified several limitations of the current evidence on methods for obesity-related natural experiments and other evaluations of obesity prevention and control policies and programs. First, the current evidence base not does yet take advantage of the plethora of available data sources on obesity. Natural experiment studies used many national health surveys such as National Health and Nutrition Examination Survey 124 and BRFSS.125 The Patient Centered Outcomes Research Institute (PCORI) has established PCORNet with 13 Clinical Data Research and 20 Patient Powered Research networks to link patient data longitudinally across patient organizations as well as large health systems in the United States.126 Although the infrastructure is still under development and testing, PCORNet provides an example of how health care data and outcomes could be used to evaluate obesity prevention policies and programs, especially through linkages with other public health data sources. In addition, the Natural Experiments for Translation in Diabetes collaborations provide examples of how health systems are using natural experiment approaches to evaluate diabetes management and prevention approaches, including telephonic health coaching programs, insurance policies involving high deductibles, early diabetes detection in primary care, and national scaling of the Diabetes Prevention Program.127 Further, to improve the ability of obesity researchers to use and link data sources with spatial attributes, these data should contain consistent spatial units of analysis that can be easily linked together. For example, census data and traffic data may come in with different spatial units, which makes it difficult to link these data together. As with other obesity prevention and control studies, natural experiments have many of the same challenges in terms of obtaining valid and reliable measures of dietary intake, physical activity and weight status. All obesity prevention studies should aim to find practical ways to obtain high quality, reliable measures and ensure that the assessment of these variables in other data systems be of the highest possible quality. In our review, we identified several different measures for obesity outcomes. For example, for childhood obesity, we described the use of standard measures, such as change in body mass index (BMI) z-score or BMI percentile; however, several studies used other non-standard measures in children, such as absolute BMI change or weight change in children. We identified a need for population-based studies to use objective measures of diet (e.g., 24 hour recalls) and physical activity (e.g., accelerometers), and to expand measures of body composition beyond body mass index. Additionally, we identified very few studies that reported on co-outcomes or unintended consequences associated with policies or programs.

This review provided a unique opportunity to describe the current evidence in terms of study design and analytic approaches, including natural experiment studies and experimental approaches to evaluate policy and program interventions for obesity prevention and control. Although RCTs are the gold standard for clinical efficacy research, they are often impractical and/or costly when testing the effectiveness of policies or programs that are new or have been adapted from other settings. The MRC’s definition of natural experiments involves studies with “unplanned variation in the exposure of interest.”41 Because experimental studies also have limitations and biases, in our review we chose to include both experimental designs where the exposure of interest was controlled, as well as non-experimental studies with “unplanned variation” in the exposure. Among the natural experiment studies, we expected to find common non-experimental designs, such as instrumental variables, regression discontinuity, propensity score, and interrupted time series methods. However, we showed that few studies used these study designs

Page 74: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

61

that could improve causal inferences. It is possible that use of instrumental variables is limited by difficulties in selecting appropriate instrumental variables with the data typically available to researchers in this field. By far the most common design was cross-sectional regression models. In addition, many studies failed to present a table of covariate balance across groups to enable assessment of risk of bias due to confounding. Among the natural experiment study designs, an evaluation of Arkansas’ Act 1220 was rated as having lower risk of bias.115 They used a comparison group design using a difference in difference approach to assess the effect of Arkansas Act 1220’s mandate on BMI screening in public schools on BMI z-score, using the Centers for Disease Control and Prevention (CDC)’s YRBS.115 This study received a higher EPHPP rating because it used both valid and reliable data collection tools, was rated as being generalizable, and neither participants nor outcome assessors knew the exposure status of the participants. For pre/post studies (i.e., where an exposed group was compared to itself in a prior time period), the average number of time points pre- and post-intervention was 1 and 1.4, respectively. For difference-in-difference studies, which use changes in time and across groups, there were an average of 1 and 1.1 time points pre- and post-intervention. The MRC recommends multiple pre- and post- measures to improve the design of natural experiment studies, especially when a control group is not available as in the pre/post designs.41 Among the studies included in this review, all pre/post studies measured variables at a single time point pre-intervention, and 86 percent had a single measure post-intervention. Depending on the research question, the type of natural experiment being evaluated, and the stability of the outcome of interest, adding multiple time points should improve study validity, but could have cost/time implications for researchers and funders. A standard definition of selection bias is “bias in the estimated association or effect of an exposure on an outcome that arises from the procedures used to select individuals into the study or the analysis”.128 When the selection involves conditioning on a factor that is affected by the exposure or a cause of the exposure, and also affected by the outcome or a cause of the outcome, selection bias can arise even in the absence of a causal effect of exposure on outcome, i.e., under the causal null hypothesis. Because such uses of “selection bias” do not imply lack of internal validity, it is more appropriate to use the expressions “lack of generalizability” or “low external validity.” To include concerns about external validity, which is particularly relevant to natural experiment studies, we applied the EPHPP’s definition of selection bias to natural experiment studies, assessing to what extent the individuals selected to participate in the study are likely to be representative of the target population.91

Future Research Needs and Opportunities We identified a large number of natural experiment and experimental studies assessing the

effects of obesity programs and policies on body weight, BMI, diet and physical activity. Our results highlight a need for future research to use better methodological standards to enhance validity and reporting. To improve the use of data systems and create linkages, we suggest that obesity researchers make greater use of the health information technology infrastructure to conduct pragmatic evaluations of obesity-related policies and programs. Since many people do not obtain longitudinal continuity care, communities need ways to collect ongoing health and behavioral information, as well as link with health care systems, community-level, school, and public health data sources to aim for completeness. Several good examples are Michigan’s

Page 75: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

62

Project Healthy Schools129 and Shape Up Somerville (MA).94 Michigan’s Project Healthy Schools represented a collaboration between the University of Michigan and local community organizations, including public schools, to provide an opportunity for long-term (four years reported in the included study129) data collection to assess the effect of a wellness program in school. Shape Up Somerville provided another example of longitudinal data collection in one community to assess the long-term effects of a multilevel childhood obesity prevention program, supported by the CDC.94 In the future, schools and school systems could collect health information, in particular standard weight and height measures, from their students and then link these data with other sources to increase their ability to evaluate policies and programs aimed at obesity prevention. Ongoing data collection could address the need for having “baseline” assessments available for natural experiment researchers. To enhance validity and trustworthiness of future research, natural experiments in obesity will need to graduate their approaches from simple two-time point pre/post assessments to include approaches that capture individuals at multiple time points and use multiple comparison/control groups that each have their limitations. To improve the selection of and design of valid comparisons groups, researchers could consider approaches such as propensity score matching and regression discontinuity.44, 103, 130 For example, regression discontinuity methods take advantage of existing rules or cutoff points that determine receipt of the intervention of interest (e.g., individuals above some BMI threshold receive a program; those below that threshold do not). Persons just above or just below the cutoff are assumed to be very similar, so comparing these groups allows for a valid estimate of the effect of the intervention.130 Propensity score methods are also underused in obesity research; many studies did not even report the similarity of exposed and unexposed groups, making it hard to know whether they were or were not similar on observed confounders. Propensity score methods can help ensure the comparison of groups that are similar on observed confounders and, thus, avoid extrapolation and model dependence. Also, accompanying sensitivity analyses can be used to assess robustness of the results to a potential unobserved confounder. Key Question 6 highlighted methodologic and analytic advances that could help to strengthen efforts to estimate the effect of programs, policies, or built environment changes on obesity prevention and control. These advances include consistent use of data dictionaries, reporting standards on linkage methods of data sources, data sources with long-term public health surveillance of obesity and health behavioral outcomes and use of study designs with multiple pre- and post- exposure time points. Because natural experiments are often conducted to evaluate feasibility, implementation barriers, future research is needed to evaluate studies in terms of implementation outcomes, as current risk of bias instruments do not address feasibility or scalability. Finally, there is growing interest in using systems science approaches to tackle complex population health problems like obesity,131, 132 but we do not currently have risk of bias or other methods to evaluate the quality or risks of bias in for studies that apply these new methodologies. Finally, our review highlights the need for methodological standards in study design, analyses, and reporting of data system features and data linkages. Despite guidance from the MRC41 and the PCORI methodology committee,133 no standards like the Consolidated Standards Of Reporting Trials for trials exist for natural experiment or policy evaluation studies. We recommend standard reporting guidelines to enhance the rigor and consistency of research using natural experiments.

Page 76: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

63

Conclusions Our systematic review identified a large heterogeneous sample of natural experiment studies and data sources that have been used to estimate the effect of programs, policies, or built environment changes on obesity prevention and control. The studies used a wide variety of outcome measures and analytic methods, often with substantial risk of bias. The findings reinforce the need for methodological and analytic advances that would strengthen efforts to improve obesity prevention and control.

Page 77: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

64

References 1. Flegal KM, Kruszon-Moran D, Carroll MD,

et al. Trends in Obesity Among Adults in the United States, 2005 to 2014. JAMA. 2016 Jun 07;315(21):2284-91. doi: 10.1001/jama.2016.6458. PMID: 27272580.

2. Ogden CL, Carroll MD, Kit BK, et al. Prevalence of obesity and trends in body mass index among US children and adolescents, 1999-2010. JAMA. 2012 Feb 1;307(5):483-90. doi: 10.1001/jama.2012.40. PMID: 22253364.

3. Ogden CL, Carroll MD, Curtin LR, et al. Prevalence of high body mass index in US children and adolescents, 2007-2008. JAMA. 2010 Jan 20;303(3):242-9. doi: 10.1001/jama.2009.2012. PMID: 20071470.

4. Ogden CL, Carroll MD, Kit BK, et al. Prevalence of childhood and adult obesity in the United States, 2011-2012. JAMA. 2014 Feb 26;311(8):806-14. doi: 10.1001/jama.2014.732. PMID: 24570244.

5. Global action plan for the prevention and control of noncommunicable diseases 2013–2020 World Health Organization. Geneva, Switzerland: Organization WH; 2013. http://apps.who. int/iris/bitstream/10665/94384/1/9789241506236_ eng.pdf

6. Swinburn BA, Sacks G, Hall KD, et al. The global obesity pandemic: shaped by global drivers and local environments. Lancet. 2011 Aug 27;378(9793):804-14. doi: 10.1016/s0140-6736(11)60813-1. PMID: 21872749.

7. Mokdad AH, Ford ES, Bowman BA, et al. Prevalence of obesity, diabetes, and obesity-related health risk factors, 2001. JAMA. 2003 Jan 01;289(1):76-9. PMID: 12503980.

8. Peitz GW, Troyer J, Jones AE, et al. Association of body mass index with increased cost of care and length of stay for emergency department patients with chest pain and dyspnea. Circ Cardiovasc Qual Outcomes. 2014 Mar;7(2):292-8. doi: 10.1161/circoutcomes.113.000702. PMID: 24594550.

9. Yusuf S, Hawken S, Ounpuu S, et al. Obesity and the risk of myocardial infarction in 27,000 participants from 52 countries: a case-control study. Lancet. 2005 Nov 05;366(9497):1640-9. doi: 10.1016/s0140-6736(05)67663-5. PMID: 16271645.

10. Lu Y, Hajifathalian K, Ezzati M, et al. Metabolic mediators of the effects of body-mass index, overweight, and obesity on coronary heart disease and stroke: a pooled analysis of 97 prospective cohorts with 1.8 million participants. Lancet. 2014 Mar 15;383(9921):970-83. doi: 10.1016/s0140-6736(13)61836-x. PMID: 24269108.

11. Arterburn DE, Maciejewski ML, Tsevat J. Impact of morbid obesity on medical expenditures in adults. Int J Obes (Lond). 2005 Mar;29(3):334-9. doi: 10.1038/sj.ijo.0802896. PMID: 15685247.

12. Fitzgerald S, Kirby A, Murphy A, et al. Obesity, diet quality and absenteeism in a working population. Public Health Nutr. 2016 May 27:1-9. doi: 10.1017/s1368980016001269. PMID: 27230727.

13. Gortmaker SL, Swinburn BA, Levy D, et al. Changing the future of obesity: science, policy, and action. Lancet. 2011 Aug 27;378(9793):838-47. doi: 10.1016/s0140-6736(11)60815-5. PMID: 21872752.

14. Hammond RA. Complex systems modeling for obesity research. Prev Chronic Dis. 2009 Jul;6(3):A97. PMID: 19527598.

15. Rossen LM, Schoendorf KC. Measuring health disparities: trends in racial-ethnic and socioeconomic disparities in obesity among 2- to 18-year old youth in the United States, 2001-2010. Ann Epidemiol. 2012 Oct;22(10):698-704. doi: 10.1016/j.annepidem.2012.07.005. PMID: 22884768.

16. Committee on Evaluating Progress of Obesity Prevention Effort, Food and Nutrition Board, Institute of Medicine. Evaluating Obesity Prevention Efforts: A Plan for Measuring Progress. Washington DC: 2013 by the National Academy of Sciences; 2013.

Page 78: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

65

17. Falbe J, Thompson HR, Becker CM, et al. Impact of the Berkeley Excise Tax on Sugar-Sweetened Beverage Consumption. Am J Public Health. 2016 Oct;106(10):1865-71. doi: 10.2105/ajph.2016.303362. PMID: 27552267.

18. Elbel B, Kersh R, Brescoll VL, et al. Calorie labeling and food choices: a first look at the effects on low-income people in New York City. Health Aff (Millwood). 2009 Nov-Dec;28(6):w1110-21. doi: 10.1377/hlthaff.28.6.w1110. PMID: 19808705.

19. Restrepo BJ. Calorie Labeling in Chain Restaurants and Body Weight: Evidence from New York. Health Econ. 2016 Jul 24doi: 10.1002/hec.3389. PMID: 27451966.

20. Vadiveloo MK, Dixon LB, Elbel B. Consumer purchasing patterns in response to calorie labeling legislation in New York City. Int J Behav Nutr Phys Act. 2011 May 27;8:51. doi: 10.1186/1479-5868-8-51. PMID: 21619632.

21. Dubowitz T, Ghosh-Dastidar M, Cohen DA, et al. Diet And Perceptions Change With Supermarket Introduction In A Food Desert, But Not Because Of Supermarket Use. Health Aff (Millwood). 2015 Nov;34(11):1858-68. doi: 10.1377/hlthaff.2015.0667. PMID: 26526243.

22. Bere E, Veierod MB, Bjelland M, et al. Free school fruit--sustained effect 1 year later. Health Educ Res. 2006 Apr;21(2):268-75. doi: 10.1093/her/cyh063. PMID: 16219630.

23. Bere E, Veierod MB, Klepp KI. The Norwegian School Fruit Programme: evaluating paid vs. no-cost subscriptions. Prev Med. 2005 Aug;41(2):463-70. doi: 10.1016/j.ypmed.2004.11.024. PMID: 15917042.

24. Sisnowski J, Street JM, Merlin T. Improving food environments and tackling obesity: A realist systematic review of the policy success of regulatory interventions targeting population nutrition. PLoS One. 2017;12(8):e0182581. doi: 10.1371/journal.pone.0182581. PMID: 28783757.

25. . Executive summary: Guidelines (2013) for the management of overweight and obesity in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and the Obesity Society published by the Obesity Society and American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Based on a systematic review from the The Obesity Expert Panel, 2013. Obesity (Silver Spring). 2014 Jul;22 Suppl 2:S5-39. doi: 10.1002/oby.20821. PMID: 24961825.

26. Strategies to Prevent Obesity and Other Chronic Diseases: The CDC Guide to Strategies to Increase the Consumption of Fruits and Vegetables Centers for Disease Control and Prevention. Department of Health and Human Services; 2011. https://www.cdc.gov/obesity/downloads/strategies-fruits-and-vegetables.pdf

27. Strategies to Prevent Obesity and Other Chronic Diseases: The CDC Guide to Strategies to Increase Physical Activity in the Community Centers for Disease Control and Prevention. Atlanta, GA: U.S. Department of Health and Human Services; 2011. https://www.cdc.gov/obesity/downloads/PA_2011_WEB.pdf

28. Mastellos N, Gunn LH, Felix LM, et al. Transtheoretical model stages of change for dietary and physical exercise modification in weight loss management for overweight and obese adults. Cochrane Database Syst Rev. 2014 Feb 05(2):CD008066. doi: 10.1002/14651858.CD008066.pub3. PMID: 24500864.

29. CHAMPS in the community. San Francisco, CA: University of California, San Francisco 2008. http://dne2.ucsf.edu/public/champs/community/action/index.html#sim. Accessed on August 14, 2017.

30. Orchard TJ, Temprosa M, Goldberg R, et al. The effect of metformin and intensive lifestyle intervention on the metabolic syndrome: the Diabetes Prevention Program randomized trial. Ann Intern Med. 2005 Apr 19;142(8):611-9. PMID: 15838067.

Page 79: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

66

31. National Diabetes Prevention Program. Atlanta, GA: Centers for Disease Control and Prevention; 2017. https://www.cdc.gov/diabetes/prevention/index.html. Accessed on August 14, 2017.

32. Obesity Treatment Options Setin Primary Care forUnderserved Populations: Pragmatic Clinical Trialsto Evaluate Real-World Comparative Effectiveness Patient-CenteredOutcomesResearchInstitute. Washington, D.C.: 2014. http://www.pcori.org/assets/2014/02/PCORI-PFA-2014-Spring-Obesity.pdf

33. About PCORnet. PCORnet 2017. http://www.pcornet.org/about-pcornet/. Accessed on August 14, 2017.

34. Sonneville KR, Long MW, Ward ZJ, et al. BMI and Healthcare Cost Impact of Eliminating Tax Subsidy for Advertising Unhealthy Food to Youth. Am J Prev Med. 2015 Jul;49(1):124-34. doi: 10.1016/j.amepre.2015.02.026. PMID: 26094233.

35. Economos CD, Hammond RA. Designing effective and sustainable multifaceted interventions for obesity prevention and healthy communities. Obesity (Silver Spring). 2017 Jul;25(7):1155-6. doi: 10.1002/oby.21893. PMID: 28653500.

36. Mayne SL, Auchincloss AH, Michael YL. Impact of policy and built environment changes on obesity-related outcomes: a systematic review of naturally occurring experiments. Obes Rev. 2015 May;16(5):362-75. doi: 10.1111/obr.12269. PMID: 25753170.

37. Wheeler BW, Gunnell D, Metcalfe C, et al. The population impact on incidence of suicide and non-fatal self harm of regulatory action against the use of selective serotonin reuptake inhibitors in under 18s in the United Kingdom: ecological study. BMJ. 2008 Mar 08;336(7643):542-5. doi: 10.1136/bmj.39462.375613.BE. PMID: 18276667.

38. Mackay DF, Irfan MO, Haw S, et al. Republished paper: Meta-analysis of the effect of comprehensive smoke-free legislation on acute coronary events. Postgrad Med J. 2011 Apr;87(1026):311-6. doi: 10.1136/pgmj.2010.199026rep. PMID: 21459780.

39. Pell JP, Haw S, Cobbe S, et al. Smoke-free legislation and hospitalizations for acute coronary syndrome. N Engl J Med. 2008 Jul 31;359(5):482-91. doi: 10.1056/NEJMsa0706740. PMID: 18669427.

40. Sims M, Maxwell R, Bauld L, et al. Short term impact of smoke-free legislation in England: retrospective analysis of hospital admissions for myocardial infarction. BMJ. 2010 Jun 08;340:c2161. doi: 10.1136/bmj.c2161. PMID: 20530563.

41. Medical Research Council. Using natural experiments to evaluate population health intervcentions. London, UK: Medical Research Council; 2010. https://www.mrc.ac.uk/documents/pdf/natural-experiments-to-evaluate-population-health-interventions/. Accessed on May 18, 2017.

42. Craig P, Cooper C, Gunnell D, et al. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Community Health. 2012 Dec;66(12):1182-6. doi: 10.1136/jech-2011-200375. PMID: 22577181.

43. Ludwig J, Sanbonmatsu L, Gennetian L, et al. Neighborhoods, obesity, and diabetes--a randomized social experiment. N Engl J Med. 2011 Oct 20;365(16):1509-19. doi: 10.1056/NEJMsa1103216. PMID: 22010917.

44. Stuart EA, Huskamp HA, Duckworth K, et al. Using propensity scores in difference-in-differences models to estimate the effects of a policy change. Health Serv Outcomes Res Methodol. 2014 Dec 01;14(4):166-82. doi: 10.1007/s10742-014-0123-z. PMID: 25530705.

45. Whitlock EP, Lopez SA, Chang S, et al. AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program. J Clin Epidemiol. 2010 May;63(5):491-501. doi: 10.1016/j.jclinepi.2009.03.008. PMID: 19540721.

Page 80: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

67

46. Mellor JM. Do cigarette taxes affect children's body mass index? The effect of household environment on health. Health Econ. 2011 Apr;20(4):417-31. doi: 10.1002/hec.1598. PMID: 21394814.

47. Department of Health and Human Services. The Surgeon General's Call To Action To Prevent and Decrease Overweight and Obesity. Rockville MD; 2001.

48. The Human Development Index 2016. One World Nations Online; 2016. http://www.nationsonline.org/oneworld/human_development.htm. Accessed on August 14, 2017.

49. Woo Baidal JA, Nelson CC, Perkins M, et al. Childhood obesity prevention in the women, infants, and children program: Outcomes of the MA-CORD study. Obesity (Silver Spring). 2017 Jul;25(7):1167-74. doi: 10.1002/oby.21865. PMID: 28653498.

50. Franckle RL, Falbe J, Gortmaker S, et al. Student obesity prevalence and behavioral outcomes for the massachusetts childhood obesity research demonstration project. Obesity (Silver Spring). 2017 Jul;25(7):1175-82. doi: 10.1002/oby.21867. PMID: 28653502.

51. Taveras EM, Perkins M, Anand S, et al. Clinical effectiveness of the massachusetts childhood obesity research demonstration initiative among low-income children. Obesity (Silver Spring). 2017 Jul;25(7):1159-66. doi: 10.1002/oby.21866. PMID: 28653504.

52. Helmerhorst HJ, Brage S, Warren J, et al. A systematic review of reliability and objective criterion-related validity of physical activity questionnaires. Int J Behav Nutr Phys Act. 2012 Aug 31;9:103. doi: 10.1186/1479-5868-9-103. PMID: 22938557.

53. Measureas Registry Resources. National Collaboration on Childhood Obesity Research; 2014. http://nccor.org/nccor-tools/measures/other-resources. Accessed on September 1, 2016.

54. England CY, Andrews RC, Jago R, et al. A systematic review of brief dietary questionnaires suitable for clinical use in the prevention and management of obesity, cardiovascular disease and type 2 diabetes. Eur J Clin Nutr. 2015 Sep;69(9):977-1003. doi: 10.1038/ejcn.2015.6. PMID: 25711954.

55. Cummins S, Flint E, Matthews SA. New neighborhood grocery store increased awareness of food access but did not alter dietary habits or obesity. Health Aff (Millwood). 2014 Feb;33(2):283-91. doi: 10.1377/hlthaff.2013.0512. PMID: 24493772.

56. MacDonald JM, Stokes RJ, Cohen DA, et al. The effect of light rail transit on body mass index and physical activity. Am J Prev Med. 2010 Aug;39(2):105-12. doi: 10.1016/j.amepre.2010.03.016. PMID: 20621257.

57. Kaushal N. Do food stamps cause obesity? Evidence from immigrant experience. J Health Econ. 2007 Sep 1;26(5):968-91. doi: 10.1016/j.jhealeco.2007.01.006. PMID: 17382418.

58. Chen R, Smyser M, Chan N, et al. Changes in awareness and use of calorie information after mandatory menu labeling in restaurants in King County, Washington. Am J Public Health. 2015 Mar;105(3):546-53. doi: 10.2105/ajph.2014.302262. PMID: 25602868.

59. Cheskin LJ, Frutchey R, McDermott AY, et al. Motivating systems-oriented research on environmental and policy changes for obesity prevention. Pediatr Obes. 2016 Apr 6doi: 10.1111/ijpo.12132. PMID: 27060703.

60. Sturm R, Hattori A. Diet and obesity in Los Angeles County 2007-2012: Is there a measurable effect of the 2008 "Fast-Food Ban"? Soc Sci Med. 2015 May;133:205-11. doi: 10.1016/j.socscimed.2015.03.004. PMID: 25779774.

61. Sekhobo JP, Edmunds LS, Dalenius K, et al. Neighborhood disparities in prevalence of childhood obesity among low-income children before and after implementation of New York City child care regulations. Prev Chronic Dis. 2014;11:E181. doi: 10.5888/pcd11.140152. PMID: 25321632.

Page 81: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

68

62. Schwartz AE, Leardo M, Aneja S, et al. Effect of a School-Based Water Intervention on Child Body Mass Index and Obesity. JAMA Pediatr. 2016 Mar;170(3):220-6. doi: 10.1001/jamapediatrics.2015.3778. PMID: 26784336.

63. McKenzie TL, Cohen DA, Sehgal A, et al. System for Observing Play and Recreation in Communities (SOPARC): Reliability and Feasibility Measures. J Phys Act Health. 2006 Feb;3 Suppl 1:S208-S22. PMID: 20976027.

64. Sugovic M, Turk P, Witt JK. Perceived distance and obesity: It's what you weigh, not what you think. Acta Psychol (Amst). 2016 Mar;165:1-8. doi: 10.1016/j.actpsy.2016.01.012. PMID: 26854404.

65. Brown BB, Smith KR, Tharp D, et al. A Complete Street Intervention for Walking to Transit, Nontransit Walking, and Bicycling: A Quasi-Experimental Demonstration of Increased Use. J Phys Act Health. 2016 Nov;13(11):1210-9. doi: 10.1123/jpah.2016-0066. PMID: 27334024.

66. Brown BB, Werner CM, Tribby CP, et al. Transit Use, Physical Activity, and Body Mass Index Changes: Objective Measures Associated With Complete Street Light-Rail Construction. Am J Public Health. 2015 Jul;105(7):1468-74. doi: 10.2105/ajph.2015.302561. PMID: 25973829.

67. Branas CC, Cheney RA, MacDonald JM, et al. A difference-in-differences analysis of health, safety, and greening vacant urban space. Am J Epidemiol. 2011 Dec 01;174(11):1296-306. doi: 10.1093/aje/kwr273. PMID: 22079788.

68. Cohen DA, Marsh T, Williamson S, et al. Impact and cost-effectiveness of family Fitness Zones: a natural experiment in urban public parks. Health Place. 2012 Jan;18(1):39-45. doi: 10.1016/j.healthplace.2011.09.008. PMID: 22243905.

69. Fitzhugh EC, Bassett DR, Jr., Evans MF. Urban trails and physical activity: a natural experiment. Am J Prev Med. 2010 Sep;39(3):259-62. doi: 10.1016/j.amepre.2010.05.010. PMID: 20709258.

70. Merom D, Bauman A, Vita P, et al. An environmental intervention to promote walking and cycling--the impact of a newly constructed Rail Trail in Western Sydney. Prev Med. 2003 Feb;36(2):235-42. PMID: 12590999.

71. Parker KM, Rice J, Gustat J, et al. Effect of bike lane infrastructure improvements on ridership in one New Orleans neighborhood. Ann Behav Med. 2013 Feb;45 Suppl 1:S101-7. doi: 10.1007/s12160-012-9440-z. PMID: 23334767.

72. Human Development Reports. United Nations Development Programme. http://hdr.undp.org/en Accessed on August 14, 2017.

73. Taber DR, Chriqui JF, Powell LM, et al. Banning all sugar-sweetened beverages in middle schools: reduction of in-school access and purchasing but not overall consumption. Arch Pediatr Adolesc Med. 2012 Mar;166(3):256-62. doi: 10.1001/archpediatrics.2011.200. PMID: 22064875.

74. Wright K, Norris K, Newman Giger J, et al. Improving healthy dietary behaviors, nutrition knowledge, and self-efficacy among underserved school children with parent and community involvement. Child Obes. 2012 Aug;8(4):347-56. doi: 10.1089/chi.2012.0045. PMID: 22867074.

75. Liu J, Kuo T, Jiang L, et al. Food and drink consumption among 1-5-year-old Los Angeles County children from households receiving dual SNAP and WIC v. only WIC benefits. Public Health Nutr. 2016 Sep 9:1-8. doi: 10.1017/s1368980016002329. PMID: 27609603.

76. Tester JM, Leung CW, Crawford PB. Revised WIC Food Package and Children's Diet Quality. Pediatrics. 2016 May;137(5)doi: 10.1542/peds.2015-3557. PMID: 27244804.

77. Slater S, Chriqui J, Chaloupka FJ, et al. Joint use policies: are they related to adolescent behavior? Prev Med. 2014 Dec;69 Suppl 1:S37-43. doi: 10.1016/j.ypmed.2014.08.032. PMID: 25199731.

Page 82: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

69

78. Kim J. Are physical education-related state policies and schools' physical education requirement related to children's physical activity and obesity? J Sch Health. 2012 Jun;82(6):268-76. doi: 10.1111/j.1746-1561.2012.00697.x. PMID: 22568462.

79. Sabia JJ, Nguyen TT, Rosenberg O. High School Physical Education Requirements and Youth Body Weight: New Evidence from the YRBS. Health Econ. 2016 Aug 30doi: 10.1002/hec.3399. PMID: 27576770.

80. Stephens RL, Xu Y, Lesesne CA, et al. Relationship between child care centers' compliance with physical activity regulations and children's physical activity, New York City, 2010. Prev Chronic Dis. 2014 Oct 16;11:E179. doi: 10.5888/pcd11.130432. PMID: 25321630.

81. Webb E, Laverty A, Mindell J, et al. Free Bus Travel and Physical Activity, Gait Speed, and Adiposity in the English Longitudinal Study of Ageing. Am J Public Health. 2016 Jan;106(1):136-42. doi: 10.2105/ajph.2015.302907. PMID: 26562118

82. Elbel B, Mijanovich T, Kiszko K, et al. The Introduction of a Supermarket via Tax-Credits in a Low-Income Area. Am J Health Promot. 2017 Jan;31(1):59-66. doi: 10.4278/ajhp.150217-QUAN-733. PMID: 26389982.

83. Panter J, Heinen E, Mackett R, et al. Impact of New Transport Infrastructure on Walking, Cycling, and Physical Activity. Am J Prev Med. 2016 Feb;50(2):e45-53. doi: 10.1016/j.amepre.2015.09.021. PMID: 26585051.

84. Burke RM, Meyer A, Kay C, et al. A holistic school-based intervention for improving health-related knowledge, body composition, and fitness in elementary school students: an evaluation of the HealthMPowers program. Int J Behav Nutr Phys Act. 2014 Jun 26;11:78. doi: 10.1186/1479-5868-11-78. PMID: 24969618.

85. Flego A, Herbert J, Waters E, et al. Jamie's Ministry of Food: quasi-experimental evaluation of immediate and sustained impacts of a cooking skills program in Australia. PLoS One. 2014;9(12):e114673. doi: 10.1371/journal.pone.0114673. PMID: 25514531.

86. Wilcox HC, Kharrazi H, Wilson RF, et al. Data Linkage Strategies to Advance Youth Suicide Prevention: A Systematic Review for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med. 2016 Oct 4doi: 10.7326/m16-1281. PMID: 27699389.

87. Kharrazi H, Weiner JP. IT-enabled Community Health Interventions: Challenges, Opportunities, and Future Directions. EGEMS (Wash DC). 2014;2(3):1117. doi: 10.13063/2327-9214.1117. PMID: 25848627.

88. Roalfe AK, Holder RL, Wilson S. Standardisation of rates using logistic regression: a comparison with the direct method. BMC Health Serv Res. 2008 Dec 29;8:275. doi: 10.1186/1472-6963-8-275. PMID: 19113996.

89. Gorely T, Morris JG, Musson H, et al. Physical activity and body composition outcomes of the GreatFun2Run intervention at 20 month follow-up. Int J Behav Nutr Phys Act. 2011 Jul 18;8:74. doi: 10.1186/1479-5868-8-74. PMID: 21767356.

90. Sigmund E, El Ansari W, Sigmundova D. Does school-based physical activity decrease overweight and obesity in children aged 6-9 years? A two-year non-randomized longitudinal intervention study in the Czech Republic. BMC Public Health. 2012 Jul 29;12:570. doi: 10.1186/1471-2458-12-570. PMID: 22892226.

91. EPHPP: Quality Assessment Tool for Quantitative Studies. http://www.ephpp.ca/PDF/Quality%20Assessment%20Tool_2010_2.pdf. Accessed on August 14, 2017.

92. Eagle TF, Gurm R, Smith CA, et al. A middle school intervention to improve health behaviors and reduce cardiac risk factors. Am J Med. 2013 Oct;126(10):903-8. doi: 10.1016/j.amjmed.2013.04.019. PMID: 23932159.

Page 83: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

70

93. Taber DR, Chriqui JF, Chaloupka FJ. Differences in nutrient intake associated with state laws regarding fat, sugar, and caloric content of competitive foods. Arch Pediatr Adolesc Med. 2012 May;166(5):452-8. doi: 10.1001/archpediatrics.2011.1839. PMID: 22566546.

94. Folta SC, Kuder JF, Goldberg JP, et al. Changes in diet and physical activity resulting from the Shape Up Somerville community intervention. BMC Pediatr. 2013 Oct 04;13:157. doi: 10.1186/1471-2431-13-157. PMID: 24093936.

95. Sterne JA, Hernan MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016 Oct 12;355:i4919. doi: 10.1136/bmj.i4919. PMID: 27733354.

96. EPHPP: Quality Assessment Tool for Quantitative Studies. Effective Public Health Practice Project; 2009. http://www.ephpp.ca/tools.html. Accessed on November 17, 2016.

97. Armijo-Olivo S, Stiles CR, Hagen NA, et al. Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research. J Eval Clin Pract. 2012 Feb;18(1):12-8. doi: 10.1111/j.1365-2753.2010.01516.x. PMID: 20698919.

98. Wells G. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa, Canada: Ottawa Hospital Research Institute; 2014. http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Accessed on May 18, 2017.

99. Basu S, Meghani A, Siddiqi A. Evaluating the Health Impact of Large-Scale Public Policy Changes: Classical and Novel Approaches. Annu Rev Public Health. 2017 Mar 20;38:351-70. doi: 10.1146/annurev-publhealth-031816-044208. PMID: 28384086.

100. Population Health Methods: Difference-in-Difference Estimation. Columbia University. https://www.mailman.columbia.edu/research/population-health-methods/difference-difference-estimation. Accessed on November 2, 2017.

101. Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol. 2000 Aug;29(4):722-9. PMID: 10922351.

102. Baiocchi M, Cheng J, Small DS. Instrumental variable methods for causal inference. Stat Med. 2014 Jun 15;33(13):2297-340. doi: 10.1002/sim.6128. PMID: 24599889.

103. Biglan A, Ary D, Wagenaar AC. The value of interrupted time-series experiments for community intervention research. Prev Sci. 2000 Mar;1(1):31-49. PMID: 11507793.

104. Interrupted time series (ITS) analyses. EPOC Resources for review authors. The Cochrane Collaboration; 2017. http://epoc.cochrane.org/resources/epoc-resources-review-authors. Accessed on November 2, 2017.

105. Linden A, Adams JL, Roberts N. Evaluating disease management programme effectiveness: an introduction to the regression discontinuity design. J Eval Clin Pract. 2006 Apr;12(2):124-31. doi: 10.1111/j.1365-2753.2005.00573.x. PMID: 16579820.

106. Imbens GW, Lemieux T. Regression discontinuity designs: A guide to practice. J Econom. 2008;142:615-35.

107. What Works Clearinghouse: Preview of Regression Discontinuity Design Standards. Institute of Education Sciences; 2015. https://ies.ed.gov/ncee/wwc/Document/258. Accessed on November 2, 2017.

108. Bauhoff S. The effect of school district nutrition policies on dietary intake and overweight: a synthetic control approach. Econ Hum Biol. 2014 Jan;12:45-55. doi: 10.1016/j.ehb.2013.06.001. PMID: 23891422.

Page 84: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

71

109. Ferdinand AO, Sen B, Rahurkar S, et al. The relationship between built environments and physical activity: a systematic review. Am J Public Health. 2012 Oct;102(10):e7-e13. doi: 10.2105/ajph.2012.300740. PMID: 22897546.

110. Sarink D, Peeters A, Freak-Poli R, et al. The impact of menu energy labelling across socioeconomic groups: A systematic review. Appetite. 2016 Apr 1;99:59-75. doi: 10.1016/j.appet.2015.12.022. PMID: 26723238.

111. Sacco J, Lillico HG, Chen E, et al. The influence of menu labelling on food choices among children and adolescents: a systematic review of the literature. Perspect Public Health. 2016 Jul 19doi: 10.1177/1757913916658498. PMID: 27436235.

112. Sobol-Goldberg S, Rabinowitz J, Gross R. School-based obesity prevention programs: a meta-analysis of randomized controlled trials. Obesity (Silver Spring). 2013 Dec;21(12):2422-8. doi: 10.1002/oby.20515. PMID: 23794226.

113. Wang Y, Wu Y, Wilson RF, et al. Childhood Obesity Prevention Programs: Comparative Effectiveness Review and Meta-Analysis. Rockville MD; 2013.

114. Gudzune K, Hutfless S, Maruthur N, et al. Strategies to prevent weight gain in workplace and college settings: a systematic review. Prev Med. 2013 Oct;57(4):268-77. doi: 10.1016/j.ypmed.2013.03.004. PMID: 23523689.

115. Gee KA. School-Based Body Mass Index Screening and Parental Notification in Late Adolescence: Evidence From Arkansas's Act 1220. J Adolesc Health. 2015 Sep;57(3):270-6. doi: 10.1016/j.jadohealth.2015.05.007. PMID: 26115907.

116. Nguyen BT, Powell LM. Supplemental nutrition assistance program participation and sugar-sweetened beverage consumption, overall and by source. Prev Med. 2015 Dec;81:82-6. doi: 10.1016/j.ypmed.2015.08.003. PMID: 26303370.

117. Gibson D. Long-term Food Stamp Program participation is positively related to simultaneous overweight in young daughters and obesity in mothers. J Nutr. 2006 Apr;136(4):1081-5. PMID: 16549483.

118. Molitor F, Sugerman S, Yu H, et al. Reach of Supplemental Nutrition Assistance Program-Education (SNAP-Ed) interventions and nutrition and physical activity-related outcomes, California, 2011-2012. Prev Chronic Dis. 2015 Mar 12;12:E33. doi: 10.5888/pcd12.140449. PMID: 25764139.

119. Leung CW, Blumenthal SJ, Hoffnagle EE, et al. Associations of food stamp participation with dietary quality and obesity in children. Pediatrics. 2013 Mar;131(3):463-72. doi: 10.1542/peds.2012-0889. PMID: 23439902.

120. Hilmers A, Chen TA, Dave JM, et al. Supplemental Nutrition Assistance Program participation did not help low income Hispanic women in Texas meet the dietary guidelines. Prev Med. 2014 May;62:44-8. doi: 10.1016/j.ypmed.2014.01.016. PMID: 24530319.

121. Wong SH, Huang WY, Cerin E, et al. Home and neighbourhood environment: association with children's physical activity and obesity-related dietary behaviour. Hong Kong Med J. 2016 Dec;22 Suppl 6(6):43-7. PMID: 27807318.

122. Thom G, Lean M. Is There an Optimal Diet for Weight Management and Metabolic Health? Gastroenterology. 2017 May;152(7):1739-51. doi: 10.1053/j.gastro.2017.01.056. PMID: 28214525.

123. Maddock J, Takeuchi L, Nett B, et al. Evaluation of a statewide program to reduce chronic disease: The Healthy Hawaii Initiative, 2000-2004. Eval Program Plann. 2006;29(3):293-300. doi: 10.1016/j.evalprogplan.2005.12.007. PMID: 2006-11996-009.

124. National Health and Nutrition Examination Survey. Atlanta, GA: Centers for Disease Control and Prevention; 2017. https://www.cdc.gov/nchs/nhanes/.

Page 85: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

72

125. Behavioral Risk Factor Surveillance System. Atlanta, GA: Centers for Disease Prevention; 2017. https://www.cdc.gov/brfss/. Accessed on May 18, 2017.

126. Fleurence RL, Curtis LH, Califf RM, et al. Launching PCORnet, a national patient-centered clinical research network. J Am Med Inform Assoc. 2014 Jul-Aug;21(4):578-82. doi: 10.1136/amiajnl-2014-002747. PMID: 24821743.

127. Ackermann RT, Kenrik Duru O, Albu JB, et al. Evaluating diabetes health policies using natural experiments: the natural experiments for translation in diabetes study. Am J Prev Med. 2015 Jun;48(6):747-54. doi: 10.1016/j.amepre.2014.12.010. PMID: 25998925.

128. Porta M. A Dictionary of Epidemiology. New York, NY: Oxford University Press; 2014. http://irea.ir/files/site1/pages/dictionary.pdf. Accessed on December 11, 2017.

129. Corriveau N, Eagle T, Jiang Q, et al. Sustained Benefit Over Four-Year Follow-Up of Michigan's Project Healthy Schools. Am J Public Health. 2015 Dec;105(12):e19-25. doi: 10.2105/ajph.2015.302835. PMID: 26469650.

130. Rosenbaum P, Rubin D. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41-55.

131. Lee BY, Bartsch SM, Mui Y, et al. A systems approach to obesity. Nutr Rev. 2017 Jan;75(suppl 1):94-106. doi: 10.1093/nutrit/nuw049. PMID: 28049754.

132. Johnston LM, Matteson CL, Finegood DT. Systems science and obesity policy: a novel framework for analyzing and rethinking population-level planning. Am J Public Health. 2014 Jul;104(7):1270-8. doi: 10.2105/ajph.2014.301884. PMID: 24832406.

133. Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI). Methodological standards and patient-centeredness in comparative effectiveness research: the PCORI perspective. JAMA. 2012 Apr 18;307(15):1636-40. doi: 10.1001/jama.2012.466. PMID: 22511692.

Page 86: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

A-1

Appendix A. Acronyms Table A1. List of acronyms

Acronym Definition

AHRQ Agency for Healthcare Research and Quality

APPLE Alberta Project Promoting Active Living and Healthy Eating in Schools

ATLAS Active Teen Leaders Avoiding Screen-time

AVHPSP Annapolis Valley Health Promoting School Project

BMI Body Mass Index

BRFSS Behavioral Risk Factor Surveillance System

CDC Centers for Disease Control and Prevention

CHALK Choosing Healthy & Active Lifestyles for Kids

CHL Children’s Healthy Living

CI Confidence interval

CLASS Classification of Laws Associated with School Students

CNRA Child Nutrition and WIC Reauthorization Act

CONSORT Consolidated Standards of Reporting Trials

CPPW Communities Putting Prevention to Work

CSHP Coordinated School Health Program

DC District of Columbia

DPP Diabetes Prevention Program

EB4K with Play Energy Balance for Kids with Play

EHR Electronic health records

EMR electronic medical records

EPHPP Effective Public Health Practice Project

HDI Human Development Index

HEAL-CHI Healthy Eating Active Living–Community Health Initiative (HEAL-CHI)136

Healthy ONES The Healthy Options for Nutrition Environments in Schools

HEROES Healthy, Energetic, Ready, Outstanding, Enthusiastic, Schools

HFH Healthy Foods Hawaii

HFHF Healthy Foods, Healthy Families

Page 87: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

A-2

HIPAA Health Insurance Portability and Accountability Act

HLCK Healthy Living Cambridge Kids

HSP Healthy Schools Program

HUD Department of Housing and Urban Development

ICAPS Intervention Centered on Adolescents’ Physical activity and Sedentary behavior

IDEFICS The Identification and prevention of Diet and lifestyle-induced health EFfects In Children and infantS

IOM Institute of Medicine

JAWS Jog and Walk Stars

KNF Kids N Fitness

KPS Kearney Public School

KQ Key Question

MOTMGC Men on the Move: Growing Communities

MRC UK Medical Research Council

MSBE Michigan State Board of Education

M-SPAN Middle-School Physical Activity and Nutrition

MTO Moving to Opportunity

NCCOR The National Collaborative on Childhood Obesity Research

NCHWTF North Carolina Health and Wellness Trust Fund

NEXT-D Natural Experiments for Translation in Diabetes

NHANES National Health and Nutrition Examination Survey

NIH National Institutes of Health

NPS National Park Service

NSCH National Survey of Children’s Health

Page 88: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

A-3

Acronym Definition

P2P Pathways to Prevention Program

PA Physical activity

PACE Partnership for an Active Community Environment

PCORI The Patient Centered Outcomes Research Institute

PCORNet The National Patient-Centered Clinical Research Network

PICOTS Populations, Interventions, Comparators, Outcomes, Timing, Setting

POD Preventing Obesity by Design

RCT Randomized Controlled Trial

ROBINS-I The Risk Of Bias In Non-randomized Studies – of Interventions

SNAP Supplemental Nutrition Assistance Program

SNPI School Nutrition Policy Initiative

SOPARC System for observing play and recreation in communities

SOPLAY System for observing play and leisure activity in youth

SPARK Sports, Play, and Active Recreation for Kids

SRDR Systematic Review Data Repository

SRTS Safe Routes to School

SSB Sugar sweetened beverages

TEP Technical expert panel

TFN Texas Fitness Now

UK United Kingdom

US United States

WIC Women, Infants, and Children

YRBSS Youth Risk Behavior Surveillance System

Page 89: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

B-1

Appendix B. Glossary of Key Terms

24-hour dietary recall: 24-hour dietary recall is aimed at capturing a comprehensive and detailed accounting of all foods, beverages, and in some cases, supplements, consumed on a given day. Traditionally, 24-hour recalls have been administered by an interviewer. Multiple-pass methods are used to improve accuracy and may be implemented using computerized systems.1

BMI z-score: Number of standard deviations away from the population mean body mass index (BMI); in other words, the degree to which an individual’s measurement deviates from what is expected for that individual.

Body mass index (BMI): An indirect measure of body fat, calculated as the ratio of a person’s body weight in kilograms to the square of a person’s height in meters. BMI (kg/m2) = weight (kilograms) ÷ height (meters)2 BMI (lb/in2) = weight (pounds) ÷ height (inches)2 × 703 In children and youth, BMI is interpreted using growth charts for age and gender and is referred to as BMI-for-age and sex, which is used to assess underweight, overweight, and obesity. According to the Centers for Disease Control and Prevention (CDC), a child with a BMI that is equal to or greater than the 95th percentile is considered to be obese. A child with a BMI that is equal to or between the 85th and 95th percentile is considered to be overweight.2

Calorie A calorie is defined as the amount of heat required to change the temperature of one gram of water from 14.5 degrees Celsius to 15.5 degrees Celsius. In this report, “calorie” is used synonymously with “kilocalorie,” the unit of measure for energy obtained from food and beverages.

Case control study: A retrospective study design where the investigators gather ‘cases’ of people who already have the outcome of interest and ‘controls’ who do not. Both groups are then questioned or their records examined about whether they received the intervention exposure of interest.3

Changes in the Built Environment: Built environments are the totality of places built or designed by humans, including buildings, grounds around buildings, layout of communities, transportation infrastructure, and parks and trails.4

Note: examples include supermarkets, farmer’s markets, as well as infrastructure

Page 90: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

B-2

Cohort (one group pre + post (before and after): The same group is pretested, given an intervention, and tested immediately after the intervention. The intervention group, by means of the pretest, act as their own control group.3

Cohort analytic (two group pre and post): An observational study design where groups are assembled according to whether or not exposure to the intervention has occurred. Exposure to the intervention is not under the control of the investigators. Study groups might be nonequivalent or not comparable on some feature that affects outcome.3

Competitive Foods and beverages offered at schools other than meals and foods snacks served through the federally reimbursed school lunch, breakfast, and after-school snack programs. Competitive foods includes food and beverages items sold through à la carte lines, snack bars, student stores, vending machines, and school fundraisers.

Controlled Clinical Trial (CCT): An experimental study design where the method of allocating study subjects to intervention or control groups is open to individuals responsible for recruiting subjects or providing the intervention. The method of allocation is transparent

before assignment, e.g. an open list of random numbers or allocation by date of birth, etc.3

Data source: Datasets, including both primary and secondary sources.

Data system: A data system involves the systematic collection of data, such as in a database, as well as the information technology infrastructure to maintain and operate the system.5

Electronic health record (EHR): A digital version of a patient’s paper chart. EHRs are real-time, patient-centered records that make information available instantly and securely to authorized users. While an EHR does contain the medical and treatment histories of patients, an EHR system is built to go beyond standard clinical data collected in a provider’s office and can be inclusive of a broader view of a patient’s care.6

Environment The external influences on the life of an individual or community.

Page 91: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

B-3

Food frequency questionnaires (FFQ): Food frequency questionnaires gather information about the frequency with which different foods and beverages are consumed over some period of time, often the last month or year. They may capture total diet or particular aspects of the diet. A questionnaire aimed at capturing total diet can be lengthy, requiring 30-60 minutes to complete.1

Food records/diaries: Food records or diaries (referred to as records subsequently, for simplicity) are intended to capture a detailed account of all foods, beverages, and possibly, supplements consumed on one or more days. Records are often kept for a period of one, three, or seven days. The distinction between recalls and records is that with a recall, the respondent reports (i.e., recalls, relying on memory) what was consumed yesterday (or over the past 24 hours) whereas with a record, the respondent keeps track of (i.e., records in real time) what he or she consumes.

Interrupted time series: A time series consists of multiple observations over time. Observations can be on the same units (e.g. individuals over time) or on different but similar units (e.g. student achievement scores for particular grade and school). Interrupted time series analysis requires knowing the specific point in the series when an intervention occurred.3

Messaging about nutrition and/or physical activity: Consistent with the 2012 IOM Report that described “Message Environments”, policies and programs were classified as having specific messages or information about nutrition and /or physical activity (e.g. posted flyers or informational campaigns), which included social marketing strategies.7

Natural experiment: Natural experiment refers to ways of evaluating policy, programmatic and environmental interventions using unplanned variation in exposure to assess the impact on health outcomes. The key features of these definitions are that: (1) the intervention (policy, program, environment change) is not undertaken for the purposes of research; and (2) the variation in exposure and outcomes is analyzed using methods that attempt to make causal inferences. Outside of a randomized controlled trial it is rare for variation in exposure to an intervention to be random, so special care is needed in the design, reporting and interpretation of evidence from natural experimental. Definition adapted from Craig, 2012.8

Obesity An excess amount of subcutaneous body fat in proportion to lean body mass. In adults, a BMI of 30 or greater is considered obese. In this report, obesity in children and adolescents refers to age- and sex-specific BMIs that are equal to or greater than the 95th percentile of the CDC BMI growth charts. In most children, these values are known to indicate elevated body fat and to reflect the comorbidities associated with excessive body fatness.

Page 92: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

B-4

Observation (physical activity): Direct observation is considered to be a gold standard method of physical activity assessment because behavior is directly observed. Observation typically involves the choice of a participant to observe (because it is not possible to observe all participants at the same time), when to watch (because it is not practical to try to observe continuously for extended periods), and how to record the behavior.9

Physical activity:

Moderate- On an absolute scale, moderate-intensity physical activity is intensity completed at 3.0 to 5.9 times the intensity of rest. On a scale physical activity that is relative to an individual’s personal capacity, moderate intensity physical activity is usually a 5 or 6 on a scale of 0 to 10.

Vigorous- On an absolute scale, vigorous-intensity physical activity is intensity completed at 6.0 or more times the intensity of rest. On a scale physical activity relative to an individual’s personal capacity, vigorous-intensity physical activity is usually a 7 or 8 on a scale of 0 to 10.

Policy: is broadly defined to include both formal public policies at local, state and federal levels of government, and organizational level policies, such as those implemented by large organizations, worksites or school districts. Examples include, but are not limited to, the development of supermarkets in underserved areas, calorie labeling requirements, taxes on foods and/or beverages, after-school and summer programs, modification of the built (or human-made) environments to encourage walking or cycling for transportation or leisure.10

Population-based: Individuals sampled are broadly representative of the targeted, general population, such as the neighborhood or children in a school system. In this review, obesity prevention and control policies and programs broadly target adults and/or children and the communities they live in.

Prevention With regard to obesity,

primary prevention represents avoiding the occurrence of obesity in a population

secondary prevention represents early detection of disease through screening with the purpose of limiting its occurrence

tertiary prevention involves preventing the sequelae of obesity in childhood and adulthood.

Program: Set of activities initiated by governmental or other organizational bodies to enhance obesity prevention and control. Examples might include programs implemented worksites,

Page 93: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

B-5

healthcare organizations, after-school or summer programs, or communities that can be expected to improve obesity related behaviors such as energy intake and activity level. 10

Randomized controlled trial (RCT): An experimental design where investigators randomly allocate eligible people to an intervention or control group. A rater should describe a study as an RCT if the randomization sequence allows each study participant to have the same chance of receiving each intervention and the investigators could not predict which intervention was next.3

Rural: Relating to the country or countryside. In this report populations were designated “rural” by study authors, or not defined. The systematic review study group did not define these populations.

Questionnaires (diet): Brief instruments that enable the collection of basic information about particular foods or beverages or other dietary behaviors. Screeners may query the frequency of intake of certain foods or beverages and thus may be thought of as short food frequency questionnaires, usually without questions regarding portion sizes.1

Questionnaires (Physical Activity): include various self-report surveys, such as physical activity questionnaires and diaries, that capture a participant’s perception and interpretation of physical activity behavior. These tools also can be defined as subjective measures because they rely on the person's ability to interpret and recall physical activity and are generally categorized by mode of administration: self-administered or interview (most are self-administered).9

Suburban: An outlying area adjacent to a city or town. In this report populations were designated “suburban” by study authors, or not defined. The systematic review study group did not define these populations.

Urban: Relating to or a characteristic of a city. In this report populations were designated “urban” by study authors, or not defined. The systematic review study group did not define these populations.

Wearable technology: A category of technology devices that can be worn by a consumer and often include tracking information related to health and fitness.11

Page 94: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

B-6

Appendix B References

1. Measures Registry User Guides: Overview of Individual Diet Measures. National Collaborative on Childhood Obesity Research; 2016. http://www.nccor.org/tools-mruserguides/individual-diet/overview-of-individual-diet-measures/#objective. Accessed on October 31, 2017.

2. About Child & Teen BMI. Atlants, GA: Centers for Disease Control and Prevention; 2015. https://www.cdc.gov/healthyweight/assessing/bmi/childrens_bmi/about_childrens_bmi.html. Accessed on August 11, 2017.

3. EPHPP: Quality Assessment Tool for Quantitative Studies Dictionary. Effective Public Health Practice Project. http://www.ephpp.ca/PDF/QADictionary_dec2009.pdf. Accessed on November 1, 2017.

4. Transportation Research Board and Institute of Medicine. Does the Built Environment Influence Physical Activity? Examining the evidence. Special Report 282. National Academies Press. Washington, DC: 2005. http://onlinepubs.trb.org/onlinepubs/sr/sr282.pdf

5. Wilcox H, Wissow L, Kharrazi H, et al. Data Linkage Strategies To Advance Youth Suicide Prevention. Evidence Report/Technology Assessment No. 222. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. 290-2012-00007-I.) AHRQ Publication No. 16-E001-EF. Agency for Healthcare Research and Quality. Rockville, MD: 2016. www.effectivehealthcare.ahrq.gov/reports/final.cfm.

6. What is an electronic health record (EHR)? Health IT; 2013. https://www.healthit.gov/providers-

professionals/faqs/what-electronic-health-record-ehr. Accessed on August 11, 2017.

7. Committee on Evaluating Progress of Obesity Prevention Effort, Food and Nutrition Board, Institute of Medicine. Evaluating Obesity Prevention Efforts: A Plan for Measuring Progress. Washington DC: 2013 by the National Academy of Sciences; 2013.

8. Craig P, Cooper C, Gunnell D, et al. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Community Health. 2012 Dec;66(12):1182-6. doi: 10.1136/jech-2011-200375. PMID: 22577181.

9. Overview of Physical Activity Assessment Tools. National Collaborative on Childhood Obesity Research; 2016. http://www.nccor.org/tools-mruserguides/individual-physical-activity/overview-of-physical-activity-assessment-tools/#summary. Accessed on November 1, 2017.

10. Department of Health and Human Services, Part 1. Overview Information. Department of Health and Human Services. http://grants.nih.gov/grants/guide/pa-files/PA-16-165.html. Accessed on October 24, 2016.

11. Beal V. Wearable Technology. IT Business Edge; 2017. http://www.webopedia.com/TERM/W/wearable_technology.html. Accessed on August 11, 2017.

Page 95: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

C-1

Appendix C. Detailed Search Strategies Table C1. PubMed search strategy

# Term/string

1 obesity[mh]

2 obese[tiab]

3 obesity[tiab]

4 overweight[tiab]

5 “over weight”[tiab]

6 BMI[tiab]

7 “body mass index”[tiab]

8 “Body mass index”[mh]

9 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8

10 "Policy"[Mesh]

11 policy[tiab]

12 policies[tiab]

13 law[tiab]

14 Purchasing[tiab]

15 Purchases[tiab]

16 “food labeling”[mh]

17 “calorie information”[tiab]

18 “calorie labeling”[tiab]

19 Environment[tiab]

20 10 OR 11 OR 12 OR 13 OR 14 OR 15 OR 16 OR 17 OR 18 OR 19

21 9 AND 20

Date limited: 2000 to present

Page 96: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

C-2

Table C2. CINAHL/PsycINFO/EconLit search strategy Results from 4 November 2016

Search ID# Search Terms Search Options

S48 S33 AND S46 Limiters - Published Date: 20000101-20161031; Exclude MEDLINE records; Publication Year: 2000-2016; Exclude Dissertations

Search modes - Boolean/Phrase

S47 S33 AND S46 Search modes - Boolean/Phrase

S46 S34 OR S35 OR S36 OR S37 OR S38 OR S39 OR S40 OR S41 OR S42 OR S43 OR S44 OR S45

Search modes - Boolean/Phrase

S45 TI environment OR AB environment Search modes - Boolean/Phrase

S44 TI "calorie labeling" OR AB "calorie labeling"

Search modes - Boolean/Phrase

S43 TI "caloric information" OR AB "caloric information"

Search modes - Boolean/Phrase

S42 TI "calorie information" OR AB "calorie information"

Search modes - Boolean/Phrase

S41 TI "food labeling" OR AB "food labeling"

Search modes - Boolean/Phrase

S40 TI purchases OR AB purchases Search modes - Boolean/Phrase

S39 TI purchasing OR AB purchasing Search modes - Boolean/Phrase

S38 TI law OR AB law Search modes - Boolean/Phrase

S37 TI policies OR AB policies Search modes - Boolean/Phrase

S36 TI policy OR AB policy Search modes - Boolean/Phrase

S35 (MH "Food Labeling+") Search modes - Boolean/Phrase

S34 (MH "Public Policy+") Search modes - Boolean/Phrase

S33 S25 OR S26 OR S27 OR S28 OR S29 OR S30 OR S31 OR S32

Search modes - Boolean/Phrase

Page 97: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

C-3

Search ID# Search Terms Search Options

S32 TI BMI OR AB BMI Search modes - Boolean/Phrase

S31 TI "body mass index" OR AB "body mass index"

Search modes - Boolean/Phrase

S30 TI "over weight" OR AB "over weight" Search modes - Boolean/Phrase

S29 TI overweight OR AB overweight Search modes - Boolean/Phrase

S28 TI obese OR AB obese Search modes - Boolean/Phrase

S27 TI obesity OR AB obesity Search modes - Boolean/Phrase

S26 (MH "body mass index+") Search modes - Boolean/Phrase

Page 98: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

C-4

Table C2. CINAHL/PsycINFO/EconLit search strategy (continued) Search

ID# Search Terms Search Options

S25 (MH "Obesity+") Search modes - Boolean/Phrase

S24 S9 AND S22 Limiters - Published Date: 20000101-20161031; Exclude MEDLINE records; Publication Year: 2000-2016; Exclude Dissertations

Search modes - Boolean/Phrase

S23 S9 AND S22 Search modes - Boolean/Phrase

S22 S10 OR S11 OR S12 OR S13 OR S14 OR S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21

Search modes - Boolean/Phrase

S21 TI environment OR AB environment Search modes - Boolean/Phrase

S20 TI "calorie labeling" OR AB "calorie labeling"

Search modes - Boolean/Phrase

S19 TI "caloric information" OR AB "caloric information"

Search modes - Boolean/Phrase

S18 TI "calorie information" OR AB "calorie information"

Search modes - Boolean/Phrase

S17 TI "food labeling" OR AB "food labeling"

Search modes - Boolean/Phrase

S16 TI purchases OR AB purchases Search modes - Boolean/Phrase

S15 TI purchasing OR AB purchasing Search modes - Boolean/Phrase

S14 TI law OR AB law Search modes - Boolean/Phrase

S13 TI policies OR AB policies Search modes - Boolean/Phrase

S12 TI policy OR AB policy Search modes - Boolean/Phrase

Page 99: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

C-5

Table C2. CINAHL/PsycINFO/EconLit search strategy (continued) Search

ID# Search Terms Search Options

S11 (MH "Food Labeling+") Search modes - Boolean/Phrase

S10 (MH "Public Policy+") Search modes - Boolean/Phrase

S9 S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8

Search modes - Boolean/Phrase

S8 TI BMI OR AB BMI Search modes - Boolean/Phrase

S7 TI "body mass index" OR AB "body mass index"

Search modes - Boolean/Phrase

S6 TI "over weight" OR AB "over weight" Search modes - Boolean/Phrase

S5 TI overweight OR AB overweight Search modes - Boolean/Phrase

S4 TI obese OR AB obese Search modes - Boolean/Phrase

S3 TI obesity OR AB obesity Search modes - Boolean/Phrase

S2 (MH "body mass index+") Search modes - Boolean/Phrase

S1 (MH "Obesity+") Search modes - Boolean/Phrase

Page 100: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

C-6

Table C3. Supplemental search strategy (PubMed) # Term/string

1 Exercise[mh]

2 exercise[tiab]

3 “physical activity”[tiab]

4 “motor activity”[mh]

5 “motor activity”[tiab]

6 “energy metabolism”[mh]

7 “energy expenditure”[tiab]

8 diet[mh]

9 diet[tiab]

10 dietary[tiab]

11 “energy intake”[mh]

12 “caloric intake”[tiab]

13 calories[tiab]

14 fruit[tiab]

15 vegetable[tiab]

16 “sugar sweetened beverage”[tiab]

17 SSB[tiab]

18 “fast foods”[mh]

19 “fast food”[tiab]

20 “fast foods”[tiab]

21 intake[tiab]

22 consumption[tiab]

23 13 OR 14 OR 15 OR 16 OR 17 OR 18 OR 19 OR 20

24 21 OR 22

25 23 AND 24

26 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 OR 11 OR 12

27 26 OR 25

28 "Transportation"[Mesh]

Page 101: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

C-7

# Term/string

29 transportation[tiab]

30 transit[tiab]

31 recreation[Mesh]

32 recreation[tiab]

33 "Environment Design"[Mesh]

34 "Walking"[Mesh]

35 walk[tiab]

36 walking[tiab]

37 "Bicycling"[Mesh]

38 bike[tiab]

39 bicycling[tiab]

40 biking

41 greening[tiab]

42 28 OR 29 OR 30 OR 31 OR 32 OR 33 OR 34 OR 35 OR 36 OR 37 OR 38 OR 39 OR 40 OR 41

43 27 AND 42

Limit to systematic reviews AND limit to 2013 through 2017

Page 102: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-1

Appendix D. Abstraction Forms Figure D1. Abstract screening form

Page 103: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-2

Figure D2. Article screening form

Page 104: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-3

Figure D3. Data source abstraction form

Page 105: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-4

Page 106: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-5

Figure D4. Design specific abstraction form

Page 107: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-6

Page 108: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-7

Page 109: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-8

Page 110: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-9

Page 111: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-10

Page 112: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-11

Page 113: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-12

Figure D5. Study and participant characteristics abstraction form

Page 114: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-13

Page 115: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-14

Page 116: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-15

Page 117: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-16

Page 118: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-17

Figure D6. Intervention abstraction form

Page 119: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-18

Page 120: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-19

Page 121: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-20

Figure D7. General outcomes abstraction form

Page 122: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-21

Page 123: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-22

Page 124: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-23

Page 125: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-24

Page 126: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-25

Page 127: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-26

Page 128: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-27

Page 129: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-28

Page 130: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-29

Page 131: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-30

Page 132: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-31

Figure D8. Future research needs abstraction form

Page 133: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-32

Figure D9. Risk of bias abstraction form

Page 134: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-33

Page 135: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-34

Page 136: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-35

Page 137: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-36

Page 138: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

D-37

Page 139: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

E-1

Appendix E. Data System Classification/Coding Schema

Data System Specs and Meta Info Data System Definition Exists [Y/N] Accessible [Y/N] Sharable [Y/N] Outcome [Y/N] Qualifies [Y/N] US vs Non-US Non-US [Y/N] Country [TXT] Data System Type Primary [Y/N] Non-sharable [Y/N] Sharable [Y/N] Secondary [Y/N] Data System Meta Database Name [TXT] Database Abbreviation [TXT] Data Custodian / Owner Name [TXT] Data Custodian Abbreviation [TXT] Web Links (URL) Generic/Dedicated Website [URL] Data Dictionary [URL] Download (e.g., public use file) [URL] Parent System (e.g., national system) [URL] Additional Links #1 [URL] Additional Links #2 [URL] Additional Links #3 [URL] Data System Information Availability Dedicated Web Page [Y/N] Summary of Data System [Y/N] (e.g., description and purpose) Some Detailed Information [Y/N] (e.g., data elements) Very Detailed Information [Y/N] (e.g., data quality reports) Data Accessible (free or for a fee) [Y/N] Publicly Available [Y/N] (e.g., public use file with download link) Requires Automated Registration [Y/N] (e.g., signing up) Requires Manual Confirmation [Y/N] (e.g., email request)

Page 140: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

E-2

Data Dictionary (any level) [Y/N] Formal Data Dictionary [Y/N] (e.g., XML, PDF) Informal Data Dictionary [Y/N] (e.g., survey questions) Proxy Data Dictionary [Y/N] (e.g., summary reports) Data Use and Functions Purpose [TXT] Research [Y/N] Clinical [Y/N] Administrative [Y/N] Pub Health [Y/N] Other [TXT] Geographic Coverage Countries Country [TXT] U.S.-based [Y/N] Other (e.g., multi-country) [TXT] U.S. Coverage Setting [TXT] National [Y/N] Regional (e.g., east coast) [Y/N] State-level [Y/N] Area (smaller than state) [Y/N] County or City-level [Y/N] Zip code or Census Tract [Y/N] Smaller than Census Tract (e.g., Block Group) [Y/N] Other (e.g., territory) [Y/N] U.S. States Alabama (AL) [Y/N] Alaska (AK) [Y/N] Arizona (AZ) [Y/N] Arkansas (AR) [Y/N] California (CA) [Y/N] Colorado (CO) [Y/N] Connecticut (CT) [Y/N] Delaware (DE) [Y/N] Florida (FL) [Y/N] Georgia (GA) [Y/N] Hawaii (HI) [Y/N] Idaho (ID) [Y/N] Illinois (IL) [Y/N] Indiana (IN) [Y/N] Iowa (IA) [Y/N] Kansas (KS) [Y/N]

Page 141: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

E-3

Kentucky (KY) [Y/N] Louisiana (LA) [Y/N] Maine (ME) [Y/N] Maryland (MD) [Y/N] Massachusetts (MA) [Y/N] Michigan (MI) [Y/N] Minnesota (MN) [Y/N] Mississippi (MS) [Y/N] Missouri (MO) [Y/N] Montana (MT) [Y/N] Nebraska (NE) [Y/N] Nevada (NV) [Y/N] New Hampshire (NH) [Y/N] New Jersey (NJ) [Y/N] New Mexico (NM) [Y/N] New York (NY) [Y/N] North Carolina (NC) [Y/N] North Dakota (ND) [Y/N] Ohio (OH) [Y/N] Oklahoma (OK) [Y/N] Oregon (OR) [Y/N] Pennsylvania (PA) [Y/N] Rhode Island (RI) [Y/N] South Carolina (SC) [Y/N] South Dakota (SD) [Y/N] Tennessee (TN) [Y/N] Texas (TX) [Y/N] Utah (UT) [Y/N] Vermont (VT) [Y/N] Virginia (VA) [Y/N] Washington (WA) [Y/N] West Virginia (WV) [Y/N] Wisconsin (WI) [Y/N] Wyoming (WY) [Y/N] Other (e.g., islands, territories) [Y/N] Demographic Coverage Population Denominator General Population [Y/N] Community level [Y/N] Schools [Y/N] Programs [Y/N] Households [Y/N] Other [Y/N] Age Coverage Age Range (text/numerical) [Y/N] Infant [Y/N]

Page 142: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

E-4

Preschool [Y/N] Elementary School [Y/N] Middle School [Y/N] High School [Y/N] Teen/Adolescent/Youth (not bound to school age) [Y/N] Adult [Y/N] Gender Male [Y/N] Female [Y/N] Other Demographics Socio-economic Status [Y/N] Ethnicity [Y/N] Race [Y/N] Data Granularity Aggregation Unit Individual [Y/N] Aggregated [Y/N] Demographic [Y/N] Geographic [Y/N] Entity Payer [Y/N] Provider [Y/N] Employer [Y/N] Educational Institute (e.g., school) [Y/N] Other [TXT] Outcomes/Variables of Interest Primary Outcomes [Y/N] Weight [Y/N] Height [Y/N] BMI [Y/N] Diet [Y/N] Physical Activity [Y/N] Secondary [Intermediate] Outcome [Y/N] Policy [Y/N] Program [Y/N] Build Environment [Y/N] Other Geographical Information [Y/N] Data System Scalability Data Linkage Linked [Y/N] Linked Data (i.e., external DB names) [TXT] Linked Method [TXT] Data Collection Method Collection Method [TXT] Survey (self-report) [Y/N] EHR/EMR (any medical record) [Y/N]

Page 143: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

E-5

Panel / Longitudinal (cohort) [Y/N] Cross-sectional [Y/N] Other (objective measure) [Y/N] Data Updates Ongoing [Y/N] Stopped [Y/N] Year Started [YYYY] Year Stopped [YYYY] Data Governance Data Access Unrestricted [Y/N] Public Use File Link [URL] Registration Required [Y/N] Data Commodity Free [Y/N] Fee [Y/N] Other (e.g., special licensing structure) [Y/N] Data Privacy (HIPAA Status) Compliant Limited Protected

Page 144: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

F-1

Appendix F. Approach to Identifying Methodological/Analytic Advances That Would Help

Strengthen Efforts To Estimate the Effect of Programs, Policies, or Built Environment Changes on

Obesity Prevention and Control Advancing Research Methods for Evaluation of Natural Experiments in Obesity Prevention and Control

Thank you for agreeing to be an advisor for the Johns Hopkins University, Evidence-base Practice Center project, “Advancing Research Methods for Evaluation of Natural Experiments in Obesity Prevention and Control”. Part of the project involves identifying methodological and analytic advances that would help to strengthen efforts to estimate the effect of programs, policies, or built environment changes on obesity prevention and control (e.g. data system features, approaches to linking data, and analytic or study design approaches to reduce bias) We would appreciate your input on this portion of the project. For each question listed below, our team has identified potential methodological and analytic advances that could help strengthen efforts to determine the effect of programs, policies, or built environment changes on obesity prevention and control. We ask that you identify the methodological or analytic advances you think would advance the field of research in obesity prevention and control, especially evaluation of natural experiments in which the intervention is not controlled by researchers. After reviewing each list, please check all items that would help to advance the field, and then add anything else you believe would help to advance the field.

Page 145: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

F-2

Key Question 1: What population-based data sources have been used in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes?

What features or types of data sources could help researchers advance methods for conducting studies, especially natural experiments, of how programs, policies or built environment changes affect obesity prevention and control?

Check all that apply.

▢ A process that encourages and facilitates dissemination of publicly available information about the existence and location of data sources that include information about obesity-related outcomes.

▢ A process that encourages and facilitates dissemination of publicly available information about the existence and location of data sources that include information about programs, policies, or built environment changes that could affect obesity prevention and control

▢ Consistent use of standard terminology in data sources that include obesity-related information, including such concepts as: classification of interventions intended to contribute to obesity prevention and control (e.g., governmental program or policy, non-governmental program or policy, or built environment change); and units of measurement of obesity-related outcomes

▢ Consistent use of data dictionaries and codebooks that define all elements, and that are are readily accessible and searchable

▢ Consistent use of a standard format for describing how data were collected and validated (including data quality control processes)

▢ Explanation of how each data source was intended to be used

▢ Description of the timing of interventions in data sources containing information about programs, policies, or built environment changes that could affect obesity prevention and control

▢ Inclusion of BMI and other health behavior data in data sources covering small geographic areas.

Please add any additional features or types of data sources that could help researcher advance methods for conducting studies on obesity prevention and control.

Page 146: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

F-3

Key Question 2. What methods have been used to link different population-based data sources?

What methodological advances would help to facilitate better or more frequent linkage of population-based data sources for studies, especially natural experiments, in obesity prevention and control?

Check all that apply.

▢ Consistent use of standard procedures for linking health care data from different sources

▢ Development and use of new methods for linking data on social determinants of health with health care data

▢ Development of reporting standards by publishers for the description of how data sources are linked when studies report more than one data source in manuscripts

▢ Adoption of standard policies and procedures for exchanging/sharing data

Please add any additional methodological advances that would help facilitate better or more frequent linkage of population-based data sources.

Page 147: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

F-4

Key Question 3. What obesity measures, dietary physical behaviors, and other outcomes have been assessed in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control?

What methodological advances (e.g. measures and data collection procedures) would help to strengthen the assessment of obesity-related outcomes, including dietary and physical activity behaviors, in studies, especially natural experiments, of how programs, policies or built environment changes affect obesity prevention and control?

Check all that apply.

▢ Standardized measures and methods for obesity-related outcomes across populations and studies to better facilitate comparisons.

▢ Improving data collection measures for park-based and transportation-based studies to assess changes in individual and population physical activity beyond observing park or transportation use.

▢ Assessment of the validity of intermediate outcomes such as studies of purchasing behavior correlated to behavioral and health outcomes.

▢ Longer term surveillance of measures to measure the impact of an intervention on changes in obesity measures over time

▢ Inclusion of BMI and other health behavior data in data sources covering small geographic areas.

▢ Establishing standards for the collection of obesity-related outcomes in adults and children.

▢ Consistent use of validated measures of obesity-related outcomes across studies.

▢ Consistent use of terminology in describing measures of obesity-related outcomes used in surveys or observational studies

Please add any additional methodological advances that would help to strengthen the assessment of obesity-related outcomes.

Page 148: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

F-5

Key Question 4. Which experimental and non-experimental methods have been used in studies of how programs, policies or built environment changes affect or are associated with obesity prevention and control outcomes?

Key Question 5. What are the risks of bias in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes?

What study design and analytic approaches would help to strengthen methods and reduce the risk of bias encountered in studies, especially natural experiments, of how programs, policies or built environment changes affect obesity prevention and control?

Check all that apply.

▢ Consistent use of standards for terms and reporting in studies of obesity prevention and control using natural experiment designs, including: study design, testing assumptions, sensitivity analyses, observed (and adjusted for) confounders, and unobserved confounders

▢ Design-specific reporting standards for observational study methods used in natural experiments related to obesity prevention and control

▢ Greater use of time-series designs instead of simple pre-post comparisons in studies of obesity prevention and control

▢ Greater use of stronger natural experiment designs such as instrumental variables and regression discontinuity

▢ Development and use of validated instruments for measuring the “exposures” in studies of how programs, policies, or built environment changes affect obesity prevention and control

▢ Development and use of a specific practical and validated tool for assessing the risk of bias in observational studies of how programs, policies, or built environment changes affect obesity prevention and control

Please add any additional study design and analytic approaches that would help strengthen methods and reduce the risk of bias encountered in studies on obesity prevention and control.

Page 149: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-1

Appendix G. Included Studies 1. Alaimo K, Oleksyk SC, Drzal NB, et

al. Effects of changes in lunch-time competitive foods, nutrition practices, and nutrition policies on low-income middle-school children's diets. Child Obes. 2013 Dec;9(6):509-23. doi: 10.1089/chi.2013.0052. PMID: 24215386.

2. Anderson JV, Bybee DI, Brown RM,

et al. 5 a day fruit and vegetable intervention improves consumption in a low income population. J Am Diet Assoc. 2001 Feb;101(2):195-202. doi: 10.1016/s0002-8223(01)00052-9. PMID: 11271692.

3. Anderson LM, Aycock KE, Mihalic

CA, et al. Geographic differences in physical education and adolescent BMI: have legal mandates made a difference? J Sch Nurs. 2013 Feb;29(1):52-60. doi: 10.1177/1059840512453602. PMID: 22815346.

4. Angelopoulos PD, Milionis HJ,

Grammatikaki E, et al. Changes in BMI and blood pressure after a school based intervention: the CHILDREN study. Eur J Public Health. 2009 Jun;19(3):319-25. doi: 10.1093/eurpub/ckp004. PMID: 19208697.

5. Anthamatten P, Brink L, Lampe S, et

al. An assessment of schoolyard renovation strategies to encourage children's physical activity. Int J Behav Nutr Phys Act. 2011 Apr 09;8:27. doi: 10.1186/1479-5868-8-27. PMID: 21477325.

6. Ashfield-Watt PA, Welch AA, Godward S, et al. Effect of a pilot community intervention on fruit and vegetable intakes: use of FACET (Five-a-day Community Evaluation Tool). Public Health Nutr. 2007 Jul;10(7):671-80. doi: 10.1017/s1368980007382517. PMID: 17381948.

7. Ask AS, Hernes S, Aarek I, et al.

Serving of free school lunch to secondary-school pupils - a pilot study with health implications. Public Health Nutr. 2010 Feb;13(2):238-44. doi: 10.1017/s1368980009990772. PMID: 19650962.

8. Audrey S, Procter S, Cooper A, et al.

Employer schemes to encourage walking to work: feasibility study incorporating an exploratory randomised controlled trial. Public Health Res. 2015 Mar;3(4)doi: 10.3310/phr03040. PMID: 25763450.

9. Ayala GX, Baquero B, Laraia BA, et

al. Efficacy of a store-based environmental change intervention compared with a delayed treatment control condition on store customers' intake of fruits and vegetables. Public Health Nutr. 2013 Nov;16(11):1953-60. doi: 10.1017/s1368980013000955. PMID: 23561842.

10. Azevedo LB, Burges Watson D,

Haighton C, et al. The effect of dance mat exergaming systems on physical activity and health-related outcomes in secondary schools:

Page 150: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-2

results from a natural experiment. BMC Public Health. 2014 Sep 12;14:951. doi: 10.1186/1471-2458-14-951. PMID: 25217144.

11. Backman D, Gonzaga G, Sugerman

S, et al. Effect of fresh fruit availability at worksites on the fruit and vegetable consumption of low-wage employees. J Nutr Educ Behav. 2011 Jul-Aug;43(4 Suppl 2):S113-21. doi: 10.1016/j.jneb.2011.04.003. PMID: 21683280.

12. Baker EA, Barnidge EK, Schootman

M, et al. Adaptation of a Modified DASH Diet to a Rural African American Community Setting. Am J Prev Med. 2016 Sep 12doi: 10.1016/j.amepre.2016.07.014. PMID: 27633485.

13. Barnidge EK, Hipp PR, Estlund A, et

al. Association between community garden participation and fruit and vegetable consumption in rural Missouri. Int J Behav Nutr Phys Act. 2013 Nov 19;10:128. doi: 10.1186/1479-5868-10-128. PMID: 24252563.

14. Barroso CS, Kelder SH, Springer

AE, et al. Senate Bill 42: implementation and impact on physical activity in middle schools. J Adolesc Health. 2009 Sep;45(3 Suppl):S82-90. doi: 10.1016/j.jadohealth.2009.06.017. PMID: 19699442.

15. Bastian KA, Maximova K,

McGavock J, et al. Does School-Based Health Promotion Affect Physical Activity on Weekends? And, Does It Reach Those Students Most in Need of Health Promotion?

PLoS One. 2015;10(10):e0137987. doi: 10.1371/journal.pone.0137987. PMID: 26488168.

16. Bauhoff S. The effect of school

district nutrition policies on dietary intake and overweight: a synthetic control approach. Econ Hum Biol. 2014 Jan;12:45-55. doi: 10.1016/j.ehb.2013.06.001. PMID: 23891422.

17. Bauman A, McLean G, Hurdle D, et

al. Evaluation of the national 'Push Play' campaign in New Zealand--creating population awareness of physical activity. N Z Med J. 2003 Aug 08;116(1179):U535. PMID: 14513082.

18. Benjamin Neelon SE, Namenek

Brouwer RJ, Ostbye T, et al. A community-based intervention increases physical activity and reduces obesity in school-age children in North Carolina. Child Obes. 2015 Jun;11(3):297-303. doi: 10.1089/chi.2014.0130. PMID: 25938983.

19. Bere E, Hilsen M, Klepp KI. Effect

of the nationwide free school fruit scheme in Norway. Br J Nutr. 2010 Aug;104(4):589-94. doi: 10.1017/s0007114510000814. PMID: 20350345.

20. Bere E, Veierod MB, Bjelland M, et

al. Free school fruit--sustained effect 1 year later. Health Educ Res. 2006 Apr;21(2):268-75. doi: 10.1093/her/cyh063. PMID: 16219630.

21. Bere E, Veierod MB, Bjelland M, et

al. Outcome and process evaluation

Page 151: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-3

of a Norwegian school-randomized fruit and vegetable intervention: Fruits and Vegetables Make the Marks (FVMM). Health Educ Res. 2006 Apr;21(2):258-67. doi: 10.1093/her/cyh062. PMID: 16219631.

22. Bere E, Veierod MB, Klepp KI. The

Norwegian School Fruit Programme: evaluating paid vs. no-cost subscriptions. Prev Med. 2005 Aug;41(2):463-70. doi: 10.1016/j.ypmed.2004.11.024. PMID: 15917042.

23. Bere E, Veierod MB, Skare O, et al.

Free School Fruit--sustained effect three years later. Int J Behav Nutr Phys Act. 2007 Feb 19;4:5. doi: 10.1186/1479-5868-4-5. PMID: 17309800.

24. Beresford SA, Thompson B, Bishop

S, et al. Long-term fruit and vegetable change in worksites: Seattle 5 a Day follow-up. Am J Health Behav. 2010 Nov-Dec;34(6):707-20. PMID: 20604696.

25. Berger-Jenkins E, Rausch J, Okah E,

et al. Evaluation of a Coordinated School-Based Obesity Prevention Program in a Hispanic Community: Choosing Healthy and Active Lifestyles for Kids/Healthy Schools Healthy Families. American Journal of Health Education. 2014;45(5):261-70. doi: 10.1080/19325037.2014.932724. PMID: 103883129. Language: English. Entry Date: 20140909. Revision Date: 20150820. Publication Type: Journal Article.

26. Blake H, Zhou D, Batt ME. Five-year workplace wellness intervention in the NHS. Perspect Public Health. 2013 Sep;133(5):262-71. doi: 10.1177/1757913913489611. PMID: 23771680.

27. Blum JE, Davee AM, Beaudoin CM,

et al. Reduced availability of sugar-sweetened beverages and diet soda has a limited impact on beverage consumption patterns in Maine high school youth. J Nutr Educ Behav. 2008 Nov-Dec;40(6):341-7. doi: 10.1016/j.jneb.2007.12.004. PMID: 18984489.

28. Bolton KA, Kremer P, Gibbs L, et al.

The outcomes of health-promoting communities: being active eating well initiative-a community-based obesity prevention intervention in Victoria, Australia. Int J Obes (Lond). 2017 Apr 25doi: 10.1038/ijo.2017.73. PMID: 28321132.

29. Bonsergent E, Agrinier N, Thilly N,

et al. Overweight and obesity prevention for adolescents: a cluster randomized controlled trial in a school setting. Am J Prev Med. 2013 Jan;44(1):30-9. doi: 10.1016/j.amepre.2012.09.055. PMID: 23253647.

30. Bonvin A, Barral J, Kakebeeke TH,

et al. Effect of a governmentally-led physical activity program on motor skills in young children attending child care centers: a cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2013 Jul 08;10:90. doi: 10.1186/1479-5868-10-90. PMID: 23835207.

Page 152: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-4

31. Bowling AB, Moretti M, Ringelheim K, et al. Healthy Foods, Healthy Families: combining incentives and exposure interventions at urban farmers' markets to improve nutrition among recipients of US federal food assistance. Health Promot Perspect. 2016;6(1):10-6. doi: 10.15171/hpp.2016.02. PMID: 27123431.

32. Branas CC, Cheney RA, MacDonald

JM, et al. A difference-in-differences analysis of health, safety, and greening vacant urban space. Am J Epidemiol. 2011 Dec 01;174(11):1296-306. doi: 10.1093/aje/kwr273. PMID: 22079788.

33. Brown AL, Khattak AJ, Rodriguez

DA. Neighbourhood Types, Travel and Body Mass: A Study of New Urbanist and Suburban Neighbourhoods in the US. Urban Stud. 2008;45(4):963-88.

34. Brown BB, Smith KR, Tharp D, et

al. A Complete Street Intervention for Walking to Transit, Nontransit Walking, and Bicycling: A Quasi-Experimental Demonstration of Increased Use. J Phys Act Health. 2016 Nov;13(11):1210-9. doi: 10.1123/jpah.2016-0066. PMID: 27334024.

35. Brown BB, Werner CM, Smith KR,

et al. Environmental, behavioral, and psychological predictors of transit ridership: Evidence from a community intervention. J Environ Psychol. 2016 Jun;46:188-96. doi: 10.1016/j.jenvp.2016.04.010. PMID: 27672237.

36. Brown BB, Werner CM, Tribby CP, et al. Transit Use, Physical Activity, and Body Mass Index Changes: Objective Measures Associated With Complete Street Light-Rail Construction. Am J Public Health. 2015 Jul;105(7):1468-74. doi: 10.2105/ajph.2015.302561. PMID: 25973829.

37. Brownson RC, Baker EA, Boyd RL,

et al. A community-based approach to promoting walking in rural areas. Am J Prev Med. 2004 Jul;27(1):28-34. doi: 10.1016/j.amepre.2004.03.015. PMID: 15212772.

38. Brownson RC, Hagood L, Lovegreen

SL, et al. A multilevel ecological approach to promoting walking in rural communities. Prev Med. 2005 Nov-Dec;41(5-6):837-42. doi: 10.1016/j.ypmed.2005.09.004. PMID: 16256183.

39. Brusseau TA, Hannon J, Burns R.

The Effect of a Comprehensive School Physical Activity Program on Physical Activity and Health-Related Fitness in Children From Low-Income Families. J Phys Act Health. 2016 Aug;13(8):888-94. doi: 10.1123/jpah.2016-0028. PMID: 27144329.

40. Burke RM, Meyer A, Kay C, et al. A

holistic school-based intervention for improving health-related knowledge, body composition, and fitness in elementary school students: an evaluation of the HealthMPowers program. Int J Behav Nutr Phys Act. 2014 Jun 26;11:78. doi: 10.1186/1479-5868-11-78. PMID: 24969618.

Page 153: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-5

41. Buscail C, Menai M, Salanave B, et

al. Promoting physical activity in a low-income neighborhood of the Paris suburb of Saint-Denis: effects of a community-based intervention to increase physical activity. BMC Public Health. 2016 Jul 29;16:667. doi: 10.1186/s12889-016-3360-y. PMID: 27473296.

42. Busch V, De Leeuw JR, Zuithoff

NP, et al. A Controlled Health Promoting School Study in the Netherlands: Effects After 1 and 2 Years of Intervention. Health Promot Pract. 2015 Jul;16(4):592-600. doi: 10.1177/1524839914566272. PMID: 25566994.

43. Caballero B, Clay T, Davis SM, et

al. Pathways: a school-based, randomized controlled trial for the prevention of obesity in American Indian schoolchildren. Am J Clin Nutr. 2003 Nov;78(5):1030-8. PMID: 14594792.

44. Caldwell EM, Miller Kobayashi M,

DuBow WM, et al. Perceived access to fruits and vegetables associated with increased consumption. Public Health Nutr. 2009 Oct;12(10):1743-50. doi: 10.1017/s1368980008004308. PMID: 19105861.

45. Calise TV, Heeren T, DeJong W, et

al. Do neighborhoods make people active, or do people make active neighborhoods? Evidence from a planned community in Austin, Texas. Prev Chronic Dis. 2013 Jun 20;10:E102. doi: 10.5888/pcd10.120119. PMID: 23786909.

46. Camacho-Rivera M, Rosenbaum E,

Yama C, et al. Low-Income Housing Rental Assistance, Perceptions of Neighborhood Food Environment, and Dietary Patterns among Latino Adults: the AHOME Study. J Racial Ethn Health Disparities. 2017 Jun;4(3):346-53. doi: 10.1007/s40615-016-0234-z. PMID: 27129854.

47. Capogrossi K, You W. The Influence

of School Nutrition Programs on the Weight of Low-Income Children: A Treatment Effect Analysis. Health Econ. 2016 Jul 6doi: 10.1002/hec.3378. PMID: 27381591.

48. Cawley J, Meyerhoefer C, Newhouse

D. The Correlation of Youth Physical Activity with State Policies. Contemporary Economic Policy. 2007;25(4):506-17. doi: http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291465-7287. PMID: 0952344 Alternate Accession Number: EP27555031.

49. Cawley J, Meyerhoefer C, Newhouse

D. The impact of state physical education requirements on youth physical activity and overweight. Health Econ. 2007;16(12):1287-301. doi: 10.1002/hec.1218. PMID: 2007-19974-001.

50. Cheadle A, Rauzon S, Spring R, et

al. Kaiser Permanente's Community Health Initiative in Northern California: evaluation findings and lessons learned. Am J Health Promot. 2012 Nov-Dec;27(2):e59-68. doi: 10.4278/ajhp.111222-QUAN-462. PMID: 23113787.

Page 154: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-6

51. Chen C, Chou S-Y, Thornton RJ. The Effect of Household Technology on Weight and Health Outcomes among Chinese Adults: Evidence from China's 'Home Appliances Going to the Countryside' Policy. Journal of Human Capital. 2015 Fall;9(3):364-401. doi: http://www.jstor.org/action/showPublication?journalCode=jhumancapital. PMID: 1541217.

52. Chomitz VR, McGowan RJ, Wendel

JM, et al. Healthy Living Cambridge Kids: a community-based participatory effort to promote healthy weight and fitness. Obesity (Silver Spring). 2010 Feb;18 Suppl 1:S45-53. doi: 10.1038/oby.2009.431. PMID: 20107461.

53. Cleland V, Dwyer T, Blizzard L, et

al. The provision of compulsory school physical activity: associations with physical activity, fitness and overweight in childhood and twenty years later. Int J Behav Nutr Phys Act. 2008 Feb 29;5:14. doi: 10.1186/1479-5868-5-14. PMID: 18312621.

54. Cochrane T, Davey R, Iqbal Z, et al.

NHS health checks through general practice: randomised trial of population cardiovascular risk reduction. BMC Public Health. 2012 Nov 01;12:944. doi: 10.1186/1471-2458-12-944. PMID: 23116213.

55. Coffield JE, Metos JM, Utz RL, et

al. A multivariate analysis of federally mandated school wellness policies on adolescent obesity. J Adolesc Health. 2011 Oct;49(4):363-70. doi:

10.1016/j.jadohealth.2011.01.010. PMID: 21939866.

56. Cohen DA, Marsh T, Williamson S,

et al. Impact and cost-effectiveness of family Fitness Zones: a natural experiment in urban public parks. Health Place. 2012 Jan;18(1):39-45. doi: 10.1016/j.healthplace.2011.09.008. PMID: 22243905.

57. Cohen JF, Kraak VI,

Choumenkovitch SF, et al. The CHANGE study: a healthy-lifestyles intervention to improve rural children's diet quality. J Acad Nutr Diet. 2014 Jan;114(1):48-53. doi: 10.1016/j.jand.2013.08.014. PMID: 24126295.

58. Coleman KJ, Shordon M, Caparosa

SL, et al. The healthy options for nutrition environments in schools (Healthy ONES) group randomized trial: using implementation models to change nutrition policy and environments in low income schools. Int J Behav Nutr Phys Act. 2012 Jun 27;9:80. doi: 10.1186/1479-5868-9-80. PMID: 22734945.

59. Cortinez-O'Ryan A, Albagli A,

Sadarangani KP, et al. Reclaiming streets for outdoor play: A process and impact evaluation of "Juega en tu Barrio" (Play in your Neighborhood), an intervention to increase physical activity and opportunities for play. PLoS One. 2017;12(7):e0180172. doi: 10.1371/journal.pone.0180172. PMID: 28671984.

60. Coyle KK, Potter S, Schneider D, et

al. Distributing free fresh fruit and

Page 155: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-7

vegetables at school: results of a pilot outcome evaluation. Public Health Rep. 2009 Sep-Oct;124(5):660-9. doi: 10.1177/003335490912400508. PMID: 19753944.

61. Cradock AL, Barrett JL, Carter J, et

al. Impact of the Boston Active School Day policy to promote physical activity among children. Am J Health Promot. 2014 Jan-Feb;28(3 Suppl):S54-64. doi: 10.4278/ajhp.130430-QUAN-204. PMID: 24380467.

62. Cradock AL, McHugh A, Mont-

Ferguson H, et al. Effect of school district policy change on consumption of sugar-sweetened beverages among high school students, Boston, Massachusetts, 2004-2006. Prev Chronic Dis. 2011 Jul;8(4):A74. PMID: 21672398.

63. Crespo NC, Elder JP, Ayala GX, et

al. Results of a multi-level intervention to prevent and control childhood obesity among Latino children: the Aventuras Para Ninos Study. Ann Behav Med. 2012 Feb;43(1):84-100. doi: 10.1007/s12160-011-9332-7. PMID: 22215470.

64. Cullen KW, Watson K, Zakeri I.

Improvements in middle school student dietary intake after implementation of the Texas Public School Nutrition Policy. Am J Public Health. 2008 Jan;98(1):111-7. doi: 10.2105/ajph.2007.111765. PMID: 18048778.

65. Cullen KW, Watson K, Zakeri I, et

al. Exploring changes in middle-

school student lunch consumption after local school food service policy modifications. Public Health Nutr. 2006 Sep;9(6):814-20. PMID: 16925889.

66. Cummins S, Findlay A, Petticrew

CHM, et al. Reducing inequalities in health and diet: findings from a study on the impact of a food retail development. Environ Plann A 2008;40(2):402-22.

67. Cummins S, Flint E, Matthews SA.

New neighborhood grocery store increased awareness of food access but did not alter dietary habits or obesity. Health Aff (Millwood). 2014 Feb;33(2):283-91. doi: 10.1377/hlthaff.2013.0512. PMID: 24493772.

68. Cummins S, Petticrew M, Higgins C,

et al. Large scale food retailing as an intervention for diet and health: quasi-experimental evaluation of a natural experiment. J Epidemiol Community Health. 2005 Dec;59(12):1035-40. doi: 10.1136/jech.2004.029843. PMID: 16286490.

69. Datar A, Nicosia N. The effect of

state competitive food and beverage regulations on childhood overweight and obesity. Journal of Adolescent Health. 2016doi: 10.1016/j.jadohealth.2016.09.003. PMID: 2016-54655-001.

70. Day ME, Strange KS, McKay HA, et

al. Action schools! BC--Healthy Eating: effects of a whole-school model to modifying eating behaviours of elementary school children. Can J Public Health. 2008

Page 156: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-8

Jul-Aug;99(4):328-31. PMID: 18767281.

71. De Cocker KA, De Bourdeaudhuij

IM, Brown WJ, et al. Effects of "10,000 steps Ghent": a whole-community intervention. Am J Prev Med. 2007 Dec;33(6):455-63. doi: 10.1016/j.amepre.2007.07.037. PMID: 18022061.

72. De Cocker KA, De Bourdeaudhuij

IM, Brown WJ, et al. Four-year follow-up of the community intervention '10,000 steps Ghent'. Health Educ Res. 2011 Apr;26(2):372-80. doi: 10.1093/her/cyr015. PMID: 21393377.

73. De Coen V, De Bourdeaudhuij I,

Vereecken C, et al. Effects of a 2-year healthy eating and physical activity intervention for 3-6-year-olds in communities of high and low socio-economic status: the POP (Prevention of Overweight among Pre-school and school children) project. Public Health Nutr. 2012 Sep;15(9):1737-45. doi: 10.1017/s1368980012000687.

74. de Greeff JW, Hartman E,

Mullender-Wijnsma MJ, et al. Effect of Physically Active Academic Lessons on Body Mass Index and Physical Fitness in Primary School Children. J Sch Health. 2016 May;86(5):346-52. doi: 10.1111/josh.12384. PMID: 27040472.

75. De Henauw S, Huybrechts I, De

Bourdeaudhuij I, et al. Effects of a community-oriented obesity prevention programme on indicators

of body fatness in preschool and primary school children. Main results from the IDEFICS study. Obes Rev. 2015 Dec;16 Suppl 2:16-29. doi: 10.1111/obr.12346. PMID: 26707013.

76. de Meij JS, Chinapaw MJ, van

Stralen MM, et al. Effectiveness of JUMP-in, a Dutch primary school-based community intervention aimed at the promotion of physical activity. Br J Sports Med. 2011 Oct;45(13):1052-7. doi: 10.1136/bjsm.2010.075531. PMID: 21112875.

77. de Silva-Sanigorski A, Elea D, Bell

C, et al. Obesity prevention in the family day care setting: impact of the Romp & Chomp intervention on opportunities for children's physical activity and healthy eating. Child Care Health Dev. 2011 May;37(3):385-93. doi: 10.1111/j.1365-2214.2010.01205.x. PMID: 21276039.

78. de Visser R, Sylvester R, Rogers R,

et al. Changes in School Health Program Improve Middle School Students' Behaviors. Am J Health Behav. 2016 Sep;40(5):568-77. doi: 10.5993/ajhb.40.5.3. PMID: 27561859.

79. Dill J, McNeil N, Broach J, et al.

Bicycle boulevards and changes in physical activity and active transportation: findings from a natural experiment. Prev Med. 2014 Dec;69 Suppl 1:S74-8. doi: 10.1016/j.ypmed.2014.10.006. PMID: 25456802.

Page 157: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-9

80. Dubowitz T, Ghosh-Dastidar M, Cohen DA, et al. Diet And Perceptions Change With Supermarket Introduction In A Food Desert, But Not Because Of Supermarket Use. Health Aff (Millwood). 2015 Nov;34(11):1858-68. doi: 10.1377/hlthaff.2015.0667. PMID: 26526243.

81. Dzewaltowski DA, Estabrooks PA,

Welk G, et al. Healthy youth places: a randomized controlled trial to determine the effectiveness of facilitating adult and youth leaders to promote physical activity and fruit and vegetable consumption in middle schools. Health Educ Behav. 2009 Jun;36(3):583-600. doi: 10.1177/1090198108314619. PMID: 18469366.

82. Eagle TF, Gurm R, Smith CA, et al.

A middle school intervention to improve health behaviors and reduce cardiac risk factors. Am J Med. 2013 Oct;126(10):903-8. doi: 10.1016/j.amjmed.2013.04.019. PMID: 23932159.

83. Economos CD, Hyatt RR, Goldberg

JP, et al. A community intervention reduces BMI z-score in children: Shape Up Somerville first year results. Obesity (Silver Spring). 2007 May;15(5):1325-36. doi: 10.1038/oby.2007.155. PMID: 17495210.

84. Elbel B, Mijanovich T, Kiszko K, et

al. The Introduction of a Supermarket via Tax-Credits in a Low-Income Area. Am J Health Promot. 2017 Jan;31(1):59-66. doi: 10.4278/ajhp.150217-QUAN-733. PMID: 26389982.

85. Elbel B, Moran A, Dixon LB, et al.

Assessment of a government-subsidized supermarket in a high-need area on household food availability and children's dietary intakes. Public Health Nutr. 2015 Oct;18(15):2881-90. doi: 10.1017/s1368980015000282. PMID: 25714993.

86. Elinder LS, Heinemans N, Hagberg

J, et al. A participatory and capacity-building approach to healthy eating and physical activity—SCIP-school: A 2-year controlled trial. Int J Behav Nutr Phys Act. 2012;9doi: 10.1186/1479-5868-9-145. PMID: 2013-04234-001.

87. Eriksen K, Haraldsdottir J, Pederson

R, et al. Effect of a fruit and vegetable subscription in Danish schools. Public Health Nutr. 2003 Feb;6(1):57-63. doi: 10.1079/phn2002356. PMID: 12581466.

88. Ermetici F, Zelaschi RF, Briganti S,

et al. Association between a school-based intervention and adiposity outcomes in adolescents: The Italian "EAT" project. Obesity (Silver Spring). 2016 Mar;24(3):687-95. doi: 10.1002/oby.21365. PMID: 26833570.

89. Esquivel M, Nigg CR, Fialkowski

MK, et al. Head Start Wellness Policy Intervention in Hawaii: A Project of the Children's Healthy Living Program. Child Obes. 2016 Feb;12(1):26-32. doi: 10.1089/chi.2015.0071. PMID: 26771119.

Page 158: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-10

90. Evans CE, Ransley JK, Christian MS, et al. A cluster-randomised controlled trial of a school-based fruit and vegetable intervention: Project Tomato. Public Health Nutr. 2013 Jun;16(6):1073-81. doi: 10.1017/s1368980012005290. PMID: 23237386.

91. Fairclough SJ, McGrane B, Sanders

G, et al. A non-equivalent group pilot trial of a school-based physical activity and fitness intervention for 10-11 year old english children: born to move. BMC Public Health. 2016 Aug 24;16(1):861. doi: 10.1186/s12889-016-3550-7. PMID: 27553010.

92. Falbe J, Thompson HR, Becker CM,

et al. Impact of the Berkeley Excise Tax on Sugar-Sweetened Beverage Consumption. Am J Public Health. 2016 Oct;106(10):1865-71. doi: 10.2105/ajph.2016.303362. PMID: 27552267.

93. Farley TA, Meriwether RA, Baker

ET, et al. Safe play spaces to promote physical activity in inner-city children: results from a pilot study of an environmental intervention. Am J Public Health. 2007 Sep;97(9):1625-31. doi: 10.2105/ajph.2006.092692. PMID: 17666701.

94. Farmer VL, Williams SM, Mann JI,

et al. The effect of increasing risk and challenge in the school playground on physical activity and weight in children: a cluster randomised controlled trial (PLAY). Int J Obes (Lond). 2017 May;41(5):793-800. doi:

10.1038/ijo.2017.41. PMID: 28186099.

95. Ferreira RA, Benicio MH. [Obesity

in Brazilian women: association with parity and socioeconomic status]. Rev Panam Salud Publica. 2015 May;37(4-5):337-42. PMID: 26208205.

96. Finch M, Wolfenden L, Morgan PJ,

et al. A cluster randomized trial of a multi-level intervention, delivered by service staff, to increase physical activity of children attending center-based childcare. Prev Med. 2014 Jan;58:9-16. doi: 10.1016/j.ypmed.2013.10.004. PMID: 24145204.

97. Fitzpatrick C, Datta GD, Henderson

M, et al. School food environments associated with adiposity in Canadian children. Int J Obes (Lond). 2017 Mar 14doi: 10.1038/ijo.2017.39. PMID: 28186100.

98. Flego A, Herbert J, Waters E, et al.

Jamie's Ministry of Food: quasi-experimental evaluation of immediate and sustained impacts of a cooking skills program in Australia. PLoS One. 2014;9(12):e114673. doi: 10.1371/journal.pone.0114673. PMID: 25514531.

99. Fogarty AW, Antoniak M, Venn AJ,

et al. Does participation in a population-based dietary intervention scheme have a lasting impact on fruit intake in young children? Int J Epidemiol. 2007 Oct;36(5):1080-5. doi: 10.1093/ije/dym133. PMID: 17602183.

Page 159: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-11

100. Foster GD, Sherman S, Borradaile

KE, et al. A policy-based school intervention to prevent overweight and obesity. Pediatrics. 2008 Apr;121(4):e794-802. doi: 10.1542/peds.2007-1365. PMID: 18381508.

101. Fox MK, Dodd AH, Wilson A, et al.

Association between school food environment and practices and body mass index of US public school children. J Am Diet Assoc. 2009 Feb;109(2 Suppl):S108-17. doi: 10.1016/j.jada.2008.10.065. PMID: 19166665.

102. French SA, Harnack LJ, Hannan PJ,

et al. Worksite environment intervention to prevent obesity among metropolitan transit workers. Prev Med. 2010 Apr;50(4):180-5. doi: 10.1016/j.ypmed.2010.01.002. PMID: 20079369.

103. Frongillo EA, Fawcett SB, Ritchie

LD, et al. Community Policies and Programs to Prevent Obesity and Child Adiposity. Am J Prev Med. 2017 Jul 05doi: 10.1016/j.amepre.2017.05.006. PMID: 28688728.

104. Fu Y, Gao Z, Hannon JC, et al.

Effect of the SPARK Program on Physical Activity, Cardiorespiratory Endurance, and Motivation in Middle-School Students. J Phys Act Health. 2016 May;13(5):534-42. doi: 10.1123/jpah.2015-0351. PMID: 26528889.

105. Fuller D, Gauvin L, Kestens Y, et al.

Impact evaluation of a public bicycle share program on cycling: a case

example of BIXI in Montreal, Quebec. Am J Public Health. 2013 Mar;103(3):e85-92. doi: 10.2105/ajph.2012.300917. PMID: 23327280.

106. Fung C, McIsaac JL, Kuhle S, et al.

The impact of a population-level school food and nutrition policy on dietary intake and body weights of Canadian children. Prev Med. 2013 Dec;57(6):934-40. doi: 10.1016/j.ypmed.2013.07.016. PMID: 23891787.

107. Gatto NM, Martinez LC, Spruijt-

Metz D, et al. LA sprouts randomized controlled nutrition, cooking and gardening programme reduces obesity and metabolic risk in Hispanic/Latino youth. Pediatr Obes. 2017 Feb;12(1):28-37. doi: 10.1111/ijpo.12102. PMID: 26909882.

108. Geaney F, Harrington J, Perry IJ.

P42 The impact of a catering initiative in determining food choices and salt intake in the public sector. Journal of Epidemiology & Community Health. 2010;64:A50-A. PMID: 104133563. Language: English. Entry Date: 20140108. Revision Date: 20150710. Publication Type: Journal Article.

109. Geaney F, Kelly C, Di Marrazzo JS,

et al. The effect of complex workplace dietary interventions on employees' dietary intakes, nutrition knowledge and health status: a cluster controlled trial. Prev Med. 2016 Aug;89:76-83. doi: 10.1016/j.ypmed.2016.05.005. PMID: 27208667.

Page 160: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-12

110. Gebel K, Bauman AE, Reger-Nash B, et al. Does the environment moderate the impact of a mass media campaign to promote walking? Am J Health Promot. 2011 Sep-Oct;26(1):45-8. doi: 10.4278/ajhp.081104-ARB-269. PMID: 21879942.

111. Gee KA. School-Based Body Mass

Index Screening and Parental Notification in Late Adolescence: Evidence From Arkansas's Act 1220. J Adolesc Health. 2015 Sep;57(3):270-6. doi: 10.1016/j.jadohealth.2015.05.007. PMID: 26115907.

112. Gibson D. Long-term Food Stamp

Program participation is positively related to simultaneous overweight in young daughters and obesity in mothers. J Nutr. 2006 Apr;136(4):1081-5. PMID: 16549483.

113. Giles-Corti B, Bull F, Knuiman M,

et al. The influence of urban design on neighbourhood walking following residential relocation: longitudinal results from the RESIDE study. Soc Sci Med. 2013 Jan;77:20-30. doi: 10.1016/j.socscimed.2012.10.016. PMID: 23206559.

114. Gittelsohn J, Kim EM, He S, et al. A

food store-based environmental intervention is associated with reduced BMI and improved psychosocial factors and food-related behaviors on the Navajo nation. J Nutr. 2013 Sep;143(9):1494-500. doi: 10.3945/jn.112.165266. PMID: 23864511.

115. Gittelsohn J, Vijayadeva V, Davison N, et al. A food store intervention trial improves caregiver psychosocial factors and children's dietary intake in Hawaii. Obesity (Silver Spring). 2010 Feb;18 Suppl 1:S84-90. doi: 10.1038/oby.2009.436. PMID: 20107467.

116. Gleason PM, Dodd AH. School

Breakfast Program but Not School Lunch Program Participation Is Associated with Lower Body Mass Index. J Am Diet Assoc. 2009 Feb;109(2):S118-S28. doi: 10.1016/j.jada.2008.10.058. PMID: 19166666.

117. Goetzel RZ, Roemer EC, Pei X, et

al. Second-year results of an obesity prevention program at the Dow Chemical Company. J Occup Environ Med. 2010 Mar;52(3):291-302. doi: 10.1097/JOM.0b013e3181d46f0b. PMID: 20190646.

118. Goldsby TU, George BJ, Yeager

VA, et al. Urban Park Development and Pediatric Obesity Rates: A Quasi-Experiment Using Electronic Health Record Data. Int J Environ Res Public Health. 2016 Apr 08;13(4):411. doi: 10.3390/ijerph13040411. PMID: 27070635.

119. Goodman A, Sahlqvist S, Ogilvie D.

New walking and cycling routes and increased physical activity: one- and 2-year findings from the UK iConnect Study. Am J Public Health. 2014 Sep;104(9):e38-46. doi: 10.2105/ajph.2014.302059. PMID: 25033133.

Page 161: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-13

120. Goodman A, van Sluijs EM, Ogilvie D. Impact of offering cycle training in schools upon cycling behaviour: a natural experimental study. Int J Behav Nutr Phys Act. 2016 Mar 08;13:34. doi: 10.1186/s12966-016-0356-z. PMID: 26956383.

121. Gorely T, Morris JG, Musson H, et

al. Physical activity and body composition outcomes of the GreatFun2Run intervention at 20 month follow-up. Int J Behav Nutr Phys Act. 2011 Jul 18;8:74. doi: 10.1186/1479-5868-8-74. PMID: 21767356.

122. Gorham G, Dulin-Keita A, Risica

PM, et al. Effectiveness of Fresh to You, a Discount Fresh Fruit and Vegetable Market in Low-Income Neighborhoods, on Children's Fruit and Vegetable Consumption, Rhode Island, 2010-2011. Prev Chronic Dis. 2015 Oct 15;12:E176. doi: 10.5888/pcd12.140583. PMID: 26469949.

123. Gustat J, Rice J, Parker KM, et al.

Effect of changes to the neighborhood built environment on physical activity in a low-income African American neighborhood. Prev Chronic Dis. 2012;9:E57. PMID: 22338597.

124. Haerens L, De Bourdeaudhuij I,

Maes L, et al. The effects of a middle-school healthy eating intervention on adolescents' fat and fruit intake and soft drinks consumption. Public Health Nutr. 2007 May;10(5):443-9. doi: 10.1017/s1368980007219652. PMID: 17411463.

125. Haerens L, Deforche B, Maes L, et al. Body mass effects of a physical activity and healthy food intervention in middle schools. Obesity (Silver Spring). 2006 May;14(5):847-54. doi: 10.1038/oby.2006.98. PMID: 16855194.

126. Harding MC, Bott QD, Jonas CE.

The Malaekahana Path: An Ecological Model-based Intervention for Increasing Walking in Rural Hawai'i. J Phys Act Health. 2017 Jul 06:1-11. doi: 10.1123/jpah.2017-0330. PMID: 28682662.

127. Hardy LL, King L, Kelly B, et al.

Munch and Move: evaluation of a preschool healthy eating and movement skill program. Int J Behav Nutr Phys Act. 2010 Nov 03;7:80. doi: 10.1186/1479-5868-7-80. PMID: 21047434.

128. He M, Beynon C, Sangster Bouck

M, et al. Impact evaluation of the Northern Fruit and Vegetable Pilot Programme - a cluster-randomised controlled trial. Public Health Nutr. 2009 Nov;12(11):2199-208. doi: 10.1017/s1368980009005801. PMID: 19476675.

129. Heelan KA, Abbey BM, Donnelly

JE, et al. Evaluation of a Walking School Bus for Promoting Physical Activity in Youth. J Phys Act Health. 2009 Sep;6(5):560-7. PMID: 19953832.

130. Heelan KA, Bartee RT, Nihiser A, et

al. Healthier School Environment Leads to Decreases in Childhood Obesity: The Kearney Nebraska Story. Child Obes. 2015

Page 162: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-14

Oct;11(5):600-7. doi: 10.1089/chi.2015.0005. PMID: 26440386.

131. Hendy HM, Williams KE, Camise

TS. Kid's Choice Program improves weight management behaviors and weight status in school children. Appetite. 2011 Apr;56(2):484-94. doi: 10.1016/j.appet.2011.01.024. PMID: 21277924.

132. Hennessy E, Oh A, Agurs-Collins T,

et al. State-level school competitive food and beverage laws are associated with children's weight status. J Sch Health. 2014 Sep;84(9):609-16. doi: 10.1111/josh.12181. PMID: 25117896.

133. Herrick H, Thompson H, Kinder J, et

al. Use of SPARK to promote after-school physical activity. J Sch Health. 2012 Oct;82(10):457-61. doi: 10.1111/j.1746-1561.2012.00722.x. PMID: 22954164.

134. Hilmers A, Chen TA, Dave JM, et al.

Supplemental Nutrition Assistance Program participation did not help low income Hispanic women in Texas meet the dietary guidelines. Prev Med. 2014 May;62:44-8. doi: 10.1016/j.ypmed.2014.01.016. PMID: 24530319.

135. Hobin E, Erickson T, Comte M, et

al. Examining the impact of a province-wide physical education policy on secondary students' physical activity as a natural experiment. Int J Behav Nutr Phys Act. 2017 Jul 19;14(1):98. doi: 10.1186/s12966-017-0550-7. PMID: 28724390.

136. Hobin E, So J, Rosella L, et al.

Trajectories of objectively measured physical activity among secondary students in Canada in the context of a province-wide physical education policy: a longitudinal analysis. J Obes. 2014;2014:958645. doi: 10.1155/2014/958645. PMID: 24672714.

137. Hoefkens C, Lachat C, Kolsteren P,

et al. Posting point-of-purchase nutrition information in university canteens does not influence meal choice and nutrient intake. Am J Clin Nutr. 2011 Aug;94(2):562-70. doi: 10.3945/ajcn.111.013417. PMID: 21677060.

138. Hoelscher D, Ory M, Dowdy D, et

al. Effects of funding allocation for Safe Routes to School programs on active commuting to school and related behavioral, knowledge, and psychosocial outcomes: Results from the Texas Childhood Obesity Prevention Policy Evaluation (T-COPPE) study. Environ Behav. 2016;48(1):210-29. doi: 10.1177/0013916515613541. PMID: 2016-02263-012.

139. Hoelscher DM, Moag-Stahlberg A,

Ellis K, et al. Evaluation of a student participatory, low-intensity program to improve school wellness environment and students' eating and activity behaviors. Int J Behav Nutr Phys Act. 2016 May 13;13:59. doi: 10.1186/s12966-016-0379-5. PMID: 27178056.

140. Hollis JL, Sutherland R, Campbell L,

et al. Effects of a 'school-based' physical activity intervention on

Page 163: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-15

adiposity in adolescents from economically disadvantaged communities: secondary outcomes of the 'Physical Activity 4 Everyone' RCT. Int J Obes (Lond). 2016 Oct;40(10):1486-93. doi: 10.1038/ijo.2016.107. PMID: 27430652.

141. Hosey GM, Samo M, Gregg EW, et

al. Socioeconomic and demographic predictors of selected cardiovascular risk factors among adults living in Pohnpei, Federated States of Micronesia. BMC Public Health. 2014 Aug 31;14:895. doi: 10.1186/1471-2458-14-895. PMID: 25175388.

142. Howlett E, Davis C, Burton S. From

food desert to food oasis: The potential influence of food retailers on childhood obesity rates. Journal of Business Ethics. 2016;139(2):215-24. doi: 10.1007/s10551-015-2605-5. PMID: 2015-12722-001.

143. Hu Y, van Lenthe FJ, Judge K, et al.

Did the English strategy reduce inequalities in health? A difference-in-difference analysis comparing England with three other European countries. BMC Public Health. 2016 Aug 24;16(1):865. doi: 10.1186/s12889-016-3505-z. PMID: 27558269.

144. Huberty J, Beets M, Beighle A.

Effects of a policy-level intervention on children's pedometer-determined physical activity: preliminary findings from Movin' Afterschool. J Public Health Manag Pract. 2013 Nov-Dec;19(6):525-8. doi: 10.1097/PHH.0b013e31829465fa. PMID: 23676476.

145. Huberty JL, Beets MW, Beighle A,

et al. Environmental modifications to increase physical activity during recess: preliminary findings from ready for recess. J Phys Act Health. 2011 Sep;8 Suppl 2:S249-56. PMID: 21918239.

146. Hughes RJ, Edwards KL, Clarke GP,

et al. Childhood consumption of fruit and vegetables across England: a study of 2306 6-7-year-olds in 2007. Br J Nutr. 2012 Aug;108(4):733-42. doi: 10.1017/s0007114511005939. PMID: 22321148.

147. Hunter S, Leatherdale ST, Storey K,

et al. A quasi-experimental examination of how school-based physical activity changes impact secondary school student moderate- to vigorous- intensity physical activity over time in the COMPASS study. Int J Behav Nutr Phys Act. 2016 Jul 29;13:86. doi: 10.1186/s12966-016-0411-9. PMID: 27473113.

148. Jago R, McMurray RG, Drews KL,

et al. HEALTHY intervention: fitness, physical activity, and metabolic syndrome results. Med Sci Sports Exerc. 2011 Aug;43(8):1513-22. doi: 10.1249/MSS.0b013e31820c9797. PMID: 21233778.

149. Janssen M, Twisk JW, Toussaint

HM, et al. Effectiveness of the PLAYgrounds programme on PA levels during recess in 6-year-old to 12-year-old children. Br J Sports Med. 2015 Feb;49(4):259-64. doi: 10.1136/bjsports-2012-091517. PMID: 23293007.

Page 164: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-16

150. Jennings A, Cassidy A, Winters T, et

al. Positive effect of a targeted intervention to improve access and availability of fruit and vegetables in an area of deprivation. Health Place. 2012 Sep;18(5):1074-8. doi: 10.1016/j.healthplace.2012.05.001. PMID: 22705164.

151. Jia P, Li M, Xue H, et al. School

environment and policies, child eating behavior and overweight/obesity in urban China: the childhood obesity study in China megacities. Int J Obes (Lond). 2017 May;41(5):813-9. doi: 10.1038/ijo.2017.2. PMID: 28074059.

152. Johnson R, Robertson W, Towey M,

et al. Changes over time in mental well-being, fruit and vegetable consumption and physical activity in a community-based lifestyle intervention: a before and after study. Public Health. 2017 May;146:118-25. doi: 10.1016/j.puhe.2017.01.012. PMID: 28404463.

153. Jones J, Wyse R, Finch M, et al.

Effectiveness of an intervention to facilitate the implementation of healthy eating and physical activity policies and practices in childcare services: a randomised controlled trial. Implement Sci. 2015 Oct 25;10:147. doi: 10.1186/s13012-015-0340-z. PMID: 26498746.

154. Jordan KC, Erickson ED, Cox R, et

al. Evaluation of the Gold Medal Schools program. J Am Diet Assoc. 2008 Nov;108(11):1916-20. doi:

10.1016/j.jada.2008.08.002. PMID: 18954584.

155. Jurg ME, Kremers SP, Candel MJ, et

al. A controlled trial of a school-based environmental intervention to improve physical activity in Dutch children: JUMP-in, kids in motion. Health Promot Int. 2006 Dec;21(4):320-30. doi: 10.1093/heapro/dal032. PMID: 16963784.

156. Just DR, Wansink B. School lunch

debit card payment systems are associated with lower nutrition and higher calories. Obesity (Silver Spring). 2014 Jan;22(1):24-6. doi: 10.1002/oby.20591. PMID: 23929600.

157. Kain J, Uauy R, Albala, et al.

School-based obesity prevention in Chilean primary school children: methodology and evaluation of a controlled study. Int J Obes Relat Metab Disord. 2004 Apr;28(4):483-93. doi: 10.1038/sj.ijo.0802611. PMID: 14993915.

158. Kamada M, Kitayuguchi J, Inoue S,

et al. A community-wide campaign to promote physical activity in middle-aged and elderly people: a cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2013 Apr 09;10:44. doi: 10.1186/1479-5868-10-44. PMID: 23570536.

159. Kastorini CM, Lykou A,

Yannakoulia M, et al. The influence of a school-based intervention programme regarding adherence to a healthy diet in children and adolescents from disadvantaged areas in Greece: the DIATROFI

Page 165: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-17

study. J Epidemiol Community Health. 2016 Jul;70(7):671-7. doi: 10.1136/jech-2015-205680. PMID: 26763974.

160. Kern E, Chan NL, Fleming DW, et

al. Declines in student obesity prevalence associated with a prevention initiative - King County, Washington, 2012. MMWR Morb Mortal Wkly Rep. 2014 Feb 21;63(7):155-7. PMID: 24553199.

161. Keyte J, Harris S, Margetts B, et al.

Engagement with the National Healthy Schools Programme is associated with higher fruit and vegetable consumption in primary school children. J Hum Nutr Diet. 2012 Apr;25(2):155-60. doi: 10.1111/j.1365-277X.2011.01208.x. PMID: 22128770.

162. Kim J. Are physical education-

related state policies and schools' physical education requirement related to children's physical activity and obesity? J Sch Health. 2012 Jun;82(6):268-76. doi: 10.1111/j.1746-1561.2012.00697.x. PMID: 22568462.

163. Kim K, Hong SA, Yun SH, et al. The

effect of a healthy school tuck shop program on the access of students to healthy foods. Nutr Res Pract. 2012 Apr;6(2):138-45. doi: 10.4162/nrp.2012.6.2.138. PMID: 22586503.

164. King MH, Lederer AM, Sovinski D,

et al. Implementation and evaluation of the HEROES initiative: a tri-state coordinated school health program to reduce childhood obesity. Health Promot Pract. 2014 May;15(3):395-

405. doi: 10.1177/1524839913512835. PMID: 24334542.

165. Kloek GC, van Lenthe FJ, van

Nierop PW, et al. Impact evaluation of a Dutch community intervention to improve health-related behaviour in deprived neighbourhoods. Health Place. 2006 Dec;12(4):665-77. doi: 10.1016/j.healthplace.2005.09.002. PMID: 16253541.

166. Kubik MY, Lytle LA, Story M.

Schoolwide food practices are associated with body mass index in middle school students. Arch Pediatr Adolesc Med. 2005 Dec;159(12):1111-4. doi: 10.1001/archpedi.159.12.1111. PMID: 16330732.

167. LaCaille LJ, Schultz JF, Goei R, et

al. Go!: results from a quasi-experimental obesity prevention trial with hospital employees. BMC Public Health. 2016 Feb 19;16:171. doi: 10.1186/s12889-016-2828-0. PMID: 26893128.

168. Lachapelle U, Frank LD. Transit and

health: mode of transport, employer-sponsored public transit pass programs, and physical activity. J Public Health Policy. 2009;30 Suppl 1:S73-94. doi: 10.1057/jphp.2008.52. PMID: 19190584.

169. LaRowe TL, Tomayko EJ, Meinen

AM, et al. Active Early: one-year policy intervention to increase physical activity among early care and education programs in Wisconsin. BMC Public Health. 2016 Jul 20;16:607. doi:

Page 166: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-18

10.1186/s12889-016-3198-3. PMID: 27439770.

170. Lemon SC, Wang ML, Wedick NM,

et al. Weight gain prevention in the school worksite setting: results of a multi-level cluster randomized trial. Prev Med. 2014 Mar;60:41-7. doi: 10.1016/j.ypmed.2013.12.010. PMID: 24345602.

171. Lemon SC, Zapka J, Li W, et al. Step

ahead a worksite obesity prevention trial among hospital employees. Am J Prev Med. 2010 Jan;38(1):27-38. doi: 10.1016/j.amepre.2009.08.028. PMID: 20117554.

172. Lent MR, Vander Veur SS, McCoy

TA, et al. A randomized controlled study of a healthy corner store initiative on the purchases of urban, low-income youth. Obesity (Silver Spring). 2014 Dec;22(12):2494-500. doi: 10.1002/oby.20878. PMID: 25311881.

173. Leung CW, Blumenthal SJ,

Hoffnagle EE, et al. Associations of food stamp participation with dietary quality and obesity in children. Pediatrics. 2013 Mar;131(3):463-72. doi: 10.1542/peds.2012-0889. PMID: 23439902.

174. Liao Y, Siegel PZ, Zhou H, et al.

Reduced Disparity in Vegetable Consumption in 16 Disadvantaged Black Communities: A Successful 5-Year Community-Based Participatory Intervention. J Racial Ethn Health Disparities. 2015 Jun;2(2):211-8. doi: 10.1007/s40615-014-0065-8. PMID: 26150921.

175. Linde JA, Nygaard KE, MacLehose RF, et al. HealthWorks: results of a multi-component group-randomized worksite environmental intervention trial for weight gain prevention. Int J Behav Nutr Phys Act. 2012 Feb 16;9:14. doi: 10.1186/1479-5868-9-14. PMID: 22340088.

176. Ling J, King KM, Speck BJ, et al.

Preliminary assessment of a school-based healthy lifestyle intervention among rural elementary school children. J Sch Health. 2014 Apr;84(4):247-55. doi: 10.1111/josh.12143. PMID: 24617908.

177. Liu J, Kuo T, Jiang L, et al. Food

and drink consumption among 1-5-year-old Los Angeles County children from households receiving dual SNAP and WIC v. only WIC benefits. Public Health Nutr. 2016 Sep 9:1-8. doi: 10.1017/s1368980016002329. PMID: 27609603.

178. Llargues E, Franco R, Recasens A, et

al. Assessment of a school-based intervention in eating habits and physical activity in school children: the AVall study. J Epidemiol Community Health. 2011 Oct;65(10):896-901. doi: 10.1136/jech.2009.102319.

179. Lorentzen C, Ommundsen Y, Jenum

AK, et al. The "Romsas in Motion" community intervention: mediating effects of psychosocial factors on forward transition in the stages of change in physical activity. Health Educ Behav. 2009 Apr;36(2):348-65. doi: 10.1177/1090198107308372. PMID: 18065570.

Page 167: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-19

180. Lubans DR, Smith JJ, Plotnikoff RC,

et al. Assessing the sustained impact of a school-based obesity prevention program for adolescent boys: the ATLAS cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2016 Aug 20;13:92. doi: 10.1186/s12966-016-0420-8. PMID: 27542825.

181. Ludwig J, Sanbonmatsu L,

Gennetian L, et al. Neighborhoods, obesity, and diabetes--a randomized social experiment. N Engl J Med. 2011 Oct 20;365(16):1509-19. doi: 10.1056/NEJMsa1103216. PMID: 22010917.

182. Lv J, Liu QM, Ren YJ, et al. A

community-based multilevel intervention for smoking, physical activity and diet: short-term findings from the Community Interventions for Health programme in Hangzhou, China. J Epidemiol Community Health. 2014 Apr;68(4):333-9. doi: 10.1136/jech-2013-203356. PMID: 24297972.

183. MacDonald JM, Stokes RJ, Cohen

DA, et al. The effect of light rail transit on body mass index and physical activity. Am J Prev Med. 2010 Aug;39(2):105-12. doi: 10.1016/j.amepre.2010.03.016. PMID: 20621257.

184. Maddock J, Takeuchi L, Nett B, et

al. Evaluation of a statewide program to reduce chronic disease: The Healthy Hawaii Initiative, 2000-2004. Eval Program Plann. 2006;29(3):293-300. doi: 10.1016/j.evalprogplan.2005.12.007. PMID: 2006-11996-009.

185. Madsen K, Linchey J, Gerstein D, et

al. Energy Balance 4 Kids with Play: Results from a Two-Year Cluster-Randomized Trial. Child Obes. 2015 Aug;11(4):375-83. doi: 10.1089/chi.2015.0002. PMID: 26061799.

186. Madsen K, Thompson H, Adkins A,

et al. School-community partnerships: a cluster-randomized trial of an after-school soccer program. JAMA Pediatr. 2013 Apr;167(4):321-6. doi: 10.1001/jamapediatrics.2013.1071. PMID: 23440308.

187. Madsen KA. School-based body

mass index screening and parent notification: a statewide natural experiment. Arch Pediatr Adolesc Med. 2011 Nov;165(11):987-92. doi: 10.1001/archpediatrics.2011.127. PMID: 21727262.

188. Madsen KA, Cotterman C, Crawford

P, et al. Effect of the healthy schools program on prevalence of overweight and obesity in California schools, 2006–2012. Preventing Chronic Disease: Public Health Research, Practice, and Policy. 2015;12 PMID: 2016-24674-001.

189. Magarey AM, Pettman TL, Wilson

A, et al. Changes in Primary School Children's Behaviour, Knowledge, Attitudes, and Environments Related to Nutrition and Physical Activity. ISRN Obes. 2013;2013:752081. doi: 10.1155/2013/752081. PMID: 24555153.

190. Malakellis M, Hoare E, Sanigorski

A, et al. School-based systems

Page 168: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-20

change for obesity prevention in adolescents: outcomes of the Australian Capital Territory 'It's Your Move!'. Aust N Z J Public Health. 2017 Jul 27doi: 10.1111/1753-6405.12696. PMID: 28749562.

191. Masse LC, de Niet-Fitzgerald JE,

Watts AW, et al. Associations between the school food environment, student consumption and body mass index of Canadian adolescents. Int J Behav Nutr Phys Act. 2014 Mar 26;11(1):29. doi: 10.1186/1479-5868-11-29. PMID: 24666770.

192. Mead EL, Gittelsohn J, Roache C, et

al. A community-based, environmental chronic disease prevention intervention to improve healthy eating psychosocial factors and behaviors in indigenous populations in the Canadian arctic. Health Education & Behavior. 2013;40(5):592-602. doi: 10.1177/1090198112467793. PMID: 2013-33808-011.

193. Miewald C, Holben D, Hall P. Role

of a food box program in fruit and vegetable consumption and food security. Can J Diet Pract Res. 2012 Summer;73(2):59-65. doi: 10.3148/73.2.2012.59. PMID: 22668838.

194. Miller HJ, Tribby CP, Brown BB, et

al. Public transit generates new physical activity: Evidence from individual GPS and accelerometer data before and after light rail construction in a neighborhood of Salt Lake City, Utah, USA. Health

Place. 2015 Nov;36:8-17. doi: 10.1016/j.healthplace.2015.08.005.

195. Molitor F, Sugerman S, Yu H, et al.

Reach of Supplemental Nutrition Assistance Program-Education (SNAP-Ed) interventions and nutrition and physical activity-related outcomes, California, 2011-2012. Prev Chronic Dis. 2015 Mar 12;12:E33. doi: 10.5888/pcd12.140449. PMID: 25764139.

196. Morrill BA, Madden GJ, Wengreen

HJ, et al. A Randomized Controlled Trial of the Food Dudes Program: Tangible Rewards are More Effective Than Social Rewards for Increasing Short- and Long-Term Fruit and Vegetable Consumption. J Acad Nutr Diet. 2016 Apr;116(4):618-29. doi: 10.1016/j.jand.2015.07.001. PMID: 26297598.

197. Morton KL, Corder K, Suhrcke M, et

al. School polices, programmes and facilities, and objectively measured sedentary time, LPA and MVPA: Associations in secondary school and over the transition from primary to secondary school. Int J Behav Nutr Phys Act. 2016;13 PMID: 2016-21082-001.

198. Mullally ML, Taylor JP, Kuhle S, et

al. A province-wide school nutrition policy and food consumption in elementary school children in Prince Edward Island. Can J Public Health. 2010 Jan-Feb;101(1):40-3. PMID: 20364537.

199. Mumford KG, Contant CK,

Weissman J, et al. Changes in

Page 169: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-21

physical activity and travel behaviors in residents of a mixed-use development. Am J Prev Med. 2011 Nov;41(5):504-7. doi: 10.1016/j.amepre.2011.07.016. PMID: 22011422.

200. Murphy S, Moore GF, Tapper K, et

al. Free healthy breakfasts in primary schools: a cluster randomised controlled trial of a policy intervention in Wales, UK. Public Health Nutr. 2011 Feb;14(2):219-26. doi: 10.1017/s1368980010001886. PMID: 20602868.

201. Nanney MS, MacLehose R, Kubik

MY, et al. Recommended school policies are associated with student sugary drink and fruit and vegetable intake. Prev Med. 2014 May;62:179-81. doi: 10.1016/j.ypmed.2014.01.026. PMID: 24518003.

202. Nanney MS, MacLehose RF, Kubik

MY, et al. School Obesity Prevention Policies and Practices in Minnesota and Student Outcomes: A Longitudinal Cohort Study. Am J Prev Med. 2016 Nov;51(5):656-63. doi: 10.1016/j.amepre.2016.05.008. PMID: 27320703.

203. Naul R, Schmelt D, Dreiskaemper D,

et al. 'Healthy children in sound communities' (HCSC/gkgk)--a Dutch-German community-based network project to counteract obesity and physical inactivity. Fam Pract. 2012 Apr;29 Suppl 1:i110-i6. doi: 10.1093/fampra/cmr097. PMID: 22399539.

204. Naylor PJ, Macdonald HM,

Warburton DE, et al. An active

school model to promote physical activity in elementary schools: action schools! BC. Br J Sports Med. 2008 May;42(5):338-43. doi: 10.1136/bjsm.2007.042036. PMID: 18272538.

205. Naylor PJ, Macdonald HM, Zebedee

JA, et al. Lessons learned from Action Schools! BC--an 'active school' model to promote physical activity in elementary schools. J Sci Med Sport. 2006 Oct;9(5):413-23. doi: 10.1016/j.jsams.2006.06.013. PMID: 16884957.

206. Nehme EK, Perez A, Ranjit N, et al.

The Effect of New Shower Facilities on Physical Activity Behaviors of Employees: A Quasi-experiment. J Phys Act Health. 2017 Feb;14(2):98-107. doi: 10.1123/jpah.2015-0418. PMID: 27775466.

207. Neumark-Sztainer DR, Friend SE,

Flattum CF, et al. New moves-preventing weight-related problems in adolescent girls a group-randomized study. Am J Prev Med. 2010 Nov;39(5):421-32. doi: 10.1016/j.amepre.2010.07.017. PMID: 20965379.

208. Nguyen BT, Powell LM.

Supplemental nutrition assistance program participation and sugar-sweetened beverage consumption, overall and by source. Prev Med. 2015 Dec;81:82-6. doi: 10.1016/j.ypmed.2015.08.003. PMID: 26303370.

209. Nicklas T, Lopez S, Liu Y, et al.

Motivational theater to increase consumption of vegetable dishes by preschool children. Int J Behav Nutr

Page 170: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-22

Phys Act. 2017 Feb 07;14(1):16. doi: 10.1186/s12966-017-0468-0. PMID: 28166788.

210. Oh AY, Hennessy E, McSpadden

KE, et al. Contextual Influences on Weight Status Among Impoverished Adolescents: Neighborhood Amenities for Physical Activity and State Laws for Physical Education Time Requirements. J Phys Act Health. 2015 Jun;12(6):875-8. doi: 10.1123/jpah.2013-0303. PMID: 25109235.

211. Olsho LE, Payne GH, Walker DK, et

al. Impacts of a farmers' market incentive programme on fruit and vegetable access, purchase and consumption. Public Health Nutr. 2015 Oct;18(15):2712-21. doi: 10.1017/s1368980015001056. PMID: 25919225.

212. Ortega AN, Albert SL, Chan-Golston

AM, et al. Substantial improvements not seen in health behaviors following corner store conversions in two Latino food swamps. BMC Public Health. 2016 May 11;16:389. doi: 10.1186/s12889-016-3074-1. PMID: 27169514.

213. Panter J, Heinen E, Mackett R, et al.

Impact of New Transport Infrastructure on Walking, Cycling, and Physical Activity. Am J Prev Med. 2016 Feb;50(2):e45-53. doi: 10.1016/j.amepre.2015.09.021. PMID: 26585051.

214. Parsons WG, Garcia GM, Hoffman

PK. Evaluating school wellness policy in curbing childhood obesity in Anchorage, Alaska. J Sch Nurs. 2014 Oct;30(5):324-31. doi:

10.1177/1059840513513155. PMID: 24316497.

215. Pate RR, Ward DS, Saunders RP, et

al. Promotion of physical activity among high-school girls: a randomized controlled trial. Am J Public Health. 2005 Sep;95(9):1582-7. doi: 10.2105/ajph.2004.045807. PMID: 16118370.

216. Pbert L, Druker S, Barton B, et al. A

School-Based Program for Overweight and Obese Adolescents: A Randomized Controlled Trial. J Sch Health. 2016 Oct;86(10):699-708. doi: 10.1111/josh.12428. PMID: 27619760.

217. Perry CL, Bishop DB, Taylor GL, et

al. A randomized school trial of environmental strategies to encourage fruit and vegetable consumption among children. Health Educ Behav. 2004 Feb;31(1):65-76. PMID: 14768658.

218. Peterson KE, Spadano-Gasbarro JL,

Greaney ML, et al. Three-Year Improvements in Weight Status and Weight-Related Behaviors in Middle School Students: The Healthy Choices Study. PLoS One. 2015;10(8):e0134470. doi: 10.1371/journal.pone.0134470. PMID: 26295837.

219. Pope M. Preventing Weight Gain in

Children Who Are School Age and African-American. Pediatr Phys Ther. 2016 Summer;28(2):207-16. doi: 10.1097/pep.0000000000000243. PMID: 26914717.

Page 171: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-23

220. Powell LM, Chriqui J, Chaloupka FJ. Associations between state-level soda taxes and adolescent body mass index. J Adolesc Health. 2009 Sep;45(3 Suppl):S57-63. doi: 10.1016/j.jadohealth.2009.03.003. PMID: 19699437.

221. Quigg R, Reeder AI, Gray A, et al.

The Effectiveness of a Community Playground Intervention. Journal of Urban Health-Bulletin of the New York Academy of Medicine. 2012 Feb;89(1):171-84. doi: 10.1007/s11524-011-9622-1.

222. Ransley JK, Greenwood DC, Cade

JE, et al. Does the school fruit and vegetable scheme improve children's diet? A non-randomised controlled trial. J Epidemiol Community Health. 2007 Aug;61(8):699-703. doi: 10.1136/jech.2006.052696. PMID: 17630369.

223. Reger-Nash B, Bauman A, Booth-

Butterfield S, et al. Wheeling walks: evaluation of a media-based community intervention. Fam Community Health. 2005 Jan-Mar;28(1):64-78. PMID: 15625507.

224. Reger-Nash B, Bauman A, Cooper

L, et al. WV Walks: replication with expanded reach. J Phys Act Health. 2008 Jan;5(1):19-27. PMID: 18209251.

225. Reilly JJ, Kelly L, Montgomery C, et

al. Physical activity to prevent obesity in young children: cluster randomised controlled trial. BMJ. 2006 Nov 18;333(7577):1041. doi: 10.1136/bmj.38979.623773.55. PMID: 17028105.

226. Restrepo BJ. Calorie Labeling in Chain Restaurants and Body Weight: Evidence from New York. Health Econ. 2016 Jul 24doi: 10.1002/hec.3389. PMID: 27451966.

227. Reynolds KD, Franklin FA, Binkley

D, et al. Increasing the fruit and vegetable consumption of fourth-graders: results from the high 5 project. Prev Med. 2000 Apr;30(4):309-19. doi: 10.1006/pmed.1999.0630. PMID: 10731460.

228. Ridgers ND, Fairclough SJ, Stratton

G. Twelve-month effects of a playground intervention on children's morning and lunchtime recess physical activity levels. J Phys Act Health. 2010 Mar;7(2):167-75. PMID: 20484755.

229. Ridgers ND, Stratton G, Fairclough

SJ, et al. Children's physical activity levels during school recess: a quasi-experimental intervention study. Int J Behav Nutr Phys Act. 2007 May 21;4:19. doi: 10.1186/1479-5868-4-19. PMID: 17517136.

230. Ridgers ND, Stratton G, Fairclough

SJ, et al. Long-term effects of a playground markings and physical structures on children's recess physical activity levels. Prev Med. 2007 May;44(5):393-7. doi: 10.1016/j.ypmed.2007.01.009. PMID: 17335891.

231. Riis J, Grason H, Strobino D, et al.

State school policies and youth obesity. Matern Child Health J. 2012 Apr;16 Suppl 1:S111-8. doi: 10.1007/s10995-012-1000-4. PMID: 22527761.

Page 172: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-24

232. Ritchie LD, Rosen NJ, Fenton K, et

al. School Breakfast Policy Is Associated with Dietary Intake of Fourth- and Fifth-Grade Students. J Acad Nutr Diet. 2016 Mar;116(3):449-57. doi: 10.1016/j.jand.2015.08.020. PMID: 26433452.

233. Robles B, Montes CE, Nobari TZ, et

al. Dietary Behaviors among Public Health Center Clients with Electronic Benefit Transfer Access at Farmers' Markets. J Acad Nutr Diet. 2017 Jan;117(1):58-68. doi: 10.1016/j.jand.2016.07.012. PMID: 27618576.

234. Rogers VW, Hart PH, Motyka E, et

al. Impact of Let's Go! 5-2-1-0: a community-based, multisetting childhood obesity prevention program. J Pediatr Psychol. 2013 Oct;38(9):1010-20. doi: 10.1093/jpepsy/jst057. PMID: 23933841.

235. Rush E, McLennan S, Obolonkin V,

et al. Project Energize: whole-region primary school nutrition and physical activity programme; evaluation of body size and fitness 5 years after the randomised controlled trial. Br J Nutr. 2014 Jan 28;111(2):363-71. doi: 10.1017/s0007114513002316. PMID: 23867069.

236. Rushakoff JA, Zoughbie DE, Bui N,

et al. Evaluation of Healthy2Go: A country store transformation project to improve the food environment and consumer choices in Appalachian Kentucky. Prev Med Rep. 2017 Sep;7:187-92. doi:

10.1016/j.pmedr.2017.06.009. PMID: 28706778.

237. Sabia JJ, Nguyen TT, Rosenberg O.

High School Physical Education Requirements and Youth Body Weight: New Evidence from the YRBS. Health Econ. 2016 Aug 30doi: 10.1002/hec.3399. PMID: 27576770.

238. Sadler RC, Gilliland JA, Arku G. A

food retail-based intervention on food security and consumption. Int J Environ Res Public Health. 2013 Aug 05;10(8):3325-46. doi: 10.3390/ijerph10083325. PMID: 23921626.

239. Sallis JF, McKenzie TL, Conway

TL, et al. Environmental interventions for eating and physical activity: a randomized controlled trial in middle schools. Am J Prev Med. 2003 Apr;24(3):209-17. PMID: 12657338.

240. Sanchez-Vaznaugh EV, Sanchez

BN, Baek J, et al. 'Competitive' food and beverage policies: are they influencing childhood overweight trends? Health Aff (Millwood). 2010 Mar-Apr;29(3):436-46. doi: 10.1377/hlthaff.2009.0745. PMID: 20194985.

241. Schanzenbach D. Do School

Lunches Contribute to Childhood Obesity? : Harris School of Public Policy Studies, University of Chicago, Working Papers: 0513; 2005.

242. Schwartz AE, Leardo M, Aneja S, et

al. Effect of a School-Based Water Intervention on Child Body Mass

Page 173: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-25

Index and Obesity. JAMA Pediatr. 2016 Mar;170(3):220-6. doi: 10.1001/jamapediatrics.2015.3778. PMID: 26784336.

243. Sekhobo JP, Edmunds LS, Dalenius

K, et al. Neighborhood disparities in prevalence of childhood obesity among low-income children before and after implementation of New York City child care regulations. Prev Chronic Dis. 2014;11:E181. doi: 10.5888/pcd11.140152. PMID: 25321632.

244. Sharma SV, Markham C, Chow J, et

al. Evaluating a school-based fruit and vegetable co-op in low-income children: A quasi-experimental study. Prev Med. 2016 Oct;91:8-17. doi: 10.1016/j.ypmed.2016.07.022. PMID: 27471022.

245. Shive SE, Morris MN. Evaluation of

the energize your life! Social marketing campaign pilot study to increase fruit intake among community college students. J Am Coll Health. 2006 Jul-Aug;55(1):33-9. doi: 10.3200/jach.55.1.33-40. PMID: 16889313.

246. Sigmund E, El Ansari W,

Sigmundova D. Does school-based physical activity decrease overweight and obesity in children aged 6-9 years? A two-year non-randomized longitudinal intervention study in the Czech Republic. BMC Public Health. 2012 Jul 29;12:570. doi: 10.1186/1471-2458-12-570. PMID: 22892226.

247. Simon C, Schweitzer B, Oujaa M, et

al. Successful overweight prevention in adolescents by increasing physical

activity: a 4-year randomized controlled intervention. Int J Obes (Lond). 2008 Oct;32(10):1489-98. doi: 10.1038/ijo.2008.99. PMID: 18626482.

248. Slater S, Chriqui J, Chaloupka FJ, et

al. Joint use policies: are they related to adolescent behavior? Prev Med. 2014 Dec;69 Suppl 1:S37-43. doi: 10.1016/j.ypmed.2014.08.032. PMID: 25199731.

249. Spence S, Delve J, Stamp E, et al.

The impact of food and nutrient-based standards on primary school children's lunch and total dietary intake: a natural experimental evaluation of government policy in England. PLoS One. 2013;8(10):e78298. doi: 10.1371/journal.pone.0078298. PMID: 24205190.

250. Steenhuis I, Van Assema P, Van

Breukelen G, et al. The impact of educational and environmental interventions in Dutch worksite cafeterias. Health Promot Int. 2004 Sep;19(3):335-43. doi: 10.1093/heapro/dah307. PMID: 15306618.

251. Stephens RL, Xu Y, Lesesne CA, et

al. Relationship between child care centers' compliance with physical activity regulations and children's physical activity, New York City, 2010. Prev Chronic Dis. 2014 Oct 16;11:E179. doi: 10.5888/pcd11.130432. PMID: 25321630.

252. Story M, Hannan PJ, Fulkerson JA,

et al. Bright Start: Description and main outcomes from a group-

Page 174: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-26

randomized obesity prevention trial in American Indian children. Obesity (Silver Spring). 2012 Nov;20(11):2241-9. doi: 10.1038/oby.2012.89. PMID: 22513491.

253. Stratton G, Mullan E. The effect of

multicolor playground markings on children's physical activity level during recess. Prev Med. 2005 Nov-Dec;41(5-6):828-33. doi: 10.1016/j.ypmed.2005.07.009. PMID: 16137756.

254. Sturm R, Hattori A. Diet and obesity

in Los Angeles County 2007-2012: Is there a measurable effect of the 2008 "Fast-Food Ban"? Soc Sci Med. 2015 May;133:205-11. doi: 10.1016/j.socscimed.2015.03.004. PMID: 25779774.

255. Sturm R, Powell LM, Chriqui JF, et

al. Soda Taxes, Soft Drink Consumption, And Children's Body Mass Index. Health Affairs. 2010 May-Jun;29(5):1052-8. doi: 10.1377/hlthaff.2009.0061.

256. Taber DR, Chriqui JF, Chaloupka FJ.

Differences in nutrient intake associated with state laws regarding fat, sugar, and caloric content of competitive foods. Arch Pediatr Adolesc Med. 2012 May;166(5):452-8. doi: 10.1001/archpediatrics.2011.1839. PMID: 22566546.

257. Taber DR, Chriqui JF, Perna FM, et

al. Weight status among adolescents in States that govern competitive food nutrition content. Pediatrics. 2012 Sep;130(3):437-44. doi:

10.1542/peds.2011-3353. PMID: 22891223.

258. Taber DR, Chriqui JF, Perna FM, et

al. Association between state physical education (PE) requirements and PE participation, physical activity, and body mass index change. Prev Med. 2013 Nov;57(5):629-33. doi: 10.1016/j.ypmed.2013.08.018. PMID: 23978523.

259. Taber DR, Chriqui JF, Powell LM, et

al. Banning all sugar-sweetened beverages in middle schools: reduction of in-school access and purchasing but not overall consumption. Arch Pediatr Adolesc Med. 2012 Mar;166(3):256-62. doi: 10.1001/archpediatrics.2011.200. PMID: 22064875.

260. Taber DR, Stevens J, Evenson KR,

et al. State policies targeting junk food in schools: racial/ethnic differences in the effect of policy change on soda consumption. Am J Public Health. 2011 Sep;101(9):1769-75. doi: 10.2105/ajph.2011.300221. PMID: 21778484.

261. Tak NI, Te Velde SJ, Brug J. Ethnic

differences in 1-year follow-up effect of the Dutch Schoolgruiten Project - promoting fruit and vegetable consumption among primary-school children. Public Health Nutr. 2007 Dec;10(12):1497-507. doi: 10.1017/s1368980007000456. PMID: 17610757.

262. Tak NI, Te Velde SJ, Brug J. Long-

term effects of the Dutch Schoolgruiten Project--promoting

Page 175: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-27

fruit and vegetable consumption among primary-school children. Public Health Nutr. 2009 Aug;12(8):1213-23. doi: 10.1017/s1368980008003777. PMID: 18940029.

263. Tarp J, Domazet SL, Froberg K, et

al. Effectiveness of a School-Based Physical Activity Intervention on Cognitive Performance in Danish Adolescents: LCoMotion-Learning, Cognition and Motion - A Cluster Randomized Controlled Trial. PLoS One. 2016;11(6):e0158087. doi: 10.1371/journal.pone.0158087. PMID: 27341346.

264. Te Velde SJ, Brug J, Wind M, et al.

Effects of a comprehensive fruit- and vegetable-promoting school-based intervention in three European countries: the Pro Children Study. Br J Nutr. 2008 Apr;99(4):893-903. doi: 10.1017/s000711450782513x. PMID: 17953787.

265. Tester JM, Leung CW, Crawford PB.

Revised WIC Food Package and Children's Diet Quality. Pediatrics. 2016 May;137(5)doi: 10.1542/peds.2015-3557. PMID: 27244804.

266. Tomlin D, Naylor PJ, McKay H, et

al. The impact of Action Schools! BC on the health of Aboriginal children and youth living in rural and remote communities in British Columbia. Int J Circumpolar Health. 2012 Mar 19;71:17999. doi: 10.3402/ijch.v71i0.17999. PMID: 22456048.

267. Toussaint LL, Housholder K,

Janssen K, et al. Slowing BMI

Growth Trajectories in Elementary School-Aged Children: The Northeast Iowa Food and Fitness Initiative. Fam Community Health. 2017 Jul/Sep;40(3):192-7. doi: 10.1097/fch.0000000000000151. PMID: 28525438.

268. Utter J, Denny S, Dyson B. School

gardens and adolescent nutrition and BMI: Results from a national, multilevel study. Prev Med. 2016 Feb;83:1-4. doi: 10.1016/j.ypmed.2015.11.022. PMID: 26657347.

269. Vadiveloo MK, Dixon LB, Elbel B.

Consumer purchasing patterns in response to calorie labeling legislation in New York City. Int J Behav Nutr Phys Act. 2011 May 27;8:51. doi: 10.1186/1479-5868-8-51. PMID: 21619632.

270. Van Cauwenberghe E, De

Bourdeaudhuij I, Maes L, et al. Efficacy and feasibility of lowering playground density to promote physical activity and to discourage sedentary time during recess at preschool: a pilot study. Prev Med. 2012 Oct;55(4):319-21. doi: 10.1016/j.ypmed.2012.07.014. PMID: 22846504.

271. Vander Ploeg KA, McGavock J,

Maximova K, et al. School-based health promotion and physical activity during and after school hours. Pediatrics. 2014 Feb;133(2):e371-8. doi: 10.1542/peds.2013-2383. PMID: 24420806.

272. Vasquez A, Sherwood NE, Larson

N, et al. A novel dietary

Page 176: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-28

improvement strategy: examining the potential impact of community-supported agriculture membership. Public Health Nutr. 2016 Oct;19(14):2618-28. doi: 10.1017/s1368980015003638. PMID: 26857484.

273. Veugelers PJ, Fitzgerald AL.

Effectiveness of school programs in preventing childhood obesity: a multilevel comparison. Am J Public Health. 2005 Mar;95(3):432-5. doi: 10.2105/ajph.2004.045898. PMID: 15727972.

274. Waters E, Gibbs L, Tadic M, et al.

Cluster randomised trial of a school-community child health promotion and obesity prevention intervention: findings from the evaluation of fun 'n healthy in Moreland! BMC Public Health. 2017 Aug 03;18(1):92. doi: 10.1186/s12889-017-4625-9. PMID: 28774278.

275. Weaver RG, Webster CA, Egan C, et

al. Partnerships for Active Children in Elementary Schools: Outcomes of a 2-Year Pilot Study to Increase Physical Activity During the School Day. Am J Health Promot. 2017 Jan 01:890117117707289. doi: 10.1177/0890117117707289. PMID: 28482678.

276. Webb E, Laverty A, Mindell J, et al.

Free Bus Travel and Physical Activity, Gait Speed, and Adiposity in the English Longitudinal Study of Ageing. Am J Public Health. 2016 Jan;106(1):136-42. doi: 10.2105/ajph.2015.302907. PMID: 26562118

277. Webb E, Netuveli G, Millett C. Free bus passes, use of public transport and obesity among older people in England. J Epidemiol Community Health. 2012 Feb;66(2):176-80. doi: 10.1136/jech.2011.133165. PMID: 21911850.

278. Wells L, Nelson M. The National

School Fruit Scheme produces short-term but not longer-term increases in fruit consumption in primary school children. Br J Nutr. 2005 Apr;93(4):537-42. PMID: 15946417.

279. Wells NM, Myers BM, Henderson

CR, Jr. School gardens and physical activity: a randomized controlled trial of low-income elementary schools. Prev Med. 2014 Dec;69 Suppl 1:S27-33. doi: 10.1016/j.ypmed.2014.10.012. PMID: 25456803.

280. Wendel ML, Benden ME, Zhao H, et

al. Stand-Biased Versus Seated Classrooms and Childhood Obesity: A Randomized Experiment in Texas. Am J Public Health. 2016 Oct;106(10):1849-54. doi: 10.2105/ajph.2016.303323. PMID: 27552276.

281. West ST, Shores KA. The impacts of

building a greenway on proximate residents' physical activity. J Phys Act Health. 2011 Nov;8(8):1092-7. PMID: 22039127.

282. Whaley SE, McGregor S, Jiang L, et

al. A WIC-Based Intervention to Prevent Early Childhood Overweight. J Nutr Educ Behav. 2010 May-Jun;42(3):S47-S51. doi:

Page 177: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-29

10.1016/j.jneb.2010.02.010. PMID: 20399409

283. Whetstone LM, Kolasa KM, Collier

DN. Participation in community-originated interventions is associated with positive changes in weight status and health behaviors in youth. Am J Health Promot. 2012 Sep-Oct;27(1):10-6. doi: 10.4278/ajhp.100415-QUAN-117. PMID: 22950920.

284. Whitt-Glover MC, Ham SA, Yancey

AK. Instant Recess(R): a practical tool for increasing physical activity during the school day. Prog Community Health Partnersh. 2011 Fall;5(3):289-97. doi: 10.1353/cpr.2011.0031. PMID: 22080777.

285. Williamson DA, Champagne CM,

Harsha DW, et al. Effect of an environmental school-based obesity prevention program on changes in body fat and body weight: a randomized trial. Obesity (Silver Spring). 2012 Aug;20(8):1653-61. doi: 10.1038/oby.2012.60. PMID: 22402733.

286. Wilson DK, Van Horn ML, Siceloff

ER, et al. The Results of the "Positive Action for Today's Health" (PATH) Trial for Increasing Walking and Physical Activity in Underserved African-American Communities. Ann Behav Med. 2015 Jun;49(3):398-410. doi: 10.1007/s12160-014-9664-1. PMID: 25385203.

287. Woodward-Lopez G, Gosliner W,

Samuels SE, et al. Lessons learned from evaluations of California's

statewide school nutrition standards. Am J Public Health. 2010 Nov;100(11):2137-45. doi: 10.2105/ajph.2010.193490. PMID: 20864696.

288. Wright K, Giger JN, Norris K, et al.

Impact of a nurse-directed, coordinated school health program to enhance physical activity behaviors and reduce body mass index among minority children: a parallel-group, randomized control trial. Int J Nurs Stud. 2013 Jun;50(6):727-37. doi: 10.1016/j.ijnurstu.2012.09.004. PMID: 23021318.

289. Wright K, Norris K, Newman Giger

J, et al. Improving healthy dietary behaviors, nutrition knowledge, and self-efficacy among underserved school children with parent and community involvement. Child Obes. 2012 Aug;8(4):347-56. doi: 10.1089/chi.2012.0045. PMID: 22867074.

290. Wrigley N, Warm D, Margetts B.

Deprivation, Diet, and Food-Retail Access: Findings from the Leeds ‘Food Deserts' Study. Environ Plann A 2003;35(1):151-88.

291. Yildirim M, Arundell L, Cerin E, et

al. What helps children to move more at school recess and lunchtime? Mid-intervention results from Transform-Us! cluster-randomised controlled trial. Br J Sports Med. 2014 Feb;48(3):271-7. doi: 10.1136/bjsports-2013-092466. PMID: 24124036.

292. Zhou Z, Ren H, Yin Z, et al. A

policy-driven multifaceted approach for early childhood physical fitness

Page 178: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

G-30

promotion: impacts on body composition and physical fitness in young Chinese children. BMC Pediatr. 2014 May 05;14:118. doi: 10.1186/1471-2431-14-118. PMID: 24886119.

293. Zhu X, Lu Z, Yu C, et al. Walkable

communities: impacts on residents’ physical and social health. World Health Des 2013;7:68-75.

294. Zhu X, Yu CY, Lee C, et al. A retrospective study on changes in residents' physical activities, social interactions, and neighborhood cohesion after moving to a walkable community. Prev Med. 2014 Dec;69 Suppl 1:S93-7. doi: 10.1016/j.ypmed.2014.08.013. PMID: 25158208.

Page 179: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-1

Appendix H. Evidence Tables Evidence Table H1a. Intervention descriptions of included studies that follow a natural experiment method

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Anderson, 20131 Child Nutrition and WIC Reauthorization Act (CNWICRA) Physical activity environment School Child US

Anthamatten, 20112 Learning Landscape Physical activity environment School Child US

Azevedo, 20143 Dance mat systems in public schools Pysical and built environment School Child Non-US

Barnidge, 20134 Healthier Missouri Communities Food and beverage environment Community or neighborhood Adult US

Barroso, 20095 Texas Senate Bill 42 (SB42) Physical activity environment School Child US

Bauhoff, 20146 Senate Bill SB 677; Los Angeles Unified School District Nutrition Policy

Food and beverage environment School Child US

Bauman, 20037 ‘Push Play’ initiative Messaging environment Community or neighborhood Adult Non-US

Bere, 20108 Fruits and Vegetables Make the Marks Food and beverage environment School Child Non-US

Berger-Jenkins, 20149 Choosing Healthy & Active Lifestyles for Kids (CHALK) Food and beverage environment

Physical activity environment

School Child US

Bolton, 201710 Health-Promoting Communities: Being Active Eating Well Physical activity environment

School and early child care environment

Social marketing environment

Other or multiple (define) Child Non-US

Bowling, 201611 Healthy Foods, Healthy Families (HFHF) Food and beverage environment Community or neighborhood

Food assistance programs

Adult US

Branas, 201112 Vacant lot greening program Physical and Built environment

Parks and recreation environment

Community or neighborhood Adult US

Page 180: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-2

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Brown, 200813 New urbanist neighborhood (in comparison to traditional suburban neighborhood)

Physical and Built environment

Parks and recreation environment

Community or neighborhood Adult US

Brown, 201514 Moving Across Places Physical and Built environment

Transportation environment

Community or neighborhood Adult US

Brown, 201615 Complete streets design Physical and Built environment

Transportation environment

Community or neighborhood Adult US

Brown, 201616 Moving Across Places Physical and Built environment

Transportation environment

Community or neighborhood Adult US

Burke, 201417 The HealthMPowers program Food and beverage environment

Physical activity environment

School Child US

Buscail, 201618 “For health, I move in my neighborhood!” Physical activity environment Community or neighborhood Adult Non-US

Caldwell, 200919 Colorado Trust Food and beverage environment Community or neighborhood Both US

Page 181: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-3

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Calise, 201320 Residents moving to a new urbanist neighborhood Physical and Built environment

Parks and recreation environment

Transportation environment

Community or neighborhood Adult US

Camacho-Rivera, 201721 AHOME Food and beverage environment Food assistance programs Adult US

Capogrossi, 201622 School Breakfast Program (SBP) and National School Lunch program

Food and beverage environment School Child US

Cawley, 200723 YRBSS merged with state policies Physical activity environment School Child US

Cawley, 200724 YRBSS merged with state policies Physical activity environment School Child US

Chen, 201525 The Home Appliances Going to the Countryside Policy changes Community or neighborhood Adult Non-US

Cleland, 200826 Compulsory school physical activity Physical activity environment School Adult Non-US

Coffield, 201127 Child Nutrition and WIC Reauthorization Act (CNRA) Policy changes School Child US

Cohen, 201228 Fitness Zone Physical activity environment

Parks and recreation environment

Community or neighborhood Both US

Cohen, 201429 Creating Healthy, Active and Nurturing Growing-up Environments (CHANGE) study: serve healthier school breakfast and lunch

Food and beverage environment School Child US

Coyle, 200930 Mississippi Fresh Fruit and Vegetable Program (MFVP) Food and beverage environment School Child US

Cradock, 201131 Boston Public Schools Snack and Beverage Policy Food and beverage environment School Child US

Cradock, 201432 Boston Active School Day Policy Physical activity environment School Child US

Cullen, 200633 National School-lunch program and local school-level food policy changes

Food and beverage environment School Child US

Cullen, 200834 Texas Public School Nutrition Policy Food and beverage environment School Child US

Page 182: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-4

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Cummins, 200535 Impact of a super store on a community's food intake Food and beverage environment Community or neighborhood Adult Non-US

Cummins, 200836 New food retail development on a deprived area Physical and Built environment Community or neighborhood Both Non-US

Cummins, 201437 Pennsylvania Fresh Food Financing Initiative Physical and Built environment Community or neighborhood Adult US

Datar, 201638 Competitive foods and beverages policies in public schools in multple states: NY, WA, NC, GA, TX, KY, TN, LA, GA, OK, KS

Food and beverage environment School (include summer school and after care)

Child US

De Cocker, 200739 10,000 Steps Ghent--to increas PA Messaging environment Community or neighborhood Adult Non-US

de Visser, 201640 Project Healthy Schools in two Cities in Michigan Food and beverage environment

Physical activity environment

Messaging environment

School Child US

Page 183: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-5

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Dill, 201441 installation of new bicycle boulevards. Physical and Built environment

Transportation environment

Community or neighborhood Both US

Dubowitz, 201542 Healthy Food Financing Initiative Physical and Built environment

Food and beverage environment

Community or neighborhood Adult US

Elbel, 201543 New York City's Food Retail Expansion to Support Health (FRESH)

Food and beverage environment Community or neighborhood Adult US

Elbel, 201744 FRESH: opening a 17,000 square foot supermarket receiving FRESH program incentives

Physical and Built environment, Food and beverage environment,

Community or neighborhood Adult US

Falbe, 201645 Excise tax on SSB Food and beverage environment Community or neighborhood Adult US

Fitzpatrick, 201746 Food practices and policies within schools and neighborhoods Food and beverage environment School (include summer school and after care)

Child Non-US

Flego, 201447 Jamie's Ministry of Food Food and beverage environment Community or neighborhood Adult Non-US

Fogarty, 200748 National Schools Fruit Scheme (NSFS) Food and beverage environment School Child Non-US

Fox, 200949 National School Lunch Program Food and beverage environment School Child US

Frongillo, 201750 Healthy Communities Study (HCS) Other Community or Neighborhood Child US

Fuller, 201351 BIcycle-taXI: a public bicycle share program Physical activity environment

Transportation environment

Community or neighborhood Adult Non-US

Fung, 201352 The Food and Nutrition Policy for Nova Scotia Public Schools Food and beverage environment School Child Non-US

Gee, 201553 Act 1220 (Arkansas): BMI screening Policy changes School Child US

Gibson, 200654 Food Stamp program Food and beverage environment Food assistance programs Child US

Page 184: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-6

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Giles-Corti, 201355 RESIDential Environmental Project (RESIDE) Physical and Built environment

Parks and recreation environment

Transportation environment

Community or neighborhood Adult Non-US

Gleason, 200956 School meals Food and beverage environment School (include summer school and after care)

Child US

Goldsby, 201657 Railroad Park Physical and Built environment

Parks and recreation environment

Community or neighborhood Child US

Goodman, 201458 a traffic-free bridge was built over Cardiff Bay; Kenilworth, a traffic-free bridge was built over a busy trunk road; and Southampton, an informal riverside footpath was turned into a boardwalk

Physical and Built environment

Transportation environment

Community or neighborhood Adult Non-US

Goodman, 201659 Bikeability- cycle training in schools Physical activity environment Transportation environment Child Non-US

Page 185: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-7

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Gorely, 201160 GreatFun2Run Messaging environment School Child Non-US

Gorham, 201561 Fresh to You Markets Food and beverage environment Community or neighborhood Both US

Harding, 201762 The Mālaekahana Bike Path Physical and Built environment Community or neighborhood Adult US

Heelan, 201563 Kearney Public School (KPS) District; population dose study Food and beverage environment

Physical activity environment

School Child US

Hennessy, 201464 Multiple state laws for 50 states and DC Policy changes School Child US

Herrick, 201265 Sports, Play, and Active Recreation for Kids (SPARK) Physical activity environment School Child US

Hilmers, 201466 The Supplemental Nutrition Assistance Program (SNAP) Food and beverage environment Community or neighborhood

Food assistance program

Adult US

Hobin, 201467 Mandatory province-wide PE policy Physical activity environment School Child Non-US

Hobin, 201768 Province PE policy on MVPA (moderate to vigorous physical activity)

Physical and Built environment School (include summer school and after care)

Child Non-US

Hoelscher, 201669 Fuel Up to Play 60 (FUTP60) Food and beverage environment School Child Non-US

Hoelscher, 201670 2005 Safe Routes to School (SRTS) initiative Physical activity environment School Child US

Howlett, 201671 SNAP (Food stamp program) Food and beverage environment Food assistance programs Child US

Hu, 201672 Reducing Health inequalities : an action report and Tackling Health Inequalities: a program of Action

Policy changes Community or neighborhood Adult Non-US

Hughes, 201273 The School Fruit and Vegetable Scheme (SFVS) Food and beverage environment School Child Non-US

Hunter, 201674 COMPASS: Changes to PA policies, recreational programming, use of public health units (i.e., a government health agency that carries out community health programs), and environment/equipment

Physical activity environment, School and early child care environment,

School (include summer school and after care)

Child Non-US

Page 186: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-8

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Jennings, 201275 Mobile Food Store (MFS) Food and beverage environment Community or neighborhood Adult Non-US

Jia, 201776 Childhood Obesity Study in China Megacities (COCM) Food and beverage environment School (include summer school and after care)

Child Non-US

Johnson, 201777 One Body One Life (OBOL) healthy lifestyle programme Physical activity environment Community or neighborhood Adult Non-US

Just, 201478 Food purchases by debit-card Food and beverage environment School Child Non-US

Kern, 201479 Communities Putting Prevention to Work (CPPW) Food and beverage environment

Physical activity environment

School Child US

Keyte, 201280 National Healthy Schools Progamme Food and beverage environment School Child Non-US

Kim, 201281 Changing food offerings at school tuck shops Food and beverage environment School Child Non-US

Kim, 201282 Physical education state policies Policy changes School Child US

Page 187: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-9

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

King, 201483 HEROES (Healthy, Energetic, Ready, Outstanding, Enthusiastic, Schools) Initiative

Food and beverage environment School Child US

Kubik, 200584 Schoolwide food policies and practices Food and beverage environment School Child US

Lachapelle, 200985 Mean distance walked per day for transportation purposes Physical and Built environment

Transportation environment

Community or neighborhood Adult US

LaRowe, 201686 The Active Early program Physical activity environment School Child US

Leung, 201387 Supplemental Nutrition Assistance Program (SNAP), formerly the Food Stamp Program

Food and beverage environment Food assistance programs Child US

Liao, 201588 Reach 2010 Food and beverage environment Community or neighborhood Adult US

Ling, 201489 Comprehensive school-based intervention on healthy behavior Food and beverage environment School Child US

Liu, 201690 WIC and SNAP Food and beverage environment Food assistance programs Child US

MacDonald, 201091 Light-rail transit use Physical and Built environment

Transportation environment

Community or neighborhood Adult US

Maddock, 200692 Coordinated School Health Program (CSHP) Physical activity environment

Messaging environment

School Child US

Madsen, 201193 Let’s Go! Messaging environment School Child US

Madsen, 201594 Healthy Schools Program (HSP) Food and beverage environment

Physical activity environment

School Child US

Malakellis, 201795 The Australian Capital Territory (ACT) Food and beverage environment, Physical activity environment,

School (include summer school and after care)

Child Non-US

Masse, 201496 Healthier nutrition guidelines Food and beverage environment School Child US

Page 188: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-10

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Miewald, 201297 Food Box Program ( Fraser Region Harvest Box Program) Food and beverage environment Community or neighborhood Adult Non-US

Miller, 201598 Light rail transit use Physical activity environment

Transportation environment

Community or neighborhood Adult US

Molitor, 201599 Supplemental Nutrition Assistance Program–Education (SNAP-ED)

Food and beverage environment Food assistance programs Both US

Morton, 2016100 SPEEDY Physical activity environment School (include summer school and after care)

Child Non-US

Mullally, 2010101 Prince Edward Island school nutrition policy Food and beverage environment School Child Non-US

Mumford, 2011102 mixed-use redevelopment community in metropolitan Atlanta (Atlantic Station) that promotes walking and physical activity

Physical and Built environment

Parks and recreation environment

Transportation environment

Community or neighborhood Adult US

Nanney, 2014103 The School Obesity-related Policy Evaluation study (ScOPE) Policy changes School Child US

Page 189: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-11

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Nanney, 2016104 School-specific policies/program Food and beverage environment

Physical activity environment

School Child US

Neelon, 2015105 Mebane on the Move Intervention Physical and Built environment Community or neighborhood Child US

Nehme, 2017106 Workplace showers Physical and Built environment Employer or workplace Adult US

Nguyen, 2015107 Supplemental Nutrition Assistance Program (SNAP) Food and beverage environment Food assistance programs Adult US

Oh, 2015108 Analysis of CLASS and NSCH Physical activity environment School Child US

Olsho, 2015109 Health Bucks: Farmers' market incentive program Food and beverage environment Food assistance programs Adult US

Panter, 2016110 The Cambridge Guided Busway Physical and Built environment

Parks and recreation environment

Transportation environment

Community or neighborhood Adult Non-US

Parsons, 2014111 Anchorage School District’s School Wellness Policy Policy changes School Child US

Peterson, 2015112 Healthy Choice Food and beverage environment

Physical activity environment

School Child US

Powell, 2009113 State-level grocery story and vending machine soda taxes Food and beverage environment School Child US

Quig, 2012114 playground upgrade program Physical and Built environment Community or neighborhood Child US

Reger-Nash, 2005115 Wheeling Walks Physical activity environment Community or neighborhood Adult US

Reger-Nash, 2008116 A social marketing intervention promoted walking

Messaging environment Community or neighborhood Adult US

Restrepo, 2016117 Calorie Labeling Laws in New York City jurisdictions Messaging environment Community or neighborhood Adult US

Ridgers, 2007118 Sporting Playgrounds' Initiative: playground redesign intervention across time on children's recess physical activity levels

Physical and Built environment School Child Non-US

Page 190: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-12

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Ridgers, 2007119 Sporting Playgrounds' Initiative Physical activity environment School Child Non-US

Riis, 2012120 results of different school nutrition policies Policy changes School Child US

Ritchie, 2016121 Breakfast service policy Food and beverage environment School Child US

Robles, 2017122 Electronic Benefit Transfer (EBT) at farmers markets Food and beverage environment Community or neighborhood Adult US

Rushakoff, 2017123 Healthy2Go: received training and technical assistance to increase availability and awareness of healthy foods

Food and beverage environment Community or neighborhood Child US

Sabia, 2016124 state high school physical education requirements Physical activity environment School Child US

Sadler, 2013125 an independent grocery store (Witherbee’s Market) at the center of the Carriage Town neighborhood

Physical and Built environment Community or neighborhood Adult US

Sanchez-Vaznaugh, 2010126 California Childhood Obesity Prevention Act: Obesity Prevention Motion; Obesity Prevention Motion

Food and beverage environment School Child US

Page 191: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-13

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Schanzenbach, 2005127 National School Lunch Program Food and beverage environment School Child US

Schwartz, 2016128 Michigan State Board of Education (MSBE) nutrition policy Physical and Built environment School Child US

Sekhobo, 2014129 WIC enrolled children Physical and Built environment Community or neighborhood Child US

Slater, 2014130 Joint Use Policies Policy changes School Child US

Spence, 2013131 Introduction of food and nutrient-based standards Food and beverage environment School Child Non-US

Stephens, 2014132 Regulations governing minimum PA standards in child care centers

Physical activity environment School Child US

Stratton, 2005133 multicolor school playground markings Physical activity environment School Child Non-US

Sturm, 2010134 State Soda Tax Food and beverage environment School Child US

Sturm, 2015135 Los Angeles Fast Food ban Food and beverage environment Community or neighborhood Adult US

Taber, 2011136 Policy Changes Targeting Junk Food in School vending machines Food and beverage environment School Child US

Taber, 2012137 Laws governing competitive food nutrition content Physical activity environment School Child Non-US

Taber, 2012138 All Sugar Sweetened Beverage Ban and the only soda ban Food and beverage environment School Child US

Taber, 2013139 State competitive food laws and state physical education laws Food and beverage environment

Physical activity environment

School Child US

Tak, 2007140 Schoolgruiten project Food and beverage environment School Child Non-US

Tak, 2009141 Schoolgruiten Project, a Dutch primary school-based intervention providing free fruit and vegetables (F&V).

Food and beverage environment School Child Non-US

Tester, 2016142 WIC Food and beverage environment Food assistance programs Child US

Page 192: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-14

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

Toussaint, 2017143 The Northwest Iowa Food and Fitness Initiative: Regional Safe Routes to School Programming (see www.saferoutestoschools.org), Walking School Buses, and bike rodeos

Physical and Built environment, Food and beverage environment, Physical activity environment

Community or neighborhood Child US

Utter, 2016144 School gardens Food and beverage environment School (include summer school and after care)

Child Non-US

Vadiveloo, 2011145 Calorie-labeling policy Messaging environment Community or neighborhood Adult US

Veugelers, 2005146 Annapolis Valley Health Promoting School Project (AVHPSP) Food and beverage environment School Child Non-US

von Hippel, 2015147 Texas Fitness Now (TFN) Physical activity environment School Child US

Webb, 2012148 English longitudinal study of ageing Physical and Built environment

Transportation environment

Community or neighborhood Adult Non-US

Webb, 2016149 Free bus transportation for older people Physical activity environment

Transportation environment

Community or neighborhood Adult Non-US

Wells, 2005150 National School Fruit Scheme (NSFS) Food and beverage environment School Child Non-US

Page 193: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-15

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change Target of intervention Adult/Child/Both US or Non-US

West, 2011151 Building new greenway Physical and Built environment

Parks and recreation environment

Community or neighborhood Adult US

Whetstone, 2012152 North Carolina Health and Wellness Trust Fund (NCHWTF) Food and beverage environment

Physical activity environment

School Child US

Woodward-Lopez, 2010153 Senate Bill 12 (SB 12), California Food and beverage environment School Child US

Wrigley, 2003154 Food Deserts in British Cities project ( Seacroft project) Food and beverage environment Community or neighborhood Adult Non-US

Zhu, 2013155 LEED for Neighborhood Development (LEED-ND) Physical and Built environment

Transportation environment

Community or neighborhood Adult US

Zhu, 2014156 Moving to a walkable community Physical and Built environment

Transportation environment

Community or neighborhood Adult US

BMI=Body Mass Index; No.=number; PE=physical education; SNAP= Supplemental Nutrition Assistance Program; SSB=sugar sweetened beverage; US=United States; WIC=Women, Infants, and Children program; YRBSS=Youth Risk Behavior Surveillance System

Page 194: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-16

Evidence Table H1b. Intervention descriptions of included studies that follow an experimental method

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Alaimo, 2013157 Michigan State Board of Education (MSBE) nutrition policy Food and beverage environment

Messaging environment

School Child US

Anderson, 2001158 Michigan Farmers' Market Nutrition Program Food and beverage environment Community or neighborhood Adult US

Angelopoulos, 2009159 The CHILDREN study Food and beverage environment

Physical activity environment

School Child Non-US

Ask, 2010160 Free School lunch Food and beverage environment School Child Non-US

Audrey, 2015161 Walk to Work Physical and Built environment

Physical activity environment

Transportation environment

Employer or workplace Adult Non-US

Ayala, 2013162 Fruit and vegetable promotion in stores with staff training and installing new equipment

Physical and Built environment

Messaging environment

Community or neighborhood Adult US

Backman, 2011163 Fresh fruit availability at worksites Food and beverage environment Employer or workplace Adult US

Baker, 2016164 Men on the Move: Growing Communities (MOTMGC) Food and beverage environment Community or neighborhood Adult US

Bastian, 2015165 APPLE Physical activity environment School Child US

Bere, 2005166 School Fruit Programme Food and beverage environment School Child Non-US

Bere, 2006167 Norwegian School Fruit Programme ( no cost to parents) AND Fruit and Vegetables Make the Marks (FVMM) educational programme ( Combined)

Food and beverage environment School Both Non-US

Bere, 2006168 Fruits and Vegetables Make the Marks intervention Food and beverage environment School Child Non-US

Page 195: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-17

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Bere, 2007169 Fruits and Vegetables Make the Marks (FVMM) Food and beverage environment School Child Non-US

Beresford, 2010170 The 5 a Day intervention: newsletters and promotions to encourage healthy eating

Messaging environment Employer or workplace Adult US

Blum, 2008171 Reduced availability of sugar sweetened beverage and diet soda in a la carte and vending programs in Maine public high schools.

Food and beverage environment School Child US

Bonsergent, 2013172 The PRomotion de l’ALImentation et de l’ActivitéPhysique (PRALIMAP) trial

Food and beverage environment

Physical activity environment

School Child Non-US

Bonvin, 2013173 “Youp’là Boug" Physical activity environment School Child Non-US

Busch, 2015174 Utrecht Healthy School Food and beverage environment School Child Non-US

Caballero, 2003175 Pathways Program (food protion focused on providing lower fat foods)

Food and beverage environment

Physical activity environment

School Child US

Chomitz, 2010176 Healthy Living Cambridge Kids (KLCK) Food and beverage environment

Physical activity environment

School

Community or neighborhood

Child US

Cochrane, 2012177 NHS Health Check program + additional lifestyle support vs. NHS health check alone

Healthcare environment

Physical activity environment

Primary care centers Adult Non-US

Page 196: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-18

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Coleman, 2012178 The Healthy Options for Nutrition Environments in Schools (Healthy ONES)

Food and beverage environment School Child US

Cortinez-O'Ryan, 2017179 “Juega en tu Barrio” (Play in your Neighborhood)

Physical activity environment Community or neighborhood Child Non-US

Crespo, 2012180 Community Health Advisor Food and beverage environment

Physical activity environment

School Child US

Day, 2008181 Action Schools! BC - Healthy Eating, a school-based fruit and vegetable (FV) intervention

Food and beverage environment School Child Non-US

De Coen, 2012182 Prevention of Overweight among pre-school and school children (POP) using the Healthy food framework

Food and beverage environment School Child Non-US

de Greeff, 2016183 Fit en Vaardig op school Physical activity environment School (include summer school and after care)

Child Non-US

De Henauw, 2015184 The Identification and prevention of Dietary- and lifestyle-induced health EFfects In Children and infantS (IDEFICS)

Food and beverage environment

Physical activity environment

School

Community or neighborhood

Child Non-US

de Meij, 2011185 JUMP-in Physical activity environment

Physical and built environment

School Child Non-US

Dunton, 2015186 Physical activity intervention Physical activity environment School Child US

Dzewaltowski, 2009187 The Healthy Youth Places Food and beverage environment

Physical activity environment

Physical and built environment

School Both US

Eagle, 2013188 Project Healthy Schools Food and beverage environment School Child US

Page 197: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-19

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Economos, 2007189 Shape-Up Somerville Food and beverage environment

Physical activity environment

Community or neighborhood Child US

Elinder, 2012190 Stockholm County Implementation Programme in school (SCIP-school)

Food and beverage environment

Physical activity environment

School Child Non-US

Eriksen, 2003191 Fruit and vegetable subscription Food and beverage environment School Child Non-US

Ermetici, 2016192 The Italian EAT project Food and beverage environment

Messaging environment

School Child Non-US

Esquivel, 2016193 Children’s Healthy Living Program for Remote Underserved Minority Populations in the Pacific Region (CHL)

Policy change School Child US

Evans, 2013194 Project Tomato: environment of the school promotes the eating of fruit and vegetables

Food and beverage environment School Child Non-US

Fairclough, 2016195 Born to Move Physical activity environment School (include summer school and after care)

Child Non-US

Page 198: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-20

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Farley, 2007196 Opening access to schoolyard outside of regular operation hours Physical and Built environment Community or neighborhood Child US

Farmer, 2017197 Cluster randomized intervention called PLAY Physical activity environment School (include summer school and after care)

Child Non-US

Finch, 2014198 Multi-level intervention to influence children’s’ PA behaviors Physical activity environment School Child US

Foster, 2008199 School Nutrition Policy Initiative (SNPI) Food and beverage environment School Child US

French, 2010200 Route H Food and beverage environment

Physical activity environment

Employer or workplace Adult US

Fu, 2016201 SPARK (Sports, Play and Active Recreation for Kids) Physical activity environment School (include summer school and after care)

Child US

Gatto, 2017202 LA Sprouts: gardening, nutrition and cooking intervention Food and beverage environment School (include summer school and after care)

Child US

Geaney, 2016203 Food Choice at Work Food and beverage environment Employer or workplace Adult Non-US

Gittelsohn, 2010204 The Healthy Foods Hawaii (HFH) Physical and Built environment

Food and beverage environment

Community or neighborhood Adult US

Gittelsohn, 2013205 Navajo Healthy Stores (NHS) program Physical and Built environment

Food and beverage environment

Community or neighborhood Adult US

Goetzel, 2010206 Health promotion messaging and counseling Messaging environment Employer or workplace Adult US

Gustat, 2012207 Partnership for an Active Community Environment (PACE) and city of New Orleans

Physical and Built environment

Parks and recreation environment

Community or neighborhood Adult US

Page 199: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-21

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Haerens, 2006208 Physical Activity and Healthy Food Intervention Food and beverage environment

Physical activity environment

School Child US

Haerens, 2007209 A School-based intervention program on healthy food Food and beverage environment School Child Non-US

Hardy, 2010210 Munch and Move Food and beverage environment

Physical activity environment

School Child US

He, 2009211 the Northern Fruit and Vegetable Pilot Programme (NFVPP) Food and beverage environment School Child Non-US

Hendy, 2011212 Kid's Choice Program Physical activity environment School Child US

Hoefkens, 2011213 Posting point-of-purchase nutrition-information Messaging environment University Adult Non-US

Hollis, 2016214 Physical Activity 4 Everyone Physical activity environment School Child US

Huberty, 2011215 Ready for Recess Physical activity environment School Child US

Jago, 2011216 The HEALTHY Intervention Physical activity environment School Child US

Janssen, 2015217 PLAYgrounds Physical activity environment School Child Non-US

Jones, 2015218 Increase implementation of healthy eating and PA policies in child-care services

Physical activity environment School Child Non-US

Page 200: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-22

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Jordan, 2008219 Gold Medal Schools program Other School Child US

Jurg, 2006220 JUMP-in Physical activity environment School Child US

Kain, 2004221 'Healthy School' initiative Food and beverage environment

Physical activity environment

School Child Non-US

Kamada, 2013222 The COMMUNICATE (COMMUNIty-wide Campaign To promote Exercise)

Messaging environment Community or neighborhood Adult US

Kastorini, 2016223 Diatrofi Food and beverage environment School Child Non-US

Kloek, 2006224 Working on Healthy Neighborhoods Physical activity environment

Food and beverage environment

Messaging environment

Community or neighborhood Adult Non-US

LaCaille, 2016225 Go! Food and beverage environment

Physical activity environment

Employer or workplace Adult US

Lemon, 2010226 Step Ahead trial Food and beverage environment

Physical activity environment

Messaging environment

Employer or workplace Adult US

Lemon, 2014227 Multi-level weight-gain prevention Food and beverage environment

Food and beverage environment

Employer or workplace Adult US

Page 201: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-23

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Lent, 2014228 Healthy Store Intervention Physical and Built environment

Food and beverage environment

School Child US

Linde, 2012229 HealthWorks Food and beverage environment

Messaging environment

Employer or workplace Adult US

Llargues, 2011230 The Avall study: educational material on healthy food and necessary equipment to facilitate educational games on healthy foods

Food and beverage environment School Child Non-US

Lorentzen, 2009231 The “Romsås in Motion” study: labeled walking paths, street lighting, gritting of walking paths in winter

Physical and Built environment

Physical activity environment

Messaging environment

Community or neighborhood Adult Non-US

Lubans, 2016232 ATLAS Physical activity environment

Messaging environment

School Child US

Ludwig, 2011233 HUD assignment for urban housing. Physical and Built environment Community or neighborhood Adult US

Lv, 2014234 Community Interventions for Health: build walking trails, bike service system

Messaging environment

Work environment

Physical and Built environment

Community or neighborhood Adult US

Madsen, 2013235 America SCORES Physical activity environment School Child US

Madsen, 2015236 Energy Balance for Kids with Play (EB4K with Play) Food and beverage environment

Physical activity environment

School Child US

Page 202: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-24

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Mead, 2013237 Healthy Foods North Physical and Built environment

Food and beverage environment

Community or neighborhood Adult Non-US

Morrill, 2016238 Food Dudes Food and beverage environment School Child US

Murphy, 2011239 Welsh Assembly Government’s Primary School Free Breakfast Initiative (PSFBI)

Food and beverage environment School Child Non-US

Naylor, 2006240 Action Schools Physical activity environment School Child Non-US

Naylor, 2008241 Action Schools BC Physical activity environment School Child Non-US

Neumark-Sztainer, 2010242 New Moves intervention Physical activity environment School Child US

Nicklas, 2017243 Head Start: classroom videotaped (DVD) puppet shows. Social marketing environment School (include summer school and after care)

Child US

Ortega, 2016244 Proyecto Mercado FRESCO Food and beverage environment Community or neighborhood Adult US

Pate, 2005245 Lifestyle Education for Activity Program Physical activity environment School Child US

Pbert, 2016246 Lookin’ Good Feelin’ Good: School nurse intervention and after-school exercise program

Physical activity environment School Child US

Perry, 2004247 The Cafeteria Power Plus Intervention Food and beverage environment School Child US

Ploeg, 2014248 Sporting Playgrounds' Initiative: The Alberta Project Promoting Active Living and Healthy Eating in Schools (APPLE Schools)

Physical activity environment School Child Non-US

Pope, 2016249 Project CHOICE (Center for Healthy Options and Community Empowerment)- community-based participatory research study

Food and beverage environment School (include summer school and after care)

Child US

Ransley, 2007250 School fruit and vegetable scheme (SFVS) Food and beverage environment School Child Non-US

Page 203: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-25

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Reilly, 2006251 Movement and Activity Glasgow Intervention in Children (MAGIC)

Physical activity environment School Child Non-US

Reynolds, 2000252 High 5 Food and beverage environment School Child US

Ridgers, 2010253 "National 10 million sporting playground initiative" Physical activity environment School Child Non-US

Rush, 2014254 Project Energize Food and beverage environment

Physical activity environment

School Child Non-US

Sallis, 2003255 Middle-School Physical Activity and Nutrition (M-SPAN) Food and beverage environment School Child US

Sharma, 2016256 Brighter Bites Food and beverage environment School Child Non-US

Shive, 2006257 Energize Your Life! Food and beverage environment

Messaging environment

University Adult US

Sigmund, 2012258 "Healthy Schools" Project (PA program component) Physical activity environment School Child Non-US

Page 204: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-26

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Simon, 2008259 ‘Intervention Centered on Adolescents’ Physical activity and Sedentary behavior’ (ICAPS)

Physical activity environment School Child US

Steenhuis, 2004260 Changing food availability in Dutch cafeterias Food and beverage environment Employer or workplace Adult Non-US

Story, 2012261 Bright Start Food and beverage environment

Physical activity environment

School Child US

Tarp, 2016262 The Learning, Cognition & Motion (LCoMotion) study Physical activity environment School (include summer school and after care)

Child US

Te Velde, 2008263 Pro Children Food and beverage environment School Child US

Van Cauwenberghe 2012264 Lowering recess playground density Physical and Built environment School Adult Non-US

Waters, 2017265 Fun ‘n healthy in Moreland!: Healthy lunch options, healthy snacks, fruit breaks, upgraded taps, school water policy/water bottles, schools breakfast, apple slinky machines, fruit deliveries, cooking gardens

Food and beverage environment

Physical activity environment

School and early child care environment

School (include summer school and after care)

Child Non-US

Wells, 2014266 School garden Physical and built environment School Child US

Wendel, 2016267 Standing desks in classrooms Physical activity environment School Child US

Whitt-Glover, 2011268 Instant Recess Physical activity environment School Child US

Williamson, 2012269 Environmental Modifications (EMs) Physical activity environment School Child US

Wilson, 2015270 Positive Action for Today’s Health Physical and Built environment

Messaging environment

Community or neighborhood Adult US

Wright, 2012271 Competitive Food Laws (varies by state) in 40 States Food and beverage environment School Child US

Page 205: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-27

Author, year Intervention name/description if not named

Goal of the program

policy

or built environment change Target of intervention Adult/Child/Both US or Non-US

Wright, 2013272 Kids N Fitness Physical activity environment School Child US

Yildirim, 2014273 Transform-Us! (T-Us) Physical activity environment School Child Non-US

Zhou, 2014274 PA policy changes; PA curriculum Food and beverage environment

Physical activity environment

School Child Non-US

ATLAS= Active Teen Leaders Avoiding Screen-time; HUD=US Department of Housing and Urban Development; NHS=National Health Services; No.=number; PA=physical activity; US=United States

Page 206: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-28

Evidence Table H1c. Intervention descriptions of included studies that follow other study design methods Author, year Intervention name/description if not named Goal of the program, policy, or

built environment change Target of intervention Adult/Child/Both US or Non-US

Ashfield-Watt 2007275 Uk Department of health funded pilot community based intervention to improve fruit and vegetable intakes in five economically deprived areas in England

Food and beverage environment Community or neighborhood Adult Non-US

Blake, 2013276 The workplace wellness programme Food and beverage environment Employer or workplace Adult Non-US

Brownson 2004277 Bootheel Walking Promotion Project (creation of walking trails)

Physical activity environment Community or neighborhood Adult US

Brownson, 2005278 Multilevel community intervention Physical activity environment Community or neighborhood Adult US

Brusseau, 2016279 Comprehensive School Physical Activity Program (CSPAP)

Physical activity environment School (include summer school and after care)

Child US

Cheadle, 2012280 Healthy Eating Active Living–Community Health Initiative (HEAL-CHI)

Physical and Built environment

Physical activity environment

School Child US

De Cocker, 2011281 10 000 steps Ghent: walking circuits in parks, street signs in parking lots

Physical and Built environment

Messaging environment

Community or neighborhood Adult Non-US

de Silva-Sanigorski, 2011282 Water Jet installation in schools Messaging environment School Child US

Geaney, 2010283 Structured catering initiative Food and beverage environment Employer or workplace Child Non-US

Gebel, 2011284 Wheeling Walks: mass media campaign influence on walking differently

Messaging environment Community or neighborhood Adult US

Heelan, 2009285 Walking school Bus Physical activity environment School Child US

Huberty, 2013286 Movin’ Afterschool Physical activity environment School Child US

Magarey, 2013287 Eat Well Be Active (EWBA) Community Programs Physical activity environment School Child Non-US

Naul, 2012288 Healthy Children in Sound Communities (HCSC)/gkgk project Physical activity environment School Child Non-US

Rogers, 2013289 Romp & Chomp Messaging environment School Child US

Page 207: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-29

Author, year Intervention name/description if not named Goal of the program, policy, or built environment change

Target of intervention Adult/Child/Both US or Non-US

Taber, 2002290 State laws regarding the nutrition content of competitive foods sold in high schools

Food and beverage environment School Child US

Tomlin, 2012291 Action Schools! BC Physical activity environment School Child Non-US

Vasquez, 2016292 Community-Supported Agriculture (CSA) Food and beverage environment Employer or workplace Adult US

Weaver, 2017293 Partnerships for Active Children in Elementary Schools (PACES)

Physical activity environment School (include summer school and after care)

Child US

Whaley, 2010294 Supplemental Nutrition Program for Women, Infants, and Children (PHFE-WIC)

Food and beverage environment

Physical activity environment

Food assistance programs Child US

No.=number; US=United States

Page 208: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-30

Evidence Table H2a. Study and participant characteristics of included studies that follow a natural experiment method Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Anderson, 20131 2003 to 2007 83253 NA 40464 (48.3-48.7) Grades:9-12 NR

Anthamatten, 20112 2005 to 2006 3688 12 months NR NR African American, 2- 65

Hispanic, 32-92

Anglo, 2- 7

Asian, 1-3

Azevedo, 20143 2010 to 2012 497 12 months NR 11.2 NR

Barnidge, 20134 2011 NR 1 month 40 (28.4) NR Black (Non-Hispanic), 34.8

White (Non-Hispanic), 54.6

Other (including multiple race), 5.7

Barroso, 20095 2006 to 2008 3327 24-36 months NR NR NR

Bauhoff, 20146 2001 to 2006 35899 24 months NR 12-15 African American, 6-11

Hispanic, 39-72

White, 10-40

Bauman, 20037 1999 to 2002 665 3 years (45.9- 50.2) NR NR

Bere, 20108 2001 to 2008 2887 NA (48-50) 10-12 NR

Berger-Jenkins, 20149 2008 to 2010 796 2 years (51.7) 8.98 Hispanic, 60.5

Black (Non-Hispanic), 24.6

White (Non-Hispanic),12.6

Other, 2.3

Page 209: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-31

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Bolton, 201710 2008 to 2010 3648 2 years NR 5 NR

Bowling, 201611 2013 to 2013 425 17 weeks 24 (5.2) 34.5 Asian, 8.9

Black, 7.9

Cape Verdean, 0.7

Hispanic, 46.2

Multi-Ethnic, 4.8

White, 30.1

Other, 3.1

Branas, 201112 1999 to 2008 NR NR NR 36.42-36.98 (Median) NR

Brown, 200813 2003 453 2 months (27.9- 63.5) 37.18-46.84 NR

Brown, 201514 2012 to 2013 537 NR NR 37.8-43.9 Hispanic, 25

Brown, 201615 NR NR NR NR ≥18 NR

Page 210: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-32

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Brown, 201616 2012 to 2013 910 12 months NR NR NR

Burke, 201417 2011 to 2013 23347 7 months NR 10 - 17 White, 18-51

Black, 26-61

Hispanic, 1-14

Asian/Pacific Islander, 1-18

Two or more races, 2- 5

Buscail, 201618 2013 to 2015 199 24 months (35.7-41.8) 38.1 - 40.6 NR

Caldwell, 200919 4 to 16 weeks 266 NR 53 All ages White, 79.7

Not White, 6.4

Hispanic, 7.5

Not Hispanic, 83.8

Calise, 201320 2009 424 3 months NR 20-39 White (Non-Hispanic), 88.6

Camacho-Rivera, 201721 2011 to 2012 362 NA NR 44.8-47.4 NR

Capogrossi, 201622 1998 to 2007 5910 NR (43(55) NR Black, 7-33

Hispanic, 17-26

Cawley, 200723 1999 to 2003 36833 36 months 17987 Grades: High school NR

Cawley, 200724 1999 to 2003 44164 NA (50) 14-18 White, 64

Chen, 201525 2004 to 2009 18769 3 years 7032 45.4 - 47.46 NR

Page 211: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-33

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Cleland, 200826 2004 to 2006 5170 20 years NR 31.0 (2.6) NR

Coffield, 201127 2007 to 2009 40713 2 years NR 15 - 19 White, 84

Non-White, 16

Cohen, 201228 2008 to 2010 NR NR (45.6) All ages White (Non-Hispanic), 2.1

Black (Non-Hispanic), 8.5

Asian/Other, 0.8

Hispanic, 74.1

Cohen, 201429 2007 to 2009 1230 NR (41-47) 8.6 NR

Coyle, 200930 2004 to 2005 NR 12 months NR NR NR

Cradock, 201131 2004 to 2006 2091 24 months (45 ( 53) NR White, 11-72

Black, 16-43

Page 212: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-34

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Cradock, 201432 2011 to 2011 455 3 months 187 (48) 10.2 (0.8) White, Non-Hispanic, 2

Black (Non-Hispanic), 59

Hispanic, 31

Asian, 7

Other, 2

Cullen, 200633 2001 to 2002 2790 2 years NR NR Hispanic, 61

White, 34

African American, 3

Asian

other, 2

Cullen, 200834 2001 to 2006 2671 3 years NR NR NR

Cummins, 200535 NR 412 NA 118 (28.6) ≥16 NR

Cummins, 200836 2001 NR 1 year 118 (28.6) NR NR

Cummins, 201437 2006 to 2010 1440 48 months 134 (20) 54 (14.8) Black, 84

White, 9

Hispanic, 2

Other, 4

Page 213: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-35

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Datar, 201638 2013 to 2015 894 18 months NR 13.17 White (non-Hispanic), 40.6

Black (non-Hispanic), 20.4

Hispanic, 24.8

Other (including multiple race), 14.2

De Cocker, 200739 2005 to 2006 1682 1 year (47.2-47.5) 25-75 NR

de Visser, 201640 2013 to 2014 2510 6 weeks (48.5-51.1) 11-12 White, 34.8-58.7

Black, 13.7-44.4

Hispanic, 4.4

Asian, 15.1-0.7

Dill, 201441 2010 to 2013 490 5 days NR 40.8-43.1 NR

Dubowitz, 201542 2011 to 2014 1372 36 months (25) 53.3 African American, 95.2

Other, 4.8

Page 214: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-36

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Elbel, 201543 2011 to 2012 850 12 months NR NR Black (Non-Hispanic), 21.9-46

Hispanic, 47.5-70.7

Other (including multiple race), 3.3-8.0

Elbel, 201744 2011 to 2014 3998 3 years (36) NR Hispanic, 50

Black (non-Hispanic), 43

Other (including multiple race), 7.7

Falbe, 201645 2014 to 2015 990 12 months (40-54) 39-46 (15-17) African American, 31-34

Hispanic, 20-38

White, 16-32

Other, 13-17

Fitzpatrick, 201746 2005 to 2011 431 2 years (54) 8-10 NR

Flego, 201447 2011 to 2013 1960 2 years NR NR NR

Fogarty, 200748 2003 to 2005 29050 NA NR 4-6 NR

Fox, 200949 NR 2228 NR 8.8 – 15.95 White, 50-59

Black, 14-21

Hispanic, 20-24

Other, 6-7

Page 215: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-37

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Frongillo, 201750 2013 to 2015 NR 1 year NR 9.28 Black (non-Hispanic), 20.6

Hispanic, 39.8

Other (including multiple race), 39.6

Fuller, 201351 2009 to 2010 NR NA (47.3-48.1) 47.8-49.4 NR

Fung, 201352 2003 to 2011 10723 2 years (47.9-49) Grade: 5 NR

Gee, 201553 2003 to 2009 1081 NA 536 (50.6-52.5) 15.8 - 17.6 White, 61.0-72.1

Black, 21.6-29.9

Other, 3.5-6.6

Hispanic, 1.3-4.5

Gibson, 200654 1979 to NR 14859 NA (0) 7.8 NR

Giles-Corti, 201355 NR NR 48 months (38.6-47.7) 37.2-40.7 NR

Gleason, 200956 2004 to 2005 2314 NR NR Grades: 1-12 NR

Goldsby, 201657 2009 to 2012 1443 36 months 634 (44) 10.3 Median Black, 77

White, 22

Other, 1

Hispanic, 14

Non-Hispanic, 86

Page 216: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-38

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Goodman, 201458 2010 to 2011 3516 2 years 634 (43.3) NR White (Non-Hispanic), 96.9

Goodman, 201659 2010 to 2012 3336 2 years 1667 (50) 10-11 Asian/Pacific Islander, 9

Black (non-Hispanic), 1

White (non-Hispanic), 86

Other 1

Mixed, 3

Gorely, 201160 NR 589 10 months NR 7-11 NR

Gorham, 201561 NR 960 5 months (8.5) ≥18 Hispanic, 59.2

Harding, 201762 NR NR NR NR NR NR

Heelan, 201563 2006 to 2012 2234 72 months NR Grades: Kindergarten - 5 Caucasian, 85

Hennessy, 201464 2005 to 2008 16271 NA (46.8) healthy weight

(53.9) overweight/ obese

12.5-12.7 White (Non-Hispanic), 63.6-46.7

Black (Non-Hispanic), 13.6-21.1

Hispanic, 14.9-25.1

Other, 8-7.2

Herrick, 201265 2009 to 2009 100 5 months (45) 10.4 (0.5) – 10.3 (0.6); Control - Intervention

African American 2

Asian 53

Hispanic 31

White 3

Other 11

Page 217: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-39

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Hilmers, 201466 2006 to 2007 661 NA (0) 33.89 (8.95) – 35.17 (9.12) Hispanic, 100

Hobin, 201467 2008 to 2011 477 36 months 204 (45.6) 15.2 (0.818) NR

Hobin, 201768 2007 to 2013 837 NR NR Grades: 9-12 NR

Hoelscher, 201669 2009 to 2010 32482 Spring 2010 (49.6 ( 50.0) 12.33 (1.05) – 12.9 (1.14) African American, 16.8-17.2

Hispanic, 28.6-28.1

White

other, 54.7-54.7

Hoelscher, 201670 2009 to 2012 78 schools 3 years (50.5-53.8) NR Hispanic 50-70.6

White, 19.8-28.2

Black or African American, 6.6-7.2

Other, 3.0-18.0

Howlett, 201671 2000 to 2011 NR NR NR 2-4 NR

Hu, 201672 1990 to 2010 260054 NAs (1990-2000 and 2000-2010 combined)

(46-49) 16-79 NR

Hughes, 201273 2007 2709 1 month NR NR NR

Page 218: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-40

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Hunter, 201674 2013 to 2015 19854 2 years (46.4) 15.01-15.07 White (non-Hispanic), (73.7)

Jennings, 201275 2009 to 2010 322 12 months (18.8) NR NR

Jia, 201776 2015 1648 NR 738 11.6 NR

Johnson, 201777 2011 to 2012 586 12 weeks 113 (19%) NR White (non-Hispanic), 64

Asian/Pacific Islander, 18

Black (non-Hispanic), 6

Other (including multiple race), 9

Just, 201478 NR 725 NA NR 12.64 – 12.8 White 42-51

Black 15-25

Asian 8

Kern, 201479 2004 to 2012 NR 96 months NR NR NR

Keyte, 201280 2007 511 NA (50.2-52.5) Grades: 3-4 NR

Kim, 201281 NR 2057 4-10 weeks 877 (42.6) NR NR

Kim, 201282 2002 to 2007 25251 4 years (50) 13.9-14 White, 59.5-63.1

African American, 16.2-16.9

Hispanic, 12.7-15.4

Other, 7.3-8.1

King, 201483 2008 to 2012 3385 18 months (51.6) 10.6 White, 86.3

Kubik, 200584 2000 to 2000 3588 NA (51) 14.2 White, 70

Page 219: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-41

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Lachapelle, 200985 NR 4156 NA 48-50 36.6-42.9 White (Non-Hispanic), 67.6-71

LaRowe, 201686 NR 327 12 months NR 22.9 White

Caucasian 73.3

African American 8.9

Hispanic 7.6

American Indian 8.0

Multiple Race

Ethnicities 2.1

Leung, 201387 1999 to 2008 5193 NA (46.7-50.9) 10.4-12 White (Non-Hispanic), 37.9 - 48.8

African American, 17.7 -31.2

Hispanic, 24.9 -26

Other or multiple ethnicities, 6-7.5

Liao, 201588 2001 to 2006 NR NR (40-48.4) ≥18 NR

Page 220: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-42

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Ling, 201489 2011 1508 4 months 814 (54.5) 8.3 NR

Liu, 201690 2014 to 2014 3248 NA 1677 (47) 2.30-2.47 Hispanic, 86.3

White, 4

Black, 6.5

Asian/Pacific Islander, 2.3

Other, 27

MacDonald, 201091 2006 to 2008 801 24 months NR NR NR

Maddock, 200692 2000 to 2004 NR 5 years NR NR NR

Madsen, 201193 2001 to 2008 1148000 7 years NR NR African American, 68.9

American Indian

Alaskan Native, 5.7

Asian

Pacific Islander

Filipino, 84.3

Hispanic, 289.8

White (Non-Hispanic), 238.1

Other, 63.3

Page 221: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-43

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Madsen, 201594 NR NR (51) NR Hispanic, 55-56

African American, 8

Asian, 17

White (Non-Hispanic), 13-14

Malakellis, 201795 2012 to 2014 1557 2 years NR 12-16 NR

Masse, 201496 2007 to 2008 11385 NA (41.8) 14.9 NR

Miewald, 201297 2008 192 8 months (18-22) NR NR

Miller, 201598 2012 to 2013 939 1 year (49) ≥18 Hispanic, 25

Molitor, 201599 2011 1273 NA (37.2-55.8) 5.5-49.4 Hispanic, 58.8-90.6

White (Non-Hispanic), 1.12-20.4

Asian/Pacific Islander, 2.5-12.7

Black (Non-Hispanic), 2.0-7.5

Other (including multiple race), 1.2-4.4

Morton, 2016100 2007 to 2011 2064 4 years 47.7 10.24 White (non-Hispanic), 97

Other (including multiple race), 3 ,

Mullally, 2010101 2001 to 2007 971 5 years (48.4-50.1) Grades: 5-6 NR

Mumford, 2011102 2008 to 2009 101 NR NR NR White (Non-Hispanic), 47

Black (Non-Hispanic), 33

Page 222: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-44

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Nanney, 2014103 2002 to 2006 136549 4 years NR NR NR

Nanney, 2016104 2006 to 2013 7237 6 years NR NR Minority, 9.7-14.4

Neelon, 2015105 2011 to 2012 104 1 year 7.8-8.3 White (Non-Hispanic), 76.6-85

Black (Non-Hispanic), 12.5-18.8

Hispanic, 6.3

Other, 2.5-4.7

Nehme, 2017106 2014 295 9 months 115 (39.8) NR White (non-Hispanic), 65.3

Hispanic, 21.7

Black (non-Hispanic), 7.2

Asian/Pacific Islander, 3.6

American Indian/Alaska Native, 1.4

Nguyen, 2015107 2003 to 2008 17891 NA (49) NR White, 72

Black, 11

Hispanic, 12

Other, 5

Oh, 2015108 2005 to 2007 1895 NA (51.7) 12.6 White (Non-Hispanic), 32.1

Black (Non-Hispanic), 27.6

Hispanic, 31.3

Non-Hispanic other, 7.6

Page 223: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-45

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Olsho, 2015109 2010 2287 NR NR NR NR

Panter, 2016110 2011 to 2012 1143 12 months 360 (31.5) NR NR

Parsons, 2014111 1999 to 2010 6682 60 months (51.2 2-19 Caucasian, 50.2

Minority, 49.8

Peterson, 2015112 2004 to 2009 NR NR (49.4) 12.8 White (Non-Hispanic), 70.3

Asian, 3.6

Black (Non-Hispanic), 7.4

Hispanic, 8.5

American Indian

Alaskan, 1.3 missing ethnicity

Powell, 2009113 1997 to 2006 153673 (48.06) 15.06 White, 69.94

African American, 10.26

Hispanic, 10.10

Other race, 9.69

Quig, 2012114 2007 to 2008 184 1 year 72 (46) 7.6 NR

Reger-Nash, 2005115 2001 to 2002 1472 12 months NR NR White (Non-Hispanic), 93-96

Page 224: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-46

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Reger-Nash, 2008116 2003 to 2005 1834 8 week NR 40-65 White (Non-Hispanic), 92-97

Restrepo, 2016117 2004 to 2012 74 8 years (41) 53.245 Black, 12

Hispanic, 10

Ridgers, 2007118 2003 to 2004 470 6 months (49.3) 7.9-8.4 NR

Ridgers, 2007119 2003 to 2004 297 6 weeks 150 7.8- 8.3 NR

Riis, 2012120 2003 to 2007 NR NR 10-17 NR

Ritchie, 2016121 2011 to 2012 3944 NR 49.3 9.75 Hispanic 49.2

White (Non-Hispanic) 12.5

Non-Hispanic 9.1

Asian 8.5

American Indian

Alaskan native 1.4

Native Hawaiian

Pacific Islander 1.7

Other 16.7

Page 225: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-47

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Robles, 2017122 2012 NR NA (36.6-52.3) NR White (non-Hispanic), 5.1-16.3

Black (non-Hispanic), 41.3-62.1

Hispanic, 25.9-29.4

Asian/Pacific Islander, 2.3-7.3

Other, 4.7-5.7

Rushakoff, 2017123 2013 to 2014 287 18 months NR NR White (non-Hispanic), 96

Black (non-Hispanic), 2

Hispanic, 2

Sabia, 2016124 1999 to 2011 NR NA (49) 14-18 White, 59.6

Black, 13.7

Hispanic, 9.9

Asian, 3.7

Other, 13.1

Sadler, 2013125 2009 to 2011 186 2 years (45) 55 Black (Non-Hispanic), 61

Sanchez-Vaznaugh, 2010126

2001 to 2008 567756 8 years 48350 – 293645 (50.7-51.0)

Grades: 5 and 7 White, 8.4-38.2

Hispanic, 47.6-77.8

Black, 5.5-10.0

Asian, 2.7-7.4

Filipino, 1.1-1.3

Page 226: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-48

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Schanzenbach, 2005127 NR NR NR 67.8 - 67.9 months White, 100

Schwartz, 2016128 NR 60 months 1,065,562 (50.1-50.2) Grades: Kindergarten-8 Asian, 12.0-14.6

Black, 33.2-36.5

Hispanic, 37.3-38.9

White, 13.3-14.2

Sekhobo, 2014129 NR NR 72 months NR NR NR

Slater, 2014130 2009 to 2011 Sample varies NA (48) Grades: 8, 10, and 12 White (Non-Hispanic), 59

Black (Non-Hispanic), 13

Hispanic, 15

Spence, 2013131 2003 to 2009 407 5 years 198-322 4 - 7 NR

Stephens, 2014132 2009 to 2010 1352 NR 3.39 NR

Stratton, 2005133 NR 120 5 months 51 NR NR

Sturm, 2010134 1998 to 2004 7414 NA (50.2) 134.4 months Black (Non-Hispanic), 14.2

Hispanic, 17.5

Asian/Pacific Islander, 2.3

Sturm, 2015135 2007 to 2012 12058 48 months NR) NR NR

Page 227: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-49

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Taber, 2011136 2007 to 2007 90730 NA (50.7 All ages White, 62

Black, 18

Hispanic, 15

Other, 5

Taber, 2012137 2004 to 2007 8870 36 months NR Grades: 5-8 White (Non-Hispanic), 58.9

Black (Non-Hispanic), 11.9

Hispanic, 18.5

Other Non-Hispanic, 10.7

Taber, 2012138 2004 to 2007 9170 3 years NR NR White, Non-Hispanic, 44.8-70.7

Black (Non-Hispanic), 11.8-12.2

Hispanic, 7.3-33.0

Other, Non-Hispanic, 6.7-13.1

Taber, 2013139 2004 to 2007 5510 3 years NR Grade: 8 White, Non-Hispanic, 62.0

Black (Non-Hispanic), 10.4

Hispanic, 17.4

Other, Non-Hispanic, 10.2

Tak, 2007140 NR 500 NR (45.9-50.9) Grade: 4 NR

Page 228: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-50

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Tak, 2009141 2003 to 2005 1328 2 years 346 (42.8(46.6) 9.9-10.1 NR

Tester, 2016142 2003 to 2012 1197 108 months (48.3-56.2) 2.91-3.07 White (Non-Hispanic), 30.4-51.3

Hispanic, 21.8-39.9

Black (Non-Hispanic), 21-22.4

Other/mixed, 5.9-7.3

Toussaint, 2017143 2015 to 2016 7137 1 year 3678 (51.53) 4 -12 Hispanic, 4.08

American Indian/Alaska Native, 0.21

Asian/Pacific Islander, 0.53

Black (non-Hispanic), 41.08 Asian/Pacific Islander, 0.10

White, 91.45

Multiracial, 2.55

Utter, 2016144 2012 8500 1 year 40-60% Grades: 9-13 NR

Vadiveloo, 2011145 2008 to 2008 1170 NR (34.9-41) 37.7- 40.4 White, 4.9-9.9

Black, 57.0-81.5

Hispanic, 9.3-25.5

Asian

Hawaiian Pacific Islander, 1.2-2.3

Other, 0.6-5.9

Page 229: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-51

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Veugelers, 2005146 2003 to NR 5200 NA NR Grade: 5 NR

von Hippel, 2015147 2008 to 2012 772559 (50) NR Asian, 3

Black, 13

Hispanic, 44

Native Am, 0.4

White, 40

Webb, 2012148 2004 to 2008 8773 4 years (44.7) All ages NR

Webb, 2016149 2012 to 2013 4650 NA 2091 NR NR

Wells, 2005150 2003 to NR 1492 NA NR 4-8 White British background 56.5-81.1

West, 2011151 2007 to 2008 368 1 year (47.6) NR White (Non-Hispanic), 90.2

Black (Non-Hispanic), 6.1

Hispanic, 3.7

Whetstone, 2012152 NR 2487 20 months 552 (48.3) 9.5 Caucasian, 64.7

African-American, 35.3

Hispanic origin, 2.6

Woodward-Lopez, 2010153

2005 to 2008 3527 24 months NR NR NR

Page 230: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-52

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Wrigley, 2003154 2000 to 2001 1009 1 year (15.9-21.6) NR

Zhu, 2013155 2013 to NR NR NR NR (33) NR NR

Zhu, 2014156 2013 to 2014 449 NA (50.8) 37.2 White, 61.1

N=sample size; NA=not applicable; No.=number; NR=not reported

*The range of study participant characteristics are given when the mean of the total participant sample is not reported by the study.

† Reported as mean age in years unless otherwise stated.

‡Given the heterogeneity in race/ethnicity categories reported, these categories were extracted from the studies as is.

Page 231: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-53

Evidence Table H2b. Study and participant characteristics of included studies that follow an experimental method Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Alaimo, 2013157 2007 to 2010 1777 2 years 510 (40.6) 12.3-12.4 White, 49.2

African American, 23.0

Hispanic, 15.8

Native American, 4.8

Asian and other, 8.0

Anderson, 2001158 NR 564 NR (0) NR Black (Non-Hispanic), 43.3

White (Non-Hispanic), 49.

Other (including multiple race), 7.3

Angelopoulos, 2009159 2004 to 2006 646 12 month 137(42.7)-141(45.8) 10.25 – 10.29 Greek, 88.0-90.3

Immigrants, 9.7-12.0

Ask, 2010160 2007 to 2007 156 4 months 74 Grade: 9 NR

Audrey, 2015161 2012 to 2013 187 12 months 98 (52.4) 37.8 White British, 77

White Other, 10.2

Mixed Ethnic Group, 1.1

Asian or Asian British, 2.1

Chinese, 0.5

Not disclosed

missing, 9.1

Ayala, 2013162 NR 179 13 weeks NR 32 Hispanic, 100

Backman, 2011163 2005 528 3 months (41.5) 33 Hispanic, 97

Page 232: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-54

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Baker, 2016164 2008 to 2013 794 60 months (34.5-37.3) 38.8-41.7 African-American, 100

Bastian, 2015165 2009 to 2011 454 NR (49-52.8) NR NR

Bere, 2005166 2001 to 2002 922 9 months 397 NR NR

Bere, 2006167 2001 to 2003 577 2 years NR 11.3-13.0 NR

Bere, 2006168 2001 to 2003 450 2 years 169 11.3-13.0 NR

Bere, 2007169 2002 to 2005 3315 36 months 984 (51) 11.8 NR

Beresford, 2010170 2001 to 2006 NR 4.4 years (76.1-86.1) 41.2-42.2 White (Non-Hispanic), 9.6-19.7

Blum, 2008171 2004 to 2005 581 9 month NR 15.8 White, 97.8

Bonsergent, 2013172 2006 to 2011 5354 2 years (47.1) 15.8 NR

Bonvin, 2013173 2009 to 2010 648 1 year 335 (51) 3.3 NR

Busch, 2015174 2011 to 2013 1716 24 months NR NR NR

Page 233: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-55

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Caballero, 2003175 NR 1704 3 years NR Grades: 3-5 NR

Chomitz, 2010176 2004 to 2007 3561 3 years 963 (51.8) 7.7 Asian, 10.2

Black, 37.3

Hispanic, 14

White, 37.1

Other, 1.7

Cochrane, 2012177 2009 to 2010 601 12 months (86.4-90.1) 63.3 - 63.9 White, 95.8-97.0

Coleman, 2012178 2008 to 2010 579 2 years (43) 8.9 Hispanic, 52

African American, 19

Non-Hispanic White, 19

Asian

Pacific Islander, 7

Native American, 0.3

Unknown, 2.7

Cortinez-O'Ryan, 2017179 2014 100 12 weeks NR 4-17 NR

Crespo, 2012180 2003 to 2004 808 3 years NR 5.9- 33 NR

Day, 2008181 2006 to 2006 527 12 weeks 240 (43-51) 9.9-10.1 NR

De Coen, 2012182 2008 to 2010 1589 2 years (50) 4.95 NR

de Greeff, 2016183 NR 388 2 years 80-81(41-45) Grades: 2-3 Dutch,100

Page 234: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-56

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

De Henauw, 2015184 2007 to 2010 16228 2 years NR 6.016 NR

de Meij, 2011185 2006 to 2008 2848 20 months 1435 (50.4) 8.6 Dutch, 18.5 -10.2

Moroccan, 31.6 - 30.0

Turkish, 14.3 - 23.3

Surinam, 15.9-11.3

Western, Other: 6.5 - 8.1

Non-western other: 13.2 -17.2

Dunton, 2015186 NR 130 4 months 50 (41.4-47.3) 10.1-10.3 Black, 7.5-3.8

Asian, 1.9

Hispanic

Latino, 52.8-73.1

White, 32.1- 15.4

Other, 5.7-5.8

Page 235: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-57

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Dzewaltowski, 2009187 NR NR 2 years (45-46) NR White (Non-Hispanic), 81.07-87.35

Eagle, 2013188 NR NR 10 weeks (49) NR White (Non-Hispanic), 54.2

Black (Non-Hispanic), 27.3

Asian/Pacific Islander, 5.1

Hispanic, 2.9

Other (including multiple race), 7.2

Economos, 2007189 2002 to 2005 1696 1 year NR 7.34 - 7.92 White: 37.8-5.7

Black: 6.9-25.1

Hispanic: 11.8-22.8

Elinder, 2012190 2009 to 2011 813 24 months NR 6-16 Non-Swedish background, 7- 50

Eriksen, 2003191 2000 1493 5 weeks (45-49) 6-10 NR

Ermetici, 2016192 2009 to 2011 487 2 years (50-52) 12.5 NR

Esquivel, 2016193 2013 to 2014 349 1 year (54) 2-5 Asian, 9

Multiracial, 62

Native Hawaiian and Pacific Islander, 23

White, 6

Evans, 2013194 2006 to 2008 1031 20 month (48.1-51.4) 7 White (Non-Hispanic), 92.1-93.5

Fairclough, 2016195 2015 139 2 months NR 10.7 (median) NR

Page 236: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-58

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Farley, 2007196 2003 to 2005 710 2 years (50.5) Grades: 2-8 Black (Non-Hispanic), 90-99

Farmer, 2017197 2010 to 2013 902 1 year NR 7.9-8.0 Asian/Pacific Islander, 34.0-38.0

White (non-Hispanic), 46.1-52.0

Other (including multiple race), 13.3-15.7

Finch, 2014198 2010 348 6 months (56-60) 3-5 Aboriginal, 2.9-4.6

Foster, 2008199 NR 1349 2 years (45-48) 11.2 – 11.13 Black, 44.33 - 46.83

Asian,17.09- 27.67

Hispanic, 5.83 -22.43

Other, 5.47-5.50

White, 10.68-14.17

French, 2010200 2005 to 2007 1063 18 months (79) 47 White, 63

Fu, 2016201 NR 174 9 weeks 82 12.06 NR

Gatto, 2017202 2011 to 2013 375 12 weeks 153 9.3 Hispanic,89

Page 237: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-59

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Geaney, 2016203 NR 850 7-9 months 393 (76) 18-65 NR

Gittelsohn, 2010204 2006 to 2007 234 9-11 months (5) 9.8 - 41.7 Native Hawaiian or PI, 64.0

Gittelsohn, 2013205 NR 276 15-20 months NR 45.8–48.2 NR

Goetzel, 2010206 2007 to 2008 5124 2 years (73) 44.6-44.8 White (Non-Hispanic), 77.1-79.1

Gustat, 2012207 2006 to 2009 499 2 years (34.7-45.3) 41.6– 47 African American, 85.7-100

Haerens, 2006208 2003 to 2005 NR 2 years NR NR NR

Haerens, 2007209 2003 to 2004 2840 9 months (63.4) 13.1 NR

Hardy, 2010210 2008 to 2008 430 6 months (49.4-50.3) 4.4 NR

He, 2009211 2006 1586 21 weeks 576 (45.1) 11.6 NR

Hendy, 2011212 NR 457 3 months 211 NR NR

Hoefkens, 2011213 2008 to 2009 380 7 months (59) 21 to 22 NR

Hollis, 2016214 2011 1150 24 months 244-299 (48-49) NR Aboriginal and Torres Strait Islander: 8.4-8.8

Huberty, 2011215 2009 to 2010 262 8 months (46) Grades: 1-6 White (Non-Hispanic), 4

Black (Non-Hispanic), 28

Hispanic, 29

Asian/Pacific Islander, 4

Other, 1

Page 238: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-60

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Jago, 2011216 2006 to 2009 6573 3 years (47.6) 11.2 - 11.3 59, Hispanic

19.8, black

21.3, white

Janssen, 2015217 2009 to 2010 2280 10 months (55.4-56) 8.6 – 8.7 Western, 8.1-8.6

Non-Western, 91-2

Jones, 2015218 2012 to 2014 128 12 months NR 3-5 NR

Jordan, 2008219 2005 to 2006 767 1 year (51-52) 9.0 White, 85.8-86.7

Hispanic, 7.0-7.6

American Indian

Alaska Native, 0.4-0.7

Native Hawaiian

Pacific Islander, 0.4-2.8

Asian, 0.7

African American, 0.0-2.1

Other, 2.5-2.8

Jurg, 2006220 2002 to 2003 502 10 months 245 Grades: 4-6 Foreign origin, 94-71

Page 239: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-61

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Kain, 2004221 2002 3577 6 months (52.0-53.5) 10.6 NR

Kamada, 2013222 2009 to 2010 6000 1 year 2050 (46.2-47.3) 40-79 NR

Kastorini, 2016223 2012 to 2013 3941 12 months (48.1) 3-18 NR

Kloek, 2006224 2000 to 2002 2781 2 years (47) 39-45 NR

LaCaille, 2016225 2010 to 2011 526 12 months NR 43 White (non-Hispanic),92.5

Lemon, 2010226 2005 to 2008 806 24 months (19) ≥18 Asian

Other, 1.4

Hispanic, 5.3

Non-Hispanic black, 4.7

Non-Hispanic white, 88.8

Lemon, 2014227 2010 to 2012 841 24 months 258 (33) ≥21 White, 95.9

Non-white, 4.1

Lent, 2014228 2008 to 2010 767 2 years (42.2-44.6) 10.97 - 10.99 Black, 54

White, 11.6

Hispanic, 22.9

Asian, 10.8

Page 240: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-62

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Linde, 2012229 2005 to 2009 1672 24 months 654 (39.3) All ages Non-Hispanic white, 86.8

Non-Hispanic black, 4.3

Other, Non-Hispanic, 4.4

Multiracial, Non-Hispanic, 4.4

Hispanic, 2.2

Undefined/refused-answer, 1.0

Llargues, 2011230 2006 to 2008 598 2 years NR 6.03 NR

Lorentzen, 2009231 2000 to 2003 1181 3 year NR 49.1-49.4 NR

Lubans, 2016232 2012 to 2014 361 18 months (100) 12.7 Australian, 77.2

European, 14.8

African, 1.9

Asian, 1.9

Middle eastern, 0.6

Ludwig, 2011233 2008 to 2010 4498 48 months NR All ages Black, 65-66.1

Other nonwhite, 26.8-28.1

White, 6.9-8.5

Hispanic, 30.3-33

Page 241: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-63

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Lv, 2014234 2008 to 2011 2016 2 years (44.6-49.5) 51.7-44.1 Asian/Pacific Islander, 100

Madsen, 2013235 2009 to 2010 156 1 year (60) 9.8 African American, 12

Asian, 32

Latino, 42

White, 0

Other, 14

Madsen, 2015236 2011 to 2013 1140 2 years (42.3-53.1) Grades: 3-5 White, 5.9- 6.3

Black, 9.2-12.9

Latino, 45.1-54.9

Mixed, 13.9- 14.6

Other, 15.5- 21.9

Mead, 2013237 2007 to 2009 494 12 months 68 42.4 NR

Morrill, 2016238 2011 to 2012 NR 4.5 months (51) Grades 1-5 White (Non-Hispanic), 91

Hispanic, 8

Black (Non-Hispanic), 1

American Indian/Alaska Native, 8

Asian/Pacific Islander, 1

Murphy, 2011239 2004 to 2005 4350 12 months NR 9-11 NR

Page 242: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-64

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Naylor, 2006240 2003 to 2004 441 16 months NR 11.22 Asian, 52

Caucasian, 30

Naylor, 2008241 2003 to 2004 515 18 months (50.7) 10.1 - 10.3 NR

Neumark-Sztainer, 2010242 2007 to 2009 356 9 months (0) 15.8 Black, 28.4

White, 24.4

Asian, 23.0

Hispanic, 14.3

Mixed, Other, 7.3

American Indian, 2.5

Nicklas, 2017243 2014 253 4 weeks 125(49.4) 4.43 Black (non-Hispanic), 34.39

Hispanic, 65.61

Ortega, 2016244 NR 795 24 months (22) 44.4 Mexican heritage, 85-89

Pate, 2005245 1998 to 2000 2744 2 years (0) 13.6 African-American, 46.8-50.7

Pbert, 2016246 2012 to 2013 126 8 months (57-58.6) 16.3 - 16.5 White, 63-63.2

Black, 15.8-24.1

Other, 0-7.4

Hispanic, 24.1-38.6

Page 243: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-65

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Perry, 2004247 2000 to 2002 1668 12 months (51) NR White, 90

Ploeg, 2014248 2009 to 2011 652 2 years (50.5) 10.9 NR

Pope, 2016249 2005 to 2007 222 2 years (43) Grades: Kindergarten-8 Black (non-Hispanic),100

Ransley, 2007250 2004 4595 7 months (49-51) 72.1 – 72.4 months NR

Reilly, 2006251 2002 to 2003 545 12 months NR NR NR

Reynolds, 2000252 1994 to 1995 1698 12 months NR NR NR

Ridgers, 2010253 2003 to 2004 434 12 months NR 7.9 - 8.4 NR

Rush, 2014254 2004 to 2011 6629 24 months 3215 (48.5) 7.58 - 10.69 European, 53.7-65.4

Maori, 26.6-36.6

Pacific, 5.1-6.3

Other, 2.5-3.8

Sallis, 2003255 1997 to 1999 24 12 months (51) NR nonwhite, 44.5

Sharma, 2016256 2013 to 2015 1348 16 weeks (11-48.1) 6.15-34.3 Hispanic, 71.2

Black, 23.6

White, 3.5

Other, 1.

Page 244: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-66

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Shive, 2006257 2002 835 10 weeks (23.1-35) 24.0 - 32.2 White, 64.6-82.4

Black, 0-2.6

Latino, 5.0-13.9

Asian

PI, 2.9-9.0

American Indian, 3.3-4.8

Other, 4.1-6.7

Sigmund, 2012258 2006 to 2008 176 2 years NR NR NR

Simon, 2008259 2002 to 2006 954 4 years (47.4-52.6) 11.6 – 11.7 NR

Steenhuis, 2004260 NR 5425 6 months (62) 18-64 NR

Story, 2012261 2005 to 2007 454 2 years 232 (51) 5.78-5.84 Oglala Sioux tribe(Lakota people), 99.3

Tarp, 2016262 2013 to 2014 705 20 weeks 309 12.7-13.1 NR

Te Velde, 2008263 2003 to 2005 2106 3 years 333-365 (45.7-49.4) 10.7 – 10.8 NR

Van Cauwenberghe 2012264 2011 128 1 month 69 5.1 NR

Page 245: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-67

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Waters, 2017265 2004 to 2009 3222 4-5 years NR NR NR

Wells, 2014266 2011 to 2013 285 2 years (43.6) 9.3 White (Non-Hispanic), 51.5

Black (Non-Hispanic), 30

Hispanic, 8.8

Asian/Pacific Islander, 9.7

Wendel, 2016267 2011 to 2013 193 24 months 96 (49.7) 8.8 White, 74.6

Hispanic, 7.8

Black, 7.3

Other, 10.4

Whitt-Glover, 2011268 2009 4599 1 year (51.2) NR Hispanic, 29.5

Black, 31.4

White, 32.6

Williamson, 2012269 2006 to 2009 2060 3 years 854 (41.5) NR White (Non-Hispanic), 31.6

Black (Non-Hispanic), 68.4

Wilson, 2015270 2008 to 2010 434 24 months (38) 51.12 NR

Page 246: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-68

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Wright, 2012271 2009 to 2012 305 12 months (38-42) 4-8.3 Black or African American, 1-4

Hispanic

Latino, 92-96

Mexican

Mexican American, 100-95

Wright, 2013272 2008 to 2010 251 12 months 50-51 (38-42) 8.3 – 9 Black, 1-4

Other, 4-1

Hispanic

Latino, 95-96

Yildirim, 2014273 2010 to 2010 599 5-9 months (43) 8.2 (0.4) NR

Zhou, 2014274 2010 to 2011 387 12 months 191 (53.5) 4.5 NR

N=sample size; NA=not applicable; No.=number; NR=not reported

*The range of study participant characteristics are given when the mean of the total participant sample is not reported by the study.

† Reported as mean age in years unless otherwise stated.

‡Given the heterogeneity in race/ethnicity categories reported, these categories were extracted from the studies as is.

Page 247: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-69

Evidence Table H2c. Study and participant characteristics of included studies that follow other study design methods Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Ashfield-Watt 2007275 2000 to 2002 1284 1 year NR 49-69 White (non-Hispanic), 95

Other, 4

Blake, 2013276 2006 to 2011 1452 5 year 297-264 (20.45-23.61) 41.06 - 41.65 NR

Brownson 2004277 2000 to 2002 NR NR 303 (23.4-25.9) NR White (non-Hispanic), 64.3-69.7

Black (non-Hispanic), 29.1-33.8

Other (including multiple race), 1.1-1.9

missing/unknown, 0.2

Brownson, 2005278 2003 to 2004 2470 12 months 153-207 (20.3-26.6) NR White (non-Hispanic), 94.2-94.9)

Black (non-Hispanic), 1.6-1.9)

Other (including multiple race), 3.3-3.5

Brusseau, 2016279 2014 to 2015 1460 12 weeks 730 8.4 NR

Cheadle, 2012280 2007 to 2010 3396 48 months NR NR NR

De Cocker, 2011281 2005 to 2009 886 4 years (48.6-53.4) 54.9-56.4 NR

de Silva-Sanigorski, 2011282

2004 to 2008 35157 48 months (50.1-52) 2.07 - 3.65 NR

Geaney, 2010283 NR 100 NA (20 t0 26) 18-44 NR

Gebel, 2011284 2001 to 2002 1472 3 months NR 50-65 NR

Heelan, 2009285 NR NR 2 years NR 8-8.5 NR

Huberty, 2013286 2010 to 2011 490 12 months (43) 8.7 White, 76

Magarey, 2013287 2006 to 2009 1732 3 years 604 (50.5) 11.8 NR

Page 248: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-70

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Naul, 2012288 2009 to 2010 744 13 months 291 6.96 - 7.24 NR

Rogers, 2013289 2007 to 2011 800 3 years (33) 11.3 NR

Taber, 2002290 2009 to 2010 680 NA (44.2-50.8) 15.0-15.2 White (non-Hispanic), 11.7-43.5

Black (non-Hispanic), 1.0-33.8

Hispanic, 14.7-76.6

Non-Hispanic other, 8.1-10.8

Tomlin, 2012291 2007 to 2008 148 12 months 77 NR NR

Page 249: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-71

Author, year Study dates Total N at baseline Maximum followup Male sex, n (%)* Age or grade*† Race/Ethnicity, %*‡

Vasquez, 2016292 2008 to 2010 935 NR 45 (13.9) 44 NR

Weaver, 2017293 2014 to 2016 229 2 years (50-58.4) 6.7-8.0 White (non-Hispanic), 21.4-63.0)

Black (non-Hispanic), 18.5-64.3

Other (including multiple race), 12.4-18.5

Whaley, 2010294 2007 to 2009 812 2 years (49-52) NR NR

N=sample size; NA=not applicable; No.=number; NR=not reported

*The range of study participant characteristics are given when the mean of the total participant sample is not reported by the study.

† Reported as mean age in years unless otherwise stated.

‡Given the heterogeneity in race/ethnicity categories reported, these categories were extracted from the studies as is.

Page 250: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-72

Evidence Table H3. Child BMI percentile measures Author, year Setting Level Adult/Child Measure type Child BMI Percentile Measure*

Natural Experiment Studies

Anderson, 20131 School US/Federal Child NR NR

Barroso, 20095 School State/Regional Child Measured by trained staff Standard Measure

Coffield, 201127 School US/Federal Child Measured by trained staff Self-reported height and weight were taken from the child’s first state-issued drivers license obtained between the ages 15 and 19 years.

Datar, 201638 School Local Child Measured by trained staff Height and weight measurements were also collected by the study staff during visits to the original 12 installations

Datar, 201638 School Local Child Self-reported Both child and parent reports of the child’s height (in feet and inches) and weight (pounds) were obtained for all children via the child and parent surveys

Fox, 200949 School US/Federal Child Measured by trained staff Standard Measure

Fung, 201352 School Other country Child Measured by trained staff Standard Measure

Gleason, 200956 School US/Federal Child Measured by trained staff Standard Measure

Heelan, 201563 School Local Child Measured by trained staff Standard Measure

Hennessy, 201464 School State/Regional Child Self-reported parent-reported child height and weight

Herrick, 201265 After school or summer school Non-Governmental Child Measured by trained staff Standard Measure

Kern, 201479 School Local Child NR NR

Kim, 201282 School State/Regional Child Self-reported NSCH survey

Madsen, 201193 School State/Regional Child Other BMI data retrieved from FITNESSGRAM collected data

Madsen, 201594 School Non-Governmental Child Other Student-level fitness data from Fitnessgram assessment

Nanney, 2014103 School State/Regional Child Self-reported Minnesota Student Survey (MSS)

Nanney, 2016104 School Local Child Self-reported Questionnaire

Page 251: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-73

Author, year Setting Level Adult/Child Measure type Child BMI Percentile Measure*

Oh, 2015108 Community or neighborhood , School US/Federal Child Self-reported parent-reported weight and height of child in NSCH

Parsons, 2014111 School Local Child Measured by trained staff School nurses collected

Peterson, 2015112 After school or summer school, School State/Regional Child Measured by trained staff Standard Measure

Sanchez-Vaznaugh, 2010126 School Local Child Other Fitnessgram test

Schwartz, 2016128 School Local Child Measured by trained staff standard weight and height measurement by physical education teacher

Sekhobo, 2014129 Early childhood education Local Child Measured by trained staff measured by trained WIC staff

Sturm, 2010134 School US/Federal Child Measured by trained staff height and weight were measure by study staff of the Early Childhood Longitudinal Study—Kindergarten Cohort

Taber, 2011136 School State/ Regional Child Measured by trained staff Standard Measure

Taber, 2012137 School Other country Child Measured by trained staff Standard Measure

von Hippel, 2015147 School US/Federal Child Other FitnessGram assessment

Page 252: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-74

Author, year Setting Level Adult/Child Measure type Child BMI Percentile Measure

Experimental Studies

Bonvin, 2013173 Early childhood education or daycare Other country Child Measured by trained staff Standard Measure

Coleman, 2012178 School Non-Governmental Child Measured by trained staff Standard Measure

Crespo, 2012180 Community or neighborhood , School Local Child Self-reported Calculated percentile from self-report (via parents)

Dzewaltowski, 2009187 School Local Child Measured by trained staff Standard Measure

Foster, 2008199 School Local Child Measured by trained staff Standard Measure

Gatto, 2017202 School Local Child Other Height was measured with a free-standing stadiometer (Seca, Birmingham, UK) and weight and percent body fat via bioelectrical impedance (Tanita TBF 300A, Arlington Heights, IL, USA).

Hendy, 2011212 School Local Child Measured by trained staff Standard Measure

Jago, 2011216 School US/Federal Child Measured by trained staff Standard Measure

Jago, 2011216 School Non-Governmental Child Measured by trained staff Standard Measure

Kastorini, 2016223 School Other country Child Self-reported self-reported weight and height

Lent, 2014228 School Non-Governmental Child Measured by trained staff Standard Measure

Madsen, 2013235 After school or summer school Non-Governmental Child E.H.R. E.H.R.

Pope, 2016249 School Local Child Measured by trained staff health screenings IN PHYS ED

Pate, 2005245 School Non-Governmental Child Measured by trained staff Standard Measure

Reilly, 2006251 Early childhood education or daycare Other country Child Measured by trained staff Standard Measure

Story, 2012261 School State/Regional Child Measured by trained staff Standard Measure

Page 253: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-75

Author, year Setting Level Adult/Child Measure type Child BMI Percentile Measure

Wendel, 2016267 School State/Regional Child Measured by trained staff Standard Measure

Other Study Designs

de Silva-Sanigorski, 2011282 Early childhood education or daycare Non-Governmental Child Measured by trained staff Standard Measure

Heelan, 2009285 School State/Regional Child Measured by trained staff BMI percentile

Naul, 2012288 School Other country Child Measured by trained staff Standard Measure

BMI=Body Mass Index; E.H.R.=electronic health record; No.=Number; NR=not reported; US=United States

* Standard measure” refers to the standardized method for calculating BMI, BMI-z score, and BMI-percentile as stated by the Centers for Disease Control and Prevention (CDC).295, 296

Page 254: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-76

Evidence Table H4. Child BMI-z score measures Author, year Setting Level Adult/Child Measure type Child BMI-z Measure*

Natural Experiment Studies

Bauhoff, 20146 School State/ Regional Child Other Fitnesgram Physical fitness test- measured height and weight.

Benjamin Neelon, 2015105 School Local Child Measured by trained staff Standard Measure

Bolton, 201710 School Other country Child Measured by trained staff Standard Measure

Burke, 201417 School Non-Governmental

Child Measured by trained staff Standard Measure

Capogrossi, 201622 School US/Federal Child Measured by trained staff Standard Measure

Eagle, 2013188 School State/Regional Child Measured by trained staff Standard Measure

Datar, 201638 School Local Child Measured by trained staff Height and weight measurements were also collected by the study staff during visits to the original 12 installations

Datar, 201638 School Local Child Self-reported Both child and parent reports of the child’s height (in feet and inches) and weight (pounds) were obtained for all children via the child and parent surveys

Farmer, 2017197 School Other country Child Measured by trained staff Standard Measure

Fitzpatrick, 201746 School Other country Child Measured by trained staff Standard Measure

Fox, 200949 School US/Federal Child Measured by trained staff Standard Measure

Gee, 201553 School , Other State/Regional Child Self-reported Youth Risk Behavior Survey

Gleason, 200956 School US/Federal Child Measured by trained staff Standard Measure

Goldsby, 201657 Community or neighborhood

Local Child E.H.R. E.H.R.

Herrick, 201265 After school or summer school

Non-Governmental

Child Measured by trained staff Standard Measure

Kim, 201282 School State/Regional Child Self-reported NSCH survey

Page 255: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-77

Author, year Setting Level Adult/Child Measure type Child BMI-z Measure*

Leung, 201387 Food assistance program

US/Federal Child Other NHANES

Madsen, 201193 School State/Regional Child Other BMI data retrieved from FITNESSGRAM collected data

Madsen, 201594 School Non-Governmental

Child Other Student-level fitness data from Fitnessgram assessment

Malakellis, 201795 School Other country Child Measured by trained staff Standard Measure

Sanchez-Vaznaugh, 2010126 School Local Child Other Fitnessgram test

Schwartz, 2016128 School Local Child Measured by trained staff standard weight and height measurement by physical education teacher

Whetstone, 2012152 Community or neighborhood , faith-based, School

State/Regional Child Measured by trained staff Standard Measure

Experimental Studies

Angelopoulos, 2009159 School Other country Child Measured by trained staff Standard Measure

Bonsergent, 2013172 School Other country Child Measured by trained staff Standard Measure

Page 256: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-78

Author, year Setting Level Adult/Child Measure type Child BMI-z Measure

Experimental Studies (continued)

Chomitz, 2010176 School Non-Governmental

Child Measured by trained staff Standard Measure

Coleman, 2012178 School Non-Governmental

Child Measured by trained staff Standard Measure

Crespo, 2012180 Community or neighborhood , School

Local Child Self-reported Calculated z-score from self-report (via parents)

De Coen, 2012182 School Other country Child Other BMI z-score

de Greeff, 2016183 School Other country Child Measured by trained staff Standard Measure

De Henauw, 2015184 Early childhood education or daycare School

Non-Governmental

Child Self-reported IDEFICS questionnaire

Economos, 2007189 Community or neighborhood

Non-Governmental

Child Measured by trained staff Standard Measure

Ermetici, 2016192 School , E.H.R. Other country Child Measured by trained staff Standard Measure

Esquivel, 2016193 Early childhood education or daycare

State/Regional Child Measured by trained staff Standard Measure

Fairclough, 2016195 School Other country Child Measured by trained staff Standard Measure

Foster, 2008199 School Local Child Measured by trained staff Standard Measure

Gatto, 2017202 School Local Child Other Height was measured with a free-standing stadiometer (Seca, Birmingham, UK) and weight and percent body fat via bioelectrical impedance (Tanita TBF 300A, Arlington Heights, IL, USA). BMI-z determined according to CDC cut points

Page 257: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-79

Author, year Setting Level Adult/Child Measure type Child BMI-z Measure

Gorely, 201160 School Other country Child Measured by trained staff BMI z-score

Haerens, 2006208 School Local Child Measured by trained staff Standard Measure

Hollis, 2016214 School Non-Governmental

Child Measured by trained staff Standard Measure

Jago, 2011216 School Non-Governmental

Child Measured by trained staff Standard Measure

Jordan, 2008219 School State/Regional Child Measured by trained staff Standard Measure

Kain, 2004221 School Other country Child Measured by trained staff Standard Measure

Lent, 2014228 School Non-Governmental

Child Measured by trained staff Standard Measure

Lubans, 2016232 School Local Child Measured by trained staff Standard Measure

Madsen, 2015236 School Non-Governmental

Child Measured by trained staff Standard Measure

Pbert, 2016246 School Non-Governmental

Child Measured by trained staff Standard Measure

Page 258: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-80

Author, year Setting Level Adult/Child Measure type Child BMI-z Measure

Experimental Studies (continued)

Pope, 2016249 School State/Regional Child Measured by trained staff Tanita TBF-310GS

Simon, 2008259 School Non-Governmental

Child Measured by trained staff Standard Measure

Story, 2012261 School State/Regional Child Measured by trained staff Standard Measure

Waters, 2017265 School Other country Child Measured by trained staff Standard Measure

Williamson, 2012269 School State/Regional Child Measured by trained staff Standard Measure

Wright, 2012271 School State/ Regional Child Measured by trained staff Standard Measure

Wright, 2013272 After school or summer school, School

Non-Governmental

Child Measured by trained staff Standard Measure

Zhou, 2014274 Early childhood education or daycare

Other country Child Other body composition analyzer (InBody J20, BIO-SPACE, Seoul, Korea)

Other Study Designs

de Silva-Sanigorski, 2011282 Early childhood education or daycare

Non-Governmental

Child Measured by trained staff Standard Measure

Magarey, 2013287 School Other country Child Measured by trained staff Standard Measure

Tomlin, 2012291 School Other country Child Measured by trained staff Standard Measure

BMI-z=Body Mass Index-Z score; E.H.R.=electronic health record; IDEFICS= Identification and prevention of dietary-and lifestyle-induced health effects In children and infants ; NHANES=National Health and Nutrition Examination Survey; No.=Number; NR=not reported; NSCH=National Survey of Children’s Health; US=United States

Page 259: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-81

* Standard measure” refers to the standardized method for calculating BMI, BMI-z score, and BMI-percentile as stated by the Centers for Disease Control and Prevention (CDC).295, 296

Page 260: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-82

Evidence Table H5. Child change in weight measures Author, year Setting Level Adult/child Measure type Child Weight Change Measure*

Natural Experiment Studies

Capogrossi, 201622 Food assistance program , School

US/Federal Child Measured by trained staff Standard Measure

Frongillo, 201750 Community or neighborhood

Local Child Measured by trained staff BMI and waist circumference

Hobin, 201768 School Other country Child Measured by trained staff Reports as overweight and obese

Jia, 201776 School Other country Child Measured by trained staff BMI and Weight status (Obese/overweight)- defined based on International Obesity Task Force-recommended age- and sex-specific cutoffs corresponding to BMI = 25 kg m− 2 at age 18 years

Toussaint, 2017143 School State/Regional Child Measured by trained staff BMI: Standard Measure

Utter, 2016144 School Other country Child Measured by trained staff BMI

Veugelers, 2005146 School Other country Child Measured by trained staff weight and height taken

Experimental Studies

Ask, 2010160 School Other country Child Measured by trained staff Standard Measure

Bonsergent, 2013172 School Other country Child Measured by trained staff Standard Measure

Bonvin, 2013173 Early childhood education or daycare

Other country Child Measured by trained staff Standard Measure

Caballero, 2003175 School State/Regional Child Measured by trained staff Standard measure

Cortinez-O'Ryan, 2017179 Community or neighborhood

Other country Child Measured by trained staff BMI of child captured at baseline but not reported at follow up

Farley, 2007196 Community or neighborhood

Local Child Other measured height, weight, and an estimate of body fat using bioelectrical impedance analysis

Hollis, 2016214 School Non-Governmental

Child Measured by trained staff Standard Measure

Page 261: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-83

Author, year Setting Level Adult/child Measure type Child Weight Change Measure*

Tarp, 2016262 School Other country Child Measured by trained staff BMI only: Body mass was measured to one decimal using an electronic scale (Tanita BWB-800, Tokyo, Japan)

Zhou, 2014274 Early childhood education or daycare

Other country Child Other body composition analyzer (InBody J20, BIO-SPACE, Seoul, Korea)

Page 262: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-84

Author, year Setting Level Adult/child Measure type Child Weight Change Measure

Other Study Designs

Brusseau, 2016279 School State/Regional Child Measured by trained staff Body mass index (BMI) was calculated using standard procedures taking a student’s weight in kilograms divided by the square or his or her height in meters.

BMI=Body Mass Index; No..=Number

* Standard measure” refers to the standardized method for calculating BMI, BMI-z score, and BMI-percentile as stated by the Centers for Disease Control and Prevention (CDC).295, 296

Page 263: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-85

Evidence Table H6. Child other weight outcomes Author, year Setting Level Adult/child Measure type

Natural Experiment Studies

Gibson, 200654 Food assistance program

US/Federal Child BMI

Kubik, 200584 School State/Regional Child BMI

Powell, 2009113 Community or neighborhood, School

State/Regional Child BMI

Schanzenbach, 2005127 School US/Federal Child BMI

Experimental Studies

de Meij, 2011185 After school or summer school, School

Other country Child BMI

Ploeg, 2014248 School Other country Child Probability of obesity

Rush, 2014254 School Local Child BMI

Ridgers, 2010253 School Other country Child BMI

BMI=Body Mass Index; No..=Number; US=United States

Page 264: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-86

Evidence Table H7. Adult BMI score measures Author, year Setting Level Adult/child Measure type Adult BMI Measure*

Natural Experiment Studies

Bolton, 201710 Employer or worksite

Other country Adult Measured by trained staff Standard Measure

Brown, 200813 Community or neighborhood

Local Adult Self-reported Self-reported weight and height of head of household

Brown, 201514 Transportation Local Adult Measured by trained staff "clinically measured"

Brown, 201615 Transportation, Community or neighborhood

Local Adult Self-reported BMI

Camacho-Rivera, 201721 Community or neighborhood

State/Regional Adult Measured by trained staff Standard Measure

Chen, 201525 Home Other country Adult Measured by trained staff Standard Measure

Cleland, 200826 School Other country Adult Self-reported Questionnaire ( at baseline) and International Physical Activity Questionnaire (IPAQ-L) for follow up, Other objective measure: BMI derived from measured weight and height (method not described)

Cummins, 201437 Community or neighborhood

Local Adult Self-reported self reported weight and height

Dubowitz, 201542 Community or neighborhood

Non-Governmental

Adult Self-reported Standard Measure

Hu, 201672 Reducing health inequities

Other country Adult Self-reported measured or self -reported

MacDonald, 201091 Transportation Local Adult Self-reported BMI was calculated in kg/m^2 using self-reported height and weight

Restrepo, 2016117 Community or neighborhood

State/Regional Adult Self-reported self-reported data - from BRFSS

Page 265: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-87

Author, year Setting Level Adult/child Measure type Adult BMI Measure*

Sadler, 2013125 Community or neighborhood

Local Adult Self-reported Behavioral Risk Factor Surveillance System (BRFSS)

Sturm, 2015135 Community or neighborhood

Local Adult Self-reported self reported weight and height

Webb 2016149 Transportation Other country Adult Measured by trained staff Standard Measure

Webb, 2012148 Transportation Other country Adult Measured by trained staff Standard Measure

Page 266: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-88

Author, year Setting Level Adult/child Measure type Adult BMI Measure

Experimental Studies

Baker, 2016164 Community or neighborhood

Non-Governmental

Adult Self-reported Self-reported

Cochrane, 2012177 Primary care centers

Other country Adult Measured by trained staff Standard Measure

Crespo, 2012180 Community or neighborhood , School

Local Adult Self-reported BMI

French, 2010200 Employer or worksite

Local Adult Measured by trained staff Standard Measure

Geaney, 2016203 Employer or worksite

Other country Adult Self-reported Standard Measure

Gittelsohn, 2013205 Community or neighborhood

State/Regional Adult Self-reported Self reported height and weight accepted from participants that refused trained data collector measurements, Standard Measure

Goetzel, 2010206 Employer or worksite

Non-Governmental

Adult Self-reported Biometric data were collected using standardized protocols and instruments developed by Dow Health Services

LaCaille, 2016225 Employer or worksite

State/Regional Adult Measured by trained staff Standard Measure

Lemon, 2010226 Employer or worksite

State/Regional Adult Measured by trained staff Standard Measure

Lemon, 2014227 Employer or worksite, School

Non-Governmental

Adult Self-reported Standard Measure

Linde, 2012229 Employer or worksite

Non-Governmental

Adult Measured by trained staff Standard Measure

Page 267: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-89

Author, year Setting Level Adult/child Measure type Adult BMI Measure

Ludwig, 2011233 Community or neighborhood

Local Adult Measured by trained staff Measured by trained staff

Mead, 2013237 Community or neighborhood

Other country Adult Measured by trained staff Standard Measure

Other Study Designs

Blake, 2013276 Employer or worksite

Other country Adult Self-reported Self Measured

Vasquez, 2016292 Employer or worksite

State/Regional Adult Other modified version of the Household Food Inventory checklist, which was based on the previously validated Block Food Frequency Questionnaire

BMI=Body Mass Index; BRFSS= Behavioral Risk Factor Surveillance System

* Standard measure” refers to the standardized method for calculating BMI, BMI-z score, and BMI-percentile as stated by the Centers for Disease Control and Prevention (CDC).295, 296

Page 268: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-90

Evidence Table H8. Adult change in weight measures Author, year Setting Level Adult/child Measure type Adult Weight Change Measure*

Natural Experiment

Goodman, 201458 Community or neighborhood

Other country Adult Self-reported Self-reported

Experimental

Cochrane, 2012177 Primary care centers

Other country Adult Measured by trained staff Standard Weight Measure

French, 2010200 Employer or worksite

Local Adult Measured by trained staff Standard Weight Measure

Geaney, 2016203 Employer or worksite

Other country Adult Measured by trained staff Standard Weight Measure

LaCaille, 2016225 Employer or worksite

Employer or worksite

Adult Measured by trained staff Standard Weight Measure

Lemon, 2014227 Employer or worksite, School

Non-Governmental

Adult Measured by trained staff Standard Weight Measure

BMI=Body Mass Index; BRFSS= Behavioral Risk Factor Surveillance System; No..=Number; US=United States

* Standard measure” refers to the standardized method for calculating BMI, BMI-z score, and BMI-percentile as stated by the Centers for Disease Control and Prevention (CDC).295, 296

Page 269: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-91

Evidence Table H9. Total calorie intake measures Author, year Setting

Level Adult/Child Measure type Diet Calorie Intake Measure

Natural Experiment

Chen, 201525 Home Other country Adult 24-hour recall self reported 24hr recall

Cullen, 200633 School Local/Other country

Child record/log Anonymous food record entered into Nutrition Data System (version 4.2), Objective measure: Electronic data from POS purchase machine

Dubowitz, 201542 Community or neighborhood

Non-Governmental

Adult 24-hour recall Automated self administered 24-Hour Dietary Recall (ASA24)

Fung, 201352 School Other country Child Food Frequency Questionnaire

Harvard Youth Adolescent Food Frequency Questionnaire (YAQ) adapted for Canadian settings

Hilmers, 201466 Community or neighborhood , Food assistance program

US/Federal Adult record/log 24-hour dietary food record

Leung, 201387 Food assistance program

US/Federal Child 24-hour recall NHANES

Ritchie, 2016121 School Local Child 24-hour recall 24-hour dietary-assisted recall and a 24-hour recall interview

Spence, 2013131 School Other country Child record/log Food Assessment in Schools Tool (FAST)

Taber, 2002290 School State/Regional Child 24-recall National Youth Physical Activity and Nutrition Study (NYPANS)

Experimental

Caballero, 2003175 School State/Regional Child Other Observation

Cochrane, 2012177 Primary care centers

Other country Adult Other Primary Prevention Toolkit

Foster, 2008199 School Local Child Questionnaire Youth/Adolescent Questionnaire

Page 270: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-92

Author, year Setting

Level Adult/Child Measure type Diet Calorie Intake Measure

Geaney, 2016203 Employer or worksite

Other country Adult 24-hour recall UK 3-step dietary recall (no mention of it being validated)

Gittelsohn, 2010204 Community or neighborhood

Non-Governmental

Adult NR NR

Pbert, 2016246 School Non-Governmental

Child NR NR

Ransley, 2007250 School Other country Child record/log child and diet evaluation tool (CADET)

Sharma, 2016256 Food assistance program , School

Other country Child Food Frequency Questionnaire

Block Kids Food validated food frequency questionnaire

NHANES=National Health and Nutrition Examination Survey; No.=Number; NR=not reported; US=United States

Page 271: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-93

Evidence Table H10. Change in fast food intake measures Author, year Setting

Level Adult/Child Measure type Diet Fast Food Intake Measure

Natural Experiment

Bauhoff, 20146 School State/ Regional Child Questionnaire The California Healthy Kids Survey (CHKS)

Camacho-Rivera, 201721 Community or neighborhood

State/ Regional Adult Other questionnaire Not described

Fung, 201352 School Other country Child Questionnaire Harvard Youth Adolescent Food Frequency Questionnaire (YAQ) adapted for Canadian settings

Gleason, 200956 School US/Federal Child Record/log Not specific, only reports school breakfast and lunch- Not sure we want his data

Jia, 201776 School Other country Child Other questionnaire Self reported weekly frequency of fast food and street food consumption.

King, 201483 School Local Child 24-hour recall Student Health Assessment Questionnaire/ 24-hour recall

Molitor, 201599 Home US/Federal Both Food frequency questionnaire

2011–2012 CHIS

Sturm, 2015135 Community or neighborhood

Local Adult Other questionnaire Telephone interview

Utter, 2016144 School Other country Child Other questionnaire Self-report

Vadiveloo, 2011145 Fast food chains Local Adult Questionnaire Street-intercept survey

Experimental

French, 2010200 Employer or worksite

Local Adult Food Frequency Questionnaire

self-report food frequency questionnaire (adapted from Thompson et al., 2002; Thompson et al., 1998)

Pbert, 2016246 School Non-Governmental

Child 24-hour recall 24-hour dietary recall interview22 using the Interactive Nutrition Data System (NDS, Nutrition Coordinating Center, University of Minnesota, Minneapolis, MN).

Sharma, 2016256 Food assistance program , School

Other country Child Questionnaire Self-reported questionnaire (Ding etal, 2012)

Story, 2012261 School State/Regional Child Questionnaire In-person survey completed by parents

Page 272: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-94

Author, year Setting

Level Adult/Child Measure type Diet Fast Food Intake Measure

Wright, 2012271 School State/ Regional Child Questionnaire The Child and Adolescent Trial for Cardiovascular Health After-School Student Questionnaire (ASSQ)

Other study designs

de Silva-Sanigorski, 2011282 Early childhood education or daycare

Non-Governmental

Child Questionnaire Eating and Physical Activity Questionnaire (EPAQ)

CHIS=California Health Interview Survey; No.=Number; US=United States

Page 273: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-95

Evidence Table H11. Change in fruit and vegetable consumption measures Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Natural Experiment Studies

Barnidge, 20134 Community or neighborhood

Local Adult Food frequency questionnaire

BRFSS

Bauhoff, 20146 School State/ Regional Child Questionnaire The California Healthy Kids Survey (CHKS)

Bere, 20108 School Other country Child Food frequency questionnaire

measured fruit and vegetable intake and frequency

Berger-Jenkins, 20149 School Non-Governmental

Child Questionnaire survey

Bolton, 201710 Employer or worksite, School

Other country Both Other questionnaire Victorian Population Health Survey

Bolton, 201710 Employer or worksite, School

Other country Child Other questionnaire Victorian state government Local Level Child Health and Wellbeing Survey

Bowling, 201611 Community or neighborhood , Food assistance program

State/Regional Adult Questionnaire Internally designed survey with comparison to 2013 CDC State Indicator Report on Fruits and Vegetables

Caldwell, 200919 Community or neighborhood

State/Regional Both Other questionnaire YRBS to assess F&V consumption

Coyle, 200930 School State/Regional Child 24-recall 24-hour dietary recall interview

Cullen, 200633 School Local Child record/log Anonymous food record entered into Nutrition Data System (version 4.2), Electronic data from POS purchase machine

Cullen, 200834 School State/Regional Child Record/log Anoynymous record immediately after lunch

Cummins, 200535 Community or neighborhood

Other country Adult Other questionnaire How many portions per day: fruit portion = mdium sized item; vegetabel = 3 heaping tablespoons or a medium salad bowl

Cummins, 200836 Community or neighborhood

Other country Both Record/log Self-reported in survey

Page 274: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-96

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Cummins, 201437 Community or neighborhood

Local Adult Food Frequency Questionnaire

Block Food Frequency Questionnaire

Datar, 201638 School Local Child Food frequency questionnaire

Beverage and Snack Questionnaire

de Visser, 201640 School Non-Governmental

Child Questionnaire School Physical Activity and Nutrition (SPAN) survey

Dubowitz, 201542 Community or neighborhood

Non-Governmental

Adult 24-hour recall Automated self administered 24-Hour Dietary Recall (ASA24)

Page 275: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-97

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Eagle, 2013188 School State/Regional Child Food frequency questionnaire

School-Based Nutrition Monitoring Questionnaire

Elbel, 201543 Community or neighborhood

Local Adult 24-recall 24-hour dietary recall survey

Elbel, 201543 Community or neighborhood

Local Adult Other questionnaire Eating and Physical Activity Questionnaire (EPAQ)

Elbel, 201744 Community or neighborhood

Local Adult 24-recall Dietary recall conducted by staff phone interview with participant

Elbel, 201744 Community or neighborhood

Local Adult Food frequency questionnaire

Eating and Physical Activity Questionnaire

FFung, 201352 School Other country Child Questionnaire Harvard Youth Adolescent Food Frequency Questionnaire (YAQ) adapted for Canadian settings

Flego, 201447 Community or neighborhood

Other country Adult Food frequency questionnaire

Queensland Self-Reported Health Status Survey

Dubowitz, 201542 Community or neighborhood

Local Adult 24-recall Automated SelfAdministered 24-h recall

Dubowitz, 201542 Community or neighborhood

Local Adult 24-recall Healthy Eating Index (HEI) 2010 scores

Dubowitz, 201542 Community or neighborhood

Local Child Other questionnaire Parent survey

Fogarty, 200748 School Other country Child Other questionnaire Times per week fruit was consumed; how much fruit consumed per day on average

Gee, 201553 School State/Regional Child Questionnaire Centers for Disease Control and Prevention. Youth Risk Behavior Survey (YRBS). Washington, D.C.

Gleason, 200956 School US/Federal Adult Other questionnaire Dietary Quality Index (DQI) self report for dietary behaviour measures intake of fish, red meat and meat products, starchy foods, fibre, sugary foods, fatty foods, alcohol, and fruit and vegetables.

Page 276: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-98

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Gorham, 201561 Community or neighborhood

Local Both Other questionnaire survey: cups of F&V per day

Hilmers, 201466 Community or neighborhood , Food assistance program

US/Federal Adult record/log 24-hour dietary food record

Hoelscher, 201669 School Other country Child Questionnaire School Physical Activity and Nutrition (SPAN) questionnaire. Self Administered survey of demographics, foods eaten in prior day, physical activity, attitutudes about wellness and the PUTP60 program, and participation/awareness of program

Hughes, 201273 School Other country Child Food frequency questionnaire

CADET

Page 277: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-99

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Jennings, 201275 Community or neighborhood

Other country Adult Food frequency questionnaire

East of England Lifestyle Survey

Johnson, 201777 Community or neighborhood

Other country Adult Food frequency questionnaire

portions on a ‘typical day’ and Health Survey for England requests the number of portions eaten in the last 24 h

Just, 201478 School Other country Child 24-hour recall 24-h dietary recall, but authors only considered food consumed during lunch at the school cafeteria

Keyte, 201280 School Other country Child Other questionnaire Servings of fruit-vegetable per day

King, 201483 School Local Child 24-hour recall Student Health Assessment Questionnaire/ 24-hour recall

Leung, 201387 Food assistance program

US/Federal Child Questionnaire NHANES

Liao, 201588 Community or neighborhood

US/Federal Adult Other questionnaire REACH Risk Factor Survey compared to BRFSS

Ling, 201489 Community or neighborhood , School

Other country Child Questionnaire School Physical Activity and Nutrition (SPAN) questionnaire

Liu, 201690 Food assistance program

US/Federal Child Other questionnaire Telephone survey

Maddock, 200692 Community or neighborhood, School

State/Regional Child Questionnaire self-report

Malakellis, 201795 School Other country Child Other questionnaire ABAKQ

Miewald 201297 Community or neighborhood

Other country Adult Other questionnaire Behavioral Risk Factor Surveillance System (BRFSS)

Molitor, 201599 Home US/Federal Both Food frequency questionnaire

2011–2012 CHIS

Page 278: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-100

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Mullally, 2010101 School Other country Child Questionnaire Eating Behaviour Study questionnaire

Nanney, 2014103 School State/Regional Child Questionnaire Instrument with validation

Nanney, 2016104 School Local Child Questionnaire Questionnaire

Olsho, 2015109 Community or neighborhood

Local Adult Food frequency questionnaire

The New York City Community Health Survey (CHS)

Olsho, 2015109 Community or neighborhood

Local Adult Other questionnaire Phone interviews

Page 279: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-101

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Olsho, 2015109 Community or neighborhood

Local Adult Other questionnaire Surveys by trained interviewers at farmers market

Peterson, 2015112 After school or summer school, School

State/Regional Child Questionnaire use of risk behavior survey-daily intake

Restrepo, 2016117 Community or neighborhood

State/Regional Adult Questionnaire self reported

Ritchie, 2016121 School Local Child 24-hour recall 24-hour dietary-assisted recall and a 24-hour recall interview

Robles, 2017122 Community or neighborhood

Local Adult Food frequency questionnaire

National Institute of Health’s Eating at America’s Table Quick Food Scan

Rushakoff, 2017123 Community or neighborhood

Local Child Other questionnaire Primary study questionnaire: frequency of consuming leafy greens two or more times a day

Sadler, 2013125 Community or neighborhood

Local Adult Food frequency questionnaire

Behavioral Risk Factor Surveillance System (BRFSS)

Sturm, 2015135 Community or neighborhood

Local Adult Other questionnaire Telephone interview

Tak, 2007140 School Other country Child Other questionnaire Fruit intake--unspecified

Tak, 2009141 School Other country Child Food frequency questionnaire

similar to the validated questionnaire of the Pro Children Study

Tester, 2016142 Early childhood education

US/Federal Child 24-hour recall 24-hour diet recall

Utter, 2016144 School Other country Child Other questionnaire Self-report

Veugelers, 2005146 School Other country Child Questionnaire Harvard Youth Adolescent Food Frequency Questionnaire (YAQ)

Page 280: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-102

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Wells, 2005150 School Other country Child Questionnaire Questionnaire was developed, which included a 24 h food tick list, food frequency questions, food attitude questions, questions about the NSFS and personal data ( Food tick list adapted from FAST)

Whetstone, 2012152 Community or neighborhood , faith-based, School ,

State/Regional Child Questionnaire Physical Activity and Nutrition (PAN) Monitoring Tool

Woodward-Lopez, 2010153 School State/ Regional Child Questionnaire survey

Wrigley, 2003154 Community or neighborhood

Other country Adult Other questionnaire National Diet and Nutrition Survey (NDNS)

Wrigley, 2003154 Community or neighborhood

Other country Adult Record/log Food consumption diary

Page 281: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-103

.

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Experimental Studies

Alaimo, 2013157 School State/Regional Child Food Frequency Questionnaire

Block Kids Food Frequency Questionnaires

Anderson, 2001158 Community or neighborhood

Local Adult Food frequency questionnaire

Modified version of BRFSS

Angelopoulos, 2009159 School Other country Child NR NR

Ayala, 2013162 Community or neighborhood

Local Adult Food frequency questionnaire

National Cancer Institute (NCI) Fruit and Vegetable All-Day Screener

Backman, 2011163 Employer or worksite

Local Adult Food frequency questionnaire

National Cancer Institute’s By-Meal Fruit and Vegetable Screener,

Baker, 2016164 Community or neighborhood

Non-Governmental

Adult Other Transtheoretical Model

Bere, 2005166 School Other country Child 24-recall 24-hour dietary recall

Bere, 2006167 School Other country Both Food frequency questionnaire

Food frequency questionnaire

Bere, 2006167 School Other country Child 24-recall Written 24-hour F&V recall

Bere, 2006168 After school or summer school

Other country Child 24-recall 24-hour fruit and vegetable recall

Bere, 2007169 School Other country Child 24-hour recall Self reported 24 hr recall and Food Frequency Questionnaire (with references cited for validity and reliability)

Beresford, 2010170 Employer or worksite

Non-Governmental

Adult Other questionnaire The questionnaire consisted of 46 questions pertaining to individual dietary behaviors, stages of change, taste preferences and perceptions, barriers to eating fruits and vegetables, autonomy in meal preparation, and social support

Page 282: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-104

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Cohen, 201429 School Non-Governmental

Child Food frequency questionnaire

2007 Block Food Screener

Crespo, 2012180 Community or neighborhood , School

Local Child Food frequency questionnaire

daily sevings of F&V

Day, 2008181 School Other country Child 24-recall hand-counting, using Canadian Nutrient File serving sizes,10 from a validated 24-Hour Food Recall questionnaire

Day, 2008181 School Other country Child Food frequency questionnaire

a Food Frequency Questionnaire (Food Frequency Questionnaire) adapted from the Eating at America's Table Study Quick Food Scan

De Coen, 2012182 School Other country Child Other F&V consumption

De Henauw, 2015184 Early childhood education or daycare School

Non-Governmental

Child Questionnaire Parent questionnaire

Dzewaltowski, 2009187 School Local Child Food frequency questionnaire

The Youth Adolescent Questionnaire (YAQ)

Page 283: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-105

.

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Economos, 2007189 Community or neighborhood

Non-Governmental

Child Questionnaire Instrument with validation

Elinder, 2012190 School Other country Child Questionnaire Health quesionnaire

Eriksen, 2003191 School Other country Child 24-hour recall developed for this study -precoded 24-hour recall and a short food-frequency questionnaire (Food Frequency Questionnaire)

Eriksen, 2003191 School Other country Child Food Frequency Questionnaire

developed for this study -precoded 24-hour recall and a short food-frequency questionnaire (Food Frequency Questionnaire)

Ermetici, 2016192 School Other country Child Questionnaire Italian National Institute of Health, adapted from a validated international standard questionnaire targeting adolescents

Esquivel, 2016193 Early childhood education or daycare

State/Regional Child Observation observed plate waste

Evans, 2013194 School Other country Child Food frequency questionnaire

24 h dietary assessment tool, the Child and Diet Evaluation Tool Intervention (CADET) diary

Foster, 2008199 School Local Child Questionnaire Youth/Adolescent Questionnaire

French, 2010200 Employer or worksite

Local Adult Food Frequency Questionnaire

self-report food frequency questionnaire (adapted from Thompson et al., 2002; Thompson et al., 1998)

Gatto, 2017202 School Local Child Food Frequency Questionnaire

Block Kids Food Screener

Gittelsohn, 2010204 Community or neighborhood

Non-Governmental

Adult Questionnaire the Customer Impact Questionnaire (CIQ) and the Child Customer Impact Questionnaire

Haerens, 2006208 School Local Child Food frequency questionnaire

NR

Page 284: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-106

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Haerens, 2007209 School Other country Child Food Frequency Questionnaire

pieces/week and below fruit recommendations

Hardy, 2010210 Early childhood education or daycare

State/Regional Child Observation Observation and census of lunch content

He, 2009211 School Other country Child 24-recall pre-coded 24 h fruit and vegetable recall

He, 2009211 School Other country Child Other questionnaire Pro-Children Questionnaire

Hendy, 2011212 School Local Child Food frequency questionnaire

FVFIRST

Hoefkens, 2011213 University Other country Adult record/log The awareness of participants of the relation between diet and health was measured by using the 7-point scale described by Ragaert et al (30)

Jago, 2011216 School Non-Governmental

Child Other Structured interviews, focus groups

Jones, 2015218 Early childhood education or daycare

Other country Child Observation Ball S, Benjamin S, Ward D. Development and reliability of an observation method to assess food intake of young children in child care. J Am Diet Assoc. 2007;107:656–61.

Page 285: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-107

.

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Jordan, 2008219 School State/Regional Child Questionnaire Student survey

Kastorini, 2016160 School Other country Child Food Frequency Questionnaire

KIDMED score

Kloek, 2006166 Community or neighborhood

Other country Adult Other questionnaire F&V grams/day

LaCaille, 2016168 Employer or worksite

State/Regional Adult Food frequency questionnaire

"The National Cancer Institute Multifactor Screener is a 17-item self-report food frequency questionnaire that estimates daily fruit and vegetable intake, grams of

fiber, and percent of energy from fat."

Llargues, 2011179 School Other country Child Food frequency questionnaire

food frequency questionnaire

Llargues, 2011179 School Other country Child Other questionnaire Krece Plus test

Lubans, 2016181 School Local Child Questionnaire Two items from the NSW Schools Physical Activity and Nutrition Survey (SPANS)

Lv, 2014183 Community or neighborhood , Employer or worksite

Local Adult Other questionnaire Study questionnaire

Madsen, 2015186 School Non-Governmental

Child Other digital images of students' lunch trays before and after meal consumption

Morrill, 2016197 School Local Child Other Blinded observers recorded F&V intake

Murphy, 2011201 School Other country Child Other questionnaire Dietary recall questionnaire

Neumark-Sztainer, 2010208 School Non-Governmental

Child 24-hour recall Dietary intake was assessed with one 24-hour dietary recall conducted by trained research staff (Nutrition Data System for Research software version 2006 developed by the Nutrition Coordinating Center (NCC), University of Minnesota, Minneapolis, MN.) Other measures were assessed with the New Moves survey (available at www.newmovesonline.com).

Page 286: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-108

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Nicklas, 2017210 School Local Child Observation Digital photography, Trained assessors used digital cameras to capture images of the vegetable dishes for initial serving, before additional servings, and after additional servings of vegetable dishes.

Ortega, 2016213 Community or neighborhood

Local Adult Other Townsend MS et al. Selecting items for a food behavior checklist for a limited-resource audience. J Nutr Educ Behav. 2003;35(2):69–82

Pbert, 2016217 School Non-Governmental

Child 24-hour recall 24-hour dietary recall interview22 using the Interactive Nutrition Data System (NDS, Nutrition Coordinating Center, University of Minnesota, Minneapolis, MN).

Perry, 2004218 School Non-Governmental

Child Observation Nutrition Data System (version 2.6, 1993, University of Minnesota, Minneapolis),

Ransley, 2007223 School Other country Child record/log Child and diet evaluation tool (CADET)

Reynolds, 2000228 School Local Child 24-recall 24-hour recall interview

Sharma, 2016245 Food assistance program

Other country Child Food Frequency Questionnaire

Block Kids Food validated food frequency questionnaire (for children) and itemFruits and Vegetables Screener (for parents)

Shive, 2006246 University State/Regional Adult Questionnaire Questionnaire

Page 287: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-109

.

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Steenhuis, 2004251 Employer or worksite

Other country Adult Food frequency questionnaire

F&V intake

Story, 2012253 School State/Regional Child Questionnaire In-person survey completed by parents

Te Velde, 2008265 School State/Regional Child 24-hour recall 24 hour recall method and food frequency questionnaire (Haraldsdo´ttir J, Tho´rsdo´ ttir I, de Almeida MDV, Maes L, Pe´rez Rodrigo C, Elmadfa I & Frost Andersen L (2005) Validity and reproducibility of a precoded questionnaire to assess fruit and vegetable intake in European 11- to 12-year-old schoolchildren. Ann Nutr Metab 49, 221–227.)

Te Velde, 2008265 School State/Regional Child Food Frequency Questionnaire

24 hour recall method and food frequency questionnaire (Haraldsdo´ttir J, Tho´rsdo´ ttir I, de Almeida MDV, Maes L, Pe´rez Rodrigo C, Elmadfa I & Frost Andersen L (2005) Validity and reproducibility of a precoded questionnaire to assess fruit and vegetable intake in European 11- to 12-year-old schoolchildren. Ann Nutr Metab 49, 221–227.)

Waters, 2017276 School Other country Child 24-recall 24-h record

Waters, 2017276 School Other country Child Other questionnaire Parental questionnaire covering issues such as family food habits, and usual intake of fruit, vegetable, dairy and drink consumption

Waters, 2017276 School Other country Child Other questionnaire Child questionnaire assessing food behaviours

Wright, 2012291 School State/ Regional Child Questionnaire The Child and Adolescent TRial for Cardiovascular Health After-School Student Questionnaire (ASSQ)--self administered survey for kids that measures dietary intake for previous day, healthy dietary behaviors, food knowledge, nutrition knowledge, food intentions, and dietary self efficacy.

Other Study Designs

Ashfield-Watt 20078 Community or neighborhood

Other country Adult Food frequency questionnaire

FACET – Five-a-day Community Evaluation Tool

Blake, 201328 Employer or worksite

Other country Adult Questionnaire they do not explain

Page 288: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-110

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Cheadle, 201252 After school or summer school, Community or neighborhood , Employer or worksite, School

Non-Governmental

Child Questionnaire Youth survey

de Silva-Sanigorski, 201179 Early childhood education or daycare

Non-Governmental

Child Questionnaire Eating and Physical Activity Questionnaire (EPAQ)

Geaney, 2010110 Employer or worksite

Other country Child 24-hour recall 24 hour recall and questionnaire

Kim, 2012164 School Other country Child Food frequency questionnaire

types of food purchased and how often they were consumed

Magarey, 2013190 School Other country Child Questionnaire Child nutrition questionnaire

Page 289: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-111

.

Author, year Setting Level Adult/Child Measure type Diet Fruit/Vegetable Intake Measure

Rogers, 2013235 After school or summer school, Community center (e.g., job training, youth), Community or neighborhood , Early childhood education or daycare , Employer or worksite, School

Non-Governmental

Child Questionnaire Questionnaire conducted by researcher

Tomlin, 2012267 School Other country Child 24-recall 24-hour dietary recall

Vasquez, 2016273 Employer or worksite

State/Regional Adult Food Frequency Questionnaire

Based on previously validated Block Food Frequency Questionnaire

Whaley, 2010284 Community or neighborhood

Local Child Food frequency questionnaire

Questions from WIC child

BRFSS= Behavioral Risk Factor Surveillance System; CADET=Child and Diet Evaluation Tool; CHIS= California Health Interview Survey; F&V=fruits and vegetables; FVFIRST=Fruit and Vegetable FIRST; NHANES=National Health and Nutrition Examination Survey; No.=Number; NR=not reported; US=United States; WIC= Women, Infants, and Children program

Page 290: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-112

Evidence Table H12. Change in fiber intake measures Author, year Setting Level Adult/Child Measure type Diet Fiber Measure

Natural Experiment Studies

Cullen, 200667 School Local/Other country

Child record/log Anonymous food record entered into Nutrition Data System (version 4.2), Objective measure: Electronic data from POS purchase machine

Cullen, 200866 School State/Regional Child Record/log Anonymous record immediately after lunch

Fung, 2013108 School Other country Child Questionnaire Harvard Youth Adolescent Food Frequency Questionnaire (YAQ) adapted for Canadian settings

Hilmers, 2014136 Community or neighborhood , Food assistance program

US/Federal Adult record/log 24-hour dietary food record

Leung, 2013174 Food assistance program

US/Federal Child Questionnaire NHANES

Taber, 2002257 School State/Regional Child 24-recall National Youth Physical Activity and Nutrition Study (NYPANS), written survey and 24 hour recall

Experimental Studies

Alaimo, 20133 School State/Regional Child Food Frequency Questionnaire

Block Kids Food Frequency Questionnaires

Ayala, 201311 Community or neighborhood

Local Adult Other questionnaire shorter version of a thirty-item scale used to assess behavioural strategies to increase fibre and decrease fat

LaCaille, 2016168 Employer or worksite

State/Regional Adult Food Frequency Questionnaire

The National Cancer Institute Multifactor Screener is a 17-item self-report food frequency questionnaire that estimates daily fruit and vegetable intake, grams of

fiber, and percent of energy from fat.

Gatto, 2017109 School Local Child Food Frequency Questionnaire

Block Kids Food Screener

Sharma, 2016245 Food assistance program , School

Other country Child Food Frequency Questionnaire

Block Kids Food validated food frequency questionnaire

Page 291: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-113

Author, year Setting Level Adult/Child Measure type Diet Fiber Measure

Wright, 2012291 School State/ Regional Child Questionnaire The Child and Adolescent Trial for Cardiovascular Health After-School Student Questionnaire (ASSQ)

NHANES= National Health and Nutrition Examination Survey; No.=Number; US=United States

Page 292: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-114

Evidence Table H13. Change in sugar sweetened beverage intake measures Author, year Setting Level Adult/Child Measure type Diet SSB Intake Measure

Natural Experiment Studies

Bauhoff, 201418 School State/ Regional Child Questionnaire The California Healthy Kids Survey (CHKS)

Bowling, 201633 Community or neighborhood , Food assistance program

State/Regional Adult Questionnaire Internally designed surveys

Cradock, 201164 School Local Child Questionnaire The Boston Youth Survey has a 7-day recall horizon, and the NHANES recall covers the previous 24-hour period

Cullen, 200667 School Local/Other country

Child record/log Anonymous food record entered into Nutrition Data System (version 4.2; Nutrition Coordinating Center, University of Minnesota, Minneapolis, MN, USA), diet_ssb_obj: Electronic data from POS purchase machine

Cullen, 200866 School State/Regional Child Record/log Anonymous record immediately after lunch

Datar, 201671 School Local Child Food frequency questionnaire

Beverage and Snack Questionnaire

de Visser, 201680 School Non-Governmental

Child Questionnaire the School Physical Activity and Nutrition (SPAN) survey

Elbel, 201588 Community or neighborhood

Local Adult 24-recall 24-hours dietary recall survey

Elbel, 201588 Community or neighborhood

Local Adult Other questionnaire Eating and Physical Activity Questionnaire (EPAQ)

Elbel, 201787 Community or neighborhood

State/Regional Adult Food frequency questionnaire

Eating and Physical Activity Questionnaire

Falbe, 201695 Community or neighborhood

Local Adult Questionnaire We assessed beverage consumption via interviewer-administered intercept surveys with a beverage frequency questionnaire modified from the Behavioral Risk Factor Surveillance System 2011 SSB module

Fung, 2013108 School Other country Child Questionnaire Harvard Youth Adolescent Food Frequency Questionnaire (YAQ) adapted for Canadian settings

Page 293: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-115

Author, year Setting Level Adult/Child Measure type Diet SSB Intake Measure

Hilmers, 2014136 Community or neighborhood

US/Federal Adult record/log 24-hour dietary food record

Jia, 2017152 School Other country Child Questionnaire Self-reported weekly frequency of sugary beverage

King, 2014165 School Local Child 24-hour recall Student Health Assessment Questionnaire/ 24-hour recall

Leung, 2013174 Food assistance program

US/Federal Child Questionnaire NHANES

Liu, 2016178 Food assistance program

US/Federal Child Questionnaire Telephone survey

Page 294: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-116

Author, year Setting Level Adult/Child Measure type Diet SSB Intake Measure

Malakellis, 2017191 School Other country Child Questionnaire Adolescent Behaviours Attitudes and Knowledge Questionnaire (ABAKQ)

Masse, 2014192 School Local Child Questionnaire BC Adolescent Health Survey (AHS)

Molitor, 2015196 Home US/Federal Both Food frequency questionnaire

2011–2012 CHIS

Nanney, 2014202 School State/Regional Child Questionnaire Instrument with validation

Nanney, 2016203 School Local Child Questionnaire Questionnaire

Nguyen, 2015209 After school or summer school

US/Federal Adult Questionnaire National Health and Nutrition Examination Survey

Robles, 2017234 Community or neighborhood

Local Adult Food frequency questionnaire

National Institute of Health’s Eating at America’s Table Quick Food Scan

Sturm, 2010256 School US/Federal Child Other questionnaire child food consumption questionnaire

Sturm, 2015255 Community or neighborhood

Local Adult Other questionnaire Telephone interview

Taber, 2011261 School State/ Regional Child Questionnaire Survey (not specified)

Taber, 2012260 School State/ Regional Child Questionnaire Staff administered questionnaire

Whetstone, 2012285 Community or neighborhood , faith-based, School ,

State/Regional Child Questionnaire Physical Activity and Nutrition (PAN) Monitoring Tool

Woodward-Lopez, 2010289 School State/ Regional Child Questionnaire survey

Experimental Studies

Bere, 200524 School Other country Child 24-recall 24-hour recall

Page 295: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-117

Author, year Setting Level Adult/Child Measure type Diet SSB Intake Measure

Bere, 200725 School Other country Child 24-hour recall Self reported 24 hr recall and Food Frequency Questionnaire (with references cited for validity and reliability)

Blum, 200829 School State/ Regional Child Food Frequency Questionnaire

youth food frequency questionnaire

Crespo, 201265 Community or neighborhood , School

Local Child Food frequency questionnaire

daily sevings of SSB

De Henauw, 201577 Early childhood education or daycare School

Non-Governmental

Child Questionnaire Parent questionnaire

Economos, 200786 Community or neighborhood

Non-Governmental

Child Questionnaire

Elinder, 201289 School Other country Child Questionnaire Questionnaire

Ermetici, 201691 School Other country Child Questionnaire Italian National Institute of Health, adapted from a validated international standard questionnaire targeting adolescents

Page 296: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-118

Author, year Setting Level Adult/Child Measure type Diet SSB Intake Measure

French, 2010104 Employer or worksite

Local Adult Food Frequency Questionnaire

self-report food frequency questionnaire (adapted from Thompson et al., 2002; Thompson et al., 1998)

Haerens, 2006127 School Local Child Food frequency questionnaire

NR

Haerens, 2007126 School Other country Child Food Frequency Questionnaire

glasses/day

Hardy, 2010129 Early childhood education or daycare

State/Regional Child Observation Observation and census of lunch content

Hendy, 2011133 School Local Child Food frequency questionnaire

HDRINK

Jago, 2011149 School Non-Governmental

Child Other Structured interviews, focus groups

Jordan, 2008155 School State/Regional Child Questionnaire Parent survey

Lubans, 2016181 School Local Child Questionnaire Two items from the NSW Schools Physical Activity and Nutrition Survey (SPANS)

Madsen, 2015186 School Non-Governmental

Child Questionnaire Survey questions adapted from the School Physical Activity and Nutrition Questionnaire and the Child Food Consumption Questionnaire

Pbert, 2016217 School Non-Governmental

Child 24-hour recall 24-hour dietary recall interview22 using the Interactive Nutrition Data System (NDS, Nutrition Coordinating Center, University of Minnesota, Minneapolis, MN).

Sharma, 2016245 Food assistance program , School

Other country Child Food Frequency Questionnaire

Block Kids Food validated food frequency questionnaire

Waters, 2017276 School Other country Child 24-recall 24-h record

Waters, 2017276 School Other country Child Other questionnaire Parental questionnaire covering issues such as family food habits, and usual intake of fruit, vegetable, dairy and drink consumption

Page 297: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-119

Author, year Setting Level Adult/Child Measure type Diet SSB Intake Measure

Waters, 2017276 School Other country Child Other questionnaire Child questionnaire assessing food behaviours

Page 298: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-120

Author, year Setting Level Adult/Child Measure type Diet SSB Intake Measure

Other Study Designs

de Silva-Sanigorski, 201179 Early childhood education or daycare

Non-Governmental

Child Questionnaire Eating and Physical Activity Questionnaire (EPAQ)

Kim, 2012164 School Other country Child Food frequency questionnaire

types of food purchased and how often they were consumed

Rogers, 2013235 After school or summer school, Community center (e.g., job training, youth), Community or neighborhood , Early childhood education or daycare , Employer or worksite, School

Non-Governmental

Child Questionnaire Questionnaire conducted by researcher

Tomlin, 2012267 School Other country Child 24-recall 24-hour dietary recall

Whaley, 2010284 Community or neighborhood

Local Child Food frequency questionnaire

Questions from WIC child questionnaire

ABAKQ=Adolescent Behaviours Attitudes and Knowledge Questionnaire; CHIS=California Health Interview Survey; NHANES=National Health and Nutrition Examination Survey; No.=number; NR=not reported; SSB=sugar sweetened beverage; US=United States; WIC=Women, Infants, and Children program

Page 299: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-121

Evidence Table H14. Change in physical activity measures Author, year Setting Level Adult/Child Measure type Physical activity measure

Natural Experiment Studies

Anthamatten, 20117 School State/Regional Child Observation System for Obser- ving Play and Leisure Activity in Youth (SOPLAY)

Azevedo, 201412 School Other country Child Electronic monitor Accelerometer: Light PA, vigororous PA

Azevedo, 201412 School Other country Child Observation 20m shuttle run test (to test aerobic fitness)

Barroso, 200916 School State/Regional Child Observation SOFIT (System for observing Fitness Instruction Time)

Barroso, 200916 School State/Regional Child Questionnaire SPAN Questionnaire

Bauman, 200319 Community or neighborhood

Other country Adult Questionnaire New Zealand Sport and Physical Activity Survey

Bauman, 200319 Early childhood education or daycare

Other country Adult Questionnaire Stand alone household interviews

Benjamin Neelon, 201520

School Local Child Electronic monitor accelerometer (ActiGraph GT1M, Pensacola, FL)

Bolton, 201730 Employer or worksite, School

Other country Both Questionnaire Victorian Population Health Survey

Bolton, 201730 Employer or worksite, School

Other country Child Questionnaire Victorian state government Local Level Child Health and Wellbeing Survey

Branas, 201134 Community or neighborhood

Local Adult Questionnaire Self reported (Southeastern Pennsylvania Household Health Survey)

Brown, 200835 Community or neighborhood

Local Adult Questionnaire BRFSS

Brown, 201538 Transportation Local Adult Electronic monitor accelerometer counts per minute

Page 300: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-122

Author, year Setting Level Adult/Child Measure type Physical activity measure

Brown, 201637 Transportation, Community or neighborhood

Local Adult Electronic monitor Accelerometer to measure moderate to vigorous PA

Brown, 201636 Transportation Local Adult Electronic monitor Accelerometers and GPS units

Buscail, 201643 Community or neighborhood

Other country Adult Questionnaire Recent Physical Activity Questionnaire (RPAQ) validated in French

Calise, 201347 Community or neighborhood

Local Adult Questionnaire Neighborhood Physical Activity Questionnaire (NPAQ)

Cawley, 200751 School State/Regional Child Questionnaire Survey

Cawley, 200750 School State/Regional Child Questionnaire YRBSS Questionnaire

Page 301: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-123

Author, year Setting Level Adult/Child Measure type Physical activity measure

Chen, 201553 Home Other country Adult Other Compendium of Physical Activities

Cleland, 200855 School Other country Adult Questionnaire International Physical Activity Questionnaire (IPAQ-L), Wearable device:Yamax Digiwalker pedometer

Cohen, 201258 Community or neighborhood

Local Both Observation METs: metabolic equivalents

Cohen, 201258 Community or neighborhood

Local Both Questionnaire Self-reported park use: self reprted park use , and engaging in more exercise

Cradock, 201463 School Local Child Electronic monitor accelerometers (GT3X/GT1M or MTI/CSA 7164, Actigraph, Pensacola, Florida)

De Cocker, 200773 Community or neighborhood , Transportation

Other country Adult Questionnaire Total time for physical activity, minutes/wk

De Cocker, 200773 Community or neighborhood , Transportation

Other country Adult Questionnaire Total time household PA, minutes/wk

De Cocker, 200773 Community or neighborhood , Transportation

Other country Adult Questionnaire Total time leisure time PA, minutes/wk

De Cocker, 200773 Community or neighborhood , Transportation

Other country Adult Questionnaire Total time moderate PA, minutes/wk

De Cocker, 200773 Community or neighborhood , Transportation

Other country Adult Questionnaire Total time transport-related PA, minutes/wk

De Cocker, 200773 Community or neighborhood , Transportation

Other country Adult Questionnaire Total time vigorous PA, minutes/wk

Page 302: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-124

Author, year Setting Level Adult/Child Measure type Physical activity measure

De Cocker, 200773 Community or neighborhood , Transportation

Other country Adult Questionnaire Total time walking, minutes/wk

De Cocker, 200773 Community or neighborhood , Transportation

Other country Adult Questionnaire Total time work-related PA, minutes/wk

de Visser, 201680 School Non-Governmental Child Questionnaire the School Physical Activity and Nutrition (SPAN) survey

Dill, 201481 Community or neighborhood

Local Both Electronic monitor accelerometer data

Page 303: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-125

Author, year Setting Level Adult/Child Measure type Physical activity measure

Dill, 201481 Community or neighborhood

Local Both Electronic monitor GPS tracking

Dill, 201481 Community or neighborhood

Local Both Record/log Travel diaries

Eagle, 201385 School State/Regional Child Questionnaire School-Based Nutrition Monitoring Questionnaire

Fitzpatrick, 201799 School Other country Child Electronic monitor Accelerometer (Actigraph LS 7164)

Fuller, 2013107 Community or neighborhood

Other country Adult Questionnaire International Physical Activity Questionnaire (IPAQ)

Gee, 2015113 School State/Regional Child Questionnaire Centers for Disease Control and Prevention. Youth Risk Behavior Survey (YRBS). Washington, D.C.

Giles-Corti, 2013115 Community or neighborhood

Other country Adult Questionnaire Mean minutes of transportation-related walking.

Goodman, 2014121 Community or neighborhood

Other country Adult Questionnaire 7-day recall instrument

Goodman, 2014121 Community or neighborhood

Other country Adult Questionnaire adapted version of the short form of the International Physical Activity Questionnaire40

Goodman, 2016122 School, Transportation

Other country Child Other parent report of frequency of child's cycling

Gorely, 2011123 School Other country Child Electronic monitor number of steps taken in the previous 24 hours

Herrick, 2012135 After school or summer school

Non-Governmental Child Questionnaire California Healthy Kids Survey; the NHBLI Growth and Health Study,10 and the Healthy Eating Active Communities survey,11 that assessed frequency of exercise (days per week), enjoyment of sports (I enjoy activities such as walking, playing ball, bike riding, dancing or skating) on a 4-point scale, and perception of physical activity level (Compared to most [boys/girls] your age, would you say you are: less active, about as active, or more active).

Hobin, 2014138 School Other country Child Electronic monitor Accelerometer

Page 304: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-126

Author, year Setting Level Adult/Child Measure type Physical activity measure

Hobin, 2017137 School Other country Child Electronic monitor Accelerometer

Hoelscher, 2016141 School Other country Child Questionnaire School Physical Activity and Nutrition (SPAN) questionnaire.

Hoelscher, 2016140 School State/Regional Child Questionnaire Questionnaires

Hunter, 2016148 School Other country Child Questionnaire COMPASS Student Questionnaire (Cq): MVPA measured by two questions on the Cq

Kim, 2012163 School State/Regional Child Questionnaire NSCH survey

Page 305: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-127

Author, year Setting Level Adult/Child Measure type Physical activity measure

King, 2014165 School Local Child Questionnaire Student Health Assessment Questionnaire/ 24-hour recall

Johnson, 2017153 Community or neighborhood

Other country Adult Questionnaire Number of episodes of physical activity or walking of at least 30 min per week

Lachapelle, 2009169 Community or neighborhood , Transportation

Local Adult Questionnaire Total distance walked for transportation: non-walker; walk up to 2.4 km=moderate; walk more than 2.4 km=meets PA recommendation

LaRowe, 2016170 Early childhood education or daycare

State/Regional Child Electronic monitor Child physical activity (PA) was measured using Actical triaxial accelerometers (Bio-Lynx, Montreal, Quebec, Canada)

LaRowe, 2016170 Early childhood education or daycare

State/Regional Child Questionnaire modified version of the Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC) - only at baseline

Ling, 2014177 Community or neighborhood , School

Local Child Electronic monitor Pedometer

MacDonald, 2010184 Transportation Local Adult Questionnaire International Physical Activity Questionnaire (IPAQ)

Maddock, 2006185 Community or neighborhood, School

State/Regional Child Questionnaire Surveys

Malakellis, 2017191 School Other country Child Questionnaire The Adolescent Behaviours, Attitudes, and Knowledge Questionnaire (ABAKQ)

Harding, 2017128 Community or neighborhood

State/Regional Adult Other Behavioral Risk Factor Surveillance System (BRFSS)/leisure time PA

Miller, 2015195 Transportation, Community or neighborhood

Local Adult Electronic monitor measured overall PA, transit-related PA, and other PA

Page 306: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-128

Author, year Setting Level Adult/Child Measure type Physical activity measure

Molitor, 2015196 Home US/Federal Both Questionnaire 2011–2012 CHIS

Morton, 2016198 School Other country Child Electronic monitor Actigraph (GT1M; Pensacola, FL) accelerometer

Mumford, 2011200 Community or neighborhood , Transportation

Local Adult Questionnaire The 148-item self-administered survey

Page 307: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-129

Author, year Setting Level Adult/Child Measure type Physical activity measure

Nanney, 2014202 School State/Regional Child Questionnaire Minnesota Student Survey (MSS)

Nehme, 2017207 Employer or worksite

State/Regional Adult Questionnaire Past Week Modifiable Activity Questionnaire (PWMAQ)

Nehme, 2017207 Employer or worksite

State/Regional Adult Questionnaire the International Physical Activity Questionnaire Long form (IPAQ-L)

Panter, 2016214 Transportation Other country Adult Questionnaire Recent Physical Activity Questionnaire (RPAQ)

Peterson, 2015219 After school or summer school, School

State/Regional Child Questionnaire use of risk behavior survey-daily physical activity

Quig, 2012222 Community or neighborhood

Local Child Electronic monitor GT1M (Actigraph, Fort Walton Beach, FL)

Quig, 2012222 Community or neighborhood

Local Child Questionnaire self-administered questionnaire

Reger-Nash, 2005224 Community or neighborhood

Local Adult Questionnaire Telephone survey

Reger-Nash, 2008225 Community or neighborhood

Non-Governmental Adult Questionnaire Behavioral Risk Factor Surveillance System physical activity questions and media-recall questions

Reger-Nash, 2008225 Community or neighborhood

Non-Governmental Adult Questionnaire random-digit-dial telephone surveys

Ridgers, 2007231 School Other country Child Electronic monitor physical activity levels during recess were quantified using heart rate (HR) telemetry and accelerometry

Ridgers, 2007230 School Other country Child Electronic monitor Accelerometry, ActiGraph (Model 7164, MTI Health Services, Florida, USA)

Riis, 2012232 School State/Regional Child NR instrument not described

Sabia, 2016238 School State/Regional Child Questionnaire NYRBS

Page 308: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-130

Author, year Setting Level Adult/Child Measure type Physical activity measure

Slater, 2014249 School State/Regional Child Questionnaire Days physically active for 60 min/day

Slater, 2014249 School State/Regional Child Questionnaire Vigorous exercise--no timing.

Stephens, 2014252 Early childhood education or daycare

Local Child Electronic monitor accelerometer (ActiGraph GT3X)

Stratton, 2005254 After school or summer school

Other country Child Electronic monitor Sportstester heart rate telemeters (Electro-Polar, Kempele, Finland)

Taber, 2013259 School State/ Regional Child Questionnaire Written questionnaire, students reported the numberof days they engaged in at least 20 mins of activity that made them sweat or breathe hard in the past week

Veugelers, 2005274 School Other country Child NR NR

Page 309: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-131

Author, year Setting Level Adult/Child Measure type Physical activity measure

Webb, 2016278 Transportation Other country Adult Questionnaire English Longitudinal Study of Ageing (ELSA)

West, 2011283 Community or neighborhood

Local Adult Questionnaire Non-validated survey questions

Whetstone, 2012285 Community or neighborhood , faith-based, School ,

State/Regional Child Questionnaire Physical Activity and Nutrition (PAN) Monitoring Tool

Zhu, 2013295 Community or neighborhood

Local Adult Questionnaire NR

Zhu, 2014296 Community or neighborhood

Local Adult Record/log Self-reported

Experimental Studies

Angelopoulos, 20096 School Other country Child Questionnaire Standardized questionnaire,

Audrey, 201510 Employer or worksite

Other country Adult Electronic monitor ActiGraph GT3X+; ActiGraph LLC, FL, US

Audrey, 201510 Employer or worksite

Other country Adult Questionnaire Study questionnaire

Bastian, 201517 School Non-Governmental Child Electronic monitor number of steps during weekdays and weekends Omron Hj-720ITC time-stamped piezoelectric pedometer (Omron, Toronto Ontario, Canada

Bonvin, 201332 Early childhood education or daycare

Other country Child Other Motor skill measures were adapted from the Zurich Neuromotor Assessment (ZNA) test,

Caballero, 200345 School State/Regional Child Electronic monitor motion sensor mesures PA over a 24 hour period

Page 310: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-132

Author, year Setting Level Adult/Child Measure type Physical activity measure

Caballero, 200345 School State/Regional Child Questionnaire self-reported PA

Chomitz, 201054 School Non-Governmental Child Other Fitness test scores

Cochrane, 201256 Primary care centers

Other country Adult Questionnaire General Practice Physical Activity Questionnaire

Cortinez-O'Ryan, 201761 Community or neighborhood

Other country Child Electronic monitor pedometer

Page 311: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-133

Author, year Setting Level Adult/Child Measure type Physical activity measure

Crespo, 201265 Community or neighborhood , School

Local Child Questionnaire per parents: comparison of their child's PA to others

Crespo, 201265 Community or neighborhood , School

Local Child Questionnaire per parents: sports participation over the past year

de Greeff, 201676 School Other country Child Questionnaire Eurofit physical fitness test battery

De Henauw, 201577 Early childhood education or daycare, School

Non-Governmental Child Electronic monitor The ActiGraph and the ActiTrainer (Pensacola, FL, USA) accelerometers

Dunton, 201583 School Non-Governmental Child Electronic monitor Actigraph GT2M accelerometers

Dunton, 201583 School Non-Governmental Child Record/log Time diaries completed by parents

Dzewaltowski, 200984 School Local Child Questionnaire Previous Day Physical Activity Recall (PDPAR)

Elinder, 201289 School Other country Child Electronic monitor Accelorometer (Actigraph GT1M)

Elinder, 201289 School Other country Child Questionnaire Questionnaire

Ermetici, 201691 School Other country Child Questionnaire A simple self-completion questionnaire designed by the Italian National Institute of Health, adapted from a validated international standard questionnaire targeting adolescents

Fairclough, 201694 School Other country Child Electronic monitor ActiGraph GT9X

Farley, 200796 Community or neighborhood

Local Child Observation modified System of Observing Play and Leisure Activity in Youth measure, observation within 8x8 block area of neighborhood

Farley, 200796 Community or neighborhood

Local Child Record/log Child annual self-reported survey of physical activity

Page 312: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-134

Author, year Setting Level Adult/Child Measure type Physical activity measure

Farmer, 201797 School Other country Child Electronic monitor ActiGraph GT3X

Finch, 201498 Early childhood education or daycare

State/Regional Child NR NR

Foster, 2008102 School Local Child Questionnaire Youth/Adolescent Activity Questionnaire, a self-administered 24-item questionnaire

French, 2010104 Employer or worksite

Local Adult Questionnaire Godin leisure time physical activity questionnaire

Page 313: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-135

Author, year Setting Level Adult/Child Measure type Physical activity measure

Fu, 2016106 School State/Regional Child Electronic monitor Pedometer

Goetzel, 2010119 Employer or worksite

Non-Governmental Adult Questionnaire using several health risk assessment questions

Gustat, 2012125 Community or neighborhood

Non-Governmental Adult Questionnaire self-reported PA through interviewer-administered household surveys conducted door to door, SOPLAY (System for Observing Play and Leisure Activity in Youth) methods to objectively measure neighborhood PA on streets, sidewalks, and outside public areas on every block in each of the 3 neighborhoods

Haerens, 2006127 School Local Child Questionnaire Adapted version of the International Physical Activity Questionnaire

Hardy, 2010129 Early childhood education or daycare

State/Regional Child Questionnaire Test of Gross Movement Development (TMGD-2) checklist,

Hendy, 2011133 School Local Child Electronic monitor Pedometer

Hollis, 2016142 School Non-Governmental Child NR NR

Huberty, 2011146 School State/Regional Child Electronic monitor MVPA (defined as 3 METs)

Jago, 2011149 School US/Federal Child Other Met-S, PA activity

Janssen, 2015150 School Other country Child Electronic monitor Accelerometers

Janssen, 2015150 School Other country Child Questionnaire SOPLAY protocol

Jones, 2015154 Early childhood education or daycare

Other country Child Observation System for Observing Play and Leisure in Youth (SOPLAY) tool

Jordan, 2008155 School State/Regional Child Questionnaire Parent and student survey

Page 314: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-136

Author, year Setting Level Adult/Child Measure type Physical activity measure

Jurg, 2006156 After school or summer school, School

Non-Governmental Child Other analysing participation lists

Kain, 2004158 School Other country Child Other 20 m shuttle run test

Kamada, 2013159 Community or neighborhood

Local Adult Questionnaire Self-administered

Kloek, 2006166 Community or neighborhood

Other country Adult Questionnaire 30 minutes of MVPA/week

LaCaille, 2016168 Employer or worksite

State/Regional Adult Questionnaire Two self-report measures were used to gather data about physical activity; the Godin Leisure Time Exercise Questionnaire (GLTEQ), The International Physical Activity Questionnaire (IPAQ)

Page 315: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-137

Author, year Setting Level Adult/Child Measure type Physical activity measure

Linde, 2012176 Employer or worksite

Non-Governmental Adult Electronic monitor stati use recorded by laser counter

Llargues, 2011179 School Other country Child Record/log Self-reported

Lorentzen, 2009180 Community or neighborhood

Other country Adult Questionnaire an instrument developed by Prochaska and Marcus (1994)

Lorentzen, 2009180 Community or neighborhood

Other country Adult Record/log self-reported weekly strenuous physical activity

Lubans, 2016181 School Local Child Electronic monitor actigraph accelerometers, CPM and MVPA

Lv, 2014183 Community or neighborhood , Employer or worksite

Local Adult Questionnaire International Physical Activity Questionnaire (IPAQ)

Madsen, 2013187 After school or summer school

Non-Governmental Child Electronic monitor GT1M or GT3X accelerometer (Actigraph LLC)

Madsen, 2015186 School Non-Governmental Child Electronic monitor Actigraph GT1M or GT3X accelerometer (AG; Actigraph, LLC, Pensacola, FL)

Naylor, 2006206 School Other country Child Record/log Activity Logs

Naylor, 2008205 School Other country Child Electronic monitor pedometers

Neumark-Sztainer, 2010208

School Non-Governmental Child Questionnaire Physical activity was assessed with the 3-Day Physical Activity Recall (3-DPAR) survey. (Pate RR, Ross R, Dowda M, Trost SG, Sirard JR. Validation of a three-day physical activity recall instrument in female youth. Pediatric Exercise Science 2003;15:257–265.)

Pate, 2005216 School Non-Governmental Child Questionnaire The 3-Day Physical Activity Recall (3DPAR), a modification of the Previous Day Physical Activity Recall

Pbert, 2016217 School Non-Governmental Child NR NR

Page 316: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-138

Author, year Setting Level Adult/Child Measure type Physical activity measure

Ploeg, 2014272 School Other country Child Electronic monitor Omron HJ-720 ITC time-stamped pedometer

Ridgers, 2010229 School Other country Child Electronic monitor Heart rate telemetry (The Polar Team System) and Accelorometers ( Actigraph)

Sallis, 2003240 School Local Child Observation The validated SOFIT (System for Observing Fitness Instruction Time) method27,28 was used to evaluate student physical activity in a random sample of PE classes. The SOPLAY (System for Observing Play and Leisure Activity of Youth) method29 was developed for the present study to assess the number and activity level of students during leisure times. For SOPLAY observations, all locations used for physical activity at school were identified, and observers collected data in all locations before school, after lunch, and after school on randomly selected days.

Sigmund, 2012247 School Other country Child Electronic monitor Yamax Digiwalker SW-200 pedometer (Yamax Corporation, Tokyo, Japan)

Sigmund, 2012247 School Other country Child Record/log a PA log book for inputting the Yamax data

Page 317: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-139

Author, year Setting Level Adult/Child Measure type Physical activity measure

Simon, 2008248 School Non-Governmental Child Record/log Self reported leisure PA using the Modified Activity Questionnaire for adolescents

Story, 2012253 School State/Regional Child Questionnaire Survey completed by classroom and PE teachers

Van Cauwenberghe 2012271

School Other country Adult Electronic monitor accelerometry (GT1M Actigraph; 15 s epoch)

Wells, 2014281 School Local Child Electronic monitor Actigraph GT3X+ or GT1M accelerometers

Wells, 2014281 School Local Child Observation the Physical Activity Research & Assessment tool for Garden Observation (PARAGON), direct observation was conducted by trained research staff

Wells, 2014281 School Local Child Questionnaire The Girls Health Enrichment Multi-site Study (GEMS) Activity Questionnaire (GAQ)

Tarp, 2016264 School Other country Child Electronic monitor Accelerometer (GT3X and GT3X+ devices ActiGraph LLC, Pensacola, FL,USA)

Waters, 2017276 School Other country Child Observation SOPLAY (System for Observing Play and Leisure Activity in Youth)

Waters, 2017276 School Other country Child Questionnaire Primary study parent questionnaires: covering issues such as family physical activities and child sedentary and physical activities and level of active transport

Waters, 2017276 School Other country Child Questionnaire Primary study child questionnaire: covering issues such as family physical activities and child sedentary and physical activities and level of active transport (

Whitt-Glover, 2011286 School Local Child Observation System for Observing Instructional Fitness Time (SOFIT)

Williamson, 2012287 School State/Regional Child Questionnaire Self-Administered Physical Activity Checklist (SAPAC)

Wilson, 2015288 Community or neighborhood

Non-Governmental Adult Observation Staff collected data

Wright, 2013290 After school or summer school, School

Non-Governmental Child Questionnaire Child and Adolescent Trial for Cardiovascular Health (CATCH) School Physical Activity and Nutrition (SPAN) Student Questionnaire.

Page 318: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-140

Author, year Setting Level Adult/Child Measure type Physical activity measure

Yildirim, 2014293 School Other country Child NR NR

Zhou, 2014294 Early childhood education or daycare

Other country Child Electronic monitor accelerometers (GT3X, Acti- Graph Manufacturing Technology Inc., FL., USA)

Zhou, 2014294 Early childhood education or daycare

Other country Child Other battery test from the Chinese National Measurement Standards on People’s Physical Fitness for young children

Page 319: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-141

Author, year Setting Level Adult/Child Measure type Physical activity measure

Other Study Designs

Blake, 201328 Employer or worksite

Other country Adult Questionnaire Physical Activity Questionnaire

Brownson 200439 Community or neighborhood

Local Adult Other Personal cards swiped as they entered and left the trail

Brownson 200439 Community or neighborhood

Local Adult Questionnaire The self reported (telephone survey) walking behavior assessment used a four-item scale developed at San Diego State University

Brownson, 200540 Community or neighborhood

US/Federal Adult Questionnaire NR

Brusseau, 201641 School State/Regional Child Electronic monitor Yamax DigiWalker CW600 pedometers (Tokyo, Japan) and ActiGraph wGT3X-BT triaxial accelerometers (Pensacola, FL).

Cheadle, 201252 After school or summer school, Community or neighborhood , Employer or worksite, School

Non-Governmental Child Other Fitnessgram test

De Cocker, 201174 Community or neighborhood

Other country Adult Electronic monitor Pedomotor: Yamax Digiwalker SW-200 (Yamax, Tokyo, Japan)

De Cocker, 201174 Community or neighborhood

Other country Adult Questionnaire International Physical Activity Questionnaire (IPAQ)

De Cocker, 201174 Community or neighborhood

Other country Adult Record/log Activity log

Page 320: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-142

Author, year Setting Level Adult/Child Measure type Physical activity measure

de Silva-Sanigorski, 201179

Early childhood education or daycare

Non-Governmental Child Questionnaire Romp & Chomp Audit survey

Gebel, 2011112 Community or neighborhood

Local Adult Questionnaire Participants reported the number of days per week and the minutes per day of walking for at least 10 minutes at a time

Heelan, 2009131 School State/Regional Child Electronic monitor steps per day

Heelan, 2009131 School State/Regional Child Questionnaire School-wide prevalence of walking to school

Huberty, 2013145 After school or summer school

Non-Governmental Child NR NR

Naul, 2012204 School Other country Child Other amount of specific activities

Page 321: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-143

Author, year Setting Level Adult/Child Measure type Physical activity measure

Rogers, 2013235 After school or summer school, Community center (e.g., job training, youth), Community or neighborhood , Early childhood education or daycare , Employer or worksite, School

Non-Governmental Child Questionnaire Questionnaire conducted by researcher

Tomlin, 2012267 School Other country Child Questionnaire Physical Activity Questionnaire for Children (PAQ-c) or Adolescents (PAQ-a)

Weaver, 2017277 School

Local Child Electronic monitor accelerometer (ActiGraph GT3Xþ, Shalimar, Florida)

Whaley, 2010284 Community or neighborhood

Local Child Questionnaire Questions taken from the standaed WIC child questionnaire

CPM=counts per minute; GPS=global positioning system; MET=metabolic equivalent of task; MVPA=moderate to vigorous physical activity; No.=number; NR=not reported; NSCH=National Survey of Children’s Health; NYRBS=National Youth Risk Behavior Survey; PA=physical activity; PE=physical education; SPAN=School Physical Activity and Nutrition Survey; US=United States; YRBSS=Youth Risk Behavior Surveillance System

Page 322: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-144

Evidence Table H15. Co-outcomes measures Author, year Setting Co-Outcome Co-Outcome Description

Natural Experiment Studies

Bolton, 201730 School Food environment School-related food services, the existence of nutrition/physical activity policies and the environment

Bolton, 201730 School PA environment the existence of nutrition/physical activity policies and the environment

Brown, 200835 Community or neighborhood

Commuting behavior Transport behaviour

Calise, 201347 Community or neighborhood

PA environment Neighborhood characteristics to facilitate walking

Cohen, 201258

Other METs generated per cost of equipment

Cullen, 200667 School Purchasing behavior more vending machines installed in schools

Cummins, 200868 Community or neighborhood

Other Retail shop count survey

Dubowitz, 201582 Community or neighborhood

Purchasing behavior questionnaire

Elbel, 201787 Community or neighborhood

Purchasing behavior how often they usually bought fresh fruits and vegetables; and where they usually shopped for food

Fitzpatrick, 201799 School Food environment record specific features of the school food environment

Fitzpatrick, 201799 Community or neighborhood

Food environment To identify and map convenience stores and fast-food restaurants within a 750 m road network buffer for each schoo

Goodman, 2016122 Community or neighborhood

Other PA behavior - Bikeability

Hoelscher, 2016140 School Commuting behavior 2 day self reported walking or biking to or from school

Just, 2014157 School Purchasing behavior items purchased at school cafeteria

Page 323: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-145

Author, year Setting Co-Outcome Co-Outcome Description

LaRowe, 2016170 Early childhood education or daycare

PA environment Day long observation using the Physical Activity Environment and Policy Assessment Observation (PA-EPAO) scores

Morton, 2016198 School PA environment Changes in school environment for PA from primary to secondary school

Page 324: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-146

Author, year Setting Co-Outcome Co-Outcome Description

Nanney, 2016203 School Food environment Greater school-reported soda availability was associated with an increase in intake of 0.3 daily servings of soda among ninth grade students relative to no soda availability. This association was more pronounced among ninth grade boys than among ninth grade girls, but the difference in the effect between girls and boys was not statistically significant (p=0.23).

Olsho, 2015212 Community or neighborhood

Purchasing behavior Purchase of fruit and vegetables at farmers market

Robles, 2017234 Community or neighborhood

Commuting behavior Transportation behaviors were measured by asking survey questions (un-validated questionnaire)

Rushakoff, 2017237 Community or neighborhood

Purchasing behavior Residents were surveyed about their purchasing and eating patterns over a one-month period

Schwartz, 2016243 School Purchasing behavior Milk purchases

Sturm, 2015255 Community or neighborhood

PA environment Using LA Department of Public Health data

Taber, 2012260 School Purchasing behavior Questionnaire

Vadiveloo, 2011270 Fast food chains Purchasing behavior Frequency of fast food dinner purchases

Wells, 2005280 School Purchasing behavior Parents of 4-6 year old who recieved free school fruit were asked if they had changed the amout of fruit they gave their children at home

Woodward-Lopez, 2010289 School Purchasing behavior Food and Beverage Sales

Wrigley, 2003292 Community or neighborhood

Purchasing behavior Switch to the new store in the neighborhood as their main food purchasing source

Experimental Studies

Audrey, 201510 Employer or worksite Commuting behavior Average daily commute mode of transport and time

Busch, 201544 School Food environment NR

Page 325: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-147

Author, year Setting Co-Outcome Co-Outcome Description

Coleman, 201260 School Food environment Observed changes in food/beverages on campus

French, 2010104 Employer or worksite PA environment Questionnaire

Gittelsohn, 2010117 Community or neighborhood

Purchasing behavior purchasing of healthy and unhealthy food score

Gittelsohn, 2013116 Community or neighborhood

Purchasing behavior Shelf label–driven healthy food purchasing assessed change in frequency of purchasing a food due to an NHS label ( Labelling was part of intervention)

Haerens, 2007126 School Other In girls only, fat intake and percentage of energy from fat decreased significantly more in the intervention group with parental support

Lent, 2014173 School Purchasing behavior total energy purchases, purchase of fat, sodium, carbohydrate, sugar, protein and fiber

Page 326: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-148

Author, year Setting Co-Outcome Co-Outcome Description

Mead, 2013193 Community or neighborhood

Purchasing behavior Healthy food acquisition

Sharma, 2016245 Food assistance program , School

Purchasing behavior Frequency of cooking from scratch at home, eating out, using nutrition facts labels, serving F&V at mealtimes and snacks, serving sugary cereals and sugary drinks at meals, limiting portion sizes, screen time, fried foods, fast food and/or sugary beverages, as well as eating family dinners, and requiring that children finish the food on their plate were assessed

Waters, 2017276 School PA environment Physical activity facilities/policies

Waters, 2017276 School Food environment Canteen facilities/policies

Other Study Designs

Cheadle, 201252 After school or summer school, Community or neighborhood , Employer or worksite, School

PA environment NR

F&V=fruits and vegetables; LA=Los Angeles; METs=metabolic equivalent of task; No.=Number; PA=physical activity

Page 327: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-149

Evidence Table H16. List of distinct secondary data sources by study method Outcomes Data source Population

(Adult, Child, Both)

Number of studies

Weight Diet Physical Activity

Natural Experiment Food environment atlas Adult 1 x 2006 Canadian Census Child 1 x Americans for Nonsmokers’ Rights Adult 1 x x Anchorage School District’s student health data Child 1 x Australian Schools Health and Fitness Survey (ASHFS) Adult 1 x x

Beer taxes Adult 1 x x Behavioral Risk Factor Surveillance System (BRFSS) Adult 4 x x Boston Youth Survey Child 1 x Bridging the Gap Community Obesity Measures Project (BTG_COMP) Child 3 x x x Bureau of Labor Statistic (County-level Unemployment Rates) Adult 1 x x CACI Ltd. Adult 1 x

California FITNESSGRAM Physical Fitness Test (PFT) Child 1 x x

California Health Interview Survey Both 2 x x x California Healthy Kids Survey (CHKS) Child 1 x x

Callorie Labelling Laws Adult 1 x x CDE DataQuest Child 1 x Census Tiger Files Adult 1 x x Child and Diet Evaluation Tool Child 1 x Childhood Obesity Study in China Megacities (COCM) Child 1 x x Cigarette taxes Adult 1 x x Classification of Laws Associated with School Students Child 1 x Cohort for obesity, marijuana use, physical activity, alcohol use, smoking and sedentary behavior (COMPASS)

Child 1 x

Common Core of Data (CCD) Child 1 x Community Health Information Profile (CHIP) Child 1 x x Defense Manpower Data Center Child 1 x x Dunedin City Council Child 1 x Early Childhood Longitudinal Study—Kindergarten Cohort (ECLS-K) Child 1 x x ELSA (wave 6) Adult 1 x x English Longitudinal Study of Ageing (ELSA) Adult 1 x

Page 328: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-150

Outcomes Data source Population

(Adult, Child, Both)

Number of studies

Weight Diet Physical Activity

Family Activity Study (FAS) Both 1 x

FitnessGram Child 2 x

Page 329: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-151

Outcomes Data source Population

(Adult, Child, Both)

Number of studies

Weight Diet Physical Activity

Health and Health Care Utilization Adult 1 x Health Behavior and Health Adult 1 x Health Survey for England Adult 1 x Healthy Eating, Active Communities study (HEAC) Child 1 x

Household food security survey module Adult 1 x Index of Multiple Deprivation (IMD) Child 1 x Jamie's Ministry of Food Adult 1 x LA County Food Retail Adult 1 x x Liveable Neighbourhoods Guidelines’ Adult 1 x Los Angeles County Health and Nutrition Examination Survey (LAHANES-II)

Adult 1 x

Manitoba (Youth Health Survey) Child 1 x x Military Teenagers Environment Exercise and Nutrition Study (M-TEENS)

Adult 1 x x

Minnesota Student Survey (MSS) Child 1 x x Monitoring the Future Child 1 x Montreal Epidemiological and Geographical Analysis of Population Health Outcomes and Neighborhood Effects (MEGAPHONE)

Child 1 x x

Multipurpose Family Survey Adult 1 x National Diet and Nutrition Survey data Child 1 x National Health and Nutrition Examination Survey (NHANES) Adult 1 x National Household Travel Survey Adult 1 x

National School Lunch Program (NSLP) Child 1 x x National Survey of Children's Health Child 1 x x x National Youth Physical Activity and Nutrition Study (NYPANS) Child 1 x New Zealand Ministry of Education Child 1 x NHS Health Scotland Both 1 x The Northeast Iowa Food and Fitness Initiative Child 1 x NYC School Food Database Child 1 x Ongoing Survey of Living Conditions (DLO) Adult 1 x Output area classification Child 1 x Pediatric Nutrition Surveillance System (PedNSS) Child 1 x Permanent Survey of Living Conditions (POLS) Adult 1 x Philadelphia Bureau of Revision of Taxes Adult 1 x

Page 330: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-152

Outcomes Data source Population

(Adult, Child, Both)

Number of studies

Weight Diet Physical Activity

Philadelphia Department of Licenses and Inspections Adult 1 x Philadelphia Police Department Adult 1 x Physical Education-Related State Policy Classification System (PERSPCS)

Child 1

x

x

Quebec Adipose and Lifestyle Investigation in Youth (Quality) Study Child 1 x x School Breakfast Program (SBP) Child 1 x x School Health Policies and Programs Study (SHPPS) Child 1 x x School Health Policy Database - State Child 1 x School Nutrition Dietary Assessment Study-III (SNDA-III) Child 2 x x School Nutrition-Environment State Policy Classification System Child 1 x School Wellness Study (SWS) Child 1 x Shape of the Nation Reports Child 1 x SNAP Data System of the Economic Research Service Child 1 x SNAP Supplemental Both 2 x x Socio-Economic Index for Areas (SEIFA) Child 1 x x x Southeastern Pennsylvania Household Health Survey Adult 1 x Texas Childhood Obesity Prevention Policy Evaluation Child 1 x Texas Fitness Now Child 1 x The High School Study (HSS) Child 1 x The New York City Community Health Survey (CHS) Adult 1 x The Pediatric Nutrition Surveillance System Child 1 x The School Health Profiles (Profiles) Child 1 x x Trip Identification and Analysis System (TIAS) Adult

1 x

UK index of multiple deprivation, 2004 Child 1 x

U.S. GDC Park Landmarks Adult 1 x x

US Census Adult 2 x x

US Postal Service records Adult 1

x

U.S. Department of Agriculture (USDA) Economic Research Service’s (ERS) Food Environment Atlas

Child 1 x

Utah Population Database (UPDB) Child 1 x Washington State Healthy Youth Survey (HYS) Child 1 x

Page 331: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-153

Outcomes Data source Population

(Adult, Child, Both)

Number of studies

Weight Diet Physical Activity

Westlaw Child 1 x WIC Data (LA-County) Child 1 x

Page 332: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-154

Outcomes Data source Population

(Adult, Child, Both)

Number of studies

Weight Diet Physical Activity

Youth Risk Behavior Survey Surveillance - National Child 3 x x x ZipCom commercial database Child 1 x x Experimental studies Moving to Opportunity Adult 1 x National Foundation for Educational Research Child 1 x Nutrition Data System Child 1 x Project CHOICE (Center for Healthy Options and Community Empowerment) Child

1 x

School Action Inventory Child 1 x Other study designs Child Health and Intervention Research Project (CHIRP) Child 1 x x Documentation of Community Change (DOCC) Child 1 x x Maternal and Child Health (MCH) Key Age and Stage (KA&S) Child 1 x x x Romp & Chomp Child 1 x x x

GDC=Geographic Data Center; LA=Los Angeles; NHS=National Health Services; NYC=New York City; SNAP=Supplemental Nutrition Assistance Program; US=United States ; WIC=Women, Infants, and Children

Page 333: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-155

Evidence Table H17. Identified questionnaires by study method Outcome Questionnaire Population Studies Diet Physical Activity

Natural Experiment 7-day recall instrument Adult 1 x Adolescent Behaviours, Attitudes, and Knowledge Questionnaire (ABAKQ) Child 1 x x BC Adolescent Health Survey (AHS) Adult 1 x Behavioral Risk Factor Surveillance System Both 4 x x Beverage and Snack Questionnaire Child 1 x California Health Interview Survey Both 1 x California Healthy Kids Survey Child 2 x x Child Food Consumption Questionnaire Child 1 x COMPASS Student Questionnaire (Cq) Child 1 x Dietary Quality Index (DQI) Adult 1 x Eating and Physical Activity Questionnaire (EPAQ) Adult 2 x Eating Behaviour Study questionnaire Child 1 x English Longitudinal Study of Ageing (ELSA) Adult 1 x Food tick list adapted from FAST Child 1 x Harvard Youth Adolescent Food Frequency Questionnaire (YAQ) Child 2 x Health Survey for England Adult 1 x International Physical Activity Questionnaire (IPAQ) Adult 5 x Mean minutes of transportation-related walking. Adult 1 x Minnesota Student Survey (MSS) Child 1 x National Diet and Nutrition Survey (NDNS) Adult 1 x National Health and Nutrition Examination Survey Adult 1 x National Institute of Health’s Eating at America’s Table Quick Food Scan Adult 1 x National Survey of Children’s Health (NSCH) survey Child 1 x Neighborhood Physical Activity Questionnaire (NPAQ) Adult 1 x NHANES Child 1 x Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC) Child 1 x Past Week Modifiable Activity Questionnaire (PWMAQ) Adult 1 x Phone interviews (unspecified) Adult 1 x Physical Activity and Nutrition (PAN) Monitoring Tool Child 1 x x Random-digit-dial telephone surveys (unspecified) Adult 1 x REACH Risk Factor Survey Adult 1 x Recent Physical Activity Questionnaire (RPAQ) Adult 2 x School Physical Activity and Nutrition (SPAN) questionnaire Child 5 x x School-Based Nutrition Monitoring Questionnaire Child 1 x

Page 334: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-156

Outcome Questionnaire Population Studies Diet Physical Activity

Self-administered questionnaire (unspecified) Child 1 x Southeastern Pennsylvania Household Health Survey Adult 1 x SPAN Questionnaire Child 1 x Staff administered questionnaire Child 1 x Street-intercept survey Adult 1 x Student Health Assessment Questionnaire/ 24-hour recall Child 1 x Surveys by trained interviewers at farmers market Adult 1 x The 148-item self-administered survey Adult 1 x The Boston Youth Survey Child 1 x The California Healthy Kids Survey (CHKS) Child 1 x the School Physical Activity and Nutrition (SPAN) survey Child 1 x x Use of risk behavior survey-daily intake(unspecified) Child 1 x x Victorian Population Health Survey Both 1 x Victorian state government Local Level Child Health and Wellbeing Survey Child 1 x Written questionnaire, students reported the number of days they engaged in at least 20 mins of activity that made them sweat or breathe hard in the past week Child

1 x

Youth Risk Behavior Survey (YRBS) Both 3 x x Experimental studies Adapted version of the International Physical Activity Questionnaire Child 1 x 3-Day Physical Activity Recall (3-DPAR) survey Child 3 x Block Kids Food Screener Child 1 x Child and Adolescent Trial for Cardiovascular Health (CATCH) School Physical Activity and Nutrition (SPAN) Student Questionnaire. Child

1

x

Child and Adolescent Trial for Cardiovascular Health After-School Student Questionnaire (ASSQ) Child 1 x Child Food Consumption Questionnaire Child 1 x Customer Impact Questionnaire (CIQ) Adult 1 x Eurofit physical fitness test battery Child 1 x General Practice Physical Activity Questionnaire Adult 1 x Girls Health Enrichment Multi-site Study (GEMS) Activity Questionnaire (GAQ) Child 1 x Godin leisure time physical activity questionnaire Adult 12 x In-person survey completed by parents Child 1 x International Physical Activity Questionnaire (IPAQ) Both 13 x Interviewer-administered household surveys, SOPLAY (System for Observing Play and Leisure Activity in Youth) methods Adult 1 x Italian National Institute of Health Child 1 x

Page 335: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-157

Outcome Questionnaire Population Studies Diet Physical Activity

Krece Plus test Child 1 x National Cancer Institute Multifactor Screener Adult 1 x Previous Day Physical Activity Recall (PDPAR) Child 1 x Pro-Children Questionnaire Child 1 x School Physical Activity and Nutrition Questionnaire (SPAN) Child 2 x Self-Administered Physical Activity Checklist (SAPAC) Child 1 x SOPLAY protocol Child 1 x Test of Gross Movement Development (TMGD-2) checklist, Child 1 x Youth/Adolescent Activity Questionnaire Child 2 x x Other study designs Child nutrition questionnaire Child 1 x Eating and Physical Activity Questionnaire (EPAQ) Child 1 x International Physical Activity Questionnaire (IPAQ) Adult 1 x Physical Activity Questionnaire for Children (PAQ-c) or Adolescents (PAQ-a) Child 1 x Romp & Chomp Audit survey Child 1 x WIC child questionnaire Child 1 x Youth survey Child 1 x

Page 336: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-158

References for Appendix H

1. Modified z-scores in the CDC growth charts. Centers for Disease Control and Prevention. https://www.cdc.gov/nccdphp/dnpa/growthcharts/resources/BIV-cutoffs.pdf. Accessed on October 31, 2017.

2. About Adult BMI: How is BMI calculated? Centers for Disease Control and Prevention; 2017. https://www.cdc.gov/healthyweight/assessing/bmi/adult_bmi/index.html. Accessed on October 31, 2017.

3. Alaimo K, Oleksyk SC, Drzal NB, et al. Effects of changes in lunch-time competitive foods, nutrition practices, and nutrition policies on low-income middle-school children's diets. Child Obes. 2013 Dec;9(6):509-23. doi: 10.1089/chi.2013.0052. PMID: 24215386.

4. Anderson JV, Bybee DI, Brown RM, et al. 5 a day fruit and vegetable intervention improves consumption in a low income population. J Am Diet Assoc. 2001 Feb;101(2):195-202. doi: 10.1016/s0002-8223(01)00052-9. PMID: 11271692.

5. Anderson LM, Aycock KE, Mihalic CA, et al. Geographic differences in physical education and adolescent BMI: have legal mandates made a difference? J Sch Nurs. 2013 Feb;29(1):52-60. doi: 10.1177/1059840512453602. PMID: 22815346.

6. Angelopoulos PD, Milionis HJ, Grammatikaki E, et al. Changes in BMI and blood pressure after a school based intervention: the CHILDREN study. Eur J Public Health. 2009 Jun;19(3):319-25. doi: 10.1093/eurpub/ckp004. PMID: 19208697.

7. Anthamatten P, Brink L, Lampe S, et al. An assessment of schoolyard renovation strategies to encourage children's physical activity. Int J Behav Nutr Phys Act. 2011 Apr 09;8:27. doi: 10.1186/1479-5868-8-27. PMID: 21477325.

8. Ashfield-Watt PA, Welch AA, Godward S, et al. Effect of a pilot community intervention on fruit and vegetable intakes: use of FACET (Five-a-day Community Evaluation Tool). Public Health Nutr. 2007 Jul;10(7):671-80. doi: 10.1017/s1368980007382517. PMID: 17381948.

9. Ask AS, Hernes S, Aarek I, et al. Serving of free school lunch to secondary-school pupils - a pilot study with health implications. Public Health Nutr. 2010 Feb;13(2):238-44. doi: 10.1017/s1368980009990772. PMID: 19650962.

10. Audrey S, Procter S, Cooper A, et al. Employer schemes to encourage walking to work: feasibility study incorporating an exploratory randomised controlled trial. Public Health Res. 2015 Mar;3(4)doi: 10.3310/phr03040. PMID: 25763450.

11. Ayala GX, Baquero B, Laraia BA, et al. Efficacy of a store-based environmental change intervention compared with a delayed treatment control condition on store customers' intake of fruits and vegetables. Public Health Nutr. 2013 Nov;16(11):1953-60. doi: 10.1017/s1368980013000955. PMID: 23561842.

12. Azevedo LB, Burges Watson D, Haighton C, et al. The effect of dance mat exergaming systems on physical activity and health-related outcomes in secondary schools: results from a natural experiment. BMC Public Health. 2014 Sep 12;14:951. doi: 10.1186/1471-2458-14-951. PMID: 25217144.

13. Backman D, Gonzaga G, Sugerman S, et al. Effect of fresh fruit availability at worksites on the fruit and vegetable consumption of low-wage employees. J Nutr Educ Behav. 2011 Jul-Aug;43(4 Suppl 2):S113-21. doi: 10.1016/j.jneb.2011.04.003. PMID: 21683280.

14. Baker EA, Barnidge EK, Schootman M, et al. Adaptation of a Modified DASH Diet to a Rural African American Community Setting. Am J Prev Med. 2016 Sep 12doi: 10.1016/j.amepre.2016.07.014. PMID: 27633485.

15. Barnidge EK, Hipp PR, Estlund A, et al. Association between community garden participation and fruit and vegetable consumption in rural Missouri. Int J Behav Nutr Phys Act. 2013 Nov 19;10:128. doi: 10.1186/1479-5868-10-128. PMID: 24252563.

16. Barroso CS, Kelder SH, Springer AE, et al. Senate Bill 42: implementation and impact on physical activity in middle schools. J Adolesc Health. 2009 Sep;45(3 Suppl):S82-90. doi: 10.1016/j.jadohealth.2009.06.017. PMID: 19699442.

17. Bastian KA, Maximova K, McGavock J, et al. Does School-Based Health Promotion Affect Physical Activity on Weekends? And, Does It Reach Those Students Most in Need of Health Promotion? PLoS One. 2015;10(10):e0137987. doi: 10.1371/journal.pone.0137987. PMID: 26488168.

Page 337: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-159

18. Bauhoff S. The effect of school district nutrition policies on dietary intake and overweight: a synthetic control approach. Econ Hum Biol. 2014 Jan;12:45-55. doi: 10.1016/j.ehb.2013.06.001. PMID: 23891422.

19. Bauman A, McLean G, Hurdle D, et al. Evaluation of the national 'Push Play' campaign in New Zealand--creating population awareness of physical activity. N Z Med J. 2003 Aug 08;116(1179):U535. PMID: 14513082.

20. Benjamin Neelon SE, Namenek Brouwer RJ, Ostbye T, et al. A community-based intervention increases physical activity and reduces obesity in school-age children in North Carolina. Child Obes. 2015 Jun;11(3):297-303. doi: 10.1089/chi.2014.0130. PMID: 25938983.

21. Bere E, Hilsen M, Klepp KI. Effect of the nationwide free school fruit scheme in Norway. Br J Nutr. 2010 Aug;104(4):589-94. doi: 10.1017/s0007114510000814. PMID: 20350345.

22. Bere E, Veierod MB, Bjelland M, et al. Free school fruit--sustained effect 1 year later. Health Educ Res. 2006 Apr;21(2):268-75. doi: 10.1093/her/cyh063. PMID: 16219630.

23. Bere E, Veierod MB, Bjelland M, et al. Outcome and process evaluation of a Norwegian school-randomized fruit and vegetable intervention: Fruits and Vegetables Make the Marks (FVMM). Health Educ Res. 2006 Apr;21(2):258-67. doi: 10.1093/her/cyh062. PMID: 16219631.

24. Bere E, Veierod MB, Klepp KI. The Norwegian School Fruit Programme: evaluating paid vs. no-cost subscriptions. Prev Med. 2005 Aug;41(2):463-70. doi: 10.1016/j.ypmed.2004.11.024. PMID: 15917042.

25. Bere E, Veierod MB, Skare O, et al. Free School Fruit--sustained effect three years later. Int J Behav Nutr Phys Act. 2007 Feb 19;4:5. doi: 10.1186/1479-5868-4-5. PMID: 17309800.

26. Beresford SA, Thompson B, Bishop S, et al. Long-term fruit and vegetable change in worksites: Seattle 5 a Day follow-up. Am J Health Behav. 2010 Nov-Dec;34(6):707-20. PMID: 20604696.

27. Berger-Jenkins E, Rausch J, Okah E, et al. Evaluation of a Coordinated School-Based Obesity Prevention Program in a Hispanic Community: Choosing Healthy and Active Lifestyles for Kids/Healthy Schools Healthy Families. American Journal of Health Education. 2014;45(5):261-70. doi: 10.1080/19325037.2014.932724. PMID: 103883129. Language: English. Entry Date: 20140909. Revision Date: 20150820. Publication Type: Journal Article.

28. Blake H, Zhou D, Batt ME. Five-year workplace wellness intervention in the NHS. Perspect Public Health. 2013 Sep;133(5):262-71. doi: 10.1177/1757913913489611. PMID: 23771680.

29. Blum JE, Davee AM, Beaudoin CM, et al. Reduced availability of sugar-sweetened beverages and diet soda has a limited impact on beverage consumption patterns in Maine high school youth. J Nutr Educ Behav. 2008 Nov-Dec;40(6):341-7. doi: 10.1016/j.jneb.2007.12.004. PMID: 18984489.

30. Bolton KA, Kremer P, Gibbs L, et al. The outcomes of health-promoting communities: being active eating well initiative-a community-based obesity prevention intervention in Victoria, Australia. Int J Obes (Lond). 2017 Apr 25doi: 10.1038/ijo.2017.73. PMID: 28321132.

31. Bonsergent E, Agrinier N, Thilly N, et al. Overweight and obesity prevention for adolescents: a cluster randomized controlled trial in a school setting. Am J Prev Med. 2013 Jan;44(1):30-9. doi: 10.1016/j.amepre.2012.09.055. PMID: 23253647.

32. Bonvin A, Barral J, Kakebeeke TH, et al. Effect of a governmentally-led physical activity program on motor skills in young children attending child care centers: a cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2013 Jul 08;10:90. doi: 10.1186/1479-5868-10-90. PMID: 23835207.

33. Bowling AB, Moretti M, Ringelheim K, et al. Healthy Foods, Healthy Families: combining incentives and exposure interventions at urban farmers' markets to improve nutrition among recipients of US federal food assistance. Health Promot Perspect. 2016;6(1):10-6. doi: 10.15171/hpp.2016.02. PMID: 27123431.

34. Branas CC, Cheney RA, MacDonald JM, et al. A difference-in-differences analysis of health, safety, and greening vacant urban space. Am J Epidemiol. 2011 Dec 01;174(11):1296-306. doi: 10.1093/aje/kwr273. PMID: 22079788.

35. Brown AL, Khattak AJ, Rodriguez DA. Neighbourhood Types, Travel and Body Mass: A Study of New Urbanist and Suburban Neighbourhoods in the US. Urban Stud. 2008;45(4):963-88.

36. Brown BB, Smith KR, Tharp D, et al. A Complete Street Intervention for Walking to Transit, Nontransit Walking, and Bicycling: A Quasi-Experimental Demonstration of Increased Use. J Phys Act Health. 2016 Nov;13(11):1210-9. doi: 10.1123/jpah.2016-0066. PMID: 27334024.

37. Brown BB, Werner CM, Smith KR, et al. Environmental, behavioral, and psychological predictors of transit ridership: Evidence from a community intervention. J Environ Psychol. 2016 Jun;46:188-96. doi: 10.1016/j.jenvp.2016.04.010. PMID: 27672237.

Page 338: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-160

38. Brown BB, Werner CM, Tribby CP, et al. Transit Use, Physical Activity, and Body Mass Index Changes: Objective Measures Associated With Complete Street Light-Rail Construction. Am J Public Health. 2015 Jul;105(7):1468-74. doi: 10.2105/ajph.2015.302561. PMID: 25973829.

39. Brownson RC, Baker EA, Boyd RL, et al. A community-based approach to promoting walking in rural areas. Am J Prev Med. 2004 Jul;27(1):28-34. doi: 10.1016/j.amepre.2004.03.015. PMID: 15212772.

40. Brownson RC, Hagood L, Lovegreen SL, et al. A multilevel ecological approach to promoting walking in rural communities. Prev Med. 2005 Nov-Dec;41(5-6):837-42. doi: 10.1016/j.ypmed.2005.09.004. PMID: 16256183.

41. Brusseau TA, Hannon J, Burns R. The Effect of a Comprehensive School Physical Activity Program on Physical Activity and Health-Related Fitness in Children From Low-Income Families. J Phys Act Health. 2016 Aug;13(8):888-94. doi: 10.1123/jpah.2016-0028. PMID: 27144329.

42. Burke RM, Meyer A, Kay C, et al. A holistic school-based intervention for improving health-related knowledge, body composition, and fitness in elementary school students: an evaluation of the HealthMPowers program. Int J Behav Nutr Phys Act. 2014 Jun 26;11:78. doi: 10.1186/1479-5868-11-78. PMID: 24969618.

43. Buscail C, Menai M, Salanave B, et al. Promoting physical activity in a low-income neighborhood of the Paris suburb of Saint-Denis: effects of a community-based intervention to increase physical activity. BMC Public Health. 2016 Jul 29;16:667. doi: 10.1186/s12889-016-3360-y. PMID: 27473296.

44. Busch V, De Leeuw JR, Zuithoff NP, et al. A Controlled Health Promoting School Study in the Netherlands: Effects After 1 and 2 Years of Intervention. Health Promot Pract. 2015 Jul;16(4):592-600. doi: 10.1177/1524839914566272. PMID: 25566994.

45. Caballero B, Clay T, Davis SM, et al. Pathways: a school-based, randomized controlled trial for the prevention of obesity in American Indian schoolchildren. Am J Clin Nutr. 2003 Nov;78(5):1030-8. PMID: 14594792.

46. Caldwell EM, Miller Kobayashi M, DuBow WM, et al. Perceived access to fruits and vegetables associated with increased consumption. Public Health Nutr. 2009 Oct;12(10):1743-50. doi: 10.1017/s1368980008004308. PMID: 19105861.

47. Calise TV, Heeren T, DeJong W, et al. Do neighborhoods make people active, or do people make active neighborhoods? Evidence from a planned community in Austin, Texas. Prev Chronic Dis. 2013 Jun 20;10:E102. doi: 10.5888/pcd10.120119. PMID: 23786909.

48. Camacho-Rivera M, Rosenbaum E, Yama C, et al. Low-Income Housing Rental Assistance, Perceptions of Neighborhood Food Environment, and Dietary Patterns among Latino Adults: the AHOME Study. J Racial Ethn Health Disparities. 2017 Jun;4(3):346-53. doi: 10.1007/s40615-016-0234-z. PMID: 27129854.

49. Capogrossi K, You W. The Influence of School Nutrition Programs on the Weight of Low-Income Children: A Treatment Effect Analysis. Health Econ. 2016 Jul 6doi: 10.1002/hec.3378. PMID: 27381591.

50. Cawley J, Meyerhoefer C, Newhouse D. The Correlation of Youth Physical Activity with State Policies. Contemporary Economic Policy. 2007;25(4):506-17. doi: http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291465-7287. PMID: 0952344 Alternate Accession Number: EP27555031.

51. Cawley J, Meyerhoefer C, Newhouse D. The impact of state physical education requirements on youth physical activity and overweight. Health Econ. 2007;16(12):1287-301. doi: 10.1002/hec.1218. PMID: 2007-19974-001.

52. Cheadle A, Rauzon S, Spring R, et al. Kaiser Permanente's Community Health Initiative in Northern California: evaluation findings and lessons learned. Am J Health Promot. 2012 Nov-Dec;27(2):e59-68. doi: 10.4278/ajhp.111222-QUAN-462. PMID: 23113787.

53. Chen C, Chou S-Y, Thornton RJ. The Effect of Household Technology on Weight and Health Outcomes among Chinese Adults: Evidence from China's 'Home Appliances Going to the Countryside' Policy. Journal of Human Capital. 2015 Fall;9(3):364-401. doi: http://www.jstor.org/action/showPublication?journalCode=jhumancapital. PMID: 1541217.

54. Chomitz VR, McGowan RJ, Wendel JM, et al. Healthy Living Cambridge Kids: a community-based participatory effort to promote healthy weight and fitness. Obesity (Silver Spring). 2010 Feb;18 Suppl 1:S45-53. doi: 10.1038/oby.2009.431. PMID: 20107461.

55. Cleland V, Dwyer T, Blizzard L, et al. The provision of compulsory school physical activity: associations with physical activity, fitness and overweight in childhood and twenty years later. Int J Behav Nutr Phys Act. 2008 Feb 29;5:14. doi: 10.1186/1479-5868-5-14. PMID: 18312621.

Page 339: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-161

56. Cochrane T, Davey R, Iqbal Z, et al. NHS health checks through general practice: randomised trial of population cardiovascular risk reduction. BMC Public Health. 2012 Nov 01;12:944. doi: 10.1186/1471-2458-12-944. PMID: 23116213.

57. Coffield JE, Metos JM, Utz RL, et al. A multivariate analysis of federally mandated school wellness policies on adolescent obesity. J Adolesc Health. 2011 Oct;49(4):363-70. doi: 10.1016/j.jadohealth.2011.01.010. PMID: 21939866.

58. Cohen DA, Marsh T, Williamson S, et al. Impact and cost-effectiveness of family Fitness Zones: a natural experiment in urban public parks. Health Place. 2012 Jan;18(1):39-45. doi: 10.1016/j.healthplace.2011.09.008. PMID: 22243905.

59. Cohen JF, Kraak VI, Choumenkovitch SF, et al. The CHANGE study: a healthy-lifestyles intervention to improve rural children's diet quality. J Acad Nutr Diet. 2014 Jan;114(1):48-53. doi: 10.1016/j.jand.2013.08.014. PMID: 24126295.

60. Coleman KJ, Shordon M, Caparosa SL, et al. The healthy options for nutrition environments in schools (Healthy ONES) group randomized trial: using implementation models to change nutrition policy and environments in low income schools. Int J Behav Nutr Phys Act. 2012 Jun 27;9:80. doi: 10.1186/1479-5868-9-80. PMID: 22734945.

61. Cortinez-O'Ryan A, Albagli A, Sadarangani KP, et al. Reclaiming streets for outdoor play: A process and impact evaluation of "Juega en tu Barrio" (Play in your Neighborhood), an intervention to increase physical activity and opportunities for play. PLoS One. 2017;12(7):e0180172. doi: 10.1371/journal.pone.0180172. PMID: 28671984.

62. Coyle KK, Potter S, Schneider D, et al. Distributing free fresh fruit and vegetables at school: results of a pilot outcome evaluation. Public Health Rep. 2009 Sep-Oct;124(5):660-9. doi: 10.1177/003335490912400508. PMID: 19753944.

63. Cradock AL, Barrett JL, Carter J, et al. Impact of the Boston Active School Day policy to promote physical activity among children. Am J Health Promot. 2014 Jan-Feb;28(3 Suppl):S54-64. doi: 10.4278/ajhp.130430-QUAN-204. PMID: 24380467.

64. Cradock AL, McHugh A, Mont-Ferguson H, et al. Effect of school district policy change on consumption of sugar-sweetened beverages among high school students, Boston, Massachusetts, 2004-2006. Prev Chronic Dis. 2011 Jul;8(4):A74. PMID: 21672398.

65. Crespo NC, Elder JP, Ayala GX, et al. Results of a multi-level intervention to prevent and control childhood obesity among Latino children: the Aventuras Para Ninos Study. Ann Behav Med. 2012 Feb;43(1):84-100. doi: 10.1007/s12160-011-9332-7. PMID: 22215470.

66. Cullen KW, Watson K, Zakeri I. Improvements in middle school student dietary intake after implementation of the Texas Public School Nutrition Policy. Am J Public Health. 2008 Jan;98(1):111-7. doi: 10.2105/ajph.2007.111765. PMID: 18048778.

67. Cullen KW, Watson K, Zakeri I, et al. Exploring changes in middle-school student lunch consumption after local school food service policy modifications. Public Health Nutr. 2006 Sep;9(6):814-20. PMID: 16925889.

68. Cummins S, Findlay A, Petticrew CHM, et al. Reducing inequalities in health and diet: findings from a study on the impact of a food retail development. Environ Plann A 2008;40(2):402-22.

69. Cummins S, Flint E, Matthews SA. New neighborhood grocery store increased awareness of food access but did not alter dietary habits or obesity. Health Aff (Millwood). 2014 Feb;33(2):283-91. doi: 10.1377/hlthaff.2013.0512. PMID: 24493772.

70. Cummins S, Petticrew M, Higgins C, et al. Large scale food retailing as an intervention for diet and health: quasi-experimental evaluation of a natural experiment. J Epidemiol Community Health. 2005 Dec;59(12):1035-40. doi: 10.1136/jech.2004.029843. PMID: 16286490.

71. Datar A, Nicosia N. The effect of state competitive food and beverage regulations on childhood overweight and obesity. Journal of Adolescent Health. 2016doi: 10.1016/j.jadohealth.2016.09.003. PMID: 2016-54655-001.

72. Day ME, Strange KS, McKay HA, et al. Action schools! BC--Healthy Eating: effects of a whole-school model to modifying eating behaviours of elementary school children. Can J Public Health. 2008 Jul-Aug;99(4):328-31. PMID: 18767281.

73. De Cocker KA, De Bourdeaudhuij IM, Brown WJ, et al. Effects of "10,000 steps Ghent": a whole-community intervention. Am J Prev Med. 2007 Dec;33(6):455-63. doi: 10.1016/j.amepre.2007.07.037. PMID: 18022061.

74. De Cocker KA, De Bourdeaudhuij IM, Brown WJ, et al. Four-year follow-up of the community intervention '10,000 steps Ghent'. Health Educ Res. 2011 Apr;26(2):372-80. doi: 10.1093/her/cyr015. PMID: 21393377.

Page 340: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-162

75. De Coen V, De Bourdeaudhuij I, Vereecken C, et al. Effects of a 2-year healthy eating and physical activity intervention for 3-6-year-olds in communities of high and low socio-economic status: the POP (Prevention of Overweight among Pre-school and school children) project. Public Health Nutr. 2012 Sep;15(9):1737-45. doi: 10.1017/s1368980012000687.

76. de Greeff JW, Hartman E, Mullender-Wijnsma MJ, et al. Effect of Physically Active Academic Lessons on Body Mass Index and Physical Fitness in Primary School Children. J Sch Health. 2016 May;86(5):346-52. doi: 10.1111/josh.12384. PMID: 27040472.

77. De Henauw S, Huybrechts I, De Bourdeaudhuij I, et al. Effects of a community-oriented obesity prevention programme on indicators of body fatness in preschool and primary school children. Main results from the IDEFICS study. Obes Rev. 2015 Dec;16 Suppl 2:16-29. doi: 10.1111/obr.12346. PMID: 26707013.

78. de Meij JS, Chinapaw MJ, van Stralen MM, et al. Effectiveness of JUMP-in, a Dutch primary school-based community intervention aimed at the promotion of physical activity. Br J Sports Med. 2011 Oct;45(13):1052-7. doi: 10.1136/bjsm.2010.075531. PMID: 21112875.

79. de Silva-Sanigorski A, Elea D, Bell C, et al. Obesity prevention in the family day care setting: impact of the Romp & Chomp intervention on opportunities for children's physical activity and healthy eating. Child Care Health Dev. 2011 May;37(3):385-93. doi: 10.1111/j.1365-2214.2010.01205.x. PMID: 21276039.

80. de Visser R, Sylvester R, Rogers R, et al. Changes in School Health Program Improve Middle School Students' Behaviors. Am J Health Behav. 2016 Sep;40(5):568-77. doi: 10.5993/ajhb.40.5.3. PMID: 27561859.

81. Dill J, McNeil N, Broach J, et al. Bicycle boulevards and changes in physical activity and active transportation: findings from a natural experiment. Prev Med. 2014 Dec;69 Suppl 1:S74-8. doi: 10.1016/j.ypmed.2014.10.006. PMID: 25456802.

82. Dubowitz T, Ghosh-Dastidar M, Cohen DA, et al. Diet And Perceptions Change With Supermarket Introduction In A Food Desert, But Not Because Of Supermarket Use. Health Aff (Millwood). 2015 Nov;34(11):1858-68. doi: 10.1377/hlthaff.2015.0667. PMID: 26526243.

83. Dunton G, Ebin VJ, Efrat MW, et al. The Use of Refundable Tax Credits to Increase Low-Income Children's After-School Physical Activity Level. J Phys Act Health. 2015 Jun;12(6):840-53. doi: 10.1123/jpah.2014-0058. PMID: 25184738.

84. Dzewaltowski DA, Estabrooks PA, Welk G, et al. Healthy youth places: a randomized controlled trial to determine the effectiveness of facilitating adult and youth leaders to promote physical activity and fruit and vegetable consumption in middle schools. Health Educ Behav. 2009 Jun;36(3):583-600. doi: 10.1177/1090198108314619. PMID: 18469366.

85. Eagle TF, Gurm R, Smith CA, et al. A middle school intervention to improve health behaviors and reduce cardiac risk factors. Am J Med. 2013 Oct;126(10):903-8. doi: 10.1016/j.amjmed.2013.04.019. PMID: 23932159.

86. Economos CD, Hyatt RR, Goldberg JP, et al. A community intervention reduces BMI z-score in children: Shape Up Somerville first year results. Obesity (Silver Spring). 2007 May;15(5):1325-36. doi: 10.1038/oby.2007.155. PMID: 17495210.

87. Elbel B, Mijanovich T, Kiszko K, et al. The Introduction of a Supermarket via Tax-Credits in a Low-Income Area. Am J Health Promot. 2017 Jan;31(1):59-66. doi: 10.4278/ajhp.150217-QUAN-733. PMID: 26389982.

88. Elbel B, Moran A, Dixon LB, et al. Assessment of a government-subsidized supermarket in a high-need area on household food availability and children's dietary intakes. Public Health Nutr. 2015 Oct;18(15):2881-90. doi: 10.1017/s1368980015000282. PMID: 25714993.

89. Elinder LS, Heinemans N, Hagberg J, et al. A participatory and capacity-building approach to healthy eating and physical activity—SCIP-school: A 2-year controlled trial. Int J Behav Nutr Phys Act. 2012;9doi: 10.1186/1479-5868-9-145. PMID: 2013-04234-001.

90. Eriksen K, Haraldsdottir J, Pederson R, et al. Effect of a fruit and vegetable subscription in Danish schools. Public Health Nutr. 2003 Feb;6(1):57-63. doi: 10.1079/phn2002356. PMID: 12581466.

91. Ermetici F, Zelaschi RF, Briganti S, et al. Association between a school-based intervention and adiposity outcomes in adolescents: The Italian "EAT" project. Obesity (Silver Spring). 2016 Mar;24(3):687-95. doi: 10.1002/oby.21365. PMID: 26833570.

92. Esquivel M, Nigg CR, Fialkowski MK, et al. Head Start Wellness Policy Intervention in Hawaii: A Project of the Children's Healthy Living Program. Child Obes. 2016 Feb;12(1):26-32. doi: 10.1089/chi.2015.0071. PMID: 26771119.

93. Evans CE, Ransley JK, Christian MS, et al. A cluster-randomised controlled trial of a school-based fruit and vegetable intervention: Project Tomato. Public Health Nutr. 2013 Jun;16(6):1073-81. doi: 10.1017/s1368980012005290. PMID: 23237386.

Page 341: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-163

94. Fairclough SJ, McGrane B, Sanders G, et al. A non-equivalent group pilot trial of a school-based physical activity and fitness intervention for 10-11 year old english children: born to move. BMC Public Health. 2016 Aug 24;16(1):861. doi: 10.1186/s12889-016-3550-7. PMID: 27553010.

95. Falbe J, Thompson HR, Becker CM, et al. Impact of the Berkeley Excise Tax on Sugar-Sweetened Beverage Consumption. Am J Public Health. 2016 Oct;106(10):1865-71. doi: 10.2105/ajph.2016.303362. PMID: 27552267.

96. Farley TA, Meriwether RA, Baker ET, et al. Safe play spaces to promote physical activity in inner-city children: results from a pilot study of an environmental intervention. Am J Public Health. 2007 Sep;97(9):1625-31. doi: 10.2105/ajph.2006.092692. PMID: 17666701.

97. Farmer VL, Williams SM, Mann JI, et al. The effect of increasing risk and challenge in the school playground on physical activity and weight in children: a cluster randomised controlled trial (PLAY). Int J Obes (Lond). 2017 May;41(5):793-800. doi: 10.1038/ijo.2017.41. PMID: 28186099.

98. Finch M, Wolfenden L, Morgan PJ, et al. A cluster randomized trial of a multi-level intervention, delivered by service staff, to increase physical activity of children attending center-based childcare. Prev Med. 2014 Jan;58:9-16. doi: 10.1016/j.ypmed.2013.10.004. PMID: 24145204.

99. Fitzpatrick C, Datta GD, Henderson M, et al. School food environments associated with adiposity in Canadian children. Int J Obes (Lond). 2017 Mar 14doi: 10.1038/ijo.2017.39. PMID: 28186100.

100. Flego A, Herbert J, Waters E, et al. Jamie's Ministry of Food: quasi-experimental evaluation of immediate and sustained impacts of a cooking skills program in Australia. PLoS One. 2014;9(12):e114673. doi: 10.1371/journal.pone.0114673. PMID: 25514531.

101. Fogarty AW, Antoniak M, Venn AJ, et al. Does participation in a population-based dietary intervention scheme have a lasting impact on fruit intake in young children? Int J Epidemiol. 2007 Oct;36(5):1080-5. doi: 10.1093/ije/dym133. PMID: 17602183.

102. Foster GD, Sherman S, Borradaile KE, et al. A policy-based school intervention to prevent overweight and obesity. Pediatrics. 2008 Apr;121(4):e794-802. doi: 10.1542/peds.2007-1365. PMID: 18381508.

103. Fox MK, Dodd AH, Wilson A, et al. Association between school food environment and practices and body mass index of US public school children. J Am Diet Assoc. 2009 Feb;109(2 Suppl):S108-17. doi: 10.1016/j.jada.2008.10.065. PMID: 19166665.

104. French SA, Harnack LJ, Hannan PJ, et al. Worksite environment intervention to prevent obesity among metropolitan transit workers. Prev Med. 2010 Apr;50(4):180-5. doi: 10.1016/j.ypmed.2010.01.002. PMID: 20079369.

105. Frongillo EA, Fawcett SB, Ritchie LD, et al. Community Policies and Programs to Prevent Obesity and Child Adiposity. Am J Prev Med. 2017 Jul 05doi: 10.1016/j.amepre.2017.05.006. PMID: 28688728.

106. Fu Y, Gao Z, Hannon JC, et al. Effect of the SPARK Program on Physical Activity, Cardiorespiratory Endurance, and Motivation in Middle-School Students. J Phys Act Health. 2016 May;13(5):534-42. doi: 10.1123/jpah.2015-0351. PMID: 26528889.

107. Fuller D, Gauvin L, Kestens Y, et al. Impact evaluation of a public bicycle share program on cycling: a case example of BIXI in Montreal, Quebec. Am J Public Health. 2013 Mar;103(3):e85-92. doi: 10.2105/ajph.2012.300917. PMID: 23327280.

108. Fung C, McIsaac JL, Kuhle S, et al. The impact of a population-level school food and nutrition policy on dietary intake and body weights of Canadian children. Prev Med. 2013 Dec;57(6):934-40. doi: 10.1016/j.ypmed.2013.07.016. PMID: 23891787.

109. Gatto NM, Martinez LC, Spruijt-Metz D, et al. LA sprouts randomized controlled nutrition, cooking and gardening programme reduces obesity and metabolic risk in Hispanic/Latino youth. Pediatr Obes. 2017 Feb;12(1):28-37. doi: 10.1111/ijpo.12102. PMID: 26909882.

110. Geaney F, Harrington J, Perry IJ. P42 The impact of a catering initiative in determining food choices and salt intake in the public sector. Journal of Epidemiology & Community Health. 2010;64:A50-A. PMID: 104133563. Language: English. Entry Date: 20140108. Revision Date: 20150710. Publication Type: Journal Article.

111. Geaney F, Kelly C, Di Marrazzo JS, et al. The effect of complex workplace dietary interventions on employees' dietary intakes, nutrition knowledge and health status: a cluster controlled trial. Prev Med. 2016 Aug;89:76-83. doi: 10.1016/j.ypmed.2016.05.005. PMID: 27208667.

112. Gebel K, Bauman AE, Reger-Nash B, et al. Does the environment moderate the impact of a mass media campaign to promote walking? Am J Health Promot. 2011 Sep-Oct;26(1):45-8. doi: 10.4278/ajhp.081104-ARB-269. PMID: 21879942.

Page 342: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-164

113. Gee KA. School-Based Body Mass Index Screening and Parental Notification in Late Adolescence: Evidence From Arkansas's Act 1220. J Adolesc Health. 2015 Sep;57(3):270-6. doi: 10.1016/j.jadohealth.2015.05.007. PMID: 26115907.

114. Gibson D. Long-term Food Stamp Program participation is positively related to simultaneous overweight in young daughters and obesity in mothers. J Nutr. 2006 Apr;136(4):1081-5. PMID: 16549483.

115. Giles-Corti B, Bull F, Knuiman M, et al. The influence of urban design on neighbourhood walking following residential relocation: longitudinal results from the RESIDE study. Soc Sci Med. 2013 Jan;77:20-30. doi: 10.1016/j.socscimed.2012.10.016. PMID: 23206559.

116. Gittelsohn J, Kim EM, He S, et al. A food store-based environmental intervention is associated with reduced BMI and improved psychosocial factors and food-related behaviors on the Navajo nation. J Nutr. 2013 Sep;143(9):1494-500. doi: 10.3945/jn.112.165266. PMID: 23864511.

117. Gittelsohn J, Vijayadeva V, Davison N, et al. A food store intervention trial improves caregiver psychosocial factors and children's dietary intake in Hawaii. Obesity (Silver Spring). 2010 Feb;18 Suppl 1:S84-90. doi: 10.1038/oby.2009.436. PMID: 20107467.

118. Gleason PM, Dodd AH. School Breakfast Program but Not School Lunch Program Participation Is Associated with Lower Body Mass Index. J Am Diet Assoc. 2009 Feb;109(2):S118-S28. doi: 10.1016/j.jada.2008.10.058. PMID: 19166666.

119. Goetzel RZ, Roemer EC, Pei X, et al. Second-year results of an obesity prevention program at the Dow Chemical Company. J Occup Environ Med. 2010 Mar;52(3):291-302. doi: 10.1097/JOM.0b013e3181d46f0b. PMID: 20190646.

120. Goldsby TU, George BJ, Yeager VA, et al. Urban Park Development and Pediatric Obesity Rates: A Quasi-Experiment Using Electronic Health Record Data. Int J Environ Res Public Health. 2016 Apr 08;13(4):411. doi: 10.3390/ijerph13040411. PMID: 27070635.

121. Goodman A, Sahlqvist S, Ogilvie D. New walking and cycling routes and increased physical activity: one- and 2-year findings from the UK iConnect Study. Am J Public Health. 2014 Sep;104(9):e38-46. doi: 10.2105/ajph.2014.302059. PMID: 25033133.

122. Goodman A, van Sluijs EM, Ogilvie D. Impact of offering cycle training in schools upon cycling behaviour: a natural experimental study. Int J Behav Nutr Phys Act. 2016 Mar 08;13:34. doi: 10.1186/s12966-016-0356-z. PMID: 26956383.

123. Gorely T, Morris JG, Musson H, et al. Physical activity and body composition outcomes of the GreatFun2Run intervention at 20 month follow-up. Int J Behav Nutr Phys Act. 2011 Jul 18;8:74. doi: 10.1186/1479-5868-8-74. PMID: 21767356.

124. Gorham G, Dulin-Keita A, Risica PM, et al. Effectiveness of Fresh to You, a Discount Fresh Fruit and Vegetable Market in Low-Income Neighborhoods, on Children's Fruit and Vegetable Consumption, Rhode Island, 2010-2011. Prev Chronic Dis. 2015 Oct 15;12:E176. doi: 10.5888/pcd12.140583. PMID: 26469949.

125. Gustat J, Rice J, Parker KM, et al. Effect of changes to the neighborhood built environment on physical activity in a low-income African American neighborhood. Prev Chronic Dis. 2012;9:E57. PMID: 22338597.

126. Haerens L, De Bourdeaudhuij I, Maes L, et al. The effects of a middle-school healthy eating intervention on adolescents' fat and fruit intake and soft drinks consumption. Public Health Nutr. 2007 May;10(5):443-9. doi: 10.1017/s1368980007219652. PMID: 17411463.

127. Haerens L, Deforche B, Maes L, et al. Body mass effects of a physical activity and healthy food intervention in middle schools. Obesity (Silver Spring). 2006 May;14(5):847-54. doi: 10.1038/oby.2006.98. PMID: 16855194.

128. Harding MC, Bott QD, Jonas CE. The Malaekahana Path: An Ecological Model-based Intervention for Increasing Walking in Rural Hawai'i. J Phys Act Health. 2017 Jul 06:1-11. doi: 10.1123/jpah.2017-0330. PMID: 28682662.

129. Hardy LL, King L, Kelly B, et al. Munch and Move: evaluation of a preschool healthy eating and movement skill program. Int J Behav Nutr Phys Act. 2010 Nov 03;7:80. doi: 10.1186/1479-5868-7-80. PMID: 21047434.

130. He M, Beynon C, Sangster Bouck M, et al. Impact evaluation of the Northern Fruit and Vegetable Pilot Programme - a cluster-randomised controlled trial. Public Health Nutr. 2009 Nov;12(11):2199-208. doi: 10.1017/s1368980009005801. PMID: 19476675.

131. Heelan KA, Abbey BM, Donnelly JE, et al. Evaluation of a Walking School Bus for Promoting Physical Activity in Youth. J Phys Act Health. 2009 Sep;6(5):560-7. PMID: 19953832.

132. Heelan KA, Bartee RT, Nihiser A, et al. Healthier School Environment Leads to Decreases in Childhood Obesity: The Kearney Nebraska Story. Child Obes. 2015 Oct;11(5):600-7. doi: 10.1089/chi.2015.0005. PMID: 26440386.

Page 343: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-165

133. Hendy HM, Williams KE, Camise TS. Kid's Choice Program improves weight management behaviors and weight status in school children. Appetite. 2011 Apr;56(2):484-94. doi: 10.1016/j.appet.2011.01.024. PMID: 21277924.

134. Hennessy E, Oh A, Agurs-Collins T, et al. State-level school competitive food and beverage laws are associated with children's weight status. J Sch Health. 2014 Sep;84(9):609-16. doi: 10.1111/josh.12181. PMID: 25117896.

135. Herrick H, Thompson H, Kinder J, et al. Use of SPARK to promote after-school physical activity. J Sch Health. 2012 Oct;82(10):457-61. doi: 10.1111/j.1746-1561.2012.00722.x. PMID: 22954164.

136. Hilmers A, Chen TA, Dave JM, et al. Supplemental Nutrition Assistance Program participation did not help low income Hispanic women in Texas meet the dietary guidelines. Prev Med. 2014 May;62:44-8. doi: 10.1016/j.ypmed.2014.01.016. PMID: 24530319.

137. Hobin E, Erickson T, Comte M, et al. Examining the impact of a province-wide physical education policy on secondary students' physical activity as a natural experiment. Int J Behav Nutr Phys Act. 2017 Jul 19;14(1):98. doi: 10.1186/s12966-017-0550-7. PMID: 28724390.

138. Hobin E, So J, Rosella L, et al. Trajectories of objectively measured physical activity among secondary students in Canada in the context of a province-wide physical education policy: a longitudinal analysis. J Obes. 2014;2014:958645. doi: 10.1155/2014/958645. PMID: 24672714.

139. Hoefkens C, Lachat C, Kolsteren P, et al. Posting point-of-purchase nutrition information in university canteens does not influence meal choice and nutrient intake. Am J Clin Nutr. 2011 Aug;94(2):562-70. doi: 10.3945/ajcn.111.013417. PMID: 21677060.

140. Hoelscher D, Ory M, Dowdy D, et al. Effects of funding allocation for Safe Routes to School programs on active commuting to school and related behavioral, knowledge, and psychosocial outcomes: Results from the Texas Childhood Obesity Prevention Policy Evaluation (T-COPPE) study. Environ Behav. 2016;48(1):210-29. doi: 10.1177/0013916515613541. PMID: 2016-02263-012.

141. Hoelscher DM, Moag-Stahlberg A, Ellis K, et al. Evaluation of a student participatory, low-intensity program to improve school wellness environment and students' eating and activity behaviors. Int J Behav Nutr Phys Act. 2016 May 13;13:59. doi: 10.1186/s12966-016-0379-5. PMID: 27178056.

142. Hollis JL, Sutherland R, Campbell L, et al. Effects of a 'school-based' physical activity intervention on adiposity in adolescents from economically disadvantaged communities: secondary outcomes of the 'Physical Activity 4 Everyone' RCT. Int J Obes (Lond). 2016 Oct;40(10):1486-93. doi: 10.1038/ijo.2016.107. PMID: 27430652.

143. Howlett E, Davis C, Burton S. From food desert to food oasis: The potential influence of food retailers on childhood obesity rates. Journal of Business Ethics. 2016;139(2):215-24. doi: 10.1007/s10551-015-2605-5. PMID: 2015-12722-001.

144. Hu Y, van Lenthe FJ, Judge K, et al. Did the English strategy reduce inequalities in health? A difference-in-difference analysis comparing England with three other European countries. BMC Public Health. 2016 Aug 24;16(1):865. doi: 10.1186/s12889-016-3505-z. PMID: 27558269.

145. Huberty J, Beets M, Beighle A. Effects of a policy-level intervention on children's pedometer-determined physical activity: preliminary findings from Movin' Afterschool. J Public Health Manag Pract. 2013 Nov-Dec;19(6):525-8. doi: 10.1097/PHH.0b013e31829465fa. PMID: 23676476.

146. Huberty JL, Beets MW, Beighle A, et al. Environmental modifications to increase physical activity during recess: preliminary findings from ready for recess. J Phys Act Health. 2011 Sep;8 Suppl 2:S249-56. PMID: 21918239.

147. Hughes RJ, Edwards KL, Clarke GP, et al. Childhood consumption of fruit and vegetables across England: a study of 2306 6-7-year-olds in 2007. Br J Nutr. 2012 Aug;108(4):733-42. doi: 10.1017/s0007114511005939. PMID: 22321148.

148. Hunter S, Leatherdale ST, Storey K, et al. A quasi-experimental examination of how school-based physical activity changes impact secondary school student moderate- to vigorous- intensity physical activity over time in the COMPASS study. Int J Behav Nutr Phys Act. 2016 Jul 29;13:86. doi: 10.1186/s12966-016-0411-9. PMID: 27473113.

149. Jago R, McMurray RG, Drews KL, et al. HEALTHY intervention: fitness, physical activity, and metabolic syndrome results. Med Sci Sports Exerc. 2011 Aug;43(8):1513-22. doi: 10.1249/MSS.0b013e31820c9797. PMID: 21233778.

Page 344: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-166

150. Janssen M, Twisk JW, Toussaint HM, et al. Effectiveness of the PLAYgrounds programme on PA levels during recess in 6-year-old to 12-year-old children. Br J Sports Med. 2015 Feb;49(4):259-64. doi: 10.1136/bjsports-2012-091517. PMID: 23293007.

151. Jennings A, Cassidy A, Winters T, et al. Positive effect of a targeted intervention to improve access and availability of fruit and vegetables in an area of deprivation. Health Place. 2012 Sep;18(5):1074-8. doi: 10.1016/j.healthplace.2012.05.001. PMID: 22705164.

152. Jia P, Li M, Xue H, et al. School environment and policies, child eating behavior and overweight/obesity in urban China: the childhood obesity study in China megacities. Int J Obes (Lond). 2017 May;41(5):813-9. doi: 10.1038/ijo.2017.2. PMID: 28074059.

153. Johnson R, Robertson W, Towey M, et al. Changes over time in mental well-being, fruit and vegetable consumption and physical activity in a community-based lifestyle intervention: a before and after study. Public Health. 2017 May;146:118-25. doi: 10.1016/j.puhe.2017.01.012. PMID: 28404463.

154. Jones J, Wyse R, Finch M, et al. Effectiveness of an intervention to facilitate the implementation of healthy eating and physical activity policies and practices in childcare services: a randomised controlled trial. Implement Sci. 2015 Oct 25;10:147. doi: 10.1186/s13012-015-0340-z. PMID: 26498746.

155. Jordan KC, Erickson ED, Cox R, et al. Evaluation of the Gold Medal Schools program. J Am Diet Assoc. 2008 Nov;108(11):1916-20. doi: 10.1016/j.jada.2008.08.002. PMID: 18954584.

156. Jurg ME, Kremers SP, Candel MJ, et al. A controlled trial of a school-based environmental intervention to improve physical activity in Dutch children: JUMP-in, kids in motion. Health Promot Int. 2006 Dec;21(4):320-30. doi: 10.1093/heapro/dal032. PMID: 16963784.

157. Just DR, Wansink B. School lunch debit card payment systems are associated with lower nutrition and higher calories. Obesity (Silver Spring). 2014 Jan;22(1):24-6. doi: 10.1002/oby.20591. PMID: 23929600.

158. Kain J, Uauy R, Albala, et al. School-based obesity prevention in Chilean primary school children: methodology and evaluation of a controlled study. Int J Obes Relat Metab Disord. 2004 Apr;28(4):483-93. doi: 10.1038/sj.ijo.0802611. PMID: 14993915.

159. Kamada M, Kitayuguchi J, Inoue S, et al. A community-wide campaign to promote physical activity in middle-aged and elderly people: a cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2013 Apr 09;10:44. doi: 10.1186/1479-5868-10-44. PMID: 23570536.

160. Kastorini CM, Lykou A, Yannakoulia M, et al. The influence of a school-based intervention programme regarding adherence to a healthy diet in children and adolescents from disadvantaged areas in Greece: the DIATROFI study. J Epidemiol Community Health. 2016 Jul;70(7):671-7. doi: 10.1136/jech-2015-205680. PMID: 26763974.

161. Kern E, Chan NL, Fleming DW, et al. Declines in student obesity prevalence associated with a prevention initiative - King County, Washington, 2012. MMWR Morb Mortal Wkly Rep. 2014 Feb 21;63(7):155-7. PMID: 24553199.

162. Keyte J, Harris S, Margetts B, et al. Engagement with the National Healthy Schools Programme is associated with higher fruit and vegetable consumption in primary school children. J Hum Nutr Diet. 2012 Apr;25(2):155-60. doi: 10.1111/j.1365-277X.2011.01208.x. PMID: 22128770.

163. Kim J. Are physical education-related state policies and schools' physical education requirement related to children's physical activity and obesity? J Sch Health. 2012 Jun;82(6):268-76. doi: 10.1111/j.1746-1561.2012.00697.x. PMID: 22568462.

164. Kim K, Hong SA, Yun SH, et al. The effect of a healthy school tuck shop program on the access of students to healthy foods. Nutr Res Pract. 2012 Apr;6(2):138-45. doi: 10.4162/nrp.2012.6.2.138. PMID: 22586503.

165. King MH, Lederer AM, Sovinski D, et al. Implementation and evaluation of the HEROES initiative: a tri-state coordinated school health program to reduce childhood obesity. Health Promot Pract. 2014 May;15(3):395-405. doi: 10.1177/1524839913512835. PMID: 24334542.

166. Kloek GC, van Lenthe FJ, van Nierop PW, et al. Impact evaluation of a Dutch community intervention to improve health-related behaviour in deprived neighbourhoods. Health Place. 2006 Dec;12(4):665-77. doi: 10.1016/j.healthplace.2005.09.002. PMID: 16253541.

167. Kubik MY, Lytle LA, Story M. Schoolwide food practices are associated with body mass index in middle school students. Arch Pediatr Adolesc Med. 2005 Dec;159(12):1111-4. doi: 10.1001/archpedi.159.12.1111. PMID: 16330732.

168. LaCaille LJ, Schultz JF, Goei R, et al. Go!: results from a quasi-experimental obesity prevention trial with hospital employees. BMC Public Health. 2016 Feb 19;16:171. doi: 10.1186/s12889-016-2828-0. PMID: 26893128.

169. Lachapelle U, Frank LD. Transit and health: mode of transport, employer-sponsored public transit pass programs, and physical activity. J Public Health Policy. 2009;30 Suppl 1:S73-94. doi: 10.1057/jphp.2008.52. PMID: 19190584.

Page 345: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-167

170. LaRowe TL, Tomayko EJ, Meinen AM, et al. Active Early: one-year policy intervention to increase physical activity among early care and education programs in Wisconsin. BMC Public Health. 2016 Jul 20;16:607. doi: 10.1186/s12889-016-3198-3. PMID: 27439770.

171. Lemon SC, Wang ML, Wedick NM, et al. Weight gain prevention in the school worksite setting: results of a multi-level cluster randomized trial. Prev Med. 2014 Mar;60:41-7. doi: 10.1016/j.ypmed.2013.12.010. PMID: 24345602.

172. Lemon SC, Zapka J, Li W, et al. Step ahead a worksite obesity prevention trial among hospital employees. Am J Prev Med. 2010 Jan;38(1):27-38. doi: 10.1016/j.amepre.2009.08.028. PMID: 20117554.

173. Lent MR, Vander Veur SS, McCoy TA, et al. A randomized controlled study of a healthy corner store initiative on the purchases of urban, low-income youth. Obesity (Silver Spring). 2014 Dec;22(12):2494-500. doi: 10.1002/oby.20878. PMID: 25311881.

174. Leung CW, Blumenthal SJ, Hoffnagle EE, et al. Associations of food stamp participation with dietary quality and obesity in children. Pediatrics. 2013 Mar;131(3):463-72. doi: 10.1542/peds.2012-0889. PMID: 23439902.

175. Liao Y, Siegel PZ, Zhou H, et al. Reduced Disparity in Vegetable Consumption in 16 Disadvantaged Black Communities: A Successful 5-Year Community-Based Participatory Intervention. J Racial Ethn Health Disparities. 2015 Jun;2(2):211-8. doi: 10.1007/s40615-014-0065-8. PMID: 26150921.

176. Linde JA, Nygaard KE, MacLehose RF, et al. HealthWorks: results of a multi-component group-randomized worksite environmental intervention trial for weight gain prevention. Int J Behav Nutr Phys Act. 2012 Feb 16;9:14. doi: 10.1186/1479-5868-9-14. PMID: 22340088.

177. Ling J, King KM, Speck BJ, et al. Preliminary assessment of a school-based healthy lifestyle intervention among rural elementary school children. J Sch Health. 2014 Apr;84(4):247-55. doi: 10.1111/josh.12143. PMID: 24617908.

178. Liu J, Kuo T, Jiang L, et al. Food and drink consumption among 1-5-year-old Los Angeles County children from households receiving dual SNAP and WIC v. only WIC benefits. Public Health Nutr. 2016 Sep 9:1-8. doi: 10.1017/s1368980016002329. PMID: 27609603.

179. Llargues E, Franco R, Recasens A, et al. Assessment of a school-based intervention in eating habits and physical activity in school children: the AVall study. J Epidemiol Community Health. 2011 Oct;65(10):896-901. doi: 10.1136/jech.2009.102319.

180. Lorentzen C, Ommundsen Y, Jenum AK, et al. The "Romsas in Motion" community intervention: mediating effects of psychosocial factors on forward transition in the stages of change in physical activity. Health Educ Behav. 2009 Apr;36(2):348-65. doi: 10.1177/1090198107308372. PMID: 18065570.

181. Lubans DR, Smith JJ, Plotnikoff RC, et al. Assessing the sustained impact of a school-based obesity prevention program for adolescent boys: the ATLAS cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2016 Aug 20;13:92. doi: 10.1186/s12966-016-0420-8. PMID: 27542825.

182. Ludwig J, Sanbonmatsu L, Gennetian L, et al. Neighborhoods, obesity, and diabetes--a randomized social experiment. N Engl J Med. 2011 Oct 20;365(16):1509-19. doi: 10.1056/NEJMsa1103216. PMID: 22010917.

183. Lv J, Liu QM, Ren YJ, et al. A community-based multilevel intervention for smoking, physical activity and diet: short-term findings from the Community Interventions for Health programme in Hangzhou, China. J Epidemiol Community Health. 2014 Apr;68(4):333-9. doi: 10.1136/jech-2013-203356. PMID: 24297972.

184. MacDonald JM, Stokes RJ, Cohen DA, et al. The effect of light rail transit on body mass index and physical activity. Am J Prev Med. 2010 Aug;39(2):105-12. doi: 10.1016/j.amepre.2010.03.016. PMID: 20621257.

185. Maddock J, Takeuchi L, Nett B, et al. Evaluation of a statewide program to reduce chronic disease: The Healthy Hawaii Initiative, 2000-2004. Eval Program Plann. 2006;29(3):293-300. doi: 10.1016/j.evalprogplan.2005.12.007. PMID: 2006-11996-009.

186. Madsen K, Linchey J, Gerstein D, et al. Energy Balance 4 Kids with Play: Results from a Two-Year Cluster-Randomized Trial. Child Obes. 2015 Aug;11(4):375-83. doi: 10.1089/chi.2015.0002. PMID: 26061799.

187. Madsen K, Thompson H, Adkins A, et al. School-community partnerships: a cluster-randomized trial of an after-school soccer program. JAMA Pediatr. 2013 Apr;167(4):321-6. doi: 10.1001/jamapediatrics.2013.1071. PMID: 23440308.

188. Madsen KA. School-based body mass index screening and parent notification: a statewide natural experiment. Arch Pediatr Adolesc Med. 2011 Nov;165(11):987-92. doi: 10.1001/archpediatrics.2011.127. PMID: 21727262.

Page 346: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-168

189. Madsen KA, Cotterman C, Crawford P, et al. Effect of the healthy schools program on prevalence of overweight and obesity in California schools, 2006–2012. Preventing Chronic Disease: Public Health Research, Practice, and Policy. 2015;12 PMID: 2016-24674-001.

190. Magarey AM, Pettman TL, Wilson A, et al. Changes in Primary School Children's Behaviour, Knowledge, Attitudes, and Environments Related to Nutrition and Physical Activity. ISRN Obes. 2013;2013:752081. doi: 10.1155/2013/752081. PMID: 24555153.

191. Malakellis M, Hoare E, Sanigorski A, et al. School-based systems change for obesity prevention in adolescents: outcomes of the Australian Capital Territory 'It's Your Move!'. Aust N Z J Public Health. 2017 Jul 27doi: 10.1111/1753-6405.12696. PMID: 28749562.

192. Masse LC, de Niet-Fitzgerald JE, Watts AW, et al. Associations between the school food environment, student consumption and body mass index of Canadian adolescents. Int J Behav Nutr Phys Act. 2014 Mar 26;11(1):29. doi: 10.1186/1479-5868-11-29. PMID: 24666770.

193. Mead EL, Gittelsohn J, Roache C, et al. A community-based, environmental chronic disease prevention intervention to improve healthy eating psychosocial factors and behaviors in indigenous populations in the Canadian arctic. Health Education & Behavior. 2013;40(5):592-602. doi: 10.1177/1090198112467793. PMID: 2013-33808-011.

194. Miewald C, Holben D, Hall P. Role of a food box program in fruit and vegetable consumption and food security. Can J Diet Pract Res. 2012 Summer;73(2):59-65. doi: 10.3148/73.2.2012.59. PMID: 22668838.

195. Miller HJ, Tribby CP, Brown BB, et al. Public transit generates new physical activity: Evidence from individual GPS and accelerometer data before and after light rail construction in a neighborhood of Salt Lake City, Utah, USA. Health Place. 2015 Nov;36:8-17. doi: 10.1016/j.healthplace.2015.08.005.

196. Molitor F, Sugerman S, Yu H, et al. Reach of Supplemental Nutrition Assistance Program-Education (SNAP-Ed) interventions and nutrition and physical activity-related outcomes, California, 2011-2012. Prev Chronic Dis. 2015 Mar 12;12:E33. doi: 10.5888/pcd12.140449. PMID: 25764139.

197. Morrill BA, Madden GJ, Wengreen HJ, et al. A Randomized Controlled Trial of the Food Dudes Program: Tangible Rewards are More Effective Than Social Rewards for Increasing Short- and Long-Term Fruit and Vegetable Consumption. J Acad Nutr Diet. 2016 Apr;116(4):618-29. doi: 10.1016/j.jand.2015.07.001. PMID: 26297598.

198. Morton KL, Corder K, Suhrcke M, et al. School polices, programmes and facilities, and objectively measured sedentary time, LPA and MVPA: Associations in secondary school and over the transition from primary to secondary school. Int J Behav Nutr Phys Act. 2016;13 PMID: 2016-21082-001.

199. Mullally ML, Taylor JP, Kuhle S, et al. A province-wide school nutrition policy and food consumption in elementary school children in Prince Edward Island. Can J Public Health. 2010 Jan-Feb;101(1):40-3. PMID: 20364537.

200. Mumford KG, Contant CK, Weissman J, et al. Changes in physical activity and travel behaviors in residents of a mixed-use development. Am J Prev Med. 2011 Nov;41(5):504-7. doi: 10.1016/j.amepre.2011.07.016. PMID: 22011422.

201. Murphy S, Moore GF, Tapper K, et al. Free healthy breakfasts in primary schools: a cluster randomised controlled trial of a policy intervention in Wales, UK. Public Health Nutr. 2011 Feb;14(2):219-26. doi: 10.1017/s1368980010001886. PMID: 20602868.

202. Nanney MS, MacLehose R, Kubik MY, et al. Recommended school policies are associated with student sugary drink and fruit and vegetable intake. Prev Med. 2014 May;62:179-81. doi: 10.1016/j.ypmed.2014.01.026. PMID: 24518003.

203. Nanney MS, MacLehose RF, Kubik MY, et al. School Obesity Prevention Policies and Practices in Minnesota and Student Outcomes: A Longitudinal Cohort Study. Am J Prev Med. 2016 Nov;51(5):656-63. doi: 10.1016/j.amepre.2016.05.008. PMID: 27320703.

204. Naul R, Schmelt D, Dreiskaemper D, et al. 'Healthy children in sound communities' (HCSC/gkgk)--a Dutch-German community-based network project to counteract obesity and physical inactivity. Fam Pract. 2012 Apr;29 Suppl 1:i110-i6. doi: 10.1093/fampra/cmr097. PMID: 22399539.

205. Naylor PJ, Macdonald HM, Warburton DE, et al. An active school model to promote physical activity in elementary schools: action schools! BC. Br J Sports Med. 2008 May;42(5):338-43. doi: 10.1136/bjsm.2007.042036. PMID: 18272538.

206. Naylor PJ, Macdonald HM, Zebedee JA, et al. Lessons learned from Action Schools! BC--an 'active school' model to promote physical activity in elementary schools. J Sci Med Sport. 2006 Oct;9(5):413-23. doi: 10.1016/j.jsams.2006.06.013. PMID: 16884957.

Page 347: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-169

207. Nehme EK, Perez A, Ranjit N, et al. The Effect of New Shower Facilities on Physical Activity Behaviors of Employees: A Quasi-experiment. J Phys Act Health. 2017 Feb;14(2):98-107. doi: 10.1123/jpah.2015-0418. PMID: 27775466.

208. Neumark-Sztainer DR, Friend SE, Flattum CF, et al. New moves-preventing weight-related problems in adolescent girls a group-randomized study. Am J Prev Med. 2010 Nov;39(5):421-32. doi: 10.1016/j.amepre.2010.07.017. PMID: 20965379.

209. Nguyen BT, Powell LM. Supplemental nutrition assistance program participation and sugar-sweetened beverage consumption, overall and by source. Prev Med. 2015 Dec;81:82-6. doi: 10.1016/j.ypmed.2015.08.003. PMID: 26303370.

210. Nicklas T, Lopez S, Liu Y, et al. Motivational theater to increase consumption of vegetable dishes by preschool children. Int J Behav Nutr Phys Act. 2017 Feb 07;14(1):16. doi: 10.1186/s12966-017-0468-0. PMID: 28166788.

211. Oh AY, Hennessy E, McSpadden KE, et al. Contextual Influences on Weight Status Among Impoverished Adolescents: Neighborhood Amenities for Physical Activity and State Laws for Physical Education Time Requirements. J Phys Act Health. 2015 Jun;12(6):875-8. doi: 10.1123/jpah.2013-0303. PMID: 25109235.

212. Olsho LE, Payne GH, Walker DK, et al. Impacts of a farmers' market incentive programme on fruit and vegetable access, purchase and consumption. Public Health Nutr. 2015 Oct;18(15):2712-21. doi: 10.1017/s1368980015001056. PMID: 25919225.

213. Ortega AN, Albert SL, Chan-Golston AM, et al. Substantial improvements not seen in health behaviors following corner store conversions in two Latino food swamps. BMC Public Health. 2016 May 11;16:389. doi: 10.1186/s12889-016-3074-1. PMID: 27169514.

214. Panter J, Heinen E, Mackett R, et al. Impact of New Transport Infrastructure on Walking, Cycling, and Physical Activity. Am J Prev Med. 2016 Feb;50(2):e45-53. doi: 10.1016/j.amepre.2015.09.021. PMID: 26585051.

215. Parsons WG, Garcia GM, Hoffman PK. Evaluating school wellness policy in curbing childhood obesity in Anchorage, Alaska. J Sch Nurs. 2014 Oct;30(5):324-31. doi: 10.1177/1059840513513155. PMID: 24316497.

216. Pate RR, Ward DS, Saunders RP, et al. Promotion of physical activity among high-school girls: a randomized controlled trial. Am J Public Health. 2005 Sep;95(9):1582-7. doi: 10.2105/ajph.2004.045807. PMID: 16118370.

217. Pbert L, Druker S, Barton B, et al. A School-Based Program for Overweight and Obese Adolescents: A Randomized Controlled Trial. J Sch Health. 2016 Oct;86(10):699-708. doi: 10.1111/josh.12428. PMID: 27619760.

218. Perry CL, Bishop DB, Taylor GL, et al. A randomized school trial of environmental strategies to encourage fruit and vegetable consumption among children. Health Educ Behav. 2004 Feb;31(1):65-76. PMID: 14768658.

219. Peterson KE, Spadano-Gasbarro JL, Greaney ML, et al. Three-Year Improvements in Weight Status and Weight-Related Behaviors in Middle School Students: The Healthy Choices Study. PLoS One. 2015;10(8):e0134470. doi: 10.1371/journal.pone.0134470. PMID: 26295837.

220. Pope M. Preventing Weight Gain in Children Who Are School Age and African-American. Pediatr Phys Ther. 2016 Summer;28(2):207-16. doi: 10.1097/pep.0000000000000243. PMID: 26914717.

221. Powell LM, Chriqui J, Chaloupka FJ. Associations between state-level soda taxes and adolescent body mass index. J Adolesc Health. 2009 Sep;45(3 Suppl):S57-63. doi: 10.1016/j.jadohealth.2009.03.003. PMID: 19699437.

222. Quigg R, Reeder AI, Gray A, et al. The Effectiveness of a Community Playground Intervention. Journal of Urban Health-Bulletin of the New York Academy of Medicine. 2012 Feb;89(1):171-84. doi: 10.1007/s11524-011-9622-1.

223. Ransley JK, Greenwood DC, Cade JE, et al. Does the school fruit and vegetable scheme improve children's diet? A non-randomised controlled trial. J Epidemiol Community Health. 2007 Aug;61(8):699-703. doi: 10.1136/jech.2006.052696. PMID: 17630369.

224. Reger-Nash B, Bauman A, Booth-Butterfield S, et al. Wheeling walks: evaluation of a media-based community intervention. Fam Community Health. 2005 Jan-Mar;28(1):64-78. PMID: 15625507.

225. Reger-Nash B, Bauman A, Cooper L, et al. WV Walks: replication with expanded reach. J Phys Act Health. 2008 Jan;5(1):19-27. PMID: 18209251.

226. Reilly JJ, Kelly L, Montgomery C, et al. Physical activity to prevent obesity in young children: cluster randomised controlled trial. BMJ. 2006 Nov 18;333(7577):1041. doi: 10.1136/bmj.38979.623773.55. PMID: 17028105.

Page 348: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-170

227. Restrepo BJ. Calorie Labeling in Chain Restaurants and Body Weight: Evidence from New York. Health Econ. 2016 Jul 24doi: 10.1002/hec.3389. PMID: 27451966.

228. Reynolds KD, Franklin FA, Binkley D, et al. Increasing the fruit and vegetable consumption of fourth-graders: results from the high 5 project. Prev Med. 2000 Apr;30(4):309-19. doi: 10.1006/pmed.1999.0630. PMID: 10731460.

229. Ridgers ND, Fairclough SJ, Stratton G. Twelve-month effects of a playground intervention on children's morning and lunchtime recess physical activity levels. J Phys Act Health. 2010 Mar;7(2):167-75. PMID: 20484755.

230. Ridgers ND, Stratton G, Fairclough SJ, et al. Children's physical activity levels during school recess: a quasi-experimental intervention study. Int J Behav Nutr Phys Act. 2007 May 21;4:19. doi: 10.1186/1479-5868-4-19. PMID: 17517136.

231. Ridgers ND, Stratton G, Fairclough SJ, et al. Long-term effects of a playground markings and physical structures on children's recess physical activity levels. Prev Med. 2007 May;44(5):393-7. doi: 10.1016/j.ypmed.2007.01.009. PMID: 17335891.

232. Riis J, Grason H, Strobino D, et al. State school policies and youth obesity. Matern Child Health J. 2012 Apr;16 Suppl 1:S111-8. doi: 10.1007/s10995-012-1000-4. PMID: 22527761.

233. Ritchie LD, Rosen NJ, Fenton K, et al. School Breakfast Policy Is Associated with Dietary Intake of Fourth- and Fifth-Grade Students. J Acad Nutr Diet. 2016 Mar;116(3):449-57. doi: 10.1016/j.jand.2015.08.020. PMID: 26433452.

234. Robles B, Montes CE, Nobari TZ, et al. Dietary Behaviors among Public Health Center Clients with Electronic Benefit Transfer Access at Farmers' Markets. J Acad Nutr Diet. 2017 Jan;117(1):58-68. doi: 10.1016/j.jand.2016.07.012. PMID: 27618576.

235. Rogers VW, Hart PH, Motyka E, et al. Impact of Let's Go! 5-2-1-0: a community-based, multisetting childhood obesity prevention program. J Pediatr Psychol. 2013 Oct;38(9):1010-20. doi: 10.1093/jpepsy/jst057. PMID: 23933841.

236. Rush E, McLennan S, Obolonkin V, et al. Project Energize: whole-region primary school nutrition and physical activity programme; evaluation of body size and fitness 5 years after the randomised controlled trial. Br J Nutr. 2014 Jan 28;111(2):363-71. doi: 10.1017/s0007114513002316. PMID: 23867069.

237. Rushakoff JA, Zoughbie DE, Bui N, et al. Evaluation of Healthy2Go: A country store transformation project to improve the food environment and consumer choices in Appalachian Kentucky. Prev Med Rep. 2017 Sep;7:187-92. doi: 10.1016/j.pmedr.2017.06.009. PMID: 28706778.

238. Sabia JJ, Nguyen TT, Rosenberg O. High School Physical Education Requirements and Youth Body Weight: New Evidence from the YRBS. Health Econ. 2016 Aug 30doi: 10.1002/hec.3399. PMID: 27576770.

239. Sadler RC, Gilliland JA, Arku G. A food retail-based intervention on food security and consumption. Int J Environ Res Public Health. 2013 Aug 05;10(8):3325-46. doi: 10.3390/ijerph10083325. PMID: 23921626.

240. Sallis JF, McKenzie TL, Conway TL, et al. Environmental interventions for eating and physical activity: a randomized controlled trial in middle schools. Am J Prev Med. 2003 Apr;24(3):209-17. PMID: 12657338.

241. Sanchez-Vaznaugh EV, Sanchez BN, Baek J, et al. 'Competitive' food and beverage policies: are they influencing childhood overweight trends? Health Aff (Millwood). 2010 Mar-Apr;29(3):436-46. doi: 10.1377/hlthaff.2009.0745. PMID: 20194985.

242. Schanzenbach D. Do School Lunches Contribute to Childhood Obesity? : Harris School of Public Policy Studies, University of Chicago, Working Papers: 0513; 2005.

243. Schwartz AE, Leardo M, Aneja S, et al. Effect of a School-Based Water Intervention on Child Body Mass Index and Obesity. JAMA Pediatr. 2016 Mar;170(3):220-6. doi: 10.1001/jamapediatrics.2015.3778. PMID: 26784336.

244. Sekhobo JP, Edmunds LS, Dalenius K, et al. Neighborhood disparities in prevalence of childhood obesity among low-income children before and after implementation of New York City child care regulations. Prev Chronic Dis. 2014;11:E181. doi: 10.5888/pcd11.140152. PMID: 25321632.

245. Sharma SV, Markham C, Chow J, et al. Evaluating a school-based fruit and vegetable co-op in low-income children: A quasi-experimental study. Prev Med. 2016 Oct;91:8-17. doi: 10.1016/j.ypmed.2016.07.022. PMID: 27471022.

246. Shive SE, Morris MN. Evaluation of the energize your life! Social marketing campaign pilot study to increase fruit intake among community college students. J Am Coll Health. 2006 Jul-Aug;55(1):33-9. doi: 10.3200/jach.55.1.33-40. PMID: 16889313.

Page 349: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-171

247. Sigmund E, El Ansari W, Sigmundova D. Does school-based physical activity decrease overweight and obesity in children aged 6-9 years? A two-year non-randomized longitudinal intervention study in the Czech Republic. BMC Public Health. 2012 Jul 29;12:570. doi: 10.1186/1471-2458-12-570. PMID: 22892226.

248. Simon C, Schweitzer B, Oujaa M, et al. Successful overweight prevention in adolescents by increasing physical activity: a 4-year randomized controlled intervention. Int J Obes (Lond). 2008 Oct;32(10):1489-98. doi: 10.1038/ijo.2008.99. PMID: 18626482.

249. Slater S, Chriqui J, Chaloupka FJ, et al. Joint use policies: are they related to adolescent behavior? Prev Med. 2014 Dec;69 Suppl 1:S37-43. doi: 10.1016/j.ypmed.2014.08.032. PMID: 25199731.

250. Spence S, Delve J, Stamp E, et al. The impact of food and nutrient-based standards on primary school children's lunch and total dietary intake: a natural experimental evaluation of government policy in England. PLoS One. 2013;8(10):e78298. doi: 10.1371/journal.pone.0078298. PMID: 24205190.

251. Steenhuis I, Van Assema P, Van Breukelen G, et al. The impact of educational and environmental interventions in Dutch worksite cafeterias. Health Promot Int. 2004 Sep;19(3):335-43. doi: 10.1093/heapro/dah307. PMID: 15306618.

252. Stephens RL, Xu Y, Lesesne CA, et al. Relationship between child care centers' compliance with physical activity regulations and children's physical activity, New York City, 2010. Prev Chronic Dis. 2014 Oct 16;11:E179. doi: 10.5888/pcd11.130432. PMID: 25321630.

253. Story M, Hannan PJ, Fulkerson JA, et al. Bright Start: Description and main outcomes from a group-randomized obesity prevention trial in American Indian children. Obesity (Silver Spring). 2012 Nov;20(11):2241-9. doi: 10.1038/oby.2012.89. PMID: 22513491.

254. Stratton G, Mullan E. The effect of multicolor playground markings on children's physical activity level during recess. Prev Med. 2005 Nov-Dec;41(5-6):828-33. doi: 10.1016/j.ypmed.2005.07.009. PMID: 16137756.

255. Sturm R, Hattori A. Diet and obesity in Los Angeles County 2007-2012: Is there a measurable effect of the 2008 "Fast-Food Ban"? Soc Sci Med. 2015 May;133:205-11. doi: 10.1016/j.socscimed.2015.03.004. PMID: 25779774.

256. Sturm R, Powell LM, Chriqui JF, et al. Soda Taxes, Soft Drink Consumption, And Children's Body Mass Index. Health Affairs. 2010 May-Jun;29(5):1052-8. doi: 10.1377/hlthaff.2009.0061.

257. Taber DR, Chriqui JF, Chaloupka FJ. Differences in nutrient intake associated with state laws regarding fat, sugar, and caloric content of competitive foods. Arch Pediatr Adolesc Med. 2012 May;166(5):452-8. doi: 10.1001/archpediatrics.2011.1839. PMID: 22566546.

258. Taber DR, Chriqui JF, Perna FM, et al. Weight status among adolescents in States that govern competitive food nutrition content. Pediatrics. 2012 Sep;130(3):437-44. doi: 10.1542/peds.2011-3353. PMID: 22891223.

259. Taber DR, Chriqui JF, Perna FM, et al. Association between state physical education (PE) requirements and PE participation, physical activity, and body mass index change. Prev Med. 2013 Nov;57(5):629-33. doi: 10.1016/j.ypmed.2013.08.018. PMID: 23978523.

260. Taber DR, Chriqui JF, Powell LM, et al. Banning all sugar-sweetened beverages in middle schools: reduction of in-school access and purchasing but not overall consumption. Arch Pediatr Adolesc Med. 2012 Mar;166(3):256-62. doi: 10.1001/archpediatrics.2011.200. PMID: 22064875.

261. Taber DR, Stevens J, Evenson KR, et al. State policies targeting junk food in schools: racial/ethnic differences in the effect of policy change on soda consumption. Am J Public Health. 2011 Sep;101(9):1769-75. doi: 10.2105/ajph.2011.300221. PMID: 21778484.

262. Tak NI, Te Velde SJ, Brug J. Ethnic differences in 1-year follow-up effect of the Dutch Schoolgruiten Project - promoting fruit and vegetable consumption among primary-school children. Public Health Nutr. 2007 Dec;10(12):1497-507. doi: 10.1017/s1368980007000456. PMID: 17610757.

263. Tak NI, Te Velde SJ, Brug J. Long-term effects of the Dutch Schoolgruiten Project--promoting fruit and vegetable consumption among primary-school children. Public Health Nutr. 2009 Aug;12(8):1213-23. doi: 10.1017/s1368980008003777. PMID: 18940029.

264. Tarp J, Domazet SL, Froberg K, et al. Effectiveness of a School-Based Physical Activity Intervention on Cognitive Performance in Danish Adolescents: LCoMotion-Learning, Cognition and Motion - A Cluster Randomized Controlled Trial. PLoS One. 2016;11(6):e0158087. doi: 10.1371/journal.pone.0158087. PMID: 27341346.

Page 350: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-172

265. Te Velde SJ, Brug J, Wind M, et al. Effects of a comprehensive fruit- and vegetable-promoting school-based intervention in three European countries: the Pro Children Study. Br J Nutr. 2008 Apr;99(4):893-903. doi: 10.1017/s000711450782513x. PMID: 17953787.

266. Tester JM, Leung CW, Crawford PB. Revised WIC Food Package and Children's Diet Quality. Pediatrics. 2016 May;137(5)doi: 10.1542/peds.2015-3557. PMID: 27244804.

267. Tomlin D, Naylor PJ, McKay H, et al. The impact of Action Schools! BC on the health of Aboriginal children and youth living in rural and remote communities in British Columbia. Int J Circumpolar Health. 2012 Mar 19;71:17999. doi: 10.3402/ijch.v71i0.17999. PMID: 22456048.

268. Toussaint LL, Housholder K, Janssen K, et al. Slowing BMI Growth Trajectories in Elementary School-Aged Children: The Northeast Iowa Food and Fitness Initiative. Fam Community Health. 2017 Jul/Sep;40(3):192-7. doi: 10.1097/fch.0000000000000151. PMID: 28525438.

269. Utter J, Denny S, Dyson B. School gardens and adolescent nutrition and BMI: Results from a national, multilevel study. Prev Med. 2016 Feb;83:1-4. doi: 10.1016/j.ypmed.2015.11.022. PMID: 26657347.

270. Vadiveloo MK, Dixon LB, Elbel B. Consumer purchasing patterns in response to calorie labeling legislation in New York City. Int J Behav Nutr Phys Act. 2011 May 27;8:51. doi: 10.1186/1479-5868-8-51. PMID: 21619632.

271. Van Cauwenberghe E, De Bourdeaudhuij I, Maes L, et al. Efficacy and feasibility of lowering playground density to promote physical activity and to discourage sedentary time during recess at preschool: a pilot study. Prev Med. 2012 Oct;55(4):319-21. doi: 10.1016/j.ypmed.2012.07.014. PMID: 22846504.

272. Vander Ploeg KA, McGavock J, Maximova K, et al. School-based health promotion and physical activity during and after school hours. Pediatrics. 2014 Feb;133(2):e371-8. doi: 10.1542/peds.2013-2383. PMID: 24420806.

273. Vasquez A, Sherwood NE, Larson N, et al. A novel dietary improvement strategy: examining the potential impact of community-supported agriculture membership. Public Health Nutr. 2016 Oct;19(14):2618-28. doi: 10.1017/s1368980015003638. PMID: 26857484.

274. Veugelers PJ, Fitzgerald AL. Effectiveness of school programs in preventing childhood obesity: a multilevel comparison. Am J Public Health. 2005 Mar;95(3):432-5. doi: 10.2105/ajph.2004.045898. PMID: 15727972.

275. von Hippel PT, Bradbury WK. The effects of school physical education grants on obesity, fitness, and academic achievement. Prev Med. 2015 Sep;78:44-51. doi: 10.1016/j.ypmed.2015.06.011. PMID: 26163396.

276. Waters E, Gibbs L, Tadic M, et al. Cluster randomised trial of a school-community child health promotion and obesity prevention intervention: findings from the evaluation of fun 'n healthy in Moreland! BMC Public Health. 2017 Aug 03;18(1):92. doi: 10.1186/s12889-017-4625-9. PMID: 28774278.

277. Weaver RG, Webster CA, Egan C, et al. Partnerships for Active Children in Elementary Schools: Outcomes of a 2-Year Pilot Study to Increase Physical Activity During the School Day. Am J Health Promot. 2017 Jan 01:890117117707289. doi: 10.1177/0890117117707289. PMID: 28482678.

278. Webb E, Laverty A, Mindell J, et al. Free Bus Travel and Physical Activity, Gait Speed, and Adiposity in the English Longitudinal Study of Ageing. Am J Public Health. 2016 Jan;106(1):136-42. doi: 10.2105/ajph.2015.302907. PMID: 26562118

279. Webb E, Netuveli G, Millett C. Free bus passes, use of public transport and obesity among older people in England. J Epidemiol Community Health. 2012 Feb;66(2):176-80. doi: 10.1136/jech.2011.133165. PMID: 21911850.

280. Wells L, Nelson M. The National School Fruit Scheme produces short-term but not longer-term increases in fruit consumption in primary school children. Br J Nutr. 2005 Apr;93(4):537-42. PMID: 15946417.

281. Wells NM, Myers BM, Henderson CR, Jr. School gardens and physical activity: a randomized controlled trial of low-income elementary schools. Prev Med. 2014 Dec;69 Suppl 1:S27-33. doi: 10.1016/j.ypmed.2014.10.012. PMID: 25456803.

282. Wendel ML, Benden ME, Zhao H, et al. Stand-Biased Versus Seated Classrooms and Childhood Obesity: A Randomized Experiment in Texas. Am J Public Health. 2016 Oct;106(10):1849-54. doi: 10.2105/ajph.2016.303323. PMID: 27552276.

283. West ST, Shores KA. The impacts of building a greenway on proximate residents' physical activity. J Phys Act Health. 2011 Nov;8(8):1092-7. PMID: 22039127.

284. Whaley SE, McGregor S, Jiang L, et al. A WIC-Based Intervention to Prevent Early Childhood Overweight. J Nutr Educ Behav. 2010 May-Jun;42(3):S47-S51. doi: 10.1016/j.jneb.2010.02.010. PMID: 20399409

Page 351: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

H-173

285. Whetstone LM, Kolasa KM, Collier DN. Participation in community-originated interventions is associated with positive changes in weight status and health behaviors in youth. Am J Health Promot. 2012 Sep-Oct;27(1):10-6. doi: 10.4278/ajhp.100415-QUAN-117. PMID: 22950920.

286. Whitt-Glover MC, Ham SA, Yancey AK. Instant Recess(R): a practical tool for increasing physical activity during the school day. Prog Community Health Partnersh. 2011 Fall;5(3):289-97. doi: 10.1353/cpr.2011.0031. PMID: 22080777.

287. Williamson DA, Champagne CM, Harsha DW, et al. Effect of an environmental school-based obesity prevention program on changes in body fat and body weight: a randomized trial. Obesity (Silver Spring). 2012 Aug;20(8):1653-61. doi: 10.1038/oby.2012.60. PMID: 22402733.

288. Wilson DK, Van Horn ML, Siceloff ER, et al. The Results of the "Positive Action for Today's Health" (PATH) Trial for Increasing Walking and Physical Activity in Underserved African-American Communities. Ann Behav Med. 2015 Jun;49(3):398-410. doi: 10.1007/s12160-014-9664-1. PMID: 25385203.

289. Woodward-Lopez G, Gosliner W, Samuels SE, et al. Lessons learned from evaluations of California's statewide school nutrition standards. Am J Public Health. 2010 Nov;100(11):2137-45. doi: 10.2105/ajph.2010.193490. PMID: 20864696.

290. Wright K, Giger JN, Norris K, et al. Impact of a nurse-directed, coordinated school health program to enhance physical activity behaviors and reduce body mass index among minority children: a parallel-group, randomized control trial. Int J Nurs Stud. 2013 Jun;50(6):727-37. doi: 10.1016/j.ijnurstu.2012.09.004. PMID: 23021318.

291. Wright K, Norris K, Newman Giger J, et al. Improving healthy dietary behaviors, nutrition knowledge, and self-efficacy among underserved school children with parent and community involvement. Child Obes. 2012 Aug;8(4):347-56. doi: 10.1089/chi.2012.0045. PMID: 22867074.

292. Wrigley N, Warm D, Margetts B. Deprivation, Diet, and Food-Retail Access: Findings from the Leeds ‘Food Deserts' Study. Environ Plann A 2003;35(1):151-88.

293. Yildirim M, Arundell L, Cerin E, et al. What helps children to move more at school recess and lunchtime? Mid-intervention results from Transform-Us! cluster-randomised controlled trial. Br J Sports Med. 2014 Feb;48(3):271-7. doi: 10.1136/bjsports-2013-092466. PMID: 24124036.

294. Zhou Z, Ren H, Yin Z, et al. A policy-driven multifaceted approach for early childhood physical fitness promotion: impacts on body composition and physical fitness in young Chinese children. BMC Pediatr. 2014 May 05;14:118. doi: 10.1186/1471-2431-14-118. PMID: 24886119.

295. Zhu X, Lu Z, Yu C, et al. Walkable communities: impacts on residents’ physical and social health. World Health Des 2013;7:68-75.

296. Zhu X, Yu CY, Lee C, et al. A retrospective study on changes in residents' physical activities, social interactions, and neighborhood cohesion after moving to a walkable community. Prev Med. 2014 Dec;69 Suppl 1:S93-7. doi: 10.1016/j.ypmed.2014.08.013. PMID: 25158208.

Page 352: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-1

Appendix I. Programs, Policies, or Built Environment Changes in Included Studies (U.S. Only) Table I1. Program, policy, or built environment evaluations in the United States at the government level (local, state/regional, or Federal)

2005 Safe Routes to School (SRTS) initiative1 Healthy Foods, Healthy Families (HFHF)2 Reach 20103

Act 1220 (Arkansas): NMI screening4 Healthy2Go: received training and technical assistance to increase availability and awareness of healthy foods5

Ready for Recess6

AHOME7 HEROES (Healthy, Energetic, Ready, Outstanding, Enthusiastic, Schools) Initiative8

Reduced availability of sugar sweetened beverage and diet soda in a la carte and vending programs in Maine public high schools.9

All Sugar Sweetened Beverage Ban and the only soda ban10 High 511 Regulations governing minimum PA standards in child care centers12 an independent grocery store (Witherbee’s Market) at the center of the Carriage Town neighborhood13

HUD assignment for urban housing.14 Residents moving to a new urbanist neighborhood15

Analysis of CLASS and NSCH16 installation of new bicycle boulevards.17 results of different school nutrition policies18 Anchorage School District’s School Wellness Policy19 Instant Recess20 Route H21 ATLAS22 Joint Use Policies23 School Breakfast Program (SBP) and National School Lunch program24 Bootheel Walking Promotion Project (creation of walking trails)25 Kearney Public School (KPS) District; population dose study26 School garden27 Boston Active School Day Policy28 Kid's Choice Program29 School meals30 Boston Public Schools Snack and Beverage Policy31 LA Sprouts: gardening, nutrition and cooking intervention32 School Nutrition Policy Initiative (SNPI)33

Breakfast service policy34 Learning Landscape35 School-specific policies/program36 Bright Start37 LEED for Neighborhood Development (LEED-ND)38 Schoolwide food policies and practices39 Building new greenway40 Let’s Go!41 Senate Bill 12 (SB 12), California42 California Childhood Obesity Prevention Act: Obesity Prevention Motion; Obesity Prevention Motion43

Light rail transit44, 45 Senate Bill SB 677; Los Angeles Unified School District Nutrition Policy46

Calorie Labeling Laws in New York City jurisdictions47 Los Angeles Fast Food ban48 SNAP (Food stamp program)49

Calorie-labeling policy50 Mean distance walked per day for transportation purposes51 SPARK (Sports, Play and Active Recreation for Kids)52 Child Nutrition and WIC Reauthorization Act (CNRA)53, 54 Mebane on the Move Intervention55 Standing desks in classrooms56 Children’s Healthy Living Program for Remote Underserved Minority Populations in the Pacific Region (CHL)57

Michigan Farmers' Market Nutrition Program58 State competitive food laws and state physical education laws 59

Page 353: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-2

Colorado Trust60 Michigan State Board of Education (MSBE) nutrition policy61, 62 state high school physical education requirements63 Communities Putting Prevention to Work (CPPW)64 Middle-School Physical Activity and Nutrition (M-SPAN)65 State laws regarding the nutrition content of competitive foods sold in high

schools66 Community Health Advisor67 Mississippi Fresh Fruit and Vegetable Program (MFVP)68 State Soda Tax69 Community Interventions for Health: build walking trails, bike service system70

mixed-use redevelopment community in metropolitan Atlanta (Atlantic Station) that promotes walking and physical activity71

State-level grocery story and vending machine soda taxes72

Community-Supported Agriculture (CSA)73 Moving Across Places74, 75 Step Ahead trial76 Competitive Food Laws (varies by state) in 40 States77 Moving to a walkable community78 Supplemental Nutrition Assistance Program (SNAP)79-83 Competitive foods and beverages policies in public schools in multple states: NY, WA, NC, GA, TX, KY, TN, LA, GA, OK, KS84

Multilevel community intervention85 Texas Fitness Now (TFN)86

Complete streets design87 Multi-level intervention to influence children’s’ PA behaviors88 Texas Public School Nutrition Policy89 Comprehensive School Physical Activity Program (CSPAP)90

Multiple state laws for 50 states and DC91 Texas Senate Bill 42 (SB42)92

Comprehensive school-based intervention on healthy behavior93 Munch and Move94 The Active Early program95 Coordinated School Health Program (CSHP)96 National School Lunch Program97, 98 The COMMUNICATE (COMMUNIty-wide Campaign To promote

Exercise)99 Electronic Benefit Transfer (EBT) at farmers markets100 Navajo Healthy Stores (NHS) program101 The HEALTHY Intervention102 Energize Your Life!103 New urbanist neighborhood (in comparison to traditional suburban

neighborhood)104 The Healthy Youth Places105

Environmental Modifications (EMs)106 New York City's Food Retail Expansion to Support Health (FRESH)107 The Learning, Cognition & Motion (LCoMotion) study108 Excise tax on SSB109 North Carolina Health and Wellness Trust Fund (NCHWTF)110 The Mālaekahana Bike Path111 Fitness Zone112 Opening access to schoolyard outside of regular operation hours113 The Northwest Iowa Food and Fitness Initiative: Regional Safe Routes to

School Programming (see www.saferoutestoschools.org), Walking School Buses, and bike rodeos 114

Food Dudes115 Partnerships for Active Children in Elementary Schools (PACES)116 The School Obesity-related Policy Evaluation study (ScOPE)117 Fresh fruit availability at worksites118 Pathways Program (food protion focused on providing lower fat foods)119 The Supplemental Nutrition Assistance Program (SNAP)120 Fresh to You Markets121 Pennsylvania Fresh Food Financing Initiative122 Vacant lot greening program123

Page 354: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-3

Fruit and vegetable promotion in stores with staff training and installing new equipment124

Physical Activity and Healthy Food Intervention125 Walking school Bus126

Go!127 Physical education state policies128 Wheeling Walks129, 130

Gold Medal Schools program131 playground upgrade program132 WIC133, 134 Head Start: classroom videotaped (DVD) puppet shows.135 Policy Changes Targeting Junk Food in School vending machines136 WIC and SNAP137 Health Bucks: Farmers' market incentive program138 Pro Children139 Workplace showers140 Healthier Missouri Communities141 Project CHOICE (Center

for Healthy Options and Community Empowerment)- community-based participatory research study142

YRBSS merged with state policies143, 144

Healthier nutrition guidelines145 Project Healthy Schools146 Healthy Choice147 Proyecto Mercado FRESCO148 Healthy Communities Study (HCS)149 Railroad Park150

Page 355: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-4

Table I2. Program, policy, or built environment evaluations in the United States at the nongovernment level Men on the Move: Growing Communities (MOTMGC)151 Movin’ Afterschool152 Lifestyle Education for Activity Program153 Lookin’ Good Feelin’ Good: School nurse intervention and after-school exercise program154

Healthy Eating Active Living–Community Health Initiative (HEAL-CHI)155 Healthy Schools Program (HSP)156

Project Healthy Schools in two Cities in Michigan157 Kids N Fitness158 Choosing Healthy & Active Lifestyles for Kids (CHALK)159

Physical Activity 4 Everyone160 Sports, Play, and Active Recreation for Kids (SPARK)161 Positive Action for Today’s Health162 Healthy Food Financing Initiative163 The Healthy Options for Nutrition Environments in Schools (Healthy

ONES)164 New Moves intervention165

APPLE166 HealthWorks167 ‘Intervention Centered on Adolescents’ Physical activity and Sedentary behavior’ (ICAPS)168

Energy Balance for Kids with Play (EB4K with Play)169 Partnership for an Active Community Environment (PACE) and city of New Orleans170

America SCORES171

Healthy Store Intervention172 Water Jet installation in schools173 The Cafeteria Power Plus Intervention174 Physical activity intervention175 The Healthy Foods Hawaii (HFH)176 Creating Healthy, Active and Nurturing Growing-up Environments

(CHANGE) study: serve healthier school breakfast and lunch177 The HealthMPowers program178 Healthy Living Cambridge Kids (KLCK)179 The 5 a Day intervention: newsletters and promotions to encourage healthy

eating180 Multi-level weight-gain prevention181 Shape-Up Somerville182 Health promotion messaging and counseling 183 Romp & Chomp184 JUMP-in185 A social marketing

intervention promoted walking186

Page 356: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-5

References for Appendix I

1. Hoelscher D, Ory M, Dowdy D, et al. Effects of funding allocation for Safe Routes to School programs on active commuting to school and related behavioral, knowledge, and psychosocial outcomes: Results from the Texas Childhood Obesity Prevention Policy Evaluation (T-COPPE) study. Environ Behav. 2016;48(1):210-29. doi: 10.1177/0013916515613541. PMID: 2016-02263-012.

2. Bowling AB, Moretti M, Ringelheim K, et al. Healthy Foods, Healthy Families: combining incentives and exposure interventions at urban farmers' markets to improve nutrition among recipients of US federal food assistance. Health Promot Perspect. 2016;6(1):10-6. doi: 10.15171/hpp.2016.02. PMID: 27123431.

3. Liao Y, Siegel PZ, Zhou H, et al. Reduced Disparity in Vegetable Consumption in 16 Disadvantaged Black Communities: A Successful 5-Year Community-Based Participatory Intervention. J Racial Ethn Health Disparities. 2015 Jun;2(2):211-8. doi: 10.1007/s40615-014-0065-8. PMID: 26150921.

4. Gee KA. School-Based Body Mass Index Screening and Parental Notification in Late Adolescence: Evidence From Arkansas's Act 1220. J Adolesc Health. 2015 Sep;57(3):270-6. doi: 10.1016/j.jadohealth.2015.05.007. PMID: 26115907.

5. Rushakoff JA, Zoughbie DE, Bui N, et al. Evaluation of Healthy2Go: A country store transformation project to improve the food environment and consumer choices in Appalachian Kentucky. Prev Med Rep. 2017 Sep;7:187-92. doi: 10.1016/j.pmedr.2017.06.009. PMID: 28706778.

6. Huberty JL, Beets MW, Beighle A, et al. Environmental modifications to increase physical activity during recess: preliminary findings from ready for recess. J Phys Act Health. 2011 Sep;8 Suppl 2:S249-56. PMID: 21918239.

7. Camacho-Rivera M, Rosenbaum E, Yama C, et al. Low-Income Housing Rental Assistance, Perceptions of Neighborhood Food Environment, and Dietary Patterns among Latino Adults: the AHOME Study. J Racial Ethn Health Disparities. 2017 Jun;4(3):346-53. doi: 10.1007/s40615-016-0234-z. PMID: 27129854.

8. King MH, Lederer AM, Sovinski D, et al. Implementation and evaluation of the HEROES initiative: a tri-state coordinated school health program to reduce childhood obesity. Health Promot Pract. 2014 May;15(3):395-405. doi: 10.1177/1524839913512835. PMID: 24334542.

9. Blum JE, Davee AM, Beaudoin CM, et al. Reduced availability of sugar-sweetened beverages and diet soda has a limited impact on beverage consumption patterns in Maine high school youth. J Nutr Educ Behav. 2008 Nov-Dec;40(6):341-7. doi: 10.1016/j.jneb.2007.12.004. PMID: 18984489.

10. Taber DR, Chriqui JF, Powell LM, et al. Banning all sugar-sweetened beverages in middle schools: reduction of in-school access and purchasing but not overall consumption. Arch Pediatr Adolesc Med. 2012 Mar;166(3):256-62. doi: 10.1001/archpediatrics.2011.200. PMID: 22064875.

11. Reynolds KD, Franklin FA, Binkley D, et al. Increasing the fruit and vegetable consumption of fourth-graders: results from the high 5 project. Prev Med. 2000 Apr;30(4):309-19. doi: 10.1006/pmed.1999.0630. PMID: 10731460.

12. Stephens RL, Xu Y, Lesesne CA, et al. Relationship between child care centers' compliance with physical activity regulations and children's physical activity, New York City, 2010. Prev Chronic Dis. 2014 Oct 16;11:E179. doi: 10.5888/pcd11.130432. PMID: 25321630.

13. Sadler RC, Gilliland JA, Arku G. A food retail-based intervention on food security and consumption. Int J Environ Res Public Health. 2013 Aug 05;10(8):3325-46. doi: 10.3390/ijerph10083325. PMID: 23921626.

14. Ludwig J, Sanbonmatsu L, Gennetian L, et al. Neighborhoods, obesity, and diabetes--a randomized social experiment. N Engl J Med. 2011 Oct 20;365(16):1509-19. doi: 10.1056/NEJMsa1103216. PMID: 22010917.

15. Calise TV, Heeren T, DeJong W, et al. Do neighborhoods make people active, or do people make active neighborhoods? Evidence from a planned community in Austin, Texas. Prev Chronic Dis. 2013 Jun 20;10:E102. doi: 10.5888/pcd10.120119. PMID: 23786909.

Page 357: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-6

16. Oh AY, Hennessy E, McSpadden KE, et al. Contextual Influences on Weight Status Among Impoverished Adolescents: Neighborhood Amenities for Physical Activity and State Laws for Physical Education Time Requirements. J Phys Act Health. 2015 Jun;12(6):875-8. doi: 10.1123/jpah.2013-0303. PMID: 25109235.

17. Dill J, McNeil N, Broach J, et al. Bicycle boulevards and changes in physical activity and active transportation: findings from a natural experiment. Prev Med. 2014 Dec;69 Suppl 1:S74-8. doi: 10.1016/j.ypmed.2014.10.006. PMID: 25456802.

18. Riis J, Grason H, Strobino D, et al. State school policies and youth obesity. Matern Child Health J. 2012 Apr;16 Suppl 1:S111-8. doi: 10.1007/s10995-012-1000-4. PMID: 22527761.

19. Parsons WG, Garcia GM, Hoffman PK. Evaluating school wellness policy in curbing childhood obesity in Anchorage, Alaska. J Sch Nurs. 2014 Oct;30(5):324-31. doi: 10.1177/1059840513513155. PMID: 24316497.

20. Whitt-Glover MC, Ham SA, Yancey AK. Instant Recess(R): a practical tool for increasing physical activity during the school day. Prog Community Health Partnersh. 2011 Fall;5(3):289-97. doi: 10.1353/cpr.2011.0031. PMID: 22080777.

21. French SA, Harnack LJ, Hannan PJ, et al. Worksite environment intervention to prevent obesity among metropolitan transit workers. Prev Med. 2010 Apr;50(4):180-5. doi: 10.1016/j.ypmed.2010.01.002. PMID: 20079369.

22. Lubans DR, Smith JJ, Plotnikoff RC, et al. Assessing the sustained impact of a school-based obesity prevention program for adolescent boys: the ATLAS cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2016 Aug 20;13:92. doi: 10.1186/s12966-016-0420-8. PMID: 27542825.

23. Slater S, Chriqui J, Chaloupka FJ, et al. Joint use policies: are they related to adolescent behavior? Prev Med. 2014 Dec;69 Suppl 1:S37-43. doi: 10.1016/j.ypmed.2014.08.032. PMID: 25199731.

24. Capogrossi K, You W. The Influence of School Nutrition Programs on the Weight of Low-Income Children: A Treatment Effect Analysis. Health Econ. 2016 Jul 6doi: 10.1002/hec.3378. PMID: 27381591.

25. Brownson RC, Baker EA, Boyd RL, et al. A community-based approach to promoting walking in rural areas. Am J Prev Med. 2004 Jul;27(1):28-34. doi: 10.1016/j.amepre.2004.03.015. PMID: 15212772.

26. Heelan KA, Bartee RT, Nihiser A, et al. Healthier School Environment Leads to Decreases in Childhood Obesity: The Kearney Nebraska Story. Child Obes. 2015 Oct;11(5):600-7. doi: 10.1089/chi.2015.0005. PMID: 26440386.

27. Wells NM, Myers BM, Henderson CR, Jr. School gardens and physical activity: a randomized controlled trial of low-income elementary schools. Prev Med. 2014 Dec;69 Suppl 1:S27-33. doi: 10.1016/j.ypmed.2014.10.012. PMID: 25456803.

28. Cradock AL, Barrett JL, Carter J, et al. Impact of the Boston Active School Day policy to promote physical activity among children. Am J Health Promot. 2014 Jan-Feb;28(3 Suppl):S54-64. doi: 10.4278/ajhp.130430-QUAN-204. PMID: 24380467.

29. Hendy HM, Williams KE, Camise TS. Kid's Choice Program improves weight management behaviors and weight status in school children. Appetite. 2011 Apr;56(2):484-94. doi: 10.1016/j.appet.2011.01.024. PMID: 21277924.

30. Gleason PM, Dodd AH. School Breakfast Program but Not School Lunch Program Participation Is Associated with Lower Body Mass Index. J Am Diet Assoc. 2009 Feb;109(2):S118-S28. doi: 10.1016/j.jada.2008.10.058. PMID: 19166666.

31. Cradock AL, McHugh A, Mont-Ferguson H, et al. Effect of school district policy change on consumption of sugar-sweetened beverages among high school students, Boston, Massachusetts, 2004-2006. Prev Chronic Dis. 2011 Jul;8(4):A74. PMID: 21672398.

32. Gatto NM, Martinez LC, Spruijt-Metz D, et al. LA sprouts randomized controlled nutrition, cooking and gardening programme reduces obesity and metabolic risk in Hispanic/Latino youth. Pediatr Obes. 2017 Feb;12(1):28-37. doi: 10.1111/ijpo.12102. PMID: 26909882.

33. Foster GD, Sherman S, Borradaile KE, et al. A policy-based school intervention to prevent overweight and obesity. Pediatrics. 2008 Apr;121(4):e794-802. doi: 10.1542/peds.2007-1365. PMID: 18381508.

34. Ritchie LD, Rosen NJ, Fenton K, et al. School Breakfast Policy Is Associated with Dietary Intake of Fourth- and Fifth-Grade Students. J Acad Nutr Diet. 2016 Mar;116(3):449-57. doi: 10.1016/j.jand.2015.08.020. PMID: 26433452.

Page 358: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-7

35. Anthamatten P, Brink L, Lampe S, et al. An assessment of schoolyard renovation strategies to encourage children's physical activity. Int J Behav Nutr Phys Act. 2011 Apr 09;8:27. doi: 10.1186/1479-5868-8-27. PMID: 21477325.

36. Nanney MS, MacLehose RF, Kubik MY, et al. School Obesity Prevention Policies and Practices in Minnesota and Student Outcomes: A Longitudinal Cohort Study. Am J Prev Med. 2016 Nov;51(5):656-63. doi: 10.1016/j.amepre.2016.05.008. PMID: 27320703.

37. Story M, Hannan PJ, Fulkerson JA, et al. Bright Start: Description and main outcomes from a group-randomized obesity prevention trial in American Indian children. Obesity (Silver Spring). 2012 Nov;20(11):2241-9. doi: 10.1038/oby.2012.89. PMID: 22513491.

38. Zhu X, Lu Z, Yu C, et al. Walkable communities: impacts on residents’ physical and social health. World Health Des 2013;7:68-75.

39. Kubik MY, Lytle LA, Story M. Schoolwide food practices are associated with body mass index in middle school students. Arch Pediatr Adolesc Med. 2005 Dec;159(12):1111-4. doi: 10.1001/archpedi.159.12.1111. PMID: 16330732.

40. West ST, Shores KA. The impacts of building a greenway on proximate residents' physical activity. J Phys Act Health. 2011 Nov;8(8):1092-7. PMID: 22039127.

41. Madsen KA. School-based body mass index screening and parent notification: a statewide natural experiment. Arch Pediatr Adolesc Med. 2011 Nov;165(11):987-92. doi: 10.1001/archpediatrics.2011.127. PMID: 21727262.

42. Woodward-Lopez G, Gosliner W, Samuels SE, et al. Lessons learned from evaluations of California's statewide school nutrition standards. Am J Public Health. 2010 Nov;100(11):2137-45. doi: 10.2105/ajph.2010.193490. PMID: 20864696.

43. Sanchez-Vaznaugh EV, Sanchez BN, Baek J, et al. 'Competitive' food and beverage policies: are they influencing childhood overweight trends? Health Aff (Millwood). 2010 Mar-Apr;29(3):436-46. doi: 10.1377/hlthaff.2009.0745. PMID: 20194985.

44. Miller HJ, Tribby CP, Brown BB, et al. Public transit generates new physical activity: Evidence from individual GPS and accelerometer data before and after light rail construction in a neighborhood of Salt Lake City, Utah, USA. Health Place. 2015 Nov;36:8-17. doi: 10.1016/j.healthplace.2015.08.005.

45. MacDonald JM, Stokes RJ, Cohen DA, et al. The effect of light rail transit on body mass index and physical activity. Am J Prev Med. 2010 Aug;39(2):105-12. doi: 10.1016/j.amepre.2010.03.016. PMID: 20621257.

46. Bauhoff S. The effect of school district nutrition policies on dietary intake and overweight: a synthetic control approach. Econ Hum Biol. 2014 Jan;12:45-55. doi: 10.1016/j.ehb.2013.06.001. PMID: 23891422.

47. Restrepo BJ. Calorie Labeling in Chain Restaurants and Body Weight: Evidence from New York. Health Econ. 2016 Jul 24doi: 10.1002/hec.3389. PMID: 27451966.

48. Sturm R, Hattori A. Diet and obesity in Los Angeles County 2007-2012: Is there a measurable effect of the 2008 "Fast-Food Ban"? Soc Sci Med. 2015 May;133:205-11. doi: 10.1016/j.socscimed.2015.03.004. PMID: 25779774.

49. Howlett E, Davis C, Burton S. From food desert to food oasis: The potential influence of food retailers on childhood obesity rates. Journal of Business Ethics. 2016;139(2):215-24. doi: 10.1007/s10551-015-2605-5. PMID: 2015-12722-001.

50. Vadiveloo MK, Dixon LB, Elbel B. Consumer purchasing patterns in response to calorie labeling legislation in New York City. Int J Behav Nutr Phys Act. 2011 May 27;8:51. doi: 10.1186/1479-5868-8-51. PMID: 21619632.

51. Lachapelle U, Frank LD. Transit and health: mode of transport, employer-sponsored public transit pass programs, and physical activity. J Public Health Policy. 2009;30 Suppl 1:S73-94. doi: 10.1057/jphp.2008.52. PMID: 19190584.

52. Fu Y, Gao Z, Hannon JC, et al. Effect of the SPARK Program on Physical Activity, Cardiorespiratory Endurance, and Motivation in Middle-School Students. J Phys Act Health. 2016 May;13(5):534-42. doi: 10.1123/jpah.2015-0351. PMID: 26528889.

53. Coffield JE, Metos JM, Utz RL, et al. A multivariate analysis of federally mandated school wellness policies on adolescent obesity. J Adolesc Health. 2011 Oct;49(4):363-70. doi: 10.1016/j.jadohealth.2011.01.010. PMID: 21939866.

54. Anderson LM, Aycock KE, Mihalic CA, et al. Geographic differences in physical education and adolescent BMI: have legal mandates made a difference? J Sch Nurs. 2013 Feb;29(1):52-60. doi: 10.1177/1059840512453602. PMID: 22815346.

Page 359: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-8

55. Benjamin Neelon SE, Namenek Brouwer RJ, Ostbye T, et al. A community-based intervention increases physical activity and reduces obesity in school-age children in North Carolina. Child Obes. 2015 Jun;11(3):297-303. doi: 10.1089/chi.2014.0130. PMID: 25938983.

56. Wendel ML, Benden ME, Zhao H, et al. Stand-Biased Versus Seated Classrooms and Childhood Obesity: A Randomized Experiment in Texas. Am J Public Health. 2016 Oct;106(10):1849-54. doi: 10.2105/ajph.2016.303323. PMID: 27552276.

57. Esquivel M, Nigg CR, Fialkowski MK, et al. Head Start Wellness Policy Intervention in Hawaii: A Project of the Children's Healthy Living Program. Child Obes. 2016 Feb;12(1):26-32. doi: 10.1089/chi.2015.0071. PMID: 26771119.

58. Anderson JV, Bybee DI, Brown RM, et al. 5 a day fruit and vegetable intervention improves consumption in a low income population. J Am Diet Assoc. 2001 Feb;101(2):195-202. doi: 10.1016/s0002-8223(01)00052-9. PMID: 11271692.

59. Taber DR, Chriqui JF, Perna FM, et al. Association between state physical education (PE) requirements and PE participation, physical activity, and body mass index change. Prev Med. 2013 Nov;57(5):629-33. doi: 10.1016/j.ypmed.2013.08.018. PMID: 23978523.

60. Caldwell EM, Miller Kobayashi M, DuBow WM, et al. Perceived access to fruits and vegetables associated with increased consumption. Public Health Nutr. 2009 Oct;12(10):1743-50. doi: 10.1017/s1368980008004308. PMID: 19105861.

61. Schwartz AE, Leardo M, Aneja S, et al. Effect of a School-Based Water Intervention on Child Body Mass Index and Obesity. JAMA Pediatr. 2016 Mar;170(3):220-6. doi: 10.1001/jamapediatrics.2015.3778. PMID: 26784336.

62. Alaimo K, Oleksyk SC, Drzal NB, et al. Effects of changes in lunch-time competitive foods, nutrition practices, and nutrition policies on low-income middle-school children's diets. Child Obes. 2013 Dec;9(6):509-23. doi: 10.1089/chi.2013.0052. PMID: 24215386.

63. Sabia JJ, Nguyen TT, Rosenberg O. High School Physical Education Requirements and Youth Body Weight: New Evidence from the YRBS. Health Econ. 2016 Aug 30doi: 10.1002/hec.3399. PMID: 27576770.

64. Kern E, Chan NL, Fleming DW, et al. Declines in student obesity prevalence associated with a prevention initiative - King County, Washington, 2012. MMWR Morb Mortal Wkly Rep. 2014 Feb 21;63(7):155-7. PMID: 24553199.

65. Sallis JF, McKenzie TL, Conway TL, et al. Environmental interventions for eating and physical activity: a randomized controlled trial in middle schools. Am J Prev Med. 2003 Apr;24(3):209-17. PMID: 12657338.

66. Taber DR, Chriqui JF, Chaloupka FJ. Differences in nutrient intake associated with state laws regarding fat, sugar, and caloric content of competitive foods. Arch Pediatr Adolesc Med. 2012 May;166(5):452-8. doi: 10.1001/archpediatrics.2011.1839. PMID: 22566546.

67. Crespo NC, Elder JP, Ayala GX, et al. Results of a multi-level intervention to prevent and control childhood obesity among Latino children: the Aventuras Para Ninos Study. Ann Behav Med. 2012 Feb;43(1):84-100. doi: 10.1007/s12160-011-9332-7. PMID: 22215470.

68. Coyle KK, Potter S, Schneider D, et al. Distributing free fresh fruit and vegetables at school: results of a pilot outcome evaluation. Public Health Rep. 2009 Sep-Oct;124(5):660-9. doi: 10.1177/003335490912400508. PMID: 19753944.

69. Sturm R, Powell LM, Chriqui JF, et al. Soda Taxes, Soft Drink Consumption, And Children's Body Mass Index. Health Affairs. 2010 May-Jun;29(5):1052-8. doi: 10.1377/hlthaff.2009.0061.

70. Lv J, Liu QM, Ren YJ, et al. A community-based multilevel intervention for smoking, physical activity and diet: short-term findings from the Community Interventions for Health programme in Hangzhou, China. J Epidemiol Community Health. 2014 Apr;68(4):333-9. doi: 10.1136/jech-2013-203356. PMID: 24297972.

71. Mumford KG, Contant CK, Weissman J, et al. Changes in physical activity and travel behaviors in residents of a mixed-use development. Am J Prev Med. 2011 Nov;41(5):504-7. doi: 10.1016/j.amepre.2011.07.016. PMID: 22011422.

72. Powell LM, Chriqui J, Chaloupka FJ. Associations between state-level soda taxes and adolescent body mass index. J Adolesc Health. 2009 Sep;45(3 Suppl):S57-63. doi: 10.1016/j.jadohealth.2009.03.003. PMID: 19699437.

73. Vasquez A, Sherwood NE, Larson N, et al. A novel dietary improvement strategy: examining the potential impact of community-supported agriculture membership. Public Health Nutr. 2016 Oct;19(14):2618-28. doi: 10.1017/s1368980015003638. PMID: 26857484.

Page 360: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-9

74. Brown BB, Smith KR, Tharp D, et al. A Complete Street Intervention for Walking to Transit, Nontransit Walking, and Bicycling: A Quasi-Experimental Demonstration of Increased Use. J Phys Act Health. 2016 Nov;13(11):1210-9. doi: 10.1123/jpah.2016-0066. PMID: 27334024.

75. Brown BB, Werner CM, Tribby CP, et al. Transit Use, Physical Activity, and Body Mass Index Changes: Objective Measures Associated With Complete Street Light-Rail Construction. Am J Public Health. 2015 Jul;105(7):1468-74. doi: 10.2105/ajph.2015.302561. PMID: 25973829.

76. Lemon SC, Zapka J, Li W, et al. Step ahead a worksite obesity prevention trial among hospital employees. Am J Prev Med. 2010 Jan;38(1):27-38. doi: 10.1016/j.amepre.2009.08.028. PMID: 20117554.

77. Wright K, Norris K, Newman Giger J, et al. Improving healthy dietary behaviors, nutrition knowledge, and self-efficacy among underserved school children with parent and community involvement. Child Obes. 2012 Aug;8(4):347-56. doi: 10.1089/chi.2012.0045. PMID: 22867074.

78. Zhu X, Yu CY, Lee C, et al. A retrospective study on changes in residents' physical activities, social interactions, and neighborhood cohesion after moving to a walkable community. Prev Med. 2014 Dec;69 Suppl 1:S93-7. doi: 10.1016/j.ypmed.2014.08.013. PMID: 25158208.

79. Nguyen BT, Powell LM. Supplemental nutrition assistance program participation and sugar-sweetened beverage consumption, overall and by source. Prev Med. 2015 Dec;81:82-6. doi: 10.1016/j.ypmed.2015.08.003. PMID: 26303370.

80. Gibson D. Long-term Food Stamp Program participation is positively related to simultaneous overweight in young daughters and obesity in mothers. J Nutr. 2006 Apr;136(4):1081-5. PMID: 16549483.

81. Molitor F, Sugerman S, Yu H, et al. Reach of Supplemental Nutrition Assistance Program-Education (SNAP-Ed) interventions and nutrition and physical activity-related outcomes, California, 2011-2012. Prev Chronic Dis. 2015 Mar 12;12:E33. doi: 10.5888/pcd12.140449. PMID: 25764139.

82. Girard O, Brocherie F, Morin JB, et al. Intrasession and Intersession Reliability of Running Mechanics During Treadmill Sprints. Int J Sports Physiol Perform. 2016 May;11(4):432-9. doi: 10.1123/ijspp.2015-0145. PMID: 26356384.

83. Leung CW, Blumenthal SJ, Hoffnagle EE, et al. Associations of food stamp participation with dietary quality and obesity in children. Pediatrics. 2013 Mar;131(3):463-72. doi: 10.1542/peds.2012-0889. PMID: 23439902.

84. Datar A, Nicosia N. The effect of state competitive food and beverage regulations on childhood overweight and obesity. Journal of Adolescent Health. 2016doi: 10.1016/j.jadohealth.2016.09.003. PMID: 2016-54655-001.

85. Brownson RC, Hagood L, Lovegreen SL, et al. A multilevel ecological approach to promoting walking in rural communities. Prev Med. 2005 Nov-Dec;41(5-6):837-42. doi: 10.1016/j.ypmed.2005.09.004. PMID: 16256183.

86. von Hippel PT, Bradbury WK. The effects of school physical education grants on obesity, fitness, and academic achievement. Prev Med. 2015 Sep;78:44-51. doi: 10.1016/j.ypmed.2015.06.011. PMID: 26163396.

87. Brown BB, Werner CM, Smith KR, et al. Environmental, behavioral, and psychological predictors of transit ridership: Evidence from a community intervention. J Environ Psychol. 2016 Jun;46:188-96. doi: 10.1016/j.jenvp.2016.04.010. PMID: 27672237.

88. Finch M, Wolfenden L, Morgan PJ, et al. A cluster randomized trial of a multi-level intervention, delivered by service staff, to increase physical activity of children attending center-based childcare. Prev Med. 2014 Jan;58:9-16. doi: 10.1016/j.ypmed.2013.10.004. PMID: 24145204.

89. Cullen KW, Watson K, Zakeri I. Improvements in middle school student dietary intake after implementation of the Texas Public School Nutrition Policy. Am J Public Health. 2008 Jan;98(1):111-7. doi: 10.2105/ajph.2007.111765. PMID: 18048778.

90. Brusseau TA, Hannon J, Burns R. The Effect of a Comprehensive School Physical Activity Program on Physical Activity and Health-Related Fitness in Children From Low-Income Families. J Phys Act Health. 2016 Aug;13(8):888-94. doi: 10.1123/jpah.2016-0028. PMID: 27144329.

Page 361: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-10

91. Hennessy E, Oh A, Agurs-Collins T, et al. State-level school competitive food and beverage laws are associated with children's weight status. J Sch Health. 2014 Sep;84(9):609-16. doi: 10.1111/josh.12181. PMID: 25117896.

92. Barroso CS, Kelder SH, Springer AE, et al. Senate Bill 42: implementation and impact on physical activity in middle schools. J Adolesc Health. 2009 Sep;45(3 Suppl):S82-90. doi: 10.1016/j.jadohealth.2009.06.017. PMID: 19699442.

93. Ling J, King KM, Speck BJ, et al. Preliminary assessment of a school-based healthy lifestyle intervention among rural elementary school children. J Sch Health. 2014 Apr;84(4):247-55. doi: 10.1111/josh.12143. PMID: 24617908.

94. Hardy LL, King L, Kelly B, et al. Munch and Move: evaluation of a preschool healthy eating and movement skill program. Int J Behav Nutr Phys Act. 2010 Nov 03;7:80. doi: 10.1186/1479-5868-7-80. PMID: 21047434.

95. LaRowe TL, Tomayko EJ, Meinen AM, et al. Active Early: one-year policy intervention to increase physical activity among early care and education programs in Wisconsin. BMC Public Health. 2016 Jul 20;16:607. doi: 10.1186/s12889-016-3198-3. PMID: 27439770.

96. Maddock J, Takeuchi L, Nett B, et al. Evaluation of a statewide program to reduce chronic disease: The Healthy Hawaii Initiative, 2000-2004. Eval Program Plann. 2006;29(3):293-300. doi: 10.1016/j.evalprogplan.2005.12.007. PMID: 2006-11996-009.

97. Fox MK, Dodd AH, Wilson A, et al. Association between school food environment and practices and body mass index of US public school children. J Am Diet Assoc. 2009 Feb;109(2 Suppl):S108-17. doi: 10.1016/j.jada.2008.10.065. PMID: 19166665.

98. Schanzenbach D. Do School Lunches Contribute to Childhood Obesity? : Harris School of Public Policy Studies, University of Chicago, Working Papers: 0513; 2005.

99. Kamada M, Kitayuguchi J, Inoue S, et al. A community-wide campaign to promote physical activity in middle-aged and elderly people: a cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2013 Apr 09;10:44. doi: 10.1186/1479-5868-10-44. PMID: 23570536.

100. Robles B, Montes CE, Nobari TZ, et al. Dietary Behaviors among Public Health Center Clients with Electronic Benefit Transfer Access at Farmers' Markets. J Acad Nutr Diet. 2017 Jan;117(1):58-68. doi: 10.1016/j.jand.2016.07.012. PMID: 27618576.

101. Gittelsohn J, Kim EM, He S, et al. A food store-based environmental intervention is associated with reduced BMI and improved psychosocial factors and food-related behaviors on the Navajo nation. J Nutr. 2013 Sep;143(9):1494-500. doi: 10.3945/jn.112.165266. PMID: 23864511.

102. Jago R, McMurray RG, Drews KL, et al. HEALTHY intervention: fitness, physical activity, and metabolic syndrome results. Med Sci Sports Exerc. 2011 Aug;43(8):1513-22. doi: 10.1249/MSS.0b013e31820c9797. PMID: 21233778.

103. Shive SE, Morris MN. Evaluation of the energize your life! Social marketing campaign pilot study to increase fruit intake among community college students. J Am Coll Health. 2006 Jul-Aug;55(1):33-9. doi: 10.3200/jach.55.1.33-40. PMID: 16889313.

104. Brown AL, Khattak AJ, Rodriguez DA. Neighbourhood Types, Travel and Body Mass: A Study of New Urbanist and Suburban Neighbourhoods in the US. Urban Stud. 2008;45(4):963-88.

105. Dzewaltowski DA, Estabrooks PA, Welk G, et al. Healthy youth places: a randomized controlled trial to determine the effectiveness of facilitating adult and youth leaders to promote physical activity and fruit and vegetable consumption in middle schools. Health Educ Behav. 2009 Jun;36(3):583-600. doi: 10.1177/1090198108314619. PMID: 18469366.

106. Williamson DA, Champagne CM, Harsha DW, et al. Effect of an environmental school-based obesity prevention program on changes in body fat and body weight: a randomized trial. Obesity (Silver Spring). 2012 Aug;20(8):1653-61. doi: 10.1038/oby.2012.60. PMID: 22402733.

107. Elbel B, Moran A, Dixon LB, et al. Assessment of a government-subsidized supermarket in a high-need area on household food availability and children's dietary intakes. Public Health Nutr. 2015 Oct;18(15):2881-90. doi: 10.1017/s1368980015000282. PMID: 25714993.

108. Tarp J, Domazet SL, Froberg K, et al. Effectiveness of a School-Based Physical Activity Intervention on Cognitive Performance in Danish Adolescents: LCoMotion-Learning, Cognition and Motion - A Cluster Randomized Controlled Trial. PLoS One. 2016;11(6):e0158087. doi: 10.1371/journal.pone.0158087. PMID: 27341346.

109. Falbe J, Thompson HR, Becker CM, et al. Impact of the Berkeley Excise Tax on Sugar-Sweetened Beverage Consumption. Am J Public Health. 2016 Oct;106(10):1865-71. doi: 10.2105/ajph.2016.303362. PMID: 27552267.

Page 362: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-11

110. Whetstone LM, Kolasa KM, Collier DN. Participation in community-originated interventions is associated with positive changes in weight status and health behaviors in youth. Am J Health Promot. 2012 Sep-Oct;27(1):10-6. doi: 10.4278/ajhp.100415-QUAN-117. PMID: 22950920.

111. Harding MC, Bott QD, Jonas CE. The Malaekahana Path: An Ecological Model-based Intervention for Increasing Walking in Rural Hawai'i. J Phys Act Health. 2017 Jul 06:1-11. doi: 10.1123/jpah.2017-0330. PMID: 28682662.

112. Cohen DA, Marsh T, Williamson S, et al. Impact and cost-effectiveness of family Fitness Zones: a natural experiment in urban public parks. Health Place. 2012 Jan;18(1):39-45. doi: 10.1016/j.healthplace.2011.09.008. PMID: 22243905.

113. Farley TA, Meriwether RA, Baker ET, et al. Safe play spaces to promote physical activity in inner-city children: results from a pilot study of an environmental intervention. Am J Public Health. 2007 Sep;97(9):1625-31. doi: 10.2105/ajph.2006.092692. PMID: 17666701.

114. Toussaint LL, Housholder K, Janssen K, et al. Slowing BMI Growth Trajectories in Elementary School-Aged Children: The Northeast Iowa Food and Fitness Initiative. Fam Community Health. 2017 Jul/Sep;40(3):192-7. doi: 10.1097/fch.0000000000000151. PMID: 28525438.

115. Morrill BA, Madden GJ, Wengreen HJ, et al. A Randomized Controlled Trial of the Food Dudes Program: Tangible Rewards are More Effective Than Social Rewards for Increasing Short- and Long-Term Fruit and Vegetable Consumption. J Acad Nutr Diet. 2016 Apr;116(4):618-29. doi: 10.1016/j.jand.2015.07.001. PMID: 26297598.

116. Weaver RG, Webster CA, Egan C, et al. Partnerships for Active Children in Elementary Schools: Outcomes of a 2-Year Pilot Study to Increase Physical Activity During the School Day. Am J Health Promot. 2017 Jan 01:890117117707289. doi: 10.1177/0890117117707289. PMID: 28482678.

117. Nanney MS, MacLehose R, Kubik MY, et al. Recommended school policies are associated with student sugary drink and fruit and vegetable intake. Prev Med. 2014 May;62:179-81. doi: 10.1016/j.ypmed.2014.01.026. PMID: 24518003.

118. Backman D, Gonzaga G, Sugerman S, et al. Effect of fresh fruit availability at worksites on the fruit and vegetable consumption of low-wage employees. J Nutr Educ Behav. 2011 Jul-Aug;43(4 Suppl 2):S113-21. doi: 10.1016/j.jneb.2011.04.003. PMID: 21683280.

119. Caballero B, Clay T, Davis SM, et al. Pathways: a school-based, randomized controlled trial for the prevention of obesity in American Indian schoolchildren. Am J Clin Nutr. 2003 Nov;78(5):1030-8. PMID: 14594792.

120. Hilmers A, Chen TA, Dave JM, et al. Supplemental Nutrition Assistance Program participation did not help low income Hispanic women in Texas meet the dietary guidelines. Prev Med. 2014 May;62:44-8. doi: 10.1016/j.ypmed.2014.01.016. PMID: 24530319.

121. Gorham G, Dulin-Keita A, Risica PM, et al. Effectiveness of Fresh to You, a Discount Fresh Fruit and Vegetable Market in Low-Income Neighborhoods, on Children's Fruit and Vegetable Consumption, Rhode Island, 2010-2011. Prev Chronic Dis. 2015 Oct 15;12:E176. doi: 10.5888/pcd12.140583. PMID: 26469949.

122. Cummins S, Flint E, Matthews SA. New neighborhood grocery store increased awareness of food access but did not alter dietary habits or obesity. Health Aff (Millwood). 2014 Feb;33(2):283-91. doi: 10.1377/hlthaff.2013.0512. PMID: 24493772.

123. Branas CC, Cheney RA, MacDonald JM, et al. A difference-in-differences analysis of health, safety, and greening vacant urban space. Am J Epidemiol. 2011 Dec 01;174(11):1296-306. doi: 10.1093/aje/kwr273. PMID: 22079788.

124. Ayala GX, Baquero B, Laraia BA, et al. Efficacy of a store-based environmental change intervention compared with a delayed treatment control condition on store customers' intake of fruits and vegetables. Public Health Nutr. 2013 Nov;16(11):1953-60. doi: 10.1017/s1368980013000955. PMID: 23561842.

125. Haerens L, Deforche B, Maes L, et al. Body mass effects of a physical activity and healthy food intervention in middle schools. Obesity (Silver Spring). 2006 May;14(5):847-54. doi: 10.1038/oby.2006.98. PMID: 16855194.

126. Heelan KA, Abbey BM, Donnelly JE, et al. Evaluation of a Walking School Bus for Promoting Physical Activity in Youth. J Phys Act Health. 2009 Sep;6(5):560-7. PMID: 19953832.

Page 363: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-12

127. LaCaille LJ, Schultz JF, Goei R, et al. Go!: results from a quasi-experimental obesity prevention trial with hospital employees. BMC Public Health. 2016 Feb 19;16:171. doi: 10.1186/s12889-016-2828-0. PMID: 26893128.

128. Kim J. Are physical education-related state policies and schools' physical education requirement related to children's physical activity and obesity? J Sch Health. 2012 Jun;82(6):268-76. doi: 10.1111/j.1746-1561.2012.00697.x. PMID: 22568462.

129. Reger-Nash B, Bauman A, Booth-Butterfield S, et al. Wheeling walks: evaluation of a media-based community intervention. Fam Community Health. 2005 Jan-Mar;28(1):64-78. PMID: 15625507.

130. Gebel K, Bauman AE, Reger-Nash B, et al. Does the environment moderate the impact of a mass media campaign to promote walking? Am J Health Promot. 2011 Sep-Oct;26(1):45-8. doi: 10.4278/ajhp.081104-ARB-269. PMID: 21879942.

131. Jordan KC, Erickson ED, Cox R, et al. Evaluation of the Gold Medal Schools program. J Am Diet Assoc. 2008 Nov;108(11):1916-20. doi: 10.1016/j.jada.2008.08.002. PMID: 18954584.

132. Quigg R, Reeder AI, Gray A, et al. The Effectiveness of a Community Playground Intervention. Journal of Urban Health-Bulletin of the New York Academy of Medicine. 2012 Feb;89(1):171-84. doi: 10.1007/s11524-011-9622-1.

133. Tester JM, Leung CW, Crawford PB. Revised WIC Food Package and Children's Diet Quality. Pediatrics. 2016 May;137(5)doi: 10.1542/peds.2015-3557. PMID: 27244804.

134. Sekhobo JP, Edmunds LS, Dalenius K, et al. Neighborhood disparities in prevalence of childhood obesity among low-income children before and after implementation of New York City child care regulations. Prev Chronic Dis. 2014;11:E181. doi: 10.5888/pcd11.140152. PMID: 25321632.

135. Nicklas T, Lopez S, Liu Y, et al. Motivational theater to increase consumption of vegetable dishes by preschool children. Int J Behav Nutr Phys Act. 2017 Feb 07;14(1):16. doi: 10.1186/s12966-017-0468-0. PMID: 28166788.

136. Taber DR, Stevens J, Evenson KR, et al. State policies targeting junk food in schools: racial/ethnic differences in the effect of policy change on soda consumption. Am J Public Health. 2011 Sep;101(9):1769-75. doi: 10.2105/ajph.2011.300221. PMID: 21778484.

137. Liu J, Kuo T, Jiang L, et al. Food and drink consumption among 1-5-year-old Los Angeles County children from households receiving dual SNAP and WIC v. only WIC benefits. Public Health Nutr. 2016 Sep 9:1-8. doi: 10.1017/s1368980016002329. PMID: 27609603.

138. Olsho LE, Payne GH, Walker DK, et al. Impacts of a farmers' market incentive programme on fruit and vegetable access, purchase and consumption. Public Health Nutr. 2015 Oct;18(15):2712-21. doi: 10.1017/s1368980015001056. PMID: 25919225.

139. Te Velde SJ, Brug J, Wind M, et al. Effects of a comprehensive fruit- and vegetable-promoting school-based intervention in three European countries: the Pro Children Study. Br J Nutr. 2008 Apr;99(4):893-903. doi: 10.1017/s000711450782513x. PMID: 17953787.

140. Nehme EK, Perez A, Ranjit N, et al. The Effect of New Shower Facilities on Physical Activity Behaviors of Employees: A Quasi-experiment. J Phys Act Health. 2017 Feb;14(2):98-107. doi: 10.1123/jpah.2015-0418. PMID: 27775466.

141. Barnidge EK, Hipp PR, Estlund A, et al. Association between community garden participation and fruit and vegetable consumption in rural Missouri. Int J Behav Nutr Phys Act. 2013 Nov 19;10:128. doi: 10.1186/1479-5868-10-128. PMID: 24252563.

142. Pope M. Preventing Weight Gain in Children Who Are School Age and African-American. Pediatr Phys Ther. 2016 Summer;28(2):207-16. doi: 10.1097/pep.0000000000000243. PMID: 26914717.

143. Cawley J, Meyerhoefer C, Newhouse D. The impact of state physical education requirements on youth physical activity and overweight. Health Econ. 2007;16(12):1287-301. doi: 10.1002/hec.1218. PMID: 2007-19974-001.

144. Cawley J, Meyerhoefer C, Newhouse D. The Correlation of Youth Physical Activity with State Policies. Contemporary Economic Policy. 2007;25(4):506-17. doi: http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291465-7287. PMID: 0952344 Alternate Accession Number: EP27555031.

145. Masse LC, de Niet-Fitzgerald JE, Watts AW, et al. Associations between the school food environment, student consumption and body mass index of Canadian adolescents. Int J Behav Nutr Phys Act. 2014 Mar 26;11(1):29. doi: 10.1186/1479-5868-11-29. PMID: 24666770.

Page 364: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-13

146. Eagle TF, Gurm R, Smith CA, et al. A middle school intervention to improve health behaviors and reduce cardiac risk factors. Am J Med. 2013 Oct;126(10):903-8. doi: 10.1016/j.amjmed.2013.04.019. PMID: 23932159.

147. Peterson KE, Spadano-Gasbarro JL, Greaney ML, et al. Three-Year Improvements in Weight Status and Weight-Related Behaviors in Middle School Students: The Healthy Choices Study. PLoS One. 2015;10(8):e0134470. doi: 10.1371/journal.pone.0134470. PMID: 26295837.

148. Ortega AN, Albert SL, Chan-Golston AM, et al. Substantial improvements not seen in health behaviors following corner store conversions in two Latino food swamps. BMC Public Health. 2016 May 11;16:389. doi: 10.1186/s12889-016-3074-1. PMID: 27169514.

149. Frongillo EA, Fawcett SB, Ritchie LD, et al. Community Policies and Programs to Prevent Obesity and Child Adiposity. Am J Prev Med. 2017 Jul 05doi: 10.1016/j.amepre.2017.05.006. PMID: 28688728.

150. Goldsby TU, George BJ, Yeager VA, et al. Urban Park Development and Pediatric Obesity Rates: A Quasi-Experiment Using Electronic Health Record Data. Int J Environ Res Public Health. 2016 Apr 08;13(4):411. doi: 10.3390/ijerph13040411. PMID: 27070635.

151. Baker EA, Barnidge EK, Schootman M, et al. Adaptation of a Modified DASH Diet to a Rural African American Community Setting. Am J Prev Med. 2016 Sep 12doi: 10.1016/j.amepre.2016.07.014. PMID: 27633485.

152. Huberty J, Beets M, Beighle A. Effects of a policy-level intervention on children's pedometer-determined physical activity: preliminary findings from Movin' Afterschool. J Public Health Manag Pract. 2013 Nov-Dec;19(6):525-8. doi: 10.1097/PHH.0b013e31829465fa. PMID: 23676476.

153. Pate RR, Ward DS, Saunders RP, et al. Promotion of physical activity among high-school girls: a randomized controlled trial. Am J Public Health. 2005 Sep;95(9):1582-7. doi: 10.2105/ajph.2004.045807. PMID: 16118370.

154. Pbert L, Druker S, Barton B, et al. A School-Based Program for Overweight and Obese Adolescents: A Randomized Controlled Trial. J Sch Health. 2016 Oct;86(10):699-708. doi: 10.1111/josh.12428. PMID: 27619760.

155. Cheadle A, Rauzon S, Spring R, et al. Kaiser Permanente's Community Health Initiative in Northern California: evaluation findings and lessons learned. Am J Health Promot. 2012 Nov-Dec;27(2):e59-68. doi: 10.4278/ajhp.111222-QUAN-462. PMID: 23113787.

156. Madsen KA, Cotterman C, Crawford P, et al. Effect of the healthy schools program on prevalence of overweight and obesity in California schools, 2006–2012. Preventing Chronic Disease: Public Health Research, Practice, and Policy. 2015;12 PMID: 2016-24674-001.

157. de Visser R, Sylvester R, Rogers R, et al. Changes in School Health Program Improve Middle School Students' Behaviors. Am J Health Behav. 2016 Sep;40(5):568-77. doi: 10.5993/ajhb.40.5.3. PMID: 27561859.

158. Wright K, Giger JN, Norris K, et al. Impact of a nurse-directed, coordinated school health program to enhance physical activity behaviors and reduce body mass index among minority children: a parallel-group, randomized control trial. Int J Nurs Stud. 2013 Jun;50(6):727-37. doi: 10.1016/j.ijnurstu.2012.09.004. PMID: 23021318.

159. Berger-Jenkins E, Rausch J, Okah E, et al. Evaluation of a Coordinated School-Based Obesity Prevention Program in a Hispanic Community: Choosing Healthy and Active Lifestyles for Kids/Healthy Schools Healthy Families. American Journal of Health Education. 2014;45(5):261-70. doi: 10.1080/19325037.2014.932724. PMID: 103883129. Language: English. Entry Date: 20140909. Revision Date: 20150820. Publication Type: Journal Article.

160. Hollis JL, Sutherland R, Campbell L, et al. Effects of a 'school-based' physical activity intervention on adiposity in adolescents from economically disadvantaged communities: secondary outcomes of the 'Physical Activity 4 Everyone' RCT. Int J Obes (Lond). 2016 Oct;40(10):1486-93. doi: 10.1038/ijo.2016.107. PMID: 27430652.

161. Herrick H, Thompson H, Kinder J, et al. Use of SPARK to promote after-school physical activity. J Sch Health. 2012 Oct;82(10):457-61. doi: 10.1111/j.1746-1561.2012.00722.x. PMID: 22954164.

162. Wilson DK, Van Horn ML, Siceloff ER, et al. The Results of the "Positive Action for Today's Health" (PATH) Trial for Increasing Walking and Physical Activity in Underserved African-American Communities. Ann Behav Med. 2015 Jun;49(3):398-410. doi: 10.1007/s12160-014-9664-1. PMID: 25385203.

Page 365: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-14

163. Dubowitz T, Ghosh-Dastidar M, Cohen DA, et al. Diet And Perceptions Change With Supermarket Introduction In A Food Desert, But Not Because Of Supermarket Use. Health Aff (Millwood). 2015 Nov;34(11):1858-68. doi: 10.1377/hlthaff.2015.0667. PMID: 26526243.

164. Coleman KJ, Shordon M, Caparosa SL, et al. The healthy options for nutrition environments in schools (Healthy ONES) group randomized trial: using implementation models to change nutrition policy and environments in low income schools. Int J Behav Nutr Phys Act. 2012 Jun 27;9:80. doi: 10.1186/1479-5868-9-80. PMID: 22734945.

165. Neumark-Sztainer DR, Friend SE, Flattum CF, et al. New moves-preventing weight-related problems in adolescent girls a group-randomized study. Am J Prev Med. 2010 Nov;39(5):421-32. doi: 10.1016/j.amepre.2010.07.017. PMID: 20965379.

166. Bastian KA, Maximova K, McGavock J, et al. Does School-Based Health Promotion Affect Physical Activity on Weekends? And, Does It Reach Those Students Most in Need of Health Promotion? PLoS One. 2015;10(10):e0137987. doi: 10.1371/journal.pone.0137987. PMID: 26488168.

167. Linde JA, Nygaard KE, MacLehose RF, et al. HealthWorks: results of a multi-component group-randomized worksite environmental intervention trial for weight gain prevention. Int J Behav Nutr Phys Act. 2012 Feb 16;9:14. doi: 10.1186/1479-5868-9-14. PMID: 22340088.

168. Simon C, Schweitzer B, Oujaa M, et al. Successful overweight prevention in adolescents by increasing physical activity: a 4-year randomized controlled intervention. Int J Obes (Lond). 2008 Oct;32(10):1489-98. doi: 10.1038/ijo.2008.99. PMID: 18626482.

169. Madsen K, Linchey J, Gerstein D, et al. Energy Balance 4 Kids with Play: Results from a Two-Year Cluster-Randomized Trial. Child Obes. 2015 Aug;11(4):375-83. doi: 10.1089/chi.2015.0002. PMID: 26061799.

170. Gustat J, Rice J, Parker KM, et al. Effect of changes to the neighborhood built environment on physical activity in a low-income African American neighborhood. Prev Chronic Dis. 2012;9:E57. PMID: 22338597.

171. Madsen K, Thompson H, Adkins A, et al. School-community partnerships: a cluster-randomized trial of an after-school soccer program. JAMA Pediatr. 2013 Apr;167(4):321-6. doi: 10.1001/jamapediatrics.2013.1071. PMID: 23440308.

172. Lent MR, Vander Veur SS, McCoy TA, et al. A randomized controlled study of a healthy corner store initiative on the purchases of urban, low-income youth. Obesity (Silver Spring). 2014 Dec;22(12):2494-500. doi: 10.1002/oby.20878. PMID: 25311881.

173. de Silva-Sanigorski A, Elea D, Bell C, et al. Obesity prevention in the family day care setting: impact of the Romp & Chomp intervention on opportunities for children's physical activity and healthy eating. Child Care Health Dev. 2011 May;37(3):385-93. doi: 10.1111/j.1365-2214.2010.01205.x. PMID: 21276039.

174. Perry CL, Bishop DB, Taylor GL, et al. A randomized school trial of environmental strategies to encourage fruit and vegetable consumption among children. Health Educ Behav. 2004 Feb;31(1):65-76. PMID: 14768658.

175. Dunton G, Ebin VJ, Efrat MW, et al. The Use of Refundable Tax Credits to Increase Low-Income Children's After-School Physical Activity Level. J Phys Act Health. 2015 Jun;12(6):840-53. doi: 10.1123/jpah.2014-0058. PMID: 25184738.

176. Gittelsohn J, Vijayadeva V, Davison N, et al. A food store intervention trial improves caregiver psychosocial factors and children's dietary intake in Hawaii. Obesity (Silver Spring). 2010 Feb;18 Suppl 1:S84-90. doi: 10.1038/oby.2009.436. PMID: 20107467.

177. Cohen JF, Kraak VI, Choumenkovitch SF, et al. The CHANGE study: a healthy-lifestyles intervention to improve rural children's diet quality. J Acad Nutr Diet. 2014 Jan;114(1):48-53. doi: 10.1016/j.jand.2013.08.014. PMID: 24126295.

178. Burke RM, Meyer A, Kay C, et al. A holistic school-based intervention for improving health-related knowledge, body composition, and fitness in elementary school students: an evaluation of the HealthMPowers program. Int J Behav Nutr Phys Act. 2014 Jun 26;11:78. doi: 10.1186/1479-5868-11-78. PMID: 24969618.

179. Chomitz VR, McGowan RJ, Wendel JM, et al. Healthy Living Cambridge Kids: a community-based participatory effort to promote healthy weight and fitness. Obesity (Silver Spring). 2010 Feb;18 Suppl 1:S45-53. doi: 10.1038/oby.2009.431. PMID: 20107461.

180. Beresford SA, Thompson B, Bishop S, et al. Long-term fruit and vegetable change in worksites: Seattle 5 a Day follow-up. Am J Health Behav. 2010 Nov-Dec;34(6):707-20. PMID: 20604696.

Page 366: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

I-15

181. Lemon SC, Wang ML, Wedick NM, et al. Weight gain prevention in the school worksite setting: results of a multi-level cluster randomized trial. Prev Med. 2014 Mar;60:41-7. doi: 10.1016/j.ypmed.2013.12.010. PMID: 24345602.

182. Economos CD, Hyatt RR, Goldberg JP, et al. A community intervention reduces BMI z-score in children: Shape Up Somerville first year results. Obesity (Silver Spring). 2007 May;15(5):1325-36. doi: 10.1038/oby.2007.155. PMID: 17495210.

183. Goetzel RZ, Roemer EC, Pei X, et al. Second-year results of an obesity prevention program at the Dow Chemical Company. J Occup Environ Med. 2010 Mar;52(3):291-302. doi: 10.1097/JOM.0b013e3181d46f0b. PMID: 20190646.

184. Rogers VW, Hart PH, Motyka E, et al. Impact of Let's Go! 5-2-1-0: a community-based, multisetting childhood obesity prevention program. J Pediatr Psychol. 2013 Oct;38(9):1010-20. doi: 10.1093/jpepsy/jst057. PMID: 23933841.

185. Jurg ME, Kremers SP, Candel MJ, et al. A controlled trial of a school-based environmental intervention to improve physical activity in Dutch children: JUMP-in, kids in motion. Health Promot Int. 2006 Dec;21(4):320-30. doi: 10.1093/heapro/dal032. PMID: 16963784.

186. Reger-Nash B, Bauman A, Cooper L, et al. WV Walks: replication with expanded reach. J Phys Act Health. 2008 Jan;5(1):19-27. PMID: 18209251.

Page 367: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-1

Appendix J. Data Systems Identified by the Systematic Review* Table J1. Data systems identified by the systematic review

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Natural Experiment Studies

Bauhoff, 20141 USA N Y California Healthy Kids Survey (CHKS)

WestEd http://chks.wested.org/reports/

http://chks.wested.org/administer/download/

- -

Bauhoff, 20141 USA N Y California FITNESSGRAM Physical Fitness Test (PFT)

State of California - Department of Education

http://www.cde.ca.gov/ta/tg/pf/pftresearch.asp

https://pftdata.org/resources.aspx

http://www.cde.ca.gov/ta/tg/pf/

http://data1.cde.ca.gov/dataquest/

Branas, 20112 USA N Y PA Property Taxes Philadelphia Bureau of Revision of Taxes

http://publicrecords.onlinesearches.com/Pennsylvania-Assessor-and-Property-Tax-Records.htm

- - -

Branas, 20112 USA N Y Philadelphia City Map Philadelphia Department of Licenses and Inspections

http://gsg.phila.gov/map

- - -

Branas, 20112 USA N Y US Postal Service Records

US Postal Services https://ribbs.usps.gov/index.cfm?page=address_info_systems

https://ribbs.usps.gov/addressing/documents/tech_guides/pubs/AIS_VIEWER_USER.PDF

- -

Branas, 20112 USA N Y Philadelphia Crime Data Philadelphia Police Department

https://www.opendataphilly.org/dataset/crime-incidents

https://www.opendataphilly.org/dataset/crime-incidents/resource/79134de9-56fa-41f2-b529-b660aaf1539b

- https://www.opendataphilly.org/dataset/crime-incidents/resource/c57a9de2-e300-468a-9a20-3e64e5b9b2da

Page 368: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-2

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Branas, 20112 USA N Y Southeastern Pennsylvania Household Health Survey

Philadelphia Health Management Corporation

http://www.chdbdata.org/data-data-tools

- - -

Camacho-Rivera, 20173 USA Y N The Affordable Housing as an Obesity Mediating

NYC-DofCityPlanning

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4074325/

- -

Capogrossi, 20164 USA N Y Common Core of Data (CCD)

National Center for Education Statistics

https://nces.ed.gov/ccd/ccddata.asp

- - -

Capogrossi, 20164 USA N Y Early Childhood Longitudinal Study-Kindergarten (ECLS-K)

National Center for Education and Statistics

https://nces.ed.gov/edat/

https://nces.ed.gov/ecls/instruments2011.asp

https://nces.ed.gov/ecls/dataproducts.asp

-

Chen, 20155

China N Y China Health and Nutrition Survey (CHNS)

Chinese Center for Disease Control and Prevention (CCDC)

- http://www.cpc.unc.edu/projects/china/data/questionnaires

- http://www.cpc.unc.edu/projects/china/data/datasets

Cleland, 20086

Australia N Y Australian Schools Health and Fitness Survey (ASHFS)

Australian Government

- - - -

Coffield, 20117

USA N Y Utah Population Database (UPDB)

Utah State Government

Access with RGE and IRB approval: https://healthcare.utah.edu/huntsmancancerinstitute/research/updb/access.php

- http://healthcare.utah.edu/huntsmancancerinstitute/research/updb/data/

http://healthcare.utah.edu/huntsmancancerinstitute/research/updb/data/

Cradock, 20118

USA N Y Boston Youth Survey City of Boston and the Harvard Youth Violence Prevention Center

https://www.hsph.harvard.edu/wp-content/uploads/sites/120/2012/10/2004BYSfullreport.pdf

https://cdn1.sph.harvard.edu/wp-content/uploads/sites/120/2012/10/Inst_2006_BYS_survey_FINAL.pdf

- https://www.hsph.harvard.edu/wp-content/uploads/sites/120/2012/10/Final_2006_BYS_Highlights_and_tables.pdf

Page 369: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-3

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Cummins, 20059

UK N Y UK CACI Address Files CACI Ltd. http://www.caci.co.uk/integrated-marketing/consumer-data

http://research.caciconsult.co.uk/lifestyle-data/

- -

Cummins, 200810

UK N Y Scottish Health Survey (Health Education Population Survey)

NHS Health Scotland

http://www.healthscotland.com/scotlands-health/population/HEPS.aspx

- - -

Cummins, 201411 USA N Y USDA Food Atlas US Department of Agriculture

https://www.ers.usda.gov

https://www.ers.usda.gov/data-products/

- https://www.ers.usda.gov/data-products/food-environment-atlas/data-access-and-documentation-downloads/

Datar, 201612 USA N Y State Laws for Snack Foods and Beverages

Bridging the Gap http://foods.bridgingthegapresearch.org/#ng12s/2012

http://www.bridgingthegapresearch.org/_asset/44cmz6/btg_state_srts_0511_laws_codebook_14Nov11_puse-1.pdf

http://www.bridgingthegapresearch.org/research/state_obesity-related_policies/

http://www.bridgingthegapresearch.org/_asset/kxq8n0/BTG_SRTS_hazlaw_2005_2011_with_dummyvars_03Nov11.xlsx

Fitzpatrick, 201713 Canada N Y Montreal Epidemiological and Geographical Analysis of Population Health Outcomes and Neighborhood Effects

Université de Montréal

https://www.nature.com/ijo/journal/v41/n7/full/ijo201739a.html

- -

Fitzpatrick, 201713 Canada Y N Quebec Adipose and Lifestyle Investigation in Youth (Quality) Study

QUebec Adipose and Lifestyle InvesTigation in Youth

http://www.etudequalitystudy.ca/

- -

Fitzpatrick, 201713 Canada N Y ZipCom commercial database

Tamec Inc. http://www.zipcom.ca/en/

- -

Page 370: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-4

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Frongillo, 201714 USA Y N Community Policies and Programs

University of South Carolina

https://www.ncbi.nlm.nih.gov/pubmed/28688728

- -

Gee, 201515

USA N Y National Survey of Children's Health

Centers for Disease Control and Prevention

ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/slaits/nsch_2011_2012/03_Dataset

https://www.cdc.gov/nchs/data/slaits/2011nschquestionnaire.pdf

https://www.cdc.gov/nchs/slaits/nsch.htm

-

Giles-Corti, 201316

Australia N Y Western Australian Liveable Neighbourhoods Guideline

Western Australian Department of Planning

https://www.planning.wa.gov.au/Liveable-neighbourhoods.aspx

- https://www.planning.wa.gov.au/dop_pub_pdf/mrsindex.pdf

https://www.planning.wa.gov.au/dop_pub_pdf/LN_Text_update_02.pdf

Hennessy, 201417

USA N Y Classification of Laws Associated with School Students

National Cancer Institute

https://class.cancer.gov/download.aspx

- https://class.cancer.gov/

-

Hobin, 201418

Canada N Y Youth Health Survey Canadian Cosortium - http://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvInstrumentList&Id=282165

- http://www23.statcan.gc.ca/imdb/p3Instr.pl?Function=assembleInstr&lang=en&Item_Id=314505

Hobin, 201719 Canada N Y Manitoba (Youth Health Survey)

Partners in Planning for Healthy Living

http://partners.healthincommon.ca/tools-and-resources/youth-health-survey/

http://partners.healthincommon.ca/wp-content/uploads/2013/01/YHS_A.pdf

- -

Hoelscher, 201620

USA Y N Texas Childhood Obesity Prevention Policy Evaluation

Texas State - - https://sph.uth.edu/research/centers/dell/project.htm?project=1183cfc3-c761-442e-881f-297978d00fe8

https://sph.uth.edu/research/centers/dell/resources/2013-Texas-Health-Perception-Survey_Report1.pdf

Page 371: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-5

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Howlett, 201621 USA N Y County Business Patterns

Census Bureau https://www.census.gov/programs-surveys/cbp.html

https://www2.census.gov/programs-surveys/rhfs/cbp/technical%20documentation/2015_record_layouts/county_layout_2015.txt

- https://www.census.gov/programs-surveys/cbp/data/datasets.html

Howlett, 201621 USA N Y Supplemental Nutrition Assistance Program

Department of Commerce

https://www.fns.usda.gov/pd/snap-state-activity-reports

https://www.fns.usda.gov/snap/supplemental-nutrition-assistance-program-snap

https://www.fns.usda.gov/pd/supplemental-nutrition-assistance-program-snap

Hu, 201622 Finland N Y Health Behavior and Health

National Institute for Health and Welfare

https://www.thl.fi/en/web/thlfi-en/whats-new?p_p_state=maximized&p_p_mode=view&saveLastPath=0&_58_struts_action=%2Flogin%2Flogin&p_p_id=58&p_p_lifecycle=0&_58_redirect=%2Fen%2Fweb%2Fthlfi-en%2Fresearch-and-expertwork%2Fpopulation-studies%2Fhealth-behaviour-and-health-among-the-finnish-adult-population-avtk

- https://www.thl.fi/en/web/thlfi-en/research-and-expertwork/population-studies

-

Page 372: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-6

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Hu, 201622 Italy N Y Multipurpose Family Survey

Italian National Institute of Statistics (ISTAT)

- https://www.istat.it/it/archivio/91926

Health Statics => "Life styles and risk factors"=> Body mass Index : http://dati.istat.it/?lang=en

-

Hu, 201622 Italy N Y Health and Health Care Utilization

Italian National Institute of Statistics (ISTAT)

http://www.istat.it/en/archive/129376

- - http://www.istat.it/en/files/2014/07/pills_10_luglio_en01.pdf?title=Health+conditions+and+use+of+health+services+-+10+Jul+2014+-+Full+text.pdf

Hu, 201622 Netherland N Y Permanent Survey of Living Conditions (POLS)

Netherland Government

http://statline.cbs.nl/Statweb/search/?Q=POLS-Gezondheid+1997-2009&LA=NL

- - http://www.jpi-dataproject.eu/Home/Database/311?topicId=7#

Hu, 201622 Netherland N Y Ongoing Survey of Living Conditions (DLO)

Netherland Government

- - - -

Hu, 201622 UK N Y Health Survey for England

National Health Service

- https://data.gov.uk/dataset/health_survey_for_england

https://discover.ukdataservice.ac.uk/series/?sn=2000021

http://content.digital.nhs.uk/searchcatalogue?q=Health+Survey+for+England&area=&size=10&sort=Relevance#top

Hughes, 201223 UK Y N Child and Diet Evaluation Tool

Medical Research Council - National Prevention Research Institute

http://dapa-toolkit.mrc.ac.uk/

- - -

Hughes, 201223 UK N Y Index of Multiple Deprivation

UK Government https://data.gov.uk/dataset/imd_2004

- - -

Page 373: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-7

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Hughes, 201223 UK N Y Output Area Classification

UK Office for National Statistics

http://www.opengeodemographics.com/

- - -

Page 374: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-8

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Hunter, 201624 Canada Y N Cohort for obesity, marijuana use, physical activity, alcohol use, smoking and sedentary behavior

Uni of Waterloo https://uwaterloo.ca/compass-system/compass-system-projects/compass-study

- -

Jia, 201725 China Y N Childhood Obesity Study in China Megacities

Multi-City Center for Disease Control and Prevention, China

https://www.ncbi.nlm.nih.gov/pubmed/28074059

- -

Just, 201426

USA N Y School Nutrition Dietary Assessment Study-III (SNDA-III)

US Department of Agriculture

- https://fns-prod.azureedge.net/sites/default/files/SNDAIII-Instruments.pdf

https://www.fns.usda.gov/school-nutrition-dietary-assessment-study-iii

https://fns-prod.azureedge.net/sites/default/files/SNDAIII-SummaryofFindings.pdf

Kern, 201427

USA N Y Washington State Healthy Youth Survey (HYS)

Washington State Department of Health

http://www.doh.wa.gov/DataandStatisticalReports/DataSystems/HealthyYouthSurvey/PastSurveys

http://www.doh.wa.gov/DataandStatisticalReports/DataSystems/HealthyYouthSurvey

- -

Kim, 201228

USA N Y Physical Education-Related State Policy Classification System (PERSPCS)

National Cancer Institute (UBC: Centre for Community Child Health Research)

https://www.ncbi.nlm.nih.gov/pubmed/17884575

- - http://www.ihrp.uic.edu/study/nci-physical-education-related-state-policy-classification-system-perspcs-and-school-nutrition

Kubik, 200529

USA Y N Teens Eating for Energy and Nutrition at School

Penn State + Military

- - - -

Page 375: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-9

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Lachapelle, 200930

USA N Y National Household Travel Survey

US Department of Transportation - Federal Highway Administration

https://www.nationalhouseholdtravelsurvey.com/

http://nhts.ornl.gov/2009/pub/Codebook.pdf

http://nhts.ornl.gov/ http://nhts.ornl.gov/download.shtml

Liu, 201631 USA N Y WIC Data (LA-County) Los Angeles County http://www.healthycity.org/maps/

http://www.lawicdata.org/survey/

https://www.fns.usda.gov/pd/wic-program

http://lawicdata.org/topics/obesity-2/

Liu, 201631 USA N Y SNAP Supplemental Los Angeles County http://www.cdss.ca.gov/inforesources/Research-and-Data/CalFresh-Data-Tables

http://dpss.lacounty.gov/wps/portal/dpss/main/programs-and-services/calfresh/!ut/p/b1/04_Sj9Q1NDA1NDcyNrcw1o_Qj8pLLMtMTyzJzM9LzAHxo8zi3QwMDNz9nYKN3H2ATEf_ACdvr7BAAxMzE6CCSGQFBr5hbgaeQYbGvk7u5kYWYYaE9HvpR6Xn5CcBrQrXj0JVjMUssAIDHMDRQN_PIz83VT83KsciO8tEEQADFwf6/dl4/d5/L2dJQSEvUUt3QS80SmtFL1o2X0YwMDBHT0JTMjBFNkEwQU9TSjFVOEsxUzUw/

https://www.fns.usda.gov/pd/supplemental-nutrition-assistance-program-snap

-

MacDonald, 201032 USA N Y U.S. GDC Park Landmarks

ESRI ESRI - - -

MacDonald, 201032 USA N Y Census Tiger Files Census https://www.census.gov/geo/maps-data/data/tiger-line.html

https://www.census.gov/geo/maps-data/data/tiger-line.html

- -

Madsen, 201133 USA N Y CDE DataQuest The California Department of Education (CDE)

- - - http://data1.cde.ca.gov/dataquest/

Page 376: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-10

Page 377: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-11

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Malakellis, 201734 Australia N Y Socio-Economic Index for Areas

ACT-IYM-Australia http://www.health.act.gov.au/healthy-living/healthy-children-and-young-people/its-your-move

http://health.act.gov.au/sites/default/files//Questionnaire%20Planning%20Guide-webpublication%20version_AUG15.pdf

- http://health.act.gov.au/sites/default/files//Systems%20approach%20to%20reducing%20unhealthy%20weight%20in%20Australian%20adolescents_%20ACT%20%C3%94%C3%87%C2%A3It%C3%94%C3%87%C3%96s%20Your%20Move%20%21%C3%94%C3%87%C3%98_.pdf

Masse, 201435

Canada N Y Canadian Census Statistics Canada http://www12.statcan.ca/census-recensement/2006/dp-pd/index-eng.cfm

- http://www12.statcan.gc.ca/census-recensement/2006/dp-pd/index-eng.cfm

http://www12.statcan.ca/datasets/Index-eng.cfm?Temporal=2006

Page 378: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-12

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Miewald 201236

Canada N Y Canadian Community Health Survey - Household Food Security Survey Module

Government of Canada

https://www.canada.ca/en/health-canada/services/food-nutrition/food-nutrition-surveillance/health-nutrition-surveys/canadian-community-health-survey-cchs.html

https://www.canada.ca/en/health-canada/services/food-nutrition/food-nutrition-surveillance/health-nutrition-surveys/canadian-community-health-survey-cchs/canadian-community-health-survey-cycle-2-2-nutrition-2004-income-related-household-food-security-canada-health-canada-2007.html#appa

- -

Miller, 201537 USA Y N Trip Identification and Analysis System

Westat https://www.westat.com/projects/using-gps-measure-physical-activity-levels

https://www.westat.com/expertise/statistical-research-survey-methods/survey-design

- -

Miller, 201537 USA N Y National Youth Physical Activity and Nutrition Study (NYPANS)

Centers for Disease Control and Prevention

https://www.cdc.gov/healthyyouth/data/yrbs/nypans.htm

ftp://ftp.cdc.gov/pub/data/yrbs/nypans/nypans_data_users_manual.pdf

- -

Nanney, 201638 USA N Y Minnesota Student Survey (MSS)

State of MN http://www.health.state.mn.us/divs/chs/mss/specialreports/index.html

- http://www.health.state.mn.us/divs/chs/mss/

-

Page 379: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-13

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Nanney, 201638 USA N Y The School Health Profiles (Profiles)

Centers for Disease Control and Prevention

For actual Data you need to send a request: https://www.cdc.gov/healthyyouth/data/profiles/contact.htm

https://www.cdc.gov/healthyyouth/data/profiles/questionnaires.htm

https://www.cdc.gov/healthyyouth/data/profiles/results.htm

https://www.cdc.gov/healthyschools/physicalactivity/profiles.htm

Nguyen, 201539

USA N Y National Health and Nutrition Examination Survey (NHANES)

Centers for Disease Control and Prevention - National Center for Health Statistics

https://wwwn.cdc.gov/nchs/data/nhanes/survey_contents.pdf

https://www.cdc.gov/nchs/nhanes/nhanes_questionnaires.htm

- https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx

Olsho, 201540

USA N Y NYC Community Health Survey

NYC Department of Health

http://www1.nyc.gov/site/doh/data/data-sets/community-health-survey.page

http://www1.nyc.gov/assets/doh/downloads/excel/episrv/chs-variable-crosswalk.xlsx

- http://www1.nyc.gov/site/doh/data/data-sets/community-health-survey-public-use-data.page

Parsons, 201441

USA N Y Anchorage School District’s student health data

Anchorage School District

- - - http://www.asdk12.org/data/behaviordashboard/

Peterson, 201542

USA Y N Community Health Information Profile (CHIP)

Massachusetts Dept. of Public Health

http://www.mass.gov/eohhs/researcher/community-health/masschip/

- - -

Powell, 200943 USA N Y Business list developed by Dun and Bradstreet

Dun and Bradstreet - - - -

Powell, 200943 USA N Y Monitoring the Future University of Michigan's Institute for Social Research

http://www.monitoringthefuture.org/data/data.html

- http://www.monitoringthefuture.org/

-

Page 380: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-14

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Powell, 200943 USA N Y Soda taxes The MayaTech Corporation for the Robert Wood Johnson Foundation-supported ImpacTeen project

http://impacteen.uic.edu/statetaxdata/BTG_State_Soda_Sales_Tax_Jan012011_publuse_091911.pdf

http://impacteen.uic.edu/obesitystatedata.htm

- -

Powell, 200943

USA N Y Sales Taxes Federation of Tax Administrators

- - https://www.taxadmin.org/current-tax-rates

https://www.taxadmin.org/assets/docs/Research/Rates/sales.pdf

Quig, 201244 New Zealand N Y Education Review Office - Communities of Learning

New Zealand Ministry of Education

http://www.ero.govt.nz

http://www.ero.govt.nz/how-ero-reviews/ero-reviews-of-early-childhood-services-and-kohanga-reo/#self-reports-for-early-childhood-services

- -

Quig, 201244 New Zealand N Y Dunedin City Population and Demography

Dunedin City Council

http://www.dunedin.govt.nz/

- - http://www.dunedin.govt.nz/your-council/long-term-plan-2015-2016/section-1-major-issues-and-strategies/city-profile/population-and-demography

Restrepo, 201645 USA N Y Bureau of Labor Statistic (County-level Unemployment Rates)

US Bureau of Labor COUNTY DATA: https://www.bls.gov/lau/

- - -

Page 381: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-15

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Restrepo, 201645 USA N Y Americans for Nonsmokers’ Rights

Americans for Nonsmokers’ Rights

http://www.no-smoke.org/goingsmokefree.php?id=519

- - http://www.no-smoke.org/document.php?id=675

Page 382: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-16

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Restrepo, 201645 USA N Y Callorie Labelling Laws Center for Science in the Public Interest

https://cspinet.org/resource/nutrition-labeling-standard-menu-items-chain-restaurants

- https://cspinet.org/protecting-our-health/menu-labeling

https://cspinet.org/resource/nutrition-labeling-chain-restaurants-state-and-local-lawsbillsregulations-2009-2010

Restrepo, 201645 USA N Y Behavioral Risk Factor Surveillance System (BRFSS)

Centers for Disease Control and Prevention

https://www.cdc.gov/brfss/smart/smart_data.htm

https://www.cdc.gov/brfss/questionnaires/index.htm

https://www.cdc.gov/brfss/

https://www.cdc.gov/brfss/annual_data/annual_data.htm

Restrepo, 201645 USA N Y Cigarette taxes Tax Burden on Tobacco

https://www.taxadmin.org/index.php?option=com_content&view=article&id=58:tobacco-tax-papers&catid=28:tobacco-tax&Itemid=205

https://www.healthdata.gov/dataset/tax-burden-tobacco-volume-49-1970-2014

https://www.taxadmin.org/assets/docs/Tobacco/papers/tax_burden_2014.pdf

-

Restrepo, 201645 USA N Y Beer taxes Brewer’s Almanac http://www.beerinstitute.org/statistics/taxes-paid/

- - -

Riis, 201246

USA N Y School Nutrition-Environment State Policy Classification System

National Cancer Institute (UBC: Centre for Community Child Health Research)

https://www.ncbi.nlm.nih.gov/pubmed/17884576

- - https://class.cancer.gov/data/201201/CLASS_Nutrition_SCORING_KEY_and_variable_information_132012.pdf

Robles, 201747 USA N Y Los Angeles County Health and Nutrition Examination Survey

LA-County http://publichealth.lacounty.gov/ha/

- http://publichealth.lacounty.gov/ha/hasurveyintro.htm

Page 383: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-17

Page 384: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-18

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Sabia, 201648 USA N Y Youth Risk Behavior Survey Suveillance – National

Centers for Disease Control and Prevention

https://nccd.cdc.gov/youthonline/App/Results.aspx?TT=B&OUT=0&SID=HS&QID=QNOBESE&LID=LL&YID=RY&LID2=&YID2=&COL=&ROW1=&ROW2=&HT=&LCT=&FS=&FR=&FG=&FI=&FP=&FSL=&FRL=&FGL=&FIL=&FPL=&PV=&TST=&C1=&C2=&QP=&DP=&VA=CI&CS=Y&SYID=&EYID=&SC=&SO=

https://www.cdc.gov/healthyyouth/data/yrbs/questionnaires.htm

https://www.cdc.gov/healthyyouth/data/topics/npao.htm

https://www.cdc.gov/healthyyouth/data/yrbs/data.htm

Sabia, 201648 USA N Y Shape of the Nation Reports

National Association for Sport and Physical Education and American Heart Association

http://www.shapeamerica.org/advocacy/son/2016/upload/Shape-of-the-Nation-2016_web.pdf

- http://www.shapeamerica.org/advocacy/son/index.cfm

-

Sabia, 201648 USA N Y School Health Policy Database - State

National Association of State Boards of Education

http://www.nasbe.org/healthy_schools/hs/map.php

- - http://www.nasbe.org/healthy_schools/hs/search.php

Schwartz, 201649 USA N Y NYC School Food Database

New York City Department of Education

- - - -

Schwartz, 201649 USA N Y FITNESSGRAM-NYC The Cooper Institute - - - -

Page 385: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-19

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Sekhobo, 201450 USA N Y The Pediatric Nutrition Surveillance System

Centers for Disease Control and Prevention

https://www.health.ny.gov/statistics/prevention/nutrition/pednss/

- - -

Page 386: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-20

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Sturm, 201051

USA Y N Bridging the Gap - Community Obesity Measures Project

RWJ Foundation http://www.bridgingthegapresearch.org/research/community_data/

http://www.bridgingthegapresearch.org/_asset/p5mswy/BTGCOMP_FoodStore_2012.pdf

- -

Sturm, 201552 USA N Y California Health Interview Survey

University of California Los Angeles

http://healthpolicy.ucla.edu/chis/Pages/default.aspx

http://healthpolicy.ucla.edu/chis/design/Pages/questionnairesEnglish.aspx

https://www.cdc.gov/nchs/nhis/

http://healthpolicy.ucla.edu/chis/analyze/Pages/CHIS-Data-Documentation.aspx

Sturm, 201552 USA N Y LA County Food Retail Los Angeles County Department of Public Health

- - - -

Taber, 201153 USA N Y School Health Policies and Programs Study (SHPPS)

Centers for Disease Control and Prevention

- https://www.cdc.gov/healthyyouth/data/shpps/questionnaires.htm

https://www.cdc.gov/healthyyouth/data/shpps/index.htm

https://www.cdc.gov/healthyyouth/data/shpps/data.htm

Taber, 201254

USA N Y Westlaw Thomson Reuters - - - -

Toussaint, 201755 USA N Y The Northeast Iowa Food and Fitness Initiative

Kellogg Foundation Food

http://www.iowafoodandfitness.org/site/datacomm.html

http://www.iowafoodandfitness.org/site/cyclemenu.html

-

Utter, 201656 New Zealand N Y Youth'12 Adolescent Health Research Group

https://www.fmhs.auckland.ac.nz/en/faculty/adolescent-health-research-group.html

https://cdn.auckland.ac.nz/assets/fmhs/faculty/ahrg/docs/youth12-questionnaire.pdf

- https://www.fmhs.auckland.ac.nz/assets/fmhs/faculty/ahrg/docs/2012prevalence-tables-report.pdf

Page 387: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-21

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

von Hippel, 201557 USA N Y Texas Fitness Now Texas Government http://tea.texas.gov/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=2147496810&libID=2147496807

http://tea.texas.gov/Texas_Schools/Safe_and_Healthy_Schools/Physical_Fitness_Assessment_Initiative/FITNESSGRAM%C2%AE_Procedures_Manual/

http://tea.texas.gov/Reports_and_Data/Program_Evaluations/Other_Initiatives/Program_Evaluation__Other_Initiatives/

http://tea.texas.gov/Texas_Schools/Safe_and_Healthy_Schools/Physical_Fitness_Assessment_Initiative/Fitness_Data/

von Hippel, 201557 USA Y N FitnessGram The Cooper Institute http://www.cooperinstitute.org/FitnessGram

http://tea.texas.gov/Texas_Schools/Safe_and_Healthy_Schools/Physical_Fitness_Assessment_Initiative/FITNESSGRAM%C2%AE_Procedures_Manual/

- -

Webb, 201258 UK N Y English Longitudinal Study of Ageing (ELSA)

National Health Services

- http://www.elsa-project.ac.uk/documentation

- http://www.elsa-project.ac.uk/data_elsa

Wells, 200559

UK N Y National Diet and Nutrition Survey data

United Kingdom Government (Public Health England + UK Food Standards Agency)

https://www.gov.uk/government/collections/national-diet-and-nutrition-survey

- http://webarchive.nationalarchives.gov.uk/20130402145952/http://transparency.dh.gov.uk/category/statistics/ndns/

-

Woodward-Lopez, 201060

USA Y N School Wellness Study (SWS)

Study Team - - - -

Woodward-Lopez, 201060

USA Y N the High School Study (HSS)

Study Team - - - -

Page 388: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-22

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Woodward-Lopez, 201060

USA Y N Healthy Eating, Active Communities study (HEAC)

place-based initiative

- - https://portal.hud.gov/hudportal/HUD?src=/program_offices/economic_development/place_based

http://www.cssp.org/community/neighborhood-investment/place-based-initiatives

Page 389: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-23

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Experimental studies

Ludwig, 201161

USA N Y Moving to Opportunity Housing and Urban Development

http://www.nber.org/mtopuf/#PUF

http://www.nber.org/mtopublic/instruments.html

- -

Perry, 200462

USA Y N Nutrition Data System University of Minnesota, Minneapolis

- https://drive.google.com/drive/folders/0B7tgPhfpOAbTNTZ6UDBSMUV3MmM

- -

Naylor, 200663

Canada N Y School Action Inventory British Columbia Ministry of Education

- http://www.actionschoolsbc.ca/

- -

Pope, 201664 USA Y N Center for Healthy Options and Community Empowerment

National Center on Minority Health and Health Disparities

https://www.nimhd.nih.gov/

- -

Rush, 201465

New Zealand Y N Project Energize Waikato District Health Board

- - https://www.waikatodhb.health.nz/public-health-advice/project-energize/

https://www.waikatodhb.health.nz/assets/public-health-advice/project-energize/School-engagement-in-the-Project-Energize-health-intervention-programme.pdf

Lorentzen, 200966

Norway N Y Population and Demography

Statistics Norway https://www.ssb.no/en/

https://www.ssb.no/en/befolkning/nokkeltall/summary-tables

- -

Page 390: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-24

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Evans, 201367

UK N Y UK School Surveys National Foundation of Educational Research

https://www.nfer.ac.uk/schools/school-surveys/

https://www.nfer.ac.uk/schools/school-surveys/about-the-surveys/

https://www.nfer.ac.uk/about-nfer/strategy-and-vision/annual-report/

-

Page 391: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-25

Author, year Country Primary Secondary Database Custodian Generic URL Data Dictionary URL Parent System URL Data Access Link

Other Study Designs

Cheadle, 201268

USA Y N Documentation of Community Change (DOCC)

Kaiser Permanente - - http://www.healzones.org/resources/[email protected]

-

de Silva-Sanigorski, 201169

Australia Y N Romp & Chomp Victoria State Government, Australia

- - - -

de Silva-Sanigorski, 201169

Australia N Y Maternal and Child Health (MCH) Key Age and Stage (KA&S)

Victoria State Government, Australia

- - - -

Page 392: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-26

References for Appendix J

1. Bauhoff S. The effect of school district nutrition policies on dietary intake and overweight: a synthetic control approach. Econ Hum Biol. 2014 Jan;12:45-55. doi: 10.1016/j.ehb.2013.06.001. PMID: 23891422.

2. Branas CC, Cheney RA, MacDonald JM, et al. A difference-in-differences analysis of health, safety, and greening vacant urban space. Am J Epidemiol. 2011 Dec 01;174(11):1296-306. doi: 10.1093/aje/kwr273. PMID: 22079788.

3. Camacho-Rivera M, Rosenbaum E, Yama C, et al. Low-Income Housing Rental Assistance, Perceptions of Neighborhood Food Environment, and Dietary Patterns among Latino Adults: the AHOME Study. J Racial Ethn Health Disparities. 2017 Jun;4(3):346-53. doi: 10.1007/s40615-016-0234-z. PMID: 27129854.

4. Capogrossi K, You W. The Influence of School Nutrition Programs on the Weight of Low-Income Children: A Treatment Effect Analysis. Health Econ. 2016 Jul 6doi: 10.1002/hec.3378. PMID: 27381591.

5. Chen C, Chou S-Y, Thornton RJ. The Effect of Household Technology on Weight and Health Outcomes among Chinese Adults: Evidence from China's 'Home Appliances Going to the Countryside' Policy. Journal of Human Capital. 2015 Fall;9(3):364-401. doi: http://www.jstor.org/action/showPublication?journalCode=jhumancapital. PMID: 1541217.

6. Cleland V, Dwyer T, Blizzard L, et al. The provision of compulsory school physical activity: associations with physical activity, fitness and overweight in childhood and twenty years later. Int J Behav Nutr Phys Act. 2008 Feb 29;5:14. doi: 10.1186/1479-5868-5-14. PMID: 18312621.

7. Coffield JE, Metos JM, Utz RL, et al. A multivariate analysis of federally mandated school wellness policies on adolescent obesity. J Adolesc Health. 2011 Oct;49(4):363-70. doi: 10.1016/j.jadohealth.2011.01.010. PMID: 21939866.

8. Cradock AL, McHugh A, Mont-Ferguson H, et al. Effect of school district policy change on consumption of sugar-sweetened beverages among high school students, Boston, Massachusetts, 2004-2006. Prev Chronic Dis. 2011 Jul;8(4):A74. PMID: 21672398.

9. Cummins S, Petticrew M, Higgins C, et al. Large scale food retailing as an intervention for diet and health: quasi-experimental evaluation of a natural experiment. J Epidemiol Community Health. 2005 Dec;59(12):1035-40. doi: 10.1136/jech.2004.029843. PMID: 16286490.

10. Cummins S, Findlay A, Petticrew CHM, et al. Reducing inequalities in health and diet: findings from a study on the impact of a food retail development. Environ Plann A 2008;40(2):402-22.

11. Cummins S, Flint E, Matthews SA. New neighborhood grocery store increased awareness of food access but did not alter dietary habits or obesity. Health Aff (Millwood). 2014 Feb;33(2):283-91. doi: 10.1377/hlthaff.2013.0512. PMID: 24493772.

12. Datar A, Nicosia N. The effect of state competitive food and beverage regulations on childhood overweight and obesity. Journal of Adolescent Health. 2016doi: 10.1016/j.jadohealth.2016.09.003. PMID: 2016-54655-001.

13. Fitzpatrick C, Datta GD, Henderson M, et al. School food environments associated with adiposity in Canadian children. Int J Obes (Lond). 2017 Mar 14doi: 10.1038/ijo.2017.39. PMID: 28186100.

14. Frongillo EA, Fawcett SB, Ritchie LD, et al. Community Policies and Programs to Prevent Obesity and Child Adiposity. Am J Prev Med. 2017 Jul 05doi: 10.1016/j.amepre.2017.05.006. PMID: 28688728.

15. Gee KA. School-Based Body Mass Index Screening and Parental Notification in Late Adolescence: Evidence From Arkansas's Act 1220. J Adolesc Health. 2015 Sep;57(3):270-6. doi: 10.1016/j.jadohealth.2015.05.007. PMID: 26115907.

16. Giles-Corti B, Bull F, Knuiman M, et al. The influence of urban design on neighbourhood walking following residential relocation: longitudinal results from the RESIDE study. Soc Sci Med. 2013 Jan;77:20-30. doi: 10.1016/j.socscimed.2012.10.016. PMID: 23206559.

17. Hennessy E, Oh A, Agurs-Collins T, et al. State-level school competitive food and beverage laws are associated with children's weight status. J Sch Health. 2014 Sep;84(9):609-16. doi: 10.1111/josh.12181. PMID: 25117896.

Page 393: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-27

18. Hobin E, So J, Rosella L, et al. Trajectories of objectively measured physical activity among secondary students in Canada in the context of a province-wide physical education policy: a longitudinal analysis. J Obes. 2014;2014:958645. doi: 10.1155/2014/958645. PMID: 24672714.

19. Hobin E, Erickson T, Comte M, et al. Examining the impact of a province-wide physical education policy on secondary students' physical activity as a natural experiment. Int J Behav Nutr Phys Act. 2017 Jul 19;14(1):98. doi: 10.1186/s12966-017-0550-7. PMID: 28724390.

20. Hoelscher D, Ory M, Dowdy D, et al. Effects of funding allocation for Safe Routes to School programs on active commuting to school and related behavioral, knowledge, and psychosocial outcomes: Results from the Texas Childhood Obesity Prevention Policy Evaluation (T-COPPE) study. Environ Behav. 2016;48(1):210-29. doi: 10.1177/0013916515613541. PMID: 2016-02263-012.

21. Howlett E, Davis C, Burton S. From food desert to food oasis: The potential influence of food retailers on childhood obesity rates. Journal of Business Ethics. 2016;139(2):215-24. doi: 10.1007/s10551-015-2605-5. PMID: 2015-12722-001.

22. Hu Y, van Lenthe FJ, Judge K, et al. Did the English strategy reduce inequalities in health? A difference-in-difference analysis comparing England with three other European countries. BMC Public Health. 2016 Aug 24;16(1):865. doi: 10.1186/s12889-016-3505-z. PMID: 27558269.

23. Hughes RJ, Edwards KL, Clarke GP, et al. Childhood consumption of fruit and vegetables across England: a study of 2306 6-7-year-olds in 2007. Br J Nutr. 2012 Aug;108(4):733-42. doi: 10.1017/s0007114511005939. PMID: 22321148.

24. Hunter S, Leatherdale ST, Storey K, et al. A quasi-experimental examination of how school-based physical activity changes impact secondary school student moderate- to vigorous- intensity physical activity over time in the COMPASS study. Int J Behav Nutr Phys Act. 2016 Jul 29;13:86. doi: 10.1186/s12966-016-0411-9. PMID: 27473113.

25. Jia P, Li M, Xue H, et al. School environment and policies, child eating behavior and overweight/obesity in urban China: the childhood obesity study in China megacities. Int J Obes (Lond). 2017 May;41(5):813-9. doi: 10.1038/ijo.2017.2. PMID: 28074059.

26. Just DR, Wansink B. School lunch debit card payment systems are associated with lower nutrition and higher calories. Obesity (Silver Spring). 2014 Jan;22(1):24-6. doi: 10.1002/oby.20591. PMID: 23929600.

27. Kern E, Chan NL, Fleming DW, et al. Declines in student obesity prevalence associated with a prevention initiative - King County, Washington, 2012. MMWR Morb Mortal Wkly Rep. 2014 Feb 21;63(7):155-7. PMID: 24553199.

28. Kim J. Are physical education-related state policies and schools' physical education requirement related to children's physical activity and obesity? J Sch Health. 2012 Jun;82(6):268-76. doi: 10.1111/j.1746-1561.2012.00697.x. PMID: 22568462.

29. Kubik MY, Lytle LA, Story M. Schoolwide food practices are associated with body mass index in middle school students. Arch Pediatr Adolesc Med. 2005 Dec;159(12):1111-4. doi: 10.1001/archpedi.159.12.1111. PMID: 16330732.

30. Lachapelle U, Frank LD. Transit and health: mode of transport, employer-sponsored public transit pass programs, and physical activity. J Public Health Policy. 2009;30 Suppl 1:S73-94. doi: 10.1057/jphp.2008.52. PMID: 19190584.

31. Liu J, Kuo T, Jiang L, et al. Food and drink consumption among 1-5-year-old Los Angeles County children from households receiving dual SNAP and WIC v. only WIC benefits. Public Health Nutr. 2016 Sep 9:1-8. doi: 10.1017/s1368980016002329. PMID: 27609603.

32. MacDonald JM, Stokes RJ, Cohen DA, et al. The effect of light rail transit on body mass index and physical activity. Am J Prev Med. 2010 Aug;39(2):105-12. doi: 10.1016/j.amepre.2010.03.016. PMID: 20621257.

33. Madsen KA. School-based body mass index screening and parent notification: a statewide natural experiment. Arch Pediatr Adolesc Med. 2011 Nov;165(11):987-92. doi: 10.1001/archpediatrics.2011.127. PMID: 21727262.

34. Malakellis M, Hoare E, Sanigorski A, et al. School-based systems change for obesity prevention in adolescents: outcomes of the Australian Capital Territory 'It's Your Move!'. Aust N Z J Public Health. 2017 Jul 27doi: 10.1111/1753-6405.12696. PMID: 28749562.

35. Masse LC, de Niet-Fitzgerald JE, Watts AW, et al. Associations between the school food environment, student consumption and body mass index of Canadian adolescents. Int J Behav Nutr Phys Act. 2014 Mar 26;11(1):29. doi: 10.1186/1479-5868-11-29. PMID: 24666770.

36. Miewald C, Holben D, Hall P. Role of a food box program in fruit and vegetable consumption and food security. Can J Diet Pract Res. 2012 Summer;73(2):59-65. doi: 10.3148/73.2.2012.59. PMID: 22668838.

Page 394: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-28

37. Miller HJ, Tribby CP, Brown BB, et al. Public transit generates new physical activity: Evidence from individual GPS and accelerometer data before and after light rail construction in a neighborhood of Salt Lake City, Utah, USA. Health Place. 2015 Nov;36:8-17. doi: 10.1016/j.healthplace.2015.08.005.

38. Nanney MS, MacLehose RF, Kubik MY, et al. School Obesity Prevention Policies and Practices in Minnesota and Student Outcomes: A Longitudinal Cohort Study. Am J Prev Med. 2016 Nov;51(5):656-63. doi: 10.1016/j.amepre.2016.05.008. PMID: 27320703.

39. Nguyen BT, Powell LM. Supplemental nutrition assistance program participation and sugar-sweetened beverage consumption, overall and by source. Prev Med. 2015 Dec;81:82-6. doi: 10.1016/j.ypmed.2015.08.003. PMID: 26303370.

40. Olsho LE, Payne GH, Walker DK, et al. Impacts of a farmers' market incentive programme on fruit and vegetable access, purchase and consumption. Public Health Nutr. 2015 Oct;18(15):2712-21. doi: 10.1017/s1368980015001056. PMID: 25919225.

41. Parsons WG, Garcia GM, Hoffman PK. Evaluating school wellness policy in curbing childhood obesity in Anchorage, Alaska. J Sch Nurs. 2014 Oct;30(5):324-31. doi: 10.1177/1059840513513155. PMID: 24316497.

42. Peterson KE, Spadano-Gasbarro JL, Greaney ML, et al. Three-Year Improvements in Weight Status and Weight-Related Behaviors in Middle School Students: The Healthy Choices Study. PLoS One. 2015;10(8):e0134470. doi: 10.1371/journal.pone.0134470. PMID: 26295837.

43. Powell LM, Chriqui J, Chaloupka FJ. Associations between state-level soda taxes and adolescent body mass index. J Adolesc Health. 2009 Sep;45(3 Suppl):S57-63. doi: 10.1016/j.jadohealth.2009.03.003. PMID: 19699437.

44. Quigg R, Reeder AI, Gray A, et al. The Effectiveness of a Community Playground Intervention. Journal of Urban Health-Bulletin of the New York Academy of Medicine. 2012 Feb;89(1):171-84. doi: 10.1007/s11524-011-9622-1.

45. Restrepo BJ. Calorie Labeling in Chain Restaurants and Body Weight: Evidence from New York. Health Econ. 2016 Jul 24doi: 10.1002/hec.3389. PMID: 27451966.

46. Riis J, Grason H, Strobino D, et al. State school policies and youth obesity. Matern Child Health J. 2012 Apr;16 Suppl 1:S111-8. doi: 10.1007/s10995-012-1000-4. PMID: 22527761.

47. Robles B, Montes CE, Nobari TZ, et al. Dietary Behaviors among Public Health Center Clients with Electronic Benefit Transfer Access at Farmers' Markets. J Acad Nutr Diet. 2017 Jan;117(1):58-68. doi: 10.1016/j.jand.2016.07.012. PMID: 27618576.

48. Sabia JJ, Nguyen TT, Rosenberg O. High School Physical Education Requirements and Youth Body Weight: New Evidence from the YRBS. Health Econ. 2016 Aug 30doi: 10.1002/hec.3399. PMID: 27576770.

49. Schwartz AE, Leardo M, Aneja S, et al. Effect of a School-Based Water Intervention on Child Body Mass Index and Obesity. JAMA Pediatr. 2016 Mar;170(3):220-6. doi: 10.1001/jamapediatrics.2015.3778. PMID: 26784336.

50. Sekhobo JP, Edmunds LS, Dalenius K, et al. Neighborhood disparities in prevalence of childhood obesity among low-income children before and after implementation of New York City child care regulations. Prev Chronic Dis. 2014;11:E181. doi: 10.5888/pcd11.140152. PMID: 25321632.

51. Sturm R, Powell LM, Chriqui JF, et al. Soda Taxes, Soft Drink Consumption, And Children's Body Mass Index. Health Affairs. 2010 May-Jun;29(5):1052-8. doi: 10.1377/hlthaff.2009.0061.

52. Sturm R, Hattori A. Diet and obesity in Los Angeles County 2007-2012: Is there a measurable effect of the 2008 "Fast-Food Ban"? Soc Sci Med. 2015 May;133:205-11. doi: 10.1016/j.socscimed.2015.03.004. PMID: 25779774.

53. Taber DR, Stevens J, Evenson KR, et al. State policies targeting junk food in schools: racial/ethnic differences in the effect of policy change on soda consumption. Am J Public Health. 2011 Sep;101(9):1769-75. doi: 10.2105/ajph.2011.300221. PMID: 21778484.

54. Taber DR, Chriqui JF, Perna FM, et al. Weight status among adolescents in States that govern competitive food nutrition content. Pediatrics. 2012 Sep;130(3):437-44. doi: 10.1542/peds.2011-3353. PMID: 22891223.

55. Toussaint LL, Housholder K, Janssen K, et al. Slowing BMI Growth Trajectories in Elementary School-Aged Children: The Northeast Iowa Food and Fitness Initiative. Fam Community Health. 2017 Jul/Sep;40(3):192-7. doi: 10.1097/fch.0000000000000151. PMID: 28525438.

56. Utter J, Denny S, Dyson B. School gardens and adolescent nutrition and BMI: Results from a national, multilevel study. Prev Med. 2016 Feb;83:1-4. doi: 10.1016/j.ypmed.2015.11.022. PMID: 26657347.

Page 395: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

J-29

57. von Hippel PT, Bradbury WK. The effects of school physical education grants on obesity, fitness, and academic achievement. Prev Med. 2015 Sep;78:44-51. doi: 10.1016/j.ypmed.2015.06.011. PMID: 26163396.

58. Webb E, Netuveli G, Millett C. Free bus passes, use of public transport and obesity among older people in England. J Epidemiol Community Health. 2012 Feb;66(2):176-80. doi: 10.1136/jech.2011.133165. PMID: 21911850.

59. Wells L, Nelson M. The National School Fruit Scheme produces short-term but not longer-term increases in fruit consumption in primary school children. Br J Nutr. 2005 Apr;93(4):537-42. PMID: 15946417.

60. Woodward-Lopez G, Gosliner W, Samuels SE, et al. Lessons learned from evaluations of California's statewide school nutrition standards. Am J Public Health. 2010 Nov;100(11):2137-45. doi: 10.2105/ajph.2010.193490. PMID: 20864696.

61. Ludwig J, Sanbonmatsu L, Gennetian L, et al. Neighborhoods, obesity, and diabetes--a randomized social experiment. N Engl J Med. 2011 Oct 20;365(16):1509-19. doi: 10.1056/NEJMsa1103216. PMID: 22010917.

62. Perry CL, Bishop DB, Taylor GL, et al. A randomized school trial of environmental strategies to encourage fruit and vegetable consumption among children. Health Educ Behav. 2004 Feb;31(1):65-76. PMID: 14768658.

63. Naylor PJ, Macdonald HM, Zebedee JA, et al. Lessons learned from Action Schools! BC--an 'active school' model to promote physical activity in elementary schools. J Sci Med Sport. 2006 Oct;9(5):413-23. doi: 10.1016/j.jsams.2006.06.013. PMID: 16884957.

64. Pope M. Preventing Weight Gain in Children Who Are School Age and African-American. Pediatr Phys Ther. 2016 Summer;28(2):207-16. doi: 10.1097/pep.0000000000000243. PMID: 26914717.

65. Rush E, McLennan S, Obolonkin V, et al. Project Energize: whole-region primary school nutrition and physical activity programme; evaluation of body size and fitness 5 years after the randomised controlled trial. Br J Nutr. 2014 Jan 28;111(2):363-71. doi: 10.1017/s0007114513002316. PMID: 23867069.

66. Lorentzen C, Ommundsen Y, Jenum AK, et al. The "Romsas in Motion" community intervention: mediating effects of psychosocial factors on forward transition in the stages of change in physical activity. Health Educ Behav. 2009 Apr;36(2):348-65. doi: 10.1177/1090198107308372. PMID: 18065570.

67. Evans CE, Ransley JK, Christian MS, et al. A cluster-randomised controlled trial of a school-based fruit and vegetable intervention: Project Tomato. Public Health Nutr. 2013 Jun;16(6):1073-81. doi: 10.1017/s1368980012005290. PMID: 23237386.

68. Cheadle A, Rauzon S, Spring R, et al. Kaiser Permanente's Community Health Initiative in Northern California: evaluation findings and lessons learned. Am J Health Promot. 2012 Nov-Dec;27(2):e59-68. doi: 10.4278/ajhp.111222-QUAN-462. PMID: 23113787.

69. de Silva-Sanigorski A, Elea D, Bell C, et al. Obesity prevention in the family day care setting: impact of the Romp & Chomp intervention on opportunities for children's physical activity and healthy eating. Child Care Health Dev. 2011 May;37(3):385-93. doi: 10.1111/j.1365-2214.2010.01205.x. PMID: 21276039.

Page 396: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-1

Appendix K. Individual Study Risk of Bias Ratings Table K1. Individual study EPHPP risk of bias ratings

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Natural Experiments

Anderson, 20131 M M M L H H H

Anthamatten, 20112 M M H H M H H

Azevedo, 20143 H M H H L L H

Barnidge, 20134 L M H H L H H

Barroso, 20095 M H M M L H H

Bauhoff, 20146 H M L M M H H

Bauman, 20037 M H H H H H H

Bere, 20108 M M H M L M M

Berger-Jenkins, 20149

H M M M H H H

Bolton, 201710 M M L H L H H

Bowling, 201611 M M H M L H H

Branas, 201112 M M H M H H H

Brown, 200813 H H H M M M H

Brown, 201514 L M L H L H H

Brown, 201615 L M L H L H H

Brown, 201616 M M H H L H H

Burke, 201417 M M H M L H H

Buscail, 201618 H M L H L H H

Caldwell, 200919 M M H H L H H

Calise, 201320 M H L L M M M

Camacho-Rivera, 201721

H H L M L H H

Capogrossi, 201622 L M M M L M L

Cawley, 200723 L H M M L H H

Page 397: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-2

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Cawley, 200724 L H H M L H H

Chen, 201525 L M L M L H M

Cleland, 200826 M H M M L H H

Coffield, 201127 H H M M H H H

Cohen, 201228 L M H M M H H

Coyle, 200929 L M H H L H H

Cradock, 201130 L H H M M M H

Cradock, 201431 M M M L L L L

Cullen, 200632 M M H M H H H

Cullen, 200833 L M L H L H H

Cummins, 200534 M M L H H H H

Cummins, 200835 H H L M M M H

Cummins, 201436 M M L M M M L

Datar, 201637 M H L M L H H

de Visser, 201638 H H H M L M H

De Cocker, 200739 M M L H L M M

Dill, 201440 H M H M M H H

Dubowitz, 201541 L M L M M M L

Eagle, 201342 L M H H L H H

Elbel, 201543 H M L H L H H

Page 398: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-3

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Elbel, 201744 M H L M M H H

Falbe, 201645 M H L M L L M

Fitzpatrick, 201746 H M L M M M M

Flego, 201447 L M L H L H H

Fogarty, 200748 M M H H L H H

Fox, 200949 H H H M M M H

Frongillo, 201750 H H L H L H H

Fuller, 201351 H H H H M M H

Fung, 201352 H H M H M H H

Gee, 201553 L H L L L H H

Gibson, 200654 M H M M L H H

Giles-Corti, 201355 L M H M L L M

Gleason, 200956 H H H H L H H

Goldsby, 201657 L M L L M H M

Goodman, 201458 L M L H M L M

Goodman, 201659 M M L L L L L

Gorely, 201160 M M H H L M H

Gorham, 201561 M M H H M M H

Harding, 201762 H M L H L H H

Heelan, 201563 L M H M L L M

Hennessy, 201464 L M L L L H M

Herrick, 201265 M M H H L L H

Hilmers, 201466 H H M M H H H

Hobin, 201467 M M M M H H H

Hobin, 201768 M H L M L H H

Hoelscher, 201669 M M L M L L L

Page 399: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-4

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Hoelscher, 201670 M M H H L H H

Howlett, 201671 L H H M L H H

Hu, 201672 M M L M H M M

Hughes, 201273 L M L H L H H

Hunter, 201674 H M H H M M H

Jennings, 201275 L M L H L H H

Jia, 201776 M H H M H H H

Johnson, 201777 M M L M H H H

Just, 201478 M H M M H H H

Kern, 201479 M M H M H H H

Keyte, 201280 H H H M L M H

Kim, 201281 M H H M M H H

King, 201482 M M H M H H H

Kubik, 200583 L H H M H H H

Lachapelle, 200984 H H H H H H H

LaRowe, 201685 H M M M L H H

Leung, 201386 H H M L M H H

Liao, 201587 M M H M L H H

Ling, 201488 L M L H L H H

Liu, 201689 M H M M H H H

MacDonald, 201090 M M L M L L L

Maddock, 200691 M M H M H H H

Madsen, 201192 L H L M M M M

Madsen, 201593 L H L M H H H

Malakellis, 201794 H H H H L M H

Masse, 201495 M H H M H M H

Page 400: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-5

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Miewald, 201296 M H L H L H H

Miller, 201597 M M H M L H H

Molitor, 201598 M M L M M M L

Morton, 201699 M M L M L H M

Mullally, 2010100 L H H M M M H

Mumford, 2011101 H H L H M M H

Nanney, 2014102 M M M M L M L

Nanney, 2016103 L M M M H M M

Benjamin Neelon, 2015104

H M H M M H H

Nehme, 2017105 H M L H M H H

Nguyen, 2015106 L H L M L H H

Oh, 2015107 H H H M L H H

Olsho, 2015108 M M H H M M H

Panter, 2016109 L M L H L H H

Parsons, 2014110 M H H M H H H

Peterson, 2015111 L M L H H H H

Powell, 2009112 M H H L H H H

Quigg, 2012113 H M L M H L H

Reger-Nash, 2005114 L M L H L H H

Reger-Nash, 2008115 H M L M M H H

Restrepo, 2016116 M H M H H H H

Ridgers, 2007117 M M M M M L L

Ridgers, 2007118 H M L M M L M

Riis, 2012119 M H H M L H H

Ritchie, 2016120 M H L M L H H

Page 401: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-6

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Robles, 2017121 M H H H L H H

Rushakoff, 2017122 H M L H H L H

Sabia, 2016123 M H L M L H H

Sadler, 2013124 H M L H H H H

Sanchez-Vaznaugh, 2010125

L M L M H H H

Schanzenbach, 2005126

M M L L H H H

Schwartz, 2016127 L H L M L M M

Sekhobo, 2014128 L M H M L H H

Slater, 2014129 M H L H H H H

Spence, 2013130 H H H M M H H

Stephens, 2014131 L H H M L H H

Stratton, 2005132 M M H H L M H

Sturm, 2010133 H H H H H H H

Sturm, 2015134 M H L M H M H

Taber, 2011135 L H H M M M H

Taber, 2012136 M H M H H M H

Taber, 2012137 M M M M L M L

Taber, 2012138 M M H M M M M

Taber, 2013139 M M H M H H H

Tak, 2007140 M M H H L H H

Tak, 2009141 L M L H H M H

Tester, 2016142 L M L M L H M

Toussaint, 2017143 M M L M L H M

Page 402: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-7

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Utter, 2016144 M H L M H H H

Vadiveloo, 2011145 L H L M M L M

Veugelers, 2005146 M H M M M L M

von Hippel, 2015147 M M H M M H H

Webb, 2012148 L H L M M M M

Webb, 2016149 M H L M M M M

Wells, 2005150 M M M M M M L

West, 2011151 H H L H H H H

Whetstone, 2012152 H H H M H H H

Woodward-Lopez, 2010153

L M H M L M M

Wrigley, 2003154 M M L H H M H

Zhu, 2013155 H M L H L H H

Zhu, 2014156 H H L M M M M

Experimental

Alaimo, 2013157 H M L M M M M

Anderson, 2001158 L M L H L H H

Angelopoulos, 2009159

M L L M L L L

Ask, 2010160 M H H M M H H

Audrey, 2015161 H L H H H M H

Ayala, 2013162 H L H M M M H

Backman, 2011163 L L H H L L H

Baker, 2016164 H M L M H H H

Bastian, 2015165 M M M M L L L

Bere, 2005166 M M L H L H H

Bere, 2006167 M L H H M H H

Page 403: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-8

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Bere, 2006168 H L H M L H H

Bere, 2007169 H L H M L M H

Beresford, 2010170 M L L H M M M

Blum, 2008171 M L H M L L M

Bonsergent, 2013172 L L L M L M L

Bonvin, 2013173 M L L M L L L

Busch, 2015174 M M H H L H H

Caballero, 2003175 L M H M L L M

Chomitz, 2010176 L M L M M H M

Cochrane, 2012177 H L L M L L M

Cohen, 2014178 L L H H M H H

Coleman, 2012179 H L M M L M M

Cortinez-O'Ryan, 2017180

M M L M L L L

Crespo, 2012181 M M L H L M M

Day, 2008182 M L H H L H H

de Meij, 2011183 L M L M L M L

De Coen, 2012184 M M H H L M H

De Greef, 2016185 M L L H L H H

Dunton, 2015186 M L L M L M L

Dzewaltowski, 2009187

M M L M L H M

Page 404: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-9

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Economos, 2007188 M M L M L M L

Elinder, 2012189 M M M H M L M

Eriksen, 2003190 H L H M L H H

Ermetici, 2016191 M M L H L L M

Esquivel, 2016192 M M L M L M L

Evans, 2013193 L L H H M M H

Fairclough, 2016194 M H H H L M H

Farley, 2007195 M M H M H H H

Farmer, 2017196 H L L M L M M

Finch, 2014197 M L H M L H H

Foster, 2008198 M L L M L M L

French, 2010199 M H H M M H H

Fu, 2016200 M H H M L H H

Gatto, 2017201 H L L H L L H

Geaney, 2016202 M L L H M M M

Gittelsohn, 2010203 M M M M H H H

Gittelsohn, 2013204 M L M M M H M

Goetzel, 2010205 M L L M M H M

Gustat, 2012206 M M H M L L M

Haerens, 2006207 L L M H L H H

Haerens, 2007208 H H L M L H H

Hardy, 2010209 H H L M M M H

He, 2009210 M H L H L L H

Hendy, 2011211 M L L M L H M

Hoefkens, 2011212 M M M M L H M

Hollis, 2016213 L L L M L L L

Page 405: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-10

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Huberty, 2011214 M L H H L H H

Jago, 2011215 H H L M L M H

Janssen, 2015216 M L L M L L L

Jones, 2015217 L L L M M L L

Jordan, 2008218 M L H M M M M

Jurg, 2006219 L M L M H H H

Kain, 2004220 L M H M L L M

Kamada, 2013221 M L M L M M L

Kastorini, 2016222 H M L M H H H

Kloek, 2006223 M M H M L M M

LaCaille, 2016224 H M L M L M M

Lemon, 2010225 M L L M L M L

Lemon, 2014226 H L H H H M H

Lent, 2014227 H L L M H M H

Linde, 2012228 M L L M H L M

Llargues, 2011229 L L L M L M L

Lorentzen, 2009230 H H L H M H H

Lubans, 2016231 M L L H M M M

Ludwig, 2011232 L L L M L M L

Lv, 2014233 M H H H L M H

Madsen, 2013234 H H L H M L H

Madsen, 2015235 M L L M M H M

Mead, 2013236 M H L M L M M

Morrill, 2016237 L M H M M H H

Murphy, 2011238 H L H H M L H

Page 406: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-11

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Naylor, 2006239 H L H M H L H

Naylor, 2008240 H H L H L L H

Neumark-Sztainer, 2010241

M L L H L L M

Nicklas, 2017242 M L L H H L H

Ortega, 2016243 M M L M L M L

Pate, 2005244 L L L M M L L

Pbert, 2016245 M L L M L L L

Perry, 2004246 M H H H H H H

Ploeg, 2014247 L M L M M H M

Pope, 2016248 L M H H L M H

Ransley, 2007249 M L L M L M L

Reilly, 2006250 L L L L L H M

Reynolds, 2000251 L L L H L H H

Ridgers, 2010252 M M M M H H H

Rush, 2014253 M H H H M H H

Sallis, 2003254 M L L H M L M

Sharma, 2016255 M L L M L M L

Shive, 2006256 M M H H M H H

Sigmund, 2012257 H M L M H H H

Simon, 2008258 M L L H L M M

Steenhuis, 2004259 M L H H L H H

Story, 2012260 L L L M L H M

Tarp, 2016261 M L L H L M M

Te Velde, 2008262 M H L M L M M

Van Cauwen, 2012263

H M L M L H H

Page 407: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-12

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

Verloigne, 2015264 H M H M L H H

Waters, 2017265 H L L M M H H

Wells, 2014266 M L L M M M L

Wendel, 2016267 M L L M L H M

Whitt-Glover, 2011268

L H L M H M M

Williamson, 2012269 L L L H L L M

Wilson, 2015270 M L L M M L L

Wright, 2012271 M L L M M L L

Wright, 2013272 M L L H L M M

Yildirim, 2014273 M L L M L H M

Zhou, 2014274 L M M M L L L

Other Study Design

Ashfield-Watt, 2007275

M M H H M L H

Blake, 2013276 M M L H H M H

Brownson, 2004277 M L L H L H H

Brownson, 2005278 L M L H L H H

Brusseau, 2016279 M M H M L H H

Cheadle, 2012280 H M H M H H H

De Cocker, 2011281 H H L M L H H

Page 408: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-13

Author, year Selection Bias

Study Design

Con-founders

Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall Rating

de Silva-Sanigorski, 2011282

H H H H M H H

Geaney, 2010283 H H H M L H H

Gebel, 2011284 H M L M M M M

Heelan, 2009285 M M H H L H H

Huberty, 2013286 H M H M L M H

Kim, 2012287 M M L H H M H

Magarey, 2013288 M M L M L H M

Naul, 2012289 H H H H H M H

Rogers, 2013290 H H H M H H H

Tomlin, 2012291 M M L H L H H

Vasquez, 2016292 M M L M M L L

Weaver, 2017293 L M L M L H M

Whaley, 2010294 L M L H L H H

EPHPP = Effective Public Health Practice Project; H = high risk of bias; L = low risk of bias; M = medium risk of bias

References for Appendix K

1. Anderson LM, Aycock KE, Mihalic CA, et al. Geographic differences in physical education and adolescent BMI: have legal mandates made a difference? J Sch Nurs. 2013 Feb;29(1):52-60. doi: 10.1177/1059840512453602. PMID: 22815346.

2. Anthamatten P, Brink L, Lampe S, et al. An assessment of schoolyard renovation strategies to encourage children's physical activity. Int J Behav Nutr Phys Act. 2011 Apr 09;8:27. doi: 10.1186/1479-5868-8-27. PMID: 21477325.

3. Azevedo LB, Burges Watson D, Haighton C, et al. The effect of dance mat exergaming systems on physical activity and health-related outcomes in secondary schools: results from a natural experiment. BMC Public Health. 2014 Sep 12;14:951. doi: 10.1186/1471-2458-14-951. PMID: 25217144.

4. Barnidge EK, Hipp PR, Estlund A, et al. Association between community garden participation and fruit and vegetable consumption in rural Missouri. Int J Behav Nutr Phys Act. 2013 Nov 19;10:128. doi:

Page 409: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-14

10.1186/1479-5868-10-128. PMID: 24252563.

5. Barroso CS, Kelder SH, Springer AE, et al. Senate Bill 42: implementation and impact on physical activity in middle schools. J Adolesc Health. 2009 Sep;45(3 Suppl):S82-90. doi: 10.1016/j.jadohealth.2009.06.017. PMID: 19699442.

6. Bauhoff S. The effect of school district nutrition policies on dietary intake and overweight: a synthetic control approach. Econ Hum Biol. 2014 Jan;12:45-55. doi: 10.1016/j.ehb.2013.06.001. PMID: 23891422.

7. Bauman A, McLean G, Hurdle D, et al. Evaluation of the national 'Push Play' campaign in New Zealand--creating population awareness of physical activity. N Z Med J. 2003 Aug 08;116(1179):U535. PMID: 14513082.

8. Bere E, Hilsen M, Klepp KI. Effect of the nationwide free school fruit scheme in Norway. Br J Nutr. 2010 Aug;104(4):589-94. doi: 10.1017/s0007114510000814. PMID: 20350345.

9. Berger-Jenkins E, Rausch J, Okah E, et al. Evaluation of a Coordinated School-Based Obesity Prevention Program in a Hispanic Community: Choosing Healthy and Active Lifestyles for Kids/Healthy Schools Healthy Families. American Journal of Health Education. 2014;45(5):261-70. doi: 10.1080/19325037.2014.932724. PMID: 103883129. Language: English. Entry Date: 20140909. Revision Date: 20150820. Publication Type: Journal Article.

10. Bolton KA, Kremer P, Gibbs L, et al. The outcomes of health-promoting communities: being active eating well initiative-a community-based obesity prevention intervention in Victoria, Australia. Int J Obes (Lond). 2017 Apr 25doi: 10.1038/ijo.2017.73. PMID: 28321132.

11. Bowling AB, Moretti M, Ringelheim K, et al. Healthy Foods, Healthy Families: combining

incentives and exposure interventions at urban farmers' markets to improve nutrition among recipients of US federal food assistance. Health Promot Perspect. 2016;6(1):10-6. doi: 10.15171/hpp.2016.02. PMID: 27123431.

12. Branas CC, Cheney RA, MacDonald JM, et al. A difference-in-differences analysis of health, safety, and greening vacant urban space. Am J Epidemiol. 2011 Dec 01;174(11):1296-306. doi: 10.1093/aje/kwr273. PMID: 22079788.

13. Brown AL, Khattak AJ, Rodriguez DA. Neighbourhood Types, Travel and Body Mass: A Study of New Urbanist and Suburban Neighbourhoods in the US. Urban Stud. 2008;45(4):963-88.

14. Brown BB, Werner CM, Tribby CP, et al. Transit Use, Physical Activity, and Body Mass Index Changes: Objective Measures Associated With Complete Street Light-Rail Construction. Am J Public Health. 2015 Jul;105(7):1468-74. doi: 10.2105/ajph.2015.302561. PMID: 25973829.

15. Brown BB, Smith KR, Tharp D, et al. A Complete Street Intervention for Walking to Transit, Nontransit Walking, and Bicycling: A Quasi-Experimental Demonstration of Increased Use. J Phys Act Health. 2016 Nov;13(11):1210-9. doi: 10.1123/jpah.2016-0066. PMID: 27334024.

16. Brown BB, Werner CM, Smith KR, et al. Environmental, behavioral, and psychological predictors of transit ridership: Evidence from a community intervention. J Environ Psychol. 2016 Jun;46:188-96. doi: 10.1016/j.jenvp.2016.04.010. PMID: 27672237.

17. Burke RM, Meyer A, Kay C, et al. A holistic school-based intervention for improving health-related knowledge, body composition, and fitness in elementary school students: an evaluation of the HealthMPowers program. Int J Behav Nutr Phys Act. 2014 Jun 26;11:78. doi: 10.1186/1479-5868-11-78. PMID: 24969618.

Page 410: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-15

18. Buscail C, Menai M, Salanave B, et al. Promoting physical activity in a low-income neighborhood of the Paris suburb of Saint-Denis: effects of a community-based intervention to increase physical activity. BMC Public Health. 2016 Jul 29;16:667. doi: 10.1186/s12889-016-3360-y. PMID: 27473296.

19. Caldwell EM, Miller Kobayashi M, DuBow WM, et al. Perceived access to fruits and vegetables associated with increased consumption. Public Health Nutr. 2009 Oct;12(10):1743-50. doi: 10.1017/s1368980008004308. PMID: 19105861.

20. Calise TV, Heeren T, DeJong W, et al. Do neighborhoods make people active, or do people make active neighborhoods? Evidence from a planned community in Austin, Texas. Prev Chronic Dis. 2013 Jun 20;10:E102. doi: 10.5888/pcd10.120119. PMID: 23786909.

21. Camacho-Rivera M, Rosenbaum E, Yama C, et al. Low-Income Housing Rental Assistance, Perceptions of Neighborhood Food Environment, and Dietary Patterns among Latino Adults: the AHOME Study. J Racial Ethn Health Disparities. 2017 Jun;4(3):346-53. doi: 10.1007/s40615-016-0234-z. PMID: 27129854.

22. Capogrossi K, You W. The Influence of School Nutrition Programs on the Weight of Low-Income Children: A Treatment Effect Analysis. Health Econ. 2016 Jul 6doi: 10.1002/hec.3378. PMID: 27381591.

23. Cawley J, Meyerhoefer C, Newhouse D. The impact of state physical education requirements on youth physical activity and overweight. Health Econ. 2007;16(12):1287-301. doi: 10.1002/hec.1218. PMID: 2007-19974-001.

24. Cawley J, Meyerhoefer C, Newhouse D. The Correlation of Youth Physical Activity with State Policies. Contemporary Economic Policy. 2007;25(4):506-17. doi: http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291465-7287. PMID:

0952344 Alternate Accession Number: EP27555031.

25. Chen C, Chou S-Y, Thornton RJ. The Effect of Household Technology on Weight and Health Outcomes among Chinese Adults: Evidence from China's 'Home Appliances Going to the Countryside' Policy. Journal of Human Capital. 2015 Fall;9(3):364-401. doi: http://www.jstor.org/action/showPublication?journalCode=jhumancapital. PMID: 1541217.

26. Cleland V, Dwyer T, Blizzard L, et al. The provision of compulsory school physical activity: associations with physical activity, fitness and overweight in childhood and twenty years later. Int J Behav Nutr Phys Act. 2008 Feb 29;5:14. doi: 10.1186/1479-5868-5-14. PMID: 18312621.

27. Coffield JE, Metos JM, Utz RL, et al. A multivariate analysis of federally mandated school wellness policies on adolescent obesity. J Adolesc Health. 2011 Oct;49(4):363-70. doi: 10.1016/j.jadohealth.2011.01.010. PMID: 21939866.

28. Cohen DA, Marsh T, Williamson S, et al. Impact and cost-effectiveness of family Fitness Zones: a natural experiment in urban public parks. Health Place. 2012 Jan;18(1):39-45. doi: 10.1016/j.healthplace.2011.09.008. PMID: 22243905.

29. Coyle KK, Potter S, Schneider D, et al. Distributing free fresh fruit and vegetables at school: results of a pilot outcome evaluation. Public Health Rep. 2009 Sep-Oct;124(5):660-9. doi: 10.1177/003335490912400508. PMID: 19753944.

30. Cradock AL, McHugh A, Mont-Ferguson H, et al. Effect of school district policy change on consumption of sugar-sweetened beverages among high school students, Boston, Massachusetts, 2004-2006. Prev Chronic Dis. 2011 Jul;8(4):A74. PMID: 21672398.

Page 411: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-16

31. Cradock AL, Barrett JL, Carter J, et al. Impact of the Boston Active School Day policy to promote physical activity among children. Am J Health Promot. 2014 Jan-Feb;28(3 Suppl):S54-64. doi: 10.4278/ajhp.130430-QUAN-204. PMID: 24380467.

32. Cullen KW, Watson K, Zakeri I, et al. Exploring changes in middle-school student lunch consumption after local school food service policy modifications. Public Health Nutr. 2006 Sep;9(6):814-20. PMID: 16925889.

33. Cullen KW, Watson K, Zakeri I. Improvements in middle school student dietary intake after implementation of the Texas Public School Nutrition Policy. Am J Public Health. 2008 Jan;98(1):111-7. doi: 10.2105/ajph.2007.111765. PMID: 18048778.

34. Cummins S, Petticrew M, Higgins C, et al. Large scale food retailing as an intervention for diet and health: quasi-experimental evaluation of a natural experiment. J Epidemiol Community Health. 2005 Dec;59(12):1035-40. doi: 10.1136/jech.2004.029843. PMID: 16286490.

35. Cummins S, Findlay A, Petticrew CHM, et al. Reducing inequalities in health and diet: findings from a study on the impact of a food retail development. Environ Plann A 2008;40(2):402-22.

36. Cummins S, Flint E, Matthews SA. New neighborhood grocery store increased awareness of food access but did not alter dietary habits or obesity. Health Aff (Millwood). 2014 Feb;33(2):283-91. doi: 10.1377/hlthaff.2013.0512. PMID: 24493772.

37. Datar A, Nicosia N. The effect of state competitive food and beverage regulations on childhood overweight and obesity. Journal of Adolescent Health. 2016doi: 10.1016/j.jadohealth.2016.09.003. PMID: 2016-54655-001.

38. de Visser R, Sylvester R, Rogers R, et al. Changes in School Health Program Improve Middle School Students' Behaviors. Am J Health Behav. 2016 Sep;40(5):568-77. doi: 10.5993/ajhb.40.5.3. PMID: 27561859.

39. De Cocker KA, De Bourdeaudhuij IM, Brown WJ, et al. Effects of "10,000 steps Ghent": a whole-community intervention. Am J Prev Med. 2007 Dec;33(6):455-63. doi: 10.1016/j.amepre.2007.07.037. PMID: 18022061.

40. Dill J, McNeil N, Broach J, et al. Bicycle boulevards and changes in physical activity and active transportation: findings from a natural experiment. Prev Med. 2014 Dec;69 Suppl 1:S74-8. doi: 10.1016/j.ypmed.2014.10.006. PMID: 25456802.

41. Dubowitz T, Ghosh-Dastidar M, Cohen DA, et al. Diet And Perceptions Change With Supermarket Introduction In A Food Desert, But Not Because Of Supermarket Use. Health Aff (Millwood). 2015 Nov;34(11):1858-68. doi: 10.1377/hlthaff.2015.0667. PMID: 26526243.

42. Eagle TF, Gurm R, Smith CA, et al. A middle school intervention to improve health behaviors and reduce cardiac risk factors. Am J Med. 2013 Oct;126(10):903-8. doi: 10.1016/j.amjmed.2013.04.019. PMID: 23932159.

43. Elbel B, Moran A, Dixon LB, et al. Assessment of a government-subsidized supermarket in a high-need area on household food availability and children's dietary intakes. Public Health Nutr. 2015 Oct;18(15):2881-90. doi: 10.1017/s1368980015000282. PMID: 25714993.

44. Elbel B, Mijanovich T, Kiszko K, et al. The Introduction of a Supermarket via Tax-Credits in a Low-Income Area. Am J Health Promot. 2017 Jan;31(1):59-66. doi: 10.4278/ajhp.150217-QUAN-733. PMID: 26389982.

Page 412: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-17

45. Falbe J, Thompson HR, Becker CM, et al. Impact of the Berkeley Excise Tax on Sugar-Sweetened Beverage Consumption. Am J Public Health. 2016 Oct;106(10):1865-71. doi: 10.2105/ajph.2016.303362. PMID: 27552267.

46. Fitzpatrick C, Datta GD, Henderson M, et al. School food environments associated with adiposity in Canadian children. Int J Obes (Lond). 2017 Mar 14doi: 10.1038/ijo.2017.39. PMID: 28186100.

47. Flego A, Herbert J, Waters E, et al. Jamie's Ministry of Food: quasi-experimental evaluation of immediate and sustained impacts of a cooking skills program in Australia. PLoS One. 2014;9(12):e114673. doi: 10.1371/journal.pone.0114673. PMID: 25514531.

48. Fogarty AW, Antoniak M, Venn AJ, et al. Does participation in a population-based dietary intervention scheme have a lasting impact on fruit intake in young children? Int J Epidemiol. 2007 Oct;36(5):1080-5. doi: 10.1093/ije/dym133. PMID: 17602183.

49. Fox MK, Dodd AH, Wilson A, et al. Association between school food environment and practices and body mass index of US public school children. J Am Diet Assoc. 2009 Feb;109(2 Suppl):S108-17. doi: 10.1016/j.jada.2008.10.065. PMID: 19166665.

50. Frongillo EA, Fawcett SB, Ritchie LD, et al. Community Policies and Programs to Prevent Obesity and Child Adiposity. Am J Prev Med. 2017 Jul 05doi: 10.1016/j.amepre.2017.05.006. PMID: 28688728.

51. Fuller D, Gauvin L, Kestens Y, et al. Impact evaluation of a public bicycle share program on cycling: a case example of BIXI in Montreal, Quebec. Am J Public Health. 2013 Mar;103(3):e85-92. doi: 10.2105/ajph.2012.300917. PMID: 23327280.

52. Fung C, McIsaac JL, Kuhle S, et al. The impact of a population-level school food and nutrition policy on dietary intake and body

weights of Canadian children. Prev Med. 2013 Dec;57(6):934-40. doi: 10.1016/j.ypmed.2013.07.016. PMID: 23891787.

53. Gee KA. School-Based Body Mass Index Screening and Parental Notification in Late Adolescence: Evidence From Arkansas's Act 1220. J Adolesc Health. 2015 Sep;57(3):270-6. doi: 10.1016/j.jadohealth.2015.05.007. PMID: 26115907.

54. Gibson D. Long-term Food Stamp Program participation is positively related to simultaneous overweight in young daughters and obesity in mothers. J Nutr. 2006 Apr;136(4):1081-5. PMID: 16549483.

55. Giles-Corti B, Bull F, Knuiman M, et al. The influence of urban design on neighbourhood walking following residential relocation: longitudinal results from the RESIDE study. Soc Sci Med. 2013 Jan;77:20-30. doi: 10.1016/j.socscimed.2012.10.016. PMID: 23206559.

56. Gleason PM, Dodd AH. School Breakfast Program but Not School Lunch Program Participation Is Associated with Lower Body Mass Index. J Am Diet Assoc. 2009 Feb;109(2):S118-S28. doi: 10.1016/j.jada.2008.10.058. PMID: 19166666.

57. Goldsby TU, George BJ, Yeager VA, et al. Urban Park Development and Pediatric Obesity Rates: A Quasi-Experiment Using Electronic Health Record Data. Int J Environ Res Public Health. 2016 Apr 08;13(4):411. doi: 10.3390/ijerph13040411. PMID: 27070635.

58. Goodman A, Sahlqvist S, Ogilvie D. New walking and cycling routes and increased physical activity: one- and 2-year findings from the UK iConnect Study. Am J Public Health. 2014 Sep;104(9):e38-46. doi: 10.2105/ajph.2014.302059. PMID: 25033133.

59. Goodman A, van Sluijs EM, Ogilvie D. Impact of offering cycle training in schools

Page 413: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-18

upon cycling behaviour: a natural experimental study. Int J Behav Nutr Phys Act. 2016 Mar 08;13:34. doi: 10.1186/s12966-016-0356-z. PMID: 26956383.

60. Gorely T, Morris JG, Musson H, et al. Physical activity and body composition outcomes of the GreatFun2Run intervention at 20 month follow-up. Int J Behav Nutr Phys Act. 2011 Jul 18;8:74. doi: 10.1186/1479-5868-8-74. PMID: 21767356.

61. Gorham G, Dulin-Keita A, Risica PM, et al. Effectiveness of Fresh to You, a Discount Fresh Fruit and Vegetable Market in Low-Income Neighborhoods, on Children's Fruit and Vegetable Consumption, Rhode Island, 2010-2011. Prev Chronic Dis. 2015 Oct 15;12:E176. doi: 10.5888/pcd12.140583. PMID: 26469949.

62. Harding MC, Bott QD, Jonas CE. The Malaekahana Path: An Ecological Model-based Intervention for Increasing Walking in Rural Hawai'i. J Phys Act Health. 2017 Jul 06:1-11. doi: 10.1123/jpah.2017-0330. PMID: 28682662.

63. Heelan KA, Bartee RT, Nihiser A, et al. Healthier School Environment Leads to Decreases in Childhood Obesity: The Kearney Nebraska Story. Child Obes. 2015 Oct;11(5):600-7. doi: 10.1089/chi.2015.0005. PMID: 26440386.

64. Hennessy E, Oh A, Agurs-Collins T, et al. State-level school competitive food and beverage laws are associated with children's weight status. J Sch Health. 2014 Sep;84(9):609-16. doi: 10.1111/josh.12181. PMID: 25117896.

65. Herrick H, Thompson H, Kinder J, et al. Use of SPARK to promote after-school physical activity. J Sch Health. 2012 Oct;82(10):457-61. doi: 10.1111/j.1746-1561.2012.00722.x. PMID: 22954164.

66. Hilmers A, Chen TA, Dave JM, et al. Supplemental Nutrition Assistance Program participation did not help low income

Hispanic women in Texas meet the dietary guidelines. Prev Med. 2014 May;62:44-8. doi: 10.1016/j.ypmed.2014.01.016. PMID: 24530319.

67. Hobin E, So J, Rosella L, et al. Trajectories of objectively measured physical activity among secondary students in Canada in the context of a province-wide physical education policy: a longitudinal analysis. J Obes. 2014;2014:958645. doi: 10.1155/2014/958645. PMID: 24672714.

68. Hobin E, Erickson T, Comte M, et al. Examining the impact of a province-wide physical education policy on secondary students' physical activity as a natural experiment. Int J Behav Nutr Phys Act. 2017 Jul 19;14(1):98. doi: 10.1186/s12966-017-0550-7. PMID: 28724390.

69. Hoelscher D, Ory M, Dowdy D, et al. Effects of funding allocation for Safe Routes to School programs on active commuting to school and related behavioral, knowledge, and psychosocial outcomes: Results from the Texas Childhood Obesity Prevention Policy Evaluation (T-COPPE) study. Environ Behav. 2016;48(1):210-29. doi: 10.1177/0013916515613541. PMID: 2016-02263-012.

70. Hoelscher DM, Moag-Stahlberg A, Ellis K, et al. Evaluation of a student participatory, low-intensity program to improve school wellness environment and students' eating and activity behaviors. Int J Behav Nutr Phys Act. 2016 May 13;13:59. doi: 10.1186/s12966-016-0379-5. PMID: 27178056.

71. Howlett E, Davis C, Burton S. From food desert to food oasis: The potential influence of food retailers on childhood obesity rates. Journal of Business Ethics. 2016;139(2):215-24. doi: 10.1007/s10551-015-2605-5. PMID: 2015-12722-001.

72. Hu Y, van Lenthe FJ, Judge K, et al. Did the English strategy reduce inequalities in health? A difference-in-difference analysis comparing England with three other European countries. BMC Public Health. 2016 Aug 24;16(1):865. doi:

Page 414: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-19

10.1186/s12889-016-3505-z. PMID: 27558269.

73. Hughes RJ, Edwards KL, Clarke GP, et al. Childhood consumption of fruit and vegetables across England: a study of 2306 6-7-year-olds in 2007. Br J Nutr. 2012 Aug;108(4):733-42. doi: 10.1017/s0007114511005939. PMID: 22321148.

74. Hunter S, Leatherdale ST, Storey K, et al. A quasi-experimental examination of how school-based physical activity changes impact secondary school student moderate- to vigorous- intensity physical activity over time in the COMPASS study. Int J Behav Nutr Phys Act. 2016 Jul 29;13:86. doi: 10.1186/s12966-016-0411-9. PMID: 27473113.

75. Jennings A, Cassidy A, Winters T, et al. Positive effect of a targeted intervention to improve access and availability of fruit and vegetables in an area of deprivation. Health Place. 2012 Sep;18(5):1074-8. doi: 10.1016/j.healthplace.2012.05.001. PMID: 22705164.

76. Jia P, Li M, Xue H, et al. School environment and policies, child eating behavior and overweight/obesity in urban China: the childhood obesity study in China megacities. Int J Obes (Lond). 2017 May;41(5):813-9. doi: 10.1038/ijo.2017.2. PMID: 28074059.

77. Johnson R, Robertson W, Towey M, et al. Changes over time in mental well-being, fruit and vegetable consumption and physical activity in a community-based lifestyle intervention: a before and after study. Public Health. 2017 May;146:118-25. doi: 10.1016/j.puhe.2017.01.012. PMID: 28404463.

78. Just DR, Wansink B. School lunch debit card payment systems are associated with lower nutrition and higher calories. Obesity (Silver Spring). 2014 Jan;22(1):24-6. doi: 10.1002/oby.20591. PMID: 23929600.

79. Kern E, Chan NL, Fleming DW, et al. Declines in student obesity prevalence associated with a prevention initiative - King County, Washington, 2012. MMWR Morb Mortal Wkly Rep. 2014 Feb 21;63(7):155-7. PMID: 24553199.

80. Keyte J, Harris S, Margetts B, et al. Engagement with the National Healthy Schools Programme is associated with higher fruit and vegetable consumption in primary school children. J Hum Nutr Diet. 2012 Apr;25(2):155-60. doi: 10.1111/j.1365-277X.2011.01208.x. PMID: 22128770.

81. Kim J. Are physical education-related state policies and schools' physical education requirement related to children's physical activity and obesity? J Sch Health. 2012 Jun;82(6):268-76. doi: 10.1111/j.1746-1561.2012.00697.x. PMID: 22568462.

82. King MH, Lederer AM, Sovinski D, et al. Implementation and evaluation of the HEROES initiative: a tri-state coordinated school health program to reduce childhood obesity. Health Promot Pract. 2014 May;15(3):395-405. doi: 10.1177/1524839913512835. PMID: 24334542.

83. Kubik MY, Lytle LA, Story M. Schoolwide food practices are associated with body mass index in middle school students. Arch Pediatr Adolesc Med. 2005 Dec;159(12):1111-4. doi: 10.1001/archpedi.159.12.1111. PMID: 16330732.

84. Lachapelle U, Frank LD. Transit and health: mode of transport, employer-sponsored public transit pass programs, and physical activity. J Public Health Policy. 2009;30 Suppl 1:S73-94. doi: 10.1057/jphp.2008.52. PMID: 19190584.

85. LaRowe TL, Tomayko EJ, Meinen AM, et al. Active Early: one-year policy intervention to increase physical activity among early care and education programs in Wisconsin. BMC Public Health. 2016 Jul 20;16:607. doi: 10.1186/s12889-016-3198-3. PMID: 27439770.

Page 415: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-20

86. Leung CW, Blumenthal SJ, Hoffnagle EE, et al. Associations of food stamp participation with dietary quality and obesity in children. Pediatrics. 2013 Mar;131(3):463-72. doi: 10.1542/peds.2012-0889. PMID: 23439902.

87. Liao Y, Siegel PZ, Zhou H, et al. Reduced Disparity in Vegetable Consumption in 16 Disadvantaged Black Communities: A Successful 5-Year Community-Based Participatory Intervention. J Racial Ethn Health Disparities. 2015 Jun;2(2):211-8. doi: 10.1007/s40615-014-0065-8. PMID: 26150921.

88. Ling J, King KM, Speck BJ, et al. Preliminary assessment of a school-based healthy lifestyle intervention among rural elementary school children. J Sch Health. 2014 Apr;84(4):247-55. doi: 10.1111/josh.12143. PMID: 24617908.

89. Liu J, Kuo T, Jiang L, et al. Food and drink consumption among 1-5-year-old Los Angeles County children from households receiving dual SNAP and WIC v. only WIC benefits. Public Health Nutr. 2016 Sep 9:1-8. doi: 10.1017/s1368980016002329. PMID: 27609603.

90. MacDonald JM, Stokes RJ, Cohen DA, et al. The effect of light rail transit on body mass index and physical activity. Am J Prev Med. 2010 Aug;39(2):105-12. doi: 10.1016/j.amepre.2010.03.016. PMID: 20621257.

91. Maddock J, Takeuchi L, Nett B, et al. Evaluation of a statewide program to reduce chronic disease: The Healthy Hawaii Initiative, 2000-2004. Eval Program Plann. 2006;29(3):293-300. doi: 10.1016/j.evalprogplan.2005.12.007. PMID: 2006-11996-009.

92. Madsen KA. School-based body mass index screening and parent notification: a statewide natural experiment. Arch Pediatr Adolesc Med. 2011 Nov;165(11):987-92. doi: 10.1001/archpediatrics.2011.127. PMID: 21727262.

93. Madsen KA, Cotterman C, Crawford P, et al. Effect of the healthy schools program on

prevalence of overweight and obesity in California schools, 2006–2012. Preventing Chronic Disease: Public Health Research, Practice, and Policy. 2015;12 PMID: 2016-24674-001.

94. Malakellis M, Hoare E, Sanigorski A, et al. School-based systems change for obesity prevention in adolescents: outcomes of the Australian Capital Territory 'It's Your Move!'. Aust N Z J Public Health. 2017 Jul 27doi: 10.1111/1753-6405.12696. PMID: 28749562.

95. Masse LC, de Niet-Fitzgerald JE, Watts AW, et al. Associations between the school food environment, student consumption and body mass index of Canadian adolescents. Int J Behav Nutr Phys Act. 2014 Mar 26;11(1):29. doi: 10.1186/1479-5868-11-29. PMID: 24666770.

96. Miewald C, Holben D, Hall P. Role of a food box program in fruit and vegetable consumption and food security. Can J Diet Pract Res. 2012 Summer;73(2):59-65. doi: 10.3148/73.2.2012.59. PMID: 22668838.

97. Miller HJ, Tribby CP, Brown BB, et al. Public transit generates new physical activity: Evidence from individual GPS and accelerometer data before and after light rail construction in a neighborhood of Salt Lake City, Utah, USA. Health Place. 2015 Nov;36:8-17. doi: 10.1016/j.healthplace.2015.08.005.

98. Molitor F, Sugerman S, Yu H, et al. Reach of Supplemental Nutrition Assistance Program-Education (SNAP-Ed) interventions and nutrition and physical activity-related outcomes, California, 2011-2012. Prev Chronic Dis. 2015 Mar 12;12:E33. doi: 10.5888/pcd12.140449. PMID: 25764139.

99. Morton KL, Corder K, Suhrcke M, et al. School polices, programmes and facilities, and objectively measured sedentary time, LPA and MVPA: Associations in secondary school and over the transition from primary to secondary school. Int J Behav Nutr Phys Act. 2016;13 PMID: 2016-21082-001.

Page 416: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-21

100. Mullally ML, Taylor JP, Kuhle S, et al. A province-wide school nutrition policy and food consumption in elementary school children in Prince Edward Island. Can J Public Health. 2010 Jan-Feb;101(1):40-3. PMID: 20364537.

101. Mumford KG, Contant CK, Weissman J, et al. Changes in physical activity and travel behaviors in residents of a mixed-use development. Am J Prev Med. 2011 Nov;41(5):504-7. doi: 10.1016/j.amepre.2011.07.016. PMID: 22011422.

102. Nanney MS, MacLehose R, Kubik MY, et al. Recommended school policies are associated with student sugary drink and fruit and vegetable intake. Prev Med. 2014 May;62:179-81. doi: 10.1016/j.ypmed.2014.01.026. PMID: 24518003.

103. Nanney MS, MacLehose RF, Kubik MY, et al. School Obesity Prevention Policies and Practices in Minnesota and Student Outcomes: A Longitudinal Cohort Study. Am J Prev Med. 2016 Nov;51(5):656-63. doi: 10.1016/j.amepre.2016.05.008. PMID: 27320703.

104. Benjamin Neelon SE, Namenek Brouwer RJ, Ostbye T, et al. A community-based intervention increases physical activity and reduces obesity in school-age children in North Carolina. Child Obes. 2015 Jun;11(3):297-303. doi: 10.1089/chi.2014.0130. PMID: 25938983.

105. Nehme EK, Perez A, Ranjit N, et al. The Effect of New Shower Facilities on Physical Activity Behaviors of Employees: A Quasi-experiment. J Phys Act Health. 2017 Feb;14(2):98-107. doi: 10.1123/jpah.2015-0418. PMID: 27775466.

106. Nguyen BT, Powell LM. Supplemental nutrition assistance program participation and sugar-sweetened beverage consumption, overall and by source. Prev Med. 2015 Dec;81:82-6. doi: 10.1016/j.ypmed.2015.08.003. PMID: 26303370.

107. Oh AY, Hennessy E, McSpadden KE, et al. Contextual Influences on Weight Status Among Impoverished Adolescents: Neighborhood Amenities for Physical Activity and State Laws for Physical Education Time Requirements. J Phys Act Health. 2015 Jun;12(6):875-8. doi: 10.1123/jpah.2013-0303. PMID: 25109235.

108. Olsho LE, Payne GH, Walker DK, et al. Impacts of a farmers' market incentive programme on fruit and vegetable access, purchase and consumption. Public Health Nutr. 2015 Oct;18(15):2712-21. doi: 10.1017/s1368980015001056. PMID: 25919225.

109. Panter J, Heinen E, Mackett R, et al. Impact of New Transport Infrastructure on Walking, Cycling, and Physical Activity. Am J Prev Med. 2016 Feb;50(2):e45-53. doi: 10.1016/j.amepre.2015.09.021. PMID: 26585051.

110. Parsons WG, Garcia GM, Hoffman PK. Evaluating school wellness policy in curbing childhood obesity in Anchorage, Alaska. J Sch Nurs. 2014 Oct;30(5):324-31. doi: 10.1177/1059840513513155. PMID: 24316497.

111. Peterson KE, Spadano-Gasbarro JL, Greaney ML, et al. Three-Year Improvements in Weight Status and Weight-Related Behaviors in Middle School Students: The Healthy Choices Study. PLoS One. 2015;10(8):e0134470. doi: 10.1371/journal.pone.0134470. PMID: 26295837.

112. Powell LM, Chriqui J, Chaloupka FJ. Associations between state-level soda taxes and adolescent body mass index. J Adolesc Health. 2009 Sep;45(3 Suppl):S57-63. doi: 10.1016/j.jadohealth.2009.03.003. PMID: 19699437.

113. Quigg R, Reeder AI, Gray A, et al. The Effectiveness of a Community Playground Intervention. Journal of Urban Health-Bulletin of the New York Academy of

Page 417: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-22

Medicine. 2012 Feb;89(1):171-84. doi: 10.1007/s11524-011-9622-1.

114. Reger-Nash B, Bauman A, Booth-Butterfield S, et al. Wheeling walks: evaluation of a media-based community intervention. Fam Community Health. 2005 Jan-Mar;28(1):64-78. PMID: 15625507.

115. Reger-Nash B, Bauman A, Cooper L, et al. WV Walks: replication with expanded reach. J Phys Act Health. 2008 Jan;5(1):19-27. PMID: 18209251.

116. Restrepo BJ. Calorie Labeling in Chain Restaurants and Body Weight: Evidence from New York. Health Econ. 2016 Jul 24doi: 10.1002/hec.3389. PMID: 27451966.

117. Ridgers ND, Stratton G, Fairclough SJ, et al. Children's physical activity levels during school recess: a quasi-experimental intervention study. Int J Behav Nutr Phys Act. 2007 May 21;4:19. doi: 10.1186/1479-5868-4-19. PMID: 17517136.

118. Ridgers ND, Stratton G, Fairclough SJ, et al. Long-term effects of a playground markings and physical structures on children's recess physical activity levels. Prev Med. 2007 May;44(5):393-7. doi: 10.1016/j.ypmed.2007.01.009. PMID: 17335891.

119. Riis J, Grason H, Strobino D, et al. State school policies and youth obesity. Matern Child Health J. 2012 Apr;16 Suppl 1:S111-8. doi: 10.1007/s10995-012-1000-4. PMID: 22527761.

120. Ritchie LD, Rosen NJ, Fenton K, et al. School Breakfast Policy Is Associated with Dietary Intake of Fourth- and Fifth-Grade Students. J Acad Nutr Diet. 2016 Mar;116(3):449-57. doi: 10.1016/j.jand.2015.08.020. PMID: 26433452.

121. Robles B, Montes CE, Nobari TZ, et al. Dietary Behaviors among Public Health Center Clients with Electronic Benefit Transfer Access at Farmers' Markets. J Acad

Nutr Diet. 2017 Jan;117(1):58-68. doi: 10.1016/j.jand.2016.07.012. PMID: 27618576.

122. Rushakoff JA, Zoughbie DE, Bui N, et al. Evaluation of Healthy2Go: A country store transformation project to improve the food environment and consumer choices in Appalachian Kentucky. Prev Med Rep. 2017 Sep;7:187-92. doi: 10.1016/j.pmedr.2017.06.009. PMID: 28706778.

123. Sabia JJ, Nguyen TT, Rosenberg O. High School Physical Education Requirements and Youth Body Weight: New Evidence from the YRBS. Health Econ. 2016 Aug 30doi: 10.1002/hec.3399. PMID: 27576770.

124. Sadler RC, Gilliland JA, Arku G. A food retail-based intervention on food security and consumption. Int J Environ Res Public Health. 2013 Aug 05;10(8):3325-46. doi: 10.3390/ijerph10083325. PMID: 23921626.

125. Sanchez-Vaznaugh EV, Sanchez BN, Baek J, et al. 'Competitive' food and beverage policies: are they influencing childhood overweight trends? Health Aff (Millwood). 2010 Mar-Apr;29(3):436-46. doi: 10.1377/hlthaff.2009.0745. PMID: 20194985.

126. Schanzenbach D. Do School Lunches Contribute to Childhood Obesity? : Harris School of Public Policy Studies, University of Chicago, Working Papers: 0513; 2005.

127. Schwartz AE, Leardo M, Aneja S, et al. Effect of a School-Based Water Intervention on Child Body Mass Index and Obesity. JAMA Pediatr. 2016 Mar;170(3):220-6. doi: 10.1001/jamapediatrics.2015.3778. PMID: 26784336.

128. Sekhobo JP, Edmunds LS, Dalenius K, et al. Neighborhood disparities in prevalence of childhood obesity among low-income children before and after implementation of New York City child care regulations. Prev Chronic Dis. 2014;11:E181. doi: 10.5888/pcd11.140152. PMID: 25321632.

129. Slater S, Chriqui J, Chaloupka FJ, et al. Joint use policies: are they related to adolescent

Page 418: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-23

behavior? Prev Med. 2014 Dec;69 Suppl 1:S37-43. doi: 10.1016/j.ypmed.2014.08.032. PMID: 25199731.

130. Spence S, Delve J, Stamp E, et al. The impact of food and nutrient-based standards on primary school children's lunch and total dietary intake: a natural experimental evaluation of government policy in England. PLoS One. 2013;8(10):e78298. doi: 10.1371/journal.pone.0078298. PMID: 24205190.

131. Stephens RL, Xu Y, Lesesne CA, et al. Relationship between child care centers' compliance with physical activity regulations and children's physical activity, New York City, 2010. Prev Chronic Dis. 2014 Oct 16;11:E179. doi: 10.5888/pcd11.130432. PMID: 25321630.

132. Stratton G, Mullan E. The effect of multicolor playground markings on children's physical activity level during recess. Prev Med. 2005 Nov-Dec;41(5-6):828-33. doi: 10.1016/j.ypmed.2005.07.009. PMID: 16137756.

133. Sturm R, Powell LM, Chriqui JF, et al. Soda Taxes, Soft Drink Consumption, And Children's Body Mass Index. Health Affairs. 2010 May-Jun;29(5):1052-8. doi: 10.1377/hlthaff.2009.0061.

134. Sturm R, Hattori A. Diet and obesity in Los Angeles County 2007-2012: Is there a measurable effect of the 2008 "Fast-Food Ban"? Soc Sci Med. 2015 May;133:205-11. doi: 10.1016/j.socscimed.2015.03.004. PMID: 25779774.

135. Taber DR, Stevens J, Evenson KR, et al. State policies targeting junk food in schools: racial/ethnic differences in the effect of policy change on soda consumption. Am J Public Health. 2011 Sep;101(9):1769-75. doi: 10.2105/ajph.2011.300221. PMID: 21778484.

136. Taber DR, Chriqui JF, Chaloupka FJ. Differences in nutrient intake associated with state laws regarding fat, sugar, and caloric content of competitive foods. Arch Pediatr Adolesc Med. 2012 May;166(5):452-8. doi: 10.1001/archpediatrics.2011.1839. PMID: 22566546.

137. Taber DR, Chriqui JF, Perna FM, et al. Weight status among adolescents in States that govern competitive food nutrition content. Pediatrics. 2012 Sep;130(3):437-44. doi: 10.1542/peds.2011-3353. PMID: 22891223.

138. Taber DR, Chriqui JF, Powell LM, et al. Banning all sugar-sweetened beverages in middle schools: reduction of in-school access and purchasing but not overall consumption. Arch Pediatr Adolesc Med. 2012 Mar;166(3):256-62. doi: 10.1001/archpediatrics.2011.200. PMID: 22064875.

39. Taber DR, Chriqui JF, Perna FM, et al. Association between state physical education (PE) requirements and PE participation, physical activity, and body mass index change. Prev Med. 2013 Nov;57(5):629-33. doi: 10.1016/j.ypmed.2013.08.018. PMID: 23978523.

140. Tak NI, Te Velde SJ, Brug J. Ethnic differences in 1-year follow-up effect of the Dutch Schoolgruiten Project - promoting fruit and vegetable consumption among primary-school children. Public Health Nutr. 2007 Dec;10(12):1497-507. doi: 10.1017/s1368980007000456. PMID: 17610757.

141. Tak NI, Te Velde SJ, Brug J. Long-term effects of the Dutch Schoolgruiten Project--promoting fruit and vegetable consumption among primary-school children. Public Health Nutr. 2009 Aug;12(8):1213-23. doi: 10.1017/s1368980008003777. PMID: 18940029.

142. Tester JM, Leung CW, Crawford PB. Revised WIC Food Package and Children's Diet Quality. Pediatrics. 2016 May;137(5)doi: 10.1542/peds.2015-3557. PMID: 27244804.

Page 419: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-24

143. Toussaint LL, Housholder K, Janssen K, et al. Slowing BMI Growth Trajectories in Elementary School-Aged Children: The Northeast Iowa Food and Fitness Initiative. Fam Community Health. 2017 Jul/Sep;40(3):192-7. doi: 10.1097/fch.0000000000000151. PMID: 28525438.

144. Utter J, Denny S, Dyson B. School gardens and adolescent nutrition and BMI: Results from a national, multilevel study. Prev Med. 2016 Feb;83:1-4. doi: 10.1016/j.ypmed.2015.11.022. PMID: 26657347.

145. Vadiveloo MK, Dixon LB, Elbel B. Consumer purchasing patterns in response to calorie labeling legislation in New York City. Int J Behav Nutr Phys Act. 2011 May 27;8:51. doi: 10.1186/1479-5868-8-51. PMID: 21619632.

146. Veugelers PJ, Fitzgerald AL. Effectiveness of school programs in preventing childhood obesity: a multilevel comparison. Am J Public Health. 2005 Mar;95(3):432-5. doi: 10.2105/ajph.2004.045898. PMID: 15727972.

147. von Hippel PT, Bradbury WK. The effects of school physical education grants on obesity, fitness, and academic achievement. Prev Med. 2015 Sep;78:44-51. doi: 10.1016/j.ypmed.2015.06.011. PMID: 26163396.

148. Webb E, Netuveli G, Millett C. Free bus passes, use of public transport and obesity among older people in England. J Epidemiol Community Health. 2012 Feb;66(2):176-80. doi: 10.1136/jech.2011.133165. PMID: 21911850.

149. Webb E, Laverty A, Mindell J, et al. Free Bus Travel and Physical Activity, Gait Speed, and Adiposity in the English Longitudinal Study of Ageing. Am J Public Health. 2016 Jan;106(1):136-42. doi: 10.2105/ajph.2015.302907. PMID: 26562118

150. Wells L, Nelson M. The National School Fruit Scheme produces short-term but not

longer-term increases in fruit consumption in primary school children. Br J Nutr. 2005 Apr;93(4):537-42. PMID: 15946417.

151. West ST, Shores KA. The impacts of building a greenway on proximate residents' physical activity. J Phys Act Health. 2011 Nov;8(8):1092-7. PMID: 22039127.

152. Whetstone LM, Kolasa KM, Collier DN. Participation in community-originated interventions is associated with positive changes in weight status and health behaviors in youth. Am J Health Promot. 2012 Sep-Oct;27(1):10-6. doi: 10.4278/ajhp.100415-QUAN-117. PMID: 22950920.

153. Woodward-Lopez G, Gosliner W, Samuels SE, et al. Lessons learned from evaluations of California's statewide school nutrition standards. Am J Public Health. 2010 Nov;100(11):2137-45. doi: 10.2105/ajph.2010.193490. PMID: 20864696.

154. Wrigley N, Warm D, Margetts B. Deprivation, Diet, and Food-Retail Access: Findings from the Leeds ‘Food Deserts' Study. Environ Plann A 2003;35(1):151-88.

155. Zhu X, Lu Z, Yu C, et al. Walkable communities: impacts on residents’ physical and social health. World Health Des 2013;7:68-75.

156. Zhu X, Yu CY, Lee C, et al. A retrospective study on changes in residents' physical activities, social interactions, and neighborhood cohesion after moving to a walkable community. Prev Med. 2014 Dec;69 Suppl 1:S93-7. doi: 10.1016/j.ypmed.2014.08.013. PMID: 25158208.

157. Alaimo K, Oleksyk SC, Drzal NB, et al. Effects of changes in lunch-time competitive foods, nutrition practices, and nutrition policies on low-income middle-school children's diets. Child Obes. 2013

Page 420: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-25

Dec;9(6):509-23. doi: 10.1089/chi.2013.0052. PMID: 24215386.

158. Anderson JV, Bybee DI, Brown RM, et al. 5 a day fruit and vegetable intervention improves consumption in a low income population. J Am Diet Assoc. 2001 Feb;101(2):195-202. doi: 10.1016/s0002-8223(01)00052-9. PMID: 11271692.

159. Angelopoulos PD, Milionis HJ, Grammatikaki E, et al. Changes in BMI and blood pressure after a school based intervention: the CHILDREN study. Eur J Public Health. 2009 Jun;19(3):319-25. doi: 10.1093/eurpub/ckp004. PMID: 19208697.

160. Ask AS, Hernes S, Aarek I, et al. Serving of free school lunch to secondary-school pupils - a pilot study with health implications. Public Health Nutr. 2010 Feb;13(2):238-44. doi: 10.1017/s1368980009990772. PMID: 19650962.

161. Audrey S, Procter S, Cooper A, et al. Employer schemes to encourage walking to work: feasibility study incorporating an exploratory randomised controlled trial. Public Health Res. 2015 Mar;3(4)doi: 10.3310/phr03040. PMID: 25763450.

162. Ayala GX, Baquero B, Laraia BA, et al. Efficacy of a store-based environmental change intervention compared with a delayed treatment control condition on store customers' intake of fruits and vegetables. Public Health Nutr. 2013 Nov;16(11):1953-60. doi: 10.1017/s1368980013000955. PMID: 23561842.

163. Backman D, Gonzaga G, Sugerman S, et al. Effect of fresh fruit availability at worksites on the fruit and vegetable consumption of low-wage employees. J Nutr Educ Behav. 2011 Jul-Aug;43(4 Suppl 2):S113-21. doi: 10.1016/j.jneb.2011.04.003. PMID: 21683280.

164. Baker EA, Barnidge EK, Schootman M, et al. Adaptation of a Modified DASH Diet to a Rural African American Community Setting. Am J Prev Med. 2016 Sep 12doi: 10.1016/j.amepre.2016.07.014. PMID: 27633485.

165. Bastian KA, Maximova K, McGavock J, et al. Does School-Based Health Promotion Affect Physical Activity on Weekends? And, Does It Reach Those Students Most in Need of Health Promotion? PLoS One. 2015;10(10):e0137987. doi: 10.1371/journal.pone.0137987. PMID: 26488168.

166. Bere E, Veierod MB, Klepp KI. The Norwegian School Fruit Programme: evaluating paid vs. no-cost subscriptions. Prev Med. 2005 Aug;41(2):463-70. doi: 10.1016/j.ypmed.2004.11.024. PMID: 15917042.

167. Bere E, Veierod MB, Bjelland M, et al. Free school fruit--sustained effect 1 year later. Health Educ Res. 2006 Apr;21(2):268-75. doi: 10.1093/her/cyh063. PMID: 16219630.

168. Bere E, Veierod MB, Bjelland M, et al. Outcome and process evaluation of a Norwegian school-randomized fruit and vegetable intervention: Fruits and Vegetables Make the Marks (FVMM). Health Educ Res. 2006 Apr;21(2):258-67. doi: 10.1093/her/cyh062. PMID: 16219631.

169. Bere E, Veierod MB, Skare O, et al. Free School Fruit--sustained effect three years later. Int J Behav Nutr Phys Act. 2007 Feb 19;4:5. doi: 10.1186/1479-5868-4-5. PMID: 17309800.

170. Beresford SA, Thompson B, Bishop S, et al. Long-term fruit and vegetable change in worksites: Seattle 5 a Day follow-up. Am J Health Behav. 2010 Nov-Dec;34(6):707-20. PMID: 20604696.

171. Blum JE, Davee AM, Beaudoin CM, et al. Reduced availability of sugar-sweetened beverages and diet soda has a limited impact on beverage consumption patterns in Maine high school youth. J Nutr Educ Behav. 2008 Nov-Dec;40(6):341-7. doi: 10.1016/j.jneb.2007.12.004. PMID: 18984489.

172. Bonsergent E, Agrinier N, Thilly N, et al. Overweight and obesity prevention for

Page 421: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-26

adolescents: a cluster randomized controlled trial in a school setting. Am J Prev Med. 2013 Jan;44(1):30-9. doi: 10.1016/j.amepre.2012.09.055. PMID: 23253647.

173. Bonvin A, Barral J, Kakebeeke TH, et al. Effect of a governmentally-led physical activity program on motor skills in young children attending child care centers: a cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2013 Jul 08;10:90. doi: 10.1186/1479-5868-10-90. PMID: 23835207.

174. Busch V, De Leeuw JR, Zuithoff NP, et al. A Controlled Health Promoting School Study in the Netherlands: Effects After 1 and 2 Years of Intervention. Health Promot Pract. 2015 Jul;16(4):592-600. doi: 10.1177/1524839914566272. PMID: 25566994.

175. Caballero B, Clay T, Davis SM, et al. Pathways: a school-based, randomized controlled trial for the prevention of obesity in American Indian schoolchildren. Am J Clin Nutr. 2003 Nov;78(5):1030-8. PMID: 14594792.

176. Chomitz VR, McGowan RJ, Wendel JM, et al. Healthy Living Cambridge Kids: a community-based participatory effort to promote healthy weight and fitness. Obesity (Silver Spring). 2010 Feb;18 Suppl 1:S45-53. doi: 10.1038/oby.2009.431. PMID: 20107461.

177. Cochrane T, Davey R, Iqbal Z, et al. NHS health checks through general practice: randomised trial of population cardiovascular risk reduction. BMC Public Health. 2012 Nov 01;12:944. doi: 10.1186/1471-2458-12-944. PMID: 23116213.

178. Cohen JF, Kraak VI, Choumenkovitch SF, et al. The CHANGE study: a healthy-lifestyles intervention to improve rural children's diet quality. J Acad Nutr Diet. 2014 Jan;114(1):48-53. doi: 10.1016/j.jand.2013.08.014. PMID: 24126295.

179. Coleman KJ, Shordon M, Caparosa SL, et al. The healthy options for nutrition environments in schools (Healthy ONES) group randomized trial: using implementation models to change nutrition policy and environments in low income schools. Int J Behav Nutr Phys Act. 2012 Jun 27;9:80. doi: 10.1186/1479-5868-9-80. PMID: 22734945.

180. Cortinez-O'Ryan A, Albagli A, Sadarangani KP, et al. Reclaiming streets for outdoor play: A process and impact evaluation of "Juega en tu Barrio" (Play in your Neighborhood), an intervention to increase physical activity and opportunities for play. PLoS One. 2017;12(7):e0180172. doi: 10.1371/journal.pone.0180172. PMID: 28671984.

181. Crespo NC, Elder JP, Ayala GX, et al. Results of a multi-level intervention to prevent and control childhood obesity among Latino children: the Aventuras Para Ninos Study. Ann Behav Med. 2012 Feb;43(1):84-100. doi: 10.1007/s12160-011-9332-7. PMID: 22215470.

182. Day ME, Strange KS, McKay HA, et al. Action schools! BC--Healthy Eating: effects of a whole-school model to modifying eating behaviours of elementary school children. Can J Public Health. 2008 Jul-Aug;99(4):328-31. PMID: 18767281.

183. de Meij JS, Chinapaw MJ, van Stralen MM, et al. Effectiveness of JUMP-in, a Dutch primary school-based community intervention aimed at the promotion of physical activity. Br J Sports Med. 2011 Oct;45(13):1052-7. doi: 10.1136/bjsm.2010.075531. PMID: 21112875.

184. De Coen V, De Bourdeaudhuij I, Vereecken C, et al. Effects of a 2-year healthy eating and physical activity intervention for 3-6-year-olds in communities of high and low socio-economic status: the POP (Prevention of Overweight among Pre-school and school children) project. Public Health Nutr. 2012 Sep;15(9):1737-45. doi: 10.1017/s1368980012000687.

Page 422: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-27

185. de Greeff JW, Hartman E, Mullender-Wijnsma MJ, et al. Effect of Physically Active Academic Lessons on Body Mass Index and Physical Fitness in Primary School Children. J Sch Health. 2016 May;86(5):346-52. doi: 10.1111/josh.12384. PMID: 27040472.

186. Dunton G, Ebin VJ, Efrat MW, et al. The Use of Refundable Tax Credits to Increase Low-Income Children's After-School Physical Activity Level. J Phys Act Health. 2015 Jun;12(6):840-53. doi: 10.1123/jpah.2014-0058. PMID: 25184738.

187. Dzewaltowski DA, Estabrooks PA, Welk G, et al. Healthy youth places: a randomized controlled trial to determine the effectiveness of facilitating adult and youth leaders to promote physical activity and fruit and vegetable consumption in middle schools. Health Educ Behav. 2009 Jun;36(3):583-600. doi: 10.1177/1090198108314619. PMID: 18469366.

188. Economos CD, Hyatt RR, Goldberg JP, et al. A community intervention reduces BMI z-score in children: Shape Up Somerville first year results. Obesity (Silver Spring). 2007 May;15(5):1325-36. doi: 10.1038/oby.2007.155. PMID: 17495210.

189. Elinder LS, Heinemans N, Hagberg J, et al. A participatory and capacity-building approach to healthy eating and physical activity—SCIP-school: A 2-year controlled trial. Int J Behav Nutr Phys Act. 2012;9doi: 10.1186/1479-5868-9-145. PMID: 2013-04234-001.

190. Eriksen K, Haraldsdottir J, Pederson R, et al. Effect of a fruit and vegetable subscription in Danish schools. Public Health Nutr. 2003 Feb;6(1):57-63. doi: 10.1079/phn2002356. PMID: 12581466.

191. Ermetici F, Zelaschi RF, Briganti S, et al. Association between a school-based intervention and adiposity outcomes in adolescents: The Italian "EAT" project. Obesity (Silver Spring). 2016

Mar;24(3):687-95. doi: 10.1002/oby.21365. PMID: 26833570.

192. Esquivel M, Nigg CR, Fialkowski MK, et al. Head Start Wellness Policy Intervention in Hawaii: A Project of the Children's Healthy Living Program. Child Obes. 2016 Feb;12(1):26-32. doi: 10.1089/chi.2015.0071. PMID: 26771119.

193. Evans CE, Ransley JK, Christian MS, et al. A cluster-randomised controlled trial of a school-based fruit and vegetable intervention: Project Tomato. Public Health Nutr. 2013 Jun;16(6):1073-81. doi: 10.1017/s1368980012005290. PMID: 23237386.

194. Fairclough SJ, McGrane B, Sanders G, et al. A non-equivalent group pilot trial of a school-based physical activity and fitness intervention for 10-11 year old english children: born to move. BMC Public Health. 2016 Aug 24;16(1):861. doi: 10.1186/s12889-016-3550-7. PMID: 27553010.

195. Farley TA, Meriwether RA, Baker ET, et al. Safe play spaces to promote physical activity in inner-city children: results from a pilot study of an environmental intervention. Am J Public Health. 2007 Sep;97(9):1625-31. doi: 10.2105/ajph.2006.092692. PMID: 17666701.

196. Farmer VL, Williams SM, Mann JI, et al. The effect of increasing risk and challenge in the school playground on physical activity and weight in children: a cluster randomised controlled trial (PLAY). Int J Obes (Lond). 2017 May;41(5):793-800. doi: 10.1038/ijo.2017.41. PMID: 28186099.

197. Finch M, Wolfenden L, Morgan PJ, et al. A cluster randomized trial of a multi-level intervention, delivered by service staff, to increase physical activity of children attending center-based childcare. Prev Med. 2014 Jan;58:9-16. doi: 10.1016/j.ypmed.2013.10.004. PMID: 24145204.

198. Foster GD, Sherman S, Borradaile KE, et al. A policy-based school intervention to prevent

Page 423: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-28

overweight and obesity. Pediatrics. 2008 Apr;121(4):e794-802. doi: 10.1542/peds.2007-1365. PMID: 18381508.

199. French SA, Harnack LJ, Hannan PJ, et al. Worksite environment intervention to prevent obesity among metropolitan transit workers. Prev Med. 2010 Apr;50(4):180-5. doi: 10.1016/j.ypmed.2010.01.002. PMID: 20079369.

200. Fu Y, Gao Z, Hannon JC, et al. Effect of the SPARK Program on Physical Activity, Cardiorespiratory Endurance, and Motivation in Middle-School Students. J Phys Act Health. 2016 May;13(5):534-42. doi: 10.1123/jpah.2015-0351. PMID: 26528889.

201. Gatto NM, Martinez LC, Spruijt-Metz D, et al. LA sprouts randomized controlled nutrition, cooking and gardening programme reduces obesity and metabolic risk in Hispanic/Latino youth. Pediatr Obes. 2017 Feb;12(1):28-37. doi: 10.1111/ijpo.12102. PMID: 26909882.

202. Geaney F, Kelly C, Di Marrazzo JS, et al. The effect of complex workplace dietary interventions on employees' dietary intakes, nutrition knowledge and health status: a cluster controlled trial. Prev Med. 2016 Aug;89:76-83. doi: 10.1016/j.ypmed.2016.05.005. PMID: 27208667.

203. Gittelsohn J, Vijayadeva V, Davison N, et al. A food store intervention trial improves caregiver psychosocial factors and children's dietary intake in Hawaii. Obesity (Silver Spring). 2010 Feb;18 Suppl 1:S84-90. doi: 10.1038/oby.2009.436. PMID: 20107467.

204. Gittelsohn J, Kim EM, He S, et al. A food store-based environmental intervention is associated with reduced BMI and improved psychosocial factors and food-related behaviors on the Navajo nation. J Nutr. 2013 Sep;143(9):1494-500. doi: 10.3945/jn.112.165266. PMID: 23864511.

205. Goetzel RZ, Roemer EC, Pei X, et al. Second-year results of an obesity prevention program at the Dow Chemical Company. J Occup Environ Med. 2010 Mar;52(3):291-302. doi: 10.1097/JOM.0b013e3181d46f0b. PMID: 20190646.

206. Gustat J, Rice J, Parker KM, et al. Effect of changes to the neighborhood built environment on physical activity in a low-income African American neighborhood. Prev Chronic Dis. 2012;9:E57. PMID: 22338597.

207. Haerens L, Deforche B, Maes L, et al. Body mass effects of a physical activity and healthy food intervention in middle schools. Obesity (Silver Spring). 2006 May;14(5):847-54. doi: 10.1038/oby.2006.98. PMID: 16855194.

208. Haerens L, De Bourdeaudhuij I, Maes L, et al. The effects of a middle-school healthy eating intervention on adolescents' fat and fruit intake and soft drinks consumption. Public Health Nutr. 2007 May;10(5):443-9. doi: 10.1017/s1368980007219652. PMID: 17411463.

209. Hardy LL, King L, Kelly B, et al. Munch and Move: evaluation of a preschool healthy eating and movement skill program. Int J Behav Nutr Phys Act. 2010 Nov 03;7:80. doi: 10.1186/1479-5868-7-80. PMID: 21047434.

210. He M, Beynon C, Sangster Bouck M, et al. Impact evaluation of the Northern Fruit and Vegetable Pilot Programme - a cluster-randomised controlled trial. Public Health Nutr. 2009 Nov;12(11):2199-208. doi: 10.1017/s1368980009005801. PMID: 19476675.

211. Hendy HM, Williams KE, Camise TS. Kid's Choice Program improves weight management behaviors and weight status in school children. Appetite. 2011 Apr;56(2):484-94. doi: 10.1016/j.appet.2011.01.024. PMID: 21277924.

212. Hoefkens C, Lachat C, Kolsteren P, et al. Posting point-of-purchase nutrition

Page 424: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-29

information in university canteens does not influence meal choice and nutrient intake. Am J Clin Nutr. 2011 Aug;94(2):562-70. doi: 10.3945/ajcn.111.013417. PMID: 21677060.

213. Hollis JL, Sutherland R, Campbell L, et al. Effects of a 'school-based' physical activity intervention on adiposity in adolescents from economically disadvantaged communities: secondary outcomes of the 'Physical Activity 4 Everyone' RCT. Int J Obes (Lond). 2016 Oct;40(10):1486-93. doi: 10.1038/ijo.2016.107. PMID: 27430652.

214. Huberty JL, Beets MW, Beighle A, et al. Environmental modifications to increase physical activity during recess: preliminary findings from ready for recess. J Phys Act Health. 2011 Sep;8 Suppl 2:S249-56. PMID: 21918239.

215. Jago R, McMurray RG, Drews KL, et al. HEALTHY intervention: fitness, physical activity, and metabolic syndrome results. Med Sci Sports Exerc. 2011 Aug;43(8):1513-22. doi: 10.1249/MSS.0b013e31820c9797. PMID: 21233778.

216. Janssen M, Twisk JW, Toussaint HM, et al. Effectiveness of the PLAYgrounds programme on PA levels during recess in 6-year-old to 12-year-old children. Br J Sports Med. 2015 Feb;49(4):259-64. doi: 10.1136/bjsports-2012-091517. PMID: 23293007.

217. Jones J, Wyse R, Finch M, et al. Effectiveness of an intervention to facilitate the implementation of healthy eating and physical activity policies and practices in childcare services: a randomised controlled trial. Implement Sci. 2015 Oct 25;10:147. doi: 10.1186/s13012-015-0340-z. PMID: 26498746.

218. Jordan KC, Erickson ED, Cox R, et al. Evaluation of the Gold Medal Schools program. J Am Diet Assoc. 2008 Nov;108(11):1916-20. doi: 10.1016/j.jada.2008.08.002. PMID: 18954584.

219. Jurg ME, Kremers SP, Candel MJ, et al. A controlled trial of a school-based

environmental intervention to improve physical activity in Dutch children: JUMP-in, kids in motion. Health Promot Int. 2006 Dec;21(4):320-30. doi: 10.1093/heapro/dal032. PMID: 16963784.

220. Kain J, Uauy R, Albala, et al. School-based obesity prevention in Chilean primary school children: methodology and evaluation of a controlled study. Int J Obes Relat Metab Disord. 2004 Apr;28(4):483-93. doi: 10.1038/sj.ijo.0802611. PMID: 14993915.

221. Kamada M, Kitayuguchi J, Inoue S, et al. A community-wide campaign to promote physical activity in middle-aged and elderly people: a cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2013 Apr 09;10:44. doi: 10.1186/1479-5868-10-44. PMID: 23570536.

222. Kastorini CM, Lykou A, Yannakoulia M, et al. The influence of a school-based intervention programme regarding adherence to a healthy diet in children and adolescents from disadvantaged areas in Greece: the DIATROFI study. J Epidemiol Community Health. 2016 Jul;70(7):671-7. doi: 10.1136/jech-2015-205680. PMID: 26763974.

223. Kloek GC, van Lenthe FJ, van Nierop PW, et al. Impact evaluation of a Dutch community intervention to improve health-related behaviour in deprived neighbourhoods. Health Place. 2006 Dec;12(4):665-77. doi: 10.1016/j.healthplace.2005.09.002. PMID: 16253541.

224. LaCaille LJ, Schultz JF, Goei R, et al. Go!: results from a quasi-experimental obesity prevention trial with hospital employees. BMC Public Health. 2016 Feb 19;16:171. doi: 10.1186/s12889-016-2828-0. PMID: 26893128.

225. Lemon SC, Zapka J, Li W, et al. Step ahead a worksite obesity prevention trial among hospital employees. Am J Prev Med. 2010 Jan;38(1):27-38. doi: 10.1016/j.amepre.2009.08.028. PMID: 20117554.

Page 425: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-30

226. Lemon SC, Wang ML, Wedick NM, et al. Weight gain prevention in the school worksite setting: results of a multi-level cluster randomized trial. Prev Med. 2014 Mar;60:41-7. doi: 10.1016/j.ypmed.2013.12.010. PMID: 24345602.

227. Lent MR, Vander Veur SS, McCoy TA, et al. A randomized controlled study of a healthy corner store initiative on the purchases of urban, low-income youth. Obesity (Silver Spring). 2014 Dec;22(12):2494-500. doi: 10.1002/oby.20878. PMID: 25311881.

228. Linde JA, Nygaard KE, MacLehose RF, et al. HealthWorks: results of a multi-component group-randomized worksite environmental intervention trial for weight gain prevention. Int J Behav Nutr Phys Act. 2012 Feb 16;9:14. doi: 10.1186/1479-5868-9-14. PMID: 22340088.

229. Llargues E, Franco R, Recasens A, et al. Assessment of a school-based intervention in eating habits and physical activity in school children: the AVall study. J Epidemiol Community Health. 2011 Oct;65(10):896-901. doi: 10.1136/jech.2009.102319.

230. Lorentzen C, Ommundsen Y, Jenum AK, et al. The "Romsas in Motion" community intervention: mediating effects of psychosocial factors on forward transition in the stages of change in physical activity. Health Educ Behav. 2009 Apr;36(2):348-65. doi: 10.1177/1090198107308372. PMID: 18065570.

231. Lubans DR, Smith JJ, Plotnikoff RC, et al. Assessing the sustained impact of a school-based obesity prevention program for adolescent boys: the ATLAS cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2016 Aug 20;13:92. doi: 10.1186/s12966-016-0420-8. PMID: 27542825.

232. Ludwig J, Sanbonmatsu L, Gennetian L, et al. Neighborhoods, obesity, and diabetes--a randomized social experiment. N Engl J Med. 2011 Oct 20;365(16):1509-19. doi: 10.1056/NEJMsa1103216. PMID: 22010917.

233. Lv J, Liu QM, Ren YJ, et al. A community-based multilevel intervention for smoking, physical activity and diet: short-term findings from the Community Interventions for Health programme in Hangzhou, China. J Epidemiol Community Health. 2014 Apr;68(4):333-9. doi: 10.1136/jech-2013-203356. PMID: 24297972.

234. Madsen K, Thompson H, Adkins A, et al. School-community partnerships: a cluster-randomized trial of an after-school soccer program. JAMA Pediatr. 2013 Apr;167(4):321-6. doi: 10.1001/jamapediatrics.2013.1071. PMID: 23440308.

235. Madsen K, Linchey J, Gerstein D, et al. Energy Balance 4 Kids with Play: Results from a Two-Year Cluster-Randomized Trial. Child Obes. 2015 Aug;11(4):375-83. doi: 10.1089/chi.2015.0002. PMID: 26061799.

236. Mead EL, Gittelsohn J, Roache C, et al. A community-based, environmental chronic disease prevention intervention to improve healthy eating psychosocial factors and behaviors in indigenous populations in the Canadian arctic. Health Education & Behavior. 2013;40(5):592-602. doi: 10.1177/1090198112467793. PMID: 2013-33808-011.

237. Morrill BA, Madden GJ, Wengreen HJ, et al. A Randomized Controlled Trial of the Food Dudes Program: Tangible Rewards are More Effective Than Social Rewards for Increasing Short- and Long-Term Fruit and Vegetable Consumption. J Acad Nutr Diet. 2016 Apr;116(4):618-29. doi: 10.1016/j.jand.2015.07.001. PMID: 26297598.

238. Murphy S, Moore GF, Tapper K, et al. Free healthy breakfasts in primary schools: a cluster randomised controlled trial of a policy intervention in Wales, UK. Public Health Nutr. 2011 Feb;14(2):219-26. doi: 10.1017/s1368980010001886. PMID: 20602868.

239. Naylor PJ, Macdonald HM, Zebedee JA, et al. Lessons learned from Action Schools! BC--an 'active school' model to promote

Page 426: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-31

physical activity in elementary schools. J Sci Med Sport. 2006 Oct;9(5):413-23. doi: 10.1016/j.jsams.2006.06.013. PMID: 16884957.

240. Naylor PJ, Macdonald HM, Warburton DE, et al. An active school model to promote physical activity in elementary schools: action schools! BC. Br J Sports Med. 2008 May;42(5):338-43. doi: 10.1136/bjsm.2007.042036. PMID: 18272538.

241. Neumark-Sztainer DR, Friend SE, Flattum CF, et al. New moves-preventing weight-related problems in adolescent girls a group-randomized study. Am J Prev Med. 2010 Nov;39(5):421-32. doi: 10.1016/j.amepre.2010.07.017. PMID: 20965379.

242. Nicklas T, Lopez S, Liu Y, et al. Motivational theater to increase consumption of vegetable dishes by preschool children. Int J Behav Nutr Phys Act. 2017 Feb 07;14(1):16. doi: 10.1186/s12966-017-0468-0. PMID: 28166788.

243. Ortega AN, Albert SL, Chan-Golston AM, et al. Substantial improvements not seen in health behaviors following corner store conversions in two Latino food swamps. BMC Public Health. 2016 May 11;16:389. doi: 10.1186/s12889-016-3074-1. PMID: 27169514.

244. Pate RR, Ward DS, Saunders RP, et al. Promotion of physical activity among high-school girls: a randomized controlled trial. Am J Public Health. 2005 Sep;95(9):1582-7. doi: 10.2105/ajph.2004.045807. PMID: 16118370.

245. Pbert L, Druker S, Barton B, et al. A School-Based Program for Overweight and Obese Adolescents: A Randomized Controlled Trial. J Sch Health. 2016 Oct;86(10):699-708. doi: 10.1111/josh.12428. PMID: 27619760.

246. Perry CL, Bishop DB, Taylor GL, et al. A randomized school trial of environmental

strategies to encourage fruit and vegetable consumption among children. Health Educ Behav. 2004 Feb;31(1):65-76. PMID: 14768658.

247. Vander Ploeg KA, McGavock J, Maximova K, et al. School-based health promotion and physical activity during and after school hours. Pediatrics. 2014 Feb;133(2):e371-8. doi: 10.1542/peds.2013-2383. PMID: 24420806.

248. Pope M. Preventing Weight Gain in Children Who Are School Age and African-American. Pediatr Phys Ther. 2016 Summer;28(2):207-16. doi: 10.1097/pep.0000000000000243. PMID: 26914717.

249. Ransley JK, Greenwood DC, Cade JE, et al. Does the school fruit and vegetable scheme improve children's diet? A non-randomised controlled trial. J Epidemiol Community Health. 2007 Aug;61(8):699-703. doi: 10.1136/jech.2006.052696. PMID: 17630369.

250. Reilly JJ, Kelly L, Montgomery C, et al. Physical activity to prevent obesity in young children: cluster randomised controlled trial. BMJ. 2006 Nov 18;333(7577):1041. doi: 10.1136/bmj.38979.623773.55. PMID: 17028105.

251. Reynolds KD, Franklin FA, Binkley D, et al. Increasing the fruit and vegetable consumption of fourth-graders: results from the high 5 project. Prev Med. 2000 Apr;30(4):309-19. doi: 10.1006/pmed.1999.0630. PMID: 10731460.

252. Ridgers ND, Fairclough SJ, Stratton G. Twelve-month effects of a playground intervention on children's morning and lunchtime recess physical activity levels. J Phys Act Health. 2010 Mar;7(2):167-75. PMID: 20484755.

253. Rush E, McLennan S, Obolonkin V, et al. Project Energize: whole-region primary school nutrition and physical activity programme; evaluation of body size and fitness 5 years after the randomised controlled trial. Br J Nutr. 2014 Jan 28;111(2):363-71. doi:

Page 427: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-32

10.1017/s0007114513002316. PMID: 23867069.

254. Sallis JF, McKenzie TL, Conway TL, et al. Environmental interventions for eating and physical activity: a randomized controlled trial in middle schools. Am J Prev Med. 2003 Apr;24(3):209-17. PMID: 12657338.

255. Sharma SV, Markham C, Chow J, et al. Evaluating a school-based fruit and vegetable co-op in low-income children: A quasi-experimental study. Prev Med. 2016 Oct;91:8-17. doi: 10.1016/j.ypmed.2016.07.022. PMID: 27471022.

256. Shive SE, Morris MN. Evaluation of the energize your life! Social marketing campaign pilot study to increase fruit intake among community college students. J Am Coll Health. 2006 Jul-Aug;55(1):33-9. doi: 10.3200/jach.55.1.33-40. PMID: 16889313.

257. Sigmund E, El Ansari W, Sigmundova D. Does school-based physical activity decrease overweight and obesity in children aged 6-9 years? A two-year non-randomized longitudinal intervention study in the Czech Republic. BMC Public Health. 2012 Jul 29;12:570. doi: 10.1186/1471-2458-12-570. PMID: 22892226.

258. Simon C, Schweitzer B, Oujaa M, et al. Successful overweight prevention in adolescents by increasing physical activity: a 4-year randomized controlled intervention. Int J Obes (Lond). 2008 Oct;32(10):1489-98. doi: 10.1038/ijo.2008.99. PMID: 18626482.

259. Steenhuis I, Van Assema P, Van Breukelen G, et al. The impact of educational and environmental interventions in Dutch worksite cafeterias. Health Promot Int. 2004 Sep;19(3):335-43. doi: 10.1093/heapro/dah307. PMID: 15306618.

260. Story M, Hannan PJ, Fulkerson JA, et al. Bright Start: Description and main outcomes from a group-randomized obesity prevention trial in American Indian children. Obesity

(Silver Spring). 2012 Nov;20(11):2241-9. doi: 10.1038/oby.2012.89. PMID: 22513491.

261. Tarp J, Domazet SL, Froberg K, et al. Effectiveness of a School-Based Physical Activity Intervention on Cognitive Performance in Danish Adolescents: LCoMotion-Learning, Cognition and Motion - A Cluster Randomized Controlled Trial. PLoS One. 2016;11(6):e0158087. doi: 10.1371/journal.pone.0158087. PMID: 27341346.

262. Te Velde SJ, Brug J, Wind M, et al. Effects of a comprehensive fruit- and vegetable-promoting school-based intervention in three European countries: the Pro Children Study. Br J Nutr. 2008 Apr;99(4):893-903. doi: 10.1017/s000711450782513x. PMID: 17953787.

263. Van Cauwenberghe E, De Bourdeaudhuij I, Maes L, et al. Efficacy and feasibility of lowering playground density to promote physical activity and to discourage sedentary time during recess at preschool: a pilot study. Prev Med. 2012 Oct;55(4):319-21. doi: 10.1016/j.ypmed.2012.07.014. PMID: 22846504.

264. Verloigne M, Ahrens W, De Henauw S, et al. Process evaluation of the IDEFICS school intervention: putting the evaluation of the effect on children's objectively measured physical activity and sedentary time in context. Obes Rev. 2015 Dec;16 Suppl 2:89-102. doi: 10.1111/obr.12353. PMID: 26707019.

265. Waters E, Gibbs L, Tadic M, et al. Cluster randomised trial of a school-community child health promotion and obesity prevention intervention: findings from the evaluation of fun 'n healthy in Moreland! BMC Public Health. 2017 Aug 03;18(1):92. doi: 10.1186/s12889-017-4625-9. PMID: 28774278.

266. Wells NM, Myers BM, Henderson CR, Jr. School gardens and physical activity: a randomized controlled trial of low-income elementary schools. Prev Med. 2014 Dec;69

Page 428: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-33

Suppl 1:S27-33. doi: 10.1016/j.ypmed.2014.10.012. PMID: 25456803.

267. Wendel ML, Benden ME, Zhao H, et al. Stand-Biased Versus Seated Classrooms and Childhood Obesity: A Randomized Experiment in Texas. Am J Public Health. 2016 Oct;106(10):1849-54. doi: 10.2105/ajph.2016.303323. PMID: 27552276.

268. Whitt-Glover MC, Ham SA, Yancey AK. Instant Recess(R): a practical tool for increasing physical activity during the school day. Prog Community Health Partnersh. 2011 Fall;5(3):289-97. doi: 10.1353/cpr.2011.0031. PMID: 22080777.

269. Williamson DA, Champagne CM, Harsha DW, et al. Effect of an environmental school-based obesity prevention program on changes in body fat and body weight: a randomized trial. Obesity (Silver Spring). 2012 Aug;20(8):1653-61. doi: 10.1038/oby.2012.60. PMID: 22402733.

270. Wilson DK, Van Horn ML, Siceloff ER, et al. The Results of the "Positive Action for Today's Health" (PATH) Trial for Increasing Walking and Physical Activity in Underserved African-American Communities. Ann Behav Med. 2015 Jun;49(3):398-410. doi: 10.1007/s12160-014-9664-1. PMID: 25385203.

271. Wright K, Norris K, Newman Giger J, et al. Improving healthy dietary behaviors, nutrition knowledge, and self-efficacy among underserved school children with parent and community involvement. Child Obes. 2012 Aug;8(4):347-56. doi: 10.1089/chi.2012.0045. PMID: 22867074.

272. Wright K, Giger JN, Norris K, et al. Impact of a nurse-directed, coordinated school health program to enhance physical activity behaviors and reduce body mass index among minority children: a parallel-group, randomized control trial. Int J Nurs Stud. 2013 Jun;50(6):727-37. doi: 10.1016/j.ijnurstu.2012.09.004. PMID: 23021318.

273. Yildirim M, Arundell L, Cerin E, et al. What helps children to move more at school recess and lunchtime? Mid-intervention results from Transform-Us! cluster-randomised controlled trial. Br J Sports Med. 2014 Feb;48(3):271-7. doi: 10.1136/bjsports-2013-092466. PMID: 24124036.

274. Zhou Z, Ren H, Yin Z, et al. A policy-driven multifaceted approach for early childhood physical fitness promotion: impacts on body composition and physical fitness in young Chinese children. BMC Pediatr. 2014 May 05;14:118. doi: 10.1186/1471-2431-14-118. PMID: 24886119.

275. Ashfield-Watt PA, Welch AA, Godward S, et al. Effect of a pilot community intervention on fruit and vegetable intakes: use of FACET (Five-a-day Community Evaluation Tool). Public Health Nutr. 2007 Jul;10(7):671-80. doi: 10.1017/s1368980007382517. PMID: 17381948.

276. Blake H, Zhou D, Batt ME. Five-year workplace wellness intervention in the NHS. Perspect Public Health. 2013 Sep;133(5):262-71. doi: 10.1177/1757913913489611. PMID: 23771680.

277. Brownson RC, Baker EA, Boyd RL, et al. A community-based approach to promoting walking in rural areas. Am J Prev Med. 2004 Jul;27(1):28-34. doi: 10.1016/j.amepre.2004.03.015. PMID: 15212772.

278. Brownson RC, Hagood L, Lovegreen SL, et al. A multilevel ecological approach to promoting walking in rural communities. Prev Med. 2005 Nov-Dec;41(5-6):837-42. doi: 10.1016/j.ypmed.2005.09.004. PMID: 16256183.

279. Brusseau TA, Hannon J, Burns R. The Effect of a Comprehensive School Physical Activity Program on Physical Activity and Health-Related Fitness in Children From Low-Income Families. J Phys Act Health. 2016 Aug;13(8):888-94. doi: 10.1123/jpah.2016-0028. PMID: 27144329.

Page 429: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-34

280. Cheadle A, Rauzon S, Spring R, et al. Kaiser Permanente's Community Health Initiative in Northern California: evaluation findings and lessons learned. Am J Health Promot. 2012 Nov-Dec;27(2):e59-68. doi: 10.4278/ajhp.111222-QUAN-462. PMID: 23113787.

281. De Cocker KA, De Bourdeaudhuij IM, Brown WJ, et al. Four-year follow-up of the community intervention '10,000 steps Ghent'. Health Educ Res. 2011 Apr;26(2):372-80. doi: 10.1093/her/cyr015. PMID: 21393377.

282. de Silva-Sanigorski A, Elea D, Bell C, et al. Obesity prevention in the family day care setting: impact of the Romp & Chomp intervention on opportunities for children's physical activity and healthy eating. Child Care Health Dev. 2011 May;37(3):385-93. doi: 10.1111/j.1365-2214.2010.01205.x. PMID: 21276039.

283. Geaney F, Harrington J, Perry IJ. P42 The impact of a catering initiative in determining food choices and salt intake in the public sector. Journal of Epidemiology & Community Health. 2010;64:A50-A. PMID: 104133563. Language: English. Entry Date: 20140108. Revision Date: 20150710. Publication Type: Journal Article.

284. Gebel K, Bauman AE, Reger-Nash B, et al. Does the environment moderate the impact of a mass media campaign to promote walking? Am J Health Promot. 2011 Sep-Oct;26(1):45-8. doi: 10.4278/ajhp.081104-ARB-269. PMID: 21879942.

285. Heelan KA, Abbey BM, Donnelly JE, et al. Evaluation of a Walking School Bus for Promoting Physical Activity in Youth. J Phys Act Health. 2009 Sep;6(5):560-7. PMID: 19953832.

286. Huberty J, Beets M, Beighle A. Effects of a policy-level intervention on children's pedometer-determined physical activity: preliminary findings from Movin' Afterschool. J Public Health Manag Pract. 2013 Nov-Dec;19(6):525-8. doi: 10.1097/PHH.0b013e31829465fa. PMID: 23676476.

287. Kim K, Hong SA, Yun SH, et al. The effect of a healthy school tuck shop program on the access of students to healthy foods. Nutr Res Pract. 2012 Apr;6(2):138-45. doi: 10.4162/nrp.2012.6.2.138. PMID: 22586503.

288. Magarey AM, Pettman TL, Wilson A, et al. Changes in Primary School Children's Behaviour, Knowledge, Attitudes, and Environments Related to Nutrition and Physical Activity. ISRN Obes. 2013;2013:752081. doi: 10.1155/2013/752081. PMID: 24555153.

289. Naul R, Schmelt D, Dreiskaemper D, et al. 'Healthy children in sound communities' (HCSC/gkgk)--a Dutch-German community-based network project to counteract obesity and physical inactivity. Fam Pract. 2012 Apr;29 Suppl 1:i110-i6. doi: 10.1093/fampra/cmr097. PMID: 22399539.

290. Rogers VW, Hart PH, Motyka E, et al. Impact of Let's Go! 5-2-1-0: a community-based, multisetting childhood obesity prevention program. J Pediatr Psychol. 2013 Oct;38(9):1010-20. doi: 10.1093/jpepsy/jst057. PMID: 23933841.

291. Tomlin D, Naylor PJ, McKay H, et al. The impact of Action Schools! BC on the health of Aboriginal children and youth living in rural and remote communities in British Columbia. Int J Circumpolar Health. 2012 Mar 19;71:17999. doi: 10.3402/ijch.v71i0.17999. PMID: 22456048.

292. Vasquez A, Sherwood NE, Larson N, et al. A novel dietary improvement strategy: examining the potential impact of community-supported agriculture membership. Public Health Nutr. 2016 Oct;19(14):2618-28. doi: 10.1017/s1368980015003638. PMID: 26857484.

293. Weaver RG, Webster CA, Egan C, et al. Partnerships for Active Children in Elementary Schools: Outcomes of a 2-Year Pilot Study to Increase Physical Activity During the School Day. Am J Health Promot. 2017 Jan 01:890117117707289. doi:

Page 430: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

K-35

10.1177/0890117117707289. PMID: 28482678.

294. Whaley SE, McGregor S, Jiang L, et al. A WIC-Based Intervention to Prevent Early Childhood Overweight. J Nutr Educ Behav.

2010 May-Jun;42(3):S47-S51. doi: 10.1016/j.jneb.2010.02.010. PMID: 20399409

Page 431: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

L-1

Appendix L. Comparison of Two Tools for Assessing Study Risk of Bias Table L1. Comparison of EPHPP and NOS ratings on select natural experiment studies*

EPHPP Newcastle-Ottawa Scale

Author, Year Selection Bias

Study Design

Confounders Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall rating

Selection, Max=4

Comparability, Max=2

Outcome, Max =3

Barroso, 20091 M H H M M H H 2 1 2

Berger-Jenkins, 20142

H H L M H H H 2 2 1

Brown, 20153 L M L H L H H 4 2 1

Brown, 20164 L M L H L H H 4 1 2

Calise, 20135 M H L L M M M 3 0 1

Cummins, 20146

M M L H L H H 2 2 1

Jennings, 20127 L M L H L H H 3 0 1

Just, 20148 M H H M H H H 3 0 2

Kim, 20129 M L M M L M H 3 2 1

King, 201410 M L H H H M M 3 0 2

LaRowe, 201611

H M M M L H H 4 1 3

Maddock, 200612

M H H M L H H 3 0 1

Peterson, 201513

L H L M L H H 3 1 2

Page 432: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

L-2

EPHPP Newcastle-Ottawa Scale

Author, Year Selection Bias

Study Design

Confounders Blinding Data Collection Methods

Withdrawals and Drop Outs

Overall rating

Selection, Max=4

Comparability, Max=2

Outcome, Max =3

Reger-Nash, 200814

H M L M M H H 2 1 1

Restrepo, 201615

M H M H H H H 2 2 1

Riis, 201216 M H H M M H H 3 2 1

Taber, 201217 M H M H H M H 2 2 1

Veugelers, 200518

M L L M M L L 2 2 1

Webb, 201219 L H M M H H H 3 2 2

Zhu, 201320

H M L H L H H 2 0 1

EPHPP = Effective Public Health Practice Project; H = high risk of bias; L = low risk of bias; M = medium risk of bias; NOS=Newcastle-Ottawa Scale

Page 433: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

L-3

References for Appendix L

1. Barroso CS, Kelder SH, Springer AE, et al. Senate Bill 42: implementation and impact on physical activity in middle schools. J Adolesc Health. 2009 Sep;45(3 Suppl):S82-90. doi: 10.1016/j.jadohealth.2009.06.017. PMID: 19699442.

2. Berger-Jenkins E, Rausch J, Okah E, et al. Evaluation of a Coordinated School-Based Obesity Prevention Program in a Hispanic Community: Choosing Healthy and Active Lifestyles for Kids/Healthy Schools Healthy Families. American Journal of Health Education. 2014;45(5):261-70. doi: 10.1080/19325037.2014.932724. PMID: 103883129.

3. Brown BB, Werner CM, Tribby CP, et al. Transit Use, Physical Activity, and Body Mass Index Changes: Objective Measures Associated With Complete Street Light-Rail Construction. Am J Public Health. 2015 Jul;105(7):1468-74. doi: 10.2105/ajph.2015.302561. PMID: 25973829.

4. Brown BB, Smith KR, Tharp D, et al. A Complete Street Intervention for Walking to Transit, Nontransit Walking, and Bicycling: A Quasi-Experimental Demonstration of Increased Use. J Phys Act Health. 2016 Nov;13(11):1210-9. doi: 10.1123/jpah.2016-0066. PMID: 27334024.

5. Calise TV, Heeren T, DeJong W, et al. Do neighborhoods make people active, or do people make active neighborhoods? Evidence from a planned community in Austin, Texas. Prev Chronic Dis. 2013 Jun 20;10:E102. doi: 10.5888/pcd10.120119. PMID: 23786909.

6. Cummins S, Flint E, Matthews SA. New neighborhood grocery store increased awareness of food access but did not alter dietary habits or obesity. Health Aff (Millwood). 2014 Feb;33(2):283-91. doi: 10.1377/hlthaff.2013.0512. PMID: 24493772.

7. Jennings A, Cassidy A, Winters T, et al. Positive effect of a targeted intervention to improve access and availability of fruit and vegetables in an area of deprivation. Health Place. 2012 Sep;18(5):1074-8. doi: 10.1016/j.healthplace.2012.05.001. PMID: 22705164.

8. Just DR, Wansink B. School lunch debit card payment systems are associated with lower nutrition and higher calories. Obesity (Silver Spring). 2014 Jan;22(1):24-6. doi: 10.1002/oby.20591. PMID: 23929600.

9. Kim J. Are physical education-related state policies and schools' physical education requirement related to children's physical activity and obesity? J Sch Health. 2012 Jun;82(6):268-76. doi: 10.1111/j.1746-1561.2012.00697.x. PMID: 22568462.

10. King MH, Lederer AM, Sovinski D, et al. Implementation and evaluation of the HEROES initiative: a tri-state coordinated school health program to reduce childhood obesity. Health Promot Pract. 2014 May;15(3):395-405. doi: 10.1177/1524839913512835. PMID: 24334542.

11. LaRowe TL, Tomayko EJ, Meinen AM, et al. Active Early: one-year policy intervention to increase physical activity among early care and education programs in Wisconsin. BMC Public Health. 2016 Jul 20;16:607. doi: 10.1186/s12889-016-3198-3. PMID: 27439770.

12. Maddock J, Takeuchi L, Nett B, et al. Evaluation of a statewide program to reduce chronic disease: The Healthy Hawaii Initiative, 2000-2004. Eval Program Plann. 2006;29(3):293-300. doi: 10.1016/j.evalprogplan.2005.12.007. PMID: 2006-11996-009.

13. Peterson KE, Spadano-Gasbarro JL, Greaney ML, et al. Three-Year Improvements in Weight Status and Weight-Related Behaviors in

Page 434: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

L-4

Middle School Students: The Healthy Choices Study. PLoS One. 2015;10(8):e0134470. doi: 10.1371/journal.pone.0134470. PMID: 26295837.

14. Reger-Nash B, Bauman A, Cooper L, et al. WV Walks: replication with expanded reach. J Phys Act Health. 2008 Jan;5(1):19-27. PMID: 18209251.

15. Restrepo BJ. Calorie Labeling in Chain Restaurants and Body Weight: Evidence from New York. Health Econ. 2016 Jul 24doi: 10.1002/hec.3389. PMID: 27451966.

16. Riis J, Grason H, Strobino D, et al. State school policies and youth obesity. Matern Child Health J. 2012 Apr;16 Suppl 1:S111-8. doi: 10.1007/s10995-012-1000-4. PMID: 22527761.

17. Taber DR, Chriqui JF, Chaloupka FJ. Differences in nutrient intake associated with state laws regarding fat, sugar, and caloric content of competitive foods. Arch Pediatr Adolesc Med. 2012 May;166(5):452-8. doi: 10.1001/archpediatrics.2011.1839. PMID: 22566546.

18. Veugelers PJ, Fitzgerald AL. Effectiveness of school programs in preventing childhood obesity: a multilevel comparison. Am J Public Health. 2005 Mar;95(3):432-5. doi: 10.2105/ajph.2004.045898. PMID: 15727972.

19. Webb E, Netuveli G, Millett C. Free bus passes, use of public transport and obesity among older people in England. J Epidemiol Community Health. 2012 Feb;66(2):176-80. doi: 10.1136/jech.2011.133165. PMID: 21911850.

20. Zhu X, Lu Z, Yu C, et al. Walkable communities: impacts on residents’ physical and social health. World Health Des 2013;7:68-75.

Page 435: Methods for Evaluating Natural Experiments in Obesity ...• Natural experiments most commonly used regression models comparing exposed and unexposed groups at one time. • Natural

L-5