evaluating and measuring the impact of citizen diplomacy- current status and future directions...

8/12/2019 Evaluating and Measuring the Impact of Citizen Diplomacy- Current Status and Future Directions Roundtable

1/22

Evaluating and Measuring the Impact ofCitizen Diplomacy:

Current Status and Future Directions

Rajika Bhandari & Raisa Belyavina

Institute of International Education (IIE)

Report prepared for:U.S. Citizenship Diplomacy Summit

Washington, DCNovember 17-19, 2010


2/22

Table of Contents

Executive Summary 1Recommendations and Action Steps for the Future 2

I. The Changing Landscape of Program EvaluationDefining Citizen Diplomacy 3Why Measure the Impact of Citizen Diplomacy Programs? 3Challenges in Measuring the Impact of Citizen Diplomacy Programs 4Focus and Methodology of Existing Evaluations 4

II. Case Studies in Evaluating Citizen Diplomacy Programs

A. High School Exchange Programs

AFS Intercultural Programs 5

B. International Volunteer ProgramsCenter for Social Development: International Volunteer Service Impact Study (IVIS) 6Peace Corps 7

C. Postsecondary Exchange and Study Abroad ProgramsU.S. and Visiting Fulbright Scholar Program 7Georgia Learning Outcomes of Students Studying Abroad Research Initiative (GLOSSARI) 9Benjamin A. Gilman Scholarship Program 9Study Abroad for Global Engagement (SAGE) Project 10

D. Professional and Leadership Exchange Programs

Open World Leadership Center 11The Hubert H. Humphrey Fellowship Program 11The International Visitor Leadership Program (IVLP) 12

E. Summary 13Figure 1: Key Outcomes - Comparison of U.S. and International Participant Outcomes 13Summary Table of Evaluation Methodology and Key Outcomes of Twelve Case Studies 14

Acknowledgments 16

Bibliography 17

IIE Resources 18


3/22

Executive Summary

As more Americans study abroad, become

internationally mobile in their jobs, or engage involunteering activities overseas, it is important tounderstand the impact of such activity on their homeand host institutions, the wider communities and publicdiplomacy activities at large. People-to-people contactis becoming one of the key elements of diplomacy,and citizen diplomats increasingly complement thetraditional forms of political diplomacy.

The U.S. government has provided significant support forcitizen diplomacy programs such as the Peace Corps andthe International Visitor Leadership Program (IVLP), ashave private donors, corporations, and individuals whohave contributed their resources to various programsinvolving international exchange. In the last 50 years,substantial human and financial investments have beenmade in these types of programs. Yet, there is still muchto learn about the full scope of their impact on the U.S.and the world. While some efforts have been madeto measure the impact of citizen diplomacy programs,further evaluation and documentation is needed toidentify what is working and what is not working; leveragelessons learned for program improvement; replicate

successful program strategies; and provide a rationalefor continued support for these types of programs.In-depth evaluation data on such programs can also helpimprove programs and increase access for populations thathave not traditionally engaged in citizen diplomacy activities.

This report provides an overview of the current landscapeof evaluating citizen diplomacy programs, taking a lookat the methodology and findings of evaluations of anarray of such initiatives in the United States. The goalof the report is to take stock of what has been learnedthrough these evaluations, and also to identify next

steps and recommendations for future studies of citizendiplomacy programs.

Based on an extensive review of evaluations of differenttypes of citizen diplomacy programs, this report findsthat while some efforts have been made to evaluate therange of citizen diplomacy programs, comprehensiveassessments of such programs continues to pose a

challenge, as the associated outcomes and impacts are

often intangible, not immediate, and qualitative ratherthan quantitative. What are usually measured are outputssuch as the number of participants and their degree ofsatisfaction with programs. Yet citizen diplomacy is oftenmost impactful in its multiplier effects on institutions,communities, and societies. These effects are easilyunderestimated due to their longitudinal nature and thedifficulty in measuring them. By definition, long-termimpact takes many years to manifest, often long afterprogram funding has ended and the implementing agencyhas lost touch with alumni.

The lack of standardized measurement methods presentsanother challenge. Due to the wide variety of citizendiplomacy programs, it is impossible to measure all outputswith the same tools. Different programs have differentmissions, ranging from language study to cultural exchangeto economic development. Furthermore, some programs aretargeted for the cultural or educational enrichment of U.S.participants abroad, while others focus on the developmentof host communities. Because of the significant variationacross goals, approaches, and methodologies of citizendiplomacy programs and their evaluations, much of the

evaluative data generated is not comparable.

In addition, most evaluations continue to rely on self-reports and participant perceptions of a programs impact.While this type of qualitative data is important, it relies onparticipants memories and does not allow for an unbiasedassessment of program impact. Although a few evaluationsreviewed in this report have attempted more rigorousapproaches, such as the use of comparison groups and arandom assignment design, much of the field continues torely on self-reported information.

Institute of International Education 1


4/22

Executive Summary (cont.)

Recommendations and Action Steps for the Future

Based on an extensive review of evaluations of different types

of citizen diplomacy programs, this report offers the followingrecommendations for expanding the role of evaluation in citizendiplomacy programs:

1) Because the impact of international exchanges can beunderestimated due to the difficulty in measuring the far-reachingeffects of interpersonal interaction, it is critical to continue todevelop and conduct impact studies and for program sponsorsto provide funding for long-term evaluations and alumni follow-up once the program has concluded. A potential model forsuch longitudinal impact studies is being launched by the FordFoundation International Fellowships Program, with a 10-year

alumni tracking initiative that IIE will conduct on behalf of theprogram sponsor, assessing the impact of graduate study abroadon over 4,300 emerging leaders from disadvantaged/marginalizedcommunities within the developing world.

2) Many in the fieldboth from the program side and amongevaluation expertsrecommend some sort of standardization inthe evaluation methodology of such programs, such as the use ofexternal instruments like the Intercultural Development Inventory(IDI) or the Beliefs, Events, and Values Inventory (BEVI). Commonmeasurement approaches would also facilitate the comparison ofdifferent programs, their strategies, and their effectiveness.1

3) Although rigorous impact evaluations using random assignmentand control groups are difficult to conduct, they can be importantin establishing cause-and-effect when looking at the impact ofprograms on participants and alumni. Program administrators,evaluators, and funding agencies should be encouraged to designthese types of evaluation studies even if it means focusing onsamples rather than entire populations of alumni.

4) In addition to well-established data collection approachessuch as surveys, interviews, and focus groups, evaluators of

citizen diplomacy programs should also explore the use of newerapproaches such as the use of social media as a tool for datacollection, and utilizing more in-depth qualitative approaches likeparticipatory and action research.

5) Documenting the impact of citizen diplomacy programs shouldnot be the sole responsibility of a single group (the implementingagency, for example). Various stakeholders including programorganizers, funders, third party evaluators, and the academic and

research community should collaborate to develop best practicesfor measuring the impact of citizen diplomacy programs.

6) To foster and reinforce a culture of accountability, evaluation

should be an integral part of project development and executionFunders of programs should require that evaluation be a corecomponent of the program from inception to conclusion andbeyond, and that a portion of the project budget be devoted toevaluation activities. In the absence of such requirements andwithout sufficient funding allocated for evaluation, many programsconduct cursory evaluations as the program winds down, almost asan afterthought, thereby missing critical opportunities during thecourse of the program to collect valuable evaluation data.

7) There needs to be an increased emphasis on and funding for

alumni programs to guarantee sustainability of citizen diplomacyprograms and assure lasting impact. Alumni programs help to buildnetworks that increase the multiplier effect of citizen diplomacyand promote continued cross-national cultural and educationacollaboration, and work exchanges and partnerships. Such programsalso allow easier access to program participants, making longitudinaquantitative and qualitative evaluation data easier to collect. TheFulbright Legacy Funds Alumni Impact Awards offer one model forsuch alumni engagement, as does the U.S. Department of Statesalumni website.2

8) There needs to be more professionalization of the sub-field of

evaluating citizen diplomacy programs. The U.S. Citizen DiplomacySummit held in November 2010, provided a rare opportunity foresearchers and program staff to come together to share experiencesand knowledge in evaluating their respective programs. There is aclear need for more such opportunities that foster the sharing ofbest practices in how to evaluate citizen diplomacy programs. Oneconcrete step toward this goal might be to ensure that such activitiesare fully represented in professional evaluation and research forumssuch as the American Evaluation Association, American EducationaResearch Association (AERA), and the Comparative and InternationaEducation Society (CIES), among others.

2 Evaluating and Measuring the Impact of Citizen Diplomacy


5/22

Defining Citizen Diplomacy

The shape of the world a generation from now will be influencedfar more by how well we communicate the values of our society to

others than by our military or diplomatic superiority.-Senator J. William Fulbright, 1964

In todays interconnected world, economics, politics, andsecurity are more intertwined among nations than ever before.Yet the impetus for communicating with people across bordersincreasingly lies with individual citizens who engage with theworld through live and digital communication channels ona daily basis. Citizen diplomacy is a concept that involvestwo seemingly disparate ideas: private citizens engagingin individual endeavors that serve their own interests; anddiplomacy, which includes a framework for cooperation

between countries. Taken together, citizen diplomacy refers toan array of actions and activities that individuals can partakein that contribute to deepening ties between individuals andcommunities and to advancing the goals of public diplomacy.Citizen diplomacy is thus an integral part of public diplomacy.With the growing numbers of Americans who conductbusiness abroad, U.S. students traveling and studying abroadat increasing rates each year, and digital interconnectivity thatfacilitates interaction between people who have not previouslyhad a forum for engagement, the role of citizen diplomacy isbecoming more integral to international relations.

There are numerous types of activities that can be classifiedas citizen diplomacy endeavors, whether or not they aredefined explicitly as such. Various organizations spanning allsectors, including businesses, NGOs, faith-based institutions,and education and youth services, involve elements ofcitizen diplomacy. Additionally, government-sponsoredcitizen diplomacy programs have a long-standing traditionin the United States. The current report examines severaltypes of citizen diplomacy programs, including government-sponsored and non governmental initiatives that focusprimarily on cultural and educational exchanges in the

following categories:

High School/Youth Exchange ProgramsInternational VolunteeringProfessional and Leadership ExchangesPostsecondary Exchanges (e.g., study abroad,fellowships, post-graduate programs)

I. The Changing Landscape of Program Evaluation

Why Measure the Impact of Citizen Diplomacy Programs?

Research on citizen diplomacy has existed for as long as there hasbeen support for the exchange of ideas between people and acrossnations. Scholars and program implementers have collectedinformation on the outcomes of programs, although much ofthe information gathered in the past has been anecdotal andfocused mostly on the impact on individual participants. Rigorousquantitative evaluation studies have been far less common thanqualitative studies. Today, as both public and private fundersrequire more data-driven evaluations with measurable programresults and outcomes, collecting quantitative data is no longer anoption but a requirement. Such data gathering allows for moremeaningful cross-sectional, cross-sectorial, and longitudinalanalyses, which are increasingly becoming the standard in the field

of program evaluation.

According to a study conducted in 2007 by the Center for SocialDevelopment at Washington University in St. Louis, the value ofU.S. volunteer work abroad in 2005 was nearly $3 billion (Lough,McBride, & Sherraden, 2007). This figure calculated using dataprovided in the Current Population Survey volunteer supplementof the U.S. Bureau of the Census demonstrated the economicbenefits to host countries and communities receiving internationalvolunteers from the United States. Approximately one millionvolunteers from the U.S. devote their time to international serviceprojects annually. Quantifying the net worth of volunteer work

abroad provides a concrete example of the added economicvalue to host countries. Although the economic value added bysecondary and postsecondary exchanges is more challenging tomeasure, there is a positive economic impact in this sphere aswell. According to the 2010 Open Doors Report on InternationalEducational Exchange, published annually by the Institute ofInternational Education with support from the U.S. Department ofState, the 680,000 international students enrolled in U.S. highereducation contributed nearly $20 billion to the U.S. economy in2009/10. U.S. students abroad (over 260,000 as of 2008/09) alsocontribute financially to their host countries, but the amount is

harder to calculate since many pay tuition to their home campuses(Chow & Bhandari, 2010).

Beyond establishing the economic value of citizen diplomacyprograms, quantitative data on program outcomes allows foran objective assessment of the non monetary impact of suchprograms. The Peace Corps Program, for example, has determinedthrough its evaluation that the social interactions of volunteers



6/22

with remote communities positively affect the opinionabout the United States held by local residents who wouldotherwise never have any contact with Americans. TheBenjamin A. Gilman International Scholarship Program

supports study abroad for U.S. students who would nototherwise have an opportunity for international study atthe undergraduate level and has measurable impact ontheir academic life and career goals. The positive impacton participants educational and professional goals iscaptured in longitudinal program evaluations.

As the role of soft diplomacy in an increasinglyinterconnected world becomes more significant, keepingrecord of qualitative and anecdotal information ofprogram outcomes will continue to have great value

because it provides perspective on how programs affectlives. This evidence captures the data and the stories ofhow individuals transform their societies on micro andmacro levels, and reinforces what most experts believeintuitively to be true: international learning opportunities,in whichever form and at whatever stage in life, havetransformational effects on individuals and communities.In recent years, the shift to quantitative assessments ofcitizen diplomacy programs across sectors has started totake place, and they will continue to be required by fundersand will become the standard for program evaluations.

Challenges in Measuring the Impact of CitizenDiplomacy Programs

It is indisputable that there is value in internationaleducational exchange programs, volunteering, andprofessional development opportunities that bring togethercounterparts in public and private sectors from aroundthe world. It is also clear that these opportunities makea significant contribution to public diplomacy. Measuringthe results of soft diplomacy poses a challenge, however,since outcomes and impacts of such programs are often

intangible and not immediate. What is usually measuredare outputs such as the number of participants and theirrelative degree of satisfaction with various programs. Yetcitizen diplomacy is often most impactful in its multipliereffects on institutions, communities, and societies. Andthese very effects can be easily underestimated becauseof the difficultly in measuring them.

I. The Changing Landscape of Program Evaluation (cont.)


Another challenge to measuring the impact of citizen diplomacprograms is the lack of standardized measurement methodsDue to the large variety of citizen diplomacy programs, it iimpossible to measure all outputs with the same tools. Differenprograms have different missions, ranging from language study to

cultural exchange to economic development. Furthermore, someprograms are targeted for the cultural or educational enrichmenof American participants abroad, while others focus more on thedevelopment of host communities. Despite these different goalsinternational programs do share overlapping objectives, includingthe promotion of mutual understanding between peoples andcultures, increased awareness of shared values and world views

and meaningful exchanges of knowledge.

Focus and Methodology of Existing Evaluations

Evaluating the impact of citizen diplomacy involves looking ashort-term and long-term change: attitudinal, behavioral, andinstitutional. Such assessments can be costly, but the exercisprovides invaluable data that allows researchers to measure thetangible impacts that experts know to be anecdotally true. Inthe past, research design has often not been rigorous, and mosevaluations done today are moving toward quantitative models.

Many different impacts are measured in evaluation studiesdepending on the goals of the programs and the objectives othe evaluation. Some assessments focus on estimating theindividual gains that result from participation in internationa

educational and cultural exchange programs; others look at thecontributions to local communities and institutions with whichAmerican citizens engage; still others measure the impact onindividual participants career advancement, professional skillsand language acquisition.

Many evaluations focus on measuring short-term results onindividuals and on communities. The most common assessmentools employed are participant and beneficiary constituent surveysinterviews, and focus groups. Mid-term and long-term studies aremore challenging to conduct, but as several case studies in thireport show, they are not impossible. These studies are particularlyencouraging, since large-scale and long-term impacts are oftenthe overarching goal of citizen diplomacy programs. Evaluations osuch programs have typically included an array of methodologicaapproaches, including quantitative and qualitative measures. Thuse of social media to gather quick data from program alumnespecially from those dispersed around the world, constitutes anew development in evaluating citizen diplomacy programs.


7/22

This report provides an overview of a number of evaluationsof different types of citizen diplomacy programs, highlightingvarious methodological approaches to measuring theimpact of programs on participants or on the communities

in which they were hosted. In addition to U.S.-governmentmandated evaluations of citizen diplomacy programs,different institutions that are involved with internationalexchanges and volunteering programs have conducteddetailed assessments of programs, both internally andthrough third-party evaluators. This section covers four

II. Case Studies in Evaluating Citizen Diplomacy Programs

A. High School Exchange Programs

AFS Intercultural Programs

AFS is an international, voluntary, non-governmental,non-profit organization that provides intercultural learn-ing opportunities to help people develop the knowledge,skills and understanding needed to create a more justand peaceful world.3 AFS organizes programs that fosterreal-life experiential learning and includes a high schoolexchange program in which students live with homestayfamilies in the host country.

To measure outcomes of its high school exchange pro-

grams, AFS commissioned two evaluations by HammerConsulting: an Educational Results Impact Study (2005)and a Long-Term Impact Study (2008).

The first of the two studies was conducted in nine coun-tries and in six languages, with over 2,100 secondary schoolparticipants (1,500 AFS participants and 638 of their peerswho did not participate in the AFS program, included as acontrol group). The study compared intercultural compe-tence, anxiety, networking, and knowledge, as well as for-eign language fluency, between AFS participants and theirpeers. Using the Intercultural Developmental Inventory(IDI)4to measure impact, AFS based the study on devel-opmental scores to determine whether AFS participantsshowed substantial growth in intercultural competence.A decreased level of anxiety with other cultures was mea-sured, as well an increase in knowledge of other cultures,and higher levels of foreign language fluency. The studydetermined that AFS programs achieved the mission of in-

creasing the competence of participants in all of these areas.Additionally, many AFS participants developed expansive in-tercultural networks and friendships with individuals whoseculture differed from their own. Through strengtheningthese skills and building on these international experiences,AFS participants became global citizens.

The second study followed the same methodology as thefirst, but focused on the long-term impact that participationin AFS high school programs has on an individuals choices

pertaining to international life, knowledge, and awarenessof other cultures. This web-based study surveyed AFS par-ticipants from the 1980s, and compared the findings witha control group of their peers. Nearly 2,000 AFS programalumni in 15 different countries and over 500 of their peersresponded to the survey. This study also employed develop-mental scores using IDI, and demonstrated that AFS alumnihad greater cultural competence than their peers. Partici-pants levels of anxiety with other cultures were lower; theycontinued to have greater levels of comfort around othercultures, and maintained larger intercultural networks andfriendships with individuals from cultures other than theirown. AFS alumni were fluent in more languages, and had ahigher rate of studying or living abroad at a later point in theirlives. They were also more likely to encourage their childrento study abroad. Finally, the study showed that comparedto the control group survey respondents, AFS participantsattained higher levels of education.


categories of programs and includes eleven examples ofprogram evaluations.

The case studies also showcase some key findings to illustrate

the measured results found through the assessment, andsuggest some best practices as well as continuing challenges.The table on pages 14-15 provides a snapshot of the keyfeatures of the evaluation case studies reviewed in thisreport.


8/22

Center for Social Development: InternationalVolunteer Service Impact Study (IVIS)

The Center for Social Development at the George WarrenBrown School of Social Work at Washington Universityin St. Louis developed new research methods to measurethe impact of service on volunteers, organizations, andbeneficiaries through its International Volunteering ImpactsStudy (IVIS) (Lough, McBride, & Sherraden, 2009). Thisongoing study involves multiple programs that differ acrossa range of characteristics and uses a multi-method approachfor measuring outcomes: a quasi-experimental volunteer

survey comparing international volunteers to a groupthat did not volunteer internationally, and cross-sectionalinterviews and focus groups comparing host organizationsand beneficiaries to a matched sample of those that do notwork with international volunteers.

Most recently, the study focused on a short-term non-professional program and a long-term professional program. Thestudy was undertaken using multiple methodologies to measurethe results among volunteers. Using a 45-item survey, the goalwas to assess four relevant outcomes of volunteers acrossfour outcomes related to citizen diplomacy: 1) internationalawareness, 2) intercultural relations, 3) international socialcapital, and 4) international career intentions. The statisticaltechnique of generalized linear modeling was used to assessdifferences between treatment and comparison groups acrosstime, and statistically significant findings were observed acrosseach of the four outcomes.

International volunteers reported statistically significantgrowth in international awareness, international socialcapital, and international career intentions. Some of the keyfindings are listed below.

- Intercultural relations scores at the end of the study periodhad increased for both groups as compared to scores on thebaseline survey from the beginning of the study.

- Results from the baseline survey indicate that the individualswho had self-selected to volunteer for these programs, buthad not yet undertaken this particular international volunteerservice, did not score significantly differently on any of the four

outcomes than their comparison group prior to commencingtheir program.

- The comparison group reported no change in their ownperceived level of international awareness, while the volunteersindicated a significantly increased level in internationalawareness.

- For intercultural relations, the scores of both programparticipants and the comparison group increased; therefore,the research concludes no statistical difference for the twogroups over the study period.

- International social capital, according to a comparison ofbaseline and post-test surveys, grew for both groups, butsubstantially more for the volunteers, with the long-termprogram participants reporting even higher results than theshort-term program participants.

- The results for the international career intentions categoryindicate a sizeable increase in the intention to work abroador in an international development-related career amongvolunteers after their program experience, compared bothto the groups scores before the test and to the comparison

groups survey at the end of the study.

Overall, the conclusions of the IVIS indicate a positivecorrelation between international volunteer service andinternational awareness, international social capital, andinternational career intentions. While there was an increasein volunteers perceived intercultural relations over thecourse of the study period, this trend was also observed forthe comparison group.

The IVIS study underscores the need for rigorous studies

that show impacts on citizen diplomacy and also highlightssome of the continued challenges in capturing this data. As afollow-up to the IVIS study, the intended upcoming phasesof research are: 1) to complete a third round of surveysfocusing on behaviors and potential impacts over time, thentriangulate the results with data from the host countries; and2) to extend the comparative research design to partners andsimilar programs.

B. International Volunteer Programs


II. Case Studies in Evaluating Citizen Diplomacy Programs (cont.)


9/22

Americans, and 3) what facets of the interactions correlatewith positive and negative opinions of Americans.

The projects undertaken by Peace Corps volunteers are vastlydifferent with highly qualitative results, making it difficult toemploy a single measurement methodology. The evaluationstudy was designed as a multisite research project with a built-in process to work with host country research teams. Despitevarious challenges, including noncomprehensive baseline data,differing levels of in-country research expertise, and multiplelanguages, the Peace Corps successfully completed the studyof measuring the impact of volunteers on the individuals andcommunities they serve and the resulting change of opinion

of America and American people.

Results showed that interaction with volunteers has apositive effect on the level of understanding of Americans.Prior to the host communitys experience with Peace Corpsvolunteers, the average level of understanding of Americanswas limited to moderate; and afterward, a majority reportedto have a moderate to thorough understanding. Results alsoshowed increasingly positive opinions of Americans. Prior tothe volunteer interactions, most survey respondents beganwith a neutral or positive opinion of Americans; afterward,the level of positive opinion grew among the majority of

respondents. Individuals reporting an unchanged opinionfrequently began with a highly positive opinion and tendedto have had substantial interaction with Americans beforetheir Peace Corps experience.

Peace Corps

The Peace Corps is one of the most longstanding internationalvolunteer programs supported by the U.S. government.

Volunteers live and serve within their host communitiesand receive stipends at the economic level of their hostcolleagues. During the 27-month experience, volunteersdevelop people-to-people friendship and provide service tothe communities of their residence. Their relationships (withfriends, neighbors, and colleagues) and their acceptancewithin a community are the primary framework throughwhich the mission and goals of the Peace Corps are carriedout.

In 2008, the first evaluation on measuring Peace Corps

impact on citizen diplomacy was conducted internally by theOffice of Research, Evaluation, and Measurement, upon therequest of the Office of Management and Budget (OMB).This evaluation set out to measure the second of PeaceCorps three goals: helping promote a better understandingof Americans on the part of the peoples served.5

The Peace Corps developed its own model for conductingthe study. A semi-structured survey was translated intomultiple languages and local researchers were hired ineight countries to conduct the study, which included 880individual respondents. The study measured community

members understanding of Americans prior to theirinteraction with the volunteers. The surveys were designedto learn the extent to which: 1) volunteers enable hostcommunity members to gain a deeper understandingof Americans, 2) interaction with volunteers positivelyor negatively impacts community members opinions of


C. Postsecondary Exchange and Study Abroad Programs

U.S. and Visiting Fulbright Scholar Program

The largest and most prominent postsecondary exchangeprogram supported by the U.S. government is the FulbrightProgram, sponsored by the U.S. Department of State. Thisflagship program provides funding for students, educators,and professionals to engage in international exchangeat the graduate and post graduate level. The programsupports over 7,000 U.S. and international students and

scholars annually for study, teaching, or research outside oftheir home country. The goals of the Fulbright Program areto: 1) increase mutual understanding between the peopleof the United States and the people of other countries, 2)strengthen the ties that unite the United States with othernations, 3) promote international cooperation for educationaland cultural advancement, and 4) assist in the developmentof friendly, sympathetic, and peaceful relations between theUnited States and other countries of the world.



10/22

U.S Fulbright Scholar Program

In 1999, the Office of Policy and Evaluation of the Bureau

of Educational and Cultural Affairs of the U.S. Departmentof State contracted SRI International to assess anddocument the impact and outcomes of the U.S. FulbrightScholar Program. This evaluation was intended to ascertainwhether the U.S. Fulbright Scholar Program was achievingits legislative goals and to assess the broader impacts ofthe program on individuals and institutions both in theUnited States and in the Fulbright Scholars host countries,including how participation in the program contributesto the professional and personal lives, activities, andachievements of program alumni.

In 2001, SRI surveyed a stratified random sample ofover 1,000 U.S. Fulbright Scholar Program alumni whosegrants began between 1976 and 1999. Scholars werenotified of the survey by e-mail, and over 800 Scholaralumni 80 percent completed the questionnaire eitherelectronically or on paper. The SRI assessment of the U.S.Fulbright Scholar Program found strong quantitative andqualitative evidence that the program is achieving itslegislative mandate of promoting mutual understandingand cooperation between the United States and othernations and that it has diverse and often powerful impacts

not only on the scholars themselves, but also on theircolleagues, students, friends, and families. The multipliereffect is a significant contributing factor to the far reaching

impacts of the program.

Visiting Fulbright Scholar Program

The Visiting Fulbright Scholar Program was establishedin 1946 and provides funding and support for scholarsfrom around the world to conduct research, teach, andcollaborate with peers in the United States. The goal of the

program is to foster an exchange of ideas among scholarsand build cross-national understanding between countries.About 800 grantees come to the U.S. each year and over40,000 scholars have participated in the program since itsinception.


The Outcome Assessment of the Visiting Fulbright Scholar

Program was conducted by SRI International in June 2005and includes an evaluation of visiting scholars to U.S.higher education institutions. The evaluation focuses on theindividual impacts that the Fulbright grant has on visitingscholars; it also includes an assessment of factors beyondindividual gains, including multiplier effects upon thescholars return to their home countries.

Sixteen countries were selected by the U.S. Departmentof State as the focus of the evaluation study, based upongeographic representation, political salience, longevityof the program, and the type of program administration.To understand the complexities of the Visiting ScholarProgram, the SRI assessment team held a series of initialinterviews with Fulbright Program sponsors in the UnitedStates from the Office of Academic Exchange Programsin the U.S. Department of States Bureau of Educationaland Cultural Affairs and with key representatives from theCouncil for International Exchange of Scholars (CIES), whichadministers the program. SRI obtained contact informationfor over 3,300 program alumni and received nearly 1,900valid completed questionnaires.

The results of the Fulbright Program were measuredaccording to outcomes in learning, behaviors, and linkagesthat represent key components of mutual understanding.The findings of this assessment are based primarily on thequantitative results of the survey, but they are supplementedby qualitative information obtained in open-ended questions,as well as through individual interviews and focus groups.

According to the evaluation, 99 percent of visiting scholarsreport a better understanding of the U.S. and 96 percentshared their experiences through media or cultural activitieswhen returning to their home country.6It is worth noting

that the evaluation yielded evidence of only short-termoutcomes and outputs. This is not a shortcoming of thisspecific study and its methodology; instead, it reflects theongoing challenge of evaluating the long-term impacts ofeducational exchange programs.



11/22

Georgia Learning Outcomes of StudentsStudying Abroad Research Initiative (GLOSSARI)

In 2000, the University System in the state of Georgiaundertook a major project to conduct research on theacademic impacts of study abroad programs. The researchspanned 35 diverse higher education institutions in thestate and included an analysis of quantitative data onover 19,100 students who participated in study abroadprograms as well as another 17,900 students in the controlgroup. The study focused on capturing the academicoutcomes and related indicators of those students whostudied abroad in comparison to those who did not. Thestudy focused on several major components including self-reported learning outcomes, course-specific examinations,

academic indicators such as graduation rates, and effectson career goals.

Methodologically, the study and control groups were closelymatched for comparability in order to identify and correlatethe impacts of participation in study abroad programs onacademic outcomes. In addition to quantitative analysisof student learning outcomes, the study also focused ondevelopment of intercultural competency (Sutton & Rubin,2004) and administered a questionnaire to 440 studyabroad participants and about half as many students inthe control group at 13 higher education institutions in the

state of Georgia. It is unique in its scope and comprehensivesystem-wide approach to data collection.

The findings show that students who study abroad havedemonstrably higher academic outcomes, as well ashigher graduation rates and more developed interculturalcompetencies than their peers who do not study abroad.Some of the key findings include:

The four-year graduation rate of students who studyabroad increased 7.5 percent.

Four-year graduation rates for African-American

students who studied abroad were 31 percent higherthan those who did not study abroad and 18 percenthigher for other non-white students.Study abroad does not hinder the academic successof at-risk students but enhances it.


Benjamin A. Gilman Scholarship Program

Since its inception in 2001, the Gilman Scholarship Programhas supported U.S. students with financial need who maynot otherwise have an opportunity to study abroad, as wellas students from diverse backgrounds and students goingto non traditional study abroad destinations. This programis sponsored by the U.S. Department of States Bureau ofEducational and Cultural Affairs (ECA) and offers scholarshipfunding for U.S. undergraduate students receiving Pell fundsto study abroad for up to one year.

An evaluation of the Gilman Scholarship Program wascompleted by Macro International in 2009. The goal of thisevaluation was to measure the outcomes of the program

and its impact on the participants academic performanceand career goals as well as to capture some of the multipliereffects on home institutions and host countries. A40-question survey was the main tool for data collection.A total of 1,330 program alumni responded to the survey,representing a response rate of over 51 percent. The surveyincluded a participant self-assessment of foreign languagecompetency as well as questions on the impact of theprogram on academic achievement and professional goals.The survey tool also allowed program alumni to write abouttheir experience abroad, and nearly a thousand respondentsfilled out the open-ended portion of the survey.

The evaluation found that 99 percent of respondents studyabroad experiences had a positive impact on their academiclife and influenced their career goals. Seventy-seven percentreported that participating in a study abroad programinfluenced their academic achievements after the program.Two-thirds of those who pursued language study during theirtime abroad continued to study that language upon return,and those who studied critical need languages7 were mostlikely to continue to further pursue study of the language.The evaluation also tallied various accomplishments of

program alumni: obtaining additional funding and fellowshipsfor further study, academic achievements, participation inleadership roles on and off campus, and attaining internshipsand jobs.

The impact of program alumni upon their return was alsomeasured by assessing effects on the home institution. Overhalf of the respondents encouraged their peers on campus tostudy abroad. Three-fourths of Gilman alumni also reported

II. Case Studies in Evaluating Citizen Diplomacy Programs (cont.


12/22

that studying abroad improved relationships with theirfamilies, and a number of respondents provided anecdotalevidence of multiplier effects within their families andcommunities.

Gilman Scholarship participants had an impact on thecountries and institutions by which they were hosted.About one-fourth of participants reported positiveexperiences related to cross-cultural exchange, andone-fourth of program alumni wrote about addressingnegative stereotypes about Americans and the U.S. Manyparticipants also reported tangible outcomes with directbenefits to the host community, including volunteeringand starting non profit organizations.

Nearly 90 percent of Gilman alumni maintain contact withat least one person in the host country, with approximatelyhalf of the program alumni keeping personal contactsand another half maintaining professional or academicrelationships. Additionally, about one-fifth of theparticipants have returned to the host country, and thatpercentage is larger for those who have already receivedtheir undergraduate degrees and started their careers.

Study Abroad for Global Engagement (SAGE) Project

The Study Abroad for Global Engagement (SAGE) Project

is a collaborative endeavor between the University ofMinnesota and the Forum on Education Abroad, withfunding from the U.S. Department of Education. SAGEis a longitudinal study that began in 2006 and examinesempirically the long-term impact of undergraduate studyabroad on the global engagement of former study abroadparticipants. Using both quantitative and qualitative datafrom study abroad participants spanning nearly 50 years,the SAGE research team conceptualized, developed, andtested a multidimensional model of global engagementto find how outcomes are affected by the Four Ds:

demographics, destination, duration, and depth.

The SAGE project used a sequential mixed methodsresearch design that incorporated an online GlobalEngagement Survey (GES) and individual interviews. TheGES was made up of 56 questions: twenty-two questionscovered global engagement dependent variables, andthe rest focused on demographic, education, and career


information. The goal of individual interviews was to focuson the nature of global engagement and commitments, andidentify links to previous study abroad experiences throughqualitative analysis.

A total study population of 21,569 study abroad alumnirepresenting 22 U.S. higher education institutions was invited toparticipate in the study. Over 6,390 individuals completed theGES, and 63 respondents were interviewed (all of whom wererandomly selected from the 2,500 participants who agreed tobe interviewed by the research team). In addition to quantitativeregression analysis of the survey data, the study also incorporatedqualitative methods through open-ended survey questions andin-depth case studies of specific participants.

The projects principle outcome was an assessment ofglobal engagement, which included five components: 1)civicengagement, 2) philanthropy (volunteering and monetarydonations), 3) knowledge production (in print, online, and digitalmedia), 4) social entrepreneurship; and 5) voluntary simplicity inones lifestyle.8During the study, participants were asked abouttheir level of involvement in the five factors of engagement, andto rate how such involvement was affected by their study abroadexperiences. The results indicated that study abroad experiencehad the greatest influence on voluntary simplicity, socialentrepreneurship, and civic engagement, while philanthropy wasaffected the least by studying abroad.

The evaluation also captured data on the academicachievement of students who studied abroad. Over 60percent of participants have enrolled in graduate programssince completing their undergraduate degree. Of those whoare pursuing graduate education, 35 percent are involved inglobally-oriented academic programs, and indicated thattheir study abroad experiences has influenced their careerdecisions to a large degree.



13/22


D. Professional and Leadership Exchange Programs

Open World Leadership Center

The Open World Leadership Center is one of elevenagencies in the Legislative Branch serving Congress. It wasestablished in 1999 as a grant-giving agency in a bicameraland nonpartisan initiative. The agencys mission is toenhance understanding and capabilities for cooperationbetween the United States and the countries of Eurasiaand the Baltic States by developing a network of leaders inthe region who have gained significant, firsthand exposureto Americas democratic, accountable government andits free-market system.9Through Open World programs,

current and future Eurasian leaders are invited to come tothe United States for two weeks, stay with American hostfamilies to experience and gain a higher understandingof American home life, and engage with Americancounterparts within the same professional sector. Thereare also reciprocal opportunities for Americans to travelabroad to this region. The goal of these exchanges is forthe Eurasian and American participants to bring back ideasto their home countries. According to Ambassador WilliamBurns, the current Under Secretary of State for PoliticalAffairs, The strength of Open World lies in its ability to

build lasting and ongoing ties and relations.10

The challenge in evaluating this program is how to measurethe impact of relationships that are formed through theparticipation in professional and leadership programs,many of which are short-term exchanges with long-termprospects. Many outcomes of Open World programs areseen in the long-term, but there is pressure from keystakeholders to see immediate results.

To capture the results of its programs, both immediatelyand longitudinally, Open World has devised a methodology

that includes eight bins (categories) of citizen diplomacyimpacts and actions that are measurable. These include:1) Partnerships (e.g. creation of sister courts); 2) Projects (e.g.Health Fairs); 3) Multipliers (e.g. presentations to others);4) Press (foreign and domestic); 5) Reciprocal Visits (e.g.Russia hosting American visitors); 6) Benefit to Americans(e.g. presentations); 7) Professional Advancement; and 8)Contributions.

The data is collected in a number of ways. Facilitators andhost institutions provide feedback and results reports afterthe completion of the program. Alumni also check in regularlywith regional program representatives to provide updates onoutcomes and new developments that can be attributed toparticipation in the Open World leadership exchange.

Additional data is collected electronically and throughtelephone contact. This methodology allows for thecontinuous recording of program outcomes, of which over3,800 have already been captured, reflecting outcomesof programs since 1993. The bin system records only

specific program results that fall within one of the eightaforementioned categories. This system of measurementprovides both the macro picture of results throughquantifying information as well as anecdotal evidence thatelaborates on what the numbers signify. Ambassador JohnOKeefe, Executive Director of Open World, also notes thatthis system of concretizing impact ensures that the programevaluation measures what is done but also calls for reflectingon the significance of program outcomes.

The Hubert H. Humphrey Fellowship Program

The Humphrey Fellowship Program provides a one-year,non-degree professional development opportunity forprofessionals from around the world who demonstrateleadership skills and a commitment to public service. Thisprogram supports an exchange of knowledge betweenrising leaders in public services fields around the world andthe people they live and work with in the United States. Italso aims to establish long-term impactful and productiverelationships between program participants and theircounterparts in the U.S. Approximately 200 Humphrey

Fellows are selected annually and are hosted by U.S.universities and professional affiliation sites. In the thirty-two years of the Humphrey Fellowship Program, over 4,400fellows from 157 countries have been hosted in the U.S.

An assessment of program impact was commissioned in1998 by the Office of Policy and Evaluation in the Bureau ofEducational and Cultural Affairs (ECA) and was conducted



14/22


by Macro International, Inc. (2000). In addition, theevaluation was intended to offer a model for evaluatingother international exchange programs.

Macro collected baseline data, conducted an assessmentof the selection process, and completed an in-depth studyof three universities that host Humphrey fellows. A surveywas administered to approximately 200 fellows and casestudies were conducted in five countries.

The evaluation focused on the following types of outcomesand impacts: work experience, development of knowledgeand skills, contact with American and other fellows, andunderstanding of the United States and Americans. Withinthese categories, a number of facts were assessed, including

fellows employment sectors prior to the fellowship andafterward; the readjustment experience upon return totheir home countries; implementation of new ideas at workafter the fellowship, including the use of new technologies,development of knowledge and skills; and continuedcontact with Americans and other fellows.

The evaluation generated qualitative and quantitative datathat documents the impact of the Humphrey FellowshipProgram on the participants. The interview and surveyresponses of program participants demonstrate thatsignificant results have been achieved that meet the goals

of the program.

A new evaluation of the Humphrey Fellowship Programcommenced in 2011. The U.S. Department of StateDepartment has contracted Social Solutions International,Inc. (SSi) to conduct a ten-year study of the program.

The International Visitor Leadership Program (IVLP)

The International Visitor Leadership Program (IVLP)is supported by the U.S. Department of States

Bureau of Educational and Cultural Affairs (ECA) topromote international professional exchange and buildunderstanding between the people of the United Statesand other countries. IVLP participants are selected basedon their potential as emerging leaders and participate inthematic programs for up to three weeks. This programhas existed for over 70 years and has hosted over135,000 professionals from around the world, receivingapproximately 1,400 participants annually.

ORC Macro conducted an Outcome Assessment of theprogram, focusing on the outcomes of IVLP participantsfrom four countries of the former Soviet Union who wereparticipants in the program between 1996 and 2001

(ORC Macro, 2006). The objective of the study was todetermine the impact of the IVLP exchange experience onthe professional lives of alumni, their affiliated organizationsand home countries, and to ascertainin quantitativeand qualitative termsthe effectiveness of the programin achieving its legislative mandate of increasing mutualunderstanding between the people of the United States andthe people of other countries.11

ORC Macro worked with local researchers in Russia toconduct over 800 in-person interviews and four focus groups

with IVLP alumni. This post-program outcome assessmentallowed the evaluators to measure impacts of the programas well as the changed perceptions of program participantson major regional events unfolding at the time the outcomeassessment was administered.

The evaluation assessed four program outcomes: 1) alumnisatisfaction; 2) professional and cultural learning; 3) effectson behavior/application of knowledge, and; 4) linkages, tiesand institutional changes. The study found that the IVLPprogram has demonstrable results that show short-termand long-term impacts on programs participants as well as

multiplier effects on participants home countries as well ason the United States.

Many participants in the IVLP program are in highpositions in their respective sectors and have influenceover potential points of interest with the United States.Over 60 percent reported introducing new ideas andknowledge to their work, and more than half introducedeither new initiatives or new policies based on theknowledge gained during the IVLP program.Nearly a quarter of the participants (24 percent)established new exchanges.

The evaluation found increased awareness and sharing ofinformation regarding the U.S. and contribution to the socialand civic development in the home countries of participants.Anecdotal information on the participants experience wasalso collected in the program assessment.



15/22


The case studies presented in this report illus-trate a variety of goals for conducting evaluationsthat measure program outcomes and attempt to

quantify the impacts of citizen diplomacy. This variationreflects the fact that each program is unique, as are theprogram outcomes. Some evaluations focus on measuringpersonal impacts on American citizens who travel abroad,while others measure impacts on communities that areserved by international volunteers. Yet others evaluate themultiplier effects in host or home countries, while someexamine the long-term program impacts on educationalattainment.

The diagram below illustrates key outcomes and skillsgained from participation in various international citizendiplomacy programs. There are a number of overlappingkey outcomes between U.S. and international participants,including long-term multiplier effects.


Partnerships

Reciprocal visits

Multiplier effectson home and hostcommunities

Understanding of theU.S. and Americans

English language skills

Academic impacts

Professionaladvancement

Creation of new

programs/NGOs

Interculturalcompetency

Foreign languagefluency

Academic impacts

Professional

advancementInternational careers

U.S. Participants International Participants

There is also variation in the overall approach of programevaluations presented in this report. Some organizationschoose to do their own internal evaluations of programs,often with the goal of organizational learning and improve-ment, while others opt to have external experts conductthe research to ensure an unbiased assessment of the suc-cesses and failures of the program. Given the internationaland cross-cultural emphasis of many of the programs, theevaluations often include locally-based experts and researchassistants who conduct some or all of the research and datacollection, often in languages of the country/community. Arange of methodologies and assessment tools are used inconducting evaluations of programs.12 The summary tableon the next page includes the evaluation methodolgy andkey outcomes of 12 case studies. Key recommendations ofthe report are included in the executive summary.

Figure 1: Key Outcomes - Comparison of U.S. and International Participant Outcomes

E. Summary


16/22



Program Year ofEvaluation

Evaluator Focus of Evaluation EvaluationMethodology

Key Findings/Outcomes

High School Exchange ProgramsAFS InterculturalPrograms(Educational ResultsImpact Study)

2002-2005 Hammer Consulting Impact on foreign language fluencyand intercultural competence,anxiety, networking, and knowledge

IDI, developmental scoressurvey

Positive impacts on intercultucompetency, comfort in adjusto intl environment, networkincreased foreign languagefluency.

AFS InterculturalPrograms (Long-termImpact Study)

2007-2008 Hammer Consulting Long-term impacts of participationin AFS HS programs oninternational life, knowledge, andinternational cultural awareness

Web-based survey(developmental scoresusing IDI)

Positive long-term impactson intercultural competency,comfort in adjusting to intlenvironment, networking,international lifestyle choices,academic impacts, maintaineforeign language fluency.

International Volunteer ProgramsCenter for SocialDevelopment

2010 Center for SocialDevelopment

Assess international awareness,intercultural relations, internationalsocial capital, international careerintentions

Quasi-experimentalvolunteer survey andcross-sectional interviewsand focus groups

Increase in internationalawareness, international sociacapital, and international careintentions.

Peace Corps 2008 Office of Research,Evaluation, andMeasurement,Peace Corps

Measure achievement of PeaceCorps mission of helpingpromote a better understandingof Americans on the part of thepeoples served

Semi-structured, multi-language survey

Interaction with volunteershas a positive effect on level ounderstanding of Americans;increasingly positive opinion oAmericans.

Postsecondary Exchange and Study Abroad Programs

Fulbright U.S. ScholarProgram

2001 SRI International Broad impacts of program onindividuals and institutions in the

Fulbright scholars host and homecountries; impact on personal andprofessional life; achievement ofFulbright legislative goals

Online and hard copysurveys

Fulbright achieving legislativemandate of promoting mutua

understanding and cooperatiobetween the U.S. and othernations; impacts on scholars acolleagues, students, friends,and families through multiplieeffects.

Fulbright VisitingScholar Program

2005 SRI International Impact on individual participants invisiting Fulbright scholar program;assessment of factors beyondindividual gain, including multipliereffects on scholars home countries

Interviews, questionnaires Fulbright achieving legislativemandate; visiting scholars repan increased understanding ofthe U.S.

GeorgiaLearning Outcomes

of Students StudyingAbroad ResearchInitiative (GLOSSARI)

2000onward

University Systemof Georgia

Impact of study abroad programson academic and learning

outcomes and interculturalcompetency

Quantitative analysis ofstudent outcomes data;

questionnaires

Increased academic performahigher graduation rates, and

more intercultural competencof students who studied abrostudy abroad does not hinderacademic outcomes of at-riskstudents but enhances it.

Benjamin A. GilmanScholarship Program

2009 Macro International Impact on participants, homeuniversity, host country

Survey, including open-ended questions; focusgroup; interviews

Increased language proficiencacademic and career impacts.

Summary Table of Evaluation Methodology and Key Outcomes of Twelve Case Studies


17/22



Program Year ofEvaluation

Evaluator Focus of Evaluation EvaluationMethodology

Key Findings/Outcomes

Study Abroad for

Global Engagement(SAGE) Project

2006 University of

Minnesota andForum on EducationAbroad

Long-term impact of undergraduate

study abroad on global engagementof study abroad participants

Global Engagement

Survey; open-endedsurveys; individualinterviews; case studies

Study abroad influences choic

in social entrepreneurship, civengagement, and voluntarysimplicity in ones life. It also ha positive effect on academicachievement.

Professional and Leadership Exchange Programs

Open WorldLeadership Center

Ongoing Open WorldLeadership Center

1) Partnerships, 2) Projects,3) Multipliers, 4) Press,5) Reciprocal Visits, 6) Benefitto Americans, 7) ProfessionalAdvancement, 8) Contributions

Post-program feedbackand results reports fromfacilitators and hostinstitutions, alumnicheck-ins, electronic andtelephone communication

All categories of program haveachieved measurable results; o3,800 outcomes recorded.

The Hubert

H. HumphreyFellowship Program

1998 Macro International Work experience, development

of knowledge and skills, contactwith American and other fellows,and understanding of the U.S. andAmericans

Baseline data collection;

assessment of selectionprocess; in-depth casestudies; surveys

Increased employment in

public sectors and NGOs;implementation of new ideas work; development of knowleand skills; understanding of thU.S. and Americans.

The InternationalVisitor LeadershipProgram (IVLP)

2006 ORC Macro Determine the impact of exchangeexperience on the professionallives of alumni, their organizations,and home countries; evaluate theeffectiveness of the program inachieving its legislative mandate ofincreasing mutual understandingbetween the people of the U.S. andother countries

In-person interviews andfocus groups

Positive short and long termimpacts on 1) alumni satisfact2) professional and culturallearning; 3) effects on behavioapplication of knowledge, and4) linkages, ties and institutiochanges.

Summary Table of Evaluation Methodology and Key Outcomes of Twelve Case Studies


18/22

1See the SAGE Handbook for Intercultural Competence, editedby Darla Deardorff for an expansive list of various instruments for measuringintercultural competency.

2 More information is available on the following websites:

http://www.iie.org/en/The-Power-of-Giving/Giving-Opportunities/Fulbright-Legacy-Fund-Endowmenthttps://alumni.state.gov

3 http://www.afs.org/afs_or/view/what_we_do

4IDI was developed by Mitchell R. Hammer and Milton Bennett. It is a statistically proven measure of intercultural development stages followingBennetts Developmental Model of Intercultural Sensitivity.

5http://www.peacecorps.gov/index.cfm?shell=about.mission

6http://www.sri.com/policy/csted/reports/international/executive_report.pdf

7A list of critical need languages can be found at http://www.iie.org/en/Who-We-Are/News-and-Events/Press-Center/Press-Releases/2006/2006-09-05-Gilman-Scholarship-Spring-2007-Supplement-Awards

8SAGE defines voluntary simplicity as the effort to lead a more modest, simple lifestyle. Examples are riding a bike to work, taking a job that pays lessbut contributes more to the common good, or being motivated to use recycled products and to practice active recycling. http://www.cehd.umn.edu/projects/sage/GlobalEngagementSurvey.pdf

9http://www.openworld.gov/press/print.php?id=95&lang=1

10http://www.openworld.gov/news/print.php?id=304&lang=1

11http://exchanges.state.gov/media/pdfs/office-of-policy-and-evaluations/completed-program-evaluations/executive-summary/ivlp-eurasia-study-executive-summary_january-2006.pdf

12See endnote 1.

Endnotes



19/22

Ailes, C.P. & Russell, S.H. (2002). Outcome Assessment of the U.S. Fulbright Scholar Program: Executive Report. Arlington, VA: SRI International.

Chow, P. & Bhandari, R. (2010). Open Doors 2010: Report on International Educational Exchange. New York: Institute of International Education.

Deardorff, D., Ed. (2009). The SAGE Handbook of Intercultural Competence. Thousand Oaks, CA: Sage Publications.

Emery, A., Horwitz, I., & Jennings, H. Draft Evaluation of the Benjamin A. Gilman Scholarship Program. Washington, DC: Macro International, Inc.

Hammer, M.R., Bennett, M.J., & Wiseman, R. (2003). Measuring Intercultural Sensitivity: The Intercultural Development Inventory. InternationalJournal of Intercultural Relations, 27(4), 421-443.

Hammer, M.R. (2005). The Educational Results Study: Assessment of the Impact of the AFS Study Abroad Experience. New York: AFS InterculturalPrograms, Inc.

Hansel, B. & Chen, Z. (2008). AFS Long Term Impact Study: Report 1: 20 to 25 years after the exchange experience, AFS alumni are compared withtheir peers. New York: AFS Intercultural Programs, Inc.

Hansel, B. (2008). AFS Long Term Impact Study: Report 2: Looking at intercultural sensitivity, anxiety, and experience with other cultures. New York:

AFS Intercultural Programs, Inc.

Kerley, J. (2010). Peace Corps Host Country Impact Studies [PowerPoint slides]. Presented for the Peace Corps Office of Strategic Information,Research, and Planning.

Lough, B.J., McBride, A.M., & Sherraden, M.S. (2007). The Estimated Economic Value of a US Volunteer Abroad (CSD Working Papers RP07-29). SaintLouis, MO: Center for Social Development. Retrieved from http://csd.wustl.edu/Publications/Documents/WP07-29.pdf.

Lough, B.J., McBride, A.M., & Sherraden, M.S. (2009). Measuring Volunteer Outcomes: Development of the International Volunteer Impacts Survey(CSD Working Papers RP09-31). Saint Louis, MO: Center for Social Development. Retrieved from csd.wustl.edu/Publications/Documents/WP09-31.pdf.

Macro International. (2000). Evaluation of the Hubert H. Humphrey Fellowship Program. Washington, DC: Macro International, Inc. Retrieved fromhttp://exchanges.state.gov/media/pdfs/office-of-policy-and-evaluations/completed-program-evaluations/executive-summary/humphrey-fellowship-exec-summary_jan-2000.pdf

ORC Macro. (2006). International Visitor Leadership Program outcome assessment: Executive Summary. Calverton, MD: ORC Macro. Retrieved fromhttp://exchanges.state.gov/media/pdfs/office-of-policy-and-evaluations/completed-program-evaluations/executive-summary/ivlp-eurasia-study-executive-summary_january-2006.pdf

Paige, R.M., Fry, G.W., Stallman, E.M., Josic, J., & Jon, J. Study Abroad for Global Engagement: The long-term impact of mobility exercises. Berlin: AFSInterkulturelle Begegnungen. Retrieved from http://en.afs60.de/web/index.php?TID=99.

Paige, R.M.Stallman, E.M., Jon, J., & LaBrack, B. (2009). Proceedings from NAFSA Annual 2009 Conference: Study abroad for global engagement: Thelong-term impact of international experiences. Minneapolis, MN: University of Minnesota. Retrieved from: http://www.cehd.umn.edu/projects/sage/NAFSA-5-29-09Presentation.pdf.

Redden, E. (2010). Academic outcomes of study abroad. Inside Higher Ed, July 13. Retrieved from http://www.insidehighered.com/news/2010/07/13/abroad.

SRI International Center for Science, Technology, and Economic Development. (2005). Outcome Assessment of the Visiting Fulbright Scholar Program.Arlington, VA: SRI International.

Sutton, R. & Rubin, D. (2004). The GLOSSARI Project: Initial findings from a system-wide research initiative on study abroad learning outcomes.Frontiers: The Interdisciplinary Journal of Study Abroad 10, 65-82. Retrieved from www.frontiersjournal.com/issues/vol10/vol10-04_SuttonRubin.pdf.

Bibliography



20/22

We are grateful to our expert colleagues who have contributed to this report by sharing their research and findings onevaluations that measure the impact of citizen diplomacy. The material and content shared by the panel presenters at theU.S. Citizen Diplomacy Summit in Washington, DC, on November 17, 2010, is included in this report.

Presenters at the Roundtable on Evaluating and Measuring the Impact of Citizen Diplomacy:

Allan Goodman, Chair, President and CEO, Institute of International Education

Rick Ruth, Senior Advisor, Office of the Assistant Secretary, Bureau of Education and Cultural Affairs, U.S. Department ofState

Ambassador John OKeefe, Executive Director, Open World Leadership Center, Library of Congress

Janet Kerley, Chief, Research Evaluation and Measurement, Peace CorpsBenjamin Lough, Project Manager, Center for Social Development, Washington University in St. Louis; Resident Consultant,Office of the Executive Coordinator, United Nations Volunteers ProgramFrancisco (Tachi) Cazal, President & CEO, AFS Intercultural Programs

We would also like to extend special thanks to Ann Schodde, President and CEO, U.S. Center for Citizen Diplomacy.

Acknowledgments



21/22

The Institute of International Education, founded in 1919, is a leading not-for-profit educational and cultural exchangeorganization in the United States. IIE has a network of 18 offices worldwide and more than 1,100 college and universitymembers. In collaboration with governments, corporations, private foundations, individuals, and other sponsors, IIE designs

and implements scholarship programs and programs of study, training, and exchange for students, educators, scholars, andprofessionals from all sectors. These programs include, for example, the Fulbright and Humphrey Fellowships and the GilmanScholarships administered for the U.S. Department of State, the Boren Scholarships and Fellowships and The LanguageFlagship administered for the National Security Education Program, and the International Fellowships Program, implementedfor the Ford Foundation. www.iie.org.

OPEN DOORS RESOURCEShttp://www.iie.org/opendoors

The Open Doors Report on International Educational Exchange, supported by the U.S. Department of States Bureau ofEducational and Cultural Affairs, provides an annual, comprehensive statistical analysis of academic mobility between the

United States and other nations, and trend data over 60 years.

CENTER FOR INTERNATIONAL PARTNERSHIPS IN HIGHER EDUCATIONhttp://www.iie.org/cip

The IIE Center for International Partnerships in Higher Education assists colleges and universities in developing and sustaininginstitutional partnerships with their counterparts around the world. A major initiative of the Center is the InternationalAcademic Partnerships Program, funded by the U.S. Department of Educations Fund for the Improvement of PostsecondaryEducation (FIPSE).

ATLAS OF STUDENT MOBILITY

http://www.iie.org/projectatlas

Project Atlas tracks migration trends of the millions of students who pursue education outside of their home countrieseach year. Data are collected on global student mobility patterns, country of origin, as well as leading host destinations forhigher education.

IIE WHITE PAPERSwww.iie.org/StudyAbroadCapacity

IIE White Papers address various issues of increasing capacity in the U.S. and abroad to help pave the way for substantialstudy abroad growth.

What International Students Think About U.S. Higher Education (May 2011)Expanding Study Abroad Capacity at U.S. Colleges and Universities (May 2009)Promoting Study Abroad in Science and Technology Fields (March 2009)Expanding U.S. Study Abroad in the ArabWorld: Challenges & Opportunities (February 2009)Expanding Education Abroad at Community Colleges (September 2008)Exploring Host Country Capacity for Increasing U.S. Study Abroad (May 2008)Current Trends in U.S. Study Abroad & the Impact of Strategic Diversity Initiatives (May 2007)

IIE Resources



22/22