evidence of success: assessing student learning outcomes in international education dr. darla k....

Evidence of Success:Evidence of Success: Assessing Student Assessing Student

Learning Outcomes in Learning Outcomes in International Education International Education

Dr. Darla K. DeardorffDr. Darla K. Deardorff

Association of International Association of International Education AdministratorsEducation Administrators

Duke UniversityDuke UniversityD.K. Deardorff, Duke University, 2012D.K. Deardorff, Duke University, 2012

OverviewOverview Definitions, Context, Key QuestionsDefinitions, Context, Key Questions

Framing Assessment: Logic ModelFraming Assessment: Logic Model

Framing Global Human Resources Outcomes: Framing Global Human Resources Outcomes: Intercultural Competence FrameworkIntercultural Competence Framework

Implementing Assessment Implementing Assessment

D.K. Deardorff, Duke University, 2012D.K. Deardorff, Duke University, 2012

DefinitionsDefinitions

Assessment – information on and for Assessment – information on and for student learning student learning

Evaluation – applied research, Evaluation – applied research, systematic investigation for program systematic investigation for program improvementimprovement

Key: Defined criteria (objectives)Key: Defined criteria (objectives)D.K. Deardorff, Duke University, 2012D.K. Deardorff, Duke University, 2012

LevelsLevels

InstitutionInstitution

Program/DepartmentProgram/Department

StudentStudent

Inputs (resources), Outputs (numbers), Inputs (resources), Outputs (numbers), Outcomes at each levelOutcomes at each level


Assessment-International Assessment-International Education ContextsEducation Contexts

““At Home”At Home” – – curriculum, extra-curricular activities, curriculum, extra-curricular activities, community involvement/impact, domestic/internat’l community involvement/impact, domestic/internat’l student interaction, policy, financial/economic, campus student interaction, policy, financial/economic, campus climateclimate

AbroadAbroad (Cross-border)(Cross-border) – – involves cross-involves cross-border delivery of education through exchange, distance border delivery of education through exchange, distance and e-learning, branch campuses, partnerships, host and e-learning, branch campuses, partnerships, host community impactcommunity impact


Key QuestionsKey Questions

What is the evidence of student What is the evidence of student success in international education?success in international education?

How will students know they have How will students know they have been successful in their learning?been successful in their learning?

How will others know that our How will others know that our students are successful ? students are successful ?


ASSESSMENT APPROACHASSESSMENT APPROACH

PROGRAM LOGIC MODEL…PROGRAM LOGIC MODEL…

……Program “road map:” Program “road map:”

1) 1) Where are you going? Where are you going?

2) How will you get there? 2) How will you get there?

3) How will you know when you’ve 3) How will you know when you’ve arrived?arrived?


APPROACHAPPROACH

Moving beyond Moving beyond

OUTPUTS to OUTCOMESOUTPUTS to OUTCOMES

What are indicators of success in What are indicators of success in international education?international education?

What are meaningful outcomes (results) of What are meaningful outcomes (results) of internationalization efforts?internationalization efforts?


__________________________________________________________________________

INTERNATIONALIZATIONat institutions of higher education

Inputs/Resources

needed for implementation of components of internationalization|

Activities/Components of Internationalization(college leadership, faculty involvement, curriculum, study abroad, international

students/scholars/faculty, international co-curricular units) |

Outputs of Internationalization(i.e., number of international students, number of study abroad programs, number of students

studying foreign languages, etc.)|

Outcomes of InternationalizationIntercultural competence – what is it?

How do higher education administrators define it? intercultural experts?How can it be assessed?

= Long-Term Impact of Internationalization

_______________________________________________________________General program logic model applied to internationalization. (Deardorff, 2004,)D.K. Deardorff, Duke University, 2012D.K. Deardorff, Duke University, 2012

Program Logic Model - InputsProgram Logic Model - Inputs

WHAT is invested in the programWHAT is invested in the program

Inputs include resources such as Inputs include resources such as faculty, staff, money, time, faculty, staff, money, time, partnerspartners


Program Logic Model - Program Logic Model - ActivitiesActivities

What we DOWhat we DO

Includes learning opportunities - Includes learning opportunities - training, curriculum, education training, curriculum, education abroad, etc.abroad, etc.


Program Logic Model - Program Logic Model - OutputsOutputs

WHO participatesWHO participates

Includes numbers of international Includes numbers of international students, % of students studying abroad, students, % of students studying abroad, number of international courses, etc.number of international courses, etc.


Program Logic Model - Program Logic Model - OUTCOMESOUTCOMES

The meaningful RESULTS of what we do The meaningful RESULTS of what we do

Provides the Provides the meaningmeaning behind the outputs behind the outputs (numbers)(numbers)


Program Logic Model - Program Logic Model - ImpactImpact

What are theWhat are the long-term long-term RESULTS?RESULTS?

Includes decisions, policy Includes decisions, policy changes, impact on conditions changes, impact on conditions (human, economic, civic, etc)(human, economic, civic, etc)


Framing AssessmentFraming Assessment

InputsInputs ActivitiesActivities OutputsOutputs OutcomesOutcomes ImpactImpact


ASSESSING OUTCOMESASSESSING OUTCOMES

Starting point Starting point ((Alignment is key! ):Alignment is key! ):

Mission StatementMission Statement intointo

Goals Goals into into

Measurable ObjectivesMeasurable Objectives


Framing Global Human Framing Global Human Resources Outcomes Resources Outcomes

Defining intercultural competence….Defining intercultural competence….

Knowledge, Skills, AttitudesKnowledge, Skills, Attitudes

Internal and External Outcomes Internal and External Outcomes (see Deardorff, 2006, 2009 – Deardorff ICC (see Deardorff, 2006, 2009 – Deardorff ICC

Framework handout)Framework handout)


Example: Example: Global Learning GoalGlobal Learning Goal

Goal: Develop interculturally Goal: Develop interculturally competent studentscompetent students

Outcome: Students will state 3 Outcome: Students will state 3 different cultural perspectives which different cultural perspectives which can give alternative solutions to the can give alternative solutions to the given situation/problemgiven situation/problem

Well defined, specific measurable objectives, Well defined, specific measurable objectives, aligned and integrated assessment measuresaligned and integrated assessment measures


Where to go from here…Where to go from here…

Articulate and define your goals and Articulate and define your goals and outcomes outcomes

Prioritize the key goals/outcomesPrioritize the key goals/outcomes

Determine methods based on prioritized Determine methods based on prioritized goals/outcomesgoals/outcomes

EAIE 2009- Deardorff presentation EAIE 2009- Deardorff presentation

Assessment MethodsAssessment Methods

Methods determined by clearly Methods determined by clearly articulated assessment questionsarticulated assessment questions

Include:Include: DIRECT METHODSDIRECT METHODS

INDIRECT METHODSINDIRECT METHODS

= COMBINATION OF QUANTITATIVE AND QUALITATIVE = COMBINATION OF QUANTITATIVE AND QUALITATIVE METHODSMETHODS


DIRECT METHODSDIRECT METHODS==DemonstrationDemonstration of of

student learning student learning (authentic (authentic assessment) assessment)

Embedded course assessmentEmbedded course assessment PortfoliosPortfolios PerformancePerformance TestingTesting Papers/projectsPapers/projects CapstoneCapstone


INDIRECT METHODSINDIRECT METHODS==PerceptionPerception of of student learningstudent learning

Surveys (inc. self-assessments)Surveys (inc. self-assessments) InterviewsInterviews Focus groupsFocus groups Curriculum/transcript analysisCurriculum/transcript analysis Documented dataDocumented data


Implementing Assessment Implementing Assessment

1) Assessing Institutional Context & 1) Assessing Institutional Context & ResourcesResources

2) Putting together an Assessment 2) Putting together an Assessment TeamTeam

3) Developing an Assessment Plan3) Developing an Assessment Plan

4) Garnering Support for the 4) Garnering Support for the Assessment PlanAssessment Plan

5) Implementing the Plan5) Implementing the PlanD.K. Deardorff, Duke University, 2012D.K. Deardorff, Duke University, 2012

Assessment Process/CycleAssessment Process/Cycle

Define outcomes (based on mission/goals) Define outcomes (based on mission/goals) and establish measurable criteriaand establish measurable criteria

Identify appropriate assessment methodsIdentify appropriate assessment methods Collect dataCollect data Analyze dataAnalyze data Use data Use data – design and apply changes– design and apply changes Communicate dataCommunicate data Evaluate the assessment processEvaluate the assessment process


Using Assessment ResultsUsing Assessment Results

• Improving student teaching & Improving student teaching & learninglearning

• Curriculum/program improvementCurriculum/program improvement• Inform decision-makingInform decision-making• Secure resourcesSecure resources• AccountabilityAccountability• AdvocacyAdvocacy• Academic legitimacy/faculty supportAcademic legitimacy/faculty support


Assessment – Lessons LearnedAssessment – Lessons Learned

CCollaborate - Put together Assessment ollaborate - Put together Assessment TeamTeam

AAdapt - Build on what you havedapt - Build on what you have MMeasure – what is valued (align!)easure – what is valued (align!) PPlan - Develop assessment planlan - Develop assessment plan UUse – use what you assessse – use what you assess SSupport – from leadership, upport – from leadership,

stakeholdersstakeholdersDeardorff, 2008Deardorff, 2008


Final thoughtFinal thought

“…“…..assessment efforts should not be assessment efforts should not be concerned about valuing what can be concerned about valuing what can be measured but, instead, about measured but, instead, about measuring that which is valued.” measuring that which is valued.” (Banta, et al, 1996)(Banta, et al, 1996)


Thank you….Thank you….

……. Domo Arigato!. Domo Arigato!


For further informationFor further information….….Bolin, M. C. (2007). Bolin, M. C. (2007). A Guide to Outcomes Assessment in Education AbroadA Guide to Outcomes Assessment in Education Abroad . Forum on . Forum on

Education Abroad. Education Abroad.

Deardorff, DK (2006) Identification and Assessment of Intercultural Competence in Deardorff, DK (2006) Identification and Assessment of Intercultural Competence in Journal Journal of Studies in International Educationof Studies in International Education (Fall 2006). (Fall 2006).

Deardorff, DK (2007). Principles of International Education Assessment in Deardorff, DK (2007). Principles of International Education Assessment in IIENetworkerIIENetworker (Spring 2007). (Spring 2007).

Deardorff, DK (2009). Deardorff, DK (2009). The SAGE Handbook of Intercultural CompetenceThe SAGE Handbook of Intercultural Competence, (Sage)., (Sage).

Green, M.F., Luu, D. & Burris, B. (2008). Green, M.F., Luu, D. & Burris, B. (2008). Mapping Internationalization on US Campuses: Mapping Internationalization on US Campuses: 2008 Edition. 2008 Edition. Washington DC: ACE.Washington DC: ACE.

For additional articles on these topics, please contact For additional articles on these topics, please contact

Darla K. Deardorff at Darla K. Deardorff at [email protected]

……. . ……D.K. Deardorff, Duke University, 2012D.K. Deardorff, Duke University, 2012

evidence of success: assessing student learning outcomes in international education dr. darla k....

Documents

student learning outcomes

number of students

student learning evaluation

evidence of student

number of study

evidence of success

program improvementkey

outputs numbers