annual beap presentation
DESCRIPTION
Annual BEAP Presentation. BPD Annual Conference Portland, OR March 2012. The BEAP Team. Vicky Buchan Colorado State University [email protected] - PowerPoint PPT PresentationTRANSCRIPT
Annual BEAP PresentationBPD Annual ConferencePortland, ORMarch 2012
The BEAP TeamVicky BuchanColorado State [email protected]
Brian ChristensonLewis-Clark State [email protected]://www.lcsc.edu/bchristenson
Ruth Gerritsen-McKaneUniversity of [email protected]
Tobi DeLong HamiltonLewis-Clark State College, [email protected]
Roy (Butch) RodenhiserBoise State [email protected]
Marshall Smith (Retired)[email protected]
Phil [email protected]
What BEAP Can Do For Your Program:•Student demographics•Assessment on Knowledge, Skills, &
Values•Employment information•Advising feedback•Professional development of students &
alumni/ae•Peer comparison by region, program type,
auspice & nationally•…and more
Matrix ExampleCompetency Alumn
iExit Employer FCAI FPPAI
2.1.7—Apply knowledge of human behavior and the social environment.-
A. utilize conceptual frameworks to guide the processes of assessment, intervention, and evaluation;
B. critique and apply knowledge to understand person and environment.
A. C7-12
B. C12
A. D7-12
B. D12
A. Q7-10
B. None
A. 17, 19
B. 14,22,23,56,57
A. 39-40
B. 39-40
The BEAP Matrix Can Be Found in:DeLong-Hamilton, T., Buchan, V. V., Hull, G. H., Christenson, B., Gerritsen-McKane, R., Rodenhiser, R. W., & Smith, M. L. (2011). Responding to the 2008 EPAS: Baccalaureate education assessment direct and indirect measurement. Journal of Baccalaureate Social Work, 16(1), 17-31.
Sample ProfileOverview of all Respondents
Instrument
2000-2009 2010 2011 Overall
Entrance 31,107 1,374 2,432 34,913
Exit 19,768 623 1,710 22,101
Alumni 5,445 101 212 5,758
Employer 1,963 40 74 2,077
Totals 56,483 2,138 4,428 64,849
Frequencies for Auspices of College or University, 2011Instrument Public Private
Denominational
Private Non-denomination
al
Entrance 1,442 700 286
Exit 947 477 286
Alumni 66 68 78
Employer 36 30 8
Frequencies for Program Type, 2011Instrument BSW Only Combined
BSW/MSW
Entrance 520 144
Exit 1,228 482
Alumni 164 48
Employer 66 8
Entrance Survey – Student EthnicityEthnicity 2000-2010
%N = 32,481
2011 %N = 2,620
Overall %N =
35,101European / Caucasian 62.6 53.6 58.1
African American / Black
19.8 21.8 20.8
Native American 7.5 5.0 6.3
Other Hispanic / Latino
4.9 5.4 5.2
Other 3.8 4.2 4.0
Asian / Pacific Islander 3.3 4.3 3.8
Chicano / Mexican 3.8 3.9 3.9
Puerto Rican 2.3 1.8 2.1
Exit SurveyKnowledge, Values, Skills*2010 Likert Scale changed from 0-10 to 1-9
Mean Score
2000-2009
2010 2011N=1,655
Overall
Knowledge 7.34 7.15 7.35 7.28
Values 7.42 7.34 8.34 7.70
Skills 8.67 8.26 7.55 8.16
Exit SurveyAdvising Scores*2010 Likert Scale changed from 0-10 to 1-9
Question 2000-2009N=
19,768
2010N= 623
2011N= 1,644
OverallN=
22,035
Course Selection / Curriculum Planning
6.88 6.93 7.11 6.98
Professional Advising
6.59 6.39 6.59 6.52
Career Planning
6.17 5.85 6.16 6.06
Foundation Curriculum Assessment Instrument
Purpose of the FCAI1. Provides Pre/Post test in seven major
curricular areas of a BSW program2. Provides a direct measure to assist
programs with evaluation of their curriculum
3. Assists with identification of curricular areas that may need attention
4. Provides national comparative data
Curricular Components Curriculum Area Number of
Questions
Practice 13
Human Behavior & Social Environment 10
Policy 9
Research 9
Ethics and values 8
Diversity 8
Social and Economic Justice 7
Sample HBSE Question•The concept “person-in-environment”
includes which of the following:a. Clients are influenced by their
environmentb. Clients influence their environmentc. Behavior is understood in the context of
one’s environmentd. All of the above
Sample Practice Question•Determining progress toward goal
achievement is one facet of the _____ stage.
▫a. Engagement ▫b. Evaluation ▫c. Assessment ▫d. Planning
Testing History – 2006-2011 TEST #questions N• Version 3 65 n= 305• Version 4 55 n= 381• Version 5 82 n= 286• Version 6 74 n= 36• Version 7 72 n= 318• Version 8 64 n=2231• Version 9 64 n=5839
Reliability Testing• Version 8
–Tested in two junior practice classes–Students tested twice, 2 weeks apart–Pearson’s correlation coefficient
•r = .86– a
Item difficulty index• Overall difficulty or average should be
around .5 (Cohen & Swerdlik, 2005)
• FCAI = .523 (n=415)
• “This is a very good difficulty level for the test.Not likely to misrepresent the knowledge level of test takers”.
Reliability & Effect Size
•Cronbach’s alpha = .784
•Effect Size d = 6.87
Current Data•Number of Schools: 41 from 28 States•Number of Respondents:
▫Pre-test: 3721▫Post-test: 2118
Schools By Type•BSW Only: 22•BSW/MSW Combination: 8•Not Reported: 1
Schools By Auspice•Public: 32•Private Denomination: 13•Private Non-Denomination: 6
Foundation Curriculum Assessment Instrument (FCAI)Program Name: Students are: Entering BSW Program ______ Exiting BSW Program _______Month ____, Year ____, Semester ____
I. Program Cumulative Scores Compared with all Student Scores
N = Score Range
SD t-test Value
p
Program
National
School X / National Comparison
PRE TEST
POST TEST
SD SIG
School X 33.83 (52.87%)
45.85 (71.61%)
6.18 .000
National 33.11 (51.73%)
38.96 (60.88%)
8.85 .000
Overall Scores Pre-PostBSW Entering
BSW Exiting
T-test
S.D. Sig
Raw Score
33.1 38.96 43.4 8.21 .000
% Correct
51.7 60.8
Scores By Curricular Area:Practice and HBSE
PRE-TEST
POST-TEST
S.D. SIG.
PRACTICE
7.29 8.98 2.28 .000
HBSE 5.47 6.46 1.76 .000
Scores by Curricular Area:Ethics, Diversity, and Soc/Eco Justice
PRE TEST
POST TEST
SD SIG
ETHICS 4.43 5.30 1.57 .000
DIVERSITY 4.01 4.53 1.65 .000
SOC/ECO JUSTICE
4.14 4.62 1.60 .000
Scores by Curricular Area:Policy and Research
PRE TEST
POST TEST
SD SIG
POLICY 3.94 4.59 1.81 .000
RESEARCH 3.73 4.48 1.79 .000
III. Program: BSW Student Scores by Curricular Area 1 Program : Entrance & Exit
Curricular Area Question
SuggestedCompetency
Correct responses and % Entrance
Correct responses and %
Exit
Practice
Question 1 2.1.1C 4/735.48%
39/7750.65%
Question 2 2.1.10A 46/7363.01%
65/7784.42%
Question 4 2.1.10H 21/7328.77%
43/7755.84%
Expansion beyond BSW•Based upon CSWE assertions related to
educational levels in social work education, we expanded testing to three additional groups: ▫MSW foundation students:
entering exiting
▫Advanced standing students: entering
1. Purpose of this instrument: to review and improve curriculum
2. Program will want to “monitor” scores over several years (or several cohorts) for trends.
3. The FCAI can be considered a measure of “value added” from program entry to exit.
Points to keep in Mind
Field Practicum/Placement Assessment Instrument
(FPPAI)
• 55 Likert Scale questions measuring practice behaviors linked to the EPAS 2008 competencies.
• Qualitative feedback form for each domain available for program use.
• Available online and in print format.• Individual program outcomes report with
national comparisons for EPAS 2008 Competencies & Practice Behaviors (coming next year)
• Can be used as a final field assessment and mid-test/post test design.
Assessment Methodology
FPPAI Scale
FPPAI Reporting and EPAS 2008 Overview
• FPPAI Administration Overview (Online & Paper versions)
• FPPAI Individual Reporting
• FPPAI Aggregate Reporting
• Resources available on BEAP website
Field/Practicum Placement Instrument Reporting
• More than 50 Schools utilizing the FPPAI instrument currently
• More than 1,100 administrations to date• National data comparisons available next year• Cronbach’s Alpha reliability test of internal
consistency was completed (mid-point), (n=340, .969)
• Cronbach’s Alpha reliability test of internal consistency was completed (final), (n=778, .975)
Field/Practicum Placement Instrument Reporting
Table 2: Field Placement Assessment Instrument (FPPAI) Evaluation Results: (n=340)
Section Average Minimum Maximum 1. Professional Social Work Skills and Supervision 6.56 1.71 9.00 2. Professional Communication 6.68 2.20 9.00 3. Social Work Values and Ethical Practice 6.55 2.00 9.00 4. Critical Thinking 6.33 1.50 9.00 5. Diversity 6.99 2.60 9.00 6. Human Rights, Social and Economic Justice and Policy Practice 6.44 2.00 9.00 7. Research 6.31 2.00 9.00 8. Human Behavior in the Social Environment 6.34 1.50 9.00 9. Generalist Practice 6.51 2.00 9.00 10. Social Work Practice with Individuals, Families, Groups, Organizations, & Communities 6.37 2.45 9.00
Average 6.51
EPAS Scoring
2.1.1 Identify as a professional social worker and conduct oneself accordingly
A. advocate for client access to the services of social work; 6.41 B. practice personal reflection & selfcorrection to assure continual professional development; 6.37 C. attend to professional roles and boundaries; 6.63 D. demonstrate professional demeanor in behavior, appearance, and communication; 6.84 E. engage in careerlong learning; 6.53 F. use supervision and consultation. 6.39 Total Section Score 6.53
2.1.2 Apply social work ethical principles to guide professional practice.
A. recognize and manage personal values in a way that allows professional values to guide practice; 6.49
B. make ethical decisions by applying standards of the NASW Code of Ethics and, as applicable Statement of Principles 6.62
C. tolerate ambiguity in resolving ethical conflicts; 6.65 D. apply strategies of ethical reasoning to arrive at principled decisions. 6.62 Total Section Score 6.60
2.1.3 Apply critical thinking to inform and communicate professional judgments
A. distinguish, appraise, and integrate multiple sources of knowledge, including research based knowledge, and practice wisdom; 6.31
B. analyze models of assessment, prevention, intervention, and evaluation; 6.38 C. demonstrate effective oral and written communication in working with individuals, families, groups, and colleagues. 6.68
Total Section Score 6.46
2.1.4 Engage diversity and difference in practice.
A. recognize the extent to which culture"s structures & values may oppress, marginalize, alienate, or create/ enhance privilege/ power; 6.99
B. gain sufficient self-awareness to eliminate the influence of personal biases and values in working with diverse groups; 6.86
C. recognize and communicate their understanding of the importance of difference in shaping life experiences; 6.69
D. view themselves as learners and engage those with whom they work as informants. 6.65 Total Section Score 6.80
2.1.5 Advance human rights and social and economic justice.
A. understand the forms and mechanisms of oppression and discrimination; 6.46 B. advocate for human rights and social and economic justice; 6.41 C. engage in practices that advance social and economic justice. 6.31 Total Section Score 6.39
2.1.6 Engage in research informed practice and practice informed research.
A. use practice experience to inform scientific inquiry 6.31 B. use research evidence to inform practice. 6.31 Total Section Score 6.31
2.1.7 Apply knowledge of human behavior and the social environment.
A. utilize conceptual frameworks to guide the processes of assessment, intervention, and evaluation; 6.34
B. critique and apply knowledge to understand person and environment. 6.34 Total Section Score 6.34
2.1.8 Engage in policy practice to advance social and economic wellbeing and to deliver effective social work services.
A. analyze, formulate, and advocate for policies that advance social wellbeing; 6.42 B. collaborate with colleagues and clients for effective policy action. 6.48 Total Section Score 6.45
2.1.9 Respond to contexts that shape practice.
A. continuously discover, appraise, and attend to changing locales, populations, scientific and technological relevant services; 6.33
B. provide leadership in promoting sustainable changes in service delivery and practice to improve the quality of social services. 6.28
Total Section Score 6.31
2.1.10(a)-(d) Engage, assess, intervene, and evaluate with individuals, families, groups, organizations, and communities.
A. substantively and affectively prepare for action with individuals, families, groups, organizations, and communities; 6.41
B. use empathy and other interpersonal skills; 6.72 C. develop a mutually agreedon focus of work and desired outcomes. 6.37 D. collect, organize, and interpret client data; 6.56 E. assess client strengths and limitations; 6.49 F. develop mutually agreedon intervention goals and objectives; 6.34 G. select appropriate intervention strategies. 6.37 H. initiate actions to achieve organizational goals; 6.31 I. implement prevention interventions that enhance client capacities; 6.37 J. help clients resolve problems; 6.31 K. negotiate, mediate, and advocate for clients; 6.61 L. facilitate transitions and endings. 6.30 M. critically analyze, monitor, and evaluate interventions. 6.21 Total Section Score 6.41
Table 2: Field Placement Assessment Instrument (FPPAI) Evaluation Results: (n=778)
Section Average Minimum Maximum 1. Professional Social Work Skills and Supervision 7.25 2.14 9.00 2. Professional Communication 7.36 2.33 9.00 3. Social Work Values and Ethical Practice 7.33 2.00 9.00 4. Critical Thinking 7.07 1.33 9.00 5. Diversity 7.61 2.00 9.00 6. Human Rights, Social and Economic Justice and Policy Practice 7.15 2.00 9.00 7. Research 6.77 1.00 9.00 8. Human Behavior in the Social Environment 6.99 1.00 9.00 9. Generalist Practice 7.26 2.00 9.00 10. Social Work Practice with Individuals, Families, Groups, Organizations, & Communities 7.04 1.00 9.00
Average 7.18
EPAS Scoring
2.1.1 Identify as a professional social worker and conduct oneself accordingly
A. advocate for client access to the services of social work; 7.12 B. practice personal reflection & selfcorrection to assure continual professional development; 7.18 C. attend to professional roles and boundaries; 7.38 D. demonstrate professional demeanor in behavior, appearance, and communication; 7.48 E. engage in careerlong learning; 7.23 F. use supervision and consultation. 7.12 Total Section Score 7.25
2.1.2 Apply social work ethical principles to guide professional practice.
A. recognize and manage personal values in a way that allows professional values to guide practice; 7.25
B. make ethical decisions by applying standards of the NASW Code of Ethics and, as applicable Statement of Principles 7.41
C. tolerate ambiguity in resolving ethical conflicts; 7.43 D. apply strategies of ethical reasoning to arrive at principled decisions. 7.41 Total Section Score 7.37
2.1.3 Apply critical thinking to inform and communicate professional judgments
A. distinguish, appraise, and integrate multiple sources of knowledge, including research based knowledge, and practice wisdom; 6.77
B. analyze models of assessment, prevention, intervention, and evaluation; 6.99 C. demonstrate effective oral and written communication in working with individuals, families, groups, and colleagues. 7.36
Total Section Score 7.04
2.1.4 Engage diversity and difference in practice.
A. recognize the extent to which culture"s structures & values may oppress, marginalize, alienate, or create/ enhance privilege/ power; 7.61
B. gain sufficient self-awareness to eliminate the influence of personal biases and values in working with diverse groups; 7.53
C. recognize and communicate their understanding of the importance of difference in shaping life experiences; 7.40
D. view themselves as learners and engage those with whom they work as informants. 7.37 Total Section Score 7.48
2.1.5 Advance human rights and social and economic justice.
A. understand the forms and mechanisms of oppression and discrimination; 7.17 B. advocate for human rights and social and economic justice; 7.16 C. engage in practices that advance social and economic justice. 7.05 Total Section Score 7.12
2.1.6 Engage in research informed practice and practice informed research.
A. use practice experience to inform scientific inquiry 6.77 B. use research evidence to inform practice. 6.77 Total Section Score 6.77
2.1.7 Apply knowledge of human behavior and the social environment.
A. utilize conceptual frameworks to guide the processes of assessment, intervention, and evaluation; 6.99
B. critique and apply knowledge to understand person and environment. 6.99 Total Section Score 6.99
2.1.8 Engage in policy practice to advance social and economic wellbeing and to deliver effective social work services.
A. analyze, formulate, and advocate for policies that advance social wellbeing; 7.14 B. collaborate with colleagues and clients for effective policy action. 7.15 Total Section Score 7.14
2.1.9 Respond to contexts that shape practice.
A. continuously discover, appraise, and attend to changing locales, populations, scientific and technological relevant services; 6.95
B. provide leadership in promoting sustainable changes in service delivery and practice to improve the quality of social services. 6.97
Total Section Score 6.96
2.1.10(a)-(d) Engage, assess, intervene, and evaluate with individuals, families, groups, organizations, and communities.
A. substantively and affectively prepare for action with individuals, families, groups, organizations, and communities; 7.09
B. use empathy and other interpersonal skills; 7.38 C. develop a mutually agreedon focus of work and desired outcomes. 7.09 D. collect, organize, and interpret client data; 7.25 E. assess client strengths and limitations; 7.20 F. develop mutually agreedon intervention goals and objectives; 7.03 G. select appropriate intervention strategies. 7.08 H. initiate actions to achieve organizational goals; 7.00 I. implement prevention interventions that enhance client capacities; 7.09 J. help clients resolve problems; 7.06 K. negotiate, mediate, and advocate for clients; 7.30 L. facilitate transitions and endings. 6.89 M. critically analyze, monitor, and evaluate interventions. 6.79 Total Section Score 7.10
How to Access BEAPhttp://beap.utah.edu/Orhttp://www.bpdonline.org•“Resource Center”•BEAP
▫Resource Center▫Accreditation Resources
A quick look at the BEAP website▫Navigating the site▫Locating the results of your instruments
Ordering BEAP Instruments▫It is very important for each batch of
forms to have a face sheet!▫See handout for ordering BEAP for the
first time.▫See handout for re-ordering BEAP.
