1 the value of ncate a study of research institutions and ncate accreditation aacte, friday,...
Post on 04-Jan-2016
218 Views
Preview:
TRANSCRIPT
1
The Value of NCATE
A Study of Research Institutions and NCATE AccreditationAACTE, Friday, February 8, 2008
Emerson J. ElliottNCATE, February 2008
2
Initial report from a study
Research institutions that are accredited by NCATE
Interviews of deans, NCATE coordinators and faculty
Perceptions of NCATE accreditation
3
Topics in this sessionA. Setting for the study
B. Who participated
C. Interview questions
D. Response themes
E. Summing up
4
A. Setting for the study AACTE Fall 2004 study of deans CADREI fall 2006, and AACTE committee
Streamline Reduce burden and cost Address time commitment required of BOE and UAB
members Dovetail program review and unit review Make more collaborative, less punitive—more like some
other specialized accreditors
Brought to the NCATE Executive Board
A. Continued, Setting Executive Board also set goals for NCATE’s
management plan (Fall 2006-Spring 2007) Reduce the burden of accreditation Improve service to institutions Increase the value and perceived value of the
accreditation process to institutions
And NCATE was concerned that: Research universities are the source of new knowledge in
educator preparation. and for NCATE unit and specialized professional standards. . .
5
6
A. Continued, Setting But faculty from research universities are
infrequently involved with NCATE Research university faculty are often not among the
participants in standards writing And—prior to a change in 2007—research university
faculty had participated rarely as examiners or on NCATE’s policy boards
[3:15 session at AACTE today] Arranged meeting with research university deans,
January 2007
7
A. Continued, Setting: NCATE meeting with research universities Outcomes
Symbiotic relationship between research on teaching and learning produced by research universities and use of that research to improve educator preparation
Continue discussions with CADREI and AACTE Urge more BOE and board candidates from research institutions Streamlining to ensure an efficient and effective system
As one way to support these outcomes, and to ground the “value” goal in the management plan, NCATE decided to ask the research institutions themselves what they think about NCATE accreditation.
8
B. Who participated?
Five research institutions accredited by NCATE Five interviews in each institution
Dean Coordinator Three faculty designated by the dean
4 associate deans 4 department chairs 6 program directors and/or SPA coordinators 1 assessment coordinator and doctoral student
9
B. continued, Who participated: Interviews
All conducted by telephone All interviewees gave explicit permission to
record the interview All interviews followed the same interview
questions All interviewees were promised anonymity
All transcribed
10
B. Continued, Who participated: Selected institutions All in the East of the Mississippi Three are private not-for-profit, two are public Three participated in the NCATE program review process,
two did not All five are doctorate granting, four are Carnegie “high” R &
D ($10 to $150 million in 2005), one is Carnegie “very high” (over $250 million in 2005)
Two have around $20 million in social sciences R & D in 2005
Enrollments for 2004 range from around 12,000 to more than 42,000
B. Continued, Who participated: What is not reflected in the participating institutions?
Institutions that NCATE does not accredit Institutions in the West “All” NCATE accredited research institutions—
selected from two recent cohorts of UAB action Sample is small But:
Still found a range of differences across institutions and individuals
11
12
B. Continued,Who participated: Accreditation experience All had accreditation visits in 2004-2006
One was having an initial visit Four were continuing visits
Two had all standards met Two required focused visits related to
standards 1 and 2 and are now fully accredited
One has a coming focused visit related to standards 1 and 2
B. Continued, Who participated: Time warp
YEAR2001 2002 2003 2004 2005 2006 2007
Semester S F S F S F S F S F S F S F
UNIT New Stds1st phase,Transition
2nd phase 3rd phase Full Full Full Full
PROGRAM
Assessment, syllabi, and program evidence New 1st
cohort
2nd
cohort3rd
cohort4th
cohort5th
cohort6th
cohort7th
cohort
Prog Rpts submitted
PR submit
PR submit
PR submit
PR submit
VISIT V V V V
UAB action
UAB UAB UAB UAB
Interviews
I
13
14
C. Interview Questions Overall topics were:
Value that respondents associate with NCATE accreditation
How NCATE does its job Advice about how accreditation could be improved
Findings from 3 questions and then 6 themes—topics that recurred in the interview responses
15
C. Continued, questions: (1) Overall reactions 72% (18 respondents) “positive”
Positive, very positive, useful learning experience, important to do
Includes 3 deans
28% (7 respondents) “negative” Technical and prescriptive requirements Time and resource intensive Too much assessment, compliance, SPA instructions
changed Includes 2 deans—different experiences
16
C. Continued, questions: (1) Overall reactions Additional observations
Many at our institution think we don’t need national accreditation—we have really high standards
The process allowed for considerable flexibility in applying standards
The team were true professionals The self-study aspect is useful
17
C. Continued, questions: (2) Most helpful
MOST helpful Deans—reflection, self study, faculty
collaboration Coordinators—assessment Faculty—fitting assessments to standards,
collegial activities among the faculty
18
C. Continued, questions: (3) Least helpful
LEAST helpful Deans—time, cost, burden, team visit Coordinators—mixed: Praxis, changing rules,
duplicate NCATE and state requirements, lack of research institution team members
Faculty Assessments—confusion, rubrics for standards, changing
requirements BOE team—composition, logistics, use of web
information
D. Response “themes” Frequently recurring topics that were not
explicit interview questions Research institution self descriptions BOE teams Program review and SPAs Defining evidence for NCATE TEAC references States—contrasting perspectives
19
D. Continued, themes: (1) Research institutions 19 comments, 12 respondents, all 5 institutions; 24 points
Character of the institutions Accreditation is not the way they think Candidates arrive at the graduate level and have completed subject content
courses Specialization Emphasis on scholarship and research Always under study
Professional life of the faculty Own projects and funding NCATE takes time from grant writing Faculty change courses all the time, and their assessments Little motivation to volunteer for NCATE work
BOE teams often don’t understand their qualities20
D. Continued, themes: (2) BOE teams 48 references (largest number), 18 respondents in 5
institutions. Distinction between one institution that characterized its visit
as “disastrous” and all other
21
Difficult visit All other institutions
Positive comments 0% 43%
Negative comments 80% 21%
Other comments 0 11%
Recommendations 20% 25%
Total 100% 100%
D. Continued, themes: (2) BOE teams Positive statements about BOE teams--12
The team—true professionals, collegial, business-like, outstanding team, chair from a large research university, respected what we do, team understood who we are, all in this together, good team, worked hard
22
D. Continued, themes: (2) BOE teams
Negative statements about BOE teams The team in “all other” institutions—6
lack of preparation, did not use web-based exhibits, lack of research university peers, overemphasis on logistics
The team in the “difficult visit” institution—16 failed to ask for information, did not listen, did not use provided
documentation, tense exit interview, members not prepared, did not make good use of the Sunday evening poster session
23
24
D. Continued, themes: (2) BOE teams
Recommendations—11 of the 48 comments Chair/members from research institutions (8) Train members about research institutions Assure that teams arrive better prepared
To use electronic data For more collegial interaction To use time not just to “find data, but to clarify and
elaborate”
D. Continued, themes: (3) Program Review and SPAs 22 references from 10 respondents in all 5
institutions; total of 51 points made Inconsistencies and changes—20
Across SPAs and between SPAs and NCATE—some would accept GPA or Praxis data, others would not
Changes during the accreditation process Certain SPAs are difficult to work with Lack of fit between the standards and graduate level
initial preparation
25
D. Continued, themes: (3) Program review and SPAs Concurrences with the program review process—14
More consistent use of assessments Agreement on just 6 to 8 assessments Writing for national recognition helped to redirect
programs Faculty worked hard “because there’s this element of
pride” Our program’s gotten better Comments by reviewers were fair, right on the money
26
D. Continued, themes: (3) Program review and SPAs Complaints—9
Praxis data not aligned; can’t get sub scores Limited feedback Took too long Just wanted numbers
Others—2 Respondent’s own institution made changes Faculty would rather fight with their professional
organization than adopt standards
27
D. Continued, themes: (3) Program review and SPAs; Recommendations—6 More consistency
Equivalent demands for all SPAs More coordination across SPAs and between SPAs and NCATE Limit the number of assessments
Better fit with graduate level preparation Write standards for initial preparation that recognize prior candidate
preparation in subject content Provide more examples of evidence for “other” professional
preparation
Reconsider way decisions are made about SPA standards Need more “we’re in this together,” less territoriality, more at-large
partners28
D. Continued, themes: (4) Evidence 24 references by 10 respondents from 4 institutions 18 comments on evidence generally
NCATE is too prescriptive too much reliance on “numbers and tables” Not a research base to back up some data requirements Institutions use grades as evidence of accomplishment Assessment data cannot be aggregated meaningfully
across different levels of programs 6 comments on evidence for diversity
Numbers fail to inform the goal of “cultural competence”29
D. Continued, themes: (4) Evidence; Recommendations Use a broader definition, not just “numbers on tables” Let institutions decide
More anthropological—look at what we’re doing and how faculty and our publics interpret that
Let institutions self-define their mission, and ask for reporting of evidence within broad parameters—limiting the number of assessments
Address particular issues Take on measuring of student learning based on what good teaching
and learning are Focus on ethnic diversity and “cultural competence”
30
D. Continued, themes: (5) TEAC 10 references made by 6 respondents from 4 institutions; total
of 13 points 7 points were set in a context of perceived problems with
NCATE NCATE is overly prescriptive Some required pieces of data are not supported by research Differences between SPAs and NCATE frustrate faculty NCATE is “bean counting” NCATE is burdensome, tedious, time intensive, costly Important qualities of a program cannot be captured in numbers Standards don’t align with the way the state is going
31
D. Continued, themes: (5) TEAC Three respondents ventured impressions of TEAC or
recognized it as an alternative There were 3 recommendations
NCATE should look at what it does “from a business perspective”, but leave room for compromise
TEAC and NCATE representatives should “work out a whole new accreditation system. . . Good aspects of both, but neither is a perfect system”
BUT don’t go “so far in another direction that we don’t have a set of standards that people have to step up to meet”
32
D. Continued, themes: (6) States 13 references by 6 respondents from just 2 institutions Starkly different portraits of states, so state influence is a
strong factor in accreditation, at least for some institutions NCATE parallels the state, should keep coordinated, make state exam
pass rate count for NCATE, can defer to our state on state assessments, would not have completed NCATE accreditation if state had not insisted
Negativity about NCATE is really directed at the state; state report is different only in minor ways—a duplicate, unnecessary, requirement; disparity in the standards that “regular” and “alternative” programs are held to but state refuses to acknowledge
33
E. Summing up Respondents found much that was good, e.g.:
I like what I’m hearing about web-based submissions Electronic submission is very helpful to organize data,
other information, Positive that NCATE is interested in what approved
programs are saying and thinking Preparing for NCATE made the regional accreditation
easy
34
E. Summing up They made recommendations that NCATE needs to
sift and consider. Particularly, Evidence Teams and recruitment Program standards and standards decisions Regional accreditor experiences Continuing accreditation, candidate data only
35
top related