www.FLDOE.org1

Spring MeetingJanuary 19, 2016

Tallahassee, Florida 

Florida Association of Professors of Educational Leadership (FAPEL)

Florida Educational Leadership Examination (FELE)

www.FLDOE.org2

Postsecondary Assessment Team for Today 

Phil Canto, Bureau ChiefDr. Suleyman Olgar, Scoring and Reporting Supervisor Lauren White, Scoring and Reporting Team Dan Moore, Test Development Team 

www.FLDOE.org3

Today’s Postsecondary Assessment Presentation

Present status of current FELE program Discuss status of research for “Next Gen” FELEReview 2015 data (critical implementation year)Review closely the FELE holistic scoring processProvide sample materials and informQ & A

www.FLDOE.org4

Educator Quality‐BHAG

“Every Florida teacher and leader is prepared, developed, supported, and supervised by educators to make teaching better.”

www.FLDOE.org5

• No planned substantive changes to current FELE content, structure, passing scores, or examination fees in 2015 or 2016;

• Ongoing enhancement and expansion of M/C item pools for all three subtests‐We constantly monitor to ensure quality control;

• With recent SBE changes to school grades, there are no impacts to FELE items and prompts.

FELE Program 411

www.FLDOE.org6

• FLDOE is evaluating several options for performance based or enhanced assessment(s) 

• May impact FELE and/or other exams in the future (current or new)

• End of 2016 to 2017 before pilot project could begin with statewide inputs

• Normal test development process is at least 2 years• Biggest question is, “What is the current FELE not giving us that a performance assessment might?”

FLDOE Research for Next Generation FELE

www.FLDOE.org7

• January 1, 2014 (New Test)• FELE 3.0 Examination

• Subtest 1: Leadership for Student Learning

• Subtest 2: Organizational Development

• Subtest 3: Systems Leadership

• Effective January 1, 2015 (New Format & Passing Scores)• Subtest 3: Systems Leadership

• Multiple‐Choice Section• Written Performance Section(Examinees must pass both sections independently to pass FELE Subtest 3)

FELE 3.0 Program Changes 

www.FLDOE.org8

* Operational pass rates are calculated for the first-time examinees taking FELE subtests from January 2, 2015 to December 31, 2015 with the approved passing scores at the November 2014 SBE meeting which became effective by January 1, 2015.Note: FELE Subtest 3 written performance assessment is holistically scored by two human raters using a scoring rubric of 1–6 range. FELE Subtest 3 written performance assessment score is the sum of the two ratings.

FELE Subtest Recommended Number Correct

Recommended Percentage Correct

Projected Passing Rates

Operational Pass Rates*N %

Subtest 1: Leadership for Student Learning 48/70 69% 58% 1,617 63%

Subtest 2: Organizational Development 48/69 70% 55% 1,568 58%

Subtest 3: System Leadership: Multiple-Choice Section

36/55 66%

34%

1,620 50%

Subtest 3: System Leadership: Written-Performance Section

7/12 59% 1,618 46%

FELE Recommended Passing Scores, Projected Pass Rates vs. 2015 Operational Pass Rates

www.FLDOE.org9

FELE 3.0 First-Time Examinee Pass Rates (2014 vs. 2015)

FELE 3.02014 2015

N Pass Rate Mean Score N Pass Rate Mean Score

FELE Subtest 1: Leadership for Student Learning 1,495 84% 219 1,617 63% 202

FELE Subtest 2: Organizational Development 1,485 90% 225 1,568 58% 200

FELE Subtest 3: System Leadership (Multiple-Choice Section) 1,485 82% 221 1,620 50% 197

FELE Subtest 3: System Leadership (Written Performance Section) N/A N/A N/A 1,618 46% 7

FELE Assessment Level 1,475 73% N/A 1,523 22% N/A

www.FLDOE.org10

FELE 3.0 Best Attempt Examinee Pass Rates (2014 vs. 2015)

FELE 3.02014 2015

N Pass Rate Mean Score N Pass Rate Mean Score

FELE Subtest 1: Leadership for Student Learning 1,495 87% 219 1,617 70% 202

FELE Subtest 2: Organizational Development 1,485 93% 226 1,568 69% 200

FELE Subtest 3: System Leadership (Multiple-Choice Section) 1,485 86% 222 1,620 63% 197

FELE Subtest 3: System Leadership (Written Performance Section) N/A N/A N/A 1,618 59% 7

FELE Assessment Level 1,475 81% N/A 1,523 42% N/A

www.FLDOE.org11

FELE Performance by Competency (2014 ‐ 2015)

Test Name Competency Name 2014 2015

Florida Educational Leadership Examination (FELE) N % Correct N % Correct

FELE Subtest 1: Leadership for Student Learning

01 (17-18) Facilitate student learning goals

1,495

69.7%

1,572

69.8%

02 (18) Prioritize student learning via leadership 70.6% 74.6%03 (15-17) Develop/implement instructional framework 71.5% 71.6%

04 (17) School environment for student learning 68.4% 70.5%

FELE Subtest 2: Organizational Development

01 (11-12) Recruitment and induction practices

1,485

72.2%

1,522

74.1%

02 (22-23) Faculty/staff development and retention 69.5% 71.6%

03 (16-17) Develop leadership w/in the organization 70.9% 71.9%

04 (17-18) Personal/professional behavior practices 70.3% 71.0%

FELE Subtest 3: System Leadership (Multiple-Choice Section)

01 (13-14) Decision-making processes

1,485

62.7%

1,574

67.2%

02 (14) School operation management practices 69.8% 68.5%

03 (14) Financial and resource management 64.4% 66.9%

04 (13) School legal practices and applications 64.8% 66.2%

FELE Subtest 3: System Leadership (Written Performance Section) 05 (1) Effective communication practices N/A N/A 1,471 62.6%

www.FLDOE.org12

Score Verification• Score verification eligibility: 

• A non‐passing score of 190 or higher• Written Performance component

FELENumber of Sessions Number of Items

ReviewedNumber of Status

Changes

2014 2015 2014 2015 2014 2015

Subtest 1: Leadership for Student Learning 11 25 119 197 0 0

Subtest 2: Organizational Development 3 22 59 163 1 0

Subtest 3: Systems LeadershipMultiple-Choice Section 23 23 254 (14) 212 1 1

Subtest 3: Systems LeadershipWritten Performance Section 0 42 0 45 0 0

www.FLDOE.org13

FELE Written Performance Assessment  (WPA) &  Holistic Scoring

• Interpretation of data and writing ability as it relates to the data analysis and target audience;

• Organization, sentence structure, and mechanics, as well as the accuracy of data interpretation

• An examinee response is judged for its total effect;• Rater makes an overall evaluation taking multiple performance factors into consideration

• Responses are rated independently by two raters and sent through a discrepancy resolution process if rater scores vary by more than 1 point.

www.FLDOE.org14

FELE WPA Responses & Holistic Scoring Process• Scoring Rubric identifies performance characteristics to be judged, includes the criteria for evaluating those features, and describes how performance varies across the scoring scale

• Anchor Set is used to calibrate raters to the rubric• Supplemental Rating Criteria lists the critical elements of data use by score points

• Calibration/Practice Set includes pre‐scored responses and used for rater calibration prior to operational scoring

www.FLDOE.org15

SCORE CRITERIA

6

The writing sample has a clearly established topic that the writer fully develops with specific details and examples, includingaccurate and thorough data interpretation, explanation, and application. The writer clearly describes the purpose of thecommunication and reflects an understanding of the target audience with plans for change appropriately aligned with data findings,all relevant implications, and analysis of trends. Point of view is consistently maintained. Organization is notably logical andcoherent. The writer demonstrates superior facility in the use of proper grammar and syntax.

5

The writing sample has a clearly established topic that is adequately developed and recognizable through specific details and/orexamples, including mostly accurate and thorough data interpretation, explanation, and application with minor errors in details not affecting conclusions. The writer adequately describes the purpose of the communication and reflects an understanding of the target audience with plans for change appropriately aligned with data findings, relevant implications, and analysis of trends. Point of view is mostly maintained. Organization follows a logical and coherent pattern. The writer demonstrates infrequent errors in the use of proper grammar and syntax.

4

The writing sample has an adequately stated topic that is developed with some specific details and/or examples, including somecomponents of data interpretation, explanation, and application. The writer describes the purpose of the communication and reflectsa basic understanding of the target audience with some unexplained terms or references. Plans for change may omit a fewnecessary details or relevant implications in aligning data findings and analysis of trends. Point of view is somewhat maintained. Organization is mostly logical and coherent. The writer demonstrates satisfactory use of proper grammar and syntax.

3

The writing sample states a topic that is developed with generalizations, with some accurate components of datainterpretation, explanation, and application. The writer somewhat describes the purpose of the communication and reflects amarginal understanding of the target audience with several unexplained terms or references. Plans for change omit somenecessary details or relevant implications in aligning data findings and analysis of trends. Explanations of some data elements may be faulty, and placement of data-related details may not be effective. Point of view is ambiguous. Organization is occasionallyillogical or incoherent. The writer demonstrates some errors in the use of proper grammar and syntax that do not detract from theoverall effect.

2

The writing sample presents an incomplete or ambiguous topic, with most components of data interpretation, explanation, andapplication omitted or inaccurate. The writer poorly describes the purpose of the communication and reflects little understanding ofthe target audience with numerous unexplained terms or references. Support is developed with generalizations with littledescription of plans for change or relevant implications in aligning data findings and analysis of trends. Explanations ofdata elements are faulty, and placement of data-related details is ineffective. Point of view is confusing and distracting. Organization is frequently illogical and incoherent. The writer demonstrates serious and frequent errors in proper grammar and syntax.

1

The writing sample has no evident topic. Numerous components of data interpretation, explanation, and application are missing orinaccurate. The writer fails to describe the purpose of the communication and does not address the target audience. Plans forchange and/or relevant implications in aligning data findings and analysis of trends are minimally addressed. Development is inadequate and/or irrelevant. Point of view has not been established. Organization is illogical and/or incoherent. The writerdemonstrates severe and pervasive flaws in the use of proper grammar and syntax.

FELE WPA Rubric & Score Scale

www.FLDOE.org16

Sample FELE WPA Prompt #1 (High School ABC)

www.FLDOE.org17

Sample SRC for FELE WPA Prompt #1 (High School ABC)

www.FLDOE.org18

Sample FELE WPA Prompt #2 (Middle School CDE)

www.FLDOE.org19

Sample SRC for FELE WPA Prompt #2 (Middle School CDE)

www.FLDOE.org20

Questions

Call850‐245‐0513

Visithttp://www.fldoe.org/accountability/assessments/posts

econdary‐assessment

www.FLDOE.org21

www.FLDOE.org