DRIVING IMPACTMetrics-Based Performance Management in Experiential Jewish Education

Josh Langer May 25, 2011

INTRODUCTIONResearch Question

Thesis

Literature Review

RESEARCH QUESTION

How can the lessons – positive and negative – of initiating metrics-based performance management in the formal educational world be leveraged to define objectives for and measure the success of experiential Jewish educators?

THESIS

The field of experiential Jewish education is increasingly gaining respect

The field generally lacks the systems to measure impact

Implementing a metrics-based performance management system will enhance BBYO’s ability to motivate and measure the success of its experiential educators

This new system could have broader implications for the maturation of the field as a whole

THEORY OF EXPERIENTIAL JEWISH EDUCATION “The teacher is not in the school to impose certain ideas or to

form certain habits in the child, but is there as a member of the community to select the influences which shall affect the child and to assist him in properly responding to these influences.” Dewey, John (1897). My Pedagogic Creed, Page 9.

Informal Jewish education: “an individual-centered and highly interactive educational approach focused on learning through experience, with knowledgeable and committed educators who use group process and a ‘curriculum’ of Jewish ideas and values to create a holistic educational culture.” Chazan, Barry (2003). “The Philosophy of Informal Jewish Education.”

“To be a truly professional informal Jewish educator one needs Judaic knowledge; a Jewish lifestyle; a knack for group dynamics; the ability to be inter-active and to listen; the ability to engage others; and the ability to impart ideas and values twenty-four hours day, seven days a week. One has to be accomplished in many areas – encompassing both content and method, Jewish and general – often demonstrating proficiency over and above that required of teachers.” Chazan, Barry (2003). “The Philosophy of Informal Jewish Education.”

THEORY OF EXPERIENTIAL JEWISH EDUCATION (CONTINUED)

Recreation – Provide participants with social comfort, fun and belonging in a Jewish context

Socialization – Provide the knowledge, skills and attitudes to be an active member of the Jewish community

Challenge – Encourage participants to undertake the challenge of stretching themselves and growing towards a more complex participation in one’s Jewish life

PROS AND CONS OF PERFORMANCE-BASED EDUCATOR MANAGEMENT

Set consistent objectives and expectations

Focus educators on student impact

Establish compelling case to donors/other stakeholders

Identify internal strengths and weaknesses

Make data-based decisions

Inconsistent implementation

Importance of setting correct measures (making art a science)

Managing educators’ concerns

Potential to over-complicate, distract from ultimate goal

Potential Benefits Potential Drawbacks

YIKES.

JOHN KOTTER’S CHANGE MODEL

1) Create a Sense of Urgency

2)2) Form a Powerful CoalitionForm a Powerful Coalition3) Create a Vision for Change

4)4) Communicate the VisionCommunicate the Vision5) Remove Obstacles

6)6) Create Short-term WinsCreate Short-term Wins7) Build on the Change

8)8) Anchor the Changes in Corporate Anchor the Changes in Corporate CultureCulture

BBYO’S PLANProposed Measures

BB

YO

’S M

EA

SU

RES

OF P

RO

GR

AM

MATIC

S

UC

CESS

* TDMI refers to the different categories of impact into which teen participants will be grouped based on the number and type of experiences they attend. T = Transformational, D = Deep, M = Moderate, I = Initial.

** Teen Connection is BBYO’s experience for 6th-8th grade students, with the objective of maintaining a connection with tweens beyond their becoming a Bar/Bat Mitzvah and transitioning them to continued Jewish participation in high school.

Metric Description

Total Teen Involvement Number of unique teens participating

T/D/M/I (a)*Categorization of teens based on cumulative program attendance (focuses on impact on teen participants)

Total Immersives Participation Number of teens participating in national immersive experiences

Regional Net Promoter Score (a)A customer-service metric that identifies the percent of the customer audience that are passionate supporters/attractors

Teen Connection Transition Rate (a)**

The percent of teens who transition from participation in BBYO’s middle school experience to participation in its high school offerings

First Time AZA/BBG Members (a)The percent of AZA/BBG members who are first-time members, indicates continued resonance of membership to teen audience

Convention Quality Rating (a)

Overall quality score as rated by teens who participate in weekend conventions/shabbatonim. Regions average 3-5 conventions per year

Godol/N'siah Quarterly RatingsRatings from the International Teen Presidents after they visit a region and observe its operation

Program Distribution by Educational Focus Areas (b)

The number of experiences offered in each EFA as a percent of the total number of experiences offered – shows diversity of programming in line with overarching educational objectives

Regional Strategic Plan Quarterly Review Rating

Ratings assigned by AEDs after review of overall progress in key metrics and progress towards regional strategic plan priorities

BBYO REGIONAL HEALTH SCORECARD

FINDINGS AND APPLICATIONConclusions and Recommendations

THE PDI PERSPECTIVE

Survey conducted of BBYO’s 34 PDI participants.

Represent multiple functions within BBYO Generally considered BBYO’s “top

performers”

How will the role of the Program Director change?

CONCLUSIONS FROM RESEARCH Decent alignment between what PDI participants believe

BBYO values most and what they believe is most important in terms of KNOWLEDGE 

PDI participants do not believe BBYO values knowledge of sacred Jewish texts, nor do PDI participants believe knowledge of sacred Jewish texts is important to success Interesting contrast to Barry Chazan

Both PDIers and BBYO value the “Ability to inspire and empower others” Beyond that item, there seems to be a gap in the understanding

of which abilities are keys to success as a program director

3 of 4 four metrics that PDI participants identify as the best indicators of success will be measures in the strategic plan The Regional Health Scorecard is on the right track

CONCLUSIONS FROM RESEARCH (CONTINUED)

The Overall Chapter Quality Score was considered to be the best indicator of success. Program Directors have least direct control over this

metric. Interesting.

The organization lacks a clear definition of the Program Director role.

BBYO does have a unique operational challenge in that it exists as a hybrid of a centralized and locally autonomous organization.

Several comments note a desire for higher program standards and for more advanced metrics. Suggests buy-in is possible.

RECOMMENDATIONS TO BBYO Develop concrete Program Director/Associate job description

that staff can consistently articulate Incorporate language from Chazan, Reimer/Bryfman into job

description

Incorporate Regional Health Scorecard details into annual performance evaluation

Add section to annual evaluation focused on knowledge development

Weight performance in skill-based areas based on relative importance to execution of job description

Initiate training to set expectations for performance as part of new hire on-boarding

Prioritize buy-in of field; follow recommendations of change model to manage transition to new system

IMPLICATIONS FOR THE FIELD OF EXPERIENTIAL JEWISH EDUCATION As the field continues to grow and mature,

professionalization of systems is needed:

Clarity of educational vision and educator performance management go hand-in-hand

Philanthropic community increasing insistence on hard data to substantiate funding

Best practices can extend across all modes of experiential education

Potential to attract more talented, serious educational professionals

THANKS!What an audience!?!

Looking forward to your questions and comments.