jan 29 kick off webinar final

Welcome to the NEBHE Developmental Math Demonstration Project Kick-Off Webinar

Participant Instructions

• Participants should be able to access the webinar’s audio through computer speakers or via telephone. If you have any difficulties accessing the webinar, please email Daren Follweiler at [email protected].

• To ensure good audio quality, please mute your audio when you are not speaking.

• All participants are automatically muted upon joining. To ask a question, please use one of the following: •  Type into the Question Box •  Raise Your Hand and we will unmute you •  Unmute by clicking the microphone icon

(at the side of your control panel)

NEBHE Developmental Math Demonstration Project January 29, 2013

NEBHE Developmental Math Demonstration Project

Presenters: Stafford Peat, Senior Consultant Monnica Chan, Director of Policy and Research

NEBHE Developmental Math Demonstration Project January 29, 2013

Participating Institutions/Systems

•  Connecticut Board of Regents for Higher Education

•  Community College System of New Hampshire

•  Community College of Vermont

•  Housatonic Community College

•  Middlesex Community College (CT)

•  Middlesex Community College (MA)

•  Mt Wachusett Community College

•  One additional Community College may join the network

NEBHE Developmental Math Demonstration Project 3

Memorandum of Understanding •  The MOU outlines the terms of partnership between

participating institutions/systems and NEBHE as well as the research questions we plan to answer.

•  Thanks to those institutions that have already sent in their MOUs.

•  Signed MOUs will be returned to the college/system with the first payment.

•  Signed forms are due by February 28, 2013.

•  Are there any questions that need to be answered before your institution/system signs the document?


NEBHE Developmental Mathematics Demonstration Project Memorandum of Understanding

The New England Board of Higher Education (NEBHE) released a Letter of Intent in November 2012 for community colleges, consortia of public two-year colleges or state community college systems to partner in the NEBHE Developmental Mathematics Demonstration Project (DMDP). Institutions and state community college systems were selected through a competitive process. The Memorandum of Understanding (MOU) below outlines the respective roles and responsibilities of colleges/systems and NEBHE commencing in January 2013 through September 2015. Community college/state community college system pilot sites will:

1. Use Khan Academy materials, as described in the college’s/system’s Letter of

Institutional Intent. 2. Help NEBHE answer research questions, listed below. 3. Adhere to the research dimensions of the project. 4. Report student data to NEBHE based upon the schedule described on pages two and three

in the NEBHE Developmental Math Demonstration Project Letter of Intent. The data for participating enrolled students include: demographic information, developmental math course information including instructional approach, student grades, course enrollment patterns, grades in subsequent math courses and Khan Academy usage. For high school students participating in project pre-matriculation programs, data will be reported in the aggregate. See Attachment A for variables that will be reported.

5. Enter into a data sharing agreement with NEBHE. The data sharing agreement specifies the specific variables to be collected, the basis for how student data is transmitted, stored, analyzed, made secure, reported and destroyed once the project has ended.

6. Comply with the Family Education Rights and Privacy Act (FERPA) and other state privacy requirements.

7. Advise NEBHE in the development of model Khan Developmental Math (DM) templates, resources, and data collection tools.

8. Provide NEBHE with Khan Academy electronic platform feedback, suggestions for enhancement, or new features for use by community colleges/adult programs through formal surveys and other informal feedback.

9. Participate in a project orientation and training session, Khan Academy network users group, annual network meetings, and other online convenings.

10. Assign a site coordinator who will be responsible for communication with institutional/system leaders, participating faculty and with NEBHE.

11. Participate in a broader regional dialogue on DM and developmental education reform. 12. Share best practices and lessons learned with Khan Academy and with NEBHE. 13. Inform NEBHE of planned major changes in project design activities as described in the

college’s/system’s Letter of Institutional Intent. Research Questions

• To what extent, and in what ways, is student performance in DM impacted by implementation of new delivery formats and Khan technology-based resources?

2

• What are the most and least effective utilizations of Khan resources? • What is the impact of the pilot implementation on DM at the institution? • What are the project’s research findings, in terms of: Khan usage, in-module/course

student performance? • What were the differences between the treatment and control groups? Between the varied

models/implementations of Khan resources? • What impact can be attributed to implementation of Khan-based resources, delivery

modes and other factors? To help answer these questions NEBHE will collect baseline, formative and summative data as well as use information collected through staff and student surveys, and onsite visits. NEBHE will:

1. Provide demonstration sites with technical assistance and ongoing coaching. 2. Organize a community of Khan Academy network project users to share information on

best practices, problem solving and for ongoing communication. 3. Host a two-day orientation and training session in Spring 2013 for demonstration site

staff. NEBHE will cover participants’ accommodations and meal expenses. 4. Provide assistance to demonstration sites in collecting and reporting data to NEBHE. 5. Comply with the Family Education Rights and Privacy Act (FERPA) and other state

privacy guidelines. 6. Convene all participating colleges/systems at least once per year for information sharing

and training. 7. Provide opportunities for web-based convenings. 8. Identify model DM course syllabi objectives and map the course objectives to the

Common Core State Standards for Mathematics and Khan Academy Videos and Practice Sets.

9. Develop other mapping tools, guides, data dictionaries and resources as needed to implement the project.

10. Identify or create other DM open educational resources. 11. Provide demonstration sites with project-related summaries of their institution/state

system as well as network results based upon data submissions. 12. Provide incentive funds at the conclusion to participating institutions/state systems based

upon the number of students divided by the number of students served across all pilot sites.

13. Work with participating colleges/systems to identify and develop potential next phase activities, as desired and as informed by project findings.

3

By signing below, the parties agree to the roles and responsibilities of this MOU. FOR THE INSTITUTION OR SYSTEM

Entity: ___________________________________

Signature: ___________________________________ Name: ____________________________________ Title: ____________________________________ Date: ____________________________________ FOR THE NEW ENGLAND BOARD OF HIGHER EDUCATION

Signature: ____________________________________ Name: ____________________________________ Title: _______________________________________ Date: ____________________________________

Data Sharing Variables •  Attachment to MOU.

•  Outlines data we would like to collect for all institutions including student age, race, developmental course taken and grade. There are also optional sets of data variables to be collected based upon submitted LOIs:

–  Variables for HS students in pre-matriculation programs

–  Variables for measuring whether Khan usage reduces institutional costs

–  Variables comparing student performance on placement exams and/or in treatment vs. control groups

•  Are there any questions about the data to be collected, including optional data sets?


Attachment A

Data Sharing Agreement Variables1

I Data variables for all participating colleges/systems

Variables for pre- or post- matriculation programs using Khan Academy include:

• Student identification numbers using a randomized protocol developed by demonstration sites/state systems

• Race/Ethnicity using IPEDS definitions • Gender • Pell Grant status (as a proxy for family income), if available • Age reported in age bands (<18,18-24, 25-34, 35+) • Name of pre-matriculation program, if applicable • Developmental course name and date taken • Developmental course number • Developmental course level • Developmental course instructional approach • Number of Khan Academy videos watched • Number of Khan Academy exercises (skills) started • Number of Khan Academy exercises (skills) proficient • Number of Khan Academy practice problems attempted • Number of Khan Academy practice problems correct • Developmental course grade • Student enrollment in a subsequent developmental or for “transfer credit” math course • Grade in subsequent math course and date taken Base-line data variables to be collected for comparative purposes by developmental course name, number and level for the 2011-12 academic year: • Student Race/Ethnicity breakdowns using IPEDS definitions • Gender ratio • % of Pell Grant recipients • Age reported in age bands (<18,18-24, 25-34, 35+) • Student enrollment rates in a subsequent developmental or for transfer credit math course • Student completion rate (C or higher) in DM course • Student completion rate (C or higher) in subsequent math course and date taken II Optional for colleges/systems that plan to serve high school students Variables to be collected for in the aggregate for students enrolled in high school participating in pre-matriculation programs include: • Name of pre-matriculation program

1 NB: In addition to these quantitative data variables, NEBHE will work with all sites to administer feedback surveys for faculty and students, per the Project Letter of Intent.

• Type of pre-matriculation program • Number of participating students reported by high school • Average number of Khan Academy videos watched • Average number of Khan Academy exercises (skills) started • Average number of Khan Academy exercises (skills) proficient • Average number of Khan Academy practice problems attempted • Average number of Khan Academy practice problems correct • Students’ enrollment in a subsequent developmental and for “transfer credit” math course

at the postsecondary level, if applicable (by number and %) • Average grade, range of grades and course completion rates in subsequent developmental

math and for “transfer credit” math course at the postsecondary level for pre-matriculation students, if applicable

• Pre-and-post placement test pass/fail data (by number and %), if applicable

III Optional for institutions that plan to collect additional data to measure Khan Academy usage impact on:

• Institutional cost

o Salaried costs before and after demonstration project o Hourly personnel costs before and after demonstration project o Fixed costs (instructional materials, etc) before and after demonstration project

• Student performance relative to college placement exams

o College placement exam(s), and cut-off score(s) o Student placement exam score prior to developmental math course o Student placement exam score at the end of developmental math course, if

applicable

• Student performance in treatment compared to control groups o Student identification numbers using a randomized protocol developed by

demonstration sites/state systems o Race/Ethnicity using IPEDS definitions o Gender o Pell Grant status (as a proxy for family income) o Age reported in age bands (<18,18-24, 25-34, 35+) o Name of pre-matriculation program, if applicable o Developmental course name and date taken o Developmental course number o Developmental course level o Developmental course instructional approach o Developmental course grade o Student enrollment in a subsequent developmental or for transfer credit math

course o Grade in subsequent math course and date taken

Data Sharing Memorandum of Agreement •  Outlines the process for sharing data on participating students

with NEBHE.

•  This MOA may be used as a model for institutions that don’t have a MOA template and will be emailed to participating colleges/systems coordinators next week.

•  May be modified to meet needs state/institutional privacy requirements.

•  MOA due May 31, 2013.

•  NEBHE doesn’t plan to collect student names/IDs, however we have been advised that a data share agreement is needed.


March Training Workshop for Participating Sites

•  Sessions will take place at the Sturbridge Host Hotel and Conference Center on March 28-29, 2013.

•  Tentative schedule has the workshop beginning in mid-morning on March 28 and ending late morning on March 29.

•  Accommodations and meal charges covered by NEBHE.

•  A formal workshop announcement including the tentative agenda will be emailed out next week.

•  Institutions may send up to five individuals and state systems may send up to 10 staff.

•  Site representatives should include the site coordinator, IR staff person, and participating faculty members.


March Training Workshop Tentative Agenda •  Khan Academy orientation by Khan Academy staff

•  Flipped classroom workshop session

•  Data reporting orientation

•  Feedback from attending math faculty who have used Khan in their math courses

•  Presentation of training resources

•  Discussion of implementation plans across sites and sharing of developmental math redesign initiatives


Upcoming Training Resources •  Draft model developmental math course objectives that

are aligned to the Common Core and Khan videos and practice sets

•  Instructor guides

•  Student guides

•  Project website with resources and list of implementation models (www.nebhe.org/devmath)


Next Steps •  Model course objectives for DM courses

•  Pathfinders to share experiences in using Khan Academy materials

•  Look for emails on MOA, March workshop announcement and registration information

•  On-going feedback


Questions & Comments?

Stafford Peat at [email protected]

jan 29 kick off webinar final

Documents