architecture and impact of an open, online, remixable, and multimedia-rich algebra 1 course

Ahrash N Bissell, Ph.D.Special Projects ManagerMonterey Institute for Technology and Education

2011 Sloan-C International Conference on Online Learning

Architecture and Impactof an

Open, Online, Remixable, and Multimedia-Rich

Algebra 1 Course

Our motivations...

• Enable free, personalized learning for every person.

• Produce resources that are cutting edge and of demonstrable quality.

• Build a community of educators and institutions who inform and are empowered by our work.

• Sustain the enterprise in the face of changing technical and financial circumstances.

Our motivations...

Source: Final report of the National Mathematics Advisory Panel

Mathematics: a national challenge

Free, personalized learning for every person

arematerials, tools, and media

used for teaching and learningthat are

free from copyright restrictionsor publicly licensed

for anyoneto use, adapt, and redistribute.

Free

, per

sona

lized

lear

ning

for e

very

per

son

Why OER?

Open educational resources...

● Bridge formal and informal education● Facilitate lifelong and distributed learning● Support individualization and localization

http://www.flickr.com/photos/mimax/303567279/ Mimax CC BY-NC-SA

Free

, per

sona

lized

lear

ning

for e

very

per

son

http://www.flickr.com/photos/mimax/303567279/

But OER can be hard to navigate...Fr

ee, p

erso

naliz

ed le

arni

ng fo

r eve

ry p

erso

n

… and even harder to use effectively.

CC

BY

http

://w

ww

.flic

kr.c

om/p

hoto

s/ni

ckra

te/4

5041

6319

6/

Free

, per

sona

lized

lear

ning

for e

very

per

son

Our conclusions...

• Resources need to work where teachers and learners actually use them:

• Online → Hippocampus

• At school → NROC Network

• The resources should be modular → think of Legos.

• The resources should support any form of instruction → online, offline, and hybrid.

Free

, per

sona

lized

lear

ning

for e

very

per

son

Our conclusions...

Most importantly

--The resources should

encourage and enableexperimentation and innovation

without the concomitant breakdown ofquality assurances

andcurricular relevance.

Requires attention to both design and distribution.Free

, per

sona

lized

lear

ning

for e

very

per

son

Funding: The Bill & Melinda Gates FoundationThe William & Flora Hewlett FoundationProject Audience:Algebra 1: Ages 13-16, first-time algebra

studentsDev Math: Ages 16-80, students have failed

math at least once

Project Goal: Increase the number of students that pass Algebra 1 (and developmental math) as a bridge to a college education

We zeroed in on math...

Cut

ting-

edge

, hig

h-qu

ality

reso

urce

s

● Provide learning activities to support an efficient path to credit-bearing courses

● Offer multiple modes of instruction● Support multiple curricular standards ● Allow for flexible course configurations with a

collection of learning objects ● Leverage the power of digital media

… and focused on these objectives.

Cut

ting-

edge

, hig

h-qu

ality

reso

urce

s

Research Focus Groups

Technology &

Engineering

Professional Development

Curriculum

Product Design

Development ApproachDevelopment Approach

Cut

ting-

edge

, hig

h-qu

ality

reso

urce

s

Research Focus Groups

Technology &

Engineering

Professional Development

Curriculum

Development ApproachDevelopment Approach

Product Design

Cut

ting-

edge

, hig

h-qu

ality

reso

urce

s

Product Development

Focus Groups

Administrators and Faculty• determine shortcomings of existing teaching materials• better understand administrator and faculty use cases• feedback for products in development• develop pilot sites and adoption interest

Students• what digital learning materials are they using?• better understand student use cases• feedback for products in development

68 roundtables. >720 students, instructors, and administrators. Representatives from 153 academic institutions in 28 states.

Focus groupsC

uttin

g-ed

ge, h

igh-

qual

ity re

sour

ces

Professional development is essential for supporting teacher success

Few instructors and institutions are satisfied with existing methods or digital options

Existing digital products: too expensive inflexible curricula proprietary management system

High value placed on flexibility, affordability, multiple pathways for learners

Administrators and Faculty

Focus Group Key FindingsC

uttin

g-ed

ge, h

igh-

qual

ity re

sour

ces

Keep it simple: struggling students and English language learners value simplicity

Real-world examples and application are key to engagement

Puzzles, animations, simple illustrations, and problem sets are important

Humor, esoteric example, and idioms trip-up or confuse struggling students, especially English language learners

Students

Focus Group Key FindingsC

uttin

g-ed

ge, h

igh-

qual

ity re

sour

ces

Review the Details in our ReportFocus Group Report

Download it under Project News at:

Cut

ting-

edge

, hig

h-qu

ality

reso

urce

s

http://www.nrocmath.org/

Single-session formative assessment, multi-session CMS-based

Correlated to 48 state frameworks and the Common Core

Designed for online and hybrid classroom use

Designed for first-time algebra learners (US 8th grade, 15 years old)

LO architecture: 12 Units, 26 Lessons, 68 Topics, covering 170 learning objectives

Rich, interactive multimedia with multiple instructional elements designed

for 15 year-old audience

Algebra 1

The details...

Semester 1, Semester 2

Cut

ting-

edge

, hig

h-qu

ality

reso

urce

s

Instructional elementsAt the topic level:

● warm-up (text)● presentation (video, audio,

animation, and graphics)● worked examples (audio and

graphics)● problems (interactive text)● review (text)● text tab (online textbook)

At the unit level:● virtual tutor (interactive video,

graphics, text)● project (text and graphics)● puzzles (game-based, interactive

animation, graphics)

Instructional Elements

At the topic level:● warm-up (text)● presentation (video, audio, animation, and

graphics)● worked examples (audio and graphics)● problems (interactive text)● review (text)● text tab (online textbook)

At the unit level:● virtual tutor (interactive video, graphics,

text)● project (text and graphics)● puzzles (game-based, interactive


The details...


At the topic level:● warm-up (text)● presentation (video, audio, animation,

and graphics)● worked examples (audio and graphics)● problems (interactive text)● review (text)● text tab (online textbook)




The details...


At the topic level:● warm-up (text)● presentation (video, audio, animation, and

graphics)● worked examples (audio and graphics)● problems (interactive text)● review (text)● text tab (online textbook)




The details...

http://nroc.remote-learner.net/

http://nroc.remote-learner.net/

A focus on distribution...

Institutions → NROCNetwork.org

Individuals → HippoCampus.org

http://www.montereyinstitute.org/media/demo.html

… which builds community...

The NROC Network... is comprised of educators, designers, technologists, and administrators working together to promote the continuous improvement of online courses through collaborative development of high-quality content and instruction.

A c

omm

unity

of e

duca

tors

and

inst

itutio

ns

… and is flexible...A

com

mun

ity o

f edu

cato

rs a

nd in

stitu

tions

Customizable by a teacher, a district, or a state.

Usable within popular LMS’s and repositories.

http://www.iowacconline.com/

https://elearn.chattanoogastate.edu/d2l/orgTools/ouHome/ouHome.asp?ou=246829

http://cccs.blackboard.com/webct/urw/lc59085021.tp0/cobaltMainFrame.dowebct

http://www.hippocampus.org/myHippo/?user=myCCCO

… while maintaining curricular relevance...A

com

mun

ity o

f edu

cato

rs a

nd in

stitu

tions Correlated with

state standards and popular textbooks.

Complete teaching materials.

… and sustaining our work.A

com

mun

ity o

f edu

cato

rs a

nd in

stitu

tions

Academy School District 20 Online (CO)Alabama ACCESS (DOE) Alameda County Office of EducationAlbuquerque Public Schools (NM)All Saints Day School (CA)Anaheim Union High School District (CA)Arizona Department of EducationBambugare Network (Sub-Saharan Africa)Bay Port High SchoolBishop O’Dowd High SchoolBethel Covenant College (Uganda)Berkeley Tutoring CoalitionButler Technology and Career DevelopmentCalifornia State University at Fullerton Catholic School K12 VirtualChattanooga State Tech.& Comm. College (TN) Chesterfield School District (VA)CORE (China)Corona-Norco Unified School DistrictColorado Community Colleges Online Colorado Online LearningCUDI (Mexico)Culver City Unified School DistricteMINTS/MissouriExeter Public Schools (CA)Fairmont Private SchoolsFlorida Distance Learning Consortium Forest Charter SchoolGeorgia Virtual School (DOE)GLBTQ Online High School (MN)Global Literacy Foundation (AZ)Grand Rapids Public SchoolsGreenville County Virtual School (SC) Grossmont Union High School (CA)Gwinnett County Online Campus (GA)Horry County Virtual SchoolKentucky Statewide Consortium (statewide) Hudson Schools of Technology (NJ) Idaho Digital Learning AcademyIDEAL-New Mexico

Data as of July 2011

Illinois Virtual School (DOE)Indian Prairie School District #204 (IL)Iowa Community Colleges Online Consortium Iowa AEA ConsortiumIvyTech Charter SchoolsJFY NetworksJoliet Junior CollegeJupiter Education ServicesLee’s Summit R7 School DistrictLos Angeles Unified School DistrictLouisiana Virtual School Lubbock Independent School District Online (TX) Maryland State Department of Education Mason City School District (OH)Michigan Virtual High SchoolMinnesota Learning Commons (statewide P-20) Minneapolis Public Schools OnlineMississippi Virtual School (DOE)Missouri Dept. of Elem. and Secondary EducationMontana Digital AcademyMontana State University at BillingsMontgomery County Public Schools (VA)MSD of Wayne TownshipMyron B. Thompson Academy (HI)National Association of Beginning TeachersNiles Township High School District (IL)Niswonger FoundationNebraska: Partnerships for Innovation (statewide)New York City Department of EducationNobel Learning CommunitiesNorth Carolina Community College SystemNorth Dakota Center for Distance EducationNorth Salinas High School (CA)Northwest Area Education Agency (IA) Odyssey Charter Schools (NV)OpenEd SolutionsOregon Department of EducationOpen High School of Utah (NV)OTANPalmetto State e-Cadamy

Pan American International High School Pierce CollegePlacer Union High School DistrictPrince Georges County Public School (MD)Portland State UniversityPoway Unified School DistrictRAI Online Charter SchoolRiverside Unified School District (CA)Salish Kootenai College (MT)San Diego County Office of EducationSan Jose Education Foundation (CA)San Luis Obispo County Board of Education (CA)Santa Barbara County Education OfficeSchool District of the Chathams (NJ)Schenectady City SchoolsSchool & Main InstituteSIATechSierra Vista High School (CA)Sinclair Community CollegeSouth Carolina Virtual School (DOE) Southlands Christian SchoolsTulare County Office of Education (CA)Ulster County BOCESUniversity of Alaska at FairbanksUniversity of California, IrvineUniversity of Georgia System Board of Regents Universidad de Guadalajara (MX)Utah Electronic High SchoolUniversity of Texas El PasoUniversity of Texas at Brownsville University of Texas at DallasValley Christian School (CA)Village Christian SchoolsVirtual Virginia (DOE)West Virginia DOEWhitfield County School District (GA)

Partner Advisors:Council of Chief State Supervisors and

Officers (CCSSO)Southern Regional Education Board (SREB)

Classic HippocampusFr

ee, p

erso

naliz

ed le

arni

ng fo

r eve

ry p

erso

n

HippoCampus Classic

24 hours

Instructor Feedback Survey, February 2011

The new HippocampusFr

ee, p

erso

naliz

ed le

arni

ng fo

r eve

ry p

erso

nLink

http://new.hippocampus.org/

Free

, per

sona

lized

lear

ning

for e

very

per

son

Easy to navigate.

Free

, per

sona

lized

lear

ning

for e

very

per

son

Easy to resize. Includes closed captions.

Free

, per

sona

lized

lear

ning

for e

very

per

son

Khan Academy worked examples.

Free

, per

sona

lized

lear

ning

for e

very

per

son

PhET interactive simulations.

Free

, per

sona

lized

lear

ning

for e

very

per

son

Interactive tutors at the unit level.

PE!!

Free

, per

sona

lized

lear

ning

for e

very

per

son

And playlists for customized pathways.

Free

, per

sona

lized

lear

ning

for e

very

per

son

Even the site itself can be customized.

Free

, per

sona

lized

lear

ning

for e

very

per

son

The institutional and personalized channels merge on branded Hippocampus sites.

Why this design?

An object size......that enablescreativity with less risk.

http://www.flickr.com/photos/maxx361/4205316487/

http://www.flickr.com/photos/qthomasbower/3640362081/

http://www.flickr.com/photos/23021987@N06/4040082375/in/pool-638814@N24/

http://www.flickr.com/photos/qthomasbower/3640362081/

How are we doing...?

250,000 visits/month250,000 visits/month

8 million page views/month8 million page views/month

25 minutes per visit average25 minutes per visit average

Res

earc

h an

d ou

tcom

es

How are we doing...?

Algebra 1, Semesters 1 and 2 published, March 2011.Dev Math Arithmetic beta released, June 2011.

34 Algebra 1 research pilots nationwide (54 more queued).20 Dev Math Arithmetic pilots queued in 15 states.

MITE Algebra 1 Pilot Study EvaluationDraft, June 30, 2011

Algebra 1 published to HippoCampus,August 2011.

Documentary film on use of of Algebra 1at Sierra Vista Alt. HS (Whittier USD).

NROC Mobile Math App under dev.Res

earc

h an

d ou

tcom

es

But what about learning outcomes?R

esea

rch

and

outc

omes

DISTRICT STATISTICS● District Enrollment● Teacher FTE● Years of NROC

Membership● Learning Management

System and Learning Object Repository

INSTRUCTIONAL DESIGNS● Classroom Based Instruction‐● Blended/Hybrid Learning● Accelerated Course● Credit Recovery● Online Independent Study● Individualized Instruction● Asynchronous Learning● Self Paced Learning‐● Online Teacher Led‐● Remedial Education● Adult Education / GED

We are still gathering data from our pilot projects. We are striving for maximum variation in implementations, including along the following categories of interest:

But what about learning outcomes?

A detailed look at one pilot study:

Sierra Vista Alternative High School● HS juniors and seniors

● 62% credit deficiencies, 80% teen parents, 20% homeless

● Most repeating Algebra (school data)

● Hispanic 94%, White 13%; 75% low income

● Whittier USD district algebra benchmark unit tests

● Traditional classroom: textbook, lecture

● Hybrid-Lab: NROC Alg, teacher-coach, self-pacedRes

earc

h an

d ou

tcom

es

Sierra Vista Pilot StudyR

esea

rch

and

outc

omes

Res

earc

h an

d ou

tcom

esSierra Vista Pilot Study

Student ID (anonymized)

Results -- preliminary, 40 students

Algebra benchmark test passing score = 60%

Traditional class ● Test on a schedule● Half scored below 60%, ● One student scored over 80%

Hybrid class: NROC Alg 1● Test when ready● 95% scored 60% or better ● Half scored over 80%

Res

earc

h an

d ou

tcom


Results -- preliminary, 40 students

Algebra benchmark test passing score = 60%

Hybrid class: NROC Alg 1● Test when ready● 95% scored 60% or better ● Half scored over 80%

Res

earc

h an

d ou

tcom


Play video here

www.NROCmath.orgMore studies are under way...

Res

earc

h an

d ou

tcom

es

http://www.nrocmath.org/

A set of rules that describesour reactions to technologies:

Dou

glas

s Ad

ams.

The

Sal

mon

of D

oubt

. 200

2. 1. Anything that is in the world when you're born is normal and ordinary and is just a natural part of the way the world works.

2. Anything that's invented between when you're fifteen and thirty-five is new and exciting and revolutionary and you can probably get a career in it.

3. Anything invented after you're thirty-five is against the natural order of things.

Whom do we serve, and how?

Questions?

[email protected]