did we change?: the power of assessing our technology investments and interventions
DESCRIPTION
A sales pitch for the question: "Did We Change?" An argument that we need to carefully measure the impact of our technology investments using multiple research strategies.TRANSCRIPT
Did we change?
Justin Reich
Doctoral Student, Harvard Graduate School of Education
Co-Director, EdTechTeacher.org
@bjfr
Blog: edtechresearcher.org
Why Change?
How do we change?
Did we change?
The lower school at the American School of Brussels is working on increasing connections between home and school and having parents more aware of and involved in children’s classroom experiences.
They are requiring all elementary teachers to create blogs to help build these connections. How could you assess, short term and long term, whether this initiative is working?
Scenario 1
InterviewsFocus Group
SurveysContent Analysis
Test ScoreAnalysis
We’re lost. But we’re making great time.
How would you test French Fry Quality at
McDonalds?
The Newmarket High School has just purchased 300 iPads to allow them to pilot a 1 to 1 program in the ninth grade.
Teachers have been asked to create “21st century learning environments,” and they can use the iPads however they wish.
How could you assess whether this pilot should expand to the entire high school?
Scenario 2
The Fakename School District has a Wordpress install for classroom blogging.
Teachers have been asked to consider Will Richardson’s definition of blogging, emphasizing student autonomy, creativity, and attentiveness to a real audience
Bud Hunt wanted to write his MA thesis about blogging in the district. What should he do?
Scenario 3
What did Bud do?• He took a sample of all of his school’s blog
posts• He read them• He categorized them• He drew some conclusions about the
kinds of learning taking place based on that analysis
Using Data to Address Instruction
Identify a Problem of
Practice
Gather data about the problem
Examine Instruction
Develop an Action Plan
Act and Assess
Further Reading: Data Wise and Data Wise in Action
Final Thoughts• Identify learning goals connected to
technology investments and interventions• Measure progress towards those goals• Draw random samples rather than
assessing everyone– (and actually make them random)
• Leverage partnerships with universities and other organizations
• Use assessment data to reflect upon and improve instruction
