seoul foreign schools all[1]
DESCRIPTION
TRANSCRIPT
March 2011
All Schools Report
Seoul Foreign SchoolsSeoul, South Korea
600 Corporate Pointe, Suite 1180
Culver City, CA 90230
www.metiri.com
Table of Contents
1Dimensions 21 Introduction
Dimension 1: Forward Thinking, Shared Vision 2
Dimension 2: Systems Thinking 4
Dimension 3: 21st Century Skills and Learning Approaches 7
Dimension 4: 21st Century Learning Environments 11
Dimension 5: Teacher Proficiency with 21st Century Learning 16
Dimension 6: Access and Infrastructure 20
Dimension 7: Accountability and Results 25
Student Outcomes/Student Perspectives 28
Student Engagement 29
Classroom Structures to Engage Learners 36
All Schools Report
Introduction
Fast Facts
Project:
Location:
Timeframe:
Accomplishment:
Dimensions21 (D21)
Dimensions21 provides schools with insights
into the elements required to translate 21st
Century learning into action. The 7 dimensions
represent the divergent and innovative thinking it
takes to ground schools in emergent cognitive,
social, and neuroscience. Metiri Group
developed metrics that gauge a school or
district’s progress in establishing 21st Century
systems of learning. Each dimension is
calibrated on an 8-point scale:
D21 Scores: Year 1 (2011)
0 1 2 3 4 5 6 7 8
1: Vision
2: SystemsThinking
3: 21st CenturySkills
4: LearningEnvironment
5: TeacherProficiency
6: Infrastructure
7:Accountability
4.4
5.0
4.4
3.5
5.0
5.3
3.7
Province
31 0Total School N = Teachers & Administrators
The 1500 students attending Seoul Foreign
School represent approximately 52 nationalities,
making SFS easily the largest and most
well-established international school on the
Korean peninsula. Our students come mainly
from the expatriate business and diplomatic
community; all must have foreign (non-Korean)
passports to enroll at our school. The majority of
our students are from the United States,
followed by the United Kingdom, Canada,
Australia, and Japan. Our graduating class
consists of approximately 85 students each
year, virtually all of whom go on to 4 year
universities in North America and elsewhere
around the world.
1
Dimension 1: Forward-Thinking, Shared Vision
“The best vision is insight.”
-Malcolm S. Forbes
Vision matters. A forward-thinking, shared vision
serves as a unifying and energizing force of
change within a school system.
It sets the targets to which all curriculum,
instruction, assessment, scheduling, progress
reporting, resourcing, and community
communications are aligned.
The indicators and key questions within this
dimension include:
· A Forward-Thinking Vision for All
Learners. Is there a 21st Century vision for
all learners that defines what it means to be
educated in a knowledge-based, global
society?
· A Sound Base in the Learning Sciences. Does the vision represent current research
findings from the cognitive, social, and
neurosciences?
· Communication and Commitment. Have stakeholder groups had a voice in shaping
the vision? Has the jurisdiction
communicated with them about the vision?
Are they generally supportive of and
committed to the vision?
Aggregate Chart
Figure B
Dimension 1 Scores
The top chart at the right indicates the current
status of Dimension 1. In order to provide
context to the numeric scores, frequency reports
from several of the survey questions follow. In
some cases the results from the survey
questions will be in tables, in others charts, and
where appropriate, the responses will be cross
tabulated or provide comparisons between
respondent groups such as teachers and
administrators.
Dimension 1 Findings
Figure D1-2: Percentage of teachers reporting levels
of support for the Seoul Foreign School vision for 21st
Century learning.
Extremely supportive 58.1%Somewhat supportive 25.8%Undecided or don't know 16.1%
Total: 100.0%
0 1 2 3 4 5 6 7 8
Administrators
Teachers
Aggregate
4.4
4.4
Dimension 1
Figure D1-1: Summary of weighted scores from teacher
and administrator surveys.
2
Dimension 1 Findings
The responses to this question reveal the extent to which teachers see each 21st Century Skill
embedded in the vision. Since most schools are focusing on a subset of these 21st Century Skills, there
may be value in comparing the teachers’ perceptions to the actual wording of the vision, or perhaps the
current emphasis on particular skills by the school or jurisdiction.
0 1 2 3 4 5 6 7 8
Social responsibility
Citizenship in a changing,global society
Individual integrity and ethics
Community connections
Cultural diversity
Digital literacy
Knowledge work andentrepreneurship
Lifelong learning
4.8
5.2
5.6
3.5
4.5
4.2
3.4
4.8
Figure D1-3: Teachers' ratings of the emphasis (on a scale of 1-8)
of each 21st Century Skill embodied in the school vision.
3
Dimension 2: Systems Thinking
"Systems thinking is a discipline for seeing
wholes, recognizing patterns and
interrelationships, and learning how to structure
those interrelationships in more effective,
efficient ways."
- Peter Senge, Schools That Learn
Fundamental to the application of systems
thinking in education is an openness to
innovation, reinvention, and formative, systemic
change driven by the vision. The indicators and
key questions within this dimension include:
Aggregate chart
Dimension 2 Scores
· Leadership. Are the school and/or
jurisdiction leaders leading a
high-performance education system that
enables each student the fullest opportunity
to achieve the vision?
· Curricula, Instruction, and Assessment.
Do the jurisdiction learning standards
integrate academic content with 21st
Century Skills? Are curricula, instruction,
and assessments aligned to provincial
standards?
· Professional Development. Do the school
and jurisdiction provide comprehensive
professional growth opportunities for
administrators, teachers, and other staff,
which build their capacity to advance the
vision?
· Culture of Learning and Innovation. Does
the school or jurisdiction encourage and
support school change that advances 21st
Century learning, and innovative,
educationally sound uses of technology?
· Policies Supportive of the Vision. Are
there established policies that formally
establish 21st Century learning and effective
technology use as a required design
element in all strategic planning, school
improvement, budgeting, human resources,
and accountability systems?
The top chart indicates the current status of
Dimension 2. Results from specific questions
follow.
Dimension 2 Findings
Figure D2-2: Percentage of teachers reporting their
agreement with the statement: Teachers are provided
the resources and support to redesign classrooms into
21st Century learning environments.
Always 16.1%Sometimes 51.6%Rarely 19.4%Never 9.7%Don't know/Not sure 3.2%
Total: 100.0%
0 1 2 3 4 5 6 7 8
Administrators
Teachers
Aggregate
5.0
5.0
Dimension 2
Figure D2-1: Summary weighted scores from teacher
and administrator surveys (2009)
4
The opportunity for high-quality, relevant professional development is critical to any change process. The
following questions provide insights into teachers’ agreements with statements about the professional
development experiences currently made available to them through their school or jurisdiction.
The statements represent characteristics and attributes of high quality professional development
experiences. The reader should note where large percentages of teachers indicate that the statement is
"Never" or "Rarely" representative of their jurisdiction or school’s professional development, and set
targets to improve that situation.
Dimension 2 Findings
Insert Figure G here
It is important that, as 21st Century Skills are integrated into lessons, it is accomplished systematically
and systemically, so all children are afforded such opportunities, not just those students who happen to
be assigned to classrooms of innovative teachers.
The data in the chart on the top of the next page represent strong indicators as to whether or not your
school or jurisdiction is taking the necessary policy actions that will result in systemic change and
integration of 21st Century Skills and effective technology uses to advance learning.
0% 20% 40% 60% 80% 100%
10%
16%
3%
16%
10%
13%
16%
29%
42%
39%
32%
55%
48%
42%
58%
35%
42%
45%
29%
29%
32%
3%
3%
13%
3%
3%
6%
3%
3%
6%
3%
6%
3%
Includes opportunities for teachers to see actual examples oftechnology applied to learning in classrooms similar to my own.
Includes opportunities for teachers to see actual examples of 21stCentury Skills applied to learning in classrooms similar to my own.
Allows teachers to practice skills acquired during professionaldevelopment in real or simulated classroom settings.
Prepares teachers to discuss specific research or theory upon whichthe training is based.
Prepares teachers to assess student work produced with technology.
Prepares teachers to assess student work related to 21st CenturySkills.
Includes time for teachers to work together, and to discuss and planfor using technology in the classroom.
Never Rarely Sometimes Always Don't Know/Not Sure
Figure D2-3: Percentage of teachers reporting on their level of agreement about these statements regarding the degree to
which professional development provided by their schools or jurisdiction:
5
Insert Figure H here
Dimension 2 Findings
The final chart in this section reports the percentage of administrators in the province who report that
teachers are required to consider innovative approaches to teaching and learning in their classrooms
(see list of innovations in the chart below). These results are strong indicators of the degree to which the
jurisdiction is systemically integrating 21st Century learning and technology across the system.
Dimension 2 Findings
Insert Figure I here
Figure D2-5: Percentage of administrators indicating their level of agreement with the statement, When
teachers implement jurisdiction curriculum, or design curricula, this school requires that they consider:
0% 20% 40% 60% 80% 100%
21st Century Skills have beenpurposefully incorporated into
learning standards.
21st Century Skills have beenpurposefully integrated into curricula
where appropriate.
Our assessments track studentprogress with 21st Century skills.
25%
14%
36%
58%
69%
44%
17%
17%
4% 16%
Rarely Sometimes Always Never
Figure D2-4: Percentage of teachers reporting on their level of agreement about these
statements (systemic integration of 21st Century Skills).
6
Dimension 3: 21st Century Skills and Learning Approaches
· Knowledge Age Literacies. Are students
acquiring and excelling at the skills needed
to be “literate” in the Knowledge Age?
· Inventive Thinking. Are students thinking
critically and creatively as they successfully
solve problems using high tech tools?
· Community Interaction. Are students
acquiring such skills?
· Generating Quality Results. Are students
learning to plan, manage, and achieve high
quality, impactful results?
· Authenticity and Engagement. Are
students being assigned rich, authentic work
that engages them and involves
construction of knowledge through
disciplined inquiry, resulting in products that
have value beyond the classroom?
The top chart indicates the current
status of Dimension 3. Results from specific
questions follow.
“For more than half a century, the United States
has led the world in scientific discovery and
innovation. It has been a beacon, drawing the
best scientists to its educational institutions,
industries, and laboratories from around the
globe. However, in today’s rapidly evolving
competitive world, the United States can no
longer take its supremacy for granted. Nations
from Europe to Eastern Asia are on a fast track
to pass the United States in scientific excellence
and technological innovation.”
--Taskforce on the Future of
American Innovation
Innovation is fueling the economy of the 21st
Century. Globalization has created new markets
and leveled the playing field citizens of all
nations around the world. As society changes,
the skills that citizens need to negotiate the
complexities of life also change. Innovative,
inventive thinking was once required for only a
limited few. Today, and tomorrow, it will be the
currency for success in virtually every field.
The elements within this dimension include:
Figure J
Aggregate Chart
Dimension 3 Scores
Dimension 3 Findings
0% 20% 40% 60% 80% 100%
Teaching critical thinking skills
Building creative thinking skills
Teaching problem solving tostudents
7%
8%
7%
Teachers Administrators
Figure D3-2: One of the most critical 21st Century Skill area
is higher order thinking. This chart provides a 2008
reference point for thinking skills curricula.
The percentage of teachers and administrators answering
“yes” to the question: Does your school have a formal
curriculum for:
0 1 2 3 4 5 6 7 8
Administrators
Teachers
Aggregate
4.4
4.4
Dimension 3
Figure D3-1: Summary weighted scores from teacher and
administrator surveys (2009)
7
The following charts are purposely matched up. The left-hand column reports current status of initiatives
related to each skill as reported by administrators. The right-hand column charts both teachers’ reports of
relevance of each skill to their content, and teachers’ comfort level with each skill. It is important to
compare not only the level indicated by the administrators and the teachers, but any gaps that exist
between the teachers’ comfort level and relevance level for each skill.
Dimension 3 Findings Dimension 3 Findings
Figure L
Dig
ital
Lit
era
cy
Dimension 3 Findings Dimension 3 Findings
Figure O Figure N
Inve
nti
ve T
hin
kin
g
0 1 2 3 4 5 6 7 8
Global Awareness
Scientific Literacy
Information Literacy
Multimodal Literacy
Figure D3-3: Administrators' mean score indicating the
degree to which a formal initiative on the 21st Century
skill is underway in their schools (scale 1-8):
0 1 2 3 4 5 6 7 8
Global Awareness
Scientific Literacy
Information Literacy
Multimodal Literacy
5.1
2.5
5.3
4.1
6.5
2.8
6.4
5.6
Teacher comfort level Relevance to content
Figure D3-4: Teachers' mean score related to comfort
level with each skill and relevance to their content
area:
0 1 2 3 4 5 6 7 8
Creativity andInnovation
Critical Thinking
Flexibility &Adaptability
Self-direction
Systems Thinking
Figure D3-5: Administrators' mean score indicating the
degree to which a formal initiative on the 21st Century
skill is underway in their schools (scale 1-8):
0 1 2 3 4 5 6 7 8
Creativity &Innovation
Critical Thinking
Flexibility &Adaptability
Self-direction
5.7
5.9
5.5
5.8
6.8
7.1
6.4
6.8
Teacher comfort level Relevance to content
Figure D3-6: Teachers' mean score related to comfort
level with each skill and relevance to their content
area:
8
The comparison of administrator and teacher perspectives on 21st Century Skills continues:
Dimension 3 Findings Dimension 3 Findings
Figure Q Figure P
Effe
ctiv
e C
om
mu
nic
atio
ns
Dimension 3 Findings Dimension 3 Findings
Figure S Figure R
Hig
h-Q
ual
ity
Pro
du
ctiv
ity
It is recommended that the reader identify any gaps that exist between the administrator and teacher
perspectives related to relevance of the 21st Century Skills or between the teacher comfort level and
relevance to content. Such gaps should be addressed through action plans at the jurisdiction and building
levels. Look for opportunities represented by teachers’ indicators for high relevance, but underdeveloped
comfort levels. In these situations the applicable professional development and teacher support can
quickly be leveraged into opportunities that positively impact students.
0 1 2 3 4 5 6 7 8
InteractiveCommunication
Teaming &Collaboration
Cross-cultural Skills
Personal & SocialResponsibility
Figure D3-7: Administrators' mean score indicating the
degree to which a formal initiative on the 21st Century
skill is underway in their schools (scale 1-8):
0 1 2 3 4 5 6 7 8
InteractiveCommunication
Teaming &Collaboration
Cross-cultural Skills
Personal & SocialResponsibility
5.5
5.9
5.3
5.7
6.8
6.6
6.0
6.4
Teacher comfort level Relevance to content
Figure D3-8: Teachers' mean score related to comfort
level with each skill and relevance to their content
area:
0 1 2 3 4 5 6 7 8
Prioritizing, Planningand Managing for
Results
Effective Use ofReal-world Tools
Productivity Category
Figure D3-9: Administrators' mean score indicating the
degree to which a formal initiative on the 21st Century
skill is underway in their schools (scale 1-8):
0 1 2 3 4 5 6 7 8
Prioritizing, Planning andManaging for Results
Effective Use of Real-worldTools
Productivity Category
4.7
3.9
4.4
6.3
6.1
5.8
Teacher comfort level Relevance to content
Figure D3-10: Teachers' mean score related to
comfort level with each skill and relevance to their
content area:
9
Teachers assign a variety of types of work to students, depending on many factors some of
which are outside of their control. For each of the following categories of student work, teachers and
administrators were asked to estimate the percentage of work that fell into that category. The totals are
more than 100% because of duplication within single assignments.
Dimension 3 Findings
Insert Figure T here
Two key factors in student engagement are opportunities for student choice and student creativity. The
following chart provides insight into such opportunities in your school or jurisdiction.
Dimension 3 Findings
Insert Figure U here
0% 20% 40% 60% 80% 100%
Skill building exercises (worksheets,problem-sets, etc.)
Applying skills through essays, short-answerproblems or exercises
Written products that summarize content relatedto the curriculum (e.g. reports)
Written products that require the creation oforiginal content
Applying skills to problems that are complex andemulate work done in the real world
41%
33%
30%
31%
37%
Teachers Administrators
Figure D3-11: Percentage of work assigned by teachers that falls in the
following categories, as reported by administrators and teachers:
0% 20% 40% 60% 80% 100%
11%
59%
38%
15%
Student-designed assignments
Teacher-designed assignments
Technology-based products defined by the teacher
Student-designed, technology-based products
Teachers Administrators
Figure D3-12: Percentage of work in classrooms in your school that falls in
the following categories, as reported by administrators and teachers:
10
Dimension 4: 21st Century Learning Environments
The learning environment is where the vision
becomes a reality, where 21st Century learning
comes to life.
· Alignment with 21st Century Vision. Do
the jurisdiction content, instruction, and
assessment align to 21st Century learning
and academic content standards?
· Informed Practice. Are educators
establishing learning environments that are
structured as respectful classroom
communities where students can work
creatively and productively, places that
motivate, interest, and scaffold students to
think critically?
· Culture of Innovation, Engagement, and
Collaboration. Are professional learning
teams working together to design and
facilitate collaborative 21st Century learning
activities with students? Are activities
evidence-based? Are students producing
high-quality work that is valued by peers,
parents, and community?
· Resources Aligned to 21st Century
Learning. Do students have access to a
wide variety of multimodal resources? Are
these sources accessible inside and outside
the school environment?
· Digital Tools: Range of Use. Do students
have the opportunity to use a range of
technologies (e.g., productivity tools,
visualization tools, research and
communication tools, etc.) to support 21st
Century learning and academic
achievement?
· Assessment for Learning. Is assessment
systematically used to inform practice? Do
students set learning goals based on
standards? Are they actively engaged in
monitoring their own progress toward those
goals?
· Local and Global Connections. Are there
formal, technology-based structures that
engage stakeholders and learners in
meaningful exchanges, interactions, and
partnerships at the local and global levels?
The top chart at the right indicates the current
status of Dimension 4. Results from specific
questions follow.
Figure V
Figure W
Dimension 4 Findings
Dimension 4 Scores
0% 20% 40% 60% 80% 100%
Strongly Agree
Agree
Disagree
Strongly Disagree
Don't Know/Not Sure
Others
4%
8%
14%
6%
11%
56%
Overall
Figure D4-2: Percentage of teachers indicating
agreement with the statement: Teachers in the
Emerge program know what the school’s expectations
are for student attainment of 21st Century Skills.
0 1 2 3 4 5 6 7 8
Administrators
Teachers
Aggregate
3.5
3.5
Dimension 4
Figure D4-1: Summary weighted scores from teacher
and administrator surveys (2008)
11
The perceptions of teachers as to the applicability of the use of technology to their specific teaching
assignments provides insights into why some classrooms are integrating technology and others are not.
Dimension 4 Findings
Insert Figure X here
If all students are to be afforded new opportunities in 21st Century learning and the innovative use of
technologies, all teachers must adopt such use systemically. Otherwise, student opportunity is
determined by which teacher’s classroom a student happens to be assigned. The chart below indicates
the extent to which teachers in your school are systemically implementing evidence-based practices.
Dimension 4 Findings
Insert Figure Y here
0% 20% 40% 60% 80% 100%
25%
23%
39%
53%
13%
36%
50%
50%
55%
28%
33%
69%
36%
13%
25%
23%
22%
7%
13%
18%
11%
7%
6%
9%
38%
Reading
Language Arts
Mathematics
Science
Social Studies
Arts
Health/PE
Insignificant Role Moderate Role Significant Role No Role
Figure D4-3: Percentage of teachers indicating the role technology plays in building skills
or proficiencies in their students in the following content areas:(NOTE: These data reflect responses only from teachers who indicated the subject was applicable to their teaching
n: Reading (28), Language Arts (22), Mathematics (18), Science (15), Social Studies (16), Arts (11), and Health/PE (8).
0% 20% 40% 60% 80% 100%
Share little or no common understanding aboutevidence-based practices. Teachers decideindividually whether and how they will make
instructional decisions.
Share some common understanding aboutevidence-based practices; however, some
teachers implement these uses and others donot.
Share a common understanding aboutevidence-based practices, and there are clearexpectations that such practices will be used.
8%
32%
3%
Figure D4-4. Percentage of teachers that reported: In my school teachers in the
same grade or subject areas:
12
The type of instructional strategy used in classrooms can augment, or inhibit, 21st Century learning and
technology use. Highly qualified teachers use a variety of strategies. As a rule of thumb, the reader might
look for a balance of use across the instructional strategies in the chart below, while ensuring that the
strategies that engage learners (i.e., interactivity, inquiry, collaboration) and those that build skills and
meet individual student needs (e.g., differentiation) are all fairly high.
Dimension 4 Findings
Insert Figure Z here
Students learn in a multiple of venues, many of which are outside the formal school day. Increasingly,
educators are recognizing the value in preparing students to be self-directed in these informal learning
spaces. This helps them ask deep questions and sustain curiosity, as they seek clarity, depth, accuracy,
comprehensiveness, and currency of the topics they are exploring. Informal learning represents
tremendous opportunity for extending students’ exploration and understanding of academic content
beyond the school day.
0 1 2 3 4 5 6 7 8
Direct Instruction (e.g., lecture, didactic questioning,demonstrations, guided reading, etc.)
Inquiry (e.g., guided inquiry, problem-based learning,learning from cases, etc.)
Mediating student thinking through questioningstrategies, thinking skills, and Habits of Mind
applications
Experiential learning (e.g., field trips, simulations,games, conducting experiments, etc.)
Collaborative teaming (e.g., students workingcollaboratively on an assigned project, etc.)
Independent study by individuals or teams (e.g., writingessays, producing videos, computer-assisted
instruction, virtual learning, journaling, researchprojects, etc.)
Interactive instruction (e.g., active learning strategies,debates, brainstorming, think/pair/share, jigsaw,
problem solving, conferencing, etc.)
Differentiation of instruction (i.e, multiple approaches tolearning a single topic)
5.2
5.1
4.5
3.9
5.7
5.5
4.6
4.3
3.7
6.6
5.9
6.3
6.6
6.0
5.8
6.3
Current UseImportance for 21stCentury Learning
Figure D4-5: Teachers’ ratings (scale 1-8) related to various instructional strategies.
13
The following chart indicates the importance teachers place on such informal learning and their
perceptions of students’ current use of such.
Dimension 4 Findings
Insert Figure AA here
Technology use in schools is shifting to more collaborative and innovative uses of Internet resources,
Web 2.0 tools, and multimedia production. The chart below provides insights from administrators as to
current use and importance of such technology uses by students.
Dimension 4 Findings
Insert Figure AB here
0 1 2 3 4 5 6 7 8
3.1
2.6
3.4
5.0
5.1
4.9
Informal learning at school (afterschool activities,
peer-interactions, etc.)
Informal learning beyond the school day(chat, text messaging, student web
browsing/searching, etc.)
Mentoring or coaching by an adult (e.g.,parent, teacher, expert, etc.)
Current UseImportance for 21stCentury Learning
Figure D4-6: Teachers’ ratings (scale 1-8) related to informal learning strategies.
0 1 2 3 4 5 6 7 8
Solve real-world problems
Produce print products
Produce multi-media, Web, digital audio,digital video, or presentation products
Conduct online research
Use drill and practice or tutorial software
Use the Internet to collaborate withstudents in your school, district, or local
community
Online communication with experts, peers,and others
Current UseImportance for 21stCentury Learning
Figure D4-7: Administrator ratings of current uses of technology by students in their schools,
and the administrators rating of the importance of such uses to 21st Century learning:
14
The Internet has literally linked the individual to social and professional networks 24/7. Those
communication avenues represent opportunities for schools to increase communication with parents,
community, and students. The following chart outlines the current status of your school’s uses of
technology to facilitate such outreach and interaction.
Dimension 4 Findings
Insert Figure AC here
After a careful review of these data, the reader will want to consider the critical questions listed on the first
page of the section, identify any gaps in your schools performance, and set targets for improvement.
0% 20% 40% 60% 80% 100%
Parents’ involvement in my students’ schoolwork
Interactions with students’ parents
Students’ work on authentic projects in their localcommunity
Students’ work on authentic projects outside theirlocal community
23%
13%
71%
61%
48%
23%
13%
13%
29%
65%
16%
26%
Technology is not usedfor this purpose
Occasionally facilitatedby technology
Strongly facilitated bytechnology
Figure D4-8: Parental or community involvement facilitated by technology.
The percentage of teachers that reported:
15
Dimension 5: Teacher Proficiency
“On a daily basis, teachers confront complex decisions that rely on many different kinds of knowledge
and judgment and that can involve high-stakes outcomes for students’ futures.”
- John Bransford, Linda Darling-Hammond, & Pamela LePage
The transition to 21st Century requires systemic
action that builds the capacity of teachers and
administrators.
· Knowledge and Facility with 21st Century
Skills. Are teachers in this school familiar
with the concept of 21st Century Skills and
with the research underpinnings and
practical applications of these skills?
· Building 21st Century Skills. Do teachers
in this school have a variety of strategies for
building these skills?
· Designing Rigorous Authentic Curricula.
Are teachers skilled in designing rich
curricula that integrates content, 21st
Century Skills, and technology, which
provides a digital age learning context?
· Differentiated Instructional Strategies.
Are teachers in this school skilled in
engaging all students in learning through a
variety of teaching and organizational
strategies that are tailored to the needs of
individual students?
· Informed Use of Data and Research. Are
teachers in this school skilled at accessing,
organizing, and acting upon available data
to make important decisions about students
and learning?
· Assessment for Learning. Do teachers in
this school have a deep understanding of
the central role of assessment in the
learning process and leverage technology
resources to assess core content and 21st
Century Skills?
· Professional Practice and Productivity.
Are teachers skilled in the use of technology
to support their own professional practices
and do they depend on technology to
Dimension 5 Scores
Figure AE
Pie chart
Dimension 5 Findings
The top chart at the right indicates the current
status of the province in this dimension. In order to
provide context to the numeric scores, frequency
reports from several of the survey questions are
provided on the following pages.
Figure D5-2: The percentage of teachers indicating
their preparedness in assessing student products
created using technology.
Expert 17.2%Intermediate 58.6%Novice 24.1%
Total: 100.0%
0 1 2 3 4 5 6 7 8
Administrators
Teachers
Aggregate
5.0
5.0
Dimension 5
Figure D5-1: Summary of weighted scores from
teacher and administrator surveys (2009)
16
The charts on this page provide teachers’ perspectives on their preparedness to scaffold conceptual
learning, and similarly, their preparedness to do the same for authentic learning. Teachers typically report
higher levels of comfort with the conceptual learning, but, for 21st Century learning, the latter is critical.
Dimension 5 Findings
Insert Figure AF here
stacked bar chart
Dimension 5 Findings
Insert Figure AG here
stacked bar chart
If a significant percentage of your teachers indicate a lack of preparedness to scaffold conceptual learning,
your school or jurisdiction will want to provide professional development, modeling, or perhaps peer
coaching. Authentic learning is key to increasing student engagement and deep understanding of
academic concepts.
0% 20% 40% 60% 80% 100%
6%
19%
6%
48%
32%
58%
52%
26%
61%
23%
42%
19% 6%
Work that requires students to read andunderstand content related to your
subject area.
Work that requires students to applyskills from your content area to
hypothetical problems or situations.
Work that requires students todemonstrate understanding of the core
ideas within your content area.
Work that is multidisciplinary andconnects skills and concepts from
multiple content areas.
Somewhat comfortable Comfortable Extremely comfortable Not comfortable
Figure D5-3: Teacher preparedness to scaffold conceptual learning.
The percentage of teachers indicating their comfort level in incorporating the following
types of assignments into their teaching and their students’ learning.
0% 20% 40% 60% 80% 100%
Work that requires students to usethe language and methods
professionals would use when dealingwith the content you teach.
Work that involves students inapplying concepts to real-world
problems.
Work that has an audience outsidethe classroom.
13%
16%
26%
26%
39%
26%
48%
39%
32%
13%
6%
16%
Not comfortable Somewhat comfortable Comfortable Extremely comfortable
Figure D5-4: Teacher preparedness to scaffold conceptual learning.
The percentage of teachers indicating their comfort level in incorporating the following
types of assignments into their teaching and their students’ learning.
17
The top chart on this page provides insights into how prepared teachers believe they are in their use of
assessment data and research to inform their decisions. The bottom chart asks about their preparedness
to assess technology-based student products and to use technology for assessment purposes.
Dimension 5 Findings
Insert Figure AH here
stacked bar chart
Dimension 5 Findings
Insert Figure AI here
stacked bar chart
Many teachers are not yet experienced in assessing student products that are multimedia based (e.g.,
animation, movies, audio files, simulations, etc.). As technology is integrated into curricula it is paramount
that teachers learn to do so against established standards of content, process, design, and purpose. On
the flip side, teachers need to become accomplished users of the technology to collect, manage, and
interpret a continuous stream of data to inform their instructional decisions. Use these charts to analyze
your staff’s current preparedness, and to set targets for improvement.
0% 20% 40% 60% 80% 100%
Use assessment data to make decisionsabout students and learning
Use research to make decisions aboutstudents and learning
Create and implement performance-basedassessments
Providing opportunities for student toself-assess based on a rubric or standard
7%
3%
7%
17%
23%
7%
10%
59%
53%
48%
50%
17%
20%
38%
40%
Not yet prepared Novice Intermediate Expert
Figure D5-5: Teacher preparedness in assessment.
The percentage of teachers indicating their preparedness to inform
the decisions and practices using data and research.
0% 20% 40% 60% 80% 100%
Use technology for analyzing student data
Assess student products created by usingtechnology
13% 27%
24%
50%
59%
10%
17%
Not yet prepared Novice Intermediate Expert
Figure D5-6:Teacher preparedness in assessment.
The percentage of teachers indicating their preparedness to inform
the decisions and practices using data, research, and technology.
18
Finally, the following two charts are the administrators’ perspectives on teacher preparedness across a
broad array of teaching strategies critical to 21st Century learning.
Dimension 5 Findings
Insert Figure AK here
stacked bar chart
Dimension 5 Findings
Insert Figure AL here
stacked bar chart
This dimension on teacher proficiency is one of the most critical. Readers are encouraged to use these
data to chart a course that leads to increased teacher preparedness to teach and learn in 21st Century
classrooms.
Figure D5-7: Teacher preparedness: 21st Century learning.
Percentage of administrators reporting on levels
of teacher preparedness related to 21st Century learning
Figure D5-8: Teacher preparedness: 21st Century learning.
Percentage of administrators reporting on levels
of teacher preparedness related to 21st Century learning
19
Dimension 6: Access and Infrastructure
The level of access to technology tools and the
robustness and reliability of the technology
infrastructure serve as critical foundations for
21st Century learning. The elements of this
dimension include:
· Range of Technology Tools. Are a wide
range of technology tools, software, and
environments available to support all
aspects of teaching and learning?
· Robust Infrastructure. Are the network and
technical infrastructure of the school
sufficient to provide seamless access to all
in the school community?
· Longitudinal Data System. Are systems in
place to provide all educators in the system
with seamless access to the data that they
need to support their professional decision
making?
· Technical Support. Is there adequate
technical support to provide timely
assistance to all users within the system?
· Technology-Ready Facilities. Is the school
building well suited to 21st Century teaching
and learning?
· Digital Learning Environments. Where
appropriate, are digital and virtual access to
learning opportunities available to all in the
school community?
· Administrative Processes and
Operations. Is technology leveraged to
ensure well-informed and efficient
administration at all levels of the school and
jurisdiction?
· Service Orientation. Are all staff with
responsibilities for infrastructure, technology
deployment, and technical support oriented
toward providing high-quality service? Do
they acknowledge the primacy of the
The top chart indicates the current status of the
province in this dimension. In order to provide
context to the numeric scores, frequency reports
from several of the survey questions are provided
on the following pages.
Dimension 6 Scores
Figure ??
aggregate
Figure ??
Pie chart
Dimension 6 Findings
0 1 2 3 4 5 6 7 8
Administrators
Teachers
Aggregate
5.3
5.3
Dimension 6
Figure D6-1: Summary of weighted scores from
teacher and administrator surveys (2009)
Figure D6-2: The percentage of teachers indicating the
degree to which technical support is provided with
little or no wait-time.
Excellent 6.5%Good 22.6%Adequate 38.7%Very poor 22.6%Non-existent 9.7%
Total: 100.0%
20
Schools want to be sure that purchased technologies are valued and used by teachers in the classroom.
The two charts on this page provide insights into how administrators', across the province, perceive the
usefulness of various technologies to the teaching responsibilities of their teachers, in comparison to
current availability.
Dimension 6 Findings
Insert Figure AO here
stacked bar chart
Dimension 6 Findings
Insert Figure AP here
stacked bar chart
Figure D6-3: Instructional technologies.
Comparision of percentage of administrators' ratings of availability of technology tools
to their perception of the usefulness of tools.
Available = Adequate to Meet Needs or Ubiquitous; Useful = Useful or Very Useful.
Figure D6-4: Technology peripherals.
Comparision of percentage of administrators' ratings of availability of technology tools
to their perception of the usefulness of tools.
Available = Adequate to Meet Needs or Ubiquitous; Useful = Useful or Very Useful.
21
Schools also want to balance their investments in high-speed networks with investments in computers
linked to that network. Imbalances in the direction of the network results in inefficiencies and untapped
potential, while imbalances in the direction of computers result in frustrations due to slow and inadequate
network capacity.
Dimension 6 Findings
Insert Figure AU here
stacked bar chart
Dimension 6 Findings
Insert Figure AQ here
stacked bar chart
0% 20% 40% 60% 80% 100%
The Internet connection is fast and reliable
Technologies are physically placed formaximum convenience and effectiveness
Access to networked files and resources isconvenient and easy to understand
100%
61%
84%
39%
16%
TrueNot True
Figure D6-5: Infrastructure.
Percentage of teachers who reported on how each statement applied to their school.
Figure D6-6: Adequacy of level of technology access.
Percentage of administrators who reported in each accessibility category for each item.
22
As those balances are achieved, the school will want to ensure accessibility to online learning structures
including blogs, wikis, communication systems, online courses, etc.
Dimension 6 Findings
Insert Figure AR here
stacked bar chart
Dimension 6 Findings
Insert Figure AS here
stacked bar chart
Just as critical as the high-speed network and adequacy of computer access, is the responsiveness of
technical support available to teachers and administrators.
Figure D6-7: Accessibility to online learning structures.
Percentage of administrators who reported in each accessibility category for each item.
Figure D6-8: Technical support for technology.
Percentage of administrators who reported in each accessibility category for each item.
23
Support also includes scaffolding and building capacity of teachers to use technology in their professional
practice. The chart below provides a snapshot of current perceptions of administrators on the extent to
which their teachers' professional use of technology is supported.
Dimension 6 Findings
Insert Figure AT here
stacked bar chart
Figure D6-9: Support for teacher's professional use of technology.
Percentage of administrators who reported in each accessibility category for each item.
24
Dimension 7: Accountability
The accountability dimension provides a look at
the alignment between goals, assessments, and
results.
The indicators include:
· Accountability System Aligned to Vision.
Has the accountability system been
redesigned to ensure that the vision is
achieved within a prescribed timeframe?
Have policies been rewritten to ensure that
planning, resource allocation, time
investment, curriculum redesign,
professional development, and other
elements of the system are orchestrated to
advance the vision?
· Clarity, Transparency, and
Consequences. Do educators, students,
parents, and community members
understand what the vision is, why it is
important, and what it means to their
respective roles in schools? Do they
understand what the vision will look like if
achieved, the assessments used to monitor
progress toward the vision, and the
consequences associated with failure to
achieve incremental milestones toward the
vision?
· Comprehensive, Prioritized Funding.
Have the school and jurisdiction analyzed
the full cost of implementing the vision over
time and have they committed sufficient
funds in the short and long term in order to
achieve the vision within established
timeframes?
· Decision Making Informed by Data and
Research/Results. Is the data analysis, in
combination with research, appropriately
informed, and does it contribute to the
continuous improvement of the system?
· Results. Are jurisdiction and school making
progress toward their goals?
Dimension 7 Scores
Figure AV
aggregate
Figure AW
Pie chart
Dimension 7 Findings
The top chart at the right indicates the current
status of your school or jurisdiction in this
dimension. In order to provide context to the
numeric scores, frequency reports from several
of the survey questions are provided on the
following pages.
0 1 2 3 4 5 6 7 8
Administrators
Teachers
Aggregate
3.7
3.7
Dimension 7
Figure D7-1: Summary of weighted scores from
teacher and administrator surveys (2009)
Figure D7-2: Percentage of teachers reporting the
level of their agreement with the statement: Students
in my classroom are clear about how they need to
demonstrate their skill level with the targeted 21st
Century Skills.
Agree 25.8%Disagree 67.7%Strongly disagree 6.5%
Total: 100.0%
25
The frequent use of data to inform instructional practices is critical in meeting all students’ needs. This
chart provides perspectives on the importance of various assessments in informing classroom practices.
Dimension 7 Findings
Insert Figure AX here
bar chart
Dimension 7 Findings
Insert Figure A3 here
bar chart
The chart below provides data on the use of such assessments to track students’ attainment of 21st
Century Skills.
0% 20% 40% 60% 80% 100%
District wide assessment
School wide assessment
Provincial assessment
End of course tests
Periodic quizzes and tests
Performance assessments (rubric based)
Technology literacy assessment
13%
25%
23%
28%
37%
11%
Figure D7-3: Assessments informing classroom practices.
The percentage of teachers reporting that the following types of assessments
were “important” or “key” factors in informing their instructional decisions.
0% 20% 40% 60% 80% 100%
Jurisdiction-wide assessment
School-wide assessment
End of course tests
Periodic quizzes and tests
Performance assessments (rubric based)
Technology literacy assessment
Classroom observations
Student self-assessment
Analysis of student products
Student peer reviews
We don’t assess 21st Century Skills
Figure D7-4: Assessments of 21st Century learning.
Percentage of administrators who indicated the following methods
were routinely used to assess student attainment of 21st Century Skills.
26
Student outcomes are ultimately the focus on 21st Century learning and effective technology uses. This
final chart provides the teachers’ perspective on the current level of student expertise with 21st Century
Skills.
Dimension 7 Findings
Insert Figure A2 here
stacked bar chart
School and jurisdiction leaders are encouraged to identify those skills targeted in their short and
long-term goals and set targets to increase the percentage of students in the Intermediate and Expert
0% 20% 40% 60% 80% 100%
3%
6%
3%
3%
41%
37%
60%
42%
23%
32%
40%
59%
43%
55%
60%
30%
48%
68%
55%
57%
38%
47%
3%
10%
3%
6%
13%
3%
10%
Global Awareness
Information Literacy
Critical Thinking
Self-direction
Interactive Communication
Teaming & Collaboration
Personal & Social Responsibility
Effective Use of Real-world Tools
Productivity
No skillNoviceIntermediateExpert
Figure D7-5: Percentage of teachers' rating of student level of expertise in 21st Century Skills.
27
Student Outcomes/Student Perspectivesby Emerge Research Team: Metiri Group and University of Calgary
Ultimately, the mission of schools is to ensure that students acquire the skills, knowledge, processes, and
dispositions that enable them to meet or exceed state and local learning standards. For increasing
numbers of schools those standards include academic and 21st Century learning goals, goals that will
prepare them to thrive in today’s global, high tech society and workplace.
Metiri's Student Outcomes/Student Perspectives report provides findings related to students, including:
Student Engagement
Student engagement represents actual current levels of engagement as reported by individual students.
Students in your school or jurisdiction completed Metiri Group’s Student Engagement Inventory online.
The data from those surveys were then analyzed and aggregated to report the percentage of students at
five different levels of engagement: Intrinsically Engaged, Tactically Engaged, Compliant, Withdrawn, or
Defiant.
Classroom Structures to Engage Students
This section looks at student perceptions of conditions in the school that have been linked by research to
increases in student engagement. The Metiri Group survey, Classroom Structures that Engage Students
(CSES), asked students in your school or jurisdiction to share their perceptions on characteristics of
learning environments that research indicates are directly related to the level of student engagement in
learning. These characteristics include choice, structure, diversity, intellectual safety, clarity, affiliation,
and authenticity. The student perceptions on these classroom structures were then clustered into three
domains (i.e., content, process, and product) for reporting purposes.
28
Student Engagement
Definition of Student Engagement
Degree to which students are actively
pursuing deep learning related to
established standards.
The measure of student engagement
comprises:
· Cognitive Engagement. A student’s
Investment in the effort required to
comprehend complex ideas and master
difficult skills.
· Behavioral Engagement. A student’s
participation in academic, social, and
extracurricular activities.
· Social/Emotional Engagement. A
student’s interdependence with classmates,
academics, teachers, and school.
Based on: Fredricks, J. A., Blumenfeld, P.
C., & Paris, A. H. (2004).
Are your students genuinely interested in the
topics they are studying? Are they highly
motivated and committed to learn? Do they
persevere when challenged with complex
topics? Are they achieving deep, authentic
learning? Are they self-directed? Answering
“yes” to these questions would suggest that your
students are engaged learners.
Unfortunately, many teachers are answering
“no” to these questions. Increasingly, schools
are finding that students feel alientated from
their schools, perceiving them as boring or
irrelevant to their lives. As a result, many
students do just enough to get by, while the
most disenfranchised simply drop out. Across
the U.S. the drop out rate is 30%, and in some
U.S. urban centers the drop out rate is reaching
50% or more. These students are not being
prepared to compete and excel in the 21st
Century.
Teachers’ interest in engagement is often driven
by the need to ameliorate low levels of academic
performance, inappropriate classroom
behaviors, and/or high numbers of drop outs. In
fact, engagement is influential in all students’
learning trajectories. Emergent research
identifies student engagement in learning as one
of the most powerful factors affecting
achievement of students at every ability level.
What exactly is student engagement? In his
book, Student Achievement in American
Secondary Schools, Fred Newmann states that
engaged learners make a “…psychological
investment in learning. They try hard to learn
what school offers. They take pride not simply in
earning the formal indicators of success
(grades), but in understanding the material and
incorporating or internalizing it in their lives” (p.
2).
For the purposes of this evaluation report,
student engagement in learning is the active
pursuit of deep learning to accomplish
established standards. Student engagement
comprises three scales: cognitive, behavioral,
and social/emotional engagement, as described
in the sidebar to the right.
Each scale is important in its own right, but the
three are also interdependent. While it is
obvious that the cognitive advances academic
achievement, emergent research also indicates
strong correlations between how emotionally
and socially engaged students are with teachers
and classmates, and how well they do
academically and whether they graduate.
Levels of Student Engagement
The Student Engagement Survey- Part B,
created by Metiri, includes a series of questions
aligned to the cognitive, behavioral, and
social/emotional elements of the definition. This
report, which is generated from your survey
data, provides your school/jurisdictionwith an
overall student engagement score as well as a
score on each of the scales (i.e., cognitive,
behavioral, and social/emotional).
Building off the research of recent engagement
theorists, a taxonomy of student engagement
29
levels was developed to distinguish different
types of engagement as listed below.
· Engaged
· Tactically Engaged
· Compliant
· Withdrawn
· Defiant
· Indeterminate
Using this taxonomy, one would expect an
engaged student to respond positively to “I like
anything I learn about in school.” Similarly, a
withdrawn student would rate a high level of
agreement with the statement “I do not go to
school activities after school. I like to leave
school as soon as I can.” Nine items were
written to reflect each of the five levels of
engagement for a total of 45 items.
Table SE-1 presents a few sample items by
engagement scale. The engagement level of
students was established by locating the level
with the highest mean across the 9 questions
within that category. Students whose responses
were mixed or were <2.5 in all categories were
classified as indeterminate. Table SE-2 shows
how the engagement levels differ in terms of
commitment and attention.
Table SE-1: Examples of statements written for the levels of student engagement
Engaged Tactical Compliant Withdrawn Defiant
After school, I go to school activities so I can be a good student.
I think that kids who do after school activities get better grades.
I always know if I did a good job on my schoolwork. When I don't do as well as I like, I think about what I can change for next t ime.
I keep a list of what I need to do to get a good grade.
Learning in school is important to me because my parents want me to get good grades.
I do just enough work in school to get by.
After I f inish schoolwork, I usually do not check over my work. I'm just glad to be done!
If I do my schoolwork, I do not care about the grades I get.
I do not go to school activit ies after school. I like to leave school as soon as I can.
I would never participate in an extracurricular activity in school.
Most of my schoolwork is junk. No one can make me do it.
High
Attention Low
Attentio n No
Atten tion Hig h
Commitment L ow
Co mmitment No
Commitmen t Diverted Attention
Engaged
Tactical
Compliant
Withdrawn
Defiant
Table SE-2: Attention and Commitment by engagement levels
30
The types of engagement are defined in more
detail below, followed by graphs of the
percentage of students in each level.
Characteristics: Engaged Learners
· Student sees the activity as personally
meaningful.
· The student's level of interest is
sufficiently high that he persists in the
face of difficulty.
· The student finds the task sufficiently
challenging that he believes he will
accomplish something of worth by doing
it.
· The student's emphasis is on optimum
performance and on "getting it right."
Characteristics: Tactically Engaged Learners
· The official reason for the work is not
the reason the student does the work-
-she substitutes her own goals for the
goals of the work.
· The substituted goals are instrumental-
-grades, class rank, college acceptance,
parental approval.
· The focus is on what it takes to get the
desired personal outcome rather than
on the nature of the task itself-
-satisfactions are extrinsic.
· If the task doesn’t promise to meet the
extrinsic goal, the student will abandon
it.
Characteristics: Compliant Students
· The work has no meaning to the student
and is not connected to what does have
meaning.
· There are no substitute goals for the
student.
· The student seeks to avoid either
confrontation or approbation.
· The emphasis is on minimums and exit
requirements: “What do I have to do to
get this over and get out?”
Characteristics: Withdrawn Students
· The student is disengaged from current
classroom activities and goals. The
student is thinking about other things or
is emotionally withdrawn from the
action.
· The student rejects both the official
goals and the official means of
achieving the goals.
· The student feels unable to do what is
being asked, or is uncertain about what
is being asked.
Characteristics: Defiant
· The student is disengaged from current
classroom activities and goals.
· The student is actively engaged in
another agenda.
· The student creates her own means
and her own goals.
· The student’s rebellion is usually seen
in acting out-and often in encouraging
others to rebel.
Figure SE-1: Overall Engagement
Baseline Data
The chart below provides data on engagement in
your schools.
0%
20%
40%
60%
80%
100%
42% 44%
12%
2%
46%
38%
8%4% 2% 3%
Engaged TacticallyEngaged
Compliant Withdrawn Defiant Indeterminate
Elementary Secondary
Elementary N=59; Secondary N=192
31
Figure SE-2: I really enjoy learning so I don’t consider it
“work,” even when it’s challenging.
Intrinsic Engagement
Figure SE-3: I focus on my schoolwork when studying and I
am constantly checking to make sure I understand the
material.
Tactical Engagement
Figure SE-4: Some of the topics I learn about in school
aren't that interesting but I still pay attention so I can get a
good grade.
Figure SE-5: Learning at school is really important to me
because it will help me to get into a good college and find a
good job when I finish school.
The following charts provide response
frequencies for individual questions. These
charts provide detailed information as to how
students responded to individual questions.
Note: If your jurisdiction used both the
elementary and secondary surveys, then the
original question asked of secondary students is
displayed. A similar question was asked of
elementary students, but was simplified to
decrease the reading level to Grade 3.
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
1%4%
9%
29%
57%
10%
22%
68%
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
14%10%
15%
34%
27%
9% 11%
19%
40%
20%
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
2% 3%
19%
44%
32%
2%
10%
18%
45%
25%
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
7%
29%
15%
31%
19%
9%
23% 26%
34%
8%
Elementary Secondary
Elementary N=59; Secondary N=192
32
Figure SE-6: When I complete my assignments I tend not to
go back and check over my work. I'm just glad to be done!
Compliant Engagement
Figure SE-8: I am not interested in the topics I learn about in
school. Usually I think about other things, or daydream.
Withdrawn Engagement
Figure SE-7: I just do enough on each assignment to get the
grades that keep my teachers and parents off my back.
Figure SE-9: I don't pay any attention to the school rules,
and I hope no one pays any attention to me.
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
34% 32%
15% 14%
5%
30% 32%
22%
10%6%
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
44%
25%
17%
10%
3%
31% 34%
18%13%
5%
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
37%
19%24%
12%8%
19%26% 25%
20%
10%
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
19%24% 25%
22%
10%10%
32%
18%
27%
13%
Elementary Secondary
Elementary N=59; Secondary N=192
33
Figure SE-10: I do not complete my assignments and have
no intention of doing so.
Defiant Engagement
Figure SE-11: I don't pay much attention to anything we
study. It's not worth my time.
The set of data on student engagement was
also analyzed to determine the degree to
cognitive, behavioral, and social/emotional
engagement reported by students.
0
1
2
3
4
5
Behavioral Emotional Cognitive
3.9
4.3
3.93.8
4.2
3.7
Elementary Secondary
Elementary N = 59; Secondary N = 192
Figure SE-12: Engagement types
Types of Engagement
Cognitive Engagement
Figure SE-13: I keep a list of exactly what is needed to get a
high grade and mentally check things off as I complete them.
Figure SE-14: I do not plan time for studying or completing
assignments.
Again, frequency responses from individual
questions are provided as additional insight into
each of these three areas.
0%
20%
40%
60%
80%
100%
49%
31%
10% 8%
2%
31% 29%
18%13% 10%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
20% 19% 20% 22%19%17%
27%22%
26%
8%
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
75%
17%
5% 3%
61%
20%
13%
3% 2%
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
76%
10% 10%
2% 2%
65%
15%12%
7%
1%
Elementary Secondary
Elementary N=59; Secondary N=192
34
Figure SE-15: I spend enough time in this school just doing
the basics. I do not participate in extra-
curricular activities that are not required, unless my parents
really push me.
Figure SE-16: I don't really think about following rules and
acting positively. It's just the way I am.
Behavioral Engagement
Social/Emotional Engagement
Figure SE-17: It's important to me that the work I do
represents my best effort and helps me grow.
Figure SE-18: School isn't all important to me. I am not
interested in what we learn.
The Emerge Research Team recommends that
the schools review this data set and set targets
for improvement related to the percentage of
students in the engaged categories. The detail in
the charts above should provide insights as the
type of strategies that will be required to meet
those targets.
Bangert-Drowns, R. L. (2001). A taxonomy of student
engagement with educational software: An
exploration of literate thinking with electronic
text. Journal of Educational Computing
Research, 24(3), 213-234.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H.
(2004). School engagement: Potential of the
concept, state of the evidence. Review of
Educational Research, 74(1), 59-109.
Schlechty, P. (2002). Working on the work. San
Francisco: Jossey-Bass.
Sources
0%
20%
40%
60%
80%
100%
81%
7% 5%2%
5%
70%
14%10%
5%1%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
2% 3%
18%
38% 39%
2%
10%
32%
56%
Secondary Elementary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
46%
32%
15%
3% 3%
40%
28%
14% 11%7%
Elementary Secondary
Elementary N=59; Secondary N=192
0%
20%
40%
60%
80%
100%
Not at alllike me
Not reallylike me
Neither likeor not like
me
A little likeme
A lot like me
36%
19%
36%
7%3%
30% 30%
21%
12%7%
Elementary Secondary
Elementary N=59; Secondary N=192
35
Classroom Structures That Engage Students (CSES)
Some learning environments are more effective
than others in scaffolding deep learning in
students. Based on emergent research (see
reference list at end of section), The Emerge
Research Team identified those classroom
structures and surveyed students to determine
which of those structures were evident in the
classrooms they attended.
Students were asked to share their perspectives
on whether the classrooms they attend offer
choice, structure, diversity, collaborative
environment, and clarity. In addition, students
were asked whether the classroom structures
interface with content draws on their prior
knowledge, offers them opportunities to
collaborate with colleagues, and provides them
with avenues for fully demonstrating their
acquired knowledge and skills.
Emergent research on highly qualified teachers,
highly effective teaching, differentiation of
instruction, engagement, and response to
intervention all emphasize how the right
classroom structures can engage students in
deep learning (Marzano, 2007). The classroom
that introduces content in ways that trigger
student interest, challenge students to think, cue
prior knowledge, provide relevancy, afford
choice, offer clear standards, formalize
collaboration, and ensure intellectual safety,
lead to deep learning by children and
adolescents.
Background
Scales
Three major elements have emerged from the
research on the effectiveness of classroom
structures in advancing deep learning through
student engagement related to Content,
Process, and Product.
Content refers to student opportunities to
engage with academic subjects in ways that
result in deep understanding of concepts,
principles, and context. Process refers to the
learning activities through which the student is
able to make sense of, or master, the content.
The element of Product engagement refers to
the structures that enable students to rehearse,
apply, extend, and demonstrate what he/she
learned through an assignment.
The Engagement Survey- Part A, created by
Metiri, was administered to gauge student
perspectives on the existence of and quality of
classroom structures that could lead to their
engagement in learning. The survey was guided
by the work of the researchers in the reference
list. The survey asked students to rate 50
statements on classroom structures, i.e., 15
items on Product, 15 items on Process, and 20
items on Content, using a 5-point scale
anchored with 1=Completely False to
5=Completely True, with 3 as a neutral
mid-point.Figure CE-1: Overall classroom structures the engage
students (CSES) score (scale 1-5)
0
1
2
3
4
5
Elementary Secondary
3.4 3.4
Elementary N = 64; Secondary N = 199
NOTE: The Overall Score on the Classroom
Structures That Engage Students (CSES)
reflects the average of the Product, Process,
and Content scales.
36
Figure CE-2: Product, Process, Content and Overall Engagement scores by level
0
1
2
3
4
5
Product Process Content Total ClassroomEngagement Score
3.9
3.2 3.13.4
3.8
3.2 3.33.4
Elementary Secondary
Elementary N = 64; Secondary N = 199
Student Rating Cut Points Score Level Definition
1. Completely False
1.0 <= x <= 1.9 No classroom structures that engage students
2. A Little False 2.0<= x <= 2.4 Moderately low number/quality of classroom structures that engage students
3. Neither True nor False 2.5.<= x <= 3.4 Neutral
4. A Little True 3.5<= x <= 3.9 Moderately high number/quality of classroom structures that engage students
5. Completely True 4.0 <= x <= 5.0 High number/quality of structures that engage students
Table CE-1. The CSES Survey Cut Points and Definitions
Higher scores reflect higher evidence/quality of
classroom structures that engage students in
learning, whereas low scores reflect less
evidence/quality of such classroom structures.
Scores greater than or equal to 3.5 imply that
students perceive high to moderate evidence of
classroom structures to engage them in
learning. Scores lower than 3.5 indicate that
students reported either low or moderately low
evidence/quality of classroom structures that
engage them (See Table above.) The mean
scores by school level are presented above,
followed by a more detailed analysis of each
scale.
Prior to calculation of factor scale scores, the
student responses to reversed or negatively
worded items were adjusted appropriately.
Cronbach’s alpha internal consistency
reliabilities for the Product, Process, and
Content scales were 0.82, 0.58, and 0.83,
respectively. The alpha for combined scales was
0.903, indicating high internal consistency for
the Classroom Engagement Survey.
37
In order to provide more detailed insight into
these scores, frequencies are reported below for
sample questions from each of the scales.
Figure CE-4. The work that I do in my classes seems like the
same kind of work that I might do for a job one day.
Product Scale
Product refers to the projects that require the
student to rehearse, apply, and extend what he
or she has learned in a unit. Some examples of
how teachers promote product engagement
include giving students’ a choice of how to
express required learning (e.g., create a movie
or interview an expert) or encouraging students
to create their own product assignments as long
as the assignments contain specific elements.
Another method to increase product
engagement includes using rubrics that align
with and extend students’ varied skills levels.
Therefore, a high score on the Product scale
would indicate that students value class
assignments and perceive them to be
meaningful, they understand the standards by
which the product will be assessed, and there is
value to the product beyond the classroom.
A Product scale was computed as the mean of
15 items measured using the five-point
Completely false-Completely true Likert-type
response scale. Two of the items included in the
Product scale are presented below by school
level.
Sample Product Items
Figure CE-3. My teachers tell us in advance exactly how a
product, project, or other assignment will be graded.
Process Scale
Process refers to the activities in which the
student engages in order to make sense of or
master the content. A high score on this scale
would indicate that students perceive the
classroom as an environment in which they can
learn through intellectual risk taking without fear
of ridicule, they can work interactively and
interdependently with others, and they perceive
those activities and tasks to be meaningful, as
do persons of importance to them.
The following graphs present two of the
individual items in the Process scale by
academic level.
Sample Process Items
Figure CE-5. In my classes, my teacher likes us to have
other students read or view our work.
0%
20%
40%
60%
80%
100%
3%
17% 19%
52%
9%10%6%
18%
42%
25%
Completelyfalse
A little false Neither trueor false
A little true Completelytrue
Elementary Secondary
Elementary N=64; Secondary N=180
0%
20%
40%
60%
80%
100%
19%16%
36%
25%
5%11%
22%28% 28%
11%
Completelyfalse
A little false Neither trueor false
A little true Completelytrue
Elementary Secondary
Elementary N=64; Secondary N=180
0%
20%
40%
60%
80%
100%
3% 5%9%
22%
61%
3% 6%10%
31%
51%
Completelyfalse
A little false Neither trueor false
A little true Completelytrue
Elementary Secondary
Elementary N=64; Secondary N=199
38
Figure CE-6. Often my teachers give assignments
that allow us to work together in groups.Using the Results Formatively
Schools interested in improving their CSES
scores should review their data, identify gaps
between their current scores and their goals,
identify areas for improvement, and then
develop and implement a strategic improvement
plan based on research.
Sources
Appleton, J. J., Christenson, S. L., Kim, D., &
Reschly, A. L. (2006). Measuring cognitive
and psychological engagement: Validation of
the student engagement instrument. Journal
of School Psychology, 44, 427-445.
Bangert-Drowns, R. L. (2001). A taxonomy of student
engagement with educational software: An
exploration of literate thinking with electronic
text. Journal of Educational Computing
Research, 24(3), 213-234.
Dweck, C. S. (2006). Mindset. New York: Random
House.
Dweck, C. S., & Leggett, E. L. (1988). A
social-cognitive approach to motivation and
personality. Psychological Review, 95(2),
256-273.
Fredricks, J., Blumenfeld, P., Friedel, J., & Paris, A.
(2003, March). School engagement. Paper
presented at the Indicators of Positive
Development Conference, Washington, DC.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H.
(2004). School engagement: Potential of the
concept, state of the evidence. Review of
Educational Research, 74(1), 59-109.
Hoek, D., Terwel, J., & Eeden, P. (1997). Effects of
training in the use of social and cognitive
strategies: An intervention study in
secondary mathematics in co-operative
groups. Educational Research and
Evaluation, 3(4), 364-389.
Jimerson, S., Campos, E., & Greif, J. (2003). Toward
an understanding of definitions and
measures of school engagement and related
terms. The California School Psychologist, 8,
7-27.
Marzano, R. (2007). The art and science of teaching:
A comprehensive framework for effective
instruction. Alexandria, VA: ASCD.
Newmann, F. (1992). Student engagement and
achievement in American secondary
schools. New York: Teachers College Press.
Schlechty, P. (2002). Working on the work. San
Content Scale
Content refers to the material being presented,
or what the student needs to learn such as
concepts, principles, and skills. To score high on
the Content scale indicates that students find
the subject matter interesting and perceive it to
be relevant, important, and attainable. To
measure content engagement, students
responded to several statements. The following
charts present the results of two of the individual
items in the Content scale.
Figure CE-7. I get to be creative in my classes.
Sample Content Items
Figure CE-8. My teachers make uninteresting
subjects exciting to learn about.
0%
20%
40%
60%
80%
100%
Completelyfalse
A little false Neither true orfalse
A little true Completelytrue
14%8%
22%
33%
23%
9%14%
24%
31%
21%
Elementary Secondary
Elementary N=64; Secondary N=181
0%
20%
40%
60%
80%
100%
Completelyfalse
A little false Neither true orfalse
A little true Completelytrue
6%
17%
28% 30%
19%
4%8% 11%
37% 39%
Elementary Secondary
Elementary N=64; Secondary N=180
0%
20%
40%
60%
80%
100%
Completelyfalse
A little false Neither true orfalse
A little true Completelytrue
6%
23%28%
31%
11%9%14% 15%
33% 30%
Elementary Secondary
Elementary N=64; Secondary N=199
39