edtech impact it operations effectieness data …ed to using digital tools in class. (student): •...

Technology Impact and Efficacy (TIE) AssessmentReport Prepared for ATLIS School

EDTECH IMPACT

IT OPERATIONS EFFECTIVENESS

DATA SECURITY

The TIE Assessment Score represents ATLIS School’s Overall Rating relative to other schools taking the assessment. Based on the responses from your school, the scores below represent how ATLIS School compares to other schools:

R E P O R T S U M M A R Y

TECHNOLOGY IMPACT AND EFFICACY SCORE

EDTECH IMPACT COMPARISON:

IT OPERATIONS COMPARISON:

DATA SECURITY COMPARISON:

A B O U T T H I S A S S E S S M E N T

The Technology Impact and Efficacy (TIE) Assessment has two goals. The first goal is to provide schools with benchmark scores for EdTech Impact, IT Operations Effectiveness, and Data Security relative to similar schools. In this report, schools are compared to other ATLIS member schools as a member benefit and to provide con-text for the benchmark. The second goal is to provide schools with specific actions, based on predictive analytics, to improve their performance in each benchmark area. The top portion of the report pro-vides reflective information on the school relative to similar schools. The bottom portion of the report provides information specific to your school and, based on the data, reports which action areas have the greatest impact on raising your scores in each benchmark area and which specific action areas your school can address in order to have the greatest impact on improving benchmark scores.

Your school falls in Quartile




3

3

1

3

Four surveys were developed and tested by a team of experienced ed-ucators and researchers to gather information for the TIE Assessment. The Admin Survey is used to gather context variables, such as IT inven-tory, staffing, budget, and policy information. Academic administrators provide standardized test score deltas, attendance information, and weigh the alignment of various educational standards to their mission and program. The Faculty and Staff Survey gathers information on use and satisfaction with technology. Skip logic is used to provide teach-ers with classroom-related items in the survey. The Student survey gathers information on use and satisfaction with technology and is for students age 13 and up. The grade break for this survey is determined by the school. The Young Student Survey is for students under 13 and is facilitated by a teacher to ensure questions are commonly understood. Items in the survey were used to create index scores, which were then tested for reliability.

EdTech Impact benchmark scores incorporate each of the areas in the ISTE Student Standards as well as Academic Performance on Standard-ized Tests. Schools then weigh the importance of each standard on the Admin survey. Each index score is weighted by the school’s response. Thus, the EdTech Impact benchmark score is reflective of how well a school performs in EdTech relative to the purposes that school seeks with educational technology.

The IT Operations Effectiveness benchmark score measures the community’s satisfaction with the quality and speed of service and the

quality and amount of technology provided. All other IT-related items are measured, analyzed, and reported on to show the impact they have on the school community’s IT effectiveness.

The Data Security benchmark score evaluates the implementation of best practices across security infrastructure, data security policies, and the community’s awareness and participation in security best practices. Index scores were created for each of those areas, and recommenda-tions are based on your school’s variance from the mean.

Educational Collaborators used MondoBrain Augmented Intelligence to perform rank correlation and interactive rule solving to evaluate the strength of each variable’s individual contribution to the overall value of each outcome benchmark. To determine the specific areas of relative strength for each school, we analyzed individual school data to deter-mine the behaviors that have had the greatest impact on improving the benchmark scores in each area. To determine the specific areas for targeted improvement for each school, we analyzed individual school data to determine the behaviors that would have the highest impact on improving the benchmark scores in each area.

M E T H O D O L O G Y

Given the assessed strengths and weaknesses of ATLIS School, we have developed a set of relative strengths and areas for targeted improvement. Below each benchmark heading, you will find the variable that is driving the area of strength or targeted improvement in your community. The stated variable alone will not be responsible for moving the needle on its associated benchmark. However, the designated variable was selected due to its correlation with other variables and statistical likelihood to have a positive impact on the benchmark score within your community.EDTECH IMPACT AREAS OF RELATIVE STRENGTH

FOR ATLIS SCHOOL:

Teachers and students discuss appropriate use and behavior relat-ed to using digital tools in class. (student):

• Students in your community report discussing appropriate use of digital tools with teachers and other students. These discussions are having a positive and significant effect on your EdTech Impact benchmark score. Schools that report discussing appropriate use and behavior are likely to report strengths in a variety of areas, an indication that this strength comes from relatively broad use of digital tools in the academic program. It is also an indicator that the community is open to a more growth oriented mindset. Well done!

I believe using technology with my school work improves my learning. (student):

• Students at your school believe that using technology with their school work improves their learning. This belief has a positive impact on your school community’s EdTech Impact benchmark score. Students with positive attitudes towards technology feel empowered to learn (ISTE Standards for Students). This strength

also correlates positively with students’ comfort with technology in the classroom, students’ feelings about their own attentiveness and engagement when using technology to learn, and students’ desire to use technology as a learning tool.

S P E C I F I C S U G G E S T I O N SF O R E D T E C H

E D T E C H

EDTECH IMPACT AREAS FOR TARGETED IMPROVEMENT FOR ATLIS SCHOOL:

My students explore local and global issues as part of my course. (faculty): • The school needs to expand opportunities to explore local and global

issues within the curriculum. When students’ classes engage in these opportunities, the experiences broaden students’ perspectives, especially when accompanied by collaboration with others outside the classroom (ISTE Standard 7 - Global Collaborator). As the world becomes more interconnected, students will need an understanding of how to interact with other people and cultures as well as how to work effectively as a team member toward a common goal(ISTE 7c) through the use collaborative technology (ISTE Standard 7d). Digital tools with collaboration features such as G Suite or Office 365, open the classroom doors for students to learn how to work and collabo-rate effectively with others (peers, experts, community members as well as people from other cultures) (ISTE 7b).

Professional development focused on global collaboration is recom-mended. Professional development sessions should not only provide teachers with ideas but also time to explore possibilities and develop a lesson/project appropriate for their classroom. In addition, leaders should offer follow up support to teachers using the collaborative technology the first few times. Many sites offer lesson plans and projects to help teachers raise awareness of local and global issues. Consider having teachers explore the ISTE site, 100 People: Global Issues Through our Lens (middle and high schools curriculum), PBS Learning, and/or U.S. Department of Education for ideas. The imple-mentation of global collaboration requires a skill set that both teach-ers and students will need to develop. This should not be a single PD session but a commitment to ongoing development and support.

S P E C I F I C S U G G E S T I O N SF O R E D T E C H

I believe I learn better by interacting with other students outside of our class. (student):

• Students indicate they do not see a value with interacting with other students outside of their classes. Seek to help students and teachers understand the value of collaborating with other students outside of the class. Work in this area is aligned with the ISTE Student Standard 7 - Global Collaborator. Improvement in this area will likely yield a significant increase in your school community’s EdTech Impact bench-mark score. Students who understand the value of collaborating with other students outside of class report engaging in more of this activity and this is correlated with more technology use and better technolo-gy literacy. Collaboration skills are crucial and collaborating with other outside of the class increases perspective and helps students devel-op problem solving skills through group work. Consider finding other classes to work with through the Global Education Conference, a free online conference with a large community of educators. Schools can find resources, projects and other teachers through Microsoft’s Skype in the Classroom. Though focused on science, excellent scholarship, including specific recommendations can be found in the article, “Collaboration, interdisciplinary thinking, and communication: new approaches to K–12 ecology education” Frontiers in Ecology and the Environment. (2015) http://onlinelibrary.wiley.com/doi/10.1890/140130/full.

Given the assessed strengths and weaknesses of ATLIS School, we have developed a set of relative strengths and areas for targeted improvement. Below each benchmark heading, you will find the variable that is driving the area of strength or targeted improvement in your community. The stated variable alone will not be responsible for moving the needle on its associated benchmark. However, the designated variable was selected due to its correlation with other variables and statistical likelihood to have a positive impact on the benchmark score within your community.

S P E C I F I C S U G G E S T I O N SF O R I T O P E R A T I O N S

IT OPERATIONS IMPACT AREAS OF RELATIVE STRENGTH FOR ATLIS SCHOOL:

Quality of service (faculty):

• Faculty at your school are satisfied with the quality of services offered by your IT staff. This plays a significant role in improving your IT Operations benchmark score. Most schools with strong quality of service scores also see teachers with higher confidence using technology in the classroom and greater quality integration of technology tools. Thus, your focus on quality service is likely im-proving the overall return on investment for technology spending.

Amount of technology resources (student):

• Students believe they are provided the appropriate amount of technology resources. This factor is having a significant impact on your IT Operations benchmark score. Satisfaction with the amount of technology likely also affects the quality integration of technol-ogy in the classroom, since those practices are strongly correlated with satisfaction with the amount of technology. Thus, by finding the right balance of providing the community with the technology they need, which your school is doing well, you are also maximiz-ing the return on investment in that technology.


I T O P E R A T I O N S

S P E C I F I C S U G G E S T I O N SF O R I T O P E R A T I O N S

IT OPERATIONS IMPACT AREAS FOR TARGETED IMPROVEMENT FOR ATLIS SCHOOL:

Wireless network (faculty):

• Seek to improve the campus wireless network for faculty. Strong performance of the wireless network is a leading indicator of perceived quality. Thus, improvement in this area will have a sig-nificant and large impact on your IT Operations benchmark score. With improvement in the performance of the network, you are likely to see increased use of technology in the classroom. Typical bottlenecks that cause low performance in this area are limited bandwidth or bandwidth shaping problems. Be sure you have adequate bandwidth. Education Superhighway recommends that a school have 1Mbps/student by 2018. Sometimes, the problem is not the bandwidth, but rather how that bandwidth is used. Cer-tain applications, such as video, can monopolize too much band-width. Other times, IT policies may be excessively limiting certain resources, causing poor performance. Review your bandwidth shaping or Quality of Service (QOS) settings to see if this may be the problem. Contact your Internet Service Provider for help with this if needed. Consider reading “Calculating Your School District’s Bandwidth Need: Network Essentials for Superintendents,” Educa-tion Superhighway, 10 February 2015, https://www.educationsuper-highway.org/calculating-your-school-districts-bandwidth-need-net-work-essentials-for- superintendents/.

Amount of technology resources (faculty):

• Seek to increase the amount of technology you provide for faculty. Improving the amount of the appropriate technology for faculty will significantly improve your IT Operations benchmark score. It will likely also improve your EdTech Impact benchmark score since many of the quality technology integration practices are strongly correlated with members of the community believing they are provided the appropriate amount of technology. When selecting technology for your community, be sure to begin by looking at the academic and administrative programs. Look for desired outcomes to select the software and hardware products that best facilitate those outcomes. Think about where members of communities work to select devices that facilitate actual practice. If students are expected to do work at home, make sure they have the resources to accomplish this. For more information on the matching technol-ogy with programmatic goals, consider reading “Using Technology to Support At-Risk Students’ Learning,” SCOPE and Alliance for Ex-cellent Education, https://edpolicy.stanford.edu/sites/defaulUfiles/scope-pubusing-technology-report.pdf, September 2014.


S P E C I F I C S U G G E S T I O N SF O R D A T A S E C U R I T Y

DATA SECURITY IMPACT AREAS OF RELATIVE STRENGTH FOR ATLIS SCHOOL:

Security Infrastructure:

• Congratulations! Your school’s infrastructure is supporting campus security. A well-built security infrastructure means that all users have timely access to accurate information and resources neces-sary for their role and yet the same information and resources are secure from unauthorized access. Information Security Manage-ment (ISM) is an ever changing responsibly that requires on-going risk assessment and active engagement in keeping abreast of de-velopments in the field on the part of technology leaders. Schools should engage in regular review of risks and develop contingency plans for breaches or violations of security policies. Resources for school leaders include: The US Department of Education https://studentprivacy.ed.gov which offers checklists, glossaries, and breach response toolkits. Members of ATLIS www.theatlis.org or CoSN http://www.cosn.org will find that both organizations provide various rubrics, templates, and samples that may be helpful.


D A T A S E C U R I T Y

S P E C I F I C S U G G E S T I O N SF O R D A T A S E C U R I T Y

DATA SECURITY AREAS FOR TARGETED IMPROVEMENT FOR ATLIS SCHOOL:

Security Communication and Awareness:

• Your school needs to improve communication about security procedures and policies with campus constituents. Each user on campus needs to understand their role in protecting the school’s data and resources. If the school does not have a security pol-icy in place, one should be written before taking other steps. A well-designed security policy can be included in other policies or provided as a separate document. The policy makes it clear to the school community that the school is committed to providing all us-ers timely access to accurate information and resources necessary for their role and ensure that the same information and resources are secure from unauthorized access. A written policy will spell out: access control for resources and data (who is allowed to use what, and for what purposes), anti-virus and malware resources and the responsibilities of the campus constituent groups re-garding them, appropriate use and misuse of IT assets, reporting policies for users and IT staff, and any other aspects of security deemed important. Schools should engage in regular review of risks and develop contingency plans for breaches or violations of security policies. If the school has these policies in place, they need to be shared and discussed more-widely on campus. Teach-ing students to manage their digital resources securely is part of the ISTE Standard for Digital Citizenship. In addition, ISTE teacher and administrator standards call on campus adults to demonstrate leadership in modeling citizenship best practices for students. In-formation Security Management (ISM) is an ever changing respon-

sibly that requires on-going risk assessment and active engage-ment in keeping abreast of developments in the field on the part of technology leaders. Schools should engage in regular review of risks and develop contingency plans for breaches or violations of security policies. The US Department of Education Protecting Student Privacy website provides many resources detailing best practices https://studentprivacy.ed.gov/ . Providence Day School (NC) has developed and published an entire Digital Citizenship Guide, available for use: https://pdsdigitalcitizenship.org.


This report is offered as a member benefit to ATLIS member schools. The TIE report can be purchased on the open market by non-member schools for $1500. A higher degree of human analysis is available for those schools interested in purchasing a consultation package. Schools interested in further analysis can contact Educational Collaborators (EC) at [email protected]. EC can provide personalized, interactive, consultations starting at $500.

Educational Collaborators will review the data provided by the client school in advance of the meeting to better understand the findings of the TIE Assessment and potentially uncover items not found through the automated report. EC will then host a video conference meeting with the client to review the findings. During the scheduled one-hour virtual meeting, the client will be provided:

• A summary of any additional findings• Summary report of all three surveys for your school

• Table of all Strengths and Areas of Targets Improvement for all EdTech Impact and IT Operations variables

• Table of correlations for all TIE variables• The tools and process for building powerful data driven

strategies through the development of a sample strategy based on one of the client’s specific recommendations

• The hour will also include Q & A throughout the meeting.

Schools may also acquire their own interactive MondoBrain portal with capacity to build multiple scenarios for further analysis. This additional service offers an interactive analytic portal with all data from your school, as well as anonymized ATLIS member data for each item of the assessment. This service provides a one-year license to the MondoBrain software, hosting, and anonymized data, and is available for $12,000. When schools purchase their own school data portal, they will receive 10 hours of training and consulting to help them maximize their investment. Contact Educational Collaborators for a demonstration of the options available.

D I G D E E P E R I N T O T H E D A T A

“Having a set of unbiased eyes look over the data and find trends that we missed was invaluable. The themes that were discovered and discussed are really going to help us guide larger conversations about the intersection of pedagogy and practice. On the operations side, low hanging fruit came to light that will improve everyone’s experience with technology,”

Peter Antupit, Chief Information Officer, Crystal Springs Uplands School.

edtech impact it operations effectieness data …ed to using digital tools in class. (student): •...

Documents