interview of tech yes teacher - university of west...

Evaluation of Tech Yes Science

Prepared by:

Kim Mulkey

Prepared for:

Woodward Academy Lower School

April 2010

Evaluation of Tech Yes Science 2

Executive Summary

Woodward Academy’s Lower School is in the process of piloting the Tech Yes Science

program in four of their sixth grade science classes. The evaluator, Kim Mulkey, used participant

observation, an interview, a Likert survey, and a collection of tallies to collect data for the

purpose of answering the following questions. How has the Tech Yes Program increased the

student’s use of a variety of technology in their learning process? What is the value of using

student mentors to aid each other in the learning process? Were the students motivated to achieve

the learning objectives of the assignment?

The program is a type of Project Based Learning (PBL) where students create a

presentation using information from an inquiry style of learning. They begin with a curricular

topic but do not have specific content objectives. PBL requires the use of higher levels of critical

thinking, while allowing for differentiated learning. The projects include real world problem

solving which allow students to see how science is truly implemented outside of a traditional

school setting.

The process of the project included students learning about ecology, creating an inquiry

question to investigate, conducting an experiment or creating a model, analyzing their data,

drawing conclusions, and creating technology enriched presentation.Sstudents shared their

project with the world by uploading it to the class Wiki (http://wafields.wikispaces.com). All of

the projects were completed at school without assistance from parents.

After analyzing the data, the key findings were positive. Students’ use of various

technologies was high. The mean use was 5.12 technologies per student per project. Mentors

were extremely valuable and are needed for the success of the program. The motivation findings

http://wafields.wikispaces.com/


were positive but had a surprising facet. The average and below average ability students

demonstrated a higher level of motivation and performed the tasks better than the higher ability

group.

The recommendations for improvements to the Tech Yes Science program would

include several suggestions. There needs to be more scaffolding for the students, including a

notebook with organizational aids, a journal, and helpful tips. File names need to be addressed

continually. The mentors need time to work on their projects without interruption. They also

need a more detailed evaluation form.

In conclusion, the Tech Yes Science program has great potential for students. The

increased use of technology, high motivation of the students, and spirit of helpfulness from the

students demonstrated qualities of a successful program. The PBL aspect allowed students to

learn independently but included the component of peer helpers.

Even though this evaluation only addressed the aforementioned questions, there are many

more aspects that were beneficial to students. Their ability to use critical thinking skills in the

development of all aspects of the project was encouraging. Since finishing the evaluation process

of this program, the evaluator has observed other classes not participating in the Tech Yes

Science program. The technology skills of the Tech Yes students are far superior while using the

resources in the lab.

It is the recommendation of this evaluator for the continuance of the Tech Yes program at

the Lower School, as well as the expansion to the remainder of the sixth grade science classes.

There is also a Tech Yes program for other content areas; therefore, the remaining curricular

areas could also benefit from this PBL style of learning. This program should not be the sole


means of learning, and traditional teaching needs to continue to be in the classroom; however,

the skills students learn by using this method are equally as valuable.


Evaluation of Tech Yes Science

Introduction

Science is an ideal discipline for inquiry learning. At Woodward Academy’s Lower

School in College Park, Georgia, fifth and sixth grade students participate in inquiry learning

projects. Mrs. Field’s sixth grade science classes have been piloting a program called Tech Yes.

This program uses existing science curriculum and infuses it with technology to create project

presentations. The program process also includes training student mentors to help other students

in their learning process.

Project Based Learning (PBL) is the basis of this program. PBL is theoretically

considered an instructional strategy of the constructivist theory. “ A core assumption of the

constructivist theory is that learners construct knowledge through activity, and the goal of the

learning experiences designed by teachers is to promote a deep understanding rather than a

superficial (and short lived) memorization” (Ramos and Paz, 2009). PBL is student centered

learning where the teacher does not give formal instruction, but becomes a facilitator. “Using

educational technology as a constructivist tool could aid students to represent their ideas,

articulate what they know, and explore, manipulate, and process information, while actively

collaborating with each other” (Juniu, 2006, p. 67)

There is sufficient research to suggest positive effects of PBL. The outcomes of PBL

include greater understanding of a topic and the processes of project creation. Higher reading

levels, engaged learners, and the ability to differentiate assignments for each learner are

additional results of PBL. Students also learn collaboration, real world problem solving, and

increased motivation. “Shepherd (1998) found that students who applied the PBL approach


received statistically higher grades on a Critical Thinking Test than did those in the comparative

group who had studied in a traditional manner (as cited in Frank & Barzilai, 2004, p.43).

Assessment of PBL is different than traditional methods of instruction. Evaluation must

be made in a formative and summative manner using rubrics and self evaluation. Many of the

skills learned in PBL cannot be measured on a traditional test. “In the future, children must enter

a workforce in which they will be judged on their performance. By implementing PBL, we are

preparing our students to meet the twenty-first century with preparedness and with a repertoire of

skills they can use successfully” (Bell, 1010, p. 43). This type of learning helps students build

problem solving skills and learn that it may take more than one attempt to accomplish a goal.

Mr. Andy Phillips, the principal of the Lower School, has requested evaluation of the

Tech Yes program to see the increase in technology use, results of using student mentors, and

effects on student motivation. Kim Mulkey evaluated this program in order to address his

questions and give suggestions for improvements of the program in the future. The stakeholders

(sixth grade students) will continue the program by participating in a second project after the

completion of this evaluation.

Purpose

The purpose of the evaluation is to determine the use of technology, success of mentors,

and the effect on motivation of the students engaged in the Tech Yes Science program at

Woodward Academy’s Lower School. The evaluation is both formative and summative. The

students will do another project beginning in April, so the recommendations from the evaluation

will be considered to make improvements. If this program is successful, the likelihood of

furthering the program in the other science classes will increase. This possibility will increase the


size of the stake holders from four classes to nine sixth grade classes. The evaluation will be

presented to the principal of the Lower School, as well as at an academy wide principals’

meeting.

Evaluation Questions

The objectives of this evaluation are specific to the Tech Yes program used by

four classes in the sixth grade. The evaluation questions include the following:

How has the Tech Yes Program increased the student’s use of a variety of technology in

their learning process?

What is the value of using student mentors to aid each other in the learning process?

Were the students motivated to achieve the learning objectives of the assignment?

Methods

The evaluation included the observation of 761 sixth grade students participating in the

pilot program for Tech Yes Science. The data collected for tallies of technology use included

742 students. There were four classes that were academic ability grouped by: class one was above

average, classes two was considered average, class three just below average, and class four was

considered the lowest ability class. The average age of the students is twelve years old. At the

beginning of the school year, 16 students participated in a mentor program to learn how to use

the technologies and how to mentor students in a positive manner.

1 One new student enrolled in January after the projects were underway. The student did not participate in the construction of a project. She shadowed another student to learn the process and then evaluated five completed projects.

2 Two students had not completed the assignment at the time of data collection. One student was absent for more than one week during construction.


Participant observation of the daily program was a portion of the data collected. This

qualitative data included a collection of student responses, questions, problems, changes, and any

other observation relevant to the pertinent evaluation questions.

The interview with the teacher included questions relevant to student learning in

comparison to student learning in the past. This teacher has always used inquiry learning, so the

questions asked for a comparison with the use of technology in the process and presentations.

The interview also asked for specific ways the mentors were used and their effectiveness.

Student motivation was also a focus of the interview. A Likert survey was used with questions

pertaining to the evaluation questions in order to gain more data. The Likert survey was not used

in the traditional method of coding, due to the fact that one teacher answered the survey. It was

included in the methods for future studies that include more than one teacher.

Each student was required to evaluate their project using a Tech Yes evaluation form.

They included the technologies they used to collect data, organize, create, and share their project.

The evaluator used the students’ forms, observations, and viewing of each project to tally the

number of technologies used by each student.

After each student finished their own self evaluation, a peer mentor evaluated the project

using the same form. The student mentor then offered suggestions for improvement. The

students made corrections and then submitted their project by uploading it to a wiki. They turned

their form into the teacher, so she could do the final evaluation. The final piece of data collected

was a tally of each mentor’s number of evaluations.


Questions Instrument

How has the Tech Yes Program increased the student’s use of a variety of technology in their learning process?

Tally from observation, student forms, and viewing of the projects.

Interview


Interview

Likert Survey

Observation

Tally


Interview

Likert Survey

Observation

The process of the program began with a unit on ecology. This was a traditional method

of teaching and assessment and was not observed by the evaluator. After the completion of the

unit, students used Inspiration (graphic organizer computer software program) to brainstorm a

topic and organize their thoughts. Each student then researched books or websites to look for

possible experiments on their topic. A question was then formulated for the basis of their project.

They used the Tech Yes book to develop supply lists and the process for the experiment or

model.

Next came a ten day experiment or model period. Students either built a model or

conducted an experiment to aid in the data collection for the answers to their inquiry question.

During this time they collected data for the project presentation. Even though reminded daily, a

few students failed to collect appropriate data.


The next phase began in the computer lab. Students began organizing their data in

various ways. They used whatever made sense for their project. The project creation phase

overlapped with the organization, and the technology presentation segment was under way. As

students finished their projects, the evaluation and presentation phase began. Students first did a

self evaluation, and then proceeded to be evaluated by a mentor. The student made

recommended changes and then uploaded their project to the class Wiki. The project was then

assessed by Mrs. Fields using two rubrics (Ecology and Wiki Post). Students were assigned a

section of the projects to view, but were not limited to that selection. They voted on their

favorites and an “Oscars” type party was held to view the winners.

Summary of Key Findings

After analyzing the data, the evaluator found the following results:

How has the Tech Yes Program increased the student’s use of a variety of technology in

their learning process?

From the beginning of the unit through the data collection phase, there were many uses of

technology. The evaluator was able to document 181 uses of technology for this portion of the

project. The evaluator only included the items that were actually documented. All of the students

used Inspiration to begin their brainstorming of topics. Use of digital cameras, Flip video

recorders, and Vernier Probes were technologies used the most for data collection besides

Inspiration. Many of the students did not list web resources as something they used, even though

they did


Camera

Flip Video

Word

PowerPoint

Excel

Movie M

aker

Photostory3

Pixie2

Web Blen

der

Microsco

pe

Smart

Capture

Comic Life

Vernier

Probes

Websit

e

Audacity

Inspiration

Wiki0

20406080

47

212 0 7 0 0 0 0 2 1 0

14 130

74

Technolgy Used in Data Collection181 Uses

The data collected during the organization and creation phase of the program included the

information in the graphs below. Excel, Movie Maker, and PhotoStory3 were the top uses of

organization. Pixie 2, Movie Maker, and PhotoStory3 were the top choices for the presentation

creation. Even though some of the technologies were used in both the organization and creation

the numbers are higher than the number of students participating in the program, so many

students used multiple technologies.

05

1015202530

0 04 5

2622

18 16

1 0 0 14

0 0

Technology Used in Organization97 Uses


01020304050

17 4

2519

43

2 2 5 1

Technology Used in Creation109 Uses

The total number of technologies documented during this evaluation was 379 uses. All

students uploaded their presentation to the class wiki at http://wafields.wikispaces.com.

Excluding the Wiki and Inspiration, the most used technologies were a digital camera, Pixie 2,

and Excel. The mean technology use for the projects was 5.12 uses per student. The highest use

per student was eight technologies and the low was three. The majority of the students (67)

created a video as their final format on the wiki.

Camera

Flip Video

Word

PowerPoint

Excel

Movie M

aker

Photostory3

Pixie2

Web Blen

der

Microsco

pe

Smart

Capture

Comic Life

Vernier

Probes

Websit

e

Audacity

Inspiration

Wiki0

204060

80

46

215 8

26 25 19

43

2 2 1 212 18

1

74 74

Total Technology74 Students Used 379 Technologies

Mean Technology Use = 5.12 per Student

http://wafields.wikispaces.com/


Video PowerPoint Website0

20406080 67

5 2

Presentation Format

This data does not include each of the 74 students joining the Wiki, file management,

uploading photos, and many scaffolding activities that took place prior to the project. Students

learned how to log into their school email, navigate to their network folders, export documents to

JPGs from various applications, and used a Google Doc form to vote on their favorite

presentations.


The mentors were trained in the fall over a series of afterschool, two hour training

sessions. The 16 students were recommended by teachers for this opportunity. The majority of

the mentors were good recommendations, but there needs to be more detailed criteria for choices

in the future. They learned about specific technologies, how to be a mentor, and about Internet

safety and netiquette. The guidelines for the mentor training came from the Tech Yes Teacher

Guide.

Mrs. Fields stated that the student mentor role was, “Huge, because everyone is doing

different things and you need as many eyes and hands as possible.” The mentors gained

confidence through their roles and other non-mentor students even helped out their peers with

content and technology related activities. Mentors were positive in their feedback, and had a


helpful attitude throughout. One example of this was observed when a student mentor rose from

his seat and went person to person in his row to see if his peers needed help.

During the mentor evaluation process toward the last phase of the projects, mentors not

only used class time for this part of the process, but attended tutorial in order to aid their fellow

students. A count of each mentor evaluation was tabulated in the following graph. Even though

mentors were not assigned specific students, and volunteered when needed, the distribution was

extremely close. The range of difference is four students, but all mentors evaluated at least three

other students. The goal would be four students, so considering the circumstances of time and

opportunity (mentors still needed to attend other class tutorials, such as math or English), the

results are desirable.

Mentor 1

Mentor 2

Mentor 3

Mentor 4

Mentor 5

Mentor 6

Mentor 7

Mentor 8

Mentor 9

Mentor 1

0

Mentor 1

1

Mentor 1

2

Mentor 1

3

Mentor 1

4

Mentor 1

5

Mentor 1

60

2

4

6

8

Mentors

Num

ber o

f Eva

luati

ons


The findings from the data were surprising. The motivation was higher for all students in

comparison to their traditional classroom motivation, but the highest achievers were the least

motivated.


The students in the above average class were more eager to “get finished” rather than to

make a better project. The above average class was excited in the beginning, but seemed to have

a harder time with frustration when faced with obstacles. They did not produce the best projects.

According to Mrs. Fields, “The weaker students experienced a higher level of motivation.

This is because they struggle with the traditional methods already, so something new is

appealing to them.” The evaluator also observed this to be accurate.

The motivation of the average and lower classes stayed higher than the above average

class for the duration of the projects. However, they were all more excited about doing the

projects rather than book, and pencil/paper activities as an alternative. The students showed their

enthusiasm with their actions. Mrs. Fields said, “The students are all almost 100% engaged.” The

evaluator observed this engagement, as well. There were few behavioral issues and the students

made good use of their time. Even though there was some frustration in all of students with the

management of various technologies, they were still working hard to accomplish their goals.

Creative students were more motivated both in the inquiry learning and in the technology aspect.

They seemed to enjoy the freedom of not having specific boundaries.

In a follow up discussion with Mrs. Fields, she was encouraged about the upcoming

spring project. She said “The students are excited and did not struggle with their topic selections.

They cheered when I told them the projected time schedule of the spring project.”

Recommendations and Conclusions

The evaluation of this program provided evidence of improved technology use, higher

motivation for students, and the importance of using student mentors. Even though the outcomes

are extremely positive, there are some areas of weakness that need to be addressed.


Students encountered issues in data collection, and realized once they started their

presentation projects that they did not document their experiments well enough. File

management was problematic for many of the students. Mentors struggled for time management

between helping others and finishing their own project. Access to the lab was a concern and

students needed to come during tutorial afterschool to finish their projects. There were many

passwords and file names to remember and the students often forgot theirs. Even though a

standardized file naming format was used, there were still issues with location and overwriting

each others’ files.

After observing these issues and discussing possible solutions with the teacher, the

evaluator recommends creating a journal notebook. The notebook should include daily pages

with data collection ideas and a place to write down the events of the day. It should include a

page with username and password reminders, as well as a place to write down all of the new

information. Continued reminders need to occur about file naming formats. The notebook needs

pages pertaining to analysis and a conclusion.

Another suggestion is for a time to be set aside for mentors to work alone on their

projects. This could be done during class time, recess, or before or after school. Mentors may

also need a refresher technology reminder just prior to beginning the technology portion of the

projects. There needs to be a more detailed evaluation form for the mentors that includes

checking for spelling, grammar, last names of students (should not be included), no names with

images, sources, and conclusion. Students can be directed to Atomic Learning for tips on how to

use various technologies.


In conclusion, the evaluator recommends for the continuance of the Tech Yes program at

the Lower School, as well as the expansion to the remainder of the sixth grade science classes. It

would be beneficial for other curricular areas to investigate Tech Yes that are not intended for a

specific content area. All disciplines could profit from this type of PBL. Traditional teaching still

needs to occur in the classroom; however, students need to be exposed to various methods of

teaching and PBL such as Tech Yes should be part of the curriculum.


References

http://techyes.net/science

Bell, S. (2010). Project-Based Learning for the 21st Century: Skills for the Future. Clearing

House, 83(2), 39-43. doi:10.1080/00098650903505415.

ChanLin, L. (2008). Technology Integration Applied to Project-Based Learning in Science.

Innovations in Education and Teaching International, 45(1), 55-65. Retrieved from ERIC

database.

Frank, M., & Barzilai, A. (2004). Integrating alternative assessment in a project-based learning

course for pre-service science and technology teachers. Assessment & Evaluation in

Higher Education, 29(1), 41-61. Retrieved from Academic Search Complete database

Hernández-Ramos, P., & De La Paz, S. (2009). Learning History in Middle School by Designing

Multimedia in a Project-Based Learning Experience. Journal of Research on Technology

in Education, 42(2), 151-173. Retrieved from Academic Search Complete database.

Juniu, S. (2006). Use of Technology for Constructivist Learning in a Performance Assessment

Class. Measurement in Physical Education & Exercise Science, 10(1), 67-79.

doi:10.1207/s15327841mpee1001_5.

http://techyes.net/science


Appendix

Contents

_Toc257611976

Interview of Tech Yes Teacher................................................................20

Tech Yes Survey......................................................................................21

Tech Yes Project Evaluation Form..........................................................22

Rubrics.....................................................................................................24

Tech Yes Journal......................................................................................26

Peer Mentor Project Review Checklist....................................................31


Interview of Tech Yes TeacherInquiry Learning Questions

1. What are your thoughts about inquiry learning?2. How many years have you been teaching using an inquiry style?3. What are the barriers to teaching in this mode?4. How many projects to you typically do per year?5. In comparison to when you teach in a traditional method, what are differences that you

see in the motivation of the students?6. Do you see a difference in the motivation of specific types of learners? Ability levels?

Technology Questions

1. How has technology changed the way you do your inquiry projects?2. What are the barriers to using technology integrated into the inquiry projects?3. What are your thoughts about the students’ engagement while using technology?4. What are your reasons for implementing a technology integrated approach?5. In comparison to when you teach not using technology, what are differences you see in

the motivation of the students?6. Has the technology changed the motivation of specific types of learners? Ability levels?

Mentors

1. What was the role of the student mentor?2. What are the benefits of using mentors in this program?3. What are the concerns of using mentors in this program?4. What training did the mentors receive?

General Questions

1. What changes do you think you will make the next time you implement this program? Why?

2. What obstacles did you overcome?3. What went well? Why?4. Do you think this program should be continued? Why? Expanded? Why?5. How has this program changed the way you view teaching?


Tech Yes Survey

1 = Strongly Agree 2 = Agree 3 = Undecided 4 = Disagree 5 = Strongly Disagree

Circle the number that best describes your experience.

1 2 3 4 5 Inquiry Learning is an effective method of learning.

1 2 3 4 5 Technology has changed my methods of teaching.

1 2 3 4 5 Inquiry learning should be independent with students doing their own work. They should not receive help from other students.

1 2 3 4 5 Students seem to be more motivated when using inquiry learning.

1 2 3 4 5 Students seem to be less motivated when using technology.

1 2 3 4 5 Technology has been beneficial to the students who struggle academically.

1 2 3 4 5 Technology has been beneficial to the students who are academically average.

1 2 3 4 5 Technology has not been beneficial to the students who are academically above average.

1 2 3 4 5 Mentors were an important component of the Tech Yes program.

1 2 3 4 5 There weren’t any obstacles in using this program.

1 2 3 4 5 The obstacles (if any) were easy to overcome.

1 2 3 4 5 The obstacles were mostly involved using the inquiry based portion.

1 2 3 4 5 The obstacles were mostly involved using the technology.

1 2 3 4 5 The obstacles were not significant.

1 2 3 4 5 Students are good helpers to each other.

1 2 3 4 5 The Tech Yes program should be expanded.

1 2 3 4 5 When students help each other, learning does not occur.

1 2 3 4 5 Students are engaged more during this type of projects than in traditional teaching methods.

1 2 3 4 5 The Mentor training was sufficient for the program.

1 2 3 4 5 The program should continue in the future.

1 2 3 4 5 There is not enough access to technology to make the program successful.


Tech Yes Project Evaluation Form


Rubrics


Tech Yes Journal

Name_______________________Class Period___________________

Tech Yes JournalData Collection


Day ___ Date____________________

□ Camera (Pictures of…)_____________________________________□ Flip Video (Video of…) _____________________________________□ Vernier Probes □ Excel Spreadsheet□ Inspiration□ Other (Please List)

_______________________________________________________

Journal of Events

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________

________________________________________________________________________________


Technology Available

Data Collection and Organization

Inspiration- Use the webbing to organize your project

Word- Use Word to type your content, so that the spell check and grammar check can be used.

Excel Spreadsheet- Keep track of daily data, create a graph, or use for charting information.

Camera- Make sure to put photos in your file.

Flip Video- Do not say names with faces. Try to do short clips and use the video tips sheet.

Microscope

Vernier Probes- Test water

Timeliner- Use to list the events that took place each day of your experiment or for sequencing of dates.

Project Creation-NO LAST NAMES OR NAMES WITH FACES

Pixie2- Use for a slide video presentation or create images to put in other programs

Movie Maker- Use if you have video and/or still images (JPGS). You can record sound, or import sound

PhotoStory3- Use only with still images. You can record sound, or import sound

Glogster- You can use videos, sounds, and text to create an online presentation.

Frames- This can be used to create stop animation or a progressive digital model. (For instance, you could make a heart and show the flow of blood through the heart).

Comic Life- This can be used to create images (JPGS) to put in programs or make several and turn it into a movie (then put in Movie Maker to add sound).

Power Point- This can be used and export slides as JPGs or put on slideshare.com to go on the wiki.

Web Blender- Use this program (similar to Pixie) to create a “Web Space” with web pages that are linked.

Audacity- Use to record and edit your voice, other sounds, and music. You will export as an MP3 to use in other programs.


Website Login, Password Help, and Tips

File Naming: Name all files will the f-period-all three initials. If there is more than one file, add a number.

Example: f3kkm or f3kkm1

Technology Video Tutorials

http://www.atomiclearning.com

Username: woodward

Password: academy

Student Gmail:

Go to the WA website Go to Student Start Page Go to Student Gmail

User Name = first initial middle initial last name 16

Password = Student ID number wa

Noodle Tools:

www.noodletools.com

Personal ID = first name.last name

Password = student ID #

Simply Box:


www.simplybox.com (NOTE: This has to be downloaded to your computer with your parents’ permission!!)

User Name = first initial middle initial last name 16

Password = student ID # wa

Nettrekker:

http://school.nettrekker.com

User Name = wa_student

Password = eagle

TechYES:

www.woodward.genyes.com

As a student – User name = first name last name


As a Peer Mentor – User name = first name.last name


Wiki Login

http://wafields.wikispaces.com

Username-- wawiki and your initials –ex wawikidf

Password—waID ---ex wa12345


Peer Mentor Project Review Checklist

Can you read the text in the presentation (watch for color and size)? Suggestions:_______________________________________

Is all spelling/grammar correct?Correctons:_____________________________________________________________________________________________________________

Did the student use only first names if there weren’t any photos of faces--- NO names with faces.

Do you understand the project or is the presentation unclear? Suggestions:______________________________________________________________________________________________________

Did the student include their data? Did the student include their conclusion? Did the student need to list resources or was the project an

original idea?

interview of tech yes teacher - university of west...

Documents