hp innovations in education project results & analysishp innovations in education project...

HP Innovations in Education Project Results & Analysis

The preliminary results are categorized into (i) Attitude survey, (ii) Service Learning project surveys, (iii) Interactive activities survey responses, (iv) Survey responses to flash-based animations, (v) Student survey response to efficacy of pre-recorded video lectures, and (vi) Student responses to different mode of content delivery. Students were asked to respond to survey questions on a 5-point Likert scale (Strongly Agree = 5, Agree = 4, Neutral = 3, Disagree = 2, and Strongly Disagree = 1). Sample results of these surveys are presented as follows. A. Attitude Survey

Figure 1 shows the attitude of students towards use of Tablet-PCs and supporting note-taking tool in ENGT 103: Introduction to Engineering Technology course. The student attitude towards use of Tablet-PCs and Windows Journal was gathered through a questionnaire consisting of twenty-two questions.

As shown in Figure 1, 50% of respondents agreed that use of Tablet-PCs and windows journal improved their note-taking skills, 80% of respondents experienced improved classroom learning, 70% of respondents agreed that use of tablet and note-taking tool improved their understanding of course material, 90% of respondents had positive experience in using instructional technology used in classroom, and 100% of respondents agreed that Tablet-PCs and Windows Journal helped them to be more organized in their course work.

B. Service Learning Project Surveys Figure 2a and Figure 2b depicts responses to service-learning project survey in ENGT 103: Introduction to Engineering Technology course. Students in ENGT 103, which is an introductory course in Engineering Technology program, were assigned an energy audit project. As part of this project they were asked to pick a house in the local community and conduct a complete energy audit of that house. Twenty-six students participated in this project. They were required to interact with the occupants/owner of the house and manufacturer/supplier of electrical appliances used in a typical home. At the end of the project students submitted a report with an excel sheet showing distribution

0%

20%

40%

60%

80%

100%

Improved

Note-‐

Taking

Classroom

Learning

Improved

Understan

ding

Positive

Experience

More

Organized

% o

f Res

pond

ents

Survey Questions

Fig. 1: Students Survey - Attitude Towards Tablet-PC & Windows Journal

ENGT 103: Introduction to Engineering Technology

1 (SD) 2 (D)

3 (N)

4 (A)

of energy consumption in the house. The graph in Figure 2a shows student responses to a survey to assess their service-learning project experience.

As seen in Figure 2a only 5% of respondents strongly disagreed that they would like to see more service-learning projects. Students experience in the service-learning project was also evaluated through brief interactive session with the instructor. Overall response from students who participated in this project was very satisfactory. Only complain from students was related to the duration in which they have to complete the project. A majority of students suggested at least three-week period instead of 2 weeks that were given to them for completing the project.

Figure 2b shows responses of community residents to the service-learning project carried out by students in this course.

As seen in Figure 2b, 100% of respondents agreed that service-learning project was clearly articulated by students, 100% of respondents agreed that students exhibited professionalism, 100% of respondents agreed that students were proficient in their interaction, 100% of respondents agreed that students were knowledgeable about the project, 90% of respondents agreed they would like to see more service-learning projects.

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Strongly Agree Agree Not Sure Disagree Strongly Disagree

% of Respondents

Level of Agreement

Fig. 2a: Student Respones to Service-Learning Project ENGT 103: Introduction to Engineering Technology

Improved Self-‐ConRidence Applied Theory

Impact of Using Tablet

Broader Appreciation for Discipline More SL Projects

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00%

Clearly

Articul

ated

Exhibit

ed

Profes

sionali

sm

ProRici

ent

Interac

tion

Knowl

edgeab

le

More

S-‐L

Project

s

% of Respondents

Survey Questions

Fig. 2b: Service-Learning Project - Responses from Residents in the Community

ENGT 103: Introduction to Engineering Technology Strongly Agree

Agree

Not Sure

Disagree

Strongly Disagree

C. Interactive Activities Survey Responses

Figure 3a-3i show students responses to efficacy of using web-based interactive activities in some of the courses in technology program.

Figure 3a depicts student responses to a set of web-based interactive activities in TCPU 450: Network Communications II course. As seen in Figure 3a, 100% of respondents found activities to be well-planned, 90% of respondents experienced improved understanding of the topic, 100% of respondents expressed that activities made them think critically, 90% of respondents agreed that activities reinforced what they learnt in class, 100% of respondents agreed that interactive activities can help improve their grades and they would like to see more such activities.

Results from similar activities in the same course and other courses are presented below.

0%

10%

20%

30%

40%

50%

60%

70%

80%

Well-‐

Planned

Improve

d Underst

anding

Think

Criticall

y

Reinforc

e Learnin

g

Improvi

ng

Grades

More

Activitie

s

% of Respondents

Survey Questions

Fig. 3a: Student Survey Responses - Switch-Forwarding Interactive Acitivities

TCPU 450: Network Communications II

SA

A

N

D

SD

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

Well-‐

Planned

Improved

Understand

ing

Think

Critically

Reinforce

Learning

Improving

Grades

More

Activities

% of Respondents

Survey Questions

Fig. 3b: Student Survey Responses - OSPF Routing Protocol Interactive Acitivities

TCPU 450: Network Communications II

SA

A

N

D

SD

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Well-‐

Planned

Improved

Understan

ding

Think

Critically

Reinforced

Lecture

Grades

Improve

More

Activities %

of Responsents

Survey Questions

Fig. 3c: Student Surveys: ACL Interactive Activities TCPU 460 Network Administration & Security w/Lab

Strongly Agree Agree

Not Sure

Disagree

Strongly Disagree

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

Well-‐Planned Improved Understanding

Think Critically Reinforced Lecture

Grades Improve

More Activities

% of Respondents

Survey Questions

Fig. 3d: Student Survey Responses: Basic Network Security Interactive Activities

ENGT 340 Digital Communications

Strongly Agree

Agree

Not Sure

Disagree

Strongly Disagree

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

Well-‐

Planned

Improved

Understan

ding

Think

Critically

Reinforce

d Lecture

Grades

Improve

More

Activities

% of Respondents

Survey Questions

Fig. 3e: Student Survey Responses: Wireless Technologies Interactive Activities



Not Sure

Disagree

Strongly Disagree

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Well-‐

Planned

Improve

d Understa

nding

Think

Critically

Reinforc

ed

Lecture

Grades

Improve

More

Activities

% of Respondents

Survey Questions

Fig. 3f: Student Survey to ISP Interactive Activities ENGT 340 Digital Communications Strongly

Agree Agree

Not Sure

Disagree

Strongly Disagree

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Well-‐

Planned

Improved

Understand

ing

Think

Critically

Reinforced

Lecture

Help

Improve

Grades

More

Activities

% of Respondents

Survey Questions

Fig. 3g: Student Surveys: Static and Dynamic Routing Interactive Activities

TCPU 360: Network Communications I w/Lab Strongly Agree

Agree

Not Sure

Disagree

Strongly Disagree

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

Well-‐

Planned

Improved

Understa

nding

Think

Critically

Reinforce

d Lecture

Grades

Improve

More

Activities

% of Respondents

Survey Questions

Fig. 3h: Student Survey Responses: Wireless Technologies Interactive Activities


Strongly Agree

Agree

Not Sure

Disagree

Strongly Disagree

D. Survey Responses to Flash-Based Animations

Figure 4a-4f show student responses to flash-based animations used in some of the computer networking courses. Flash-based animations/simulations were used to depict difficult concepts visually covered in these courses.

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Well-‐

Planned

Improved

Understan

ding

Think

Critically

Reinforced

Lecture

Help

Improve

Grades

More

Activities

% of Respondents

Survey Questions

Fig. 3i: Student Survey: Intractive Activities - Network Addressing ENGT 340 Digital Communications


Not Sure

Disagree

Strongly Disagree

0%

10%

20%

30%

40%

50%

60%

70%


% of Respondents

Level of Agreement

Fig. 4a: Student Surveys: Flash-Based Animations - Network Addressing


Improved Understanding

Reinforced Lecture

Improved Performance

More Animations

Figure 4a shows responses to a set of flash-based animations used to simplify some key topics in Network Addressing from ENGT 340 Digital Communications course. As shown in Figure 4a, 90% of the respondents agreed that flash-based animations improved their understanding of the topic, reinforced lecture content, can improve their course performance, and they would like to see more animations in the course.

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

Strongly Agree

Agree Not Sure Disagree Strongly Disagree

% of Respondents

Level of Agreement

Fig. 4b: Students Responses to Router ConXiguration Flash-Based Animations

TCPU 360: Network Communications I w/Lab

Improved Understanding Reinforced Lecture


More Animations

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00%

Strongly Agree


% of Respondents

Level of Agreement

Fig. 4c: Students Responses to ACL Flash-Based Animations TCPU 460: Network Administration & Security


Reinforced Lecture


More Animations

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Strongly Agree


% of Respondents

Level of Agreement

Fig. 4d: Students Responses to ISP Flash-Based Animations ENGT 340: Digital Communications


Reinforced Lecture


More Animations

0%

10%

20%

30%

40%

50%

60%

70%

Strongly Agree


% of Respondents

Level of Agreement

Fig. 4e: Student Survey Responses: Distance Vector Routing Protocol Flash-Based Animations

TCPU 360: Network Communication I w/Lab

Improved Understanding Reinforced Lecture


E. Student Responses to Pre-Recorded Lecture Videos

Figure 5a-5b show students survey responses to pre-recording video lectures on selected topics in TCPU 360 and ENGT 340 courses.

Students in TCPU 360: Networking Communications I w/Lab course and ENGT 340: Digital Communications were asked to watch pre-recorded video lectures on selected topics. They were given a pre-test in the beginning and then a post-test after watching the video lectures. In addition, to the pre/post test they were also given a survey to assess the effectiveness of video lectures in their learning. Figure 5a depicts student responses (cumulative) to pre-recorded video lecture survey for three topics. As seen in the result graph, 100% of respondents agreed that video lectures

0% 10% 20% 30% 40% 50% 60% 70% 80%

Strongly Agree


% of Respondents

Level of Agreement

Fig. 4f: Student Survey: Static & Dynamic Routing Flash-Based Animations

TCPU 360: Network Communications I w/Lab


Reinforced Lecture


More Simulations

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

Easy to

Understa

nd

Improve

d Understa

nding

Replace

Lecture

Slides

Improve

Performa

nce

More

Videos

% of Respondent

Survey Questions

Fig. 5a: Students Responses to Lecture Video Set Distance Vector Protocol/RIP/EIGRP

TCPU 360: Network Communications I w/Lab Strongly Agree Agree

Not Sure

Disagree

Strongly Disagree

were easy to understand, over 80% of respondents agreed that it improved their understanding of the topic, over 80% of respondents agreed that video lectures can replace PowerPoint lecture slides/notes, over 80% of respondents agreed that video lectures can improve their course performance, and over 80% of respondents would like to see more video lectures in this course.

F. Student Responses to the Mode of Content Delivery

At the end of the semester students were asked to grade their experience with different ways the course content was delivered. The delivery methods used were: Reading text/handouts, PowerPoint lecture slides, Flash-based simulations/animations, Interactive activities/puzzles, and video lectures. Figure 6a-6f presents results from student responses to mode of content delivery survey questions for TCPU 360 course and ENGT 340 course.

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Easy to

Underst

and

Improve

d Underst

anding

Replace

Lecture

Slides

Improve

Perform

ance

More

Videos

50% 50% 50%

25%

50%

37.50%

25%

12.50%

50%

25%

0.00%

25% 25%

12.50%

25%

12.50%

0.00%

12.50% 12.50%

0.00% % of Respondents

Level of Agreement

Fig. 5b: Student Survey Responses to Lecture Video Set ENGT 340: Digital Communications


As shown in Figure 6a, students rated (Average = 3.66) Interactive activities/puzzles to have the highest level of impact on their learning in TCPU 360 course, whereas, Reading textbook or class handouts had the least impact (Average = 1.833) on their performance in this course.

Figure 6b presents student responses to various learning skills that were improved through use of technology in TCPU 360 course. Five skills were identified as Interactive, Collaborative, Active Learning, Participatory Learning, and Critical Thinking.

1.833

3.5

3.5

3.66

3.166

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00

Reading Textbook/Handouts

PowerPoint Lectures/Slides

Flash-‐Based Simulations/Animations

Interactive Activities/Puzzles

Video Presentations

Mean Level of Impact

Method of Content Delivery

Fig. 6a: Student Responses to the Level of Impact of Each Method of Delivery

TCPU 360 Network Communications I w/Lab

As shown in Figure 6b, 100% of the respondents identified that Participatory Learning skill was improved through use of technology. 50% of respondents experienced interactive and collaborative skill improvement through use of technology, over 80% of respondents expressed that their Critical Thinking improved through use of technology. Only 33.33% of respondents identified that their Active Learning skill was improved through use of technology.

Figure 6c depicts student responses to preferred mode of course content delivery. Students in TCPU 360 course were asked to select their top two preferred methods of content delivery.

As seen in Figure 6c, Interactive activities/puzzles and Flash-based simulations/animations used in this course were the top two preferred methods.

83.30%

100.00%

33.33%

50.00%

50.00%

0.00% 20.00% 40.00% 60.00% 80.00% 100.00%

Critical Thinking

Participatorial

Active Learning

Collaborative

Interaction

% of Respondents

Reported Skills

Fig. 6b: Student Responses to Skills Improved Through Use of Technology

TCPU 360 Network Communications I w/Lab

0.00%

16.66%

83.33%

83.33%

16.66%

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00%





Video Presentations

% of Respondents


Fig. 6c: Student Responses to Preferred Method of Content Delivery

(Select Top Two) TCPU 360 Network Communications I w/Lab

1

4.12

4

2.87

3

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50





Video Presentations

Mean Level of Impact


Fig. 6d: Student Responses to the Level of Impact of Each Method of Delivery


62.50%

62.50%

100.00%

37.50%

62.50%

0.00% 20.00% 40.00% 60.00% 80.00% 100.00%

Critical Thinking

Participatorial

Active Learning

Collaborative

Interaction

% of Respondents

Reported Skills

Fig. 6e: Student Responses to Skills Improved Through Use of Technology


In academic year 2010-2011 (2nd year of the project) following evidence will be collected to gage impact of using HP Blade workstation-based remote computing infrastructure on both instruction and learning.

• Monitor utilization of resources on Blade workstations. • Comparing the number of courses offered web-enhanced and distance learning courses offered

before and after the availability of high performance blade workstations. • Frequency of faculty and students accessing resources on blade workstations. • Monitoring the quality of web-enhanced/distance learning courses.

Topic and Course wise Pre-Test/Post-Test

In order to quantitatively assess the impact of integrating technology-based learning in various courses, students were given pre and post test on every key topic covered. In addition to topic wise pre/post tests, students were also given course wise pre/post test in few courses. The results from these pre/post tests are shown in Figure 7a-7e. In Figure 7a, topic wise pre/post test (Average Score) comparisons from ENGT 340 course is shown. Results clearly show a significant improvement in student performance on each topic. (Note: No course wise pre/post test was given for ENGT 340 course as this course was an elective)

0.00%

50.00%

75.00%

50.00%

12.50%

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00%





Video Presentations

% of Respondents


Fig. 6f: Student Responses to Prefered Method of Content Delivery (Select Top Two)


Students in ENGT 340 were also involved in creating a course Blog. Each student was required to make at least two blog entries per week. Student could either start a new discussion on blog or respond/comment on existing discussion. Students were graded based on the quality and course relevance of their blog entry.

Figure 7b shows topic wise pre/post test comparisons from TCPU 460 (Network Admin & Security) course. Again the comparisons show a significant improvement in student performance in the topics shown in Figure 7b.

34.33 42.86

48.05 48.81 47.5

37.33 29.63

40

80.33

70.86

84.72

73.21 78.44

83.99 88.89

77.5

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

OSI M

odel

TCP/IP

Fundam

entals ISP

Network

Addressin

g 3OSI

Layers

(Video)

Wireless

Technologi

es

Remote

Access

Basic N

et

Security

Student Average % Scores

Course Topics

Fig. 7a: Topic wise Pre/Post Test Comparisons ENGT 340: Digital Communications Pre-‐Test

Post-‐Test

0

10

20

30

40

50

60

70

80

90

Network Security Cryptography Access Control List

Firewalls

Student Avaerage % Scores

Course Topics

Fig. 7b Topic wise Pre/Post Test Comparisons TCPU 460: Network Admin & Security

Pre-‐test

Post-‐test

Figure 7c shows course wise pre/post test comparisons for TCPU 460 course. The results show a significant improvement (Average post test score of 68.9% as compared to 22.6% in the pre test) in student performance in this course.

Similar results from another course are shown in Figure 7c and Figure 7d.

0 10 20 30 40 50 60 70

Pre-‐test Post-‐test


Fig 7c: TCPU 460: Course Pre/Post Test Results

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

100.00

Static&DynamicRouting IP Addressing Distance Vector Protocol (Video)

SwitchForward

Student Average % Score

Course Topics

Fig 7c: Topic wise Pre/Post Test Comparisons TCPU 360: Network Communications I w/Lab

Pre-‐test

Post-‐test

This project supported in part by an HP Innovations in Education grant.

0

10

20

30

40

50

60

70

80

Pre-‐Test Post-‐Test


Fig 7f: TCPU 360: Course Pre/Post Test Results

hp innovations in education project results & analysishp innovations in education project...

Documents