Submitted by Dr. Naomi HallBehavioral Sciences and Social Work

July 2010

Video Lecture Capture Initiative

Fall 2009 Initiative Report

Winston-Salem State University

EXECUTIVE SUMMARY

Winston-Salem State University (WSSU) has experienced a steady decline in student retention and graduation rates over the past five years. Additionally, the number of students who are dropping, failing, or withdrawing (DFW) from classes each semester has been increasing. The recently released WSSU Strategic Plan (2010-2015), acknowledges a number of contributing factors to deteriorating student outcomes. Poor student preparation has been identified as an outcome that demands systematic and timely address. One of the techniques, video lecture-capture, is being piloted by the Center for Excellence in Teaching and Learning (CETL) on the campus of WSSU. Since the core of students’ academic success centers around what happens in the classroom, CETL believes that the lecture-capture approach will be a good supplemental tool for enhancing student performance. Video lecture-capture (VLC) is a mechanism that allows faculty to either pre-record lectures or tape live lectures and upload for students to review at their leisure. The VLC has great potential to enhance teaching and learning outcomes—as indicated by surveys completed by both students and faculty. Unfortunately, there were some challenges with some of the data so the discussion of the extent to which this project enhanced teaching and learning is not included. VLC usage was significantly correlated with aggregated assessment scores and increased from video to video—generally declining after the third video. Nonetheless, overwhelmingly, students thought the VLC was helpful, beneficial, and valuable to their learning experience in the participating classes. This report identifies a number of interesting findings and trends; however, six notable findings/trends are as follows:

The vast majority of students thought the VLC was helpful, beneficial, and valuable to their learning experience in the participating classes;

The students who accessed the VLC system most frequently were those who had GPAs either in the very high or low range;

Students who worked more than 11 hours per week were more likely to utilize the videos for instruction and review;

The availability of pre-recorded and live recorded videos did not negatively impact student attendance in classes;

Data indicate that class structure should be should be taken into account when deciding whether or not to implement VLC in a particular course; and

VLC usage did not have a statistically significant impact on assessment scores in individual classes; however the impact of usage on assessment scores was seen when data was aggregated

2

Introduction

The lecture-capture initiative was started to address the increasing

problem at Winston-Salem State University (WSSU) of students who are

dropping, failing, or withdrawing (DFW) from classes each semester.

Institutional research studies have shown a constant decline in retention

each semester as a result of this problem. According to the WSSU Strategic

Plan (2010-2015), there are a number of contributing factors to

deteriorating student outcomes, but one of the main ones is poor student

preparation (WSSU, 2010). This can be interpreted as poor pre-college

preparation, and/or poor classroom preparation. A number of different

approaches have been taken across the academy to address the issue of

high DFWs. Some of these approaches included modifying tutoring services,

increasing mentorship initiatives, following up with at-risk students at

shorter intervals in the semester in order to monitor their progress,

providing remediation, among other efforts. It is important to develop,

enhance, and supplement instructional support programs so that students

3

are successful inside and outside of the classroom. Since the core of

students’ academic success centers around what happens in the classroom,

the lecture-capture approach seemed to be a good solution for enhancing

student performance. Earlier work with lecture-capture has shown a

positive improvement in the learning and performance of students who

review the videos (Chandra, 2007).

Lectures captured using digital video and made available using streamed

video allows students to review at their own pace the information the

instructor explained. They can also view any associated visual aids and

search for concepts and resources relating to those aids. Lectures captured

in this way may help students overcome weaknesses in areas such as taking

notes, paying attention over extended periods of time, or understanding

language barriers. Another very important aspect is the ability to make

remediation sessions available so that students who may lack certain skills

can remediate without having to spend class time doing so. With this in

mind, the Center for Excellence in Teaching and Learning (CETL) explored

alternative video lecture capturing solutions, which in essence, places at the

student’s disposal the core of what affects their academic success, which is

the professor’s perspective on the course material. An initial pilot (N = 72)

took place during the 2009 summer session II. A brief summary of the

findings are as follows (Hall, 2009):

4

Students who utilized Panopto overwhelmingly (85%) thought it was

helpful, beneficial and valuable to the class in which they were

enrolled.

A student’s grade point average (GPA) was correlated with whether or

not the videos were viewed, and for how long. The students who

reviewed the videos more frequently, and for longer periods of time

appear to be those students whose GPAs are hovering around the 3.1-

3.5 range.

It appears that two of the main factors that helped students decide

whether or not to access the videos were class requirements, and

time. Of those students who did not review any videos, over half said

they did not because it was not required for the course.

o The summer sessions are very fast and intense, and most of the

students who were attending classes (per Assessment data) also

worked at least 20 hours per week. Students were honest in

saying they had to prioritize what they could and could not do in

this shorten course. This speaks to making Panopto a

requirement so that students will not have to make those types

of study and review choices in the future.

Summer faculty did not see the benefit of the videos to the students.

The videos were not seen as convenient, helpful in student

preparation for quizzes and exams, or in discussions.

5

o This may be a direct reflection of the fact that all videos during

the pilot were recorded during an actual class session. Pre-

recording has the potential to have faculty teach a

concept/theory/phenomenon in a virtual one-on-one session with

PowerPoint slides, instead of moving around the classroom.

None of the faculty agreed to pre-record, so this is something

that will be explored in the fall 2009.

With the Fall 2009 Initiative, CETL investigated:

Whether this type of system enhanced the teaching and learning

processes

If it can enhance it, to what extent?

If the availability of the videos had any significant impact on

student assessment scores

If students found the video lecture-capture software helpful and/or

valuable to their learning experience

Methodology

Participants

Students participating in this project were from eight courses that

have a high percentage of DFWs at Winston-Salem State University. There

were no specific inclusion and/or exclusion criteria for students. All students

in the eight selected classes had the opportunity to participate. Additionally,

the faculty who taught those courses participated in the project as well.

Measures

6

Students were asked to complete three measures: 1) Initial Student

Assessment, 2) Individual Video Lecture Capture (VLC) Assessment, and the

3) Final Student Assessment. The Initial Student Assessment was given to

all students (N=264) at the beginning of each course and asked primarily

for demographic information such as gender, student classification, age,

major, current work status, and instructor. The Individual VLC Assessment

was an ongoing survey that asked students to provide information on the

video(s) they viewed, along with their attitudes and experience using the

software. Finally, students were asked to complete a Final Student

Assessment to ascertain their attitudes and experiences viewing the video,

using the software, and how it may have affected their performance in the

class. This assessment also asked students to provide any suggestions for

improving the videos and software, and implementing the initiative in other

classes. Faculty who participated in the initiative were asked to complete a

Faculty Assessment Form to gather feedback about their perceptions of

using the software, how they think the performance of students in class was

impacted, and any of their suggestions for future use.

Procedures

The videos were captured and uploaded using Panopto. Panopto is a

lecture-capture system software which makes portability quite easy. It

allowed faculty to record their lectures from anywhere, at anytime, using

video and audio, and captured information from PowerPoint slides, or other

text sources. From the metadata that is created, students had the

7

opportunity to search using any of the text, or PowerPoint thumbnails.

During a search, all data (video, text, audio) is synchronized at the point of

delivering the search results. Students were all instructed to sign up for a

Panopto account so that they could access the videos.

After a few trials to be sure that Panopto would not pose any problems

with its integration with Blackboard, the software was fully implemented for

use by any faculty member. Eight faculty members, mainly in the College of

Arts and Sciences and School of Health Sciences, either pre-recorded or

recorded their lectures live, and made the contents available for viewing by

their students. Additionally, faculty were provided with the measures from

which the following data were collected.

Students and faculty completed all assessment forms via SurveyMonkey

© by assessing the provided link. Faculty provided the link to the three

assessment tools to all students via email. Students clicked on the link and

were taken to the appropriate survey. Faculty were also given a link to use

for completion of the Faculty Assessment Form.

Results

Initial Student Assessment

A total of 309 students provided information for the initial survey.

However, only 264 students indicated that they consented to participate in

the study—the remaining 45 students skipped this initial question. The

following data represents only the students who indicated their informed

consent for participation. The participant pool (N=264) for this survey was

8

overwhelmingly populated by women (78.2%). When students were asked

“in which range does your age fall?” we saw that 69.1% of the students

indicated they were less than 20 years of age, 18.5% between 21 and 24

years, and 12.4% over the age of 25. The student classification was diverse

with the sample consisting of 27.7% Freshman, 32.7% Sophomore, 22.3%

Juniors, 11.9% Seniors, and 5.4% pursuing a second degree. There were 32

different major courses of study represented in this sample. The top five

most frequently indicated majors were: 1) Nursing (29.6%); 2) Psychology

(8.8%); 3) Business Administration (8.1%); 4) Mass Communication (7.3%);

and 5) Information Technology (6.9%). The other 27 majors represented

39.3%, collectively, of the pool.

Of the 264 students who completed the survey, 57 (21.6%) were

repeating the course. When asked why students were retaking the course,

43.4% indicated they had failed the course. Approximately 32% received a

‘D’ and 24.5% either dropped or withdrew from the course. The trend

seemed to be that those students who were taking the course again, did so

with a different instructor (88.9%). Forty percent (41.6%) of the students

indicated that they were currently working, and 39.5% worked 11-20 hours

per week, and 35.4% worked over 21 hours per week. An interesting finding

was discovered when examining the relationship between those students

who work, and those who had to retake, χ2 (1, N = 263) = 4.30, p = .038.

Students who worked were significantly more likely to be the ones who

were re-taking the class.

9

Although there were eight instructors participating in the study, the

vast majority of the students came from the classrooms of Drs. Nelson

Adams (PSY 2301 General Psychology-33%) and David Kump (BIO 1301

Biological Concepts-24.2%). The other instructors (see Figure 1), and the

percentage of students participating are as follows: Manjunatha Bhat (BIO

2311 Anatomy and Physiology-11.4%), Johanna Porter-Kelley (BIO 1331

General Microbiology-10.5%), Christopher Stanley (PSY 2326 Statistics for

Social/Behavioral Sciences-8.7%), Jan Jasper (FIN 2356 Personal Finance-

7.6%), Jeffrey Overholt (BIO 3311 Fundamentals of Anatomy and

Physiology-4.2%), and Sathasivam Mathiyalakan (MIS 3380 File

Structure/Database Operations-.4%).

Adams (PSY)33%

Bhat (BIO)11%

Jasper (FIN)8%

Kump (BIO)24%

Mathiyalakan (MIS)0%

Overholt (BIO)4%

Porter-Kelley (BIO)10%

Stanley (PSY)9%

Figure 1: Percentage of Participating Students

Individual VLC Assessment

There were a total of 149 survey responses for the Individual VLC

Assessment. The Individual VLC Assessment asked students to provide

10

information on the video(s) they viewed, along with their attitudes and

experience using the software. Even though there were eight faculty

members participating in the study, only six had students who completed

the surveys. The six faculty members whose class data is used for this

section are Drs. Mathiyalakan, Stanley, Kump, Adams, Porter-Kelley, and

Overholt. Students were asked to indicate which videos they reviewed

during the course. The following highlights these results:

1. Dr. Mathiyalakan—it was difficult to ascertain a pattern of video

viewing. His students represented 53.5% of those who viewed

the videos; however, there was no consistency in how students

reported the name of the video. Some students indicated an

assignment number, while others listed a chapter number. Even

though it was difficult to identify the most frequently viewed

video, a large number of students said they “viewed all of the

available videos.”

2. Dr. Stanley—his class made up 24.3% of the students who

viewed videos, and the top three videos viewed were Chi-

Square, Hypothesis testing, and Means and Variances.

3. Dr. Kump—his class made up 9.1% of the students who viewed

videos, and the top video viewed was the one related to Chapter

7 (Carbon Cycle).

4. Dr. Adams—his class made up 7.6% of the students who viewed

videos, and the top video viewed was on the Brain.

11

5. Dr. Porter-Kelley—her class made up 4.2% of the students who

viewed videos, and the top video viewed was on Chapter 7 (DNA

Replication).

6. Dr. Overholt—his class made up 1.4% of the students who

viewed videos, and the top video viewed was on the Endocrine

System.

It was important to gather information on the reasons students viewed

the videos, and how the video(s) may or may not have contributed to their

success in class. The overall tone of the students was that it was good for

review and clarification. When students were asked to indicate why they

viewed the video, overwhelmingly the response was focused on “reviewing

concepts.” When examining qualitative responses, three responses stood

out. One student said “to make sure I had an understanding of what was

going on so I am prepared for my test.” Another student indicated that

he/she had “missed half of this lecture and I wanted to see and hear it in my

own space so that I could understand it. I reviewed it totally twice and

viewed it a third time in sections that I needed to in order for me to

understand fully.” Finally, a non-traditional student said “I am returning to

school and don’t catch on as fast as the younger students. The videos

allowed me to pay attention and grasp the information at my pace.”

Students were asked if the video(s) were helpful in providing the

information they needed for the class, and 96.6% indicated it was.

12

Different instructors used different recording techniques with their

class—some pre-recorded and others recorded live. Students were asked if

they acquired any new information by watching the videos. Slightly over

half (54.7%) said they acquired new information, and one student said that

he or she was able to “reaffirm my knowledge of the subject.”

The overall viewing pattern for students appeared to be that they

viewed the entire video (93.2%) by themselves (97.3%). For those students

who did not view the entire video, interestingly, there was not much variety

in the amount of time spent viewing. Of the students who did not view the

entire video, approximately 33.3% viewed it for less than 5 minutes, 33.3%

viewed it between 6-10 minutes, and 33.3% viewed it for over 10 minutes.

We asked students if the video(s) were helpful to them, and 84.7% said

“yes.” On a scale from 0 to 10 (10 being the highest), students were asked

how beneficial was the lecture-capture video you just watched to your

understanding of concepts relating to this class? The mean rating was 8.48

(range 0-10, sd = 1.90). So, all in all, most students felt these videos were

beneficial to the course.

We wanted to see the grade point average (GPA) of those who viewed

the videos to see if those who had higher GPAs were more or less likely to

utilize the software. Of the students who used the software to view one or

more video, 35.4% had a GPA of 3.0 or better. The video(s) were viewed

most by those who had a GPA between 3.1-3.5. Of those who viewed at least

one video, 46.9% worked, in addition to being a fulltime student, and most

13

(55.2%) worked more than 20 hours per week. This initiative is still in the

preliminary phase of implementation, so we wanted to make sure to ask the

students to rate the technical quality of the Panopto software.

Overwhelmingly (89.9%), students felt the quality was either “good” or

excellent.”

We looked at the relationship between GPA and how beneficial

students thought the videos were. An analysis of variance (ANOVA) was run

on the two variables and found a significant relationship between a

student’s GPA and their ratings on the benefit of viewing the videos, F (3,

131) = 4.05, p = .004. Interestingly, students with the highest GPAs (over

3.5) and lowest GPAs (less than 2.0) found the videos most beneficial (9.35

and 9.40, respectively). A post-hoc Tukey’s test was conducted to examine

any possible differences between the five GPA groups. Two significant

differences were found between how beneficial the videos were for those

students whose GPAs were between 2.1-2.5 (8.81) and 3.1-3.5 (7.48), p

= .024; and those whose GPAs were between 3.1-3.5 and Over 3.5 (9.35), p

= .007. This finding is different than that of the initial pilot data (summer

2009). Pilot data revealed that students who were more likely to watch

these videos were those who were in the 3.1-3.5 GPA range, not those above

or below. However, the fall semester students, whose GPA is in the same

range, actually rated these videos to be the least beneficial.

Although there were no significant difference between employment

status and benefit of the videos, there were significant differences between

14

the number of hours a student works and their beneficial rating of the

videos, F (2, 63) = 10.58, p <.001. Those students who work more than 21

hours per week rated the videos as much more beneficial (9.62) than those

who worked between 11-20 hours per week (7.59), p <.001. Students who

worked less than 10 hours per week rated the benefit of the videos lower

than both groups (7.0).

Final Student Assessment

Two hundred and sixty four students completed the initial survey, but

only 150 completed the final survey (57%). As with the initial student

survey, the majority of the students who completed the final survey were

from Drs. Adams (26.2%) and Kump’s (20.1%) classes. Most of the faculty

had better initial survey rates than final survey rates. However, two

professors (Drs. Overholt and Mathiyalakan) had better final survey rates

(18.1% and 12.1%, respectively).

The students were asked to indicate the number of videos watched

over the course of the semester, and one in three students (35.6%) viewed

five or more videos. Only 5.4% of the students did not view any videos, and

the remaining 59.1% viewed between one and four of the videos available.

Of those 5.4% who did not view the video, 38.5% said they did not because

“it was not required for the course.” Unfortunately another 38.5% did not

view any of the videos because they indicated they had technical difficulties

assessing Panopto; 15.4% said they “didn’t see the benefit;” and 7.7% did

not know about the videos. Students had the opportunity to choose multiple

15

options, so the total does not equal 100%. Unlike the summer session pilot

project, most of the videos were pre-recorded (61.2%) versus taped during a

live class session (38.8%). Students were asked which style of video they

preferred, and two in five (41.6%) prefer the pre-recorded videos, while

30.2% indicated “no preference.” When asked about video viewing habits,

students provided the following information:

47.9% indicated they viewed the videos once, but went back to certain

parts of the video, while 18.8% viewed the video only one time.

26% of the students who viewed at least one video watched between

6-10 minutes of each video, and 25.3% watched at least 10 minutes of

each video.

Keeping in line with the Individual VLC Assessment, students

overwhelmingly viewed the videos alone (78.4%).

Students were asked how beneficial the videos viewed were in

understanding the concepts related to the course, and how valuable the

videos viewed were to the course. Both of these questions were on a 0-10

scale, with ‘0’ being the lowest (‘not at all’) point on the scale. More than

half of the students (58.3%) rated the benefit of the videos between an 8

and 10, 34.9% between a 4 and 7, and 6.8% between a 0 and 3. The mean

beneficial rating was 7.46 (range 0-10, sd = 2.37). Overwhelmingly,

students (65.4%) rated the value t of the videos between an 8 and 10, 29.2%

between a 4 and 7, and 5.4% between a 0 and 3. The mean value rating was

7.73 (range 0-10, sd = 2.33). Although there were some students who rated

16

the benefit and value fairly low, 99% said the videos were “very beneficial”

or “somewhat beneficial” for student learning in the course. We were

interested in any impact the initiative may have on a student’s attendance.

Once of the concerns voiced was that students may not show up to class

because the videos are available to them. This was unfounded in this study,

as 85% of the students said the video(s) had “no effect” on their attendance.

We asked students a series of eight questions designed to get their

attitudes, opinions, and experiences with using the software and videos. The

results of the eight questions are in Figure 2.

0

20

40

60

80

100

120

Figure 2: Student Perceptions/Experiences

Strongly Agree Agree Neutral Disagree Strongly Disagree

The last two questions asked of students were to provide any

suggestions to other students on how to use the videos to their benefit, and

provide any suggestions on how to improve using the videos in the future. A

17

number of the suggestions to students were similar and, therefore,

aggregated for the purposes of this report. Sixty five students (43.3%)

responded to the question, and the trend focused on two main categories: 1)

Review of material; and 2) “How to get the most from the video.” The vast

majority of comments were about using the videos to review concepts,

theories, and techniques prior to quizzes and exams. Since quizzes and

exams were the main method of assessment in the participating classes, this

is interpreted as a reinforcement of the benefit and importance of having

these videos available. Many students took this opportunity to provide

advice to other students on how to get the most from the videos. Some of

the suggestions were to “make sure you have your notes with you when you

watch the video,” “watch the whole video—sometimes the meat is at the

end,” and “pay attention because teachers say things in the videos that they

don’t in class.”

Of the 51 (34%) students who provided suggestions for improving the

use of the videos, 42% addressed the length of the videos. Students thought

the videos should be “short and focused on one topic,” and “integrated into

the class discussion more.” Suggestions for improvement also addressed the

quality and ease of access to the videos via the Blackboard system.

However, a few of the suggestions focused on the Panopto system itself.

Finally, there were eight (16%) students who thought that the student

should be able to see the Professor point out certain things or draw certain

elements when the video is combined with a PowerPoint presentation.

18

Student Assessment Information (Individual Classes)

Complete statistical information on student assessment outcomes was

submitted by two professors, Drs. Kump and Adams. Dr. Kump found that

VLC usage did not have a statistically significant effect on the quiz or exam

scores of students. While both quiz and exam scores were higher for those

students who watched the videos versus those who did not, the p-values

were not below 0.05. Dr. Kump compared the mean GPA of students (video

watchers and non-watchers), with the score received at the end of the

course. As expected, there was a strong correlation (r2 =0.6) between the

students’ GPA and the grade received at the end of the course (p < .001).

Additionally, the mean GPA (2.7) was significantly higher (p <.001) for

those who reported watching at least one video than those who did not

watch any. Dr. Adams had different results for the quizzes and exams in his

class. Those students who viewed at least one video averaged significantly

higher scores than those who did view any videos (83.9% vs. 70%,

respectively) t(43) = 4.02, p <.01. Additionally, Dr. Adams reports

significant improvement in the final exam scores of students (to which

videos were available) than the first three exams.

Panopto statistics were obtained on four professors: Bhat, Stanley,

Overholt and Porter-Kelley. After examining viewing data, it was

determined that VLC usage did not have a statistically significant effect on

the assessment scores of students in the individual classes. T-tests (students

who viewed at least one video vs. students who did not view any videos)

19

reveal differences in exam and quiz scores for students in these classes;

however, none of the differences were significant at p ≤ .05.

Looking at the aggregated VLC usage and assessment data, we see

that most students did not view the first video in any of the classes.

However, the number of students viewing videos increased significantly

from the first video to the second video, t(69) = 1.97, p =.05. It is unclear

why the increase in number of student viewing occurred, but student ‘word-

of-mouth’ and faculty encouragement may have contributed to the increase.

Following along with the increased number of students viewing the second

video, a statistically significant trend was found with number of minutes

viewed (Video 2) and corresponding assessment score (Assessment 2), F (1,

69) = 5.28, p <.05. This significant finding was not observed with any other

assessments or videos. The number of students viewing continued to

increase from the second to third videos—although not significant. The

student numbers began to decrease around each professor’s fourth video.

All participating faculty had four or five videos except Dr. Porter-Kelley,

who had eight videos for students to view.

Faculty Perceptions

Faculty who participated in the initiative were asked to complete a

Faculty Assessment Form to gather feedback about their perceptions of

using the software, how they think the performance of students in class was

impacted, and any of their suggestions for future use. Many of these

questions were the same in the Final Student Assessment—just re-worded.

20

Seven of the faculty members completed the assessment—although only six

completed most of the questions. All of the faculty (100%) recorded five or

more lectures. Most of the faculty (42.9%) indicated they spent more than

30 minutes on each lecture, while 28.6% spent between 21-30 minutes, and

the remaining 28.6% spent less than 20 minutes. All faculty (100%) said

they informed all of their students about the videos, and showed all of the

students how to access the videos. We encouraged all faculty members to

provide incentives for their students, and most provided this in the form of

points. Three faculty members provided extra credit, one gave participation

points, and one gave assignment points. The other faculty provided no

points and/or incentives for viewing the videos. One faculty member

elaborated on his/her decision to not provide any extra motivation for using

the videos, “As I stated many times before, if one of our outcomes is

increased grades we cannot inflate the grades of the experimental group by

giving points or incentives. The incentive has to be their want to do better.”

Instructors were asked how beneficial they thought the videos viewed

were in students’ understanding the concepts related to the course, and

how valuable the videos viewed were to the students. Both of these

questions were on a 0-10 scale, with ‘0’ being the lowest (‘not at all’) point

on the scale. The majority of the faculty rated the benefit and value very

high; 85% indicated the benefit between 8-10, and the 66.6% rated the

value between 8-10. This is very different than the summer session faculty

ratings. The faculty members, during the summer session, all rated the

21

benefit and value of the videos less than 5. All but one faculty member

(85.7%) felt that the videos had “no effect” on student attendance, but one

professor felt that having the videos available “reduced student

attendance.”

We also asked the faculty members a similar series of eight questions

designed to get their attitudes, opinions, and experiences with using the

software and videos. The results of the eight questions are in Figure 3.

Videos were a convenient way for students to access material

Videos helped students to prepare for quizzes

Videos helped students to prepare for exams

Videos helped me prepare for class discussions

Videos helped me in my overall effort to present course information to students

Videos helped to clarify concepts discussed in class for students

Videos were easily accessible for students

The video technical quality was good

0 20 40 60 80 100 120

Figure 3: Faculty Perceptions/Experiences

Strongly Agree Agree Disagree Strongly Disagree No Opinion

Faculty were asked, “In your opinion were there any apparent learning/

performance differences between students who viewed certain topics versus

those who did not?”Although the majority of the faculty did not analyze

their student outcome data, two out of three (66.7%) said “yes.” A

qualitative response from a faculty was as follows: “Students who used the

22

videos regularly remarked that this should be made available in every

class.” Although this in anecdotal information, and not statistically

generated, the faculty perceptions of the learning and performance

differences must be taken into consideration. The entire faculty

participating in this study recommended the expanded, and continued, use

of the VLC.

Finally, faculty were asked to “please provide any additional comments

you would like us to consider relating to the use of Panopto at WSSU.” The

open-ended responses were summed up by the following four comments:

“I think it is very useful, especially for the students who used it. I

was disappointed in the number of students who used it. The video

portion is rather useless, especially if you move around in class.

That portion needs to upgraded.”

“It would be quite valuable in conducting distance-learning or on-

line courses or for students who miss class due to an excused

absence. As a tool for enhancing learning in traditional courses, in

my opinion it does not appear to be effective, at least in the courses

I have taught. Students do not appear to want additional lecture,

even though the videos might not technically be a lecture. Other

methods of enhancing learning may be far more valuable. In the

courses I teach, this includes studying the textbook and reviewing

the notes.”

23

“I wish the system was a little more user friendly—is there a way

the students can log on through Blackboard and not an outside

link”

Continue to use an "experimental approach" where there are some

systematic comparisons. In class video versus supplemental video,

or enrichment type presentations versus repetition (of same

material from class), etc.”

Discussion

Based on the information provided by the three student surveys,

faculty assessment data, and faculty assessment forms—it appears that the

VLC has great potential to enhance teaching and learning outcomes. We

know that those students who split their time between working and being a

fulltime student were more likely to have taken the course again because of

a DFW. Additionally, we know that these students were also more likely to

utilize the videos for instruction and review. With the large number of

undergraduate students (traditional and non-traditional) working more than

11 hours per week, this may prove to be an essential learning tool for

students. Chandra (2007) indicated that one of the primary concerns of

faculty and administrators was that students would just listen/watch to the

videos and not attend the class sessions. We found that this was not a factor

with our students—this tool did not have a negative impact on student

attendance.

24

Unfortunately, we don’t have enough statistical data to discuss the

extent to which this project enhanced teaching and learning. One of the

major concerns is the way that faculty collected student assessment data.

There was a lack of consistency in collection and reporting of final

assessment data. It was very difficult to measure whether or not the

accessibility and usage of the videos were significantly related to the

assessment scores in each class. We have descriptive and qualitative

information on all classes; however, we don’t have final, correlated,

assessment data on all students. Dr. Kump found that the videos did not

significantly increase student assessment scores, while Dr. Adams found

that it did. One of the suggestions moving forward is to have a standardized

data collection format for all faculty and classes.

Data indicated that VLC usage did not have a statistically significant

effect on the assessment scores of students in the individual classes.

However, examination of the aggregated VLC usage and assessment data

reveal that a statistically significant trend was found with number of

minutes viewed and corresponding assessment score of the second video. It

appears that most students did not view the first video in any of the classes

but viewing increased with each subsequent video—plateauing and then

declining at approximately the fourth video. Speculation is made that

perhaps it took students some time to become comfortable with the Panopto

system and/or word of mouth began to spark student curiosity. Another

25

hypothesis is that continued faculty encouragement may have prompted

students to utilize the resource.

Similar to the results of the 2009 summer session II pilot,

overwhelmingly, students thought the VLC was helpful, beneficial, and

valuable to their learning experience in the participating classes. The

majority of the responses by students, quantitative and qualitative, were

positive and in favor of keeping the VLC system in classes. In fact, several

students indicated that it should be implemented in their other classes. We

found that students who had GPAs either in the very high or low range were

more likely to watch the videos, and rate them as more beneficial and

valuable. This may speak to the fact that those students who are doing very

well or very poorly are more conscientious about their performance.

Additionally, motivation may play a major factor in how students perceive

and utilize this tool. If you are at the top of your class, you may want to

engage in additional activities to stay there. If you are not doing very well,

you may be motivated to seek additional assistance to ‘pull’ your grades up.

What still remains unknown is how to motivate and encourage those

students in the ‘middle’ of the class to take advantage of the VLC.

Other interesting findings have to do with the length, type, and

content of the videos. Students seem to enjoy the shorter, concept-focused

videos over the longer, class-related videos. Additionally, two in five

students indicated they preferred the pre-recorded videos versus the ‘live’

videos. Based on some of the feedback from students and faculty, a

26

combination of both may be the best practice. For activities such as concept

explanation and instructional techniques, shorter videos may be preferred.

However, the longer ‘live’ recorded videos may be better for exam review

and content questions. It was noted by two faculty members that the class

structure should be taken into account in deciding whether or not to

implement VLC in the course. CETL will look at the structure and course

content of the classes with the instructors to see if VLC is a viable

supplemental tool.

Finally, with any technological tool, there are some challenges with

quality and usability. Students and faculty thought the quality of the videos

were pretty good. However, there were a few comments that focused on the

accessibility of the videos through an external link. If there is a way to

access the videos through Blackboard, instead of directly through Panopto,

this may increase the usage. One comment from a faculty member was that

some students are not willing to try and access the Panopto system multiple

times.

Limitations

No study, especially a quasi-experimental one, is without limitations.

The first limitation is the lack of consistency with type and amount of data

collected. There were some professors who were able to conduct descriptive

and inferential analyses, while others were only able to produce descriptive

data. Even though the data was aggregated, two professors had the

majority of students using the software. It is unclear if accessing/using the

27

software is positively correlated with the incentive given by that professor.

Secondly, students were self-reporting and may over exaggerate their

software usage because of the desire to obtain an incentive offered by their

professor. Conversely, students may have not participated because there

was not an incentive offered by their professor. A large portion of the

sample were nursing majors and this may skew the data. Some students are

theoretically more motivated to do well in this particular class, because not

doing so would mean denied entry into a specialized program.

References

Chandra, S. (2007). Lecture video capture for the masses. ITiCSE, June 23-

27.

Hall, N. M. (2009). Video lecture capture initiative: summer pilot final

report. Winston-Salem State

University: Center for Excellence in Teaching and Learning.

Winston-Salem State University (2010). Achieving academic distinction: The

plan for student

success. Winston-Salem, NC: Office of the Chancellor

28