A Case Study on the Use of Blended Learning to EncourageComputer Science Students to Study

Diana Perez-Marın • Ismael Pascual-Nieto

Published online: 28 January 2011

� Springer Science+Business Media, LLC 2011

Abstract Students tend to procrastinate. In particular,

Computer Science students tend to reduce the number of

hours devoted to study concepts after class. In this paper, a

case study on the use of Blended Learning to encourage

Computer Science students to study is described. Further-

more, an experiment in which the reaction of 131 Com-

puter Science university students to the proposal is

analyzed. The material for the preparation of an exam was

produced both in electronic and paper formats. 64 students

were asked to study using a free-text scoring system, and

67 students were asked to study with printed documenta-

tion in the same class. The students’ reactions, the results

of a pre-post-test and the answers to a voluntary and

anonymous satisfaction questionnaire were registered.

After that, students were given the option to keep studying

with the scoring system or with the printed documentation.

99% of the students chose to study with the computer, and

a higher frequency of study was registered during the

previous month to the exam.

Keywords Blended learning � Computer science �Conceptual study � Automated essay grading

Introduction

An increasing acceptance and use of Information and

Communication Technologies (ICTs) by the population has

happened in the last decades (Hopkins 1998). In particular,

a great acceptance of using e-learning has been registered.

E-learning can be defined as the use of ICTs to deliver a

broad range of learning opportunities to enhance knowl-

edge and performance (Beadle and Santy 2008).

The use of e-learning has many benefits such as pro-

viding greater flexibility of study, giving more control to

the students over their learning and assessment process,

and generating personalized and adapted content to each

student.

On the other hand, the use of e-learning has also pre-

sented several problems such as the loss of the student–

lecturer relationship (Chung and O’Neill 1997), the lack of

motivation when students have to self-regulate their study

(Lynch and Dembo 2004), the isolation feeling when stu-

dents have to study alone in front of a computer (McElrath

and McDowell 2008), and the difficulty of legally defending

scores achieved with e-learning systems (Ford 2000).

A possible solution to those problems can be found in

the use of Blended Learning (BL) or Hybrid Learning.

BL can be defined as a combination of traditional teaching

methods with the application of ICTs for education

(Graham 2006).

The use of BL is aimed to provide both the benefits of

the use of e-learning and the benefits of traditional lectures.

Some benefits reported are the following: a higher number

of students, less costs, better logistics, and higher learning

efficacy (Singh 2003; Derntl and Motschnig-Pitrik 2005;

Kim 2007).

However, the main difficulty in setting a BL program is

to decide how to put into practice the Blended Learning

methodology (Motschnig-Pitrik 2004). In particular, in

this paper, the focus is on the low number of hours that

Computer Science students devote to study concepts after

class (Andreou 2007).

D. Perez-Marın (&)

Universidad Rey Juan Carlos, Mostoles, Spain

e-mail: diana.perez@urjc.es; diana.perez.marin@gmail.com

I. Pascual-Nieto

Universidad Autonoma de Madrid, Madrid, Spain

123

J Sci Educ Technol (2012) 21:74–82

DOI 10.1007/s10956-011-9283-6

To solve that problem, we decided to put into practice a

BL setting designed to encourage the students to study

more after class. In particular, we thought of combining the

use of a free-text scoring system with the use of printed

documentation: firstly, during one face-to-face (F2F) ses-

sion at the lab and, later by letting the students to choose to

study with the on-line free-text scoring system or with

printed documentation until the exam. We applied this BL

setting during the 2007–2008 academic year in a Computer

Science course with 131 university students.

Students attended traditional lessons during three

months using the computer to solve practical exercises. In

the last session in class, they were split so that some stu-

dents could study concepts by using a free-text scoring

system, and other students could study with printed

documentation.

The same material was produced in electronic and paper

formats. We could observe how students who were given

the printed documentation wanted to study with the com-

puter (as expected), and when they were offered the chance

to study with the computer, they were more motivated to

study than before; and similarly, the students who had

studied with the free-text scoring system felt happier to be

able to keep studying with the computer system. The stu-

dents’ reactions, the results of a pre-post-test and the

answers to a voluntary and anonymous satisfaction ques-

tionnaire were registered.

The paper is organized as follows: Section ‘Literature

Review’ overviews some related work; Section ‘Case Study’

provides the details the experiment carried out; Section

‘Results and discussion’ presents the results achieved; and

finally, Section ‘Conclusions and future work’ ends the

paper with the main ideas and lines of future work.

Literature Review

The use of Blended Learning (BL) has been advocated. It is

considered a promising pedagogical approach (Ng and Tsoi

2008). The positive effects of using BL and incorporating

ICTs in education have been reported by several

researchers (Monteith 1998; Barton 1998; Dziuban et al.

2004; Rovai and Jordan 2004; Lee and Chan 2007; Wang

et al. 2008).

Dziuban, Hartman and Moskal claimed that BL courses

had the potential to improve learning outcomes and to lower

attrition rates. According to Dziuban et al. (2004), BL

courses can be better than online and traditional courses.

Similarly, when three graduate courses (traditional,

blended, and fully online) were compared, the results of the

blended courses where better than the results of fully online

courses and similar to the results of traditional courses

(Rovai and Jordan 2004). Part-time students preferred

blended learning settings that retained some form of face-

to-face teaching and e-learning (Lee and Chan 2007), and

students’ academic results of computer programming

course are improved when using BL (Wang et al. 2008).

However, there is not a well-established procedure to set

up a BL program. Different researchers have provided

several approaches based on five blending dimensions

(Singh 2003; Garrison and Kanuka 2004; Dziuban et al.

2004; Howard et al. 2006; Mortera-Gutierrez 2006; Larson

and Murray 2008; Fong 2008):

• Offline and online learning: offline learning is based on

the traditional study in a classroom, and online learning

is based on the use of Internet.

• Self-pace and collaborative learning: the study in self-

pace learning is controlled by each student working on

his/her own, while the study in collaborative learning is

shared among a group of students working together.

• Structured and unstructured learning: in structured

learning there is a premeditated program with orga-

nized content in sequence, while in unstructured

learning there is not such a program available.

• Custom and off-the-shelf content: Custom content is

specifically created for the course, while off-the-shelf

content is generic.

• Theory and just-in-time performance support: Theory

support is organized prior the beginning of the course,

while just-in-time performance support is provided

during the course just when the students demand it.

Case Study

Participants

The participants in this study were 131second-year Spanish

Computer Science university students, taking a compulsory

course entitled ‘Operating Systems’ in one semester (from

February to June). 102 (77.9%) were male students and 29

(22.1%) were female students. Their mean age was 20

(SD = 1.2, range = 19–23). None of the students had any

kind of disability. These features are generally represen-

tative of the population of second year Spanish Computer

Science university students.

Since the beginning of the course, the students were

separated into eight groups of 16 students in average

(SD = 7, range = 7–27). Each group had a different

timetable, but all the students worked in the same room.

Students worked in pairs, although each student could use a

different computer.

79.4% of the students had similar prior knowledge, as

analyzed from their scores and enrolment in previous

courses. Only 27 students (77.8% male, 22.2% female)

J Sci Educ Technol (2012) 21:74–82 75

123

were different, with outstanding scores, and they were

separated forming a special group of gifted students.

Experimental Design

A controlled experiment between subjects was designed, in

which the independent variable was the use of the com-

puter to study, and there were two dependent variables:

• The numerical scores achieved in a pre-post-test

performed before and after asking the participants to

study with/without the computer.

• The answers to a satisfaction questionnaire.

The decision of performing a between subject experi-

ment was made to avoid unwelcome interference effects

across conditions. The control group studied with the

printed documentation, and the test group studied with the

computer application. The separation into control and test

groups was randomly made. Table 1 shows the distribution

of the number of students in control and test groups.

Apparatus/Instruments

The control group students were provided with a printed

document of four pages. Those students were not allowed

to use the computers.

The test group students were provided with one PC with

Linux and Internet connection. The educational tool used

was Willow (Perez-Marın et al. 2007; Pascual-Nieto et al.

2008). Those students were not allowed to read the printed

documentation.

Willow is a free-text scoring system for conceptual

formative assessment. It provides immediate feedback to

each question answered by the student by comparing the

student’s answer to a set of correct answers previously

introduced by the lecturer.

Figure 1 shows a snapshot of the interface of Willow.

The interface emulates a student-system dialogue in an

attempt to be friendly.

Willow was chosen because it is the system that we have

developed. Thus, it is available without cost. All the same,

we believe that any other free-text scoring system with

similar feedback possibilities could also be used, and the

outcome of the experience would be the same. Please, see

Perez-Marın et al. (2009) for a list of other possible free-

text scoring systems that could be used.

Experimental Procedure

In this section, it is described the Blended Learning setting

put into practice for the case study. First of all, the lecturer

provided some answers to the following questions:

1. Think about the project/exam for the assessment of the

course.

The project for my students will consist in developing

a Linux shell. There will also be a final exam on June

for individual conceptual assessment.

2. How many students will be in each group?

The students will work in pairs.

3. Which is the role of each student in the group?

Each member of the group will be responsible for

some commands of the shell, and finally all the

commands should work even when they are called

combined.

4. How many F2F sessions will be per week?

There are going to be a 2-h F2F session per week as

established by the official timetable of the Faculty. The

F2F sessions will be from February to May. Figure 2

shows the course timeline.

5. In which website the students will be able to download

the materials?

The students will be able to download the materials

from the course web page.

6. How many times will I read the mail and answer

doubts?

The mail will be read twice a week on Mondays and

Wednesdays.

Table 1 Distribution of the

number of students in control

and test groups

* The outstanding students

belong to group 2A

Group Time Room Control

group (paper)

Test group

(computer)

Total

3A Monday 11:00–13:00 Lab 6 3 4 7

1A Monday 15:00–17:00 Lab 6 9 10 19

2B Tuesday 15:00–17:00 Lab 6 10 11 21

2A* Wednesday 12:00–14:00 Lab 6 15 12 27

3B Wednesday 10:00–12:00 Lab 6 4 4 8

2C Wednesday 15:00–17:00 Lab 6 12 10 22

1B Friday 9:00–11:00 Lab 6 7 6 13

1C Friday 15:00–17:00 Lab 6 7 7 14

76 J Sci Educ Technol (2012) 21:74–82

123

7. Can tutorship be offline and/or online via web

conference?

Offline tutorships will be on Friday morning from

11:00 to 13:00. Online tutorships will be agreed

between the students and the lecturer by mail.

All students used the computer to solve Practical

Exercises (PE). At the beginning of May, face-to-face

sessions stopped and the experiment was carried out during

the last F2F session of each group.

Students were asked to participate voluntarily during the

2 h of the class. No incentives or extra course credit were

offered in return for participation. On the other hand, the

experience was presented to the students as part of our

effort to find out best practices in education.

First of all, students were given a 5-min introductory

talk to Willow, because none of them knew the system or

how to use it. Students were also told that during the

session in the lab they would be randomly assigned to a

group in which they could only study with the printed

documentation, or with the computer using Willow.

Finally, all students were asked to complete a pre-test

with five multiple-choice questions (with four options

each) related to the last lesson studied in class. They were

given 20 min to complete the test. The lecturer told the

students that the score achieved in the test would only serve

to measure how much they have learnt in the study session

by comparing the score of the pre-test with the score of a

post-test to be done after the session in class.

When the pre-tests were gathered, each group of stu-

dents was randomly split into control and test groups to

study the same lesson for the final exam during 1 h. The

only difference was that the control group was asked to

study with the printed documentation provided by the

lecturer, while the test group was asked to study using

Willow. In both cases, the study was focused on the lesson

of the pre-post-test.

All the students of the control group were sat on the left

tables of the lab, and all the students of the test group were

sat on the right tables. The same researcher was in all the

sessions observing the reactions of the students.

Once the time of the study session finished, students

were asked to complete the post-test during 20 min.

Finally, students were asked to fill in a satisfaction ques-

tionnaire and handed in before leaving the room.

From the session in the lab to the exam, students could

choose to study using Willow or to ask for more printed

Fig. 1 Sample snapshot of

Willow

Fig. 2 Course timeline

J Sci Educ Technol (2012) 21:74–82 77

123

documentation. Table 2 shows how this BL setting covers

the five learning dimensions described in the Literature

Review.

Course Involved

The course was a semester-long, six credits compulsory

class, targeted at second-year university students in a

Computer Science Faculty. The course focused on the

concepts of Operating Systems, and how to program cer-

tain features of Operating Systems. Thus, students were

requested to solve programming exercises with the com-

puters, and answer conceptual questions in Operating

Systems.

During the experiment in class, the content of the

printed documentation and the lesson in Willow was the

same, 20 questions with two correct answers per question

in average (around 1,000 words in total).

The documentation was printed both sides in three

pages: the first page with five questions, the second page

with eight questions and the third page with seven ques-

tions. In Willow one question is shown at each time.

Willow also had other four lessons for the same course,

each one with 16, 15, 14 and 20 questions and 2 correct

answers per question in average. Students were told that

they could ask for the printed versions of those lessons if

they want to keep studying with printed documentation.

Evaluation

According to some authors, teaching effectiveness should

be ‘‘demonstrated by students’ learning of what has been

taught’’ (Simonson et al. 1998). In these studies, both

student grades and test scores serve as measures of student

achievement (Hoyt 1999). Taking that into account, the

measure chosen to check which type of study can be

highlighted as more effective is the difference between the

post-test and pre-test scores.

Each test was given a score in a 0–5 scale (0 indicating

no knowledge, and 5 complete knowledge). Therefore, the

scale of pre-post differences goes from -5 (worst case) up

to 5 (best case). A higher positive difference is used as an

indicator of a more effective type of study.

The metrics for the perceived satisfaction are the fol-

lowing: direct observation of the reactions in class, results

of the satisfaction questionnaire, and frequency of study

with the free-text scoring system or with the printed doc-

umentation until the exam in June. Positive students’

comments, higher scores in the satisfaction questionnaire,

and higher frequency of study are interpreted as indicators

of a more satisfying type of study.

Results and Discussion

Pres-post-test

Table 3 gathers the differences in the post-test and pre-test

scores for each group and for all the groups. There was a

higher increase of the post-test score when the study was

done with the computer in six out of the eight sessions

(75%).

However, according to conventional criteria, none of

those differences can be considered statistically significant.

Similarly, no statistically significant differences have been

found between the 2A group with the outstanding students

and the rest of the students. These results are, as expected,

because our goal was not to guarantee that after 1 h of

study the students will be able to have higher scores, but

just the opposite: it is necessary to study more before the

exam and during several weeks.

Table 2 Learning dimensions covered by the BL setting

Learning dimension Application

Offline and online learning The lecturer sets from the beginning the number of F2F sessions and provides online materials on the web.

Besides, s/he clearly establishes the online and offline communication channels

Self-pace and collaborative

learning

The lecturer creates the groups in which each student has a role to play and all the students have to work

together to prepare their project. Furthermore, each student can individually download the materials and

study with the free-text scoring system to study at his/her own pace

Structured and unstructured

learning

Although the lecturer clearly sets the rules from the beginning of the course, s/he should also be flexible

enough to react to unexpected events

Custom and off-the-shelf content When uploading the materials to the web the lecturer should not only use his/her notes but also links to other

generic documentation. Similarly, the questions for the free-text scoring system could be provided not only

by the lecturer but from previous exams

Theory and just-in-time

performance support

At the beginning of the course, the lecturer has to provide great support to the students so that they learn the

basics to start working. Later, s/he should only help when students find specific problems that prevent them

to keep working on their project

78 J Sci Educ Technol (2012) 21:74–82

123

Students’ Reaction

The first reaction observed is that students assigned to the

control group complaint more than students assigned to

the test group. Some comments made by the students are

the following: ‘You are lucky you are allowed to study

with the computer!’, or ‘If I finish reading the paper soon,

will I be able to study with the computer?’

It was also noticed that the time devoted to the study

with the paper was shorter: 20 min (SD = 5.2 min) in

contrast to the hour used by the students who studied with

the computer to pass all the questions in the lesson.

Therefore, we suspected that students were not carefully

reading the printed documentation. Hence, we decided to

introduce three mistakes on purpose in the printed docu-

mentation, so that students did not claim that they have

finished reading the document, at least, until they have

found the three mistakes.

All the same, the great majority of students in the con-

trol group claimed that they had found the mistakes in less

than 30 min. Moreover, they would just skim the text to

find the three mistakes as soon as possible, and they asked

if they could start studying with the computer. On the other

hand, all students who used the computer were engaged the

whole hour in the study, and some of them even asked to

stay longer.

No differences were observed for the case of the 2A

group of the outstanding students.

Satisfaction Questionnaire

All the students completed the questionnaires voluntary

and anonymously, although some of them left one or more

items in blank.

The satisfaction questionnaire consisted of five items:

four closed-answer items (two of them with three options,

and two of them with two options) and one open-answer

item.

The first and second items were related. They were

focused on finding out the students’ belief about the

computer as a tool to study in general (first item), and to

study concepts (second item).

The third and fourth items were also related. They asked

for the students’ self-reported preference towards studying

with the computer or with printed documentation. In the

case of the third item the question was what option they

would have preferred. In the case of the fourth item, the

question was if they intended to keep studying with the

printed documentation or with the computer.

Finally, the fifth item asked the students to provide any

comment they may have about the use of computers to

study after class.

Figures 3a–d show the graphs with the results.

81% of the 131 surveyed students answered that they

believe that using the computer to study is good. 10% of

the students stated that it was indifferent to them because

they did not have time to study after class. 9% of the

students left the answer in blank. No student answered that

using the computer to study was negative.

The result is similar to the second item, in which 87% of

the 131 surveyed students stated that they believe that the

computer is a good complement for conceptual study. Only

two students stated that they believe that the computer is

not a good complement for conceptual study. The reason

given is that if artificial intelligence is used and it has

errors, then it could induce students to make mistakes.

The results of the third item explain the complaints

observed in the reactions of the students in class: 65%

(more than the half) of the 131 students would have pre-

ferred to have studied with the computer instead of reading

the printed documentation.

For the future, 81% of the 131 students stated that they

intended to study with the computer instead of using the

printed documentation. Only one student asked for the

printed documentation.

Finally, regarding the open-answer item about the gen-

eral opinion of the students, the three more representative

Table 3 Unpaired t test with

two-tailed P for control and test

groups

Group Control group (study with paper) Test group (study with computer) Statistically

significant?

N Mean SD N Mean SD t test

3A 3 0 0.66 4 0.88 0.60 No

1A 9 0.94 0.77 10 0.85 1.08 No

2B 10 0.70 0.70 11 0.77 0.49 No

2A* 15 0.58 0.89 12 0.6 0.73 No

3B 4 0.28 0.59 4 0.5 0.35 No

2C 12 1.33 1.13 10 0.93 0.92 No

1B 7 0.55 0.70 6 0.58 0.89 No

1C 7 0.67 0.5 7 1.71 1.37 Not quite

ALL 67 0.74 0.86 64 0.85 0.89 No

J Sci Educ Technol (2012) 21:74–82 79

123

positive comments (out of the 40 positive comments pro-

vided by the students) were the following:

• ‘‘I like it because it helps me focusing on each question,

and to get immediate feedback’’.

• ‘‘It is a funny way to study, thanks!’’

• ‘‘I think that it is a great idea that should be extended to

other courses’’

Only 12 negative comments were provided by the stu-

dents, and all of them were addressed to the specific free-

text scoring system used. For instance:

• ‘‘The automatic free-text scoring should be improved.’’

• ‘‘Willow should be more intelligent because it does not

understand all answers.’’

It has not been found differences in the comments or

results provided by the 2A group of outstanding students

with respect to the rest of the students.

Frequency of the Study

Willow is able to register whenever each student answers a

question in the system. Figure 4 shows a graph in which

each bar corresponds to a day after the lab session and

before the final exam, and its length represents how many

questions were answered by the students that day.

The frequency of study during the first weeks was lower,

as can be expected because students tend to procrastinate in

their study. However, and unlike with traditional study

without the computer, students did not procrastinate until

only days before the final exam, but a higher frequency of

study was registered during the whole previous month to

the exam.

Furthermore, 99% of the students asked for a Willow

account so that they could study with the computer. Only

one student asked for the printed documentation for the rest

of the topics for the exam, and he also asked for a Willow

account.

This result has been found both for the students in the

2A outstanding group and the rest of the groups without

significant differences.

Conclusions and Future Work

Blended Learning can be used with Computer Science

students to encourage them to study after class during the

course.

A case study in which 131 Computer Science university

students were asked to study their Operating Systems

course following a Blended Learning setting has been

carried out. The results of the study have revealed higher

levels of engagement and higher frequency of study.

99% of the students asked to study with the computer

after class. Only one student asked for the printed docu-

mentation. These results are the same both for average

students and for outstanding students able to get the highest

scores in the exams.

It is our intention to keep researching into how this

Blended Learning setting can be adapted to other domains

Do you believe that using a computer to study isa good or a bad idea?

0102030405060708090

Good idea Indifferent Bad idea In blank

Do you consider that the computer is a good complement for conceptual study?

0,010,020,030,040,050,060,070,080,090,0

100,0

Yes No In blank

In the lab session, would you have preferred to:

0,0

10,0

20,0

30,0

40,0

50,0

60,0

70,0

Study with thecomputer

Study with the paperdocumentation

In blank

For the exam, do you intend to

0,010,020,030,040,050,060,070,080,090,0

Study with thecomputer

Study with the paperdocumentation

In blank

a

b

c

d

Fig. 3 a Results of the first item of the satisfaction questionnaire,

b Results of the second item of the satisfaction questionnaire,

c Results of the third item of the satisfaction questionnaire, d Results

of the fourth item of the satisfaction questionnaire

80 J Sci Educ Technol (2012) 21:74–82

123

to help other students to develop some study skills and

avoid procrastination. Furthermore, we would like to repeat

the experience with other free-text scoring systems to test

whether the outcome of this experience changes according

to the system used.

Acknowledgment We would like to thank the teachers and students

who participated in the case study, and the reviewers and language

checkers that have helped us to improve this paper.

References

Andreou C (2007) Understanding procrastination. J Theor Soc Behav

37:183–193

Barton R (1998) IT in practical work: assessing and increasing the

value added. In: Wellington J (ed) Practical work in school

science. Which way now?. Routledge, London, pp 237–252

Beadle M, Santy J (2008) The early benefits of a problem-based

approach to teaching social inclusion using an online virtual

town. Nurse Educ Pract 8(3):190–196

Chung G and O’Neill H (1997) Methodological approaches to online

scoring of essays. Technical report 461. UCLA National center

for research on evaluation, student standards, and testing

Derntl M, Motschnig-Pitrik R (2005) The role of structure, patterns,

and people in blended learning. The Inter High Educ 8(2):

111–130

Dziuban CD, Hartman JL, Moskal PD (2004) Blended learning.

EDUCAUSE Center for Applied Research Bulletin, 2004 (7).

On-line at: http://www.educause.edu/ECAR/BlendedLearning/

157515 (last visit 28th September 2010)

Fong J (2008) Web-based logging of classroom teaching activities for

blended learning. Advances in web based learning, LNCS 4823.

Springer, Heidelberg, pp 597–605

Ford (2000) Automated scoring of writing assessments, powerpoint

presentation for the Assessment Council. On-line at: http://www.

ipacweb.org/conf/00/ford.pdf (last visit 28th September 2010)

Garrison DR, Kanuka H (2004) Blended learning: uncovering its

transformative potential in higher education. Int High Educ

7(2):95–105

Graham C (2006) Blended learning systems. In: Bonk CJ, Graham CR

(eds) The handbook of blended learning: global perspectives,

local designs. Pfeiffer, San Francisco

Hopkins, C. 1998. Web?qmark?humanities=? A case study. Com-

puters and Texts, 16/17. On-line at: http://users.ox.ac.uk/

*ctitext2/publish/comtxt/ct16-17/hopkins.html (last visit 28th

Sep 2010)

Howard L, Remenyi Z, and Pap G (2006) Adaptive blended learning

environments. International conference on engineering educa-

tion, San Juan, Puerto Rico, pp. 23–28

Hoyt J (1999) Does the delivery method matter? Comparing

technologically delivered distance education with on-campus

instruction. Department of Institutional Research. On-line at:

http://www.uvu.edu/iri/pdfs/research/distpub.pdf (last visit 28th

Sep 2010)

Kim W (2007) Towards a definition and methodology for blended

learning. In: Fong J, Wang FL (eds) Blended learning. Prentice

Hall, Pearson Education, New Jersey, pp 1–8

Larson R, Murray M (2008) Open educational resources for blended

learning in high schools: overcoming impediments in developing

countries. J Asynchronous Learn Netw 12(1):1–19

Lee P, Chan FT (2007) Blended learning: experiences of adult learners

in Hong Kong. In: Fong J, Wang FL (eds) Blended learning.

Prentice Hall, Pearson Education, New Jersey, pp 79–87

Lynch R, Dembo M (2004) The relationship between self-regulation

and online learning in a blended learning context, international

review of research in open and distance learning 5(2). On-line at:

http://www.irrodl.org/index.php/irrodl/article/view/189/271 (last

visit 28th September 2010)

McElrath E, McDowell K (2008) Pedagogical strategies for building

community in graduate level distance education courses.

J Online Learn Teach 4(1). On-line at: http://www.pedagogy.ir/

images/pdf/pedagogical-strategies.pdf (last visit 28th Sep 2010)

Monteith M (1998) Information technology for learning enhance-

ment. Sweets and Zeitlinger Publishers, Exeter

Mortera-Gutierrez F (2006) Faculty best practices using blended

learning in e-learning and face-to-face instruction. Inter J

E-learning 5(3):313–337

Motschnig-Pitrik R (2004) An action research based framework for

researching and assessing blended learning. World conference

on educational multimedia, hypermedia and telecommunication,

Lugano, Switzerland, pp. 3976–3981

Ng P, Tsoi RH (2008) A study on the students’ perception of adopting

blended learning at sub-degree level international conference in

hybrid learning. City University of Hong Kong, Hong Kong,

pp 78–88

Pascual-Nieto I, Perez-Marın D, O’Donnell M, Rodrıguez P (2008)

Enhancing a free-text adaptive computer assisted assessment

system with self-assessment features. IEEE International con-

ference on advanced learning technologies, Santander, Spain,

pp. 252–256

Perez-Marın D, Pascual-Nieto I, Alfonseca E, Anguiano E, Rodrıguez

P (2007) A study on the impact of the use of an automatic and

adaptive free-text assessment system during a university course.

In: Fong J, Wang FL (eds) Blended learning. Prentice Hall,

Pearson Education, New Jersey, pp 186–195

Perez-Marın D, Pascual-Nieto I, Rodriguez P (2009) Computer-

assisted assessment of free-text answers. Knowl Eng Rev

24(4):353–374

Rovai AP, Jordan HM (2004) Blended learning and sense of community: a

comparative analysis with traditional and fully online graduate

courses. International review of research in open and distance

learning 5(2). On-line at: http://www.irrodl.org/index.php/irrodl/

article/view/192/274 (last visit 28th September 2010)

Simonson M, Schlosser C, Hanson D (1998) Theory and distance

education: a new discussion. In: Gillespie KH (ed) The impact of

Fig. 4 Frequency of study

using Willow

J Sci Educ Technol (2012) 21:74–82 81

123

technology on faculty development, life and work. Jossey Bass,

San Francisco, pp 60–75

Singh H (2003) Building effective blended learning programs. Educ

Technol Mag 43(6):51–54

Wang FL, Fong J, Choy M, Wong T (2008) Blended teaching and

learning of computer programming. International conference on

Web-based Learning, LNCS 4823, Springer, Edinburgh,

pp. 606–617

82 J Sci Educ Technol (2012) 21:74–82

123