running head: ict-assisted collaborative project …

Running head: ICT-ASSISTED COLLABORATIVE PROJECT-BASED LEARNING

The Underutilization of Internet and Communication Technology-assisted Collaborative

Project-Based Learning Among International Educators: A Delphi Study

Barry S. Kramer

Andrew E. Walker

Jennifer M. Brill

ICT Assisted Collaborative Project-Based Learning

2

Abstract

This study explores the barriers associated with teachers implementing Internet and

Communication Technology-assisted Collaborative Project-based Learning (ICTCPrjBL)

as a classroom teaching methodology with students. We used a Web-based Delphi

method to engage experienced educators in anonymous consensus building consisting of

three rounds of surveys. The Round 1 analysis yielded 51 barriers. The Round 2 analysis

produced descriptive statistics (range, mean, and standard deviation) on the importance of

each barrier. The Round 3 analysis confirmed 16 of the 51 (31.4%) barriers as

“moderately significant” to “very significant” to implementing ICTCPrjBL. Important

contributions of this study include: (a) identification of barriers to implementing

ICTCPrjBL that can inform the literature and promote greater utilization throughout the

educational community and (b) a cross comparison of barriers between North America,

Eastern Europe, and Africa to examine regional differences.


3

Investigating the Underutilization of Internet and Communication Technology-assisted

Collaborative Project-based Learning Among International Educators

The idea of the Internet as a community where learners could come together,

interact, and share knowledge was envisioned as a way to transform the field of education

(Kozma & Schank, 1998). The initial promise of introducing the information

superhighway into education brought with it the hope that global learning networks could

be established in which students could participate in intercultural collaborative learning

tasks (Cummins & Sayers, 1995). This led to the establishment of new teaching

methodologies such as Internet and Communication Technology-assisted Collaborative

Project-based Learning (ICTCPrjBL) that combine the use of Information and

Communication Technology (ICT), online student collaboration, and the constructivist

orientation of project-based learning (PrjBL) (Moursund & Smith, 2000).

Research has shown benefits derived from using PrjBL in a classroom

(Blumenfeld et al., 1991). There is also a good deal of research on the benefits of using

ICT for instructional purposes (British Educational and Communication and Technology

Agency, 2002). Furthermore, there is research that looks at the benefits of using ICT for

collaborative learning between classrooms (Moursund, Bielefeldt, & Underwood, 1997).

Yet despite these findings, ICTCPrjBL is one of the least used methods among teachers

who are characterized by their innovative use of technology (Kozma & Anderson, 2002).

ICT is the merging of computing and communications (Ducatel & Webster,

2000), but it can also be thought of any “activities that contribute to the display,

processing, storing, and transmission of information through electronic means” (Bruneau

& Lacroix, 2001, p. 4). BECTA, the British Educational Communications and


4

Technology Agency (2004), recommended the need for research that specifically looks at

ICT-related teaching methodologies in a study that investigated barriers and enablers that

affect the use of ICT by teachers. Their study concluded that there is little research that

looks at barriers that exist in specific areas of educational ICT use as well as barriers that

affect practitioners in specific roles. Furthermore they recognized a need to seek out

examples of ways in which educators have overcome barriers and have successfully

integrated specific types of ICT use.

The Value of Project-based Learning

PrjBL is an instructional model that “engages students in learning knowledge and

skills through an extended inquiry process structured around complex, authentic

questions and carefully designed products and tasks (Markham, Larmer, & Ravit, 2003,

p. 3)” Students are encouraged “to solve challenging problems that are authentic,

curriculum-based, and often interdisciplinary” (Solomon, 2003, p. 1). The nature of these

tasks frequently requires collaboration and teamwork. One factor that motivates students

is having the opportunity to share their findings in presentations to their peers and

community (The George Lucas Educational Foundation, 2004). A research report by

Moursund et al. (1997) credited PrjBL with the promotion of increased student

motivation, problem-solving ability, collaboration, resource-management skills, and

improved library research skills. Thomas (2000) identified research that showed gains in

student achievement, problem solving capabilities, understanding of subject matter, as

well as other specific skills that were directly related to PrjBL tasks.

PrjBL has been used in conjunction with ubiquitous laptop initiatives in the area

of Geography (Grant & Branch, 2005). It has also been used alongside Computer


5

Mediated Communication tools in high school (Lang, Peer, & Divaharan, 2005) in

middle level science (Rivet & Krajcik, 2004), and as a means of technology integration

for pre-service teachers (Gubacs, 2004). Thus, project-based learning appears to be an

effective (Stites, 1998) and well-utilized curriculum tool for promoting student learning.

Student’s Use of Internet and Communication Technology

Since The International Society for Technology in Education (ISTE) published

the National Educational Technology Standards for Students (1998) communication

technology has become a necessary component of a student’s educational experience.

Various uses of communication technology are found as performance indicators in the

recommended “Profiles for Technology Literate Students” at every age bracket. In

addition to student use, technology communication tools are one of the six major

standards for teaching practice. The use of ICT standards is not limited to those published

by ISTE. Similar standards regarding the educational use of ICT have been adopted by

Australia (Educational Network Australia, 2004), The United Kingdom (Qualifications

and Curriculum Authority, 2001), Ireland (National Center for Technology in Education,

2003), Canada (Government of Alberta, Canada, 2004; Province of British Columbia,

Canada, 2001), Poland (Ministry of National Education of Poland, 2001), Hungary

(Ministry of Education: Republic of Hungary, 2004), Russia (UNESCO Institute for

Information Technologies in Education, 2004) and many countries in Africa and Asia

who use the standards established by the United Nations Educational, Scientific and

Cultural Organization (UNESCO Asia and Pacific Regional Bureau for Education, 2003).

The Department for Education and Skills in England concluded that the “weight

of evidence suggested clearly” that “ICT provision and pupil ICT use do impact


6

positively on pupil attainment and school standards” (Pittard, Bannister, & Dunn, 2003,

p. 17). BECTA (2002) identified that the value of using ICT for teaching and learning

could be measured by increased student skills in motivation, presentation, questioning,

problem solving, information handling, and techniques of modeling. Law and Chow

(2002) in a study of primary school children in Hong Kong concluded that the most

important outcome of learning practices that used ICT was the empowerment it provided

teachers and students. Thus, educators across the world continue to ascribe value to the

use of ICT in student learning.

Internet and Communication Technology and Collaborative Project-based Learning

New pedagogical methodologies that combine collaborative PrjBL and ICT teach

students to work together, learn from each other, and share knowledge by developing

powerful online learning communities (Gordin, Gomez, Pea, & Fishman, 1996).

Cifuentes, Murphy and Davis (1998) found that when high school classrooms

collaborated using the Internet, students demonstrated an increase in self-esteem,

academic achievement, and multicultural understanding. Cifuentes et al. also found that

students grew personally and intellectually, felt empowered to achieve goals, became

comfortable with technology, provided and/or received mentorship, and learned from

each other.

Gragert (2000) contended that primary through high school students were more

motivated to learn as a result of engaging in collaborative online project work. He also

summarized teacher testimonials concluding that online collaborative project work

heightens student interests in subject content, language skills, motivation, and

opportunities for action from learning. Schultz-Zander, Butcher, and Dalmer (2002)


7

researching student cooperation concluded that problem-oriented learning facilitated by

ICT promotes students in primary, lower and upper secondary levels in teaching each

other, functioning as a learning community, and collaborating in joint partnerships with

other schools. Collaborative PrjBL facilitated by ICT has been identified as an innovative

classroom practice (Andersen, 2002) and educational projects that use ICT are an

established form of collaborative PrjBL and are found in school educational programs

throughout the world (Moursund & Smith, 2000).

Underutilization

Two major studies have shown a low usage among classroom teachers of ICT for

collaborative PrjBL-type activities. A report by the National Center for Education

Statistics (2000) found that when computer use among teachers in public schools was

examined, ICT use that involved student collaboration was the lowest of the assessed

categories at 7%. More recently, Kozma (2003) found that ICT use within innovative,

technology-rich school settings supported student project collaboration only 17% of the

time.

There is a substantial research base on the barriers that have prevented educators

from using innovative technology rich teaching practices in general (see for example,

Fabry & Higgs, 1997; Preston, Cox, & Cox, 2001; Riel & Becker, 1999). Studies also

exist specifically on the obstacles associated with the use of ICT in teaching (British

Educational and Communication and Technology Agency, 2004; Butler & Sellburn,

2002; Cox, Preston, & Cox, 1999). Barriers that frequently appear in assessments of

technology use among teachers include the lack of: Reliable Internet access (Pelgrum,

2001), teacher preparation time (Fabry & Higgs, 1997), time in student schedules


8

(Drenoyianni & Selwood, 1998), support regarding ways to integrate technology

(VanFossen, 1999), and training opportunities (Veen, 1993).

It is likely that some of the obstacles that are associated with the barriers to

adoption of technology teaching and teaching with ICT are the same as those associated

with ICTCPrjBL. Still, there has been little research that looks specifically at the barriers

that exist for ICTCPrjBL. The purpose of this study is to research this specific area of

ICT use to identify barriers teachers have encountered in implementing it as an

educational methodology. To that end, this research investigated the following questions:

• What barriers have teachers experienced in using Internet and Communication

Technology-assisted Collaborative Project-based Learning with students?

• Do these barriers vary substantially across the regions of North America, Eastern

Europe, and Africa?

With an eye towards generating recommendations for practitioners, this research also

investigated reasons for initial ICTCPrjBL adoption and success. Findings regarding

these two additional questions will be reported in future articles.

Methodology

A Web-based Delphi survey was used to collect data for this study, because it

allowed a panel of experts located in diverse geographic locations to consider an open-

ended question, present their opinions and ideas, and then work toward reaching

consensus on the effects of ICTCPrjBL barriers and the importance of successful

practices (Adler & Ziglio, 1996; Dalkey, 1969; Hemler, 1983).

Design of the Study


9

Linstone and Turoff (1975) have identified that the conventional Delphi Method

uses the following components: 1) careful development of research questions, 2)

identification of experts in the field, 3) use of a pilot study to determine the

understandability of the research questions, 4) obtainment of voluntary participation by a

panel of experts, 5) distribution of an initial questionnaire to the experts (Round 1), 6)

evaluation of the responses from Round 1 and returning the results to the experts to apply

rankings or ratings (Round 2), and 7) continuing the process until the expert panel obtains

a consensus and a final interpretive report is completed (Round 3 or more).

As is often the case, these procedures are modified to meet specific research needs

(Linstone & Turnoff, 1975; Murray & Hammons, 1995). This study amended the process

outlined above by using the Round 1 survey both to present the initial question and to

refine the panel of experts. Ultimately, all surveys were designed and delivered online.

Before each round, a small pilot study was given to 5 educators who have extensive

experience with online collaborative project work to check for functionality and clarity

(Baker, 1994).

Procedure

Developing the initial questions (Simmonds, 1977) and selecting an expert panel

(Andranovich, 1995; Bernard 1998; Delbecq, et al., 1975; Lang, 1998) are two major

concerns that every Delphi study must address. Bernard (1998) warned that in purposive

sampling, it is necessary to decide the purpose you want respondents to serve and then

develop appropriate criteria for identifying and soliciting their participation. Andranovich

(1995) emphasized that the study problem and the questions posed by the survey must


10

match the interest of the participants in order to gain participation that is meaningful.

Both of these concerns are addressed below.

Round 1

Participants were selected by screening for educators from the International

Education and Resource Network’s (iEARN) Conference (2004) because the majority of

attendees are teaching professionals from a number of countries that regularly use

ICTCPrjBL. It was our belief that the close association, dedication, and interest of the

conference participants in ICTCPrjBL practice meant that they would have an intimate

and experiential knowledge of the problem to be investigated and would invest their time

and expertise in a multi-survey research study (Delbecq, Van de Ven, & Gustafson,

1975). Since ICTCPrjBL is a relatively new teaching methodology (Moursund, 2002), we

determined that respondents who indicated that they had two or more years of experience

facilitating ICTCPrjBL and using Internet communication technologies had a level of

experience that would allow them to provide meaningful information on the

underutilization of ICTCPrjBL (Moursund, 2004). We found that the first round

participants had 6.3 average years of facilitating experience with ICTCPrjBL. Overall the

respondents had 7.6 average years of experience using Internet communication

technologies.

Teaching location was another major identifier we considered in our study. Since

our research was an international study, our sampling frame consisted of educators from

the six major populated continents. We decided to focus in on specific regions such as

Africa, Eastern Europe, and North America to determine if there were similarities and

differences in obstacles that have led to the use of ICTCPrjBL (Schmidt, Lyytinen, Keil,


11

& Cule, 2001). All other participants from Asia, Australia, Western Europe, and South

America were dropped. Africa, Eastern Europe, and North America were targeted

because there were a significant number of attendees at the iEARN Conference from each

of these regions (International Education and Resource Network, 2004) and in

anticipation barriers would vary by region. For instance, teachers in Africa have been

known to have difficulties with connectivity and Internet access; teachers in Eastern

Europe often experience resistance to change that limits their ability to implement

project-based learning activities; and teachers in North America often experience a lack

of time for training (British Educational and Communication and Technology Agency,

2004).

We sent email invitations for our Round 1 survey to 323 individuals. 294 of these

consisted of iEARN-attendees who were known to have experience with ICTCPrjBL due

to their position as a country coordinator (n =38), project leader (n =108), conference

presenter (n=113), or well-known project facilitator (n =35). The other 29 individuals

were referred by conference attendees as individuals who had a great deal of experience

with ICTCPrjBL, a sampling technique for Delphi studies used by Brill, Bishop, &

Walker (2005). We received 138 (42.7%) responses to the survey. This number

represented a large cross section of participants who had diverse experiences with

ICTCPrjBL. It also gave us the geographic diversity we were looking for among the

respondents (Cuhls, Blind, & Grupp, 2002). Table 1 provides a summary of response

rates for Round 1 and each subsequent round to assist in tracking the evolution from the

initial sample of 323.


12

Table 1. Sample Size (n), Response Frequency (f) to each Round, Response Rate by Survey Round, and Response Rate to Initial Sample. Source Sample

n Response

f Response

Rate Response

Rate To Initial Sample

Round 1 Targeted sample of iEARN conference attendees 294 116

39.5%

Referrals from other participants 29 22 75.7% Round I total 323 138 42.7% 42.7%

Round 2 Purposeful sample of experts 103 59 57.3% 18.7%

Round 3 Purposeful sample of experts 59 44 74.6% 13.6%

An analysis of the participant background data collected from Round 1 allowed us

to determine that 35 respondents did not meet our inclusion criteria, either on the basis of

expertise or because they did not come from one of the three targeted regions. The

primary researcher used the constant comparative method (Merriam, 2001) to analyze the

data collected from the remaining 103 respondents. The responses to the three open-

ended questions were hand-coded to look for recurring themes. In addition, a word

frequency count was administered as a validity check of the Round 1 analysis.

The initial analysis yielded 50 barriers. The assistance of an outside survey

analysis expert and an ICTCPrjBL expert was requested to review and provide further

validation for the data. They suggested that one barrier be divided into two different

responses to create 51 barriers. By doing this, all items that were identified by the

respondents were addressed in the final list. Our analysis of the data also yielded general

categories, such as issues regarding Internet connectivity, technology difficulties,

curriculum/program concerns, teacher training, ICT support, project-related concerns,


13

and miscellaneous items. Instead of randomizing the choices, we decided to place related

items next to each other. As an example, items associated with connectivity barriers were

grouped together so that the respondents could rate their experience with unreliable

connectivity, low bandwidth, lack of access, and the high cost of connectivity.

Round 2

In Round 2, we asked all 126 panelists from Round 1 to use a 5-point Likert scale

to rate the responses to the question on barriers based on their level of significance

(1=Not Significant, 2=Somewhat Significant, 3=Moderately Significant, 4=Very

Significant, 5=Extremely Significant). In addition, participants were given the

opportunity to add new barriers or elaborate on their responses. Of the 103 who were

asked to continue participating, 59 responded in Round 2.

The goal of the Round 2 data analysis was to baseline the degree of consensus

among the respondents regarding the barriers identified from Round 1. Descriptive

statistics were produced for each factor. A few participants supplied open-ended

comments. An analysis of these comments determined that participants either elaborated

on opinions that were already expressed or duplicated an item that already existed in the

survey.

Round 3

In Round 3 all 59 panelists from Round 2 were presented with the group mean

scores for each item from Round 1 and asked to express agreement or disagreement by

rating the items again. During this round 44 of the participants (74.6%) responded. They

recommended only minor changes in the mean ratings among all the items presented.

These final results are found in Appendix A. An examination of the inter-quartile ranges


14

between Rounds 2 and 3 showed a decrease, indicating a consensus among the experts

that is unlikely to be improved upon (Lindstone & Turoff, 1975).

Results

None of the 51 barriers achieved a mean in the “very significant” to

“extremely significant” (4.00≤ M ≤5.00) range. Twelve of the items were rated

“moderately significant” to “very significant” (3.00≤ M ≤4.00). Thirty-four of the items

were rated “somewhat significant” to “moderately significant” (2.00≤M≤3.00). And five

of the items were rated “not significant” to “somewhat significant.” Mean scores for the

barriers ranged from M=1.45 to M=3.48. Complete descriptives are available in

Appendix A.

Another way to look at the data is to do a cross comparison of mean rankings

between our three target regions. An analysis of similarity was conducted through a

reliability analysis for all of the barrier questions across all regions. A Cronbach’s alpha

was run and generated a value of .97. Initially, this seems to indicate an extremely high

level of agreement across regions with respect to ICTCPrjBL barriers. However, there are

a couple of important points: 1) by nature of the computational formula, Cronbach’s

alpha increases as the number of items increases (Kopalle & Lehmann, 1997) (51 in this

case), 2) Cronbach’s alpha increases as the inter-item correlations increase. These inter-

item correlations are robust in measuring consistency but not necessarily in measuring

agreement (Cronbach & Shavelson, 2004). Thus, it’s quite possible for the regions to

have similar trends in barriers, but different attitudes about how large the barriers are as a

whole. As reflected in Table 2, that is in fact what took place.


15

Table 2. Round 3, Question 2 - Category Means.

Barrier Categories

Composite Mean (SD) f=44

N. America Mean (SD) f=16

E. Europe Mean (SD) f=16

Africa Mean (SD) f=12

Teacher Training Issues 3.16 (1.05)

2.65 (1.16)

3.26 (0.96)

3.44 (0.85)

Technology/Technical Issues

2.97 (0.92)

2.58 (0.95)

3.00 (0.82)

3.44 (0.85)

Internet Connectivity Issues 2.80 (1.14)

2.31 (1.11)

2.80 (1.13)

3.46 (0.92)

Curriculum/Program Issues 2.69 (0.81)

2.46 (0.91)

2.68 (0.67)

3.00 (0.81)

ICT Support Issues 2.42 (1.18)

2.17 (1.01)

2.33 (1.04)

2.86 (1.51)

Project Related Issues 2.20 (0.85)

2.06 (0.80)

2.09 (0.67)

2.53 (1.09)

Miscellaneous Items 2.03 (0.82)

1.93 (0.42)

1.84 (0.67)

2.40 (1.26)

Note: 1=Not Significant, 2=Somewhat Significant, 3=Moderately Significant, 4=Very Significant, 5=Extremely Significant

The data shown above are perhaps best represented visually, which is available

below in Figure 1. Note that the trend lines, although different for each of the three

regions, are largely parallel. For almost every category, North America had the lowest

reported barriers, Eastern Europe was in the middle, and Africa reported the highest

barriers. The sole exception to this rule was North America reporting slightly higher

(M=1.93) Miscellaneous barriers than Eastern Europe (M=1.84).


16

Figure 1 – Barrier means by region

Thus the three regions agreed almost perfectly about the most pressing and least

pressing barriers to using ICTCPrjBL, but the impact of each barrier category remains

fairly relative. Technology Training, the highest barrier for North America (M=2.65) is

fairly close to the lowest reported category in Africa, Miscellaneous Items (M=2.40).

Discussion

When looking at barriers to ICTCPrjBL, it appears that the situation is complex. An

initial examination of the barriers for all participants in the composite group did not show

major obstacles for any one category, although some individual items did stand out when

individual regions were examined. As expected, the high cost of connectivity is the single

strongest factor identified by educators in Africa, only slightly edging out

Technology/Technical and Teacher Training, which are both progressively higher in

Eastern Europe and North America. Training in ICT use and ICTCPrjBL methodology

appears to spike somewhat in Europe, certainly more than the more gradual trend in both


17

North America and Africa. One item that achieved a significant rating in North America

and was supported with respondent commentary concerned inequality of access. Schools

in North America appear to find the inability to equally exchange information with

schools in other parts of the world to be a factor that limits their ability to do project

work. It is worth noting that other barriers that have appeared in previous studies did not

achieve high ratings. National examinations are often cited as an obstacle to curriculum

change (BECTA, 2004), yet this did not rate as a major obstacle in any region.

Limitations of the Study

Some participants in Africa indicated that in order to answer the survey they had to

spend their own personal funds to find a reliable connection to complete the survey.

When we were notified of this situation we were able to provide suggestions to the

participants on how to gain access to the survey by using a connection through a local

university. Nonetheless, it is likely that some potential participants were prevented from

completing the survey due to issues regarding their ability to access the survey. If that is

the case, then the internet connectivity barrier in Africa may in fact be under-reported

here.

We chose to conduct our study in English because we know that it was a language

common to all the participants (Schmidt, et al., 2001). English is the primary language

used by educators who are associated with iEARN. However, our pilot testing did reveal

that relevant terminology does vary in different regions of the world. An example of this

is that school districts are called educational agencies in the United Kingdom. We

attempted to address this potential limitation by using descriptive terms that would be

recognizable to educators in different parts of the world. An example of this would be


18

first defining and then consistently using the term ICT, which is more common in Africa

and Eastern Europe than it is in the United States. In spite of these limitations we believe

that the results of the study have important implications for research and practice.

Conclusions & Future Work

The goal of this study was to research the barriers to the current use of

ICTCPrjBL in educational practice. ICTCPrjBL is a practice that has documented

benefits in helping students to learn and acquire skills. At the same time, the research

suggests that this methodology is one of the least used among educators who regularly

use ICT. The barriers identified, whether real or perceived, do limit educators’ use of

ICTCPrjBL.

Perhaps the most interesting finding of this study is the relatively uniform

progression in barriers across three regions that are so socio-cultural diverse, both within

each region as well as between the three regions. Future work that verifies this stable

progression in the lessening of barriers across regions and over time may help justify the

generalization of models to help incorporate ICTCPrjBL in areas that are currently under-

served.

Panelist selection is the fundamental basis upon which any Delphi study is built

(Andranovich, 1995; Bernard 1998; Delbecq, et al., 1975). We chose to limit our study to

the perspectives and opinions of educators associated with ICTCPrjBL. It is likely that a

different set of barriers would have emerged if we had included students, administrators,

and other individuals associated with ICTCPrjBL. We also used a very specific

population (largely iEARN participants) to select our expert panel. There are many other


19

educators throughout the world who regularly use ICTCPrjBL whose voices may have

yielded different responses. Therefore, future work should draw from a larger population

than iEARN and include students, administrators, and others who participate in and

support ICTCPrjBL.

It appears that technological, training and connectivity barriers present obstacles

for schools in Africa to participate in ICTCPrjBL with other regions such as North

America and Eastern Europe. Consequently, schools in North America and Eastern

Europe are limited in their ability to fully participate in ICTCPrjBL with African schools,

narrowing the potential for cross-cultural collaboration and understanding, an oft-cited

intended benefit of this educational methodology. One respondent in this study

commented: “If you are really interested in project work and your students support you

and enjoy communicating with other people online, you’ll overcome difficulties.” In an

attempt to bring this sentiment to life, we plan future research on exemplary practices

using ICTCPrjBL to investigate strategies that educators have used to overcome barriers

for successful implementation. It is our hope that more educators will be motivated to

make further use of ICTCPrjBL as they see the benefits that ICTCPrjBL can bring to

learning and realize how barriers can be overcome.


20

References

Adler, M., & Erio, Z. (Eds.). (1996). Gazing into the oracle: The Delphi method and its

application to social policy and public health. London: Jessica Kingsley

Publishers.

Anderson, R.E. (2002). Guest editorial: International studies of innovative uses of ICT in

schools. Journal of Computer Assisted Learning, 18, 381-386.

Andranovich, G. (1995). Developing community participation and consensus: The Delphi

technique. Community Ventures: A Western Regional Extension Publication.

Atkinson, E. S. (1994). Key factors which affect pupils’ performance in technology

project work. In J. S. Smith (Ed.), IDATER 94. Loughborough University: Design

and Technology, 30-37.

Baker, T.L. (1994). Doing social research (2nd ed.). New York: McGraw-Hill.

Balzer, L. (2004). Internet-usage within a Delphi study. German Online Research

Conference (GOR) 2004. Retrieved October 2, 2004, from

http://www.websm.org/main./baza/baza.php?bid=857.

Bernard, H. R. (1988). Research methods in cultural anthropology. Newbury Park: Sage.

Blumenfeld, P.C., Soloway, S., Marx, R.W., Krajcik, J.S., Guzdial, M., & Palincsar, A.

(1991). Motivating project-based learning: Sustaining the doing, supporting the

learning. Educational Psychologist, 26(3 &4), 369-398.

Brill, J., Bishop, M., & Walker, A. (in press). An investigation into the competencies

required of an effective project manager: A Web-based Delphi study. Educational

Technology Research & Development.


21

British Educational and Communication and Technology Agency. (2002). ImpaCT2: The

impact of information and communications technology on pupil learning and

attainment. BECTA ICT Research, Coventry, England.

British Educational and Communication and Technology Agency. (2004). A review of the

research literature on the barriers to the uptake of ICT by teachers. BECTA ICT

Research, Coventry, England.

Bruneau, E. and Lacroix, M. (2001, November). Information sector: towards a 2007

classification. Paper presented at the ACN conference, Paris, France.

Butler, D., & Sellbom, M. (2002). Barriers to adopting technology for teaching and

learning. Educase Quarterly, 25(2), 22-28.

Cifuentes, L., Murphy, K., & Davis, T. (1998, February). Cultural connections:

Promoting self-esteem, achievement, and multicultural understanding through

distance learning. Proceedings of Selected Research and Development

Presentations at the National Convention of the Association for Educational

Communications and Technology, St. Louis, Missouri.

Cox, M., Preston, C., & Cox, K. (1999, September). What factors support or prevent

teachers from using ICT in their classrooms? Paper presented at the British

Education Research Association Annual Conference, University of Sussex at

Brighton.

Cronbach, L., & Shavelson, R. (2004) My current thoughts on coefficient alpha and

successor procedures. Educational and Psychological Measurement, 64(3), 391-

418.


22

Cuhls, K., Blind, K., & Grupp, H. (2002). Innovation for our future, Delphi ’98: New

foresight on science and technology. Technology, Innovation and Policy, 13,

Series of the Fraunhofer Institure for Systems and Innovation Research ISI,

Physica Heidledberg, 93-113.

Cummins, J., & Sayers, D. (1995). Brave new schools: Challenging cultural illiteracy

through global learning networks. New York: St. Martin’s Press.

Dalkey, N. (1969). The Delphi method: An experimental study of group opinion. Santa

Monica, CA: The Rand Corporation.

Delbecq, A., Van de Ven, A., & Gustafson, D. (1975). Group guide to nominal group

and Delphi processes. Glenview, IL: Scott, Foresman & Co.

Drenoyianni, H., & Selwood, I. (1998). Conceptions or misconceptions? Primary

teachers’ perceptions and use of computers in the classroom. Education and

Information Technologies, 3, 87-99.

Ducatel, K., Webster, J., & Herrmann, W. (Eds.). (2000). The information society in

Europe: Work and life in an age of globalization. Lanham, MD: Rowman &

Littlefield.

Educational Network Australia. (2004). Using ICT in the curriculum. The Information

and Communication Technologies (ICT) in Schools Taskforce. Retrieved October

30, 2004, from http://icttaskforce.edna.edu.au/.

Fabry, D., & Higgs, J. (1997). Barriers to the effective use of technology in education.

Journal of Educational Computing, 17(4), 385-395.


23

The George Lucas Educational Foundation. (2004). Project-based learning: At a glance.

Edutopia Online. Retrieved June 6, 2004, from

http://www.edutopia.org/php/keyword.php?id=037.

Gordin, D. N., Gomez, L. M., Pea, R. D., & Fishman, B. J. (1996, December). Using the

World Wide Web to build learning communities in K-12. Journal of Computer-

Mediated Communication, 2(3). Retrieved February 29, 2004, from

http://www.ascusc.org/jcmc/vol2/issue3/gordin.html.

Government of Alberta, Canada. (2004). Information and communication technology.

Retrieved October 30, 2004, from

http://www.learning.gov.ab.ca/k_12/curriculum/bysubject/ict/default.asp#info.

Gragert, E. (2000, September). Expanding international education through the Internet:

No longer limited to the global studies and language curriculum. Washington,

DC: White Paper prepared for the Secretary of Education's Conference on

Technology in Education.

Grant, M. & Branch, R. (2005) Project-based learning in amiddle school: Tracing

abilities through the artifacts of learning. Journal of research on technology in

education 38(1), 65-98.

Gubacs, K. (2004). Project-based learning: A student-centered approach to integrating

technology into physical education teacher education. Journal of Physical

Education, Recreation and Dance, 75(7), 33-37.

Hampton, J., Moore, P., & Thomas, H. (1973). Subjective probability and its

measurement. Journal of the Royal Statistical Society, Routledge: London and

New York, Series A, 136, 21-42.


24

Helmer, O. (1983). Looking forward: A guide to futures research. Beverley Hills: SAGE.

International Education and Resource Network. (2004). iEARN conference 2004 Web

site. Retrieved April 6, 2004, from http://www.iearn2004.net/.

International Society for Technology in Education. (1998). National educational

technology standards for students. Eugene, OR. Retrieved October 30, 2004, from

http://www.outreach.uiuc.edu/ijet/v1n2/parker/.

Klutschkowski, F., & Troth, W.A. (1995, April). Components for a substance abuse

counselor education curriculum: A Delphi study. Journal of Addictions and

Offender Counseling, 15, 34 –41.

Kopalle, P., & Lehmann, D. (1997) Alpha inflation? The impact of eliminating scale

items on Cronbach’s alpha. Organizational Behavior and Human Decision

Processes, 70(3), 189-197.

Kozma, R. B. (2003). Technology and classroom practices: An international study.

Journal of Research on Technology in Education, 36(1), 1-7.

Kozma, R. B., & Anderson, R.E. (2002). Qualitative case studies of innovative

pedagogical practices using ICT. Journal of Computer Assisted Learning, 18,

387-394.

Kozma, R., & Schank, P. (1998). Connecting with the twenty-fist century: Technology in

support of educational reform. In C. Dede (Ed.), Technology and Learning.

Washington, DC; American Society for Curriculum Development.

Lang, T. (1998). An overview of four futures methodologies. Retrieved April 6, 2004,

from http://www.soc.hamaiii.edu/~future/j7/LANG.html.


25

Lang, Q., Peer, J., & Divaharan, S. (2005). Computer mediated communication as a

collaborative tool for facilitating student-centered learning in project-based

classrooms. Educational Technology 45(4), 48-51.

Law, N., Lee, Y., & Chow, A. (2002). Practice characteristics that lead to 21st century

learning outcomes. Journal of Computer Assisted Learning, 18, 415-426.

Linstone, H. (1978). The Delphi technique. In J.Fowles, (Ed.), Handbook of Futures

Research (pp. 273-300). Westport, CT: Greenwood.

Linstone, H. A., & Turoff, M. (Eds.). (1975). The Delphi method: Techniques and

applications. Reading, MA: Addison-Wesley.

Markham, T., Larmer, J., & Ravit, J. (2003) Project based learning handbook: A guide to

standards focused project based learning for middle and high school teachers.

Buck Institute of Education, Novato, CA.

McGhee, R., & Kozma, R. (2001, April). New teacher and student roles in the

technology-supported classroom. Paper presented at the Annual Meeting of the

Educational Research Association, Seattle, WA.

Ministry of Education: Republic of Hungary. (2004). ICT in the Hungarian education.

Retrieved May 9, 2005, at http://www.om.hu/main.php?folderID=854.

Ministry of National Education of Poland. (2001). ICT in education policy. Retrieved

May 9, 2005, at http://www.men.waw.pl/.

Moursund, D. (2002). Project based learning using the Internet (2nd ed.). Eugene, OR:

ISTE.


26

Moursund, D. (2004). Discussion of 7-point expertise scale. Retrieved October 31, 2004,

from http://darkwing.uoregon.edu/~moursund/ICT-planning/discussion_of_7-

point.htm.

Moursund, D., Bielefeldt, T., & Underwood, S. (1997). Foundations for the road ahead:

Project-based learning and information technologies. A report prepared for the

National Foundation for the Improvement of Education by the International

Society for Technology in Education. Retrieved February 4, 2004, from

http://www.iste.org/research/roadahead/pbl.cfm.

Moursund, D., & Smith, I. (2000). Research on Internet use in education. The Research

and Evaluation Group of the International Society for Technology in Education

(ISTE). Retrieved February 4, 2004, from

http://www.iste.org/research/reports/tlcu/internet.cfm.

Murray, J. W., & Hammons, J. O. (1995). Delphi: A versatile methodology for

conducting qualitative research. The Review of Higher Education, 18(4), 423-436.

National Center for Education Statistics. (2000, April). Teacher use of computers and the

Internet in public schools. (NCES Publication No. 2000-090). U.S. Department of

Education: Office of Educational Research and Improvement.

National Center for Technology in Education. (2003). Teaching Skills Initiative (TSI).

Dublin, Ireland. Retrieved October 30, 2004, from

http://www.ncte.ie/NCTEInitiatives/TeachingSkillsInitiatives/.

Patton, M. Q. (2002). Qualitative evaluation and research methods. Thousand Oaks, CA:

Sage Publications.


27

Pelgrum, W. (2001). Obstacles to the integration of ICT in education: Results from a

worldwide educational assessment. Computers and Education, 37, 163-178.

Pittard, V., Bannister, P., & Dunn, J. (2003). The big pICTure: The impact of ICT on

attainment, motivation and learning. Nottinghamshire, England: Department for

Education and Skills.

Preston, C., Cox, M., & Cox, K. (2000). Teachers as innovators in learning: What

motivates teachers to use ICT. London: Teacher Training Agency/MirandaNet/

Oracle/Compaq.

Province of British Columbia, Canada. (2001). Integrating ICT into teaching and

learning. (2001). ICT Standards Guide. Retrieved October 30, 2004, from

http://www.bced.gov.bc.ca/technology/ict_guide/ictguide/ict_intro.htm.

Qualifications and Curriculum Authority. (2001). Information and communication

technology. National Curriculum in Action Web Site, Information and

Communication technology, London. Retrieved October 30, 2004, from

http://www.ncaction.org.uk/subjects/ict/index.htm.

Riel, M., & Becker, H. (1999). The beliefs, practices, and computer use of teacher

leaders. Teaching, learning and computing: 1998: National survey. The Center

for Research on Information Technology and Organizations, University of

California, Irvine. Retrieved February 11, 2004, from

http://www.crito.uci.edu/tlc/findings/aera/startpage.html.

Rivet, A. & Krajcik, J. (2004). Achieving standards in urban systemic reform: an

example of a sixth grade project-based science curriculum. Journal of Research in

Science Teaching 41(7), 669-692.


28

Schmidt, R. (1997). Managing Delphi surveys using nonparametric statistical techniques.

Decision Science, 28, 763-774.

Schmidt, R., Lyytinen, K., Keil, M., & Cule, P. (2001). Identifying software project risks:

An international Delphi study. Journal of Management Information Systems,

17(4), 5-36.

Schulz-Zander, R., Buchter, A., & Dalmer, R. (2002). The role of ICT as a promoter of

students’ cooperation. Journal of Computer Assisted Learning, 18, 438-448.

Scrimshaw, P. (2004). Enabling teachers to make successful use of ICT. British

Educational and Communication and Technology Agency. BECTA ICT

Research, Coventry, England.

Simmonds, C. (1977). The nature of futures problems. In H. Linstone & C. Simmonds

(Eds.), Futures research: New directions. London: Addison-Wesley.

Solomon, G. (2003, January). Project-based learning: A primer. Technology and

Learning, 23. Retrieved March 7, 2004, from

http://www.techlearning.com/db_area/archives/TL/2003/01/project.html.

Stites, R. (1998). Evaluation of project based learning: What does research say about

outcomes from project-based learning? Retrieved November 29, 2004, from

http://pblmm.k12.ca.us/PBLGuide/pblresch.htm.

SurveyMonkey. (1999-2004). SurveyMonkey LLC (http://www.surveymonkey.com).

Thomas, J. W. (2000, March). A review of research on project-based learning. Supported

by Autodesk Foundation, San Rafael, CA. Retrieved March 1, 2004, from

http://www.k12reform.org/foundation/pbl/research/summary.pdf.


29

UNESCO Asia and Pacific Regional Bureau for Education. (2003). ICT evaluation &

assessment & indicators (UNESCO). Bangkok, Thailand. Retrieved October 30,

2004, from http://www.unescobkk.org/ips/ict/ict10.htm.

UNESCO Institute for Information Technologies in Education. (2004). Internet

resources. Moscow, Russia. Retrieved May 9, 2005, at

http://is.iite.ru/html/internet/.

VanFossen, P. (1999, November). Teachers would have to be crazy not to use the

Internet!: Secondary social studies teachers in Indiana. Paper presented at the

Annual Meeting of the National Council for the Social Studies, Orlando, FL, 19-

21.

Veen, W. (1993). The role of beliefs in the use of information technology: Implications

for teacher education, or teaching the right thing at the right time. Journal of

Information Technology for Teacher Education, 9, 139-153.


30

Appendix A. Round 3 – Barrier Categories and Category Means Please rate the significance of the following barriers as you have experienced them in using Internet and Communication Technology-assisted Collaborative Project-based Learning with students:

Composite

Mean (SD) f=44

N. America Mean

(SD) f=16

E. Europe Mean (SD) f=16

Africa Mean (SD) f=12

Composite Rank - Teacher Training Issues 1 Teachers are not trained in how to specifically integrate

online projects into the classroom 3.48

(1.30) 2.81

(1.33) 3.56

(1.15) 4.25

(1.06) 2 Teachers do not feel they are proficient enough to use

ICT and computers for collaborative project work 3.41

(1.28) 2.69

(1.20) 3.38

(1.26) 4.42

(0.67) 3 Teachers have not received adequate training in

technology 3.30

(1.13) 2.69

(1.20) 3.38

(1.26) 4.00

(0.85) 4 Teachers simply do not know about this teaching

practice 3.18

(1.30) 2.88

(1.45) 3.06

(1.24) 3.75

(1.06) 5 Teachers do not see the value of online collaborative

project work 2.98

(1.23) 2.50

(1.26) 3.19

(1.11) 3.33

(1.23) 6 Teachers are resistant to changing the methodologies

they use to teach 2.93

(1.13) 2.63

(1.09) 3.06

(1.00) 3.17

(1.34) 7 Teachers have a fear of using technology 2.82

(1.17) 2.38

(1.09) 3.19

(1.05) 2.92

(1.31) Category Mean 3.16

(1.05) 2.65

(1.16) 3.26

(0.96) 3.44

(0.85) Composite Rank - Technology/Technical Issues

1 Students have poor computer related skills 3.48 (1.30)

2.81 (1.33)

3.56 (1.15)

4.25 (1.06)

2 Out-dated computers which cannot support ICT and project work

3.41 (1.28)

2.69 (1.20)

3.38 (1.26)

4.42 (0.67)

3 Unreliable internal network within the school 3.30 (1.13)

2.69 (1.20)

3.06 (1.24)

4.00 (0.85)

4 Lack of technical support for computers and ICT 3.18 (0.97)

2.88 (1.09)

3.31 (0.60)

3.42 (1.16)

5 Inequality of access with other schools does not allow schools to equally exchange information or support each other’s projects

3.09 (1.36)

2.94 (1.39)

2.81 (1.33)

3.67 (1.30)

6 Project work is difficult to accomplish because computers in different schools have different operating systems and software that are not compatible with each other

3.00 (1.26)

2.69 (1.30)

2.88 (1.26)

3.58 (1.08)

7 Students are not interested in using technology 2.98 (1.23)

2.50 (1.26)

3.19 (1.11)

3.33 (1.23)

8 Protection software does not allow students to access project work or research sites

2.98 (1.17)

2.50 (1.26)

3.06 (1.06)

3.50 (1.00)

9 Traditional subject oriented classes cannot use computers and ICT because it is strictly reserved for use by computer classes

2.93 (1.13)

2.63 (1.09)

3.06 (1.00)

3.17 (1.34)

10 Overall high cost of technology and equipment needed in schools for participating in online project work

2.82 (1.17)

2.38 (1.09)

3.19 (1.05)

2.92 (1.31)

11 Limited or no access to technology such as computers and ICT connections

2.66 (1.20)

2.63 (1.31)

2.50 (1.03)

2.92 (1.31)

12 Persistent technical difficulties resulting in unreliable equipment and connections

2.52 (1.09)

2.38 (1.09)

2.63 (0.81)

2.58 (1.44)

13 Not enough computers and related equipment to support ICT 2.05 1.69 1.94 2.67


31

collaborative project work (1.10) (0.79) (0.93) (1.44) Composite Rank - Internet Connectivity Issues

1 Poor, slow, and unreliable connectivity makes it difficult to download and access information

3.09 (1.36)

2.56 (1.26)

3.00 (1.41)

3.92 (1.08)

2 Low bandwidth or congested lines makes it difficult to download and access information

2.93 (1.42)

2.44 (1.36)

2.81 (1.42)

3.75 (1.22)

3 Persistent lack of access – Computers not connected - Lack of electricity to run computers

2.82 (1.42)

2.31 (1.25)

3.00 (1.41)

3.25 (1.54)

4 High cost of connectivity 2.36 (1.10)

1.94 (0.93)

2.38 (0.89)

2.92 (1.38)

Category Mean 2.80 (1.14)

2.31 (1.11)

2.80 (1.13)

3.46 (0.92)

Composite Rank - Curriculum/Program Issues 1 Projects take too much time to complete given the

limited amount of time in the school day and an already full school curriculum

2.98 (1.13)

2.81 (1.22)

3.13 (0.81)

3.00 (1.41)

2 Cultural misunderstandings among the students – Students cannot understand views and perspectives of students in other parts of the world

2.93 (1.09)

2.63 (1.20)

3.19 (0.83)

3.00 (1.21)

3 Preparation for local or national examinations takes precedence in the allocation of time leaving little time for computer related project work

2.93 (0.97)

2.88 (1.09)

2.88 (0.72)

3.08 (1.16)

4 ICT teachers have a workload that is too heavy and do not have enough time to devote to online collaborative project work

2.89 (1.35)

2.50 (1.26)

2.81 (1.17)

3.50 (1.57)

5 Local education agencies (school districts) emphasize traditional textbook learning and has a curriculum that has limited or no emphasis on the use of ICT skills or related projects

2.89 (1.28)

2.75 (1.48)

2.94 (0.93)

3.00 (1.48)

6 Teachers and students simply lose interest over time on projects that take a long time to accomplish

2.86 (1.17)

2.56 (1.31)

2.94 (1.00)

3.17 (1.19)

7 Negative experiences, such as low interaction or no responses from other schools, with past project work

2.84 (1.16)

2.88 (1.26)

2.88 (1.02)

2.75 (1.29)

8 No time for teachers to plan with others teachers involved in projects

2.73 (1.30)

2.13 (1.20)

2.69 (1.08)

3.58 (1.31)

9 Little or no time for individual classroom teachers to plan and initiate ICT projects

2.39 (1.30)

2.38 (1.09)

2.00 (1.10)

2.92 (1.68)

10 Students have too much traditional classroom work – Computer related project work is seen as an extra and has low priority

1.45 (0.87)

1.13 (0.34)

1.38 (0.62)

2.00 (1.35)

Composite Rank - ICT Support Issues 1 Lack of support for educational technology and ICT on

an local educational agency (school district) wide, institutional, or state level

2.75 (1.35)

2.50 (1.15)

2.69 (1.40)

3.17 (1.53)

2 There is little or no administrative support on a local level because administrators do not understand the value of ICT project work

2.43 (1.42)

2.00 (1.26)

2.44 (1.09)

3.00 (1.86)

3 Lack of support by others teachers needed to complete project work

2.07 (1.19)

2.00 (0.89)

1.88 (1.02)

2.42 (1.68)


2.17 (1.01)

2.33 (1.04)

2.86 (1.51)

Composite Rank – Project Related Issues


32

1 Language barriers – Students have difficulty communicating with other students online because they have difficulty communicating in non-native languages

2.57 (1.04)

2.38 (0.96)

2.44 (0.96)

3.00 (1.21)

2 Differences in school schedules influenced by exams and holidays that do not allow for continued participation or support of projects

2.34 (1.10)

2.44 (1.26)

2.13 (0.72)

2.50 (1.31)

3 Time difference between schools in different geographic regions prevent them from having synchronous communication

2.30 (1.27)

2.38 (1.45)

1.88 (0.81)

2.75 (1.42)

4 Available projects are difficult to integrate into the curriculum because they do not fit into the subject matter or structure of the curriculum

2.02 (1.02)

1.69 (0.79)

2.06 (0.93)

2.42 (1.31)

5 Actual projects are not well organized or poorly planned and have no support for schools attempting to work together

2.00 (0.94)

1.81 (0.83)

1.94 (0.77)

2.33 (1.23)

6 Available projects do not have any perceived educational value

1.98 (1.07)

1.69 (0.60)

2.13 (1.20)

2.17 (1.34)


2.06 (0.80)

2.09 (0.67)

2.53 (1.09)

Composite Rank - Miscellaneous Items 1 Technology use is dominated by male students leaving

few or little opportunities for females to learn skills and participate in online projects

2.59 (1.19)

2.81 (1.28)

2.31 (1.08)

2.67 (1.23)

2 Students will not stay on task to complete project work 2.18 (1.26)

2.25 (1.13)

2.06 (0.85)

2.42 (1.73)

3 Students lack cooperative learning skills and are unable to work together on project work

2.14 (1.17)

1.81 (1.17)

2.19 (0.91)

2.50 (1.45)

4 The structure of the school day does not allow the flexibility needed to participate in online collaborative project work

2.11 (1.04)

1.88 (0.89)

2.06 (0.85)

2.50 (1.38)

5 Parents of students have a fear of their children using ICT and are not supportive of online project work

1.82 (1.11)

1.44 (0.63)

1.69 (0.79)

2.50 (1.62)

6 Communication online and in forums is too impersonal for meaningful work

1.73 (1.04)

1.88 (0.96)

1.38 (0.81)

2.00 (1.35)

7 Lack of participation and interaction specifically by teachers within the United States

161 (1.10)

1.44 (0.63)

1.31 (0.87)

2.25 (1.60)


1.93 (0.42)

1.84 (0.67)

2.40 (1.26)

All Barriers 2.67 (0.79)

2.37 (0.77)

2.65 (0.67)

3.10 (0.83)

Note: 1=Not Significant, 2=Somewhat Significant, 3=Moderately Significant, 4=Very Significant, 5=Extremely Significant

running head: ict-assisted collaborative project …

Documents