This article was downloaded by: [University Of Pittsburgh]On: 21 October 2014, At: 20:09Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T3JH, UK

Journal of Research onTechnology in EducationPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/ujrt20

Students’ Perception ofInteractive Learning ModulesBeaumie Kima, Williams Richardb & John Dattiloc

a The University of Georgiab East Carolina Universityc The University of GeorgiaPublished online: 24 Feb 2014.

To cite this article: Beaumie Kim, Williams Richard & John Dattilo (2002) Students’Perception of Interactive Learning Modules, Journal of Research on Technology inEducation, 34:4, 453-473, DOI: 10.1080/15391523.2002.10782361

To link to this article: http://dx.doi.org/10.1080/15391523.2002.10782361

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verifiedwith primary sources of information. Taylor and Francis shall not be liablefor any losses, actions, claims, proceedings, demands, costs, expenses,damages, and other liabilities whatsoever or howsoever caused arisingdirectly or indirectly in connection with, in relation to or arising out of theuse of the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in any formto anyone is expressly forbidden. Terms & Conditions of access and use canbe found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

Students' Perception of Interactive Learning Modules

Beaumie Kim The University of Georgia

Richard Williams East Carolina University

John Dattilo The University of Georgia

Abstract The purpose of this study was to examine perceptions of interactive learning modules (ILMs) developed for use in an undergraduate recreation and leisure studies course. Interpretive analysis of qualitative data using a constant comparative method suggested that some char­acteristics of the ILMs influenced learning. Data clustered into four overarching categories: (a) contextualized information presented in the ILMs, (b) situated activities provided in the ILMs, (c) learner guidance offered in the ILMs, and (d) technical aspects of the ILMs. Findings of this study have implications for improvement and future development of the ILMs. Further research is encouraged to examine the relationship between characteristics of ILMs and achievements of students. (Keywords: case methods, computer-assisted instruc­tion, experiential learning, higher education, interactive learning modules.)

One purpose of higher education is to prepare people to address the emerging problems of society (Sanyal, 1991). In addition to the content relevant to job­related situations, there has been a need to find a teaching method to bridge the gap between formal academic instruction and on-the-job training (Ellington, Gordon, & Fowlie, 1998). Case study methods have been used to provide learners with some indirect practical experience related to the knowledge or concepts from text or instruction, especially in the areas of administration, medicine, and law (Knirk, 1991).

With advancements in technology, it is possible to provide students with ex­periential activities (e.g., active learner roles, problem-based units of learning, reality of function, consequence of their activities) through computer-based implementations such as interactive learning modules (ILMs) (Gredler, 1994). ILMs are self-contained educational modules that offer experiential learning op­portunities; are characterized by being functional and meaningful; and include interactive games, simulations, and drills (Rieber, 1992, 1998; Tennyson, 1994). Generally, researchers have reported that ILMs are effective learning aids, even though ILMs studied to date have widely varying characteristics (Wolfe, 1997). Opportunities for computer-based instruction can facilitate

The authors would like to thank the Department of Instructional Technology at the University of Georgia and the graduate students for developing some of the interactive learning modules. Appre­ciation is extended to Drs. janette R. Hill and Michael Grey in the Department of Instructional Technology at the University of Georgia, for providing feedback on earlier draft of this article.

journal of Research on Technology in Education 453

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

learning oppom.: .. ::1ities by providing explicit examples of concepts and issues to

which students can relate (Clark, 1994; Jonassen, Campbell, & Davidson, 1994; Kozma, 1994a).

Many instructors use computers to facilitate student learning, and there is a plethora of studies comparing computer-based instruction to traditional in­struction (Alessi & Tmllip, 1991). However, there has been limited documenta­tion of the characteristics of ILMs that contribute to learning (Kozma, 1991; Neuman, 1989; Reeves, 1997). There is a need to explore how the characteris­tics of the ILMs influence learning processes. Specifically, researchers recognize the need to idencfY which specific capabilities of the ILMs contribute to learner engagement (Kozma, 1994a; Neuman; Yang, 1992). Therefore, the purpose of this study was to examine the perceived effectiveness of ILMs and to determine characteristics of ILMs that contributed to their effectiveness.

APPLICABLE KNOWLEDGE

Conventionall:'· learners are responsible for building connections between the knowledge they acquire and the situation in which they are to apply this knowl­edge (Reeves, 19~1]); however, some learners are not applying much of the knowledge they have gained from learning experiences to real-world situations (Jacobson & Archodidou, 2000; Reeves). Unfortunately, researchers have shown that students wio can effortlessly reproduce acquired knowledge can barely use the same information in contexts of use (Cote, 1994; Hasselbring & Learning Technology Center at George Peabody College, 1991; Jacobson & Archodidou; Reeves). Unless learning occurs in a proper context, acquired knowledge may be inert and unavailable for use (Jonassen et al., 1994). There­fore, attempts have been made to solve this problem using different media and pedagogical tech=.ique~ to encourage learners to apply the knowledge learned in a condition of use in similar contexts (Reeves; also e.g., Cunningham & Thorkildsen, 1996; Jacobson & Archodidou; Johnson & Morris, 1997; Odom & Pourjalali, 199<5).

EFFECTIVE LEARNING ENVIRONMENTS AND CASE METHODS

Recent studies indicate that well-designed, task-relevant and experiential learning environments have the potential to support learners in transferring their knowledge :;) real-world problem solving (Brown, Hedberg, & Harper, 1994). Researchers (e.g., Anderson, Reder, & Simon, 1996; Brown, Collins, & Duguid, 1989; \!osniadou, 1996) suggest that the design of learning environ­ments focus on engaging learners in meaningful and purposeful activities and relating what is le:uned in the instructional settings with what is needed in real­world situations. Contemporary learning theorists, especially situated cognition theorists, stipulate that the ability to apply knowledge is largely determined by how knowledge is learned and used in the learning environment (Vosniadou) and emphasize d-_at learning should be specific to the context in which it is to be applied (Anderson et al.; Rieber, 1992).

How knowledge is initially acquired plays an important role in how much it can be used in other contexts (Reeves, 1992). When case materials provide task-

454 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

relevant and experiential learning environments, there are possibilities that the learner will gain meaningful and applicable knmvledge (Reeves). If the learner ide:::nifies necessary information from the case to sclve problems, then the likeli­tood of successful transfer of their knowledge to real-world situations should be enhanced (Kozma, 1994a). It is helpful if each case serves as a detailed example br appropriate actions when a similar situation occurs (Hudspeth, 1991). Ac­cording to Jacobson and Archodidou (2000) and Knirk (1991), case methods hav-e been used in several disciplines (e.g., professional education in the medi­cal, business, and legal fields), and use of case-based and problem-based ap­p::-oaches have been identified as being effective (e.g., Curtiss & Hurd, 1980; Diamantes, 1996; Reisman, 1987; Sharf & Poirier, 1988).

DELIVERY MEDIUM AND INTERACTIVE LEARNING MODULES

Kozoa ( 1991) suggested that we should take advantage of the capabilities o:=:- different media to complement instructional methods, even though the methods (e.g., case materials) primarily influence learning. As an example, the Cognition and Technology Group at Vanderbilt (CTGV, 1997) implemented videodisc technology in the Jasper Project to allow students to explore the v~deo and find relevant information to solve problems embedded in a complex social context. This application of a videodisc allowed students to associate their knowledge of solution procedures to real--vvodd-like problem situations (Kozma, 1994a).

There is ample ongoing research and development of case-based learning ma­ter~als exploiting the abilities of different technolo57 (e.g., CTGV, 1997; LeBlanc, 1997; Naidu & Oliver, 1996; Pickrell & Department of Clinical Sci­ences, Kansas State University, 1995; Ryan & Koschmann, 1994). Media and methods together can help learners construct their knowledge and deduce meanings by providing rich learning environments (Jonassen et al., 1994; Kozma, 1991, 1994b). As one of the most widely used and easily accessible me­dia today, the computer has many intrinsic capabilities that can complement the methods of instruction. The interactivity ofiLMs (e.g., making choices, an­Ewering questions, and solving problems) is intecded to engage mental pro­cesses and to enhance performance and productivi-:y (Jih & Reeves, 1992). The learner using an ILM becomes an active participant in the teaching and learn­ing process (Jih & Reeves).

With the abilities of the computer, ILMs can enable a case-based approach wherein learners encounter realistic problem cases and solve them using appro­priate .:(nowledge or concepts (Reeves, 1992). According to Tennyson (1994) and Jacobson and Archodidou (2000), contextual knowledge and skills can be obtained best through problem-oriented, contextual modules. Research and development of computer-delivered, problem- anc case-based learning have evolved recently to provide learners with environrr_ents that enhance effective application of knowledge (e.g., Foster & Bareiss, 1995; Glenn, Koshmann, & Conlee, 1997; Jonassen, 1996; Koshmann, 1995). Some research has already siown the superiority of computer modules compared to case-based teaching in class (e.g., Wolfe, 1997).

Iournal of Research on Technology in Education 455

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

DESIGNING OF ILMS TO INCORPORATE CASE METHODS Rieber (1992) and Tennyson (1994) have identified that the task for the in­

structional designer is to offer meaningful contexts and a performance-oriented learning environment. ILMs can be designed to help learners link knowledge to problem situations through active participation (Reeves, 1992). As designers, we need to develop ILMs that involve learners in applying relevant concepts in realistic experiences and assist them with interpreting and conceptualizing their experience (Baker, Jenson, & Ko~b, 1997).

Another task for the instructional designer is to use media to provide real­world-like learning environments (Jonassen et al., 1994). Learning is more likely to transfer when it is situated in environments similar to the setting in which the knowledge and skills will be applied (CTGV, 1997; Dede, 1992). By using computer learning environments, designers can incorporate elements that simulate a real-world challenge for the problem situation and stimulate curios­ity to increase emotional involvement (Dede). These efforts can provide a learn­ing environment where the learner constructs and transfers knowledge, even though exact replication of a particular context is impossible (Jonassen et al.).

METHOD This study employed qualitative methods to examine students' perception of

the implementation of ILMs during a semester course. A series of written cri­tiques and a focus group interview were analyzed to provide insight into stu­dents' perceptions of the ILMs.

Participants and Researchers Participants (1V = 54; 34 female, 20 males; ages 19-22) were third-year under­

graduate students enrolled in an recreation and leisure studies course at the University of Georgia (UGA). Approximately half of the students reported be­ing familiar with learning on computers. Participants completed the ILMs, pro­vided written critique of the ILMs, and participated in the interview. Informed consent was obtained from participants. We collected the critiques during the data collection period and analyzed them for themes and patterns. The third author was the course instructor, and the other researchers had no contact with students other than during data collection. These two researchers conducted the focus group interview.

Intervention The purpose of the course is to teach future leisure service professionals to

understand issues related to disability and inclusion and ultimately to learn to design leisure services that are inclusive of people with disabilities. We devel­oped the seven ILMs used in this study in conjunction with the UGA Depart­ment of Instructional Technology using Macro media Authorware (1999) and Macromedia Director (1999) software. Each ILM begins with an opening sce­nario that describes the situation and assigns the learner certain contextual task(s) as a recreation professional. The ILMs are all similar, in that they reflect both the concepts in the textbook and situations likely to be encountered by lei-

456 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

sure service professionals. However, the ILMs differed according to their spe­cific characteristics.

Plan a Fishing Trip is an ILM with the goal of arranging a variety of people (e.g., people with disabilities, children, staff members) in five different boats (Figure 1). Mter participants complete an arrangement, the ILM offers feed­back and an opportunity for modification. Mter final modifications are com­pleted, the ILM provides more precise feedback and a final score. The primary goal of the ILM is to have students apply their knowledge of natural propor­tions when developing inclusive services.

Choose Words Carefully is a two-part matching game. During the first part, participants read letters and choose the letter that uses sensitive terminology. During the second part, participants are required to place phrases from a sup­plied list in a letter containing blanks (Figure 2). The ILM gives immediate feedback concerning a participant's choice. The ILM was designed to have stu­dents practice using sensitive terminology that demonstrates respect for people with disabilities.

Make Park Accommodations is an ILM that places participants in the role of a park manager who must make decisions regarding accommodations for people with disabilities (Figure 3). An advisory board of experts provides imme­diate feedback about decisions. Mter an opportunity to change a decision, par­ticipants are offered final feedback from the board members from different per­spectives (e.g., financial, legal). The primary intent of this ILM is to have students apply mandates of the American with Disabilities Act while balancing concerns about the environment and those of park visitors.

Figure I. Plan a Fishing Trip.

Journal of Research on Technology in Education 457

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

~

with t'f,ln,;jic~~

P~H'l>P~ with diis:ttbiiiltl$;S

!ilP$~th dis.ord$!!'®

with ihfll~rins dl~!;i<rdews

Figure 2. Choose WVrds Carefully.

Ffee Tennl~ Clltllc Park.

mild I'!~$ we~k~n~, wh;~~t~ ~C~®'Wtm!bl!il!, ®Mbl~ng

inf,t;. ·~l?ifl.~Vy~Pr&1P;fir\'~mv~

Figure 3. Make Park Accommodations.

458 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

In Make Play Accessible, participants design a playground by choosing equip­ment from a catalog (Figure 4). A participant receives feedback after choosing all equipment for a playground. The game contains a status bar that provides information about the budget and the inclusiveness of the playground. Partici­pants can modify decisions during the game and are given final feedback about design of the playgrounds. This ILM is designed to have students apply prin­ciples of universal design to a specific recreation context, a playground.

Plan a Camp asks participants to solve problems in the planning of a camp that is inclusive of children with disabilities (Figure 5). Feedback is provided immediately after a participant makes a choice from the supplied answers. The primary goal of this ILM is for students to apply specific strategies that promote social inclusion of individuals into a camp environment.

In Make Adaptations, participants help solve various characters' barriers to leisure participation (Figure 6). Each character has a different disability. Imme­diate feedback about a participant's choice is provided. The focus of this ILM is for students to make decisions regarding specific ways to make reasonable ac­commodations that involve modifying the environment, service, or materials.

Be an Advocate requires participants to make decisions about different ways to advocate for people with disabilities (Figure 7). Participants select a reaction to a situation and are provided feedback concerning their selection. Following initial feedback, participants can modify their selection. The intent of the ILM is to require students to take the initiate to advocate for people with disabilities as they encounter various barriers in the community.

Data Collection and Analysis Participants completed seven brief (one or two pages) written critiques

throughout the semester, and they participated in a one-hour group interview conducted toward the end of the semester, once all the ILMs had been com­pleted. For the critiques, participants responded to the following two questions: "What do you recommend we change about the activity to improve it?" "What did you like best about the computer-based learning activity?" The researchers prepared questions and conducted the interview, which was tape-recorded and transcribed.

Data were analyzed using the constant comparative method described by Glaser and Strauss ( 1967). Data were coded and analyzed to help in the devel­opment of themes (Leedy, 1997; Taylor & Bogdan, 1984). Themes that emerged for each ILM assignment were compared to themes from all other ILMs. Specifically, words and phrases that students used frequently were under­lined (Bogdan & Biklen, 1992). Tables and figures were created using words and phrases that frequently appeared in the data (Yin, 1994). Figure 8 shows an example. Repeated use of certain words and phrases suggested four categories that could be used to organize the data. Data were then sorted into these cat­egories (Bogdan & Biklen).

Because scientific rigor must be established in any investigation (i.e., LeCompte & Goetz, 1982), the criteria of credibility, dependability, and con­firmability were employed in this study. Credibility (internal validity) was estab-

Journal of Research on Technology in Education 459

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

Figure 4. Make Play Accessible.

Figure 5. Plan a Camp.

460 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

Figure 6. Make Adaptations.

Figure 7. Be an Advocate.

Journal of Research on Technology in Education 461

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

Figure 8. Sample of data analysis.

lished through continuous discussion between research team members facilitat­ed by ongoing dialogue. The use of more than one investigator "in the collec­tion, reduction, confirmation, and interpretation of data is crucial in reducing the potential bias of a single individual" (Stumbo & Little, 1993, p. 289). Cod­ing checks were conducted by having more than one researcher code data from the critiques and interviews (Leedy, 1997). Dependability (reliability) occurred with the audit trail, a process to increase the stability of data. Members of the research team examined the processes of data collection and analysis conducted by the first author along with the transcripts to determine if they agreed with the interpretations. Confirmability (objectivity) was produced by reflexibility as the first author intentionally revealed to members of the research team the un­derlying epistemological assumptions considered when generating ideas about the interview and associated data. Finally, data were displayed in a spreadsheet under identified categories. Sorting the data by theme facilitated the search for relationships among the themes of the ILMs.

FINDINGS Data clustered into four overarching categories: (a) contextualized informa­

tion presented in the ILMs, (b) situated activities provided in the ILMs, (c) learner guidance offered in the ILMs, and (d) technical aspects of the ILMs. When talking about contextualized information in ILMs, participants spoke about three ideas: profiles describing clients, presence of multiple scenarios to which they were to respond, and simulated real-life situations. When discussing

462 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

situated activities in the ILMs, students talked about four topics: their ability to learn through active engagement, assume the role of practitioner, be challenged, and make decisions. Students reflected about learner guidance in the ILMs as they talked about presentation of immediate positive feedback and inclusion of debriefing at the end of the ILM with the chance to try to improve one's perfor­mance. Students provided specific comments about three technical features of the ILMs as they discussed their ability to navigate within the ILMs, the pres­ence of sound effects, and the presentation of the graphics. (See Table 1 for a summary of categories and themes.) Detailed comments related to themes are described in the following sections. (Note: Participant names have been changed to protect their identities. Participant quotations have not been edited.)

Table 1. Categories and Themes

Categories

Contextualized information

Situated activities

Learner guidance

Technical aspects

Themes

Client profiles Multiple scenarios Simulated real-life situations

Active engagement Role of practitioner Being challenged Making decisions

Immediate positive feedback Debriefing Chance to improve the performance

Navigation Sound effect Mfective graphics

Contextualized Information Presented in the ILMs Client Profiles

Participants reported that information about the individuals with disabilities presented in the ILM assisted them in applying conceprs during the activities. Sarah expressed appreciation for the characters profiles in Plan a Fishing Trip when she wrote, "The profiles were a good way to add character and realism to the activity. It was helpful to be able to read the profiles while still being able to place people in the boats." In discussing Make Adaptations, Paula related con­textual information to decision making: "I did like that it gave detailed profiles on the children. It included parts of their personalities which is very important when making decisions on how to include children in activities."

Multiple Scenarios Several of the ILMs included more than one scenario. Participants reported

that having several scenarios within an ILM helped them understand and apply

journal of Research on Technology in Education 463

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

concepts, and they often expressed interest in having additional scenarios con­tained within the ILMs. In discussing Plan a Fishing Trip, Lee commented, "I th~nk possibly providing us with more scenarios to complete would aid us in mastering the material." In a comment about Choose Words Carefully, Amy added, "If we were given the opportunity to put ourselves in more than one situation, then we would understand using sensitive terminology in a more pre­cise way." Participants reported that even ILMs with multiple scenarios could have been improved with the addition of even more scenarios that increased in difficulty. Discussing Plan a Camp, Matt remarked, "I feel the activity could be irr_prov-ed by adding more scenarios. In addition, the added activities could re­quire increasing skill, knowledge, and experience with each additional activity."

Simulated Real-Life Situations Students reported that the ILMs were an accurate reflection of real-life situa­

tions they expected to encounter and that the ILMs helped them practice ap­pl?ing concepts to real-life situations. In discussing Plan a Fishing Trip, Marie explained, "\Ve were able to apply what we are learning in the classroom to pncciol real-life situation. For me, bringing together classroom and realistic work situations helps me focus on what I need to get out of my time here." Greg added, "I really liked how it used real-life examples of situations. By doing this, it helped me to apply the book, web lesson, and my own personal experi­ences i.::1 order to make the right decisions."

Although participants generally reported appreciation for the realism of the scenarios with the IL\1s, some participants reported that there should be more real-life aspects in the ILMs. Liz suggested, "Including some 'real life' conflicts and problems that can arise on a trip similar to this one might be a good idea, too. Making each person more dynamic might be a good way to begin consid­ering e1ch individual and their needs." Terri stated, "the activity did not ac­count for those who already were open-minded, or those who were negatively affected by the activity."

Situated Activities provided in the ILMs Learning through Active Engagement

::?articipants reported that hands-on learning using a computer simulation helped them understand concepts. Many participants mentioned that it was fun to do s8mething outside class other than read the textbook. For example, in dis­cussing Plan a Fishing Trip, Amy stated, "The fact that it was so easy to ma­nipulate the boats in a virtually hands-on manner was extremely helpful to me as a spatially oriented learner. I like to do things hands on; the CD ROM activ­ity ofTered essentially that." Peter echoed these feelings in discussing Make Ad­aptations: "I love doing the hands-on work with the computer-based activities. It is much more beneficial and easier for me to remember things and actually ur:.derstand concepts through these activities rather than sitting down and read­ing sooe e:J.apters." Brenda reported that, "I feel that these activities are giving me a great deal of practice of finding solutions to barriers. By the time I get out imo the field, there will be fewer barriers that I have not been faced with."

464 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

Assuming the Role of Practitioner Being a protagonist in the ILMs seemed to help students feel as if they were

applying what they had learned to a real-world situation. For instance, in dis­cussing Plan a Fishing Trip, Jenny wrote, "It got me involved, and for a short period I actually imagined myself in the situation of being a professional recre­ation administrator. I felt like it was my obligation to make the fishing trip an enjoyable experience for everybody." In Choose Words Carefully, participants are required to supply their names at the beginning of the interaction and are referred by name throughout. This characteristic appeared to contribute to par­ticipant involvement in the ILM. As Kelly explained, "The recreation profes­sional had a personal identity and job title to personalize the situation. As a re­sult, I felt emotionally attached and fully involved in the activity as an advocate for [the character in the ILM]." Laura added, "I really enjoyed the way that the situation was tailored to me. It really put me in the 'hot seat."'

Being Challenged Participants reported that they enjoyed being challenged. When ILMs were

perceived as challenging, participants described that they needed to concentrate more on concepts relevant to the situation. For instance, Bill commented about Plan a Fishing Trip that, "I liked the fact that it was difficult. It really made me sit back and think about every aspect that I probably would have done other­wise. I wanted to go back and rearrange them again until I got it perfect." Con­versely, if an ILM was perceived as simple, participants reacted negatively. Heather explained about Choose Words Carefully, "This activity can be im­proved by making it a little more challenging. I felt that reading and selecting the correct answer did not require much thought." Making a general statement about all of the ILMs, Adrian said, "The scenarios were easy. I think it would be better if more of the answers were more closely related so that more thought has to be put into choosing which was correct."

Making Decisions Students stated that having the opportunity to make decisions during IU\1s

helped them understand, apply, and reflect on concepts. In the following com­ment about Choose Words Carefully, Jim identified decision making as a char­acteristic that helped the ILM seem more realistic and similar to a real-world situation. "It let me make decisions as though I was the director of a leisure ser­vice. It helped me understand the importance of using the proper terminology." In a statement about Make Park Accommodations, Amy expressed how deci­sion making helped her reflect on the concepts presented in the ILM. "Not only did I have to make a good decision, but also I had to make a decision that was pleasing to the majority. I liked the fact that it made me think about how others would react to my decision before I made one." In Make Park Accom­modations, participants commented that they enjoyed the immediate feedback received from the characters in the ILM relative to their decisions. Karen stated, "I enjoyed toying around with the decisions I made and seeing which members liked and disliked my decisions. I found it interesting to see the various opin-

Journal of Research on Technology in Education 465

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

ions of the board members and I enjoyed practicing the skills I am learning in a decision-making context."

Some participants reported that making decisions in some of the ILMs using only text-based information impaired their ability to learn. For example, when talking about Make Park Accommodations, Tony stated, "I would have liked to see examples of the modifi::ations that were suggested. Maybe plans or pictures would have helped me better visualize the changes that were proposed to take place." In contrast, participants were supplied visual information in Make Play Accessible and expressed that they enjoyed using the visual information in deci­sion making. Eli stated, "I enjoyed purchasing the equipment for the play­ground from the catalog and trying to facilitate accessibility." Susan added, "I like the fact that you could pick each item and then see the benefits as well as how much it costs."

Participants reported that being supplied several solutions from which to choose helped in decision .::naking and understanding concepts. Commenting on Plan a Camp, Mike wrote, "When there are a few things to choose from, it gives me a better idea of what may work, what may not work, or what may work with some modificacons." Participants expressed that making decisions from a restricted range of choices helped them apply concepts from other sources. Clayton commented, "I enjoyed the activity's multiple choice capability in being able to evaluate and make decisions based on the knowledge obtained from the book and Web le.,son." However, the restricted range of potential solu­tions to problems presented in the ILMs was a concern for some participants and appeared to reduce their ability to make decisions. In discussing Make Park Accommodations, Olivia stated, "The only thing that I do not like about this activity is that you are just given the options. There is no opportunity for com­ing up with own solutions." Similarly, Joe commented, "I recommend the op­portunity to present personal solutions to the scenarios. I feel that if students could provide their own sclutions it would be more challenging." It appeared that participants would have preferred a format for the ILMs that would have been less tightly controlled and that allowed for individually designed solutions.

Learner Guidance Offered in the ILMs Immediate Positive Feedback

Participants appeared to value immediate and accessible feedback. In Make Play Accessible, feedback remained on the screen during the activity. Lucy com­mented, "The feedback table regarding accessibility effectiveness and cost were also very helpful in remincing me to try to make the best choice." Participants valued receiving positive feedback from the ILMs. Joe stated, "I had the feeling of satisfaction when 'Gooc Job' flashed up on the screen. I almost felt like I was really doing something to better someone's quality of life in the activity." Tracy remarked, "I enjoyed the feeling of doing something good to help those with disabilities to become more accepted and celebrated in the community."

Conversely, when participants believed they did not receive enough guidance because of a lack of immediate feedback, they voiced concern. In discussing Plan a Fishing Trip, Tonya remarked, "It could have given a little more direction and

466 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

criticism when the boats were full instead of waiting or the trip to be completed for evaluated." It appears that participants would have preferred guidance to fa­cilitate the application of concepts in the ILMs. Greg remarked, "I think you should only have a certain number of attempts and then a tutorial assists with the areas that you could not come to some sort of conclusion about."

Debriefing at End of ILM and the Chance to Try Again Participants expected a debriefing at the end of an ILM and reported that,

when present, debriefings were helpful in understanding concepts presented in the ILMs. For example, Leigh commented about Be an Advocate that "the idea of having a review at the end comparing both your answers and the most cor­rect answer was beneficial to my learning." In contrast, the absence of debrief­ing appeared to impair the ability of participants to understand concepts pre­sented in the ILMs. In discussing Plan a Fishing Trip, Tonya stated, "I feldike there was not an ending, a conclusion of why I had just done the activity. I feel that there should be some sort of debriefing about the activity." Wendy re­flected a similar opinion about Plan a Fishing Trip when she said, "I feel as though there needs to be a debriefing of the activity to make it more effective. The student completing the activity will have a better understanding of why they have done the activity." Participants reported that the timing of the feed­back was important. Some students suggested that some feedback provided dur­ing the activity would be more effective if provided at the end of the activity. For example, Blake explained, "I might move the part where you get advice to the end of the activity. By doing this, you can see how well you can make the playground accessible on your own."

When performance evaluations and debriefings were present in an ILM, par­ticipants particularly valued the opportunity to modifY a decision using a final evaluation. In discussing Plan a Fishing Trip, Denise wrote, "I like the opportu­nity to go back and rearrange the people further. This is important because it allows me to learn more from my mistakes. I could immediately apply informa­tion learned from the evaluation." Commenting on Make Park Accommoda­tions, Valerie said, "I really liked how it gave you your results at the end and how you could do it until you got it right." Commenting on the same activity, Jenny stated, "The reinforcement came from the final slide that gave direct feedback to my degree of manageability, satisfaction of the people, and people's attitude toward disabilities. The result was not perfect, reflecting my continual need to improve as a future recreation leader." Commenting on the same char­acteristic, Paul explained, "Once I got the feedback from my first arrangement, I became interested in how I could improve the results. Before I knew, I had tried about 10 or 12 different combinations, and I was actually having a good time and was intrigued by the learning activity and learning a lot from it."

Technical Aspects of the ILMs Navigational Control

Being able to restart the activity and modifY decisions seemed to play an im­portant role in helping participants understand and apply concepts. Liz com-

journal of Research on Technology in Education 467

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

mented, "I liked the way you could start over any time, or go back in the activ­ity without losing the stuff you had done." Difficulty navigating through the ILMs appeared to impede the ability of participants to learn from the experi­ence, particularly wher_ it was difficult to review earlier parts of the ILMs. Mary stated, "I believe it would be good to include a 'Back' button. At times, I felt it would have been helpf ~1 to go over what I had done and rethink some of my wording." Brenda added, "It would be helpful to have a back arrow while the user is exploring the possible solutions and discussing them with the advice panel. If you want to go back to review any of the solutions you must restart the entire question over."

Presence of Sound Effects Although sound effects may not have directly contributed to learning, they

appeared to be effective in making ILMs enjoyable. Ryan commented on the ef­fective use of sound effects in Make Adaptations: "The part of the CD that I liked best was the fanfcre music that played when a question was answered cor­rectly. I also liked the part when the children cheer because they can participate in the programs due to the adaptations made."

Presentation of Graphics Similar to the role of sound effects, the quality of graphics in the ILMs ap­

peared to influence participants' opinions of the activities. For example, Jenny remarked about Make Adaptations, "My favorite part of this activity was the graphics and pictures that were used to depict particular activities." Similarly, Andre commented, "I a.m a very visual person and the more memorable the ac­tivity the more likely I am to retain the information."

DISCUSSION The purpose of this ~tudy was to examine students' perception of the influ­

ence of the ILMs in hdping them to understand course content and to apply the content to situations that simulate problems present in practice. Participants identified characteristics of the ILMs that influenced their understanding of course material and their ability to apply concepts related to the course. Find­ings of interpretive analysis of the qualitative data revealed four overarching cat­egories: (a) contextualized information presented in the ILMs, (b) situated ac­tivities provided in the ILMs, (c) learner guidance offered in the ILMs, and (d) technical aspects of the ILMs.

When considering tl-_e findings, caution is advised for several reasons. Al­though a considerable amount of data were collected from participants' written critiques and the focus group interview that allowed for an examination of per­ceptions of the ILMs, there are no other data sources. The implementation of additional qualitative .oethods such as in-depth interviews could have resulted in richer data and, perhaps, greater insights. Also, student performance mea­sures were not examined; therefore, conclusions about actual student learning are not possible within the confines of this study. In addition, because respon­dents were students enrolled in an undergraduate course, their comments may

468 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

have been influenced by the fact that the instructor read their critiques. The students may have withheld important information if they believed their grade for the course might have been negatively affected if they identified weaknesses in the ILMs.

Considering the limitations associated with this study, the data suggest that ILMs have the potential to prepare students to understand and apply concepts in real-world situations. Many participants mentioned future use of the knowl­edge and skills obtained from the ILMs with comments such as Barry's, "Now I have confidence of facilitating individuals with disabilities in these situations," and Emily's, "We will know what to do in the future. It is so true to life." As Reeves (1992) proposes, by responding to problems presented in a scenario, learners should have felt they were developing useful knowledge as opposed to inert knowledge. Nevertheless, some participants seemed concerned that the ILMs downplayed the complexity of real-world situations. Anne explained, "I think it should remind us that just because you place a person with a disability into a boat with people who do not have disabilities, does not mean that atti­tude will be changed."

Participants reported that the ILMs helped them apply concepts better when multiple situations were presented in the ILMs. Participants discussed the need of having many different situations in ILMs, reflecting Kozma's (1994a) sugges­tion that computer interactions contain different stories.

Both immediate feedback and a summary debriefing at the end of the ILM appeared to be valuable in promoting student learning. For example, partici­pants explained that receiving reactions about their decisions from the charac­ters in the ILMs provided better insights into the concepts than merely receiv­ing an indication of "right" or "wrong." It was also apparent that debriefing was a form of delayed feedback that helped participants understand concepts learned in the ILM. Through feedback and debriefing, students were able to re­flect on what they did and what they could hc.ve done better (Collins, 1991). Receiving the same information in the delayed feedback as in the immediate feedback disappointed some students because they were expecting additional in­formation or further explanations about the results.

When an ILM was perceived as challenging, participants particularly valued being able to modifY their decisions. Participants used such words or phrases as "challenging," "difficult," "learn from mistakes," and "trial run" to explain their experiences with challenging ILMs that allowed modifications. The word "diffi­cult" usually had positive connotations when the students expressed their opin­ions. For example, Ben commented, "I liked that it was difficult. It really made me to think." Perhaps because it was difficult and impossible to make perfect decisions, learning what was wrong about the decision and receiving better re­sults after modifYing it encouraged participants to learn from their mistakes. When inappropriate choices are treated as their performance that can be im­proved rather than simply as incorrect answers, students can learn valuable les­sons from their choices (Reeves, 1997). These findings suggest that the level of challenge and a chance to modifY decisions influenced the participants' percep­tions of the value of the ILMs.

journal of Research on Technology in Education 469

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

CONCLUSION

This research was focused on determining student perceptions of the charac­teristics of ILMs. The data suggest that students believed the ILMs could influ­ence their ability to understand and apply concepts in real-world situations. Participants acknowledged that they learned not only theoretical concepts but also how to apply them. The findings of this study offered practical implica­tions in improvement and future development of the ILMs. This is important, because poorly designed ILMs may result in partial use, nonuse, or misuse of the information by learners (Jih & Reeves, 1992). Thus, the ILMs may contrib­ute to the development of an effective and integrated learning environment, as described by Chiou (1992). However, it is apparent that additional research is needed to gain a more thorough understanding of effective characteristics of computer-based learning experiences. In examining the effectiveness of ILMs, future studies could track student performance on material related to the ILMs. Although participants expected future use of the knowledge and skills presented in ILMs, this does not confirm their ability to apply them in real-world situa­tions. Also, verifying the relationships between the characteristics of the ILMs and learner differences would make a significant contribution to theoretical and practical understanding of computer-based learning (Neuman, 1989). •

Contributors Beaumie Kim is doing her doctoral work in the Department of Instructional

Technology at the University of Georgia. Richard Williams is an assistant pro­fessor in the Department of Recreation and Leisure Studies at East Carolina University. John Dattilo is a professor in the Department of Recreation and Leisure Studies at the University of Georgia. (Address: Beaumie Kim, 611 Aderhold Hall, The University of Georgia, Athens, GA 30602; bkim@coe.uga.edu.)

References

Alessi, S.M., & Trollip, S. R., (1991). Computer-based instruction: Method and development (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall

Anderson, J. R., Reder, L. M., & Simon, H. A. (1996). Situated learning and education. Educational Researcher, 25( 4), 5-11.

Authorware 5 [Computer software]. (1999). San Francisco: Macromedia, Inc. Baker, A. C., Jensen, P. J., & Kolb, D. A. (1997). In conversation: Transform­

ing experience into learning. Simulation and Gaming, 28(1), 6-12. Bogdan, R. C., & Biklen, S. K. (1992). Qualitative research for education (2nd

ed.). Boston, MA: Allyn & Bacon. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the

culture oflearning. Educational Researcher, 18(1), 32-42. Brown, C., Hedberg, J., & Harper, B. (1994). Metacognition as a basis for

learning support software. Performance Improvement Quarterly, 7(2), 3-26. Chiou, G. (1992). Situated learning, metaphors, and computer-based learn­

ing environments. Educational Technology, 32(8), 7-11.

470 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

Clark, R. E. (1994). Media will never influence learning. Educational Technol­ogy Resea-rch and Development, 42(2), 21-29.

Cognition and Technology Group at Vanderbilt. (1997). The jasper Project: Lessons in curriculum, instruction, assessment, and professional development. Mahwah, NJ: Lawrence Erlbaum.

Collins, A. (1991). Cognitive apprenticeship and instructional technology. In L. Idol & B. F. Jones (Eds.), Educational values and cognitive instruction: Impli­cations fo-r reform (pp. 119-136). Hillsdale, NJ: Lawrence Erlbaum.

Cote, N. (1994, November). Overcoming the inert knowledge problem in learn­ingftom t!xpository text. Paper presented at the annual meeting of the Mid-South Educational Research Association, Nashville, TN.

Cunningham, T. H., & Thorkildsen, R. J. (1996, April). Effects of combining ca;e-basea' instruction and mindfulness activities on the acquisition, application, and transfer of complex knowledge: An experimental comparison of two multiple­ca;~e treat;-nents on videodisc. Paper presented at the annual meeting of the Ameri­can Educational Research Association, New York.

Curtiss, F. R., & Hurd, P. D. (1980). Implementation of case-study, role-play­ing and small group discussion in a course in pharmaceutical law and ethics. American journal of Pharmaceutical Education, 44( 1), 56-61.

Dede, C. J. (1992). The future of multimedia: Bridging to virtual worlds. Educatior1al Technology, 32(5), 54-60.

Diamantes, T. ( 1996, August). A case for cases: Using the case method in the prepa­ration of administrators. Paper presented at the annual meeting of the National Council of Professors in Educational Administration, Corpus Christi, TX.

Director 7 [Computer software]. (1999). San Francisco: Macromedia, Inc. Ellington, H., Gordon, M., & Fowlie, J. (1998). Using games & simulations in

the classroom. London: Kogan Page. Foster, D. A., & Bareiss, R. (1995, April). Administering the business school

ca~:e method with a goal-based scenario. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.

Glaser, B. G., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualit~ztive research. Chicago: Aldine.

Glenn, P. J., Koschmann, T., & Conlee, M. (1997, November). Theory presen­tarion and assessment in a problem-based learning group. Paper presented at the annual rr_eeting of the National Communication Association, Chicago.

Gredler, M. (1994). Designing and evaluating games and simulations: A process approach. Houston, TX: Gulf Publishing.

Hasselbring, T. S., & Learning Technology Center at George Peabody Col­lege. ( 1991). An evaluation of specific videodisc courseware on student learning in a ru.ral schuol environment. A research report to Tennessee Valley Authority, Knox­ville. Nashville, TN: Learning Technology Center at George Peabody College.

Hudspeth, D. R. (1991). Interactivity and design of case materials. Perfor­mance Improvement Quarterly, 4(1), 63-72.

Jacobson, M. J., & Archodidou, A. (2000). The knowledge mediator frame­work: Toward the design of hypermedia tools for learning. In M. J. Jacobson, & R. B. Kozma (Eds.), Innovations in science and mathematics education: Ad-

journal {}_;+~Research on Technology in Education 471

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

vanced designs for technoiogies of learning (pp. 117-161). Mahwah, NJ: Erlbaum.

Jih, H.]., & Reeves, T. C. (1992). Mental models: A research focus for inter­active learning systems. Educational Technology Research and Development, 40(3), 39-53.

Johnson,]. F., & Morris, K . .\1. (1997). Courseware is the key: Bridging lec­ture and laboratory in general chemistry. Journal of Computing in Higher Educa­tion, 8(2), 109-125.

Jonassen, D. H. (1996). Scaffolding diagnostic reasoning in case-based learn­ing environments. journt:d ofC.Jmputing in Higher Education, 8(1), 48-68.

Jonassen, D. H., Campbell,]. P., & Davidson, M. E. (1994). Learning with media: Restructuring the debate. Educational Technology Research and Develop­ment, 42(2), 31-39.

Knirk, F. G. (1991). Case materials: Research and practice. Performance Im­provement Quarterly, 4(1), 73-81.

Koschmann, T. (1995). Medjcal education and computer literacy: Learning about, through, and with computers. Academic Medicine, 70(9), 818-821.

Kozma, R. B. ( 1991). Learning with media. Review of Educational Research, 61(2), 179-211.

Kozma, R. B. (1994a). \Vill media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7-19.

Kozma, R. B. (1994b). A reply: Media and methods. Educational Technology Research and Development, 42(3), 11-14.

LeBlanc, H. P. (1997, ~\Jovember). Distributed and problem-based learning techniques for the family communication course. Paper presented at the annual meeting of the National Communication Association, Chicago.

LeCompte, M.D., & Goetz, J.P. (1982). Problems of reliability and validity on ethnographic research. Review of Educational Research, 52( 1), 31-60.

Leedy, P. D. ( 1997). Practicai research: Planning and design (6th ed.). Englewood Cliffs, NJ: Prentice-Hall.

Naidu, S., & Oliver, M. (1996). Computer-supported collaborative problem­based learning: An instn:.cdonal design architecture for virtual learning in nurs­ing education. journal of Distance Education, 11(2), 1-22.

Neuman, D. (1989). Natura1istic inquiry and computer-based instruction: Rationale, procedures, and potential. Educational Technology Research and Devel­opment, 37(3), 39-51.

Odom, M.D., & Pourjalali, H. (1996). Knowledge transfer from expert sys­tems vs. traditional instruction: Do personality traits make a difference? journal of End User Computing, 8(2), 14-20.

Pickrell, J. A., & Departmem: of Clinical Sciences, Kansas State University. (1995). Enhancing large-group problem-based learning in veterinary medical edu­cation. Program description. Manhattan: Kansas State University.

Reeves, T. C. (1992). Evaluating interactive multimedia. Educational Technol­ogy, 32(5), 47-53.

472 Summer 2002: Volume 34 Number 4

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4

Reeves, T. C. ( 1997). Evaluating what really matters in computer-based educa­tion [Online document]. Available: www.educationau.edu.au/archives/cp/ reeves.htm.

Reisman, S. (1987). Selecting educational software for use with textbooks. Technological Horizons in Education, 14(5), 80-84.

Rieber, L. P. (1992). Computer-based microworlds: A bridge between con­structivism and direct instruction. Educational Technology Research and Develop­ment, 40(1), 93-106.

Rieber, L. P. (1998). The WWILD Team: A resource site for interactive learning modules [Online document]. Available: http://itechl.coe.uga.edu/wwild/.

Ryan, C., & Koschmann, T. (1994, June). The collaborative learning labora­tory: A technology-enriched environment to support problem-based learning. Paper presented at the National Educational Computing Conference, Boston.

Sanyal, C. C. ( 1991). Higher education and the labor market. In P. G. Altbach (Ed.), Higher education: An encyclopedia (Vol. 1, pp. 147-168). New York: Garland.

Sharf, B. F., & Poirier, S. (1988). Exploring (un)common ground: Communi­cation and literature in a health care setting. Communication Education, 37(3), 224-236.

Stumbo, N.J., & Little, S. L. (1993). Confirming, interpreting and reporting naturalistic research findings. In M. J. Malkin & C.Z. Howe (Eds.). Research in therapeutic recreation: Basic concepts and methods (pp. 279-298). State College, PA: Venture.

Taylor, S. J., & Bogdan, R. (1984). Introduction to qualitative research meth­ods. New York: John Wiley & Sons.

Tennyson, R. D. (1994). The big wrench vs. integrated approaches: The great media debate. Educational Technology Research and Development, 42(3), 15-28.

Vosniadou, S. (1996). Learning environments for representational growth and cognitive flexibility. InS. Vosniadou, E. deCorte, R. Glaser, & H. Mandl (Eds.), International perspectives on the design of technology-supported learning en­vironments (pp. 13-24). Mahwah, NJ: Erlbaum.

Wolfe, J. (1997). The effectiveness of business games in strategic management course work. Simulation & Gaming, 28(4), 360-376.

Yang, Y. (1992). The effects of media on motivation and content recall: Com­parison of computer- and print-based instruction. journal of Educational Tech­nology Systems, 20(2), 95-105.

Yin, R. K. (1994). Case study research: Design and methods (2nd ed.) Thou­sand Oaks: Sage.

Journal of Research on Technology in Education 473

Dow

nloa

ded

by [

Uni

vers

ity O

f Pi

ttsbu

rgh]

at 2

0:09

21

Oct

ober

201

4