newssim project evaluation - columbia...

Columbia Center for New Media Teachingand Learning

Evaluation Project 2001

Director: Peter Sommer

Evaluation of News Sim Projecthttp://www.ccnmtl.columbia.edu/projects/newssim

Evaluator: Laura Zadoff - [email protected]

CCNMTL Evaluation Project: News Reporting Simulation________________________________________________________________________________________________

2

Table of Contents

1. DESCRIPTION OF THE PROJECT.................................................................. 3

1.1 THE NEWS SIM PROJECT......................................................................................... 31.2 HISTORY OF THE PROJECT ....................................................................................... 71.3 DESCRIPTION OF THE NEWS REPORTING SIMULATION: A FIRE SCENARIO ................ 8

2. EVALUATION PLAN ....................................................................................... 16

2.1 GOALS OF THE EVALUATION ................................................................................. 162.2 RESEARCH METHODS ........................................................................................... 16

3. CRITERIA AND APPROACH TO THE STUDY............................................ 18

3.1 EVALUATION FRAMEWORK: RESEARCH ON SIMULATIONS FOR TEACHING AND

LEARNING .................................................................................................................. 18Characterization of simulations............................................................................. 18

3.2 ACTIVITIES AND MATERIALS FOR PRACTICING REPORTING AND WRITING IN RWICOURSES.................................................................................................................... 19

4. PARAMETERS IN THE STUDY ..................................................................... 21

4.1 CHARACTERISTICS OF THE PARTICIPANTS OF THE STUDY........................................ 214.2 CHARACTERISTICS OF THE CONTEXT OF USE .......................................................... 22

5. FINDINGS.......................................................................................................... 24

5.1 STUDENTS’ EXPERIENCES WITH THE NEWS SIM ..................................................... 24Simulation-Wide Reaction of Participants ............................................................. 24

5.2 KNOWLEDGE ARCHITECTURE AND NAVIGATION .................................................... 25Simulation-wide navigation................................................................................... 25Turning points....................................................................................................... 26

5.3 EDUCATIONAL ARCHITECTURE AND COGNITIVE ACTIVITIES ........................................ 30Learning objectives .................................................................................................. 31Students’ analysis of learning with the News Sim.......................................................... 35Unexpected use of resources...................................................................................... 42

5.4 COMPARISON WITH THE DRILLS............................................................................. 42

6. DISCUSSION AND SUGGESTIONS ............................................................... 44

The notion of activity............................................................................................. 44The learning environment...................................................................................... 46

7. CONCLUSIONS ................................................................................................ 49

8. REFERENCES................................................................................................... 50

APPENDIX: SAMPLE INSTRUMENTATION ....................................................... 52


3

1. Description of the Project

1.1 The News Sim Project

The News Reporting Simulation (News Sim) is a digital simulation of news reporting and writingfor journalism students. Was developed by two professors1 in the Journalism School at ColumbiaUniversity in partnership with the Columbia Center for New Media Teaching and Learning.

The News Sim project was developed to provide journalism students the possibility of practicingtheir newsgathering skills, interviewing techniques, and lead writing in a controlled environmentthat approximates some of the conditions found in the real world by reporters covering commonnews stories. The project consists of a series of scenarios, each scenario addresses differentaspects of news reporting and presents different news beats that reporters are assigned to. NewsSim: Reporting a Fire is the first one in the series of news reporting scenarios. It is the onlyscenario developed up to now. In this goal based scenario2, the student, in the role of the reporter,is introduced to Freeport News where he is assigned to the police beat, and covers a fire in a localapartment building as a spot news story for a local newspaper.

The simulation constitutes a self-contained learning environment. It is a stand-alone piece ofsoftware that can be accessed via the Internet.3

News Sim is text-centered with video clips, photographs and audio that help create the situationby presenting the different characters and scenes to the student4.

Intended audiences

The News Sim was developed for use by students attending the course Reporting and Writing I(RW1) at Columbia University’s Graduate School of Journalism. It is intended to provide themwith a supplemental learning experience, in which they can further practice the news gatheringskills, interviewing techniques, and writing that they are being taught in the course.

Reporting and Writing I is a core course required of all journalism students in their first term ofthe Master of Arts program. The course has a high time load; 9:00am to 6:00pm, Monday toThursday. During a majority of the course, students are required to participate in street reporting,fieldwork where they gather news, and deadline writing; they write leads and stories about theevents that they cover. In most of the courses instructors use facts sheets or drills for practicingwriting skills. The different instructors that teach RWI conduct their course in different ways.

The simulation is designed with the intention of providing the students an opportunity to practicesome of the skills involved in journalism practice in an environment in which the learner has to

1 Professor John Pavlik and Professor Melvin Mencher.2 “A GBS [goal based scenario] is a learn-by-doing simulation in which students pursue a goal by practicing targetskills and using relevant content knowledge to help them achieve their goal” (p.165) Goal based scenarios place thestudent in a role and a context that has a cover story and scenario operations. (Shank, Berman, and Macpherson.Learning by doing. In Reigeluth (1999) Instructional –Design Theories and Models. Lawrence Erlbaum Assciates.NJ.3 From: News Reporting Simulation Guide for Instructors.http://www.columbia.edu/ccnmtl/projects/newssim/instructor4 It contains three video clips, twelve photo-audios and nine photographs. It features an interactive digital map, adynamic audio-script, and a timer tool.


4

make decisions (determined by the educational objectives), giving them the desired educationalexperience without the adverse consequences of making wrong decisions. The goal is that thestudents learn about the field by thinking like the members of that practice community. Theylearn by doing under a role-play simulation. Simulations are aimed to “provide the learner withpractice of the behavior he/she will be called upon to exhibit in reality” (Gatto, p.1445)

It also aims to provide the instructors with a completely controlled news-reporting environment,where the instructor knows all the facts of the story, and can therefore be assured each studentencounters the same reporting assignment.

The story

The featured story was adapted from a fiction town presented in Melvin Mencher’s textbookNews Reporting and Writing. This textbook is being used in several of the RWI sections.

The readings

Different readings from Professor Mencher’s books are embedded in the simulation6. Theyprovide background reporting and writing information. They are presented in two different waysin the simulation; one is in the background readings toolbox that is on the left side of the screenall along the simulation. The other, in sections of the simulation sequence; the readings specific towriting constitute sections five and six in the simulation sequence. They are required steps beforebeing writing the story.

The learning objectives

News Sim designers intend that by using the simulation, journalism students will be able topractice their news gathering skills, interviewing techniques, and lead writing required forreporting a spot news.

The learning objectives and goals for the students using the News Sim are:

I. Acquire basic news gathering skills:a. Pay attention to details, including dispatcher’s communications on police scanner.b. Recognize when a fire is worth covering.c. Master basics of navigating through a news archive for relevant background materials.d. Know how to get to the location of a fire quickly, efficiently and safely.e. Identify main sources for a fire story, both people and documents, web sites.f. Formulate good questions for sources, including knowing when to follow up.g. Learn to pay attention to details, including spelling of names, getting correct addresses,

titles of sources.

5 Gatto, D (1993) The use of interactive computer simulations in training. Australian Journal of EducationalTechnology, 9(2), 144-156. http://wwwasu.murdoch.edu.au/gen/asset/ajet/ajet9/su93p144.html6 - Mencher, Melvin. News Reporting and Writing. 8th ed. New York: McGraw Hill, 2000. 462-463, 491-494, and 737-746 - Mencher, Melvin. Reporter's Checklist and Notebook. New York: McGraw Hill, 1996. 18-21. - The math and grammar quiz was taken from CD-Rom Brush-Up: A Quick Guide to Basic Writing and Math Skills by MelvinMencher and Wendy Shilton, © 2000 Melvin Mencher.


5

h. Get information needed to follow up on a story or lead, including contact information forsources.

i. Learn to check the facts.

II. Develop basic lead writing skills:a. Recognize basic story elements, including the 5Ws: who, what, when, where, why, and

how.b. Learn to structure a lead for a spot news story; learn what a spot news story is.c. Develop editorial judgment for her story as to what elements are most important

including cause of fire, damage/injury that results.d. Learn to follow AP style.e. Understand how to move past level 1 story to level 2 (digging beyond surface facts)f. Learn not to back into lead (i.e. Don’t put “when” first).g. Synthesize facts and present them in orderly, intelligent, clear and cogent fashion.h. Take criticism well (accept edits and respond to them without taking it personally, getting

defensive or upset).i. Learn to avoid the passive voice; use the active voice, and find strong active verb (e.g.

don’t say the fire “burned” the building when in fact fire “gutted” the building).j. Learn to attribute facts to credible sources.

III. Master basic communication skillsa. Know how to send/receive e-mail and thus get your story to your editor.b. Communicate appropriately with an editor.

IV. Develop moral compass for reportinga. Learn about errors of omission and commission, and why errors of commission represent

an ethical challenge to journalism.b. Recognize the importance of privacy in this story.

V. Develop basic mathematics skills relevant to journalism(e.g. how to calculate a percentage and how to correctly interpret a percentage).7

The instructors’ guide – customizing possibilities

The Guide for instructors can be accessed from the splash screen of the simulation. It contains adescription of the simulation, and provides all the information that is included in the story withnotes on the accurateness, how the student obtains it, and if it should be used in the lead of thestory. It also contains the referenced information about lead and story writing and backgroundreadings.

The News Sim is intended to provide the possibility of being used in different ways by differentinstructors. Not only in the way instructors can use it in their classes but also by offering thepossibility of modifying the content to customize it to the teaching preferences of the instructor. 8

The guide for instructors offers the possibility of customizing the News Sim according toinstructor’s preferences in three ways; the first one is using the entire simulation (the seven

7 Information taken from the general project information document, different documents related to the envisioningphase of the project, and the Instructor’s guide in the News Sim.8 From: News Reporting Simulation Guide for Instructors.http://www.columbia.edu/ccnmtl/projects/newssim/instructor


6

sections), the second one skips sections two and three starting from the interviewing section, andthe third one is a partially condensed version, where the student skips the presentation sectionsbut can refer to the background readings. This last customization strategy is the one alternativethat has been built in the News Sim and is described below in this report. The News Sim isdesigned for use in combination with other in-class assignments and discussion. The decision onhow to integrate it is left to each instructor.9

One of the characteristics of the project in the proposal stage was to have students’ actions in thesimulation recorded and made available to the instructor for possible evaluation. Functioning as apractice place for students at the same time as an evaluation tool for the instructors. However, thispossibility has not yet been developed.

9 From: News Reporting Simulation Guide for Instructors.http://www.columbia.edu/ccnmtl/projects/newssim/instructor


7

1.2 History of the project

Origin

The actual development of the project began on February 2001. Developer professor, who had theoriginal idea, submitted the proposal to CCNMTL October 30th 2000. He invited the otherprofessor to help in the project later on in the development. The two professors came up with thescenario, based on previous experiences of fire reporting simulations developed by one of theprofessors and fictional stories developed by the other professor in his news reporting and writingtextbooks.

The original idea was to provide a space for beginning student reporters to practice their reportingskills, to have automatic feedback and to have access to a news story or event they might nototherwise have access to.10 The design of a simulation for journalism students that would allowapplying apprenticeship to large introductory classes would allow students to learn about the fieldby thinking like reporters and provide them with an educational experience without the adverseconsequences of making wrong decisions. The simulation would immerse the student in asituation in which she has to make decisions: the situation and the decisions to be made aredetermined by the educational objectives.11

Audiences

From the beginning of the development, the project was intended for beginner students in theMaster of Arts program in the Journalism School, specifically for the students in RWI course asthis course is designed to teach the students the basic reporting and writing skills. RWI was theoriginal target course for the project12. However, due to differences in teaching styles among theprofessors the News Sim has not yet been used in that course.

At the beginning of this term (fall 2001) the developer professor asked the three students inExploring New Media to test it (their comments are included in this evaluation). He thought thiscould be useful for his students in Exploring New Media as well even though the content is notdirectly relevant to his class.

An important fact mentioned by one of the professors and that should be considered to understandthe situation is that the purpose of RWI is to bring students who come from different backgroundsand experiences, to a common level.13 Considering this, using the simulation with students takingthe August introductory part of RWI could be a good option. The simulation was not used thisyear with students in the August part of RWI courses because the project was not ready on timefor that date. The context of use is still being defined.

10 Information taken from the project proposal.11 Information taken from the project proposal.12 Information provided by Project Manager.13 October 11th 2001 meeting with developer professors, two professors in RWI course, and Project Manager.


8

Project modifications

Considering that the professors involved were not teaching RWI, the original intention for thedevelopment was to have professors of RWI collaborate with the project. RWI faculty was notinvolved in the design but was consulted during the development of the project to providefeedback. According to this feedback the project was reshaped from its original ideas in order toaccommodate professor’s needs. Some of the reconfigurations include the following: Thebackground readings were shortened and the navigation was made simpler and more directedafter instructor suggestions. The original design had different fires to select from, and thedecision-making process was in general more complex. One of the modifications wasincorporated as an alternative design or customization considering the needs and suggestionsfrom one of the instructors. The description of this condensed version is included in the nextsection of this report.

1.3 Description of the News Reporting Simulation: A Fire Scenario

The News Sim is divided in seven sections: Introduction, Freeport News, The Police Beat,Covering the story, Lead and Story Structure, About the Lead, and Writing the Story. It alsocontains a splash screen, a transitional section, and a final section. These sections are organizedone after the other in the way they are presented in this description. The student cannot skipsections or screens.

The simulation has a linear structure and it is intended to be navigated in a single progressivedirection. The navigation to proceed to following screens and sections is mostly accomplished byclicking on text links. In some cases through binary optional choices, and in some others throughsingle selection. The navigation is strongly guided; incorrect options are cancelled by revertingback to correct screens. This navigation mode characterizes most of the simulation, but not all,the exception is the use of the map, where there are not single or binary options for navigation buta complex tool that does not show one or two visible paths but multiple possible navigationactions even though the correct path that will advance the simulation is still only one.

Splash Screen

News Reporting Simulation: a Fire Scenario openswith an image of a fire and a brief presentation of thesituation that places the student in the role of areporter:

‘A five-alarm fire rages late at night in a high-riseapartment in Freeport, a medium-sized city in theMidwest. On the police beat, you hear the report onthe police scanner. Your assignment is to cover thefire for The Freeport News within a two-hourdeadline.’

The student is prompted to start the simulation by clicking on the name of his instructor. Thesplash screen also contains links to the about this site page, help, and a guide to instructors thatare described in other section of this report. The names of two instructors: the developerinstructor and an instructor for RWI course. RWI professor option is a condensed version of the


9

simulation that skips the second and third sections; the ones introducing Freeport News and thePolice Beat. The description that follows is the developer professor’s - the complete version.

Section I: Introduction

The introduction screen has a movie clip of a fireaccompanying a text. The text gives an overview ofthe simulation and presents the student with thesituation and the actions he will be doing. It alsoprompts the student to read the background readingsand do the quiz before proceeding to Freeport News.The background readings are a series of hyperlinkedtexts indexed in a box on the left hand side of thescreen.The text warns the student about the time constrainand presents the main sections that compose the

simulation and that are identified in the horizontal menu at the top of the screen.

The background readings (they open in a different window):

Journalism Basics: A Primer, is a text written by thedeveloper professors. It is about three pages long and it presentsbasic information about reporting in general and about reportinga fire and this simulation in particular. It addresses the decisionsthat a journalist should make when reporting a fire like how tojudge if a fire is worth covering, what tools to take to the scene of the fire, or how to locatesources and some ethical issues and errors. Some of them relevant to the simulation and someabout reporting a fire in general but that are not present as possibilities in this simulation; likeconsiderations about using a tape recorder and a camera, about searching a database, about askingcontact information for following up with sources, or about checking the morgue or the newslibrary. This text also offers tips for note taking, quoting, source selection, attention to spelling,and lead writing. It references the students to the textbook for more note taking and writing tipsand to online database for examples of fire stories.

‘Police Beat’, is a two-page text taken from the textbook that describes the types of differentstories that are covered by police reporters and the range and cost of crime.

‘Fire Coverage’, is a one-page text taken from the textbook that presents how a police reporterknows about and decides to cover a fire. Examples of fire stories and considerations about therelation between the content and the time, about getting in the way of firemen, and an example ofa reporter who followed a false source on a cause of a fire (very similar to the situation presentedin the simulation).

‘Useful Math for Reporters’ is a math explanatory text from the textbook that explains howto find percentages, averages, rates, and ratios, with examples that can be related to news stories.

‘Grammar for Journalists’, explanations about agreement, dangling modifiers, misplacedwords, parallel construction, pronouns, sentence fragments, sequence of tenses, spelling, uses,jargon, clichés, and wordiness, with wrong and right examples mostly related to journalism ingeneral.


10

‘Math and Grammar Quiz’, consists of 5 multiple-choice math or grammar questions. Thepassing grade is 60%. If the students do not pass it they can take another quiz, up to four. Itspurpose is to provide students with an element that certifies them as capable of working asreporters and to make students aware of their proficiency in basic math and grammar skills thatare essential for journalists (however, it is not all directly related to this fire scenario).

Section II: Freeport News

This section starts with an introductory screen thatoffers information about Freeport News newspaper, itscirculation, penetration, readership, and onlinepresence.

The following screens in this section introduce thestudent to different people working in Freeport News:The executive Editor, the City Editor, and the SystemsEditor. In each screen the student sees a photograph ofthe introduced person, a text with information aboutthem and they can listen to this person speaking to

them. The Executive Editor welcomes and gives advice to the new reporter consisting of somereminders like not to forget about deadlines, to check the facts and that they are from reliable

sources, and sends the reporter to see the City Editor.The student has then the option to go talk to the CityEditor or to the Systems Editor, by clicking on one ofthe options offered on the screen. If he selects to gotalk to the Systems Editor (wrong option, according towhat the Executive Editor said) a new screen for theExecutive Editor makes clear that he needs to choosethe other option – go talk to the City Editor. The CityEditor’s audio assigns the reporter to the police beat,and explains to him what he has to do; go to the policestation, get to know what’s going on in the police beat,and listen to the police scanner. He explains what the

scanner is for and suggests that the reporter should meet the Systems Editor before leaving for thepolice station and to check with him before covering the any stories and to discuss the story. Andhe reminds about the deadline (in two hours) explainingwhy if it is missed the story won’t make that morningpaper, and for the next day his story would be irrelevant.The student has then the option to go to the Police Station,or to go talk to the Systems Editor, by clicking on one ofthe options offered on the screen. If he selects to go to thePolice Station (wrong option, according to what the CityEditor said) a new screen for the City Editor makes clearthat he needs to choose the other option – go talk to the Systems Editor, before heading to thePolice Station. The Systems Editor audio presents the digital technology that a reporter can use.

A new toolbox appears on the screen above the Background readings box: the reporter tools. Thetext on the screen directs the student to look at it. It contains the following titles: Mobile Phone,Digital Audio Recorder, Digital Camera, PDA, Freeport City Map, Reporter’s Checklist andNotebook, and Personal Computer. When clicking on them a window pops up containing a


11

photograph of the tool and a brief description. The only one of these tools that is active is theFreeport City Map. Even though the map can be launched from this screen at this moment in thesimulation, the students don’t know its purpose yet, and do not engage it at this time.

The student has then the option to get to the FreeportPolice Station, by clicking on one of the options offered onthe screen. If he selects to go talk to the City Editor (wrongoption, according to what the Systems Editor said) a newscreen for the Systems Editor makes clear that he needs tochoose the other option – get to the Freeport PoliceStation. When the student clicks on this option, the screenwith the Freeport City Map pops up.

Transition: The Map

In order to get to the Freeport Police Station, the reporterhas to use the map. The pop up screen contains apresentation of the interactive digital map as a needed toolto go around in Freeport City, and describes fourhypothetical situations in which a map could be needed.From that screen the student can launch the map or go tothe map tutorial. The map tutorial describes the function ofthe navigation control panel and the directory panel; themain tools that enable the user to search for persons and locations on the map. It describes theuses of the Small Scale Map, the Pan, Zoom and Reset Buttons, the Directory and CrossDirectory, and the Links from locations functions of the map. The Directories Panel is quitecomplex and less intuitive than the other features of the map tool. The directories are the mostimportant aspect of the map because they constitute the only way for navigating the map and thesimulation.

When launched, the map opens in the mainsimulation screen.

Even though the student can use the tools in thenavigation panel to look at the map, they are notuseful to go places. In order to get to the policestation, the student needs to use the directory. Byselecting the letter P in the directory, and scrollingdown the names, one can find the Police Stationlisted alphabetically. When clicking on it, aflashing star shows up on the zoomed-in map

where the police station is. To go to the location, the star should be clicked. The screen changesand the student is now in the Police Beat.

Section III: The Police Beat


12

In this section the student is at the Police Station. Thetext welcomes the student to Freeport Police Stationand describes its relationship with journalists. Thendescribes what a reporter does in the police station,specifying that ‘the reporter will use the policescanner, listening for codes that indicatemajor breaking stories, including crime,accidents or fires.’

A link under the photograph drives the student to visitthe veteran reporter.

When the student gets to this page, the voice of theveteran reporter starts automatically presenting herselfand providing advice. In the audio the veteran reporterguides the student’s attention to certain codes, tellshim that he should look at the list of codes and the firechecklist - these two resources appear in a box on theleft side of the screen. She also reminds the student toget the facts from the right sources, giving as anexample ‘ask how the fire started from the fireMarshal…’, she guides the student to the policescanner and mentions the map. The text in this screen

has information about the veteran reporter and characteristics of her working style that serve asexamples of what the student should do.

The text reinforces, once more, some aspects of reportingthat the student should learn and perform. It also guidesstudent’s attention to the new toolbox containing the firecheck list and Freeport fire codes and tells the student toprint the documents.

The fire checklist is a list of elements the reporter has tofind information about; the list is taken from thetextbookr’s Reporter Checklist and Notebook. The firecodes lists the codes that include the ones used in thescanner script.

The options on the screen offer the student to leave thepolice station or to listen to the scanner. If he selects to goto leave the Police Station (wrong option) a new screen forthe veteran reporter makes clear that he needs to choosethe other option – listen to the scanner, and reminds theimportant codes to the new reporter.


13

The text on this screen explains to the student what hehas to do, reminds him of the two most importantcodes he has to pay attention to, and prompts him tocall the city editor if he hears a fire to report. Theaudio and script play simultaneously and at a quickspeed.

The relevant fire alarm is mentioned right at thebeginning of the audio and script and repeated togetherwith the address of the fire.Other codes are mentioned after

it repeating the relevance of the fire. After checking with the city Editor,the time, in a pop up window starts counting the two hours that the reporterhas as a deadline.

The student should then use the map to go to the location of the fire. In order to get to the addressof the fire, the student should select the Cross directory on the map, and use it selecting the initialfor the road and browsing for the apartment building.

Section IV: Covering the Story

In this section the student is at the scene of the fire andhe will interview different sources. Four sources areorganized in following screens: the Fire Marshal, theHospital Official, the Building Owner, and theFirefighter.

The first screen is the Fire Marshal. The text remindsthe student of the importance of getting the statementfrom the fire marshal first, and reminds him of therelevant information he should get from him. When

playing the video, the fire marshal retells main information about the fire; time, life loss, cause ofthe fire. After listening to the fire marshal the student can interview him by selecting from a list

of eleven questions that is provided. Each of them hasan audio response from the fire marshal. According tothe relevance criteria instructed in the fire checklistand background readings all the questions are relevant.The answers are short, however not always precise. Alink at the bottom of the screen guides the student tovisit the Hospital official, from there, the buildingowner, and from the building owner, to the firefighter.The Hospital official is a video too, she talks first andthere are four possible questions to ask her. Thebuilding owner is represented in a photograph withfour questions to be asked, and the firefighter can

respond to two questions. The responses from the Fire Marshal and from the Building Ownerdiffer regarding the estimated damage from the fire. Another contradiction in the information isregarding the cause of the fire as the Fire Marshal and the firefighter mention different causes.


14

In the fourth of the chained sources to interview there is an option: to use the map to visit abuilding resident or neighbor, or to start developing the lead and story structure. By clicking onthe option of visiting a neighbor, the map pops up. Using the cross directory the student can selectfrom the names listed under the mentioned address. For most of the neighbors, a pop up windowinforms that they are unavailable or mourning. Two of them can be interviewed. They answer afew questions that are less relevant for the story. After interviewing the second neighbor, a singlelink guides the student to develop the lead and story structure screen.

Section V: Lead and Story Structure

This section presents a text with diagrams andexamples explaining how to build a two and three-element story structures. The make-believe of thesimulation is suspended with this screen that has moreof a background reading-style yet appears in the samemain window. The text in the Lead and Story Structuresection has a textbook narrative and scanned diagramsfor structuring a story. Clicking on them can enlargethe diagrams. The text explains the diagrams forwriting two and three-element stories.

Section VI: About the Lead

The text in this section prompts the student to decideon the lead of his story. There are several examples ofsingle, two and three element leads from newspaperarticles with explanations. A link at the bottom of thescreen brings an introduction for writing the story.

Section VII: Writing the Story

In this section a short text reminds the student of somewriting tips. Next the student chooses the name of hisprofessor to access the space for writing the story. Thescreen for writing the story is divided into the lead andthe story. The student is required to type his emailaddress. A copy of the story is sent to the professor,and another copy to the student. At this point, thestudent has his notes and he can access the backgroundreadings and other toolboxes from the left of thescreen.Once the story is submitted the student is invited to do

a survey and to start the simulation all over.


15

The customized version, accessed by choosing Professor Gissler from the splash screen, startswith an introduction similar to the full version but presents the address of the fire to the student,as he is not going to listen to it from the police scanner, skips sections two and three: where thestudent is introduced to Freeport News and listens to the scanner in the Police Beat. Goingdirectly to section four: ‘Covering the Story’. The student is not prompted to read the backgroundreadings but the toolboxes appear on the left of the screen throughout the simulation, so they canbe used as reference materials.


16

2. Evaluation Plan

2.1 Goals of the evaluation

The intention of this formative evaluation is to provide information that would be useful forimproving the News Reporting Simulation project. “Formative evaluation occurs prior to releasein the general student population. The feedback is used to develop the product further. Formativeevaluation is of great value to the educator because it indicates how the package can beimproved…”14 “This information can be used to ‘fine-tune’ or terminate the project. The datacollected as part of a Progress Evaluation can also contribute to, or form the basis for, aSummative Evaluation study conducted at some future date.”15

The purpose of this evaluation is to identify the effectiveness of the News Reporting Simulation:A Fire Scenario as a teaching tool. It attempts to understand the learning experiences promoted bythis simulation. By looking at these experiences the evaluation intends to provide information thatmay improve characteristics of the simulation and further develop it. In order to address thispurpose its goals are:

- To monitor participants perceptions/reactions to the News Sim.- To provide feedback on the functionality/navigation of the simulation.- To validate the usefulness of the different features and activities for the learning

objectives of the simulation.

While pursuing these goals the evaluation will allow for the discovery of unintended results andneeds that might inform future developments of the project.

2.2 Research Methods

The methods used for the evaluation included:

- Observation of student testers with video stimulated recall interviews

The observation of students using the simulation served to obtain a detailed understanding of theways students use the project and the problems they encounter. The interview with video-stimulated recall after the observation helped reveal why students use the simulation in the waythey did, and find out about intentions or possibilities that the students would find valuable andcould not experience in the simulation. The observation consisted of five students from theColumbia Journalism School, all of them in their first year at Columbia University, however, atdifferent stages of their professional careers and therefore with different level of experience. Thestudents used the News Sim in one session while being observed. After using the simulation theywere interviewed about their experience, using the video to explain the motives and thought,which guided their actions. The students were also asked to explain their reactions to different

14 http://www.monash.edu.au/informatics/TechME/evaluati.htm#Evaluation%2015 http://www.cchs.usyd.edu.au/arow/m06/formative.htm


17

aspects of the simulation and its features, about its navigation, and about their learning. Eachsession had a duration of approximately three hours. The evaluation instruments used are:Observation Schedule, Interview Questions, and Observation report. Samples of theseinstruments are included in this document.

- Focus group with students who used the simulation for Exploring New Media class

The Focus Group with the students in developer professor’s class Exploring New Media(Professor and the three students). Allowed us to elicit students’ reactions to the project andinterpret and prioritize the difficulties this specific group of students experienced. The questionsthat guided the discussion in the focus group are included in this document.

- Analysis of built in survey

At the end of the News Sim there is a survey that requests the student opinion on the relevance ofthe different features and aspects of the simulation for the learning experience. Seven studentscompleted the survey - all but one of the student testers. The data obtained from this survey wasanalyzed and crossed with the data obtained from the student interviews.

- Interviews to faculty partner

The faculty partner was interviewed at two different moments of the evaluation; in theseinterviews we discussed general topics about the use the News Simulation in Exploring NewMedia class, the learning objectives, the history of the project, and more specific elements anddecisions included in the design. The questions for these interviews are included in thisdocument.

- Interviews to a professor in RWI

The Interview to professor of RWI allowed us to obtain expert comment on differentcharacteristics of the simulation: 1- from a perspective of a professor of possible user courses; 2-to research about the objectives and teaching methods used in RWI; 3- about professor’sperception about students’ difficulties in order to consider the possibilities of the News Sim inaddressing those needs and being a useful resource for those methods. The questions asked inthese two interviews are included in this document.

- Interviews to the project manager

The interviews with the project manager provided information about the characteristics of theproject, the intentions, the history, and development of it.


18

3. Criteria and approach to the study

In order to address the purposes of this study we have researched on the field of simulations andits educational implications. In this section we would like to introduce some of the definitions,criteria, and theories underlying simulation development. We will make explicit what weunderstand as simulation, and the educational characteristics considered in the analysis throughwhich we are addressing the educational value and potential of the News Sim. We will alsointroduce some of the main features involved in teaching and learning RWI course.

3.1 Evaluation framework: research on simulations for teaching and learning

Even though there is extended research on simulation for Medical, Physical, and EnvironmentalSciences, the research on simulations for Social Sciences is more limited. No research was foundon use of simulations for learning news reporting.

Most of the simulations in social sciences (for adults) were found in business, political sciencesand legal education. These research is characterized by the fact that the simulations explored aremostly designed for collaborative experiences, having more than one participant, and using theweb as communication tool for interaction between participants “ web-based simulations in whichCMC [computer mediated communication] is used to create a space in which students playingroles connect to each other and generate the simulation activity in the field being studied.”(Naiduand Linser, 1999)

Only one simulation on news reporting was found on the Internet: City Council. www.rcameron.com/journalism/citycouncil/

City Council Simulation is a simply constructed web environment illustrated with drawings thatrepresent resources to access information. The media features are limited to hyperlink text anddrawings organized with a top frame as navigation bar.

Characterization of simulations

The existing research on simulations involved definitions for computer based simulations thatsupport their value from a constructivist learning perspective.

The following definition captures the most important research on this perspective: simulations are“based on the principles of goal-based learning, and learning by doing in authentic environmentswhere students are allowed the opportunity to acquire the intended learning outcomes by makingmistakes in safe environments” (Naidu, Ip, and Linser, 2000).

From the goal based scenario perspective (Shank, 1997) simulations present a context andscenario where the students have a role and learn by doing, by solving problems involved in theskills needed to accomplish the goal.


19

Simulations let the learner practice a skill in a realistic situation. They allow the process ofconstructing new knowledge to be situated or anchored (CTGV) in a meaningful context. In thesesituations the learner can manipulate or act within the environment in ways that would bedifficult, expensive, dangerous, inaccessible, too time consuming, etc. to be provided in reality.Safe environments allow for making mistakes and learning from them– allowing students to testtheir conceptions and therefore helping them develop skills. Therefore simulations invite to useour knowledge, test our hypothesis in a context where mistakes are not so harmful. Andconsequences to our actions help us reconfigure understandings that guide our practices. In thisprocess learners reconstruct concepts, schema, mental models, and cognitive structures in the faceof new information and experience that may conflict with earlier constructions (Reeves).Simulations Promote inductive learning as they allow observing the result of actions or decisionsthrough feedback generated by the simulation. And therefore reshaping their mental model of thesystem or task that is the basis for their decision-making and action when confronted withproblems in the target system (Rieber, 1996). These skills appear as meaningfully framed in theenvironment and therefore they allow for a more easy transference.

The constructivist perspective that knowledge is constructed on the basis of experience supportsthe use of simulations. And there are several ways of knowing. With simulations, participantslearn by doing, they make participants experience ideas and feel these ideas; knowledge is linkedto experience and developed from reflection. (Petranek, 2000)

3.2 Activities and materials for practicing reporting and writing in RWI courses

The following are some of the main activities involved in the process of teaching andlearning:

Drills/fact sheetsThese are sheets with facts to use for practicing writing stories. Students learn by writing manystories in the process of using drills or fact sheets. They allow the student to practice writing inshort periods of time. Then the professor would correct their stories and the students would lookat the corrections and good examples.

Street reportingStudents are asked to go to out to cover certain beats (geographic zones or topics) each week.The reporting activities increase in complexity by involving the students in a more active role inthe reporting.

GuestsProfessor brings a guest to the classroom to be interviewed by the students. Then studentswrite the story.

Professor revisionsProfessor correction of students’ stories. Some of these corrections are discussed in class.

Group editsThe instructor selects a few stories done by the students. He hands in a copy to each student in theclass and students have to edit them and then they discuss the edits. This peer feedback is highly


20

valued by the students because they discuss the decisions taken during the writing andnewsgathering exercise.

TextbooksSome of the RWI instructors use textbooks in their course. Some of the instructors use the NewsReporting and Writing book on their courses. The style of teaching by the textbook isthrough examples and pieces of advice from experienced reporters, describing situations,contextualizing examples, providing explanations from the authors of the exemplarwritings anecdotes. In addition the textbook offers examples of poor stories and explainswhy. It shows many examples for each suggestion with an emphasis on personal style,and the unavoidability of mistakes during the learning process.


21

4. Parameters in the study

This evaluation consisted of an in-depth analysis of the interaction and learning experience of theparticipants. The contexts in which the students are assigned to use the News Sim are not thesame ones as intended in the simulation design. As already described, the situation in which theparticipants use the News Sim influences their experiences. Therefore the findings of thisevaluation cannot be translated to what could occur in a course context. The learning that wouldtake place in those situations would be determined by the characteristics of the use, theparticipants, and the context in which the simulation is sited.

The characteristics of the student testers in this evaluation, and of the context in which theevaluation takes place are described below.

Even though the students are considered beginners when they enroll in Journalism School, theyarrive the first semester with different amounts and qualities of experiences. This is an importantelement to consider, as the heterogeneity in the level of the students is a characteristic of RWIstudents.

4.1 Characteristics of the participants of the study

Students that participated in the observation

The five students that were observed and interviewed with video stimulated recall are in their firstsemester in the Journalism school. One of the students came by request of the partner professor.The other four students responded voluntarily to two e-mail ads distributed to first semesterjournalism students. Only three students responded to the first ad and only one student to thesecond ad. This characteristic of the selection of student testers should be considered as veryrelevant. It not only shows the interest of these students, and their predisposition to it, but alsotheir time availability and confidence to participate in the study during school time.

Even though the students are all in their first semester in the Journalism School, their previousexperiences in journalism practice and their ages are very different:

Student 4: is in the Master’s project course and in RWI course. Her major and interest is in newmedia journalism. She works freelance writing feature stories. She is a literary writer of poetry.She doesn’t have work experience in reporting spot news. For RWI she had to write a court storythat was quite tough. Experience with computers: uses computers every day, for several yearsnow. She is in her mid twenties and bilingual English-Spanish.

Student 5: is in the MA program in the Journalism school. He was a reporter in Cleveland Cityfor 3 years and he worked in Africa for 6 years. The courses he is taking are: RWI, Internationalreporting, critical law and society, radio skills and TV skills. He was interested in doing thesimulation because he was curious about the simulation. He uses computers regularly. He is in hisearly thirties.


22

Student 6: is in the MA program in the Journalism school. The courses he is taking are: RWI,Foreign affairs, Critical Issues, Law and Society. He responded to the ad because he thought thatit is difficult to teach to be a journalist, especially he wonders if reporting can be taught. He was asports journalist for a year. He did not like very much the people in his job. He mentioned later inthe interview that people where he worked would do this story in 10 or 15 minutes. ‘I thinkeverybody should be able to do it’. In the future he would like to be a reporter of conflict. Hethinks that in his section of RWI the greatest emphasis is writing. He is in his early twenties

Student 7: is in the mid-career program in the Journalism School. She was interested in seeing asimulation about journalism, and that is why she responded to the ad. She previously taughtreporting in college in the past and said that if she had something like the News Sim she woulduse it in her teaching. She worked as a reporter before teaching, but never covered a fire. Shestarted using computers when she already was an adult, but now she uses them regularly. She isin her forties.

Student 8: has long experience in reporting. He has a M.A. already and he is interested in newmedia. The courses he is taking are: pro seminar for the PhD program, information systems,survey and film studies. Experience with computers: uses computers every day, for several yearsnow. He is in his forties.

Students that participated in the focus group

The Partner Faculty has asked his students in Exploring New Media, to use the simulation.There are 3 students in his class. Two of the students in Exploring New Media are students fromthe School of International and Public Affairs and one from Teachers College, they are not in theJournalism School, so they don’t have any journalism experience and they do not intend to bejournalists. The goal for these students in using the simulation was more related to exploringtechnology possibilities than learning reporting and writing. Even though the reactions andsuggestions of these students were taken into account in the evaluation process, the difference inpurposes and background was significant difference which can be observed in their descriptionsand in the stories they submitted.

Faculty participants

Only one of RWI faculty was able to participate in an interview for the evaluation. Thecomments, perceptions, and suggestions of another faculty member of RWI were obtained in agroup meeting.

An important characteristic of this project is that the faculty partner in the development of theNews Sim is not professor of the target course. This aspect worked as a limitation for theevaluation because we could not rely on his expertise for analyzing student work andperformance.

4.2 Characteristics of the context of use

As described above, the students under observation were not using the simulation according tothe intention of the Project developers. The simulation is not being used in RWI courses at the


23

present. Two of the participants are not in RWI course even though they are in their first semesterin the Journalism school. The study participants responded to the email ad and scheduled anappointment to work with the simulation because of their own interest in doing it. Theparticipants worked in the simulation in a computer lab (Butler 204). They were not giveninstructions regarding the use of the News Sim. They were observed by the evaluator. During thetwo hours or one hour and a half that the students worked with the News Sim they did not talk toanybody. They just referred to the evaluator when they had a problem. When they finished, thevideo stimulated recall interview, in a sense, worked as a way of sharing the experience orreflection. The characteristics of using the simulation as a two-hour evaluation or as arequirement part of a course may be very different specially for considering its educationalpossibilities.

For further evaluations would be very interesting to evaluate how the News Sim could beincorporated in the context of RWI courses, the range of what uses the professors envision for it,and how they present and incorporate it into their teaching. It would also be interesting to explorethe use of the News Sim by novice Journalism students.


24

5. Findings

In this section we describe the findings of the evaluation. These findings will be discussed in thefollowing section.

A first finding that emerged from the research is the innovative role of this simulation in thisdomain. The News Sim, inaugurates the use of hypermedia web-based simulations in the teachingof introductory news reporting and writing.

5.1 Students’ experiences with the News Sim

Simulation-Wide Reaction of Participants

All the students that participated in the evaluation had a positive reaction to the News Sim. Theymentioned it was worthwhile, interesting, and fun. They appreciated its thoroughness, the use ofmultimedia, and the way the different sections and resources are connected.

When using the simulation, the students we observed exhibited a strong concentration on the task.They were attentive and engaged during the entire simulation - spanning two hours. The situationand task presented by the News Sim is attractive and motivating to the students.

By looking at student experiences with the News Sim we can organize the simulation inthree parts. This constitutes a construction for analysis that will be useful to organize findings.

Part 1: Introduction

This first part includes the introduction of the student reporter to his role, task, and the context ofthe simulation. Students describe that the simulation [Student 5:] “brings you in”. It includes theIntroduction, Freeport News, and Police Beat sections.

The setting of the context of the simulation is an important aspect, especially for beginners, tounderstand the situation and meaning of their activity.

This first part is characterized by a presentation style. The students are shown or introduced to thesituation, although they have to make certain selections in the navigation they are generally beingwalked through.

Part 2: Reporting

In this part students go to the fire scene and interview different people.

All of the students valued having the possibility of audiovisual access to sources. The multiplemodes of representation give them the possibility to practice gathering the facts for writing theirstory by listening to the source. The possibility of hearing the sources is something richlyvalued by the students, and the professors as well, because it is a realistic characteristic ofthe tasks fundamental for reporting.


25

Students strongly concentrated through part 2 paying attention to the audio and takingnotes. All of the students selected all the questions available to interview the sources inan orderly manner. In this part the time appears as a constraint that reminds studentsabout the deadline. All the students mentioned this aspect as realistic and important. Onlyone of the students mentioned the time element as a constant concern during thesimulation. The other students thought that two hours would be enough to interview andwrite at their pace or just forgot about it while engaged in reporting and writing.

Part 3: Writing

This part includes the last three sections of the News Sim; Lead and Story Structure, About theLead, and Writing the Story. All the students were very engaged writing their stories.

Interestingly, none of the observed students read the readings that constitute the sections on thelead and story writing, in detail. Most of them just skimmed over the first sentences of the screenand went forward to the writing part because they were involved in the role and therefore ready towrite. The use of the readings within the simulation for beginner students is discussed further inthis report.

Another interesting characteristic of students’ work in the writing part is that most of the studentsdid not pay attention to the number of words while writing. When asked about the length of theirstories after they wrote them, they didn’t know or made an estimated assumption. The storieswritten by the students in Exploring New Media are: 164, 269, and 237 long. The stories writtenby the Journalism students are: 154, 378, 271, 345, and 259 words long. Only two of thesestudents remembered vaguely that they were asked to write 250 words when asked but theyexplained that they did not remember while writing.

5.2 Knowledge architecture and Navigation

In this section we describe the findings of the observation and commentary on the mannerstudents navigated through the simulation and the presented information.

Simulation-wide navigation

All the participants of the study found the simulation easy to navigate as a whole. Most of thetime they navigated the site in a fluid manner. And, as they expressed in the survey, they foundthat the simulation was logically organized and that it was easy to move through the readings.

All the students manifested a positive reaction about the articulation between the different media.And they took advantage of the possibility of ‘playing again’ the audios. All of them played againat least some of the sources and some stopped them in order to copy the quotes. Two studentsplayed the scanner again.


26

Turning points

In order to understand the elements that appeared as navigation challenges for the students wehave identified what we call turning points. These are moments in the navigation of thesimulation when students had a reaction, a recurrent unexpected action, a problem, or a requestfor help. These turning points allow relevant glimpses into the student experiences in learningwith the simulation. We will present them in here and relate them with the other findings in thediscussion section.

The Top Mapping Bar and the Unidentified Links

All the observed students tried to use the Top Mapping Bar to navigate the site. The TopMapping Bar is a horizontal list of section names located at the top of every screen. It showslocation in the simulation sequence by illuminating the visited sections – the names of thesections are not links. Most of the students attempted using the Top Mapping Bar to go to othersections more than once, in further screens, because they thought it might not be working well.This may be a result of an internalized convention on navigation; menus or list of sections at thetop of the screens are navigation bars that allow website users to go from one screen in the site toanother one. Students intended to use the Top Mapping Bar for going to other screens in threecircumstances: as first attempt to move, when they could not find the link to proceed, and whenthey wanted to go back or see a more advanced screen. I think this last case is related to the needto move around in a non-linear fashion, mentioned by several students - specially emphasized bythe more experienced ones.

The Yellow Link for moving to next screens present in the first three screens of the News Sim.These links appear in the middle of the screen to the right, under the video and are not-underlinedyellow text.

Some of the stuff wasn’t so intuitive, you know. I had to go through allthese stuff to get to something. It was a little confusing. I had to go to thequiz to get into the simulation, is that right? I was confused, I waslooking for things on the screen, how I get around…(Student 5)

Most of the students spent time looking for these links when they appear the first time. However,they remember and find it faster in the following screens. Like with other examples of assumeduse, the affordance of the Yellow Links is learned by the users without major problems. A

TopMappingBar

YellowLink


27

question to consider is if it is worthwhile to have the students discover these navigation non-conventionalities.

These two turning points are navigation elements that are different from what users areaccustomed as conventional.

The Sequence

There is a progression from a more linear to a more open navigation that seems to accompanystudents’ work and familiarity with the News Sim.

The linear architecture in the simulation works both as a source of familiar structure and as asource of frustration. The observation showed that the linear structure of the site makes it easierto navigate and guides beginner students through the presentation of the situation, the role, andalong the information gathering.

All the students followed the expected sequence in the first two parts; the presentation and thereporting parts. All the students asked all the questions to the sources and they all did it one byone from top to bottom. This is important to note because the questions appear on the screen forthe student to select freely. The project manager said that the intention of having a list ofquestions for each source was that the students would select the ones they consider relevant.

The difficulties come up in the writing part, where the students are writing on one screen and theywant to go back to check facts. Most (4 of 5) of the students who used the simulation, went backto check information from the interviews or the background readings after starting to write thestory. The simulation did not facilitate this action; navigating back through all the screens to getto the map or to a certain interview became a complex task. Students used the back button fromthe browser to do it, and therefore lost the story they were writing. Several students suggestedhaving menus to go back and having a separate space/screen for writing the story.

One suggestion I have: have an extra bar over here [left margin] whereif you started to write your story, you can get back individually to whichever of these people you want to talk to so you don’t have to go throughall to get to the fire chief….The fire chief was the most importantinterview, I think was the first interview, he should be. But it was difficultto go back to him… hard to get there. (Student 8)

None of the students respected the linear structure. Even though it was in different amounts, allthe students used the Back button on the browser to return to previous places in search ofinformation. These actions suggest the question of why the simulation doesn’t allow going backthat will be discussed later.

Another point where the students go against the fixed sequence expected by the simulation iswhen they are doing the quiz in the introduction part. They went back to check the readings whenthey were doing the math and grammar quiz. This is not an action anticipated by the simulationdesign because when they did so, they lost their already marked answers. The students in thefocus group asked for the possibility to check back at the readings. Also two of the observedstudents mentioned that if they are allowed to check back, it should be easier to do it, and if theintention is that they don’t then it should not be possible.


28

It was annoying. But I remembered [she remembered the answers andmarked them again]. You should be able to go back and forth or not goback. If you don’t want people to be able to go back, I think that islegitimate. But if you allow them to go back they shouldn’t have to dothat… (Student 7)

The Background readings

None of the participants read all the background readings. Most of them asked theevaluator if they were supposed to read them:

Do I have to do this background reading? (Student 5)

Am I supposed to read all of this? (Student 7)

Maybe at the beginning I was not sure if I should read through all thestuff. Not all directly turn into what I was doing (Student 6)

Some of the students skimmed them quickly; some read only the beginning of each of them. Onestudent said that if he would be doing the simulation at home he would print them and read them.

Most of the students refer to the Background readings as something that is outside of thesimulation. Mentioning that it is something that comes before and that they are not sure that hadto be read.

I thought it took too long to get to the simulation …this part [thebackground readings] doesn’t feel like part of it (Student 7)

Given the way in which the background readings are presented, students say that they were notused as resources. When they want to look for something in the readings, the full text aspect of itmakes it effortful to find specific references. Therefore students sometimes decided to go onwithout consulting them.

Other students mentioned that they didn’t read it because it didn’t seem necessary for what theywere going to do:

I wasn’t expecting a lot of background information. I guess that myanticipation was that I was going to be doing something fast so I wasconcerned about the time element and because I have some experienceas a reporter. I didn’t know if it was like in case you need help read thisor if these were very important instructions and I was not going to beable to do something later if I didn’t read it. I was a little confused withthat because I felt I was reading a lot of stuff… (Student 8)


29

It is well written but if one knows that he is going to take this test [he istalking about writing the story] one would not pay attention to all this.This has nothing to do because I am so focused on writing the story, Idon’t pay attention to it. (Student 6)

The same perception was expressed about the Lead and story structure sections. Studentsmention they don’t read them because they don’t need them, because they are ready to write theirstory, and because it cuts the energy they bring for writing the story. They suspend the make-believe involved in the simulation. Students call this part ‘the textbook’. Even though these twosections belong to the sequence as ‘screens’ instead of resources students skip them as non-required resources.

The Map as a transition tool

Four of the five observed students experienced a conflict when first faced with the map tool. Thethree students in the focus group also discussed the use of the map as a turning point.

The map worked as a source of new possibilities at the same time that generated some difficultyfor navigation. All but one student asked for help with the map after getting frustrated from tryingto figure it out for a while. This difficulty can be a result of several reasons that are detailedbelow.

Change in navigation style: The simulation initiates a navigation style that is linear. Frominception all the pages have a similar format. The students are guided on how to proceed by thespeech of the characters in the simulation. Choosing between binary options that lead them in apredetermined sequence. The text in the navigation box helps them to anticipate their destination.The map screen has a different format. It is a pop up window. And its different structureconstitutes an abrupt change in anticipation of their expected location. It doesn’t contain anoption box or a yellow link, and the purpose or goal that the student has to pursue is not repeatedon the map screen. Student 4 asked the evaluator what she had to do, as there were no instructions

[Looks at the evaluator for help] It doesn’t give me a job for doing with the map!(Student 4)

Non-intuitiveness of use: The use of the map seemed not to be intuitive to the participants. Thestudents tried different approaches to the map when trying to use it.

It took me a while to figure out how to use that map [why?] I don’t know,it was an online map, you had to figure out how to … for example if I putthe arrow on the map I thought ok, I’ll get a little hand and anything Iwant to see will enlarge that area, that is how I use my Map Quest athome. Then, most maps don’t come with a cross directory, you know?(Student 5)

One student spent some time identifying her present location and the one she wanted to reach,and then studied the route. This is also a very realistic use of a map, but was not useful to movearound in the simulation (Student 7).


30

A few students played with the directory but didn’t realize that they needed to look for the policestation in it in order to get there.

Student 8 mentioned that the directories run too fast so they could not see the options displayed ineach directory.

Most of the students started by zooming in the map, finding the place they want to go andclicking on it. This seemed to be an intuitive use of a map. Also fostered by the presentation ofthe zooming tools at the beginning of the tutorial.

The tutorial contains an explanation on how to use the map, but some students didn’t open thetutorial, others opened it, looked at the first explanations –the zoom functions- that look prettyobvious and closed it to go to the map. The explanations about the directory and cross-directory,the tool needed to use the map, are further down in the screen, therefore the students didn’t seethem, because they initially thought the instructions were too basic.

Even though after the first correct use of the map the students can use it, most of them keptconfusing the directory and cross directory. For example, trying to locate the apartment buildingin the directory or to find neighbors by address from there.

Another possible reason for the difficulty in the use of the map is that the navigation aspectscome up together in a moment where the student needs to keep his goal clear. This possibility ofnavigation opened by the map where the student has more control of the direction of thesimulation is very important.

The use of the map, as a navigation tool, offers a different, more open, less directed sequence.The metaphor of the map helps represent the navigation of the city, not a determined path. Usersmust select where to go. This tool opens a possibility of a more exploratory navigation; it allowsmore control for students. For these reasons it was very highly valued by the students.

Student 2: I think the map was pretty interesting.Student 2: I guess maybe it puts it in a larger context. You couldhave just gone from a screen with the editor to a screen with thefire marshal but you had to go, you had to actually use the mapto get from one point to another point.Student 1: in real life you have to get a map and figure out howto get there.…Student 1: yes, you needed to do work to find what you werelooking for and think exactly what you are looking for, a streetname…(Focus group)

Once the students learned how to use the map and the directories, some of them tried takingadvantage of the possibilities that the map navigation was opening, tried to go other places, likethe hospital, the Red Cross, or the city hall.

5.3 Educational Architecture and Cognitive Activities


31

Observation and interview answers lead us to find elements to consider learning possibilities ofthe News Sim.

All the students expressed that the News Sim is useful for them and that the time spent on it wasworthwhile. The seven students that answered the survey considered that the simulation helpedthem improve their understanding of covering a fire story.

I think it is good for me because sometimes I don’t write so well on atight, tight deadline (Student 6)

Has things logically, you’ve got to talk to this people, get thisinformation. Sorts out what needs to get done. (Student 5)

[Do you think it would be useful for RWI?] Absolutely. Anytime youdevelop any topic, you end up understanding the topic thirty times better,you know because you have to think of the components, the little thingsthat make up the larger skill, like interviewing… You have to break thatdown. Where do I get the information? And the more you think about itthe more clearly you understand the whole game, the whole situation.(Student 5)

Learning objectives

In this section we explore how the simulation addresses the learning objectives. Our observationand conclusions follow Wilson, “there is much learned within any instructional environment thatgoes far beyond the instructional objectives” (Wilson, 1997).

Another consideration regarding the accomplishment of the learning objectives is that here thefact that the simulation is used out of a relevant context. Several of these objectives, would beaddressed differently in the use the students and professors do in the class, they rely on theintegration of the simulation in the frame of other activities and interactions of class.

Acquire basic news gathering skills:

Pay attention to details, including dispatcher’s communications on police scanner.

From observing the students’ work with the simulation, they practice paying attention to detailswhen listening to the sources and to the scanner. They looked very concentrated and attentive tothe audio materials and they took down detailed notes. All the students got the correct addressfrom the scanner and the correct information on their stories.

Six of seven students agreed in the survey that working with the simulation helped them improvetheir understanding of attention to detail.

Students commented that they found it useful listening to the scanner. They pointed out that it isrealistic, and suggestions of the basic facts they need to get when covering a fire helped themfocus on relevant aspects. They mention they liked it, some students didn’t know about its


32

existence, some did. They all considered that being able to see it and listen to it without the realnoise was very useful.

The simulation offers a good opportunity to practice this skill. It offers a less noisy environmentthan going out to a real fire, and the information they gather should be pondered keeping in mindthey have to write a story.

Recognize when a fire is worth covering

As one student describes, the activity of recognizing the fire does not give the students theopportunity for thinking and deciding about when a fire is worth covering because there is onlyone fire and it is directly pointed to by the instructions. The audio of the experienced reportermentions what code to look for, so there is no need for the student to compare different fires anddecide. The simulation does say that not all fires are worth covering and it provides some criteriafor considering. It also shows to the student that codes are important to understand the scanner.However, the activity for listening to the scanner does not let the student practice hisunderstanding on how to recognize when a fire is worth covering.

Another student who very much appreciated the use of the scanner said:

I thought the scanner was good. … I thought that it was useful for thecodes and having to listen to the really important. But you heard the firetoo quickly. It would have been good to take a little bit longer to get thecode that you want because it came at the beginning... or having othercodes. (Student 7)

Identify main sources for a fire story, both people and documents, web sites

This goal is addressed by having the student interview the sources that are provided in sequence.The student is shown which are the main sources because: the sources are organized in order ofrelevance, the amount of speech and questions also denote the importance of the source, andbecause it is said in the background readings. After doing the simulation all the students knewwhat the main sources are for a fire story.

I learned to go to the fire marshal first at a fire… [What from whatyou’ve learned could be useful for other kind of story?] Kind of the chainof events… If you were at a murder scene you would probably need tofind the police chief as opposed to just running around and just trying tochaotically grab somebody (Student 4)

Six of the seven students that answered the survey said that doing the simulation helped themimprove their understanding of getting the facts and sources right.

Apart from being informed about the relevance of sources, the students have to put into practicethat order at the moment when they decide if they are going to believe what the fire marshal said,or what the fire fighter said regarding the cause of the fire. Should they quote the fire marshal orthe building owner regarding the value of damage? These decisions make them qualify theimportance of sources.


33

In this scenario and first version of the News Sim there are no documents or websites to identify.

Formulate good questions for sources, including knowing when to follow up

All the students selected all the questions for the sources available. Even though they hadexamples of questions to consider as model, students brought up the fact that not being able tothink of the questions by themselves was an important aspect that made the simulation differentfrom reporting on the street.

You don’t have to think the questions to ask because they already toldyou to ask them. I think in real life you have to do that on your own.(Student 6)

Only two of the seven students that participated in the survey agreed that the News Sim helpswith interviewing skills, the other five disagree or have no opinion. When asked why, thestudents that disagreed explained that knowing how to ask the question is an important skill inreporting, in the simulation they didn’t get the chance to think of the questions by themselves.

Learn to pay attention to details, including spelling of names, getting correct addresses, titles ofsources

Students put emphasis on getting the correct spelling and addresses when working on theirstories. Even though there are a few misspellings on their stories, they all mention as importantthe correctness of spelling and details. Most of the students used the map for checking andcorrecting names and addresses. This interesting finding will be explained later. Two of thestudents suggested that having names that were not common names would make it trickier andmake the students aware that they should not assume but they have to check everything.

Get information needed to follow up on a story or lead, including contact information for sources.

The situation presented in this scenario is a very rich one. The students explored this situation andcame up with ideas on how to get the information they needed, how to find out more and how tocheck what they initially heard or suspected. Examples of these are the intentions of students tofollow certain clues or going to different places than the ones allowed by the simulation. In thisscenario they did not have the possibility of getting information to contact the sources but they‘contacted’ them using the map and the Back button.

Learn to check the facts

Students used different strategies to check facts. They used the map directory and the back buttonto go back to sources that had the information to check their understanding of ambiguous topicsand look for more sources. When students are faced with contradictory information the News Simprompts them to check facts. One student mentioned that he went back several times trying tounderstand the cause of the fire.


34

Develop basic lead writing skills

Most of the students (6 of 7) who worked with the News Sim considered it an effective tool forenhancing their writing skills.

The News Sim stimulates students’ cognitive activity when writing their stories. This activityprovides the opportunity for practice and therefore keeps developing their understanding. All thestudents struggled with the decision of how to word the lead. They made several changesreconsidering and restructuring it.

With the exception of one story, all the stories submitted by the students include the basic storyelements (5Ws: who, what, when, where, why, and how). By doing it, the students practicedstructuring a lead for a spot news story, experiencing what a spot news story is as intended in thesimulation.

In the activity of writing they had the opportunity to practice the development of editorialjudgment for their story: what elements are most important including cause of fire, damage/injurythat results. The students considered these elements and mentioned the difficulty of deciding whatelements should come first and which ones should go further down in the story when asked toexplain their writing process.

When writing, they said their thinking was mostly how to rank things, what goes up in the story,and what words to use. They erased and changed words in their stories. They explained that theywere making sure to use the best words to describe. The fact that there was a conflicting issuebetween two sources made the students pay special attention to the attribution of the facts and, asthey had to consider their relevance and judge what to include, and at what risk. In making thesedecisions they practice judgement by doing the writing.

All the students, who selected to interview the neighbors and residents considered they shouldfind more information for their stories and attempt to dig beyond surface facts. Four of thestudents also attempted to pursue leads that the story suggested in order to get informationbeyond what they found out.

None of the students’ stories backed into the lead.

Master basic communication skills

The simulation sets the situation for thinking about the communication between the reporter andthe editor. However, in order to practice how to communicate appropriately with an editor, thiscommunication cannot be addressed in a use of the simulation that is not embedded in theclassroom context where the professor, in the role of the editor would be interacting with thestudent.

Develop moral compass for reporting

This goal crosses all the others. Doing the simulation offers the possibility of practice fordeveloping it.


35

Develop basic mathematics skills relevant to journalism

All the students valued the math reading; they dedicated some time to it going back to the mathreading while they were doing the quiz in order to understand how to proceed. They all passed thequiz. They found the math skills an important element for reporters, and a difficult one tomanage.

A general comment that students had was that they need an opportunity to practice it in thesimulation.

Students’ analysis of learning with the News Sim

From students comments we have identified their perceptions bout their learning experience fromworking with the simulation.

Use of media in the situation setting

Students value the use of media for practicing; they value the audio and the video for setting thesituation and the realism of the tasks. They also appreciated the possibility of playing the sourcesagain, even though that is not what happens in real interviewing, they compared it to asking thesource to repeat and they valued it as a learning possibility.

Some students mentioned the possibility or need to ‘see more’, to have more videos, and imagesof the fire and scene. One student mentioned the possibility of seeing the sources talking to themas the fire marshal. Another student referred to seeing the scene of the fire as perceiving thesituation plays a crucial role in the task.

If you are a reporter in a beat the physical place were you find yourself,in my mind, is the most important thing. That and who to talk to, ofcourse, but you need to see it, to see what the situation rises. I mean,there was an example there where it said the fire gutted the building or itdestroyed the building, you can’t evaluate that without seeing it. To saythat it was destroyed, I would want to see it, well, those aren’t completelysubjective terms but some people would say, for example the ownerwould say it got destroyed totally. Meanwhile other would say, well, itwas not really destroyed… yes, I want to see it for myself in order toput… at least to have my own kind of impression.(Student 5)

Use of resources and exploration possibilities

The majority (4) of the students agreed that the simulation was an effective tool to enhance theirinvestigative skills. Two students disagreed with that statement, and one had no opinion about it.


36

We can interpret the responses of these three students in relation to their comments, and thecomments of others, on the need of more information and investigation possibilities to beincorporated in the News Sim.

Student 6 mentioned that he wanted to pursue the arson lead in order to find out more:

That would have been very very interesting, extremely interesting, but Itried to… I wanted to talk to somebody who worked in arsoninvestigation, I couldn’t do it either. So that’s a whole line/set ofcommunications that is just shut off because I think is a trap, because if Igo there… rumors, people’s suspicions that I was explicitly told not to inthe beginning… I tried to go but I couldn’t. There was no listing forarson. [If there was a possibility would you have gone?] Oh, definitely[even though if you knew that you shouldn’t follow a cue from a nottrustful source…] it is a valued source!, and I am already suspicious ifall these numbers don’t match up. And, you know, a fire is a fire, but itwould be much more interesting if there’s a fraud.” [He tried to go placeswith the map and signs of not available kept popping up – he was upsetabout it] … The Police Beat is where I wanted to go; I was looking forarson [he goes to the Police, to the Red Cross, to the Hospital, couldn’tget there] (Student 6)

Student 7 emphasized the need of other sources of information as archives.

One thing that could be useful that I think is not here would be to call upclips, consult, do a search on this building like if there was a fire a yearago, that kind of thing. (Student 7)

Student 5 mentioned the possibility of visiting other places already in the map.

I was looking for sources to talk to, but there were couple of things that Iwanted to do there, one was, I was looking for something that saidarchives from the newspaper because if she won that award for surethese people covered that. Freeport annual town show…she won firstplace she had to be there in somewhere, may be she said something ormay be one of the judges… now that I think about it the MalloryCollege… I should have clicked on that and find one of her professors…you might find something interesting… (Student 5)

… If you look in that directory [in the map] that would be a littleunrealistic because if there is a fire in their building they [the neighbors]are not going to be home, so, it would be a problem try to contact them. Imight go to the shelter, the Red Cross thing should have popped up[when clicked in the directory] because that’s where people would be oryou call them. (Student 5)

Most of the students looked for or mentioned the possibility of accessing other resources; four ofthem mentioned the need of a clip archive, and two of them wanted to go ‘places’ in the map formore information.


37

You know, in that tool kit you might have something like a news librarywhere you can click on it and an alphabet list comes out. And havesomething called relative files. What would be really relevant to the storyif something in the … a piece of the exercise should be to find out whatstories have been written, to have a little news archive, with the name ofthe apartment complex and from 10 years ago…(Student 8)

Decision-making

Contradictory elements:

When asked about their learning experience, one of the elements most students remember is thearson cue. This seems to be a key learning experience from the simulation.

All students mentioned judging and paying attention to sources (a learning objective of thesimulation) as something they learned by giving the example of having to decide whether toinclude in their stories the reference to arson given by a firefighter, while it contradictedtestimony by the fire marshal.

I think the cause of the fire was an interesting thing. The fire marshal,who is the official source, was saying that arson is not suspected andthen you have two sources including a firefighter saying that arson wassuspected. I thought that the contradiction in the cause of the fire wasvery good because that is very common. (Student 7)

[Did you find it intellectually stimulating?] Yes …the arson issue wasthe most…it was the part I really think through it. (Student 8)

The other element of contradiction mentioned by the students is the difference between the valuesof damage of property that was said by two of the sources.

We believe that these factors are effective because they include conflicting elements that requirethe students to judge sources, evaluating the information they collected, and to decide whether toinclude it or not in their stories. Student 7 expressed that she considered that the major learningpossibilities would be related to dealing with inconsistencies and organizing information.

The task of interviewing - The questions:

The interviewing part of the simulation is represented with a list of questions to each source. Inthe intentions of the design selecting among a list of questions was offered as an activity wherethe student would need to examine the questions and select which ones he would ask, so eventhough they don’t have to ask the questions themselves the students would need to choose whichones they want to ask.


38

The fact that all the students select all the questions in the top down order indicates that thisobjective for selection is not working, probably because of the lack of constraint or disadvantageon selecting them all.

All students mentioned the questions when asked to give suggestions for the News Sim. Most ofthe suggestions given by the students regarding the questions involved having a more active role.

I wanted to say something about the interviewing. I don’t think it reallydid help with my interviewing because I wasn’t forming the questions.They were being formed for me. I know this would be a higher level oftechnology but if the reporter could form the questions that would beimportant because that’s the key, knowing what to ask. If the questionsare all provided… The reporter needs to be thinking about what do youask in this situation and that is something that you can’t get here.Actually, having to listen to what he said is actually more useful than thequestions. The fire marshal, he was talking fast, he was hard to quoteand a couple of things that he said were a little confusing, which isrealistic. So that in a way was more realistic than the questions. (Student7)

[What improvements would you do to the News Sim?] The questions.Especially for breaking news you don’t have a list, you have to workthem in your brain. It is interactive, you’ve got to talk to your source andbased on something they say; oh I need to know more about that, andyou open the question and takes you in a different direction. Thequestions when you have a whole list of them, what would you like toask? Well unless you’re really pressed with time you are going to clickon every one of them because you don’t know where the person is goingto say it or throw a good quote, so. Somehow the person who uses thiscould come up with the questions and type them in but that is probably ahigher level of development in software… [Do you think the ability ofbuilding the questions is a difficult skill to gain?] Definitely. It might bethe most important skill journalists have. You cannot go in a situationand expect to have a structured interview. It is semi-structured… that isreally where you have to have the most important skills. Listening andthinking on your feet because you are listening to what someone says,and that should lead you to other questions. That is the theory. You needto listen really well and then you… like: ok, I understood I need to askwhat time did it happen, if he tells you there were two injured, well whatwere the injuries? Well ‘burned’, then how bad was the burn? You know.That is really important. The best reporters I’ve ever worked with arereally like that, they sort of push it to get clear, they keep thinking. Idon’t assume that in every case everyone is going to hide something butsome information they might not … the police or the fire people, youhave to ask the details, asking questions on a different way… (Student 5)

They mentioned that here the questions are given so they didn’t have to make them. Studentsfound that this aspect was not realistic regarding their activity with the simulation.


39

I’ll be honest, I very rarely went to interviews with a list of questions,now for a feature story you might do that but for a news story I don’tthink. You don’t go with a set of questions, you go with a set of instinctslike what information is relevant and that is where the what, who, when,how become relevant, because you don’t want to think about what timedid it happen, you want to think about when did it happen, you ask ingeneral and your questions build on the responses you get. (Student 5)

Two students suggested choosing from longer lists of questions and incorporating an element orvariable that wouldn’t allow them to choose all. Another students suggested a system ofkeywords where answers related to certain topics would not be found if the questions posed bythe reporter didn’t include certain keywords related to them.

Rather than have questions there as options, may be it would be better tohave a set number of answers but hidden. They would be revealed only ifthe right questions are asked and that would be complicated to do[draws a tree structure] I don’t know how to program but this is thescheme, this is the way interviews do work: so anyone of these keywordswould give you an answer, if you ask how many people were burned youwould not get to the information [in another branch] that is a superiorsystem because at some point you are going to get so detailed, and somuch information, so many bits of information to deal with that is notworth it, you know, takes too much time. [That would be more like reallife…] Yes, you know there’s times where people will not just lay all out.You have to ask to take your questions. (Student 6)

The task of interviewing also concerns the decisions of who to interview, or, what it in this case isthe same, where to go.

The students mention that would be good to have more possibilities for decision-makingregarding where to go:

I think that if you had choices like ‘where do you want to go next?’ ‘Whodo you want to talk to next? You have this list of people and if you hadchoices like do you want to look for phone numbers on the directory, doyou want to go to the scene of the fire, do you want to go back to youroffice… (Student 7)

One student suggested making the time work as a constraint that would require decision-making.

I would go to the hospital, to the shelter too. But then time is a hugefactor, they should probably put a sign there, you know, if you physicallygo to the hospital and to the City Hall to check the records on this guy,you can put a time calculation saying it is going to take you threeminutes to get to the city hall and 20 minutes to go through the records…it will be 15 minutes to go to the Red Cross… and you would have to


40

prioritize which is the most important, leave the record for the secondday…(Student 5)

In order for the Sim to be an incentive for reflection it would need to have more decision points:

Again… if there were more choices where to go or what to do next… Ithink that is something you could add in and that would be interesting…for example, do you go straight out to the fire or do you make phonecalls first. You know you have only two hours to report and write thestory, when do you go back to the office, do you keep trying to contactneighbors… that kind of things… but other things… well we can discussthe contradiction about the cause of the fire, may be the importance ofthe news compared to other stories if this is a first page story (Student 7)

An important aspect of decision making that most students mentioned and all of them dealt withwhen writing their stories was the order or ranking of information in writing their stories.

The things you see here in the video [looking at the video in the partwhere he is in the writing part] that I change is not about what I need towrite but about where it needs to go. The order of things. That is themajor task when you do spot news. You don’t have to think about what towrite, it sort of writes itself, you have to think, how do I tell this to afriend, the important stuff first before you get into the details. That’straining. I am thinking of were to put things (Student 5)

I worried about … mainly because two people died, and that’s important,that should be high up. A baby is saved, that’s important, that should behigh up. And sometimes is hard to work all this stuff and to get it in away, you know (Student 6)

Possibilities of mistake and feedback

Most of the students mentioned that the way to learn about reporting and writing is by makingmistakes and studying the corrections made by the professor or the editor.

… The only way to learn about writing skills is to get the story backmarked up by the professor. That would teach me writing skills. Where Ialways got the best feedback on my writing was from the city editor afterI turned the story in. Because the city editor would say this sentence istoo long… (Student 8)

Four different students addressed the need for opportunities for making mistakes and havingconsequences for that.

I think that in terms of teaching the actual skills, that if there were moreopportunities to make mistakes I think there would be more opportunitiesto learn. Because a lot of the times, as she [student 2] had said, if youclicked the wrong button you would go to the correct screen anyway…Particularly when asking the questions, when you are interviewing thepeople, like may be someone gets upset, or offended or something and


41

says ‘I am not going to talk to you anymore’ or something like that. Andthen kind of you loose your source or your source doesn’t talk anymore. Imean it makes the site a lot more complicated but I think the studentswould learn more from that. (Student 1 of the Focus group)

I always learned in journalism when I found what I had done. You know,you almost learn by mistakes, unfortunately. I thought the question wasreally good about whether or not it was arson. And I think in a situationlike that you don’t take the word of someone who knows nothing... I thinkthe learning would come from the feedback from what they did wrong(Student 8)

Student 5 thinks these skills are learned through:

Experience and interaction with an editor, someone who has thatexperience (Student 5)

Level of users

Most of the observed students thought that the News Sim would be a useful tool in RWI course.They suggested that could be useful for the beginning of the course or for beginner student ofjournalism in general. A surprising similarity is that they all considered the News Sim forsomeone who knows less than them.

Two students mentioned the August part of the course, one student mentioned students coming toJournalism graduate school from other disciplines, two students considered undergraduate level.

Student 2: I saw it as more introductory, like for an undergraduatejournalism course. The students would use that like this is sort of ‘ theprofession is like this. This is what you would do in a normal day’ kind ofthing… I think that if you wanted to concentrate on specific skills I thinkyou would need to have more activities, more, more, I don’t know, more,may be more… more I don’t know…Student 1: when I was done I didn’t feel like ok now I can report a fire,you know, I will be nervous still. It’s like for a beginner journalist…(Focus Group)

I think for very beginners this would be great because there is alwaysthat sort of… when you are going to have to talk to a fire official, there isalways that ‘what do I say, what do I ask?’ you know. (Student 5)

You know what I would think this would have been a very interestingthing? Is on that exam they give for entering to this program. That’swhat I kept thinking. Right now they have to write a news story and afeature story… they could use this for that, this could be more daunting.(Student 8)


42

An RWI professor mentioned that considering the heterogeneous experiences of the students inRWI, News Sim could help students who come from other disciplines or have English as asecond language. He highlighted the fact that one of the objectives of RWI is to bring all thestudents to a certain level.

Unexpected use of resources

Observation evidenced unexpected ways in which the students made use of the resourcesavailable in the simulation.

It is very interesting the fact that almost all the students used the map and the directory in it, noonly for navigating the simulation but also for finding names, checking spelling, confirmingcomprehension of places. Students explained that in real life they would have a phonebook orother way to confirm this information, but in the simulation the map and its directory were theonly resource for verifying information or looking for facts and details not provided by thesources. As already mentioned in other section, students thought of other resources they woulduse if available as a dictionary, a news archive, a phone book, etc.

[You looked the directory and the map for a while] Yes, I looked at thespelling. It reminds you that you have these other resources; you have toconfirm everything, once or twice, from different sources. (Student 5)

Another interesting finding regarding the use of resources is that all the students used thecalculator in the accessories of the computer for working on the math quiz. That resource was notconsidered as part of the simulation. However, most of the students used it naturally. One studentasked the evaluator for permission to do it and another student requested the evaluator to bring itup for her.

5.4 Comparison with the drills

At the present, in RWI course, the instructional materials that are used for practicing writing spotnews stories are the drills or fact sheets. In this section we will compare some of thecharacteristics of the drills to the News Sim.

The first important characteristic is that the drills don’t address reporting skills. As all theinformation is provided to the student, the purpose of it is to practice writing. The simulation, onthe other hand, includes the reporting activity as an integral part of the task. It provides a realisticsituation, presented as a whole, and context whereby the activity is meaningful.

The contextual, case story, and multimedia elements anchor and integrate the tasks into arealistic activity.

The News Sim, using audio, allows the students to practice the skill of taking notes and thinkingabout the way of writing and spelling. It also allows the student to practice realistic quoting of thesources.


43

One student mentioned that in the drill, they sometimes are presented with too much informationand therefore they need to select what to include. He considered this a good exercise, as thelimitation of the size of the story would make it a realistic and complex task.

[Doing the drills] is kind of similar to doing the simulation but… kind ofnot I guess. Because you are not actually asking the questions. You haveto be much more selective, if you have a hundred piece… and you haveto know how to skim… and I like doing that. (Student 6)

RWI Professor thought that the drills have the time advantage. That their simplicity allowsstudents to do them very quickly and therefore be able to do many of them, maximizing theopportunities for practice. He thinks that the simulation takes too much time compared to thedrills. However, the context and realism enrich the experience. The professor especiallymentioned the value of the audio because that is how to get the information in real reporting.

…If you have somebody talking to you and then you have to take notes,and then you write the story rather than handing them a fact sheet. Itmight be useful to get this orally. (RWI professor)

Students also found it preferable to the drills:

[If you, for example, would have different simulations like this onereplacing the drills?] That would be preferable, yes. (Student 6)


44

6. Discussion and Suggestions

In this section we discuss the findings introduced in the previous section by rearticulating themstressing the overall experience promoted by News Sim.

The notion of activity

In order to refer to this overall experience we consider it important to rethink the notion ofactivity as ‘performance of understanding’. These activities are those where the students learn bydoing; they deepen their understanding when they think and reshape their practice andunderstanding.

David Perkins defines these authentic activities as performances for understanding:"Performances of understanding are activities which require students to use what they know innew ways or situations to build their understanding… students reshape, expand on, extrapolatefrom, and apply what they already know. Such performances challenge students' misconceptions,stereotypes, and tendencies toward rigid thinking. Performances of understanding help studentsbuild and demonstrate their understanding. … Performances of understanding make students'thinking visible" (Perkins, 1992). Authentic activities imply learning through meaningfulpracticing.

From this perspective, the News Sim can be understood as more than an opportunity forpracticing certain skills, but as a space in which these skills are being developed throughmeaningfully accomplishing a goal.

However, this space has to be pedagogically reshaped to manipulate the complexity of theperformances. By complexity we refer to the variables and tasks involved between theperformance and the context that frames this performance in the simulation.

As Honebein, explains “…the aim of an authentic activity is not just to simulate or replicate thephysical environment, calling it ‘authentic.’ Rather, the aim is to design an environment in whichlearners use their minds and bodies as they would if they were practitioners in a domain. It is thepurpose of the learning environment, whether it be simulation, actual practice, or independentstudy, to stimulate learners so that their thinking is related to actual practice …” (Honebein,1996). In other words, it is not just about how similar to reality the simulation is but howauthentic are the activities it provides for students to engage in performing the role. We think thatthis is an interesting conceptual dimension that may help us to interpret students’ experienceswith the News Sim.

We will proceed with the interpretation of findings by focusing on some aspects of the two mainactivities involved in the News Sim: reporting and writing. We will then consider how thedifferent parts of the simulation and its resources could accommodate to these two mainactivities.

There is an ambivalence between the richness of the context, in providing and presenting asituation/case and the way in which that richness is reflected in the actions of the students. Thatambivalence is shown in some of the activities that students have to deal with in the simulation.


45

The activities involved in reporting seem to be limiting the experiences of the students in theaspects of conjecturing, forecasting, making decisions and testing them by gaining feedback ontheir consequences. In this evaluation we found that there is a gap between what students consideras involved in the task of reporting and what they are prompted and able to do in the activities forreporting in the simulation. In the findings section of this report we described how studentsperceive the task as a more complex one than the way the simulation presents it.

As we described in the findings section, the use of the questions for interviewing is not beingexperienced by the students as a decision making activity as students don’t have the chance toconstruct the questions, nor to select or identify certain questions as the most relevant among thelist because they can see all of them all without any constraint.

Students mentioned the importance of having the chance to practice the skill of making thequestions. They suggested different imagined technological possibilities for being able toconstruct the questions themselves. In shaping these imagined alternatives they intended to bringback to the activity the needs of deciding what information for which they are looking andconsidering the sources’ previous responses in order to formulate new questions building on thoseanswers.

Another possibility to address the need of making the interviewing part in the simulation morerealistic to the interviewing task could be an intermediate option considering possibilities. Eventhough not addressing the same cognitive skills, we can suggest a way of structuring theinterviewing activity that would involve the students in a more active consideration of thequestions to ask. Having a longer list of question from which to choose and a constraint thatwould make the students select the ones he considers more useful. The constraint could be realtime (limited time for interviewing) or realistic but not real time (‘this source has time only forthree questions’).

The time constraint as presented in the News Sim right now is perceived more as a generalconsideration of the deadline but does not seem to be working as a constrain for the reportingactivities as shown by the findings.

In the reporting part there are also the activities of choosing who to talk to, or weather to go totalk to other sources, and related to this, what information to go on pursuing.

As described in the findings section, students wanted to pursue certain leads that were notallowed by the News Sim. We suggest that those possibilities that are not developed in the currentversion should be developed even though they are not considered as correct paths for thepresented scenario. Being able to pursue these ‘incorrect paths’ allow the students to explore andput their conceptions in use. Therefore being able to test them.

The structure of the News Sim as it is now reduces the possibilities of making mistakes reducingthe possibilities of the environment of working as a source of feedback for their actions andtherefore limiting the possibilities of reflection in action that characterize the learning withsimulations. The possibility of making mistakes allows for the students to test their beliefs andunderstandings. The feedback from the environment allows for an in-activity reflection that helpsreshape understandings. Addressing misconceptions and allowing the students to reconstructthem. Understandings that work as hypothesis – are put into practice in exploratory simulations.


46

The learning environment

We would suggest opening up the navigation promoted by the simulation from a linear sequenceto an environment distribution. Among other elements, this would address the possibilitiesopened by the map. The navigation style established by the map works as a source for decision-making by allowing the student to choose where to go. This open navigation would allowstudents to have to think in advance where they want to go and what sources they want tointerview. The information could be less relevant if the intention is that the students should notuse it for this kind of story, so this would work as feedback from the environment for the studentto consider.

Meanwhile the News Sim has limited options for making mistakes in the first two parts (theintroduction and the reporting part) as ‘mistaken’ paths cannot be followed, in the third part (thewriting part) the possibilities of learning from the mistakes and feedback is suspended untilprofessor corrects the stories or until the student involves in a discussion with other students.

We consider writing the story as an authentic activity. For writing their story, among othercognitive tasks, students have to judge information they have gathered, they need to ponder itsrelevance to organize the story, and also explain it by stating it in certain manner. The simulationoffers some elements of contradiction between the sources that make the student select whatinformation and what source to include and quote. These cognitive activities make the studentsdetermine or rethink their criteria for judgment. Having them use what they know (just read orknown from before) in new ways – demonstrates a performance of understanding.

This kind of cognitive activity is not fully fostered in other elements of the simulation. Studentsmentioned this point as disappointing. Students prepare for the challenge, but the limited amountof conflicting elements doesn’t allow them to employ their knowledge.

Suggestion: have more activities where students have to make decisions and put into practice thevalues or criteria they are learning. By giving the task goals a pedagogical value so that theleaning activity itself becomes a reflective process in which reevaluation of action takes place.For example, the suggestion of students to have more complex surnames would call theirattention for looking for correct spellings.

Duffy and Jonassen explain, “while an individual enters a situation with a plan, the critical aspectof performance is the ability to respond to the situational constraints – to be able to construct newplans based on the changing demands and constraints of the situation. In this view, then, plans(the principles, rules, procedures we teach) are ‘part of the subject matter of purposeful action,not something to be improved upon or transformed into axiomatic theories of action’ instruction,we believe, should not focus on transmitting plans to the learner but rather on developing theskills of the learner to construct (and reconstruct) plans in response to situational demands andopportunities. Instruction should provide contexts and assistance that will aid the individual inmaking sense of the environment as it is encountered. A plan is one part of that sense making, butplans must be constructed, tested, and revised as a function of the particular encounters in theenvironment.” (Duffy and Jonassen, 1992, p.4)

We found that all the students went back against the simulation structure when already in thewriting part. They mentioned that they were checking facts or going back to the map for checkingspelling trying to further understand an issue.


47

This possibility of going back may not be so realistic in the sense of reproducing reality, but isrealistic regarding pedagogical purposes. The possibility of going back allows for getting thefacts right, being able to reflect on the information, and exploring other possibilities. As a short-term recommendation we suggest that the top mapping bar, is made a non-linear navigation toolthat would allow and alternative navigation for students. This would give them the chance to“change their minds” or explore a different possibility of what they had done. It would allowthem to see other parts of the simulation, in advance or passed for their work. Therefore wesuggest making the possibility of going back available, eliminating the linear sequence thatcharacterizes the News Sim at the present.

The possibility of having the writing space available, as a non-sequenced element, would alsoallow for this exploration and use of resources in writing the story. It could also help the studentin the task of planning the story (and its structure) from the beginning of the process.

This possibility of a less linear navigation could also be applied to the integration of theinformation resources existing in the simulation. Under ‘information resources’ we have placedthe Lead and Story Structure and the Background Readings. We suggest incorporating theseelements, whose characteristics relate them to a more expository teaching, by placing them asoptional resources available to the student all the time, with open access. The timing of thissupport would be in control of the learner. The way these resources are presently integrated in thesimulation seems to interfere with the role-playing and make-believe of the simulation. Thishappens mainly with the Lead and Story Structure readings, but also the way in which the otherreadings refer to the students as students in a course instead of reporters in their role; makes itambiguous if these elements are part of the simulation or not.

The finding that students skip these readings in order to stay in their role made us consider thepossibility of presenting the readings as resources that can be easily accessed along thesimulation. The Background Readings could be more easily accessed by breaking them intosubheadings. A long plain text makes it more difficult to remember where you read it makingmore difficult to locate it later or to find it easily. This is an important point because studentsshould be able to access those elements as they need it in their practice and not as a previousstudy that they have to memorize before they need them or as a piece of theory that is imposed tothem. If they access it when they need it then it could work as meaningful knowledge that can bemore easily incorporated and transferred when it is associated with the need of the task (De Jong,1996). The background readings as resources would work as scaffolding for the students to usewhen they need them instead of as expository teaching inserted in the sequence. This method ofincorporating the readings allows flexibility for students with varying prior knowledge.

Another advantage of having the background readings as independent resources included in theenvironment is that it would allow more flexibility for professors to make modifications asintended in the customizing possibilities of the News Sim.

It is important that the information provided, as resources be related to the topics or possibilitiesof use of this scenario. Meaning that the students would have an opportunity to use the resourcesthat are presented to them. For example, most of the students appreciated the math reading butthey were puzzled when they saw there was no opportunity to use it in this scenario.

Another interesting suggestion that came from the students is the need to have other resourcesthat are always available to the journalist as databases and news archives. As described in thefindings section, students consider these resources as fundamental tools for journalists and itspresence in the simulation would allow them to practice an important objective of the simulation.


48

As mentioned at the beginning of this section, the main advantage of simulations is that allowsmanipulating the complexity of the environment in order to accommodate the needs of thelearning activities. As with games, the amount of challenge presented by and environment is acrucial and difficult to define aspect (Rieber, 1996). In this respect we believe that even the NewsSim is aimed for beginners, it should incorporate greater complexity so that the tasks result in‘performances of understanding’. In order to do this, greater scaffolding should be providedrather than reducing choices. An interesting possibility for providing appropriate scaffoldingcould be to have a system that tracks students’ navigation. Therefore it could offer help as anintervention by the figure of the editor. As some experienced students explained, editors’comments, questions and suggestions are an important source of learning for the novice reporter.

News Sim provides meaningful context to practice reporting and writing skills. The ambivalenceidentified in this study allows us to think about the complexities of this very context forpromoting challenging learning activities.


49

7. Conclusions

The analysis of the News Sim from a constructivist perspective, through exploring itscharacteristics with the evaluation methods, allowed us to understand how this project canpromote learning experiences and to offer some suggestions for further developments of theproject -- as were the intentions of this evaluation.

The News Sim proved to be an innovative tool with potential for practicing reporting and writingskills, that is very much valued by the student testers. The multimedia capabilities and themotivating elements of the story and goal-based simulation emerged as especially importantcharacteristics of the News Sim.

The two main aspects identified as determinant for the improvement of the learning possibilitiesare related to the characteristics of the activities and to the organization of the simulation and itsresources as a less directed learning environment. We believe that the development in these twoaspects will enhance the learning possibilities of the simulation.

The focus on the activities suggested including more opportunities for the students to be involvedin decision-making reflection required for the task. Modifying certain activities to make themmore realistic and authentic to the domain by incorporating a higher level of complexity. In manycases the quality of the choices and actions that students are asked to perform would require theincorporation of additional constrains.

The recommendations regarding the activities were linked to the observations about theenvironment. Several issues were raised regarding the need for the students to move more freelywithin the simulation. The possibility of modifying its structure from a linear path to a freerenvironment would provide opportunities for exploration, trial, error, revision, and modification –valued elements for the learning process. The possibility of incorporating information as optionaland available-on-demand resources instead of being part of the simulation steps was alsorecommended. This would allow the possibility of keeping the complexity and manipulating itslevel with the scaffolding of optional available resources.

The next phase in the evaluation process is a summative evaluation when News Sim is integratedinto the RWI courses. It will be interesting to evaluate the role of the simulation in the broaderteaching and learning process, in the context for which it was designed where students make useof it in a course-purpose situation. This will allow considering the influence of the tool onstudents’ performances and assessments, its interaction with the lectures, and with the otherelements of the course.


50

8. References

Bearman, Margaret (1997) Technology in Medical Education – Evaluation. Centre of MedicalInformatics, Monash University.http://www.monash.edu.au/informatics/TechME/evaluati.htm#Evaluation%20

De Jong, Ton (1996) Discovery Learning with Computer Simulations of ConceptualDomains. IST – Memo-96-02. University of Twente, The Netherlands.http://phoenix.sce.fct.unl.pt/simposio/Ton_de_Jong.htm

Gatto, D (1993) The use of interactive computer simulations in training. Australian Journal ofEducational Technology, 9(2), 144-156.http://wwwasu.murdoch.edu.au/gen/asset/ajet/ajet9/su93p144.html

Honebein, Peter (1996). Seven Goals for the Design of Constructivist Learning Environments. InBrent Wilson (1996) Constructivist Learning Environments: Case Studeis in InstructionalDesign. Educational Technology Publications. New Jersey. p.20).

Mencher, Melvin (2000) News Reporting and Writing. McGraw Hill. Eight edition.

Naidu, Som, Ip, Albert, and Liser, Roni. (2000) Dynamic Goal-Based Role-Play Simulation onthe Web: A Case Study. Educational Technology & Society 3 (3) ISSN 143-4522http://ifets.ieee.org/periodical/vol_3_2000/b05.html

Naidu, Som, and Liser, Roni. (1999) Web-based simulations as teaching and learning mediain Political Science. Ausweb99 - Fifth Australian World Wide Web Conference, Southern CrossUniversity. http://ausweb.scu.edu.au/aw99/papers/naidu/paper.html

Open University (1996) Evaluation Methods and Procedures for studying learners' use ofmedia. Developed by PLUM (Open University) and TELL (University of Hull)http://iet.open.ac.uk/plum/evaluation/plum.html

Perkins, David. Technology Meets Constructivism: Do They Make a Marriage? In ThomasDuffy and David Jonassen (Eds.) (1992) Constructivism and the Technology of Instruction. :A Conversation. Hillsdale, New Jersey: Lawrence Erlbaum.

Petranek, Charles. (2000) Written debriefing: The next vital step in learning withsimulations. Simulation & Gaming. Vol.31, No. 1 (March, 2000) pp.108-118.

Reeves, Tom. Evaluating What Really Matters in Computer-Based Education. University ofGeorgia http://www.educationau.edu.au/archives/cp/reeves.htm

Rieber, Lloyd (1996) Seriously Considering Play: Designing Interactive LearningEnvironments Based on the Blending of Microworlds, Simulations, and Games. EducationalTechnology Research & Development, 44 (2), 43-45. http://it.coe.uga.edu/~lrieber/play.html

Shank, Berman, and Macpherson. Learning by doing. In Reigeluth (1999) Instructional –DesignTheories and Models. Lawrence Erlbaum Assciates. NJ.


51

Wilson, Brent (1997) Reflections on Constructivism and Instructional Design. University ofColorado at Denver. To appear in C.R. Dills and A.A. Romiszowski (Eds.) InstructionalDevelopment Paradigms Englewood Cliffs NJ: Educational Technology Publications.http://carbon.cudenver.edu/~bwilson/construct.html


52

Appendix: Sample instrumentation

Sample instrumentation for student testers observations.

Questions for the students who worked with the simulation:(info about the interviewee)Name:E-mail:Why interested:Courses taken:Experience:Age:Use of computer/start using internet?:

Reactiona. What is your opinion about the simulation?b. Where there any statements, situations, manners, language, etc. that you did not

understand?c. Was the time working with the simulation worthwhile? Why?d. How realistic is the simulation? In what aspects? (Freeport, police beat, reporting,

activities: knowing about the job, deciding to cover the fire, interviewing, gathering info,organizing it, writing…)

Navigationa. Was the Sim easy to navigate? What were the hardest parts?b. Did you know where you were in each part? (Intro, Freeport, Police Beat, fire scene, etc.)c. Where you able to do what you wanted easily?d. Which of your actions involved more thinking and decision-making? (go over the sim)e. Was the amount of guidance and feedback appropriate? If not how and when?f. How informative were the results of your actions?g. Where you in control/in charge of reporting? How do you know?h. Did the responses from the sim make you question/doubt about something you

believed/knew?i. Where you too guided or too loose in certain parts? Which?

Learninga. What was your previous experience in reporting/writing? Was it useful in the simulation?

what for?b. What would have been useful for you to know and you did not?c. What has the News Sim made you better at?d. What happens in the simulation? (listen to their story: elements they find important)e. What are the most important parts/elements?f. What is required to be a good spot news reporter? How do you know? (if they don’t refer

the simulation ask them: what is the simulation considering to be a good reporter? howdo you know? How are these skills learned without the simulation? Is there anydifference between the two ways of learning these skills?

g. Did it challenge any of your ideas about reporting a fire? Which ones? How?h. Did you find the activities in the simulation intellectually stimulating? Examples?i. What from what you’ve learned about reporting a fire could be useful when you work as

reporter?


53

j. How clear were the objectives?k. How clear were the criteria for good reporting and writing? examplesl. What decisions did you make when writing, what from the sim helped you?m. How do you compare the News Sim with the information sheets (drills) that you are

given in RWI to practice writing? In what aspects can simulations like the News Sim bemore/less useful for replacing the drills?

n. What activity could be most helpful for learning each of the following objectives?

Acquire basic news gathering skills:

• Pay attention to details, including dispatcher’s communications on police scanner.• Recognize when a fire is worth covering• Master basics of navigating through a news archive for relevant background materials• Know how to get to the location of a fire quickly, efficiently and safely• Identify main sources for a fire story, both people and documents, web sites• Formulate good questions for sources, including knowing when to follow up• Learn to pay attention to details, including spelling of names, getting correct addresses, titles of

sources• Get information needed to follow up on a story or lead, including contact information for sources• Learn to check the facts.

Develop basic lead writing skills:

• Recognize basic story elements, including 5Ws: who , what, when , where, why, and how• Learn to structure a lead for a spot news story; learn what a spot news story is• Develop editorial judgment for her story as to what elements are most important including cause

of fire, damage/injury that results• Learn to follow AP style• Understand how to move past level 1 story to level 2 (digging beyond surface facts)• Learn not to back into lead (i.e. Don’t put “when” first)• Synthesize facts and present them in orderly, intelligent, clear and cogent fashion• Take criticism well (accept edits and respond to them without taking it personally, getting

defensive or upset)• Learn to avoid the passive voice; use the active voice, and find strong active verb (e.g. don’t say

the fire “burned the building when in fact fire “gutted” the building)• Learn to attribute facts to credible sources

Master basic communication skills

• Know how to send/receive e-mail and thus get your story to your editor• Communicate appropriately with an editor

Develop moral compass for reporting

• Learn about errors of omission and commission, and why errors of commission represent anethical challenge to journalism

• Recognize the importance of privacy in this story.

Develop basic mathematics skills relevant to journalism

(e.g. how to calculate a percentage and how to correctly interpret a percentage)

o. What improvements to the News Sim do you suggest? Why?


54

Observation Schedule16

Date................. Student .............................. E-mail……………………………………….. Course……………………………

Other relevant information about the characteristics of the student and use of the NewsSim ..............................

Time Observations: talk, actions, feedback, scores Notes on critical incidents

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

....... ............................................................................ ............................................

16 http://iet.open.ac.uk/plum/evaluation/Methods.html#observation on data collection: observation


55

Observation Report17

Date: …………………… Program: ………………………………….

Student: ……………………………….. Course ...................................

Time: to

Brief description of instructional context (student preparation, nature of follow-up, location, use of other materials, etc.)

Emergent issues (interesting points which emerge from this observation)

17 http://iet.open.ac.uk/plum/evaluation/Methods.html#observation on data collection: observation


56

Sample instrumentation for Faculty partner interviews.

Questions for faculty partner– interview I

1. How did you present the simulation to your students?2. What was their reaction?3. How did you expect them to use it? Did they use it that way?4. What information do you give them in advance and during their use?5. What are some important points in presenting and using the News Sim with students?6. What other activities complement the use of the simulation?7. Does the use of the simulation replace other activity that you would do in class? Why?8. What do you expect the simulation to help your students with? What is it useful for?9. Which students do you think could benefit more from the simulation? Why?10. Did the students that already used the simulation comment you on their use?11. What is your opinion of the submitted stories? (What happened-on corrected stories)12. Do you see any difference in student work this year (compare to other) that you relate to the use of

the NewsSim? Examples?13. By analyzing your student’s stories, are there any aspects the simulation should be improved?14. How was the integration between the work done in the Sim and the rest of the course work?15. What objectives of students at this level is the News Sim addressing?16. What aspects of the learning can you perceive most favored the students? Why? (other comments?)

Sample questions for RWI Professor interviews.

Questions for other professors (experts)

RWI Professor (interview I)

1. Can you describe how you teach RWI?

• What resources do you use?• What activities do you have your students do?• For what objectives each?• From you experience in teaching reporting, how/what activities help them learn most? Why?• What is implied (cognitively) in those activities?• How do you increase level of complexity?

2. How are the skills related to the situations?

3. What are key characteristics that differentiate an expert reporter from a novice?

• Way of thinking, representation of the job, automatization, trained eye, humaninteraction/convincing strategies, understanding of mistakes? (Exemplar performances)

• How can a novice get that?• What kinds of cognitive skills. General goals, unit goals, activities related to those goals, best ways

students understand, decisions, skills, kinds of activities could help get that?• Can you identify steps or stages in the domain that are important for learning it?

4. What difficulties do students experience?

• What are the aspects of reporting and writing that are most difficult to grasp by the students?• What kind of practice could help them overcome these difficulties?• What are your strategies to deal with students’ difficulties?


57

5. How do you evaluate students learning?

6. If you had the possibility of designing a simulation or tool for your students what characteristics would ithave (don’t worry about its plausibility)?

• How would you integrate it with your teaching?

7. What kind of practice students do when they don’t go to the street?, do they do drills? What is the purpose ofthem?

(Interview II)

Go over the News Sim (not replacing, evaluating how could be useful as a tool for theprofessor to use as a resource)

INSTRUCTIONAL DESIGN REVIEW (1-5)

1. News Sim provides learners with a clear knowledge of the program objectives.

2. The instructional interactions in News Sim are appropriate for the objectives.

3. The instructional design of News Sim is based on sound learning theory and principles.

4. The feedback in News Sim is clear.

5. The pace of News Sim is appropriate.

6. The difficulty level of News Sim is appropriate.

8. What students could benefit most?• What level?• What student characteristics (ESL, novice, little experience, novice writer, novice reporter, who

needs practice in writing, etc.)? Why?

9. What are students learning/practicing when doing the Sim? (cognitive processes – heuristics chart)• Which activities make students use what they know in new ways?• Which make them challenge their misconceptions?• Which help them discover for themselves rules of defined principles, preferred procedures, or

higher order skills?• What reflective experiences are required in the performances?• Examples?• How is the feedback helping?

10. Do you think students should be allowed to go back? Why? Is this something that we can allow in order• How do you get this kind of practice in your course?

11. Is the content adequate for RW1 students?ß What aspects/parts of it?

12. Is the treatment of the content adequate?ß Why?

13. How do you address this (similar) content/objectives in your teaching?• Could the simulation make your work easier in any aspect?


58

14. If you could customize the News Sim for your course, what would you do?• Suggestions on how would he use it in your class (homework, parts and discuss, reflection on important

decisions,…).

15. What could the Sim help your students with if added to your class as it is now?• In what ways?• In what ways it cannot?• What would you have to do in order to make it a good learning experience?

Questions for the focus group with students in Exploring New Media course.

1. What were your experiences in reporting and writing a spot story before doing the simulation? Didanything contrasted to what you thought about journalism?

2. What did the News Sim made you better at? What is the News Sim useful for (teaching/training -which each)? What elements were the most useful?

3. Tell me about the use of the background readings. The quiz. The map. (Easiness, usefulness,attractiveness…)

4. What would you teach someone who needs to report a story? [wait to see what they say] You canreconstruct the simulation sequence.

5. Regarding the navigation: Was there something that you wanted to do and couldn’t?6. In what aspects doing the simulation was like real reporting? Why?7. What would you change to make the simulation better? Why?


59

Results of embedded survey


60

mulation helped me improve myding of: Student Student Student Student Student Student Student Answers by number

1 2 3 4 5 6 7 S. Agree Agree No opinion Disagree S. Disagree 1 2 3 4 5

ing a fire story 1 2 1 1 2 2 1 4 3 0 0 0

g the facts and sources right 2 2 3 2 2 2 2 0 6 1 0 0

on to detail 1 1 3 2 2 2 2 2 4 1 0 0

ewing skills 3 2 3 4 4 4 2 0 2 2 3 0

makingsure that Ilisten toonlycrediblesources!,learningabou thevarioustypes ofelementsfor thistype ofstory

mulation was an effective tool to my news reporting skills. 1 2 3 1 4 2 2 2 3 1 1 0

mulation was an effective tool to my investigative skills. 2 2 3 2 4 4 2 0 4 1 2 0

mulation was an effective tool to my writing skills. 2 2 2 2 2 2 3 0 6 1 0 0

llowing features of the simulationpful:

to hear the police scanner 1 2 2 2 2 1 2 2 5 0 0 0

to veiw video 3 2 2 1 2 1 1 3 3 1 0 0

to interview people involved in the fire 2 2 2 2 4 1 1 2 4 0 1 0

to see examples of other stories 2 3 2 1 3 3 1 2 3 0 0

easy to move through the informationlowing sections of the simulation:

roun readings 1 2 2 2 1 2 2 2 5 0 0 0

and grammar quiz 4 1 3 2 1 2 2 2 3 1 1 0

ort city map 2 4 1 2 1 2 4 2 3 0 2 0

d it easy to navigate the simulationwhat I needed. 2 2 2 2 2 2 4 0 6 0 1 0

ll, the simulation was useful. 1 2 2 1 3 2 2 2 4 1 0 0

nterface for the simulation wasd logically. 1 1 2 2 2 2 2 2 5 0 0 0

ext was easy to read on the computer2 1 4 2 1 2 2 3 0 1 0

s able to use the simulation from2 4 1 3 1 1 1 1 0


61

newssim project evaluation - columbia...

Documents