a cognitive apprenticeship for scientific literacy based on journalism joseph l. polman, e. wendy...
TRANSCRIPT
A Cognitive Apprenticeship for Scientific Literacy Based on
Journalism
Joseph L. Polman, E. Wendy Saul, Alan Newman, Cathy Farrar, Nancy Singer, Eric Turley, Laura Pearce,
Jennifer Hope, Glenda McCarty, and Cynthia Graville
University of Missouri-St. Louis
International Conference of the Learning Sciences 2010
This material is based upon work supported by the National Science Foundation under Grant No. DRL-0822354
Acknowledgements
• NSF DRK-12 (Discovery Research K-12)
• Scijourn Project Team Members:
• Co-PIs: Joe Polman, Wendy Saul, Alan Newman, Cathy Farrar
• UMSL: Mike Butler, Jennifer Hope, Laura Pearce, Cindy Graville, Angela Kohnen, Glenda McCarty, Nancy Singer, Eric Turley
• SLSC: Diane Miller, Terris Grimes, David Hoffelmeyer, Rachel Mahan
• Advisory Board: Hubert Dyasi, Kevin Leander, Bill Penuel, Carol Stearns
• External Evaluator: Brian Hand, Univ of Iowa
Scientific Literacy through Science Journalism (SciJourn) Project
• Purpose:
promote public understanding of and engagement with science and technology
• Means:
involve high school aged youth (14-18 years old) in hybrid science journalism activities, centered around the creation of an authentic science news publication
Who, Where, and When
• Teachers working at schools• Urban, suburban, and rural
• Biology, Chem, Physics, English, Journalism …
• Youth program run by SLSC• Summer 20+ hrs/week for 5 weeks
• School year on Saturdays
• 2008-09: Alpha and beta semesters• 2009-10: Pilot year• 2010-11: Cadre I implements, Cadre II comparison
Screen shot here
What does a science journalist do?
• Articulate an issue of contemporary interest• Research it using multiple, credible sources
(print, interview, etc.)• Digest and make sense of information, including
complex and technical information• Construct an evidence-based story• Put it out so that multiple people can understand
Relates to Scientific Literacy as …
• Public understanding of and engagement with science
• Educate citizens to• make informed decisions about STEM-related issues
within their personal lives
• engage critically in democratic discourse and debate about STEM issues within the public sphere
• Requires• Understanding and communicating the meaning and
significance of science and technology information
Toward an Articulation of Standards for Scientific Literacy Based on
Journalism
Wendy Saul, Alan Newman, Nancy Singer, Eric Turley, Laura Pearce, and Joe Polman
Standards in original proposal
• To “flesh out” content standard in 6+1 so that that scoring rubric would have more utility for evaluating science text.
• To help teachers (and others) assess and respond to SciJourn student writing
• To help us understand and document progress in student work
These are “hybrid” standards
• What is realistic to expect from a HS student, not a professional journalist?
• Promotes educators’ scaffolding based on authentic practices
• We recognize the difference between what it takes to get published in SciJourner as opposed to what is useful for teaching (see SLAP -- no stereotyping, lying, advertising or plagiarism)
The path
• Learning from professionals
• Led to focusing on certain aspects that seemed possible to teach in a H.S.
• Led to first iteration of standards and
• Coding (Nancy and Eric) for more “tune up”
• while working in schools and at a youth development program
• Led to Alan “seeing the same damn thing over and over again”
• Other iterations
• Feedback from professionals
• Current version
The processes the standards need to support
• Articulate idea of current interest• Locate multiple, credible sources• Digest information• Shape an evidence-based story• Share in an authentic venue
Let’s look at the Standards
(It’s a recursive, iterative process we seek to promote here)
Finding a topic:Relevant
IV. Scientific information is relevant to readers.
IV.A: Reported findings are linked to local or personal concerns and new applications are considered.
IV.B: Readers’ implied questions are anticipated and addressed.
Finding a topic: Searching
I: Students are able to search effectively for and recognize relevant, credible information sources, especially on the Internet.
I.A: Knows how to use search engines and search terms
I.B.: Privileges data from credible government and nonprofit sites (e.g.; nih.gov and cancer.org) and can ascertain the credibility of other websites, using the “About Us” for clues.
I. C.: Keeps track of sources, including dates of publication, author names and expertise and home institution for purposes of attribution.
Locate multiple, credible sources
II.A Multiple sources
II.B Credible sources
II.B.1: Sources are relevant and reliable.
II.B.2: Appropriate stakeholders are consulted.
II.C.: Attributed sources
II.C.1: Except for accepted facts, ideas & theories, all assertions, numbers, details and opinions are attributed.
II.C.2: The names of the experts/organizations are given and their area of expertise/qualification is identified. Any biases or potential conflicts of interest are noted.
II.C.3: Copyright rules are followed and relevant URLs are given.
Digest information: related to factual accuracy
V.A: The story structure indicates what is more and less important from a reader’s and writer’s perspective. The science connection is noted.
V.B: The article shows an understanding of the content and is able to explain concepts and information, including methods of scientific inquiry.
V.C: Precise language is employed and scientific terms are used appropriately.
V.D: Quantitative measures are given in correct and comparable units.
V.E: The latest/up-to-date information is presented.
V.F.: The headline and photo caption accurately reflect the content of the story.
Shape the story
III. Scientific information, discoveries and technologies are contextualized; broader implications as well as reflections on past and future understandings are noted.
III. A: The import of the information for society is understood and sufficiently detailed
III.B: The article indicates which data/ideas are widely accepted in the scientific community and which are preliminary. The article sensibly weights the import of findings and, where appropriate, uses qualified rather than declarative language.
Share
• Know your audience (back to relevance)• Fore-fronting important information (back to
factually accurate)• Attributing sources (back to multiple, credible)• Responding to queries- authentic venue
Looking forward to seeing how well they work as a frame for
• transfer tasks
• assessment of student texts and
• read alouds.
Rubrics
• In original grant rubrics were supposed to enhance standards, i.e. be used primarily to assess writing samples.
• We have come to understand the difficulty of creating rubrics that are valid and reliable like those developed by NWP and WestEd.
Typical rubric scoring system designed by our teachers
Re: Attribution
All refs are correctly attributed
Most refs are correctly attributed
Some refs are correctly attributed
Few refs are attributed
No refs are attributed
We are moving toward specification sheets or
criteria assessments.Attribution:• Identify (underline and number) each ref needing
attribution.• Did each ref include all info needed so that someone
could trace back the ref?• The peer editor will:
check each ref. starring any they could not locate.
Standards now serving as the hub.
• Curriculum Development: Useful for helping students learning to read and write as well as assess the quality of science news articles.
• Research: useful in helping identify skills we associate with scientific literacy.
• Professional Development: useful for building teacher knowledge
• Outreach: Utility beyond the news story and actually beyond science, e.g. critical thinking, reading in the disciplines
Closing thoughts:
• It appears that these standards have utility beyond SciJourn (for reading as well as writing, for teaching research skills and critical thinking.)
• Prods discussion on personal interests vs “big ideas.” Provides a low bar for entering into science discourse.
• Importance of strategies for helping teachers respond to student writing-- something they and we clearly identify as needing attention.
Designing Transfer Tasks to Assess Scientific Literacy
Cathy Farrar, Joe Polman, Wendy Saul, and Alan Newman
Purpose of this strand
Develop a series of tasks that provide insight into a student’s ability to:– Evaluate expertise– Identify appropriate questions for experts– Select and use multiple credible sources to gain more
information– Determine coherence between text and image or graphic– Employ effective search strategies for more information– Link current science issues to self and others
Scientific Literacy
• Contextualization, content knowledge and skills are generally agreed upon as components of scientific literacy.
• Many of the current assessments focus on one or two of these components
• Rarely is scientific literacy measured in its “entirety” (Laugksch & Spargo, 1996) and few composite measures that get at the complexity have been published (Laugksch, 2000).
Specific Content
The moon has phases because...
a. It passes into the Earth's shadow every month
b. Only the side of the moon facing the sun is illuminated
c. The moon causes tides in the oceans
d. None of the above
2009 Torstar Syndication Services. Permission granted for up to 5 copies. All rights reserved.
Hot-button topic
Human Cloning a. Human beings have been cloned.
b. While we are not yet able to clone humans other mammals such as sheep, goats, rabbits, pigs, cats, horses, and deer have been successfully cloned.
c. The only mammal that has been cloned is a sheep.
d. Cloning has only been successfully performed on lower species such as insects.
http://www.vladozeman.com/Test.htm
Attitude or interestQ1. How connected do you feel to the natural world
• very connected • somewhat connected• not very connected• not at all connected
http://science4grownups.com/archives/2009/04/10/general/california-academy-of-sciences-scientific-literacy-poll-part-1-322
Current Assessments
• Assessments such as these measure content knowledge or attitude toward science without much focus on contextualization or skills.
• The SciJourn Scientific Literacy Assessment incorporates content knowledge, but does not rely on it. Instead its focus is assessing skills that students use to engage with science information.
SciJourn Scientific Literacy Assessment
• Korpan, Bisanz, and Bisanz (1997) posited that one hallmark of a scientifically literate individual is “the ability to make effective requests for information or to ask good questions about scientific research”
• It is in this vein that initial transfer tasks were developed
• Began as a series of tasks that utilize informational text in the form of brochures, press releases, or text with associated graphic or image.
Development of the tasks
• A series of eight tasks were developed and piloted in 10 different high school classrooms.
• Revision based on teacher feedback and pilot student responses
• Alpha composite assessment created and piloted • Reframing of the composite assessment using
Item Response Theory (Baker and Frank, 2001) • Beta composite assessment (IRT) developed and
piloted in May/June 2010
Task Description
• Authentic examples of science texts they would encounter• Brochures
• Science news articles
• Science images and graphics with captions
SciJourn Scientific Literacy Assessment
Strand Sample prompt
Locating credible information
Which hyperlinks would you choose, explain why and what you hope to find once you click this link?
Relevance Why is this information important to the general public?
Contextualization Why might high blood pressure be more common in teens today than 100 years ago?
Factual Accuracy If you had to present this information to a community group, what information would you check regarding lead before including in the presentation and why?
Multiple Credible Attributed Sources
What other information about SmartKid.com would you like to have to determine its credibility?
Validity and Reliability
• Face and content validity— science educators and science journalists
• Test-retest administration to non-study group• Developing scoring framework• Confirmatory factor analysis with pilot data and
SciJourn content standards• Development of scoring guide• Projected completion Fall 2010
Reframing and Assessing Engagement with Science and
Technology
Jennifer Hope, Glenda McCarty,
and Joe Polman
This material is based upon work supported by the National Science Foundation under Grant No. DRL-0822354
Purpose of this strand
• Inspired by Rosenzweig & Thelen’s Presence of the Past• Clarify the notion of engagement• Align the idea with recent views• Enhance understanding of ways youth are engaged in
science and tech in school and out• Develop a valid means of assessing engagement, and
change in engagement over time• For educators, as an indicator of engagement level
• As a program evaluation tool
Background• Why engagement?• What is engagement?
• time on task (Lee & Anderson, 1993; Singh, Granville, & Dika, 2002)
• interest, motivation (Thompson & Windschitl, 2002)
• perceived ability (Markowitz, 2004)
• interaction (Olitsky, 2007)
• persistent (Azevedo, 2004)
• Where? In school, and out (Zimmerman & Bell, 2007)
Our Framework
• Interest, or openness to and stance toward the science and technology in the moment
• Action, or actual involvement with science and technology ideas and tools
• Identity, or ways that the science and technology connects to people’s identity affiliations, in the past, present, and future
(not understanding or knowledge per se)
Instrument Development Method
• Initial draft based on existing research (e.g. Falk, et. al,
2001 & Durant, 1999) and our framework• Interview-style administration• Think-aloud with written administration• Focus group • Teacher & research team feedback • Case study analysis• Pilot administration (10 schools)
• Pre-survey
• Post-survey
Survey Design
Aspect Number of Questions
Types of Questions
Demographics 7 Gender, grade level, zip code
Interest 21 Level of interest in school, science, technologySpecific areas of interest in science/technology
Action 17 Frequency of activities related to science/tech Ways of spending free timeInformation sources utilized
Identity 19 Activities in everyday life related to science/techViews of self related to science/technologyFuture plans
Case Studies
• Comparison of YEST survey results with observation data from summer informal science program participants
• Four cases were analyzed using themes of interest, action, and identity
• Results of case study analysis utilized for further modification of YEST instrument
Tre
Survey ResponsesRates science knowledge at 8.5/10 compared to peersScience favorite subject, with interests in robotics, nanotechnology, and brain-computer interfacesRegular consumer of science and technology infoWishes there were “more types of courses” in science to take
Observational Data
Even more into sci-fi than science
Technology such an interest that access became a distraction to program goals
Keeps up to date on science and tech news using iPhone apps
Expressed disappointment at not going deeper into discussing “the science of everything”
High school junior; African-American male; 3rd year participant in summer science program
Khadijah
Survey Responses
Sees science as related to her routine daily activities, but more interested in technology
Particular interests in chemistry and astronomy
Usually interested in science in school, and sometimes outside of school
Observational Data
Often “plugged in” to multiple tech outlets at once
Thorough investigation of tech tools used at tv station
Stated interests never surfaced in projects
Selected personal topics, and then sought science connection
College-bound h.s. senior; African-American female attending public school in the city; 4th year program participant
Max
Survey Responses
Seeks science & technology “for fun and personal enjoyment” and “to learn more about the topics”
Considers himself an excellent learner with average grades
Notes that he will need to use computers every day of his life, including career interests
Observational DataDespite early resistance to writing, produced three science stories in 5 weeksMotivated to write about topics of personal interest: the SciJourn project, electric cars, and tattoosAssociated himself with project staff & participants sharing technological interests
High school senior; First generation Eastern European immigrant; 3rd year program participant
Alisha
Survey Responses
Good student with good grades
Interactions with science and technology are mostly “required” and/or “assigned”
Highly structured student, excels in a traditional school system model
Observational Data
Actions and identity which aligns with science, but her primary science interest was instrumental
In summer did not take ownership of her writing
Seldom shared her scientific knowledge with others
African American high school senior; College prep and EMT courses; 4th year program participant
Summary
• Many challenges of only using one instrument to assess a complex construct.
• Compared the youth's daily interactions with science and technology and evaluated that against their responses to the YEST survey.
• Analysis is to support the development and modifications of the YEST survey and show that it is possible to use only one instrument.
• Next steps: case studies and surveys of school-based participants
Reliability & Validity
• Face and content validity—teens, teachers, and SciJourn research team
• Standardized codebook• Test-retest administration to non-study group• Still to come…
• Factor analysis with pilot data
• Development of scoring guide
Building an Apprenticeship Community of Practice for Science Journalism
Joe Polman, Wendy Saul, Alan Newman, Laura Pearce, and Cindy Graville
The role of SciJourner within the community
At least 362 articles drafted and submitted for consideration
51 articles edited, revised, and published at the online SciJourner
- 45 from students/teens
- 6 from teachers
- 36 traditional articles, 3 videos, 2 audio podcasts, 3 extended photo captions, 7 book reviews
The Role of SciJourner (continued)
Social networking aspect of online publication
53 comments on stories
>1700 hits on “Heartbreak of Potter’s Syndrome”
>100 hits on 38 articles
5 Print editions of SciJourner
Hundreds of copies distributed
Excitement fueling student submissions, teacher recruitment, and … an authentic writing task driving scientific literacy
Role of Teachers in the Community
• Writers themselves• Developed and critiqued lesson plans
• elaborated and adapted ideas
• created new models: e.g., info graphics
• Involvement in the research (feedback on instruments, collection of artifacts, participation in case studies, teacher action research studies, etc.)
Boundary Objects and Brokering
• Connections across boundaries of communitiesE.g., from journalism to school to ISE
• Boundary Objects: Material and conceptual artifacts around which communities organize practicesE.g., SciJourner, “credibility”
• Brokering: Agents from one community negotiating practices in another communityE.g.,: journalists, scientists, education researchers, teachers, students
Key boundary objects and intersections of practice
1) The genre of the science news articles vs. the traditional school-based "5-paragraph essay”
2) Story pitches that connect personal interest and concern with relevant science
3) Internet searches for information, and uses of Wikipedia
4) “Credibility” and “attribution” of sources
5) “Editing” for publication, vs. “responding” to students to support their learning
6) Revisions in science classrooms
Genre wars: Science news article vs. 5 paragraph essay
• Science news article• Inverted triangle structure
• Created for communicative purposes, with pragmatic constraints
• 5 paragraph essay• Defines the norm of non-fiction writing in schools
• Created for assessment purposes
• Persuasive and expository essays
• Ask students to “set aside” in this class, but allow for hybridity while meeting the standards
Pitching and crafting story ideas
• Personal connections and interests• “Thinking about a tattoo?” (Health risks)
• “"Exceptions" to the Foster Care System” (HS graduation rates)
• “Hunting Proves both Profitable and Crucial”
• “The Heartbreak of Potter's Syndrome”
• “Playing in Blood, Sweat, and Saliva” (Astroturf)
• “There’s science in everything”• “Authentic” in the sense of personal meaning
Search
• Search engine• Wikipedia – (paddling against a strong stream)
• For technical terms
• For sources
• Directed search • WebMD, NIH, American Cancer Society …
• “There’s an organization for every issue.”
• At any website • Read “About Us”
• Use other credibility clues
“Credible” and “Attributed”
• Terms have become a strong part of the classroom discourse
• Negotiation and elaboration of their meanings drives understanding
• Attribution issues• Poor model of most familiar authoritative texts
(textbooks)
• “Works cited” from English class
“Editing” vs. “Responding”
Editing Responding
Typical agents/Brokers
Professional editor Teacher
Purpose Improve and polish the article for publication
Drive learning
Norms Blunt, no-nonsenseDetailed
Begin with positive, Remain supportive(in practice, may be broad & vacuous)
Tools MS Word’s “Track Changes”
Pencil or pen markup
“Designing for learning” while maintaining authenticity of publication requires both
Revisions
• Writing in science classrooms (typically labs) is seldom revised
• We are brokering a practice from English/Communication Arts classrooms (writing workshop models)
• Some resistance, but SciJourner as a boundary object drives authentic purpose
