TeachingWithData.org Resources for Teaching

Quantitative Literacy in the Social Sciences

John Paul DeWitt & Lynette HoelterUniversity of Michigan

ASA Annual Meeting, August 15, 2010

Presentation Outline:• Introducing the project partners• Quantitative Literacy • Introducing TeachingWithData.org

– General overview (demo of Website)– Sociology-related resources– Future directions

Project Partners• ICPSR • SSDAN• Others involved:

– American Economic Association Committee on Economic Education

– American Political Science Association– American Sociological Association– Association of American Geographers– Science Education Resource Center, Carleton

College

ICPSR• World’s oldest and largest social

science data archive– Began in 1962 as ICPR

• Membership organization with 700+ members worldwide (non-members can use many resources)

• Summer Program in Quantitative Methods of Social Research

Current Snapshot of ICPSR• Currently 7,880 studies (65,200 data sets)

– Grouped into Thematic Collections– Available in multiple formats– Federal funding allows parts of the

collection to be openly available– Data sources:

• Government• Large data collection efforts• Principal Investigators• Repurposing• Other organizations

ICPSR: Undergraduate Education• Fairly recent attention

– Response to faculty– Undergrad users are fastest growing

segment• Resources

– OLC, SETUPS, ICSC, EDRL• NSF-funded projects

– TeachingWithData.org (NSDL)– Course, Curriculum, & Laboratory

Improvement project to assess the effect of using digital materials on students’ quantitative literacy skills

7

SSDAN-OLC• SSDAN’s primary focus is to assist in the

dissemination of social data into the classroom with sites like DataCounts! and CensusScope

• ICPSRgreat track record in research, with a new attention on undergraduate education coming more recently with the welcomed Online Learning Center (OLC)

8

SSDAN: Background• Started in 1995• University-based organization that creates

demographic media and makes U.S. census data accessible to policymakers, educators, the media, and informed citizens. – web sites– user guides – hands-on classroom materials

• Integrating Data Analysis (IDA)

9

SSDAN: Classroom Products• DataCounts!

(www.ssdan.net/datacounts)– Collection of approximately 85 Data Driven Learning

Modules (DDLMs)– WebCHIP (simple contingency table software)– Datasets (repackaged decennial census and

American Community Survey)– Target audience is lower undergraduate courses

• CensusScope (www.censusscope.org)– Maps, charts, and tables – Demographic data at local, region, and national levels– Key indicators and trends back to 1960 for some

variables

10

SSDAN: DataCounts!

Quickly connects users to datasets… ..or Data Driven

Learning Modules

11

SSDAN: DataCounts!

Menu for choosing a dataset for analysis

Brief List of available dataset collections

12

SSDAN: DataCounts!Submitting a module:• Sections are clearly laid out• Forces faculty to create modules

with specific learning goals in mind.

• Makes re-use of module much easier

13

SSDAN: DataCounts!

TitleAuthor and Institution

Brief Description

Faceted browsing to refine the search• Appropriate Grade Levels• Subjects (e.g. Family, Sexuality and

Gender)• Learning Time

14

SSDAN: DataCounts!Data Driven Learning Modules are clearly laid out• Easy to read• Instructors can quickly identify

whether a module would be relevant to a specific course

15

SSDAN: DataCounts!• WebCHIP Commands for selecting variables,

creating tables, graphing, and recoding

Basic information about the dataset

Running the “marginals” command shows the categories for each variable and frequencies

16

SSDAN: DataCounts!

Students can quickly run simple cross tabulations to see distributions and test hypotheses

17

SSDAN: DataCounts!

Controlling for an additional variable allows for deeper analysis

18

SSDAN• DataCounts!

– Collection of approximately 85 Data Driven Learning Modules (DDLMs)

– WebCHIP (simple contingency table software)– Datasets (repackaged decennial census and

American Community Survey)– Target is lower undergraduate courses

• CensusScope– Maps, charts, and tables – Demographic data at local, region, and national levels– Key indicators and trends back to 1960 for some

variables

19

SSDAN: CensusScope

New ACS data with improved look & feel coming Fall 2010

20

SSDAN: CensusScope• Charts, Trends,

and Tables• All available for

states, counties, and metropolitan areas

Thinking about Quantitative Literacy (QL)

• CCLI project to measure effectiveness of using online modules to teach QL

• First need to agree on skill set representing QL in the social sciences– Most use data-based exercises to teach

content– QL/QR has gotten much recent attention

in institutional assessment, many schools requiring a QL component

What is QL?• “Statistical literacy, quantitative literacy, numeracy --

Under the hood, it is what do we want people to be able to do: Read tables and graphs and understand English statements that have numbers in them. That’s a good start,” said Milo Schield, a professor of statistics at Augsburg College and a vice president of the National Numeracy Network.

Shield was dismayed to find that, in a survey of his new students, 44 percent could not read a simple 100 percent row table and about a quarter could not accurately interpret a scatter plot of adult heights and weights.

Chandler, Michael Alison. What is Quantitative Literacy?, Washington Post, Feb. 5, 2009

Similar to Critical Thinking:• Students as participants in a

democratic society• Skills include:

– Questioning the source of evidence in a stated point

– Identifying gaps in information– Evaluating whether an argument is based

on data or opinion/inference/pure speculation

– Using data to draw logical conclusions

Quantitative Literacy• Necessary for informed citizenry• Skills learned & used within a context• Skills:

– Reading and interpreting tables or graphs and to calculating percentages and the like

– Working within a scientific model (variables, hypotheses, etc.)

– Understanding and critically evaluating numbers presented in everyday lives

– Evaluating arguments based on data– Knowing what kinds of data might be useful in answering

particular questions• For a straightforward definition/skill list, see

Samford University’s (not social science specific)

Translating to Learning Outcomes• Began with AAC&U rubric for quantitative reasoning• QL in social sciences:

– Calculation– Interpretation– Representation– Analysis– Method selection– Estimation/Reasonableness checks– Communication– Find/Identify/Generate data– Research design– Confidence

Learning Outcome Dimensions• Calculation: Ability to perform mathematical

operations• Interpretation: Ability to explain information

presented in a mathematical form (e.g., tables, equations, graphs, or diagrams)

• Representation: Ability to convert relevant information from one mathematical form to another (e.g., tables, equations, graphs or diagrams)

• Analysis: Ability to make judgments based on quantitative analysis

Learning Outcomes (con’t)• Method selection: Ability to choose the

mathematical operations required to answer a research question

• Estimation/Reasonableness Checks: Ability to recognize the limits of a method and to form reasonable predictions of unknown quantities

• Communication: Ability to use appropriate levels and types of quantitative information (data, reasoning, tools) to support a conclusion or explain a situation in a way that takes the audience into account.

Learning Outcomes (con’t)• Find/Identify/Generate Data:

Ability to identify or generate appropriate information to answer a question

• Research design: Understand the links between theory and data

• Confidence: Level of comfort in performing and interpreting a method of quantitative analysis

29

Assessment Tools and Results

QL Skills Are Marketable• Often cited by students as

something “tangible” that they have learned

• Definable skill set useful in many career paths

• Easy to tie to everyday life

Including Data Builds QL and:

• Engages students with disciplines more fully – Active learning– Better picture of how social scientists work– Prevents some of the feelings of

“disconnect” between substantive and technical courses

• Piques student interest• Opens the door to the world of data

TeachingWithData.org• National Science Digital Library – only social science

pathway• Goal: Make it easier for faculty to use real data in

classes– Undergraduate (esp. “non-methods”)– K(9)-12 efforts

• Includes survey of ~3600 social science faculty • Repository of data-related materials

– Exercises, including games and simulations– Static and dynamic maps, charts, tables– Data – Publications

• Tagged with metadata for easy searching

Major Changes since Oct. 2009• Redesign of the interface on the main page

– Guided Search from home page– Resources categorized by more general ‘resource type’ controlled vocabulary

• Data focused on tables and figures vs. data sets• Reference Shelf Data Sources, events, pedagogy• Classroom Resources Grouped like resources,

– Search box with grade level • Spring Cleaning – removed hundreds of resources• Identified items at lower levels (higher granularity)• User log-in (OpenID) and submission• Local content• Data in the News blog• Data for Online Analysis• Reading list: ability to create, save, and share

– Favorites– List of resources for course, project, or textbook– TwD and external resources

New Account Setup (OpenID)

New Account Setup

TeachingWithData.org

TeachingWithData.org

TeachingWithData.org

TeachingWithData.org

Future Changes• Professional Association editors

– Submit, edit metadata, review resources• “Report” button for review and edit

– Cleaner metadata, outdated links, etc• Comments• OpenStudy partnership?

– Ratings– Recommendations– User Collaborations (Instructor-Instructor, Instructor-

Student)– Instant feedback and help

– TRAILS indexing

OpenStudy.com

Sociology Resources

Example Resources• “Data in the News” feature – good

way to bring in current events• Lesson plans/lectures• Data-driven exercises• Data sources• Tools

Lesson Plans (Example)

More Extensive Lesson Plans (Example)

International Data & Information for Comparison (Example)

Example: Short Video on Family Change in Canada

Static Tables (Example)

Graphs & Maps (Example)

Interactive Maps (Example)

Data-Based Exercises: “Low-Tech” (Example)

Data-Based Exercises: Online (Example)

Data-Based Exercises: No Stat Software Needed (Example)

Simulations (Example)

Data for Online Analysis: No Software Needed (Example)

Educational Data Extracts for Statistics Packages (Example)

Tools for Data Visualization (Example)

Future Directions:• Include resources for high school teachers• Ability to link data to analysis and/or

visualization tools• Ability for faculty to rate and comment on

resources• Peer-reviewed materials and capability for

faculty to upload their own resources• Community building through professional

associations and networks of users

Your Turn!• What have you tried? • What has worked best? • Favorites we should include in TwD?

Acknowledgements• PI: George C. Alter, ICPSR• Co-PI: William H. Frey, SSDAN

• Funded by National Science Foundation grant DUE-0840642