A flavor of the CATALST Course:

Using randomization-based methods in an introductory

statistics course

Elizabeth Fry and Laura ZieglerCATALST Team: Joan Garfield, Andrew Zieffler, Robert delMas, Allan Rossman, Beth Chance, John Holcomb,

George Cobb, Michelle Everson, Rebekah Isaak, & Laura Le

Funded by NSF DUE-0814433

Outline of Presentation• Introduction to CATALST course• Radical content• Radical pedagogy• Radical technology• Student assessment• What we learned• Publications and references

"I argue that despite broad acceptance and rapid growth in enrollments, the consensus curriculum is still an unwitting prisoner of history. What we teach is largely the technical machinery of numerical approximations based on the normal distribution and its many subsidiary cogs. This machinery was once necessary, because the conceptually simpler alternative based on permutations was computationally beyond our reach. Before computers statisticians had no choice. These days we have no excuse. Randomization-based inference makes a direct connection between data production and the logic of inference that deserves to be at the core of every introductory course."

Inspiration for CATALST George Cobb (2005, 2007)

Cooking in Introductory Statistics

• CATALST teaches students to cook (i.e., do statistics and think statistically)

• The general “cooking” method is the exclusive use of simulation to carry out inferential analyses

• Problems and activities require students to develop and apply this type of “cooking”

Schoenfeld, A. H. (1998). Making mathematics and making pasta: From cookbook procedures to really cooking. In J. G. Greeno & S. Golman (Eds.), Thinking practices: A symposium on mathematics and science learning (pp. 299-319). Hillsdale, NJ: Lawrence Erlbaum Associates.

Radical Content• New sequence of topics; building ideas of

inference from first day• No t-tests; use of probability for simulation

and modeling (TinkerPlots™) • A coherent curriculum that builds ideas of

models, chance, simulated data• Immersion in statistical thinking• Textbook (Statistical Thinking: A Simulation

Approach to Modeling Uncertainty) written for this course includes examples using real data

Randomization-Based curriculum• No z-tests or t-testsInstead, students:• Specify a model

– Random chance, or “no difference” model

• Randomize and Repeat– Simulate what would happen under the model and

repeat many trials

• Evaluate – Compare observed result to what is expected under

the model

3 CATALST Units

• Chance Models and Simulation• Models for Comparing Groups• Estimating Models Using Data

Radical Pedagogy• Student-centered approach based on

research in cognition and learning, instructional design principles

• Minimal lectures, just-in-time as needed• Cooperative groups to solve problems• “Invention to learn” and “test and

conjecture” activities (develop reasoning; promote transfer)

• Writing; present reports; whole class discussion

Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2),129- 184.

Example from a Non-Randomization-Based CourseA student takes a 50 question multiple choice test with four options per question. She has not studied for the test, but she gets a score of 54%. Is her performance on this test better than what would be expected if she was blindly guessing on each question?

Example from a Non-Randomization-Based Course• Approach: Hypothesis testing

– H0: p = 0.25

– Ha: p > 0.25

• Check assumptions: np0 > 10 and n(1-p0) > 10

• Test statistic = = 4.75• Calculate p-value < 0.001• Conclude: Yes, the student is doing better

than random guessing.

CATALST Example: Matching Dogs to Owners

• Do dogs resemble their owners?• Research Question:

1. _____ 1)

2)2. _____

3. _____

4. _____

5. _____5)

4)

3)

6)6. _____

1. _____

2. _____

3. _____

4. _____

5. _____

6. _____

1

3

6

5

2

4

Non-Randomization-Based CourseTechnology• Students use technology (e.g. StatCrunch, Minitab, graphing calculator) to compute p-value

• The main purpose of technology is to help with calculations.

Radical Technology• Focus of the course is simulation• TinkerPlots™ software is used• Unique visual (graphical interface)

capabilities– Allows students to see the devices they select

(e.g., mixer, spinner)– Easily use these models to simulate and collect

data– Allows students to visually examine and evaluate

distributions of statistics

Konold, C., & Miller, C.D. (2005). TinkerPlots: Dynamic data exploration. [Computer software] Emeryville, CA: Key Curriculum Press.

Matching Dogs to OwnersBuilding the Model & Simulation

Radical Assessment

• Frequent and varied assessment• Assess students’ ability to reason and

think statistically • Focus less on computation and more on

understanding of concepts

CATALST Student Assessments• Homework

– Approximately 1 per in-class activity (15 in total)– Reinforces ideas from the in-class activities

• Exams– 3 group exams– 2 individual exams

• Final Exam– Basic knowledge: GOALS assessment (Goals and

Outcomes Associated with Learning Statistics)– Statistical thinking: MOST assessment (Models of

Statistical Thinking)

Non-Randomization-Based CourseExample Assessment Item• In order to set rates, an insurance company is trying to estimate the number of sick days that full time workers at a large company take per year. A sample of 50 workers is randomly selected and the sample mean number of sick days is 4 days per year, with a sample standard deviation of 1.4 days. – Find a 95% confidence interval for the population mean

number of sick days for full time workers at this company.

• Students will compute a t-interval to answer this question.

• One problem: We are estimating the average – but this may not be the best measure of center if distribution is skewed.

Assessments to Evaluate the CATALST Curriculum• GOALS (Goals and Outcomes Associated with

Learning Statistics) – 27 forced-choice items – Items assess statistical reasoning in a first

course in statistics• MOST (Models of Statistical Thinking)

– 4 open-ended items that ask students to explain how they would set up and solve a statistical problem

– 7 forced-choice follow-up items

Advantages of Randomization-Based Curriculum• Does not require much math background• You can look at messier problems like

Matching Dogs to Owners• Can make inferences about any statistic

(e.g. median), not just limited to means and proportions

• Fewer assumptions are required• Focus is on inference• Takes advantage of modern technology

Disadvantages of Randomization-Based Curriculum• Technology must be readily available in

the classroom• Students may still want or need to

learn z- and t-procedures However…• Many of our students bring laptops to

class• Our students come from fields where

they will not need to use z- and t- procedures

What We Have Learned

• We can teach students to “cook”.• Based on interview and assessment data,

students seem to be thinking statistically (even after only 6 class periods!)

• We can change the content/pedagogy of the introductory college course.

• We can use software at this level that is rooted in how students learn rather than purely analytical.

CATALST PublicationsGarfield, J., delMas, R. & Zieffler, A. (2012). Developing

statistical modelers and thinkers in an introductory, tertiary-level statistics course. ZDM: The International Journal on Mathematics Education.

Ziegler, L. and Garfield, J. (in press) Exploring student understanding of randomness with an iPod shuffle activity. Teaching Statistics.

Isaak, R., Garfield, J. and Zieffler, A. (in press). The Course as Textbook. Technology Innovations in Statistics Education.

Garfield, J., Zieffler, A., delMas, R. & Ziegler, L. (under review). A New Role for Probability in the Introductory College Statistics Course. Journal of Statistics Education.

delMas, R. , Zieffler, A. & Garfield, J. (under review). Tertiary Students' Reasoning about Samples and Sampling Variation in the Context of a Modeling and Simulation Approach to Inference. Educational Studies in Mathematics.

Contact Information

jbg@umn.edu

Joan Garfield

http://www.tc.umn.edu/~catalst/

References• Cobb, G. (2005). The introductory statistics course: A saber tooth

curriculum? After dinner talk given at the United States Conference on Teaching Statistics.

• Cobb, G. (2007). The introductory statistics course: A ptolemaic curriculum? Technology Innovations in Statistics Education, 1(1). http://escholarship.org/uc/item/6hb3k0nz#page-1

• Roy, M.M. & Christenfeld, N.J.S. (2004). Do dogs resemble their owners? Psychological Science, 15(5), 361-363.

• Schoenfeld, A. H. (1998). Making mathematics and making pasta: From cookbook procedures to really cooking. In J. G. Greeno and S. V. Goldman (Eds.), Thinking practices in mathematics and science learning (pp. 299–319). Mahwah, NJ: Lawrence Erlbaum

Matching Dogs to OwnersBuilding the Model & Simulation

Matching Dogs to OwnersBuilding the Model & Simulation

Matching Dogs to OwnersBuilding the Model & Simulation

Matching Dogs to OwnersBuilding the Model & Simulation

Multiple Choice Example Using Randomization

Mixer Stacks Spinner Bars Curve Counter

Fastest Options

Draw1

Repeat50

Question

Right

0.2500

Wrong

0.7500

Results of Sampl... Options

Question <new>

2

3

4

5

6

7

8

Right

Right

Wrong

Wrong

Wrong

Wrong

Wrong

Results of Sampler 1 Options

Wrong Right

70% 30%

Question

Circle IconHistory of Results of Sampler 1 Options

0 5 10 15 20 25 30 35 40 45 50 55 60

100% 0% 0%

percent_Question_Right

Circle Icon

p < 0.001

(Using 1,000 trials)