ICOTS 1

Student Attitudes toward Statistics from a

Randomization-Based Curriculum

Todd Swanson

Jill VanderStoep

Hope College, Holland, Michigan, USA

July 14, 2014

Overview

• Background on our curriculum

• Findings on student attitudes toward statistics from Fall 2013 and Spring 2014

July 14, 2014 ICOTS 2

History• 2005

• George Cobb challenges us to put the core logic of inference at the center of our curriculum using randomization (USCOTS)

• 2007 • Concepts of Statistical Inference: A Randomization-Based Curriculum

(Rossman, Chance, Holcomb, and Cobb)

• 2009 • Complete curriculum developed at Hope College (Tintle, VanderStoep

and Swanson)

• 2010• Joined forces (Tintle, Chance, Cobb, Rossman, Roy, Swanson,

VanderStoep) to further develop curriculum, class test at a wide variety of institutions, and assess results. (project named Introduction to Statistical Investigations)

3July 14, 2014 ICOTS

Six Guiding Principles

1. Spiral approach to learning statistical process

• Revisit throughout the book, starting with simpler data and move through a variety of more complex data situations

• Deeper and deeper look at the 6-steps as the course moves on

• Emphasizes a big picture, research-oriented view of statistical reasoning

4July 14, 2014 ICOTS

Six Guiding Principles

2. Randomization-based introduction to statistical inference

• Use simulation and randomization to first introduce statistical inference• Transition to traditional (theory-based) methods as a prediction to the

simulation/randomization results

3. Focus on logic and scope of inference

• Logic: • Significance • Estimation

• Scope: • Generalization • Causation

July 14, 2014 ICOTS 5

Six Guiding Principles

4. Integration of examples, exposition, explorations• A lot of flexibility

5. Easy-to-use technology• Freely available suite of web-applets

• Visualize simulation and randomization• Integration of simulation and theory-based approaches• Paste datasets into applets

6. Real data from genuine studies• Real, published research in most cases; some student gathered

datasets as well

July 14, 2014 ICOTS 6

Pedagogy• Built a course from the ground up that was based on

GAISE principles• Statistical literacy and thinking• Conceptual • Active• Real data• Technology to drive understanding • Assessments for continuous improvement

July 14, 2014 ICOTS 7

ICOTS 8

Increased Understanding• In an early version of the randomization-based curriculum

we showed:• Improved student understanding on key components of the logic of

inference when compared to our traditional curriculum while sacrificing little in other areas. (Tintle, et al. 2011)

• Improved retention in these same areas. (Tintle, et al. 2012)

• These results have been maintained at other institutions using more recent versions of the curriculum (Tintle 2014)

July 14, 2014

ICOTS 9

Student Attitudes Towards Statistics• With early assessments showing the randomization

curriculum doing as well as or better than the traditional curriculum conceptually, we wanted to see if there were also gains in student attitudes towards statistics.

• 725 students from Fall 2013 and Spring 2014 using Tintle et.al. Introduction to Statistical Investigations (ISI)

• 17 instructors• 12 institutions (5 universities, 5 colleges, 1 community

college, 1 high school)

July 14, 2014

ICOTS 10

Classroom settings

• Mix of • 4 credit trimester• 3 credit semester• 4 credit semester

• Always in a computer classroom • Always in a non-computer classroom• A mix of the two

July 14, 2014

ICOTS 11

Classroom settings

• The percentage of class time that was student lead discussion, group work, students working alone ranged from 15% to 65%.

• There was also quite a mix of how examples and explorations were used. • Examples: Read before class, present in class• Explorations: Do outside of class (homework), do in

class.

July 14, 2014

ICOTS 12

Instrument• SATS-36 (Student Attitudes Towards Statistics)• 36 items• 7-point Likert scale, with higher scores corresponding to

more positive attitudes• 6 attitude components to the test

• Affect• Cognitive Competence• Value• Difficulty• Interest• Effort

July 14, 2014

ICOTS 13

Administration of SATS• Pretest:

• Most were given during the first week of class, a few earlier.

• Most were given outside class.• Post:

• Most were given during the last week of class, a few later (during finals week).

• Most were given outside class.

• Most of the instructors had a small incentive for the students to complete the survey. (A few points on homework, labs, etc. or a few extra credit points.)

July 14, 2014

ICOTS 14

Reliability of Instrument• Chronbach’s alpha for each of the six components was

calculated for both pre-test and post-test administrations.• All components showed sufficient reliability (alpha > 0.70)

using same criteria as Schau (2012)

  Pre-course Post-courseAffect 0.83 0.89Cognitive 0.87 0.88Difficulty 0.86 0.90Effort 0.71 0.80Interest 0.88 0.91Value 0.82 0.80

July 14, 2014

ICOTS 15

ISI pre and post  Pre-course Post-course Change

Affect 4.13 (0.89) 4.36 (1.17) 0.23 (1.04)***

Cognitive 4.73 (0.85) 4.97 (1.00) 0.24 (0.87)***

Difficulty 3.70 (0.60) 3.97 (0.76) 0.27 (0.70)***

Effort 6.20 (0.72) 5.65 (0.98) -0.55 (0.97)***

Interest 4.73 (0.86) 4.34 (1.12) -0.39 (0.91)***

Value 5.13 (0.72) 4.92 (0.89) -0.21 (0.74)***

*p<0.05, **p<0.01, ***p<0.001

July 14, 2014

ICOTS 16

Summary ISI pre and post• Positive changes• Students enter the course with neutral attitudes on Affect

and Difficulty and a positive attitude on Cognitive Competence.

• Students leave the course with small but significant increases in these three attitude components• Affect: Less stressed/scared and tend to like/enjoy statistics

more• Difficulty: Statistics is easy/not complicated or technical more• Cognitive Competence: Understand specific concepts and in

general what Statistics is about more

July 14, 2014

ICOTS 17

Summary ISI pre and post• Negative changes• Students enter the course with positive attitudes on

Interest and Value and a very positive attitude on Effort.• Students leave the course with small but significant

decreases in these three attitude components• Interest: Less interested in learning/using statistics• Value: Usefulness/relevance in personal & professional

life less• Effort: Didn’t work as hard or spend as much time on

the class as thought they would

July 14, 2014

ICOTS 18

ISI compared to National Sample  Change (post-pre)  Randomization

(n=725)Traditional (n≈2200)

Difference1

Affect 0.23 (1.04) 0.13 (1.23) 0.10*

Cognitive 0.24 (0.87) 0.10 (1.06) 0.14***

Difficulty 0.27 (0.70) 0.15 (0.84) 0.12***

Effort -0.55 (0.97) -0.48 (1.14) -0.07

Interest -0.39 (0.91) -0.50 (1.25) 0.11*

Value -0.21 (0.74) -0.32 (0.96) 0.11**

1 P-value from independent samples t-test *p<0.05, **p<0.01, ***p<0.001

July 14, 2014

ICOTS 19

Summary ISI vs National• Positive changes• Increases in Affect, Cognitive Competence, and Difficulty

pre to post were significantly greater for students using the randomization curriculum than for students using a traditional curriculum• Affect: ISI students had a larger (p<0.05) positive increase in

how well they liked statistics and in how their stress/frustration decreased compared to National sample

• Cognitive Competence: ISI students had a larger (p<0.001) positive increase in how they perceived their intellectual skills and knowledge about Statistics compared to National sample

• Difficulty: ISI students had a larger (p<0.001) positive increase in how non-technical and easy to understand Statistics was compared to National sample

July 14, 2014

ICOTS 20

Summary ISI vs National• Negative changes• Decreases in Effort pre to post were not significantly

different for students using the randomization curriculum than for students using a traditional curriculum

• Decreases in Interest and Value pre to post were significantly different for students using the randomization curriculum than for students using a traditional curriculum• Interest: While ISI students were less interested in

learning/using/understanding statistics post course, the decrease was not as large a drop as it was for National sample (p<0.05)

• Value: While ISI students found statistics to be less relevant/useful in their personal/professional lives, the decrease was not as large as a decrease as it was for National sample (p<0.01)

July 14, 2014

ICOTS 21

Next Steps• Do weaker students do better in a randomization-based

statistics course than in a traditional statistics course?• Do certain attitudes predict conceptual understanding?• Are certain classroom settings (i.e. the amount of time

spent on student led discussion, group work, individual work) associated with conceptual understanding and attitude changes.• We have already looked at results from the author team. Our

changes (pre to post) were higher in all areas except for effort.

• Looking for participants in our assessment of concepts and attitudes, both those using theory-based curricula and randomization-based curricula.

July 14, 2014

ICOTS 22

Workshop• Title: Modifying introductory courses to use simulation

methods as the primary introduction to statistical inference• When: Saturday (08:00-17:00) • Who: Members of the ISI group and Lock5 group.

• There is still time to register

July 14, 2014

ICOTS 23

References• Cobb, G.W. (2007). The Introductory Statistics Course: a Ptolemaic Curriculum?

Technology Innovations in Statistical Education. 1(1).• Schau C and Emmioglu (2012). Do introductory statistics courses in the United

States Improve Students’ Attitudes? Statistics Education Research Journal. 11(2): 86-94.

• Schau, C. (2003). Survey of Attitudes Toward Statistics (SATS-36). [Online: http://evaluationandstatistics.com/] .

• Tintle NL (2014). Quantitative evidence for the use of simulation and randomization in the introductory statistics course. Proceedings of the International Conference on Teaching Statistics. Flagstaff, Arizona. July 2014.

• Tintle, N.L, Chance, B., Cobb, G., Rossman, A., Roy, S., Swanson, T. and VanderStoep, J. (2014). Introduction to Statistical Investigations, Preliminary edition. http://math.hope.edu/isi.

• Tintle, N., Topliff, K., VanderStoep J., Holmes V-L. & Swanson T. (2012). Retention of Statistical Concepts in a Preliminary Randomization-Based Introductory Statistics Curriculum. Statistics Education Research Journal, 11(1): 21-40.

• Tintle, N., VanderStoep J., Holmes V-L., Quisenberry B. & Swanson T. (2011). Development and Assessment of a Preliminary Randomization-Based Introductory Statistics Curriculum. Journal of Statistics Education, 19(1).

July 14, 2014

ICOTS 24

Acknowledgements• HHMI, GLCA Pathways,Teagle• NSF (DUE-1323210)• NSF (DUE-1140629)• Wiley• ISI team: Nathan Tintle, Beth Chance, George Cobb, Allan

Rossman, Soma Roy, Todd Swanson, Jill VanderStoep

• math.hope.edu/isi

July 14, 2014