Allan Rossman

Beth Chance

Overview

What do want students to know and do at the end of the course Our dream content

Top ten essentials No client disciplines

What would we cut to have time to get there Assumptions about current content in many courses Reality vs. fantasy

Are we there yet? Example assessment items

2JSM 2010

#1 Understand the statistical process of investigation Repeatedly experience the process as a

whole1. Formulate research question

2. Collect data

3. Examine the data

4. Draw inferences from the data

5. Communicate the results

3JSM 2010

#1 So what to cut?

Compartmentalizing the topics in the course Data analysis, data collection, statistical inference Instead: one categorical variable, compare two

groups on quantitative response…

Some specific techniques Example? Chi-square, ANOVA, regression Possible out of class explorations

4JSM 2010

#2 Describe how to collect relevant data to answer research question Research question vs. variable Do the data answer the question

Example: Songs about the heart “Worry questions”

>> Make sure students have an opportunity to write their own research questions and to critique measurement/data collection methods

5JSM 2010

#2 So what to cut?

Ordinal, nominal, interval, ratio scales Specifics of different sampling methods

(cluster, stratified) and experimental designs Though do make sure they realize not everything

is an SRS or CRD Acronyms!

Short-hand terminology (e.g., sampling distributions) and symbols (e.g., Ho/Ha)

6JSM 2010

#2 Assessment question

Pose a research question of interest to you that involves comparing two groups (but not one we discussed this quarter), Identify observational units, explanatory and response

variable(s), Describe a detailed plan to collect data to investigate

this question Be sure to provide a detailed enough plan that someone else

could carry out the actual data collection. Explain whether (and why) your plan will involve

random sampling and/or random assignment, or neither.

7JSM 2010

#3 Determine scope of conclusions based on data collection methods Random sampling: generalize to population Random assignment: cause/effect between

explanatory and response variables Some studies use only one, some (few) use

both, some (many) use neither

>> Get students in habit of always commenting on both of these issues whenever they summarize the conclusions of a study.

8JSM 2010

#3 So what to cut?

Nothing; this point is too important Move: Data collection issues to beginning of

course, descriptive analysis of bivariate quantitative data to end of course Students can discuss confounding variables in

context of observational studies

9JSM 2010

#3 Assessment question

Students using cursive writing on the essay portion of the SAT in 2005-06 scored significantly higher, on average, than those who used printed block letters. Can you conclude that cursive writing causes higher

scores? Explain. Different study: Identical essays were given to

graders, some with cursive writing and some with printed block letters. Those with cursive writing scored significantly higher. Can you conclude that cursive writing causes higher

scores? Explain.

10JSM 2010

#4 Appreciate value/necessity of graphing data Always start with a graph

Explain what see Example: number of letters memorized

Make sure statements/conclusions about the data follow from the graph

Sometimes the graph is enough!

11JSM 2010

#4 So what to cut?

Pie charts Choice of histogram bin width

But use technology explore different choices Normal probability plots Stemplots… Boxplots!!

12JSM 2010

#4 Assessment question

Did distribution of inter-eruption times of Old Faithful change between 1978 and 2003? If so, how? How are changes favorable for tourists? How are changes less favorable for tourists? What other interesting features are apparent, have

changed?

13JSM 2010

#5 Use proportional thinking

Especially important with categorical data, two-way tables Conditional proportions Proportion vs. percentage vs. percentage change

vs. baseline risk vs. relative risk Don’t need equal sample sizes to compare

proportions or averages Summary already takes sample size into account

to produce a “fair” comparison

14JSM 2010

#5 So what to cut?

Formal probability rules, counting rules Instead use two-way tables of counts, proportions

Bayes’ rule Simpson’s paradox

15JSM 2010

#5 Assessment question

Data from murder trial of nurse Kristen Gilbert:

Of the 74 shifts with a death, 40 (54.1%) were Gilbert shifts, not significantly more than half. Is this a reasonable calculation to perform here, to assess

the evidence against Gilbert? Explain. If not, perform a more relevant calculation and explain why it’s more relevant.

Gilbert working on shift Gilbert not working on shiftDeath occurred on shift 40 34Death did not occur on shift 217 1350

16JSM 2010

#6 Develop distributional thinking Conjecture how a variable will behave

Not everything follows a normal distribution Example: Matching variables to graphs (ala ABS)

Appreciate the nature of variability Think in terms of the distribution as an

“aggregate” Don’t let one value (data value or summary statistic)

drive a conclusion Focus on tendency, effects of outliers

17JSM 2010

#6 So what to cut?

Mode Relative frequency distributions Cumulative distributions 1.5×IQR criterion for outliers Details on calculating mean and median

Have to start making students responsible for having seen this before

18JSM 2010

#6 Assessment question

Which would have more variability: ages of customers at McDonald’s near freeway or ages of customers at snack bar on campus? Explain.

19JSM 2010

#6 Assessment question

Are pamphlets containing information for cancer patients written at an appropriate level that cancer patients can understand?

Analyze these data to address the research question. Summarize and explain your conclusions.

20JSM 2010

#7 Consider variability in data when making comparisons Comparing a particular outcome to a constant Comparing outcomes in two different groups Standardization can be a special case

Using a measure of variability to produce “ruler” for which we judge distances

Standard deviation (z-score) Box lengths…

21JSM 2010

#7 So what to cut?

Calculation of standard deviation by hand Short-cut calculation formulas (SD,

correlation) ANOVA table calculations Linear transformations on summary statistics

22JSM 2010

#7 Assessment question

Sketch a graph of data from 1950-1960 where the change observed between 1955 and 1956 would be considered noteworthy.

Now sketch a graph where the change observed between 1955 and 1956 would not be considered noteworthy.

Traffic Deaths

year

23JSM 2010

#8 Consider variation of statistics when making comparisons Averages vary less than individual values

Less and less with larger and larger samples Larger samples give more precise estimates Precision must be considered when making

conclusions Example: Three coin flips is not enough to decide

whether a coin is fair

24JSM 2010

#8 So what to cut

Rules for means and variances /n

Central Limit Theorem Instead use simulations, graphs

25JSM 2010

#8 Assessment question

In a rodeo roping contest, a contestant’s score is the average of two times. Explain why it is more fair to use this combination of two scores instead of relying only on one score.

26JSM 2010

#9 Understand the logic of inference When can “chance” be eliminated as a plausible

explanation? Consider chance variability due to random sampling or

random assignment Strength of evidence vs. proof

Cobb (2007) argued that the reasoning process of statistical significance can best be introduced via simulation of randomization tests rather than normal-based models “What if” distribution

27JSM 2010

#9 So what to cut?

Rejection region approaches Tables of probability distributions

Randomization approach does not require probability distributions

Even with traditional tests, technology can calculate p-values, critical values

But still focus on well-labeled sketches of “what if” distributions

Technical conditions 20-100% of specific (parametric) procedures

28JSM 2010

#9 Assessment question

MythBusters: Is yawning contagious?

Was MythBusters justified in concluding that the data provide strong evidence that yawning is contagious? Conduct your own analysis Explain reasoning process behind your conclusion

Yawn seed planted Yawn seed not planted TotalSubject yawned 10 4 14Subject did not yawn 24 12 36Total 34 16 50

10/34 29% 4/16 25%

29JSM 2010

#10 Consider margin of error Importance of interval estimate not only a

point estimate More than simply assessing statistical significance Estimate + 2 SE

Focus on idea of interval of plausible values Understand what parameter is being estimated

Issues that do/do not affect margin of error Random sampling Sample size Population size

30JSM 2010

#10 So what to cut?

Solving algebraically for sample size Any level other than 95% confidence

Any multiplier other than 2! Interpretation of “confidence level”

31JSM 2010

#10 Assessment question

Suppose you want to estimate the proportion of the over 305,000,000 Americans who prefer cats to dogs within a 3% margin-of-error. Approximately what sample size would you need with a random sample?

10 1,000 100,000 1,000,000 10,000,000

32JSM 2010

#1 Assessment question What type of study was

this? Advantages and disadvantages?

What graph could you examine to summarize these data?

What is meant by “a 16 percent decreased risk of death”?

What does it mean for the average life expectancy to be “significantly” longer?

Is this an appropriate headline? Explain.

JSM 2010 33

Conclusions

Fun to start from ground zero What is your bare minimum of essential content?

Make sure “stat methods” courses don’t prevent “stat literacy”

Take advantage of computer/calculator power Emphasize interpretation over calculation

Assess what you value

34JSM 2010

Questions?

Allan Rossman arossman@calpoly.edu Beth Chance bchance@calpoly.edu

http://www.rossmanchance.com/jsm2010.ppt

JSM 2010 35