Patrick D. KilgoEmory University

Department of Biostatistics and Bioinformatics

Southern Regional Council on StatisticsJune 6th, 2011

Integrating A Problem-Based Learning Approach Into Large Sections of

Graduate-Level Introductory Biostatistics Courses

Core Course SettingGraduate level biostatistics courses with associated lab

components

All incoming Master’s degree candidates in public health are required to take BIOS 500:Descriptive statisticsProbabilityCommon hypothesis tests

5 large sections – approximately 85 students per section

Two classes per week – 80 minutes per class

Follow-up regression course (BIOS 501) is optionalLinear regression / ANOVALogistic regressionSurvival analysis

What’s The Problem? RETENTION

It is common for our students to have forgotten almost everything in the intervening month between Fall and Spring semesters

Thesis season: By their second year, the average student has:Forgotten most of the statistical concepts they once

“knew”Has forgotten how to apply concepts and statistical

tests and also the programming necessary to accomplish their analysis

Resorted to roaming the halls of the third floor, beckoning any statistical-looking person for help

Problem-Based Learning (Duch, 2001)

We learn and retain when solving a problem ourselves

“Complex, real world problems are used to motivate students to identify and research the concepts and principles they need to know to work through those problems”

Small learning teams are used to collectively acquire, communicate and integrate information

“Instructor is no longer the sage on the stage but rather is the guide on the side.”

Problem-Based Learning Objectives (Duch, 2001)

Think critically and be able to analyze and solve real-world problems

Find, evaluate and use appropriate learning resources

Work cooperatively in teams and small groups

Demonstrate versatile and effective communication skills, both verbal and written

Use content knowledge and skills acquired at the university to become continual learners.

Previous PBL Biostatistics CoursesCarolyn Boyle, Mississippi State, Journal

of Statistics Education v.7, n.1 (1999)

Applied in an animal science setting

18 veterinarian students

8 cases over two semesters

The only published account of PBL in biostatistics

Goals – Excellence In These Areas …  Generating descriptive statistics

  Choosing the appropriate analysis approach when faced

with a research problem

Interpreting findings from research studies

Writing reports and communicating results of research findings following a statistical analysis

  Thinking through analytical problems and subsequently

designing studies

Working in groups to solve research problems

Beginning the statistical thinking/planning for your Master’s thesis

  Discussing statistical analysis with other faculty,

students and employers

Extra Resources Required for PBLDepartmental Support - $$$$$$$$

3 additional experienced “co-instructors”Though some disagree, I still believe that this

task is beyond the capabilities of the average TA

3 additional classrooms

Patience and flexibility on the part of the lab instructors

A ton of my time

General Framework of My PBL Class

No more tests or homework

No required textbook: I asked them to find any statistical text for reference

4 “cases” (problems) over the course of the semester

Mondays: Lecture (Taught by Kilgo)

Wednesdays: 4 PBL “breakout” sections of size 20 where cases are worked on in groups of 4-5. (Taught by Kilgo and 3 co-instructors)

Teach them deeper, not wider

Deeper, Not WiderHalf as many lectures = must be efficient

Before I presented a topic I asked myself three questions:How likely are the students to encounter this topic in

practice?How likely is the average student to remember this topic

in three weeks?Will they be taught this topic in their introductory

epidemiology course?

Sample of topics omitted:Many probability axioms and concepts (~1 lecture)Bayes rule (~1 lecture)Binomial and Poisson distributions (~2-3 lectures)Nonparametric tests (~1-2 lectures)Several statistical tests – McNemar’s Test, ANOVA (~2

lectures)

First ImplementationFall 2009, a class of 72 first-year, first semester Global

Health studentsNon-majors

PBL co-instructors: Lisa Elon – fellow faculty-level colleagueLaura Ward – staff senior biostatisticianJeff Switchenko – 5th year doctoral student

Open-ended, real-life, interesting problems in public health and medicine

Individual deliverables, even though group work was encouragedData analysis report with emphasis on methods, results,

conclusions, limitations

Cases

Case 1 – Designing a study to determine whether data collected from Automatic Crash Notifiers in cars can be used to determine the need for Level I trauma careNo data in this caseThought experiment

Case 2 – Were players accused in the Mitchell Report of taking steroids better offensive performers?Students had to make a descriptive case one

way or the other.Outliers, multiple observations per player, skew,

etc.

CasesCase 3 – Validation of an experimental

testing device designed to diagnose pre-Alzheimer’s diseaseData management, t-tests, assumption

violations, experimental design issuesReal-life problem – collaboration between

Emory and GA Tech

Case 4 – Smallpox Vaccine TrialChance to compare modern methods to

Jenner’s methodStudents had to read Jenner’s original paperChi-square tests, odds ratios, Interactions

(non-homogeneity)

Final ProjectStudents proposed a personalized final project in the

middle of the semester

Could be anything from a research interest to a personal interest:What is the effect of maternal iron supplements on neo-natal

iron levels?Do women think mustaches are more sexy when they are

ovulating?

Students asked for specific variables, guessed at their distribution and hypothesized about group differences.

Instructors generated datasets for them so that they were studying something that is interesting to them in a context they are familiar with.

First Semester PBL Evaluation … How comfortable are you with the following …?

Question Non-PBL ClassN=74

4.4 / 5.0

PBL ClassN=57

4.6 / 5.0

p-value

Generating descriptive stats 4.12 (0.84) 4.16 (0.65) 0.78

Choosing the right analysis 3.64 (0.82) 4.00 (0.79) 0.012

Interpreting your findings 3.88 (0.64) 4.04 (0.69) 0.18

Writing/Communicating results 3.58 (0.72) 4.00 (0.89) 0.003

Thinking through problems / study design 3.27 (0.90) 3.98 (0.74) <0.001

Working in groups to solve problems 3.82 (0.84) 4.23 (0.87) 0.008

Beginning the planning for your thesis 2.97 (1.02) 3.60 (0.96) <0.001

Discussing statistics with other faculty 3.53 (0.74) 3.74 (0.92) 0.15

Feedback From Students

If I could go back in time to the beginning of the semester with a choice of class formats I would …

1)Choose the lecture-only format (4/57)2)Choose the problem-based format (48/57)3)Be indifferent towards the format (5/57)

First Semester Growing Pains

Timing of cases / lectures / labs

Should have taken a TA when one was offered

Workload distribution – most of the assignments came due later in the semester

Different approaches from different co-instructors

First Semester Pleasant Surprises

Students liked SAS

Very positive course feedback

Students having an easier time working with faculty on projects

Many requests for a third course offering

Was as much a class in research writing and organization as it was biostatistics – their scientific writing greatly improved over the semester

Second Implementation – BIOS 501 Spring 2010 Only three cases – no final project

Case 1 – Predicting traffic deaths using 1964 NHTSA-type dataLinear regression, transformation, skew, outliers, missing

data, validation, confounding.

Case 2 – The evaluation of off-pump CABG compared to on-pump CABG with respect to major adverse outcomesLogistic regression, lots of covariates, confounding, fitting of

associative models, graphics, independent risk factor identification, interactions

Case 3a - Survival Analysis –The Role of Race and Race Mismatch in Determining Survival in Pediatric Heart Transplant Patients

Case 3b – The Effect of ICU LOS on Long-Term SurvivalKM curves, Cox proportional hazards regression, confounding,

etc.

Conclusions - Feedback From Students

Very positive in general

Complaints include:Workload distributionTime-consumingLearning material/working on cases concurrently“I got an 800 on the math GRE and I’m struggling in

your class … I felt like I would have done better in the traditional section”

BIOS 500Fall 2009: 4.6/5.0Fall 2010 4.2/5.0

BIOS 501Spring 2010 4.7/5.0Spring 2011 4.7/5.0

Likert Scale Question: I learned a lot in this course …

THANK YOU!