enhancing the first year experience a

‘Enhancing the First Year Experience – A Case Study From Biomedical Sciences’

Paul HaganStephen McClean

University of Ulster

Evaluation• There are many, many variables to be

considered when evaluating student performance.

• What works for one individual may not offer a solution for another.

• How do we assess the efficacy of all the ‘fixes’ we offer and how do they impact on each of the many experience groups which we have to accommodate.

• We need to use some medium-weight statistics to get at the detail of the ‘fixes’ versus student performance with prior educational experience factored in.

EVALUATION

• This evaluation step is CRUCIAL in the whole exercise and it is actually the detail of the EVALUATION which I am communicating, and what it can offer.

• We all need to justify what we do in terms of committing additional resources to remedial classes.

• These robust analyses give us the ammunition we require to promote and justify these activities.

Normal Distribution fits are for the two groups of students

1. Those with previous ‘A’ level experience

2. Those without ‘A’ level experience

2005-2006 - 62% 82%

2006-2007 – 68% 81%

2007-2008 65% 84%

EX_Mark_2

Frequency

105907560453015

14

12

10

8

6

4

2

0

Mean StDev N62.37 23.23 7681.79 14.38 59

A_OR_AS_201

Histogram of EX_Mark_2Normal

BOXPLOTS

Outlier an unusually large or smallobservation. Values beyond thewhiskers are outliers.

By default, the top of the box is thethird quartile (Q3). 75% of the datavalues are less than or equal to thisvalue.

By default, the upperwhisker extends to thisadjacent value thehighest data valuewithin the upper limit. Upper limit = Q3 + 1.5(Q3 - Q1)

Median the middle of thedata. Half of theobservations are less thanor equal to it.

By default, the bottom of thebox is the first quartile (Q1)25% of the data values areless than or equal to thisvalue.

By default, the lower whiskerextends to this adjacent value thelowest value within the lowerlimit.Lower limit = Q1- 1.5 (Q3 - Q1)

Mean

CA_Mark

EX_M

ark

9080706050403020100

100

80

60

40

20

0

Scatterplot of EX_Mark vs CA_Mark

Good correlation between coursework and examination mark. Three students with low coursework and > pass marks in the examination obviously defied all attempts to improve their coursework submissions.

CA_Mark

EX_M

ark

90807060504030

100

80

60

40

20

0

Scatterplot of EX_Mark vs CA_Mark

2005-2006

2006-2007

Final_CA

Exam

_To

tal

90807060504030

100

80

60

40

20

0

Scatterplot of Exam_Total vs Final_CA

Possibility of producing a ‘tailor made’ report for individual student.

Position in cohort can be highlighted from registration number.

ABSENCES

EX_M

ark

50403020100

100

80

60

40

20

0

Scatterplot of EX_Mark vs ABSENCES

The expected correlation between attendance and examination score, increased absence

from lectures, practicals and tutorials leads to depressed examination marks. Outliers suggest that some poor attendees have

sufficient prior ‘knowledge’ or alternative access to course materials, to enable a pass.

2005-2006 2006-2007

ABSENCES

EX_M

ark

403020100

100

90

80

70

60

50

40

30

20

10

Scatterplot of EX_Mark vs ABSENCES

NUMBER OF EXTRA TUTORIALS ATTENDED

EX_M

ark

543210

100

80

60

40

20

0

Boxplot of EX_Mark by EXTRA-TUTS

STUDENTS HAVING STUDIED TO 'A' ORTHE EQUIVALENT OF 'AS' CHEMISTRY

In all 6 extra ‘voluntary’ tutorials were offered in semester one. All students attending at least one

extra tutorial passed the exam.

2005-2006


EX_M

ark

210

100

90

80

70

60

50

Boxplot of EX_Mark vs EXTRA_TUTS

STUDENTS HAVING STUDIED TO 'A' OR THEEQUIVALENT OF 'AS' LEVEL CHEMISTRY

2006-2007

A total of two examination revision sessions were offered immediately prior to the examination after the Semester one examinations.


EX_M

ark

_1

6543210

100

80

60

40

20

0

Boxplot of EX_Mark_1 by EXTRA-TUTS_1

STUDENTS WITH NO 'A' LEVEL CHEMISTRY

10 RECORDED ABSENCES

Students with no ‘A’ level chemistry experience (‘A’ or ‘AS’ level) or with no chemistry at Irish Higher or Irish Ordinary level improve their probability of passing by attending at least one extra ‘voluntary tutorial’

It can be shown that the students who failed to achieve at least a pass grade were not sufficiently motivated to attend extra tutorial classes and this lack of motivation would be reflected in their level and quality of examination preparation

2005-2006 2006-2007


EX_M

ark

_1

6543210

100

90

80

70

60

50

40

30

20

10

Boxplot of EX_Mark_1 vs EXTRA_TUTS_1

STUDENTS WTH NO 'A' LEVEL CHEMISTRY

BOTHEX_1

EX_M

ark

_1

10

100

80

60

40

20

0

Boxplot of EX_Mark_1 by BOTHEX_1

MEAN = 74

MEAN = 53

STUDENTS WITH NO PREVIOUS 'A' LEVELCHEMISTRY AT ANY LEVEL

Attendance at both pre-examination revision tutorials improved the mean examination mark of students without chemistry ‘A’ levels by 21% on average. None of these students gained lower than a pass grade.

BOTHEX_1

EX_M

ark

_1

10

100

90

80

70

60

50

40

30

20

10

Boxplot of EX_Mark_1 vs BOTHEX_1

MEAN = 64MEAN = 71

STUDENTS WITH NO PREVIOUS 'A'LEVEL CHEMISTRY AT ANY LEVEL

BOTHEX_1

Exam

_To

tal_

1

10

100

90

80

70

60

50

40

30

20

10

Boxplot of Exam_Total_1 vs BOTHEX_1

MEAN = 60

MEAN = 76

STUDENTS WITH NO PREVIOUS 'A'LEVEL CHEMISTRY AT ANY LEVEL

Picture is broadly similar in 2007 - 2008

BOTHEX

EX_M

ark

10

100

90

80

70

60

50

40

30

Boxplot of EX_Mark by BOTHEX

STUDENTS WITH PREVIOUS 'A' LEVELCHEMISTRY

MEAN = 81 MEAN = 83

36 RECORDED ABSENCES

Students with previous ‘A’ level experience generally do not exhibit a significantly improved examination performance by attending both pre-examination revision tutorials. It is however not

detrimental to their performance and may be viewed as a ‘confidence-building’ exercise.

2005-2006 2006-2007

BOTHEX

EX_M

ark

10

100

90

80

70

60

50

Boxplot of EX_Mark vs BOTHEX

MEAN = 81

MEAN = 84

STUDENTS WITH PREVIOUS'A' LEVEL CHEMISTRY

Multivariate analysis 2006-2007

First Component

Seco

nd C

om

ponent

0.50.40.30.20.10.0-0.1-0.2-0.3

0.50

0.25

0.00

-0.25

-0.50

EX_Mark

HOME_EC_A

PHYSICS

BIOLOGY

SCIENCE_A

VCE_SCIENCE

MATHS

CHEM_ASCHEM_A

ABSENCES

Loading Plot of ABSENCES, ..., EX_Mark

A_OR_AS

EX_M

ark

10

100

80

60

40

20

0

Boxplot of EX_Mark by A_OR_AS

STUDENT WITH 36 ABSENCES

MEAN = 82

MEAN = 62

Students with ‘A’, ‘AS’, or equivalent level chemistry, on average, score least 15% higher in the examination.

A_OR_AS

EX_M

ark

10

100

90

80

70

60

50

40

30

20

10

Boxplot of EX_Mark vs A_OR_AS

MEAN = 67

MEAN = 81

2005-2006 2006-2007

WIFT

A_OR_AS

EXA

M/100

10

90

80

70

60

50

40

30

Boxplot of EXAM/ 100 vs A_OR_AS

MEAN = 56

MEAN = 66

WHAT’S IN IT FOR THEM?

2005 - 2006

Examination marks for second semester BIOCHEMISTRY module

A_OR_AS

Exam

_To

tal

10

100

90

80

70

60

50

40

30

20

10

Boxplot of Exam_Total vs A_OR_AS

MEAN = 63

MEAN = 84

2007 - 2008

BIOCHEMISTRY

EXAM MARK

NO ’A’ LEVELS

%PRIOR ‘A’ LEVELS %

2005/6 56 662006/7 57 612007/8 63 84

Conclusions

• Attendance is a crucial factor• Specific groups can be

identified as being deficient in chemistry and mathematics

• Targeted tutorials are very effective

• Practice MCQ’s appreciated by students

• Some ‘A’ levels better than others.

Pass Rate /%

2003/4 642004/5 732005/6 912006/7 962007/8 91