Dr. Cath ELLIS

University of Huddersfield, United Kingdom

Using Grading Analytics to Improve Student Learning

Presented by:

Using Grademark Analytics to Improve Student Learning

Dr Cath Ellis

University of Huddersfield

Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimising, learning and the environment in which it occurs.

(LAK cited in Ferguson)

Implicit within this definition are the assumptions that learning analytics make use of pre-existing, machine-readable data, that its techniques can be used to handle large data sets of data that would not be practicable to deal with manually. (Ferguson)

Business intelligence

Learning Analytics

Assessment analytics

Social Network Analytics

Discourse analytics

Nudge analytics

Academic Analytics

Content Analytics

Educational Data Mining

Accountability

Transparency

Improvement

Improvement

Informed decision making

Reduced attrition

Good degree results

Operationalisation

Staff resistance

Online learning only

Impenetrable

Depth

• Granularity

Breadth

• Large data sets

Assessment Analytics

Degree attainment

Progression

Module

Individual assessment

Learning outcome

Errors/strengths

Persistence Improvement

Assessment analytics: Blind spot

Analytics instead of assessment

• Constructivist learning theory

Data reliability

• Inter- and intra-rater reliability

• Validity of assessment design

Data availability

• Paper-based marking

• Not enough granularity

Electronic

Assessment

Management

E

A

M

Institution

Evaluation Retention Recruitment

Academic staff

Expertise Adds value Informs practice

Students

Meaningful Investment Guides behaviour

Assessment analytics

Marking

• Final mark

• Assessment criteria

• Common comments/errors

Feedback

• Final mark

• Assessment criteria

• Map Learning Outcomes

Originality checking

• Unoriginal text sources

Automatic marking

• MCQ

• Short answer/free text

Certification

• Discriminate between levels of achievement and students

• Selection for further study and employment

• License to practice

Student Learning

• Motivating students, steering their approach

• Inform teaching strategies

Quality Assurance

• Evidence for stakeholders

• Judge standards

Lifelong Learning

• Encourage skills development

• Self-evaluation and self-regulation

Why assess? Assessment of

learning Assessment for

learning

Bloxham and Boyd 2007

Motivating Students

Skills development

Discriminate between levels

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Criterion 5

Criterion 4

Criterion 3

Criterion 2

Criterion 1

% Students

Ru

bri

c C

rite

ria

ICCT Summative One Rubric Results 2010-11

Fail

3rd

2.2

2.1

1st

Inform teaching strategies

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Criterion 1 09-10

Criterion 1 10-11

Criterion 2 09-10

Criterion 2 10-11

Criterion 3 09-10

Criterion 3 10-11

Criterion 4 09-10

Criterion 4 10-11

Criterion 5 09-10

Criterion 5 10-11

% Students

Ru

bri

c C

rite

ria

2009-10/2010-11 Rubric Result Comparison

1st

2.1

2.2

3rd

fail

Motivating students

License to practice

Evidence for stakeholders

Judge standards

Self-evaluation

Granularity

References

• Bloxham, Sue, and Pete Boyd. 2007. Developing Assessment in Higher Education: A Practical Guide. 1st ed. Open University Press.

• Ferguson, R. 2012. The State of Learning Analytics in 2012: A Review and Future Challenges. Technical Report. UK: Knowledge Media Institute, The Open University.