a database for storing and retrieving classified and benchmarked exam questions for ict courses

A Database for Storing and Retrieving Classified and

Benchmarked Exam Questions for ICT Courses

Repository of Wisdom ALTA Forum May 9th, 2014

James Harland james.harland@rmit.edu.au

Joint work with Margaret Hamilton, Daryl D’Souza,

Raymond Lister (UTS), Ali Reza (UTS), Donna Teague (QUT)

A Database for Storing and Retrieving Classified and

Benchmarked Exam Questions for ICT Courses

Repository of Wisdom ALTA Forum May 9th, 2014

James Harland james.harland@rmit.edu.au

Joint work with Margaret Hamilton, Daryl D’Souza,

Raymond Lister (UTS), Ali Reza (UTS), Donna Teague (QUT)

Repository of Wisdom

But First …

Repository of Wisdom ALTA Forum May 9th, 2014

What?

Repository of Wisdom

Database of exam questions for introductory programming

Hosted on NECTAR Research Cloud (!!) Classification based on BABELnot project

framework for questions Syllabus specification based on ProGoSs Developing front-end with comments and ratings

features Ready by 30th June

ALTA Forum May 9th, 2014

Question Attributes

Repository of Wisdom ALTA Forum May 9th, 2014

Programming Language

Topic

Style Course level

Marks Skill Required

Open/closed Difficulty

Explicitness Linguistic Complexity

Conceptual Complexity

Intellectual Complexity

Code Length External domain references

Question Attributes

Repository of Wisdom ALTA Forum May 9th, 2014

Topic Up to three from a given listStyle multiple choice, short answer, program code, Parsons

problem, graphical representationSkill Required

knowledge recall, trace code, explain code, write code, modify code, debug code, design program, test program

Open/closed Closed if exactly one answer. Open otherwiseIntellectual knowledge, comprehension, application, analysis,

evaluation, synthesis (Bloom levels)Conceptual Low, medium, high (depending on rules about

concepts)Code Length Low, medium, high External Low, medium, high Linguistic Low, medium, high Explicitness High, medium, lowDifficulty Low, medium, high

Question Attributes

Repository of Wisdom ALTA Forum May 9th, 2014

Question itself is in PDF or similar

Answers and Marking Guide (optional)

Performance data (optional) Often anonymised Variable level of detail Can have several per question

Comments and ratings (1-5 stars)

Front page

Repository of Wisdom

Content …

ALTA Forum May 9th , 2014

Heading

Repository of Wisdom

Content …

ALTA Forum May 9th , 2014

Heading

Repository of Wisdom

Content …

ALTA Forum May 9th , 2014

Heading

Repository of Wisdom

Content …

ALTA Forum May 9th , 2014

Heading

Repository of Wisdom

Content …

ALTA Forum May 9th , 2014

Conclusion

Repository of Wisdom

Try it here Comments and Ratings are relatively new Will also be adding keyword search (soon!) Due for completion by 30th June Aim is to enable, not prescribe “Build it (IT?) and they will come … “ Comments and feedback most welcome

ALTA Forum May 9th , 2014

QUESTIONS?