![Page 1: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/1.jpg)
Open Standards and Specifications for Outcome-
based Assessment
Israel GutiérrezUniversidad Carlos III de Madrid
Madrid, April 12th 2010
![Page 3: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/3.jpg)
Computer-based assessmentReduce effort and costs: permit continuous assessmentAssessment re-use: interoperability needed
Variety of assessment formatsProprietary: MoodleXML, HotPotatoes, Blackboard Ad hoc: OpenMark IMS QTI
Open Standards and Specifications for Assessment
![Page 4: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/4.jpg)
Outcome-based Assessment Universities developing an
outcome-based learning strategy Adaptation to European
Space for Higher Education Improve course structure for
accreditation (ABET)
Creation of new coursesLearning outcomesAssessment as a key process
![Page 5: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/5.jpg)
Outline
Analysis of assessment standards and specifications
Usage of assessment formats in Europe
Interoperability issues
Next generation assessment scenarios
Best practices
![Page 6: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/6.jpg)
Methodology
Analysis of Assessment Standards and Specifications
Metamodels (UML diagrams) from MDAUnderstand better studied
formatsAbstraction from syntax
(XML binding)Focus on content
![Page 7: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/7.jpg)
Studied Assessment Formats
IMS QTI
OpenMarkSuML
FML
General purposeAssessment purpose
QAML
![Page 8: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/8.jpg)
Qualitative ComparisonKey features
Multiple response allowanceCorrect response indicationMetadata capabilitiesHybrid question management
Question types (from FREMA)Constructed: essay, short answer, FIB, etc.Constrained: MCQ, MRQ, TF, matching, etc.Miscellaneous: simulations, practical
sessions
![Page 10: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/10.jpg)
Assessment Formats Usage in Europethe current picture – methodology and sample description
• Data collection• Personal interview• In-depth report
• Qualitative analysis• Sample characteristics
• Higher education• Multicultural• Multilingual
Series10
2
4
6
8
10
12
14
Assessment Scenarios
Evaluation Scenarios
Tools & Services
Pilots
Sample size
Project results: WP6
![Page 12: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/12.jpg)
Assessment Formats Study: Resultsthe current picture – key conclusions
Yes22%
No78%
Re-use of assessement resources
• Scarce re-use of assessment resources• Scarce acceptance of IMS QTI specification• High technological diversity
– LMS, ad-hoc tools, etc.• Diversity of assessment methodologies
– formative/summative, various types of activities, face-to-face/blended/elearning settings, etc.
![Page 13: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/13.jpg)
Interoperability Issues: IMS QTIScarce use of the specification
Incomplete support in LMS i.e. adaptive material
Flexibility handicaps
QTI profiles
Versions instability: 1.2.1 or 2.1?Versions are not backwards compatible
![Page 14: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/14.jpg)
Assessment Interoperability
Moodle
.LRN Clix
QTIexport QTIMoodle
QTIlrn QTIclix
QTI 1.2.1
QTI 1.2.1
QTI 1.2.1
![Page 15: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/15.jpg)
Next Generation Assessment Scenarios • Hints
• Adaptation• Instructional
design
![Page 16: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/16.jpg)
Assessment Resources Interoperability ProposalAssessment resources
![Page 17: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/17.jpg)
Objective: Gather and develop best practices about interoperable content as support to higher education and learning outcome oriented
Results• ICOPER Reference Model• Gathering of recommendations• Open ICOPER Content Space• One of the most successful eContent+ projects since 2004
![Page 18: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/18.jpg)
Open ICOPER Content Space• Federated repository of educational resources
![Page 19: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/19.jpg)
Outcome-based Assessment AppUse cases
Search for assessment resources in a repository Filter by intended learning outcomes Filter by assessment methods
Import/re-use of assessment resourcesPublish/share assessment resources
Online demo: http://mozart.gast.it.uc3m.es:9000/
![Page 20: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/20.jpg)
Outcome-based Assessment AppUse cases (cont.)
Annotate assessment resources Peer review Teacher reflection Student feedback
Update learner’s profile
StakeholdersFaculty membersCourse developers
Online demo: http://mozart.gast.it.uc3m.es:9000/
![Page 21: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/21.jpg)
Outcome-based Assessment App
Learning outcome-based search
Learning outcome profile update
Implementation for dotLRN based on
ICOPER LOM profile
![Page 22: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/22.jpg)
Outcome-based Assessment App: EvaluationEngineering evaluation
ImplementersTool developers
End user evaluationFaculty membersCourse developers
Epistemological evaluation ResearchersStandardization bodies
![Page 23: 2011 04 12 (uned) emadrid igutierrez uc3m estandares especificaciones evaluacion orientada competencias](https://reader036.vdocuments.site/reader036/viewer/2022062511/54b94fed4a79593d0f8b4570/html5/thumbnails/23.jpg)
ICOPER Assessment appAssessment resources should include information
about Intended learning outcomesRecommended assessment methodsAnnotations about resources
Usage of integrated environment for outcome-based assessment