learning analytics – opportunities for iso/iec jtc 1/sc36 standardisation
TRANSCRIPT
![Page 1: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/1.jpg)
Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36
standardisation Tore Hoel
Oslo and Akershus University College of Applied Sciences Norway
ISO/IEC JTC 1/SC36 WG8 meeting, 29 November 2015Hangzhou, China
![Page 2: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/2.jpg)
The Learning Analytics Landscape
What standards are needed?
![Page 3: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/3.jpg)
Characteristics of Educational Big Data
• Grain size of recordable and analysable data has become smaller– every pen stroke, every keystroke is recorded
• Sources of evidence are (more) varied– tests, essay scoring, learning games, social interactions, affects,
body sensors, intelligent tutors, simulations, semantic mapping, LMS data…
– Unstructured (e.g., . log files, clicks, timestamps)– When structured different schemas are used
• How do we bring these data together to form a overall view of an individual learner or a cohort of learners?
(Cope, B., & Kalantzis, M., 2015)
![Page 4: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/4.jpg)
What data practices are emerging?
• Multi-scalar Data Collection– Embedded, simultaneous collection of data that can be used
for different purposes at different scales– Semantically legible datapoint (learner-actionable feedback):
«teachable moment»• Self-describing, structured data meanings immediately evident ➔
to learners, teachers, others• Sample size n= all• Data and interventions are not separate: Recursive micro
intervention result redesign cycles ➔ ➔• More widely distributed data collection roles
(Cope, B., & Kalantzis, M., 2015)
![Page 5: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/5.jpg)
Need for new Education Data Standards supporting Learning Analytics• Harmonization of Activity Stream Specifications (ADL xAPI, IMS
Caliper, W3C Activity Streams)• Building Vocabularies – Profiles – Recipes – Communities of
Practice• Storage designs – centralised data warehouses or distributed
Learning Record/Event Stores• Extract, Transform and Load (ETL) tools for data storage
• Privacy and Data Protection – how to do Privacy-by-design in this field?
• Sharing of Algorithms and Predictive Models
![Page 6: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/6.jpg)
Harmonization of Activity Streams
![Page 7: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/7.jpg)
Activity Streams• Work started around 2009 by a group from IBM, Google,
Microsoft, MySpace, Facebook, VMware a.o., • First version published in 2011 • 2014 W3C Social Web Working Group took over the specification • Working draft version 2.0 published October 2015
In its simplest form, an activity consists of an actor, a verb, an an object, and a target. It tells the story of a person performing an action on or with an object -- "Geraldine posted a photo to her album" or "John shared a video". In most cases these components will be explicit, but they may also be implied. (Activity Streams Working Group, 2011)
![Page 8: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/8.jpg)
Experience API (xAPI)
• 1st version 2013 (component of ADL Training and Learning Architecture)
• A Statement consists of an <actor (learner)>, a <verb>, an <object>, with a <result>, in a <context>. There is no constraint on what these objects should be.
• Learning Record Store: a system that stores learning information• xAPI is dependent on the presence of LRS to function
• Offered for standardisation in IEEE August 2014 – “it wasn’t the slam dunk [they were] naively hoping it would be” (Silvers, 2014)
• End of 2015 a new Data Interoperability Standards Consortium (not-for-profit organization in the State of Pennsylvania, USA) to be the steward of Experience API
![Page 9: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/9.jpg)
IMS Caliper Analytics
• White paper 2013• Public release v 1.0 October 2015• Information model buried in Sensor APIs• Metric Profiles• Base Metric Profile, Session, Annotation, Assignable,
Assessment, Outcome, Reading, Media• IMS Learning Sensor API: defines basic learning events gathered
as learning metics across learning environments• Leveraging of IMS LTI/LIS/QTI
![Page 10: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/10.jpg)
IMS Caliper
![Page 11: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/11.jpg)
Source: Yong-Sang Cho
![Page 12: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/12.jpg)
Vocabularies
![Page 13: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/13.jpg)
Talking about learning activities• Looser coupled systems, diverse Communities of Practice lead to
more diverse schemas and data models• Interoperability could be promoted by more efficient sharing of
vocabularies• Encourage smaller vocabularies / ontologies
IMS Caliper
xAPI Communities
![Page 14: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/14.jpg)
How to promote more interoperable vocabularies for education?• "Document standards" for vocabularies have severe limitations!• Communities of Practice (ref xAPI) are part of the solution…• … but serious stewardship issues• What could ISO offer in terms of dynamic vocabulary
management?
![Page 15: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/15.jpg)
Storage
![Page 16: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/16.jpg)
Apereo Dimond model
![Page 17: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/17.jpg)
Search architecture middle layer
(Hoel & Chen, 2015)
![Page 18: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/18.jpg)
MIT Open Personal Data Store / Safe Answers• openPDS allows users to
collect, store, and give fine-grained access to their data in the cloud.
• openPDS also protects users’ privacy by only sharing anonymous answers, not raw data.
• openPDS can also engage in privacy-preserving group computations to aggregate data across users without the need to share sensitive data with an intermediate entity.
http://openpds.media.mit.edu/#architecture
![Page 19: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/19.jpg)
Extract - Transform - Load tools
• When data are coming from different sources in different structures, one need tools to extract, transform and load data into data stores
• There are Open Source ( e.g., Pentaho Kettle and Talend), but most are commercial software
• Are ETL tools a possible hot spot for standardsefforts?
![Page 20: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/20.jpg)
SC36 20748-1 Data Storing & Processing
![Page 21: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/21.jpg)
Challenges for standardisation
• Privacy and Data Ownership issues – how to turn these «soft» requirements into «hard» ones?
• The role of Personal Data Stores in Learning Analytics• Harmonization of data schemes prior to analysis• Import / export facilities with ontology building (and automatic
reasoning technologies) as part of the storage solutions • Publishing and Sharing of data for research and comparison and
testing of predictive models, student models, etc.
![Page 22: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/22.jpg)
Privacy
![Page 23: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/23.jpg)
Implications for designs when Surveillance turns into Sousveillance?
Image credit: http://commons.wikimedia.org/wiki/File:SurSousVeillanceByStephanieMannAge6.png
![Page 24: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/24.jpg)
When Privacy is affecting all LA processes
• Privacy-By-Design is the overall design principle. What does it mean for the LA processes?• Data Sharing• Search• Storing• Analysing• Visualising
![Page 25: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/25.jpg)
Sharing Algorithms & Predictive Models
![Page 26: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/26.jpg)
How to support sharing?
• Exemplar predictive models are needed to advance learning analytics
• Besides a Culture for sharing data, algorithms and predictive models, what else is needed?• Parallel data streams from production systems to support
development and research• How to deal with anonymization?
• How to get data for R&D from cloud-based systems?• How do we talk about these algorithms and models (create a
vocabulary for tagging)• Where to host the resources (stewardship, openness policies,
open repositories)
![Page 27: Learning Analytics – Opportunities for ISO/IEC JTC 1/SC36 standardisation](https://reader036.vdocuments.site/reader036/viewer/2022062821/588767df1a28ab22358b4d1f/html5/thumbnails/27.jpg)
References
• Cho, Yong-Sang (2015) Quick review xAPI and IMS Caliper - Principle of both data capturing technologies. Online at http://www.slideshare.net/zzosang/quick-review-xapi-and-ims-caliper-principle-of-both-data-capturing-technologies
• Cope, B., & Kalantzis, M. (2015). Sources of Evidence-of-Learning: Learning and assessment in the era of big data. Open Review of Educational Research, 2(1)
• Hoel, T. & Chen, W. (2015). Privacy in Learning Analytics – Implications for System Architecture. In Watanabe, T. and Seta, K. (Eds.) Proceedings of the 11th International Conference on Knowledge Management. Online at http://hoel.nu/publications/Hoel_Chen_ICKM15_final_preprint.pdf