managing data-driven learning projects
TRANSCRIPT
![Page 1: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/1.jpg)
Managing Data-Driven Learning Projects
Aaron E. Silvers & Megan Torrance,
Learning Solutions 2015 Orlando, FLUSE THE TWITTERS!!!!!
@aaronesilvers @MMTorrance #LScon
![Page 2: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/2.jpg)
“ - Robert DeNiro, RONIN (1998)
“I never walk into a place I don't know how to walk out of.”
#KeepingTheEndInMind
![Page 3: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/3.jpg)
Data-Driven Learning Projects
What doThere are a couple of ways data, like xAPI data, opens new doors for Learning & Development projects.
look like?
![Page 4: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/4.jpg)
Adaptive Learning
There’s a lot of promise for
adaptive learning in the
enterprise. What exactly
does “Adaptive Learning”
mean? Well…
![Page 5: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/5.jpg)
Adaptive Learning works like the story of Goldilocks and the Three Bears
Photo by SlyOwl - http://flic.kr/p/j9fFYy
![Page 6: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/6.jpg)
First, Goldilocks (“online
self-paced eLearning”)
can choose her porridge…
Photo by SlyOwl - http://flic.kr/p/j9fFYy
![Page 7: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/7.jpg)
Online, Self-Paced Learning• Pre-testing preselects content
based on a pre-tests or learner’s history to “test out” of content. This porridge is too hot (arbitrary).
• Linear eLearning delivers the same content to every user. This porridge is too cold (not all that adaptive).
• Micro-adaptive adjusts the sequence of content based on a learner’s reactions in real-time.This porridge is just right.
Photo by SlyOwl - http://flic.kr/p/j9fFYy
![Page 8: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/8.jpg)
\
Next, Goldilocks has to
choose among the Micro-
Adaptive systems…
Photo by SlyOwl - http://flic.kr/p/j9fFYy
![Page 9: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/9.jpg)
Micro-Adaptive Systems• Preference-based systems adapt to
the style or medium preferred by each learnerThis chair is too big (it’s imaginary).
• Rule-based systems are pre-programmed with closed loops of “if… then…” logic. This chair is too small (“rules”).
• Algorithmically sequenced systems calculate the sequence of content dynamically.This chair is just right.
Photo by Viracocha - http://bit.ly/1A6fcdM
![Page 10: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/10.jpg)
\
Next, Goldilocks has to
choose among different
types of Micro-adaptive,
algorithmically
sequenced systems…
Photo by SlyOwl - http://flic.kr/p/j9fFYy
![Page 11: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/11.jpg)
Algorithmically-Sequenced Systems
• Test-driven systems are based on automated test sequencing. This bed is too hard (so testy).
• Memorization systems prioritizes repetitions based on data about the individual, the learning activity and the interaction between the two. This bed is too soft (narrow applications).
• Proficiency-driven systems look broadly at what someone is trying to learn, pulling from a wide range of content. This chair is just right.
Photo by BlueStarMedia - http://bit.ly/1y0IEVZ
![Page 12: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/12.jpg)
Adaptive Learning
Indeed, lots of promise for
adaptive learning but the
way forward has some
bumps.
Challenges
![Page 13: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/13.jpg)
You probably don’t have enough content.
Photo by Tim Ebbs - http://flic.kr/p/aiBFtv
![Page 14: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/14.jpg)
Photo by Brian Corredor - http://flic.kr/p/dLW2aT
The LMS is a fortress for content.
![Page 15: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/15.jpg)
IA
Nav
LLR&A Au T Re
Inv Feed Look G
Sta Weed Find M
Use Need DFor more on Content Strategy:
http://bit.ly/contentWrangling
You need a content strategy.
![Page 16: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/16.jpg)
Photo by woodleywonderworks - http://flic.kr/p/4Lx3mh
Defining competencies
while the nature of work
changes is hard.
![Page 17: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/17.jpg)
Photo by woodleywonderworks - http://flic.kr/p/4Lx3mh
Adaptive Learning
relies on tracking lots of
learning activity data.
This means way more
interaction design and
development when
creating content.
![Page 18: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/18.jpg)
Learning Analytics
The Big Data approach
…i.e. looking for a needle in
haystacks without knowing
what the needle looks like.
![Page 19: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/19.jpg)
Photo by woodleywonderworks - http://flic.kr/p/4Lx3mh
Remember all that
tracking we’re doing?
Where adaptive systems
keep the data mostly out
of view, Analytics are
meant to show you what
the data means.
![Page 20: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/20.jpg)
WHO TAKES WHAT LEARNING PATH? WHEN? WHAT IS THEIR CAREER, AGE, ETC.? WHAT PSYCHOMETRIC TESTING INFORMATION IS AVAILABLE? ITEM ANALYSIS & PRE/POST TESTING? HOW MANY TIMES DID IT TAKE TO “PASS”? WHAT PERFORMANCE DATA DO WE HAVE? WHAT COURSES AND LEARNING PATHS DO THE BEST PERFORMERS TAKE? WHAT KINDS OF COURSES ARE PEOPLE ASKING FOR? WHAT FEEDBACK ARE WE GETTING FROM COURSE EVALUATIONS ACROSS MULTIPLE CURRICULA, TOPICS, INDUSTRY BENCHMARKING, ETC.?
There are many
questions we can ask
right now. Things we
“should” be doing
already, even if we don’t
see immediate benefits.
![Page 21: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/21.jpg)
There’s the stuff we’re already doing…
• We’re collecting data with our LMS databases
• Our CRMs and ERPs and ESNs are capturing a lot of information about employee activity
• HRIS systems and Talent Management have uniform (kinda standardized) ways of describing people.
![Page 22: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/22.jpg)
Making Sense of It All…
• Sentiment Analysis - What do the words people use tell us about their disposition to learn?
• Engagement Analysis - What’s the activity level with learning content?
• Cohort Analysis - Who forms what groups for what reasons?
• Keyword Analysis - How do people seek info & what do they find?
• Conversion Rate - How many people respond (i.e. comment)?
• Amplification Rate - How many times is something shared?
• Applause Rate - How many likes/favorites/bookmarks?
• Economic Value - Short/Long Term Revenue/Cost Savings?
![Page 23: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/23.jpg)
Getting Started While Already in Motion
Matching structured and
unstructured data so you
can actually do something
with it in the real world.
![Page 24: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/24.jpg)
How Our Data Is Structured
STRUCTURED tabular, orderly
UNSTRUCTURED not tabular not tagged
wild wild west
![Page 25: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/25.jpg)
How Our Data Is Structured
STRUCTURED tabular, orderly
UNSTRUCTURED not tabular not tagged
wild wild west
Data-type
metatagging #LSCon
#omg1Distotsawesome
![Page 26: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/26.jpg)
How Our Data Is Structured
STRUCTURED tabular, orderly
UNSTRUCTURED not tabular not tagged
wild wild west
LMS
![Page 27: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/27.jpg)
It Would Be So Simple, Except…
![Page 28: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/28.jpg)
Learning Informatics
The Little Data approach
…i.e. known needles in
small haystacks you make.
![Page 29: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/29.jpg)
Speaking of small haystacks
![Page 30: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/30.jpg)
Speaking of small haystacks
![Page 31: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/31.jpg)
Speaking of small haystacks
…getting bigger
…and less structured
![Page 32: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/32.jpg)
Activity streams are
interesting … for a while.
![Page 33: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/33.jpg)
And data visualizations are
pretty … for a while.
Phase 2 will involve far
more complex interactions
and data requirements to
define.
![Page 34: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/34.jpg)
The RISC - VTA PDF Annotator
app developed by Float
Mobile Learning allows an
employee to use PDF job aids
like they read/highlight/
annotate on Amazon Kindle.
![Page 35: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/35.jpg)
In this case, there are only
four activities that matter:
highlighting, underlining or
noting a word or passage and
leaving a note for the page.
![Page 36: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/36.jpg)
The data collected has
a known pattern to it,
so specific reports can
be created that add
distinct value back to
the organization about
how they’re used.
![Page 37: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/37.jpg)
…and that data can represented
in different ways for different
uses. In this data visualization,
content owners and designers
can see exactly which pages in a
PDF have the most annotations,
and what kind — helpful if they
need to know where to make
edits.
![Page 38: Managing Data-Driven Learning Projects](https://reader033.vdocuments.site/reader033/viewer/2022042817/55a8c0b51a28abec318b47ce/html5/thumbnails/38.jpg)
Data-Driven Learning Projects
So…How do we manage that? This is where y’all ask the tough questions and we share what works :)