YouEDU

Staging Intelligent Interventionsin MOOC Discussion ForumsAkshay AgrawalJagadish VenkatramanShane LeonardAndreas Paepcke

Agrawal 2

Agrawal 3

Agrawal 4

An Intelligent Forum

Agrawal 5

An Intelligent Forum

• Phase 1: Detect Confusion

Agrawal 6

An Intelligent Forum

• Phase 1: Detect Confusion• Phase 2: Intervene Automatically

Agrawal 7

Challenges

Agrawal 8

Challenges

• Scale:

Agrawal 9

Challenges

• Scale:

• Vocabulary of confusion is domain-dependent

Agrawal 10

Challenges

• Scale:

• Vocabulary of confusion is domain-dependent

• TA overloaded

Agrawal 11

Challenges

• Scale:

• Vocabulary of confusion is domain-dependent

• TA overloaded– lack of forum structure

Agrawal 12

• No tagged forum datasets• Availability of large-scale forum data

Agrawal 13

Why are forums still dumb?

YouEDU

• Pre-work: Dataset– Stanford MOOCPosts

• Phase 1: Confusion Detection– Classify individual posts w.r.t. confusion

• Phase 2: Intervention–Map confused posts to video snippets

Agrawal 14

MOOCPosts Dataset

• 30,000 posts,11 courses– Humanities/Sciences, Medicine,

Education

• Human-coded, 6 dimensions– confusion, sentiment, urgency [1-7]– opinion, question, answer [0/1]

Agrawal 15

http://datastage.stanford.edu/StanfordMoocPosts/

Agrawal 16

Phase 1: Detecting Confusion

Agrawal 17

Agrawal 18

Agrawal 19

Agrawal 20

Classifier Performance

Agrawal 21

Recap: Phase 1

Phase 2: Staging Interventions

Agrawal 22

Agrawal 23

Agrawal 24

Agrawal 25

Agrawal 26

Agrawal 27

Agrawal 28

Agrawal 29

Agrawal 30

Interventions: Evaluation

• 20 confused posts, course in statistics

• Six recommendations for each– randomize order

• 3 human raters– Label each snippet as relevant or

irrelevant

• k-precision– precision of retrieved documents,

limited to the first k recommendationsAgrawal 31

Agrawal 32

YouEDU Architecture

Agrawal 33

Classification Suitehttp://github.com/akshayka/edxclassify

MOOCPosts Datasethttp://datastage.stanford.edu/StanfordMoocPosts/

Contactakshayka@cs.stanford.edu

Agrawal 34

Agrawal 35

Agrawal 36

Agrawal 37

Agrawal 38