emoocs2017 who wants to chat on a mooc v1.2

Who wants to chat on a MOOC?

Lessons from a peer recommender system

François Bouchet, Hugues Labarthe, Rémi Bachelet & Kalina Yacef

Download this slideshow : https://goo.gl/JVoNYnMay 23rd, 2017, Madrid, Spain Read the paper online : https://hal.archives-ouvertes.fr/hal-01522736

Previous papers @eMOOCs

• eMOOCs 2015 – Does peer grading work? Comparing instructor and peer assessment

– Do MOOC students come back for more? Recurring Students

• eMOOCs 2016– Increasing MOOC completion : social interactions

• eMOOCs 2017 (tomorrow 12:00)– Who wants to chat on a MOOC? THIS PAPER

– Typology of MOOC activity patterns. Learners who never rest?

• ongoing (QPES 2017 – french)

– Partnering with teachers running your MOOC : paper and workshop

2

Peer Recommendation in a MOOC

• Technology and tools:– Recommender systems (RS) help in e-learning to choose classes to follow, resources to read… (Manouselis et al. 2011, Klašnja-Milićević, Ivanović & Nanopoulos 2015)

– Peer recommender systems (PRS) help on social networks, datingwebsites…

• Problem:– Attrition plagues MOOC, but students interacting with each other have better odds to get certified (Yang, Wen & Rose, 2014) and collaboration helps (Ferschke et al. 2015)

Solution ? A PRS for MOOCs3

Can a PRS help students in a MOOC?

• Yes! (Labarthe et al. 2016) on GdP6– Increased performance

– Dropout rate reduced and happens later (if at all)

4

PRS widget

in GdP6

Chat widget

in GdP6

Can a PRS help students in a MOOC?

• Yes! (Labarthe et al. 2016) on GdP6

• But not many use it… Why?– Who to contact?

• Choosing can be overwhelming

• Solution: contact a group and not a single person

5

Can a PRS help students in a MOOC?

• Yes! (Labarthe et al. 2016) on GdP6

• But not many use it… Why?– Who to contact?

– Why are these persons suggested to me?

• Hidden algorithm parameters for experimental reasons: random vs. sociodemography vs. progress – small benefit for socio (Bouchet et al. 2017)

• Solution: show the recommendation strategy and let the students choose

6

A redesigned PRS on GdP8 & 9

7

Minimized by default,

click to expand

(bottom right-hand corner)

A redesigned PRS on GdP8 & 9

8

Blinks + sound

when a message is received

A redesigned PRS on GdP8 & 9

9

A click expands the PRS & chat

2-in-1 module

Show the list of threads one has created

or been invited to

A redesigned PRS on GdP8 & 9

10

A click expands the PRS & chat

2-in-1 module

Show the list of threads one has created

or been invited to

Message creator Date of creation

Number of

messages

in the thread

First message in the thread

A redesigned PRS on GdP8 & 9

11

A click on a thread opens it,

revealing the messages

in antichronological order

One can read messages

and contribute

A redesigned PRS on GdP8 & 9

12

Or add someone as a favorite,

in the third tab

A redesigned PRS on GdP8 & 9

13

Where one can contact again

someone already known, individually

A redesigned PRS on GdP8 & 9

14

The second and central tab allows

to create a new discussion thread

with a group of 20 persons, chosen:

• Randomly

• Based on sociodemographics similarities

(4 questions asked the first time)

• Based on progress in the MOOC

(as shown by quizzes)

A redesigned PRS on GdP8 & 9

15

When clicking on either button,

one chooses the topic of the conversation

(in a non restrictive list of hashtags)

and the initial message to send

Who wants to chat and benefits from the PRS?

Can a PRS help students in a MOOC?

• Yes! (Labarthe et al. 2016) on GdP6

• But not many use it… Why?– Who to contact?

– Why are these persons suggested to me?

– Not everyone wants to chat…

16

?

Research questions

1. How much did students use the provided PRS in GdP8?

2. What were the differences between students using and not using the PRS?

3. Why did some people not want to use the PRS?

17

Methodology: students sample

• PRS deployed on the specialization modules (week 4-7)

• N = 6,170 students whologged in at least once on this separate platform

18

• PRS log data, 3 levels of involvement:

– opening/closing of the PRS

– opening of a discussion thread in the PRS

– sending of a message in a thread

• Research questionnaires, 7 variables considered (Likert scales):

– GoalBetterThanOthers

– FearOfBeingJudged

– WillToShareFeelings

– WillToInteractToLearn

– PerceivedSocializationUsefulness

– PerformedSocialization

– LackContactWithOthers

Methodology: data collected & treatment

19

For each variable:

• Aggregation (when

multiple questions)

• Dichotomization

(removing

intermediate value)

Multiple Pearson’s

Chi-squared tests

H0 : « the use of the

PRS is independent

from each variable »

Result 1: use of the PRS

• 3025 (49.03%) students opened the PRS

• 570 (9.24%) opened a thread

• 206 (3.34%) sent a message

A minority of users:– comparable to typical use of forums in MOOCs

– lower than in platforms encouraging conversational learning (e.g. FutureLearn)

Sample large enough for inferential statistical analyses

20

Result 2: PRS users vs. non-users

• Main motivation: sharing feelings, more than learning with others

• Fear of being judged: important factor for not using a PRS

• Used by students who were already using other socialization tools

• Also used by those lacking contact with others (sometimes the same ones)

21

Opened

the chat

Opened a

thread

Sent a

message

GoalBetterThanOthers **

FearOfBeingJudged * **

WillToShareFeelings *** *** ***

WillToInteractToLearn *

PerceivedSocialization

Usefulness*** *** ***

PerformedSocialization *** *** ***

LackContactWithOthers *** *** ***

* p<.05, ** p<.01, *** p<.001

Result 3: reasons of the PRS non-users

1631 students who did not use it:– 718 have not seen it

– 406 do not remember whether they had seen it or no

– Others: (7 possible reasons)

• mostly a lack of question to ask

• preference for forums (to reach more? Used to the tool?)

• positive result: more ready to contact others in the future

22

Discussion

• ¼ of PRS non-users had no relevant question + PRS users want to share feelings PRS is used to reassure oneself and to assess progress of others

• Hypothesis consistent with the fact participants who feared judgment of others opened threads as much as those who did not fear judgment, but sent significantly less messages– lurking behavior ?

23

Conclusion• A peer-recommender system:

• can help MOOC students• is used by specific type of students• fills a gap (no competition with available means of communication)• seems to be used indirectly to compare oneself to others should a dedicated tool be developed for that?

• Limits & perspectives:• Only during the second half of the MOOC: previous attrition, students most

interested might have dropped off before GdP9 experiment (from day 1)• {{todo}} Final grade not correlated• Content of discussions not analyzed here• Tool currently embedded in MOOC towards an open source widget?• EdRecSys2017@UMAP2017 Bratislava next July : Comparing peer recommendation

strategies

• Perspective : implementation on a larger scale done now24

Thanks for listening!

• Twitter : @R_Bachelet, Googleplus : +Rémi Bachelet

Read the paper online : https://hal.archives-ouvertes.fr/hal-01522736