using sna to provide feedback on course discussion (aera 2015 presentation)

Using Social Network Analytics To Provide Feedback On Course Discussion: Learner Preferences and Reactions Vanessa Dennen Florida State University [email protected]

April 2015 AERA • Chicago, IL 1

What happens in an online class?


•  1 response •  1 reply

Students Discuss

• Points for posts

• Content loosely considered

Instructor Grades


Students Discuss

What happens in an online class?



Students Discuss

• Points for posts

• Content loosely considered

Instructor Grades


Students Discuss

How can we get students more engaged? Do students realize that they are supposed to be more engaged? Do students realize that they are not engaged?

Potential Solution • Use social network analysis and providing formative

feedback. • THIS STUDY (survey):

•  Student reactions to and preferences for 3 different types of discussion participation feedback

Formative – Instructor

Formative – Student

Summative – Instructor

Summative – Student


Research Questions 1.  Which type of feedback do students prefer on their

online discussion performance? 2.  When SNA-based feedback is given, do student prefer it

to be ego- or whole-network focused? 3.  To what extent to students find sociograms of

discussion interactions useful as a form of feedback? 4.  Will students use SNA-based feedback to guide their

actions in future class discussion?


Discussion Feedback • DF1. Generic, brief text • DF2. Personal position in class network with comparative

information and whole class sociogram (relative benchmark)

• DF3. Personal network information only and ego network sociogram (course benchmark)


The Survey • Section 1: Demographics/Perspectives on Online

Learning • Section 2: Reaction to DF1 • Section 3: Reaction to DF2 • Section 4: Reaction to DF3 • Section 5: Comparing DF2 and DF3


Participants • Enrolled in online, undergraduate communications course • N=59 (12 male, 47 female) • First online course = 10 students • Main reasons for taking course

•  Convenience (56%) •  Required (44%)


Reaction to Feedback DF1 DF2 DF3

Too Brief 15 (25%) 1 (2%) 6 (10%)

Just Right 43 (73%) 11 (19%) 21 (36%)

More than needed 1 (2%) 47 (80%) 32 (54%)

Satisfactory, but I don’t need feedback

12 (20%) 1 (2%) 4 (7%)

Satisfactory, exactly as wanted

34 (58%) 12 (20%) 19 (32%)

Would like more substantive feedback

13 (22%) N/A N/A

Would like less substantive feedback

N/A 33 (56%) 26 (44%)


Actions Based on Feedback DF1 DF2 DF3

Ignore 6 (10%) 33 (56%) 20 (34%)

Read with interest

19 (32%) 19 (32%) 28 (47%) Read and change behavior 34 (58%) 7 (12%) 11 (19%)

Prefer DF2 Prefer DF3 No Preference 15% 63% 22%

Comparing DF2 & DF3


Reaction to Sociograms • Ego network was preferred over whole class • Despite inclusion of directions, most claimed they could

not understand the sociograms • Time stamps on surveys suggest they spent very little

time trying to understand the sociograms

Mean Time to Read (pilot)

Mean Time on Page (survey)

DF1 0:10 min 0:11 min

DF2 1:47 min 0:48 min

DF3 1:39 min 0:36 min


Most and Least Useful Parts of Sociograms

DF2 DF3 Most Useful Nothing (n = 23)

Comparison to classmates (n = 19) Use of visual information (n = 5)

Nothing (n = 16) Can see own participation (n = 11) Simpler than DF2 (n = 9)

Least Useful Confusing (n=23) Too much information (n = 10) Comparison to classmates (n = 8)

Confusing (n = 28) Nothing (n = 9) Not enough information to improve performance (n = 5)


Trends in Comments • Sociograms confused students (didn’t want to think that

hard) • Mixed sentiments about whether or not they should be

concerned with how classmates performed

A Culture of No Interaction “In my opinion, most students (including myself) are most interested in how he or she can do well in the class. I don’t even check to see if anyone has responded to my posts. We are not concerned about others. It is an online class, and requires little to no interaction with others, and should stay that way.”


Discussion / Conclusions • Feedback preferences will vary widely • Feedback needs to be

•  Easy to digest •  Immediately relevant to the student

• Allowing choice may be helpful •  Provide whole class information in a public area of course •  Provide brief personal feedback individually

• Data and visual literacy skills are necessary before analytic data can be used for formative feedback


Thank you! Slides can be found at:

http://slideshare.net/vanessadennen

Contact: [email protected] @vdennen


using sna to provide feedback on course discussion (aera 2015 presentation)

Education