Uncovering Algorithms: Looking Inside the

Facebook News Feed

Motahhare Eslami, Kevin Hamilton, Karrie Karahalios, Cedric Langbort, Aimee Rickman, Christian Sandvig

“The preferential display of our flights,

and the corresponding increasein our market share, is the

competitiveraison d’être for having createdthe [SABRE] system in the first

place.”

Robert Crandall, CEOAmerican Airlines

SOURCE: ReadWrite

1. How can research on algorithms proceed without access to the algorithm?

Then, initial results from a study about the Facebook news feed, investigating:

2. What is the algorithm doing for a particular person?

3. How should we usefully visualize it?4. How do people make sense of the

algorithm?

Questions

Social Science Audits

8

Our proposal: The Algorithm Audit

11

Five illustrative research designs

Get the Code

Ask the Users

Scrape Everything

Sock Puppets

Collaborative Audit

Our Audit

How aware are users of the role of algorithms and filters in their news sources?

Kevin Lynch, “Wayfinding”

• Interviewing/surveying users

• Exposing hidden algorithmic processes with social media prompts using the API and studying the effects of knowledge on use

• Working with users to try and deduce algorithmic processes and the design rationale behind them

Our Approach

FeedVis

Aware 37.5%

Unaware/Uncertain 62.5%

Levels of Awareness

Paths to Awareness

“but some people have...I mean I have 250 friends but some people have thousand... [and if it wants to show everything], there is no way, it couldn't... it crashes your computer [laughing].” (P24)

“… Plus like the phone newsfeed is different from the laptop newsfeed sometimes like I look at something on my phone but then when I go online, there's a ton more. So, it's also “where” you look.” (P26)

“I read this a while ago, a year or two ago. That, all these keys of things that you have to do to get into people news feed.” (P21)

Unaware“I bet it would be on my news feed. I probably would catch [it] at some point during the day.”  (P30)

“... and probably I don't scroll down enough but you know [you] cannot always scroll down so much, you just go like a few posts and that’s it.” (P9)

“I don't know for sure [...] because I know there are some friends I guess, it seems like I don't see anything by them very often. I don't know if they really use Facebook that much or […]" (P36)

“My wife also uses Facebook and she has occasionally asked me if I saw a post by her friend and I will not have seen it and I don't know why I didn't see it.” (P36)

Exploring Awareness Factors

Membership Duration

Seen Content/Total Content

Friendship Network Size

Usage Frequency

Activity Level (Poster, Light Poster, Listener)

Facebook Page/Group Management

Newsfeed Parameter Manipulation (top/recent stories, block people, hide posts)

Reactions to FeedVis

“So do they actually hide these things for [from] me? [mentioning to the unseen stories] Heeeeeeeeeeeeeeeey!” (P1)

“It's kind of intense because... have you seen the movie "The Matrix"? It's kind of waking up in the Matrix in a way ...I mean you have what you think as your reality of like what they choose to show you.”(P19)

What the hell Facebook? [after scrolling Content View] Well, I’m super frustrated… (P3)

Initial Surprise

(In seen/not seen content-centric view)

“So, there is some algorithm or something or some rules to choose these white things that would not appear to me? [....]It’s very interesting. I never know that FB really hides something! So they are doing some data mining or machine learning thing?” (P1)

“For now, I cannot really understand how they categorize these people.  Actually this is my brother [pointing to sometimes seen] and actually he needs to be here [pointing to almost seen]. “(P1)

(In people-centric view)

“I think she needs support for that [mentioning to a not shown story]; if I see it, then I will say something [to support her].” (P8)

(In content-centric view)

Folk Theories

After Follow-up

After Learning about the Algorithm, People Changed their Behavior

Reading Behavior

Changing interaction behavior

Using the most recent feature

Changing news feed settings (ex. hiding posts)

Posting Behavior

Relationship Changes

Unfriending people

“After our discussion, I actually went back and started experimenting a little with the newsfeed and discussing with some friends on ways to streamline what I was receiving. Since then, I’ve become more interested in checking my Facebook because it does not seem as cluttered with random information I have no interest in.” (P10)

Conclusion

A Question, an Admonishment, and a Call to Arms

What do you users need to know?Transparency alone is not enough!Create infrastructure for algorithm

auditing!