Brexit through the Lens of Data Science

Kenneth Benoit (joint work with Akitaka Matsuo)

Department of MethodologyLSE

Text mining and data/social science

• Treats text as “data” that informs us about the authors of the text - the text itself is only incidental

• Key is some form of comparison, to gain insights about differences

• Involves the application of statistical models to judge differences using probability statements

Analyzing Brexit through Twitter

• EU-funded project

• We collected some 26 million Tweets from January - July 2016

• capture based on #hashtags, @usernames, and search terms

Hashtags: #betterdealforbritain #betteroffout #brexit #euref #eureferendum #eusummit #getoutnow #leaveeu #no2eu #notoeu #strongerin #ukineu #voteleave #wewantout #yes2eu #yestoeu brexit

Usernames:

@vote_leave @brexitwatch @eureferendum @ukandeu @notoeu @leavehq @ukineu @leaveeuofficial @ukleave_eu @strongerin @yesforeurope @grassroots_out @stronger_in

Search terms

Users in data

• 3.6M unique users

• number of tweets:

• average: 7.2

• median: 1 (more than 50% had only one tweet)

• max: 81.1K

First application: Predicting Leave v. Remain users

• Method: Naive Bayes classifier

• Data source: combined tweet corpus at user level

• Creating training data

• Select “power-users” (more than 100 tweets in the corpus, 15K users)

• Check the use of pre-determined set of “leave” and “remain” hashtags

• Calculate the difference in the use of leave and remain hashtags. Construct training data from top and bottom 10% of power users

Predictive accuracy based on feature types

No Hashtags Mentions

Hashtags Mentions

Mentions

Hashtags

All Features

0% 20% 40% 60% 80% 100%Accuracy

Distribution Bernoulli multinomial

• Use bag of words method (unigrams of #hashtags and @user_mentions, remove features used less than 20 users)

our version

Predicting Leave v. Remain

N %

Remain 9,780,223 36.93%

Neutral 7,786,297 29.40%

Leave 8,914,207 33.66%

Patterns of posts, Leave v. Remain

1e+04

1e+06

Jan Feb Mar Apr May Jun Juldate

N

side Remain Neutral Leave

our version

MP side predictions by party

Conservative

Democratic Unionist Party

Green Party

Independent

Labour

Liberal Democrat

Plaid Cymru

Scottish National Party

Sinn Fein

Social Democratic and Labour Party

UK Independence Party

Ulster Unionist Party

0 50 100 150 200 250

sideRemain

Neutral

Leave

MP predictions from Tweets

A few well-known Twitter users

@kenbenoit

@lseabra@lseanhubbard

@lsebrexitvote

@lseechr1

@lseekford

@lseenterprise

@lsefton

@lsegreenbelt

@lsehistory

@lsei

@lseideas

@lsekaras

@lselangcentre@lselawreview@lselectric_dm

@lself1202

@lsellier@lsenerazzurra

@lsengers

@lsengler@lsenior15@lsergy

@lserio9762

@lserofthyear@lseror@lsesalter

@lsesu

@lsethy@lsetlse

@lseupr

@sarahobolt

@simonjhix

10

1000

0.0 0.4 0.8 1.2Probability of Remain

Num

ber o

f Twe

ets

sideaaa

Remain

Neutral

Leave

Leave and Remain Hashtags

Leave

Remain

Neutral

#leaveeu#brexit#euref

#forex

#ukineu #eu

#eureferendum

#referendum

#remain

#bbc

#davidcameron

#labourinforbritain

#farage#trading

#rt

#uk

#sky

#voteremain

#news

#fintech

#nexit

#money

#leave

#tories

#bbcqt

#euco

#itv

#bbcdebate

#fx#ukip

#worldnews

#gove

#merkel

#grassrootsout

#british

#itveuref

#gbp #brexitthemovie

#labour

#eusummit

#gold #frexit

#marr

#out

#nhs

#euistheproblem

#registertovote

#cameron#business

#realestate

#forexnews

#germany

#ids#tcot

#strongerout

#lbc

#europe

#pound

#european

#ttip

#usa

#greece

#dodgydave

#freedom

#

#juncker

#voteukip

#brexit's

#beleave

#commonwealth

#forextrading

#georgeosborne

#ft

#incampaign

#bbcdp

#grexit

#tech

#eupol

#ivoted

#r4today

#fishingforleave

#eurefresults#bregret

#gogogo

#newsnight

#finance

#notmyvote

#uktostay

#fed#investment

#labourin

#euarmy

#ukpolitics

#gibraltar

#boe

#putin

#tomorrowspaperstoday

#savethenhs

#investing

#live

#immigration

#39

#loveeuropeleaveeu

#brexitrisks

#sovereignty

#yes2eu

#teamfollowback

#panamapapers

#votetoleave

#trump2016

#strongertogether

#turnup

#bbcsp#regrexit

#bbcpapers

#love

#auspol

#world

#migrantcrisis

#c4debate

#politics

#bluehand

#dexit

#press

#jobs

#bbcnews#startup

#gbpusd

#cornwall

#obama#sterling

#scotland

#swexit

#london

#stoptheeu

#fail

#ukref

#britin

#believeinbritain

#voteleave's

#stock

#jeremycorbyn

#nwo

#euro2016

#union

#whitegenocide

#brexiters

#syria

#remainineu

#bojo

#follow

#turkey

#economy

#bnp

#voteleavetakecontrol

#smallbusiness

#betterin

#trump

#saferin

#skynews

#birmingham

#remainin

#racism

#property

#vote

#brussels

#dontwalkaway

#stocks

#corbyn#europeday

#usd

#bbcr4today #health

#8217

#pmqs

#lsebrexitvote#peace

#german

#voteleaveeu

#isis

#peston

#euro

#specbrexit

#takecontrolday#feelthebern

#eudebate

#wewantourfishback

#forbes

#tory

#stopthewar

#pjnet#wewantout

#europeanunion

#jobsearch#breakingnews

#topstories

#irexit

#twibbon

#marrshow

#migrants

#rmt

#ge2015

#greatbritain

#reuters#indyref2

#snp

#england

#remainers

#flexcit

#skypapers

Followership network analysis, followees’ positions(Barberà 2015)

guardian

ft

thesunbbc5live

bbcr4today

bbclaurak

bbcnewsnightbbcquestiontime uklabour

timeshighered

chukaumunna

prosyn

faisalislam

guardianopinionchathamhouse

adamboultonskyspectator

conversationukpa

juliahb1

lsepoliticsblog

cbitweets

marrshow

cityam

albertonardelli

edvaizey

jamesdelingpole

jdportes

b_judah

brandonlewis

rcorbettmep

freeman_george

jamescleverlymrrbourne

ruthleaecon

martin_durkin

number10gov

independent

peston

standardnews

campbellclaret

kayburley

metrouk

gdnpolitics

carolinelucas

houseofcommonsnataliebendavidallengreenlordashcroft

rhonddabryantconhome

telepolitics

shippersunboundjolyonmaugham

ukipnfkn

telegraph

telegraphnewsafp

nigel_farage

euronews

europarl_en

guidofawkesprisonplanet

frasernelson

ukip

thescotsman

montie

paulwaugh

leaveeuofficial

danhannanmep

quigleyp

johnrentouldpjhodgesmrharrycole

toadmeister

tonyparsonsuk

douglascarswell

davidjo52951945

openeuropeiainmartin1

samcoatestimestnewtondunn

michaelpdeacon

end_of_europe

timothy_stanley

pimpmytweeting

oflynnmep

holbornlolz

breitbartlondon

suzanneevans1

dvatw

colrichardkemp

raheemkassam

jongauntuk__news

2tweetaboutit

wallaceme

richardcalhoun

davidcoburnukip

capx

redhotsquirrel

fight4uk

ironwand

liarpoliticians

rogerhelmermep

bluehand007

a_liberty_rebel

bernerlap

bbcpropaganda

wantenglandback

ajcdeanemarcherlord1

cllrbsilvester

y_euroscepticsmargotljparker

euroguido

stardust193

mikkil

theredrag

heatstreet

drteckkhong

theordinaryman2

ukleave_eu

brugesgroup

dianejamesmep

eureferendum

politicssense

uniforbritainfarmers4britain

andrew_lilicoandywigmore

lizbilney

petenorth303

skynews

irishtimesdwnews

jamesmelville

scientists4eu

euromovehealthierin

cer_grant

financialtimesskynewsbreak

theopaphitis

huffpostuk

sputnikint

suttonnicklouisemensch

bbcnormans

bv

theipaper

stellacreasy

politicshome

reutersuk

harryslaststand

strongerin

bbckamal

michaelwhite

the_iod

fullfactbonn1egreer

fastft

hugodixon

benpbradshaw

sturdyalexwdjstraw

nadinedorriesmp chrisshipitvnsoamesmp

georgemagnus1

sundersays

katehoeymp

timloughton

bernardjenkin robdothutton

theresecoffeydrgerardlyons

drphillipleemp

ukandeulsebrexitvote

bloombergtvmarkets

britainelects

rtuknews

msmithsonpbjimwaterson

marycreaghmp

stephenkb

uklabourin

mrtcharris

bbcnews

bbcbusiness

cnbc

bbcbreakfast

edconwaysky

victorialive

chrisgiles_

bbcrealitycheckstevebakerhw

simplysimontfa

thescepticisle

whitewednesday

thegreenpartylabourpress

vote_leave

peoplesnhs

socialistvoice

paulnuttallukipandrealeadsom

stewart4pboro

labourleave

mkpdavies

xxplwxx

brexitthemovie

cheekylatte

beeahoney_

k69atie

racheljoyce

bristolcomsense

womenforbritain

brexitnoww

2053pam

stronger_ln

agapanthus49

roundlike

akabilky

newsweekeuractiv

politicoeurope

sophyridgesky

huffpostukpol

simon_nixonipsosmori

mike_fabricant

liamfoxmp

brexitwatch

nadhimzahawiemmareynoldsmp

goodwinmjanna_soubry

greghands

cer_london

nickherbertmp

samgyimah mrjohnnicolson

catherinemep

strongerinpress

alynsmithmepjohnharris1969

timesredbox

kawczynskimp

lucycthomas

davidgauke

david_cameron

britinfluence

conservativesinrachael_swindon

leftfootfwd

infactsorg

reuters

peterjukes

lbc

theeconomist

wef

businessinsiderpollstationuk

betteroffout

johnredwood

consforbritain

fortunemagazine

itvnews

indyvoicesopendemocracy

telebusinessscotnational

ibtimesuk

thomasbrakeelectoralcommuk

bitetheballot

skymurnaghanmollymep

another_europestudents4europeremainineu

env4eurweareeuropeuk

mrrae1000

anandmenon1

polnyypesets

red13charlie

angelneptustartrevdick

angiemeader

thetimes

sj_powell

dcbmep

veteransbritain

john_mills_jml

owen_patersonmp

nilegardiner

leavehq

thomasevansukip

gjon777

hbaldwinmp

ben4ipswichdamiancollins

mpritchardmp

sciencebritain

brexpats

truemagic68

australiaunwra6elcontador2000

jane_collinsmep

38_degrees

davidickebbcworldatone

daily_express

networksmanagerproudpatriot101

dailymailuk

forbritain

juliet777777

steven_woolfe

stop_the_eu

helengoodmanmp

vote_leavemedia

leavethe_eu

gavthebrexit

euvoteleave23rd

jasemarkrutter

patriotic_britthe_uk_needsyou

55masseybeverleytruth

willowbrookwolflewtonserena5

fishingforleave

a_brexit_dog

nathangillmep

dwalingen

alexicon83

brexit_news

helenhims

chunkymarkunitetheunion

slatukip

the_tucdiligenttruth

annietrev

georgegalloway

justinegreeningsdoughtymp

guyoppermangeraldhowarthandrewrosindellhenrysmithmp

davidtcdavies

murrisonmpfsb_policy

otto_englishmirrorpolitics

globalbritain

liberalislandjude_kd

−2.5

0.0

2.5

5.0

−3 −2 −1 0 1 2Remain Score

Popu

larit

yPlot of top 300 hashtags. Cross side edges are highlighted. There are some connections, but the number of edges is smaller than the next figure.

Followership network analysis, followers distribution(Barberà 2015)

0.0

0.1

0.2

0.3

0.4

−2 0 2Remain Score

Den

sity

Plot of top 300 hashtags. Cross side edges are highlighted. There are some connections, but the number of edges is smaller than the next figure.

Sentiment Analysis

• Looks up terms from the Linguistic Inquiry and Word Count, a psychological dictionary

• Contains categories about: • positive and negative emotion • politics • power • quantitative language • tentative language • sadness • future v. past orientation

example: “reward” language

Leave

Neutral

Remain

0.000 0.002 0.004 0.006

Reward Language

Leave

Neutral

Remain

0.0 0.4 0.8 1.2

Positive v. Negative Emotion Language

Leave

Neutral

Remain

0.0000 0.0005 0.0010 0.0015 0.0020

Sad Language

Leave

Neutral

Remain

−0.4 −0.3 −0.2 −0.1 0.0

Future v. Past Language

Leave

Neutral

Remain

0.000 0.002 0.004 0.006

Tentative Language

Leave

Neutral

Remain

0.000 0.005 0.010

Power Language

Leave

Neutral

Remain

0.000 0.002 0.004 0.006

Quantitative Language

Sentiment Analysis conclusions: Leave was more

• reward-oriented • positive • assertive of power • less quantitative • less tentative • less sadness • future-, versus past-, oriented

−2

0

2

Jan Feb Mar Apr May Jun JulDate

Posi

tive

v. N

egat

ive E

mot

ion

Lang

uage

side Leave Neutral Remain

0.005

0.010

0.015

0.020

Jan Feb Mar Apr May Jun JulDate

Tent

ative

Lan

guag

e

side Leave Neutral Remain

Topic models

• Using unsupervised machine learning • detect topics in twitter conversation • topic distributions across sides

• Data • tweets from Leave and Remain accounts identified from NB classification • combine tweets from each account • 700,000 accounts are included

• Method • Structural Topic Model (STM) by Roberts, Stewart, and Tingley (2014) • Estimate models with 10-40 topics (incremented by 5) • covariate in topic prevalence: predicted sides of accounts

STM Results ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Donald TrumpIslam

Leave : DebateEU and other "Exit"s

Leave: Take ControlJo Cox and Xenophobia

Immigration and IdentityLeave: Messaging

EU and SovereigntyLeave: GrassrootsImmigration Control

Vote Mobilisation: LeaveBrexit and EEA

Brexit and MarketRemain: Youth

Brexit and EconomyScotland and Northern Ireland

Smaller countries "Exit"sRemain: Labour and NHS

Remain : MessagingAnger Against Leave Campaign

Vote Mobilisation: Remain

−0.10 −0.05 0.00 0.05 0.10Effect of Remain

• Selected topics from 40 topics model

STM Results (example of leave topics)

menrallisupport

free

attack

polickidmigrantthi#u

kip anti

#bluehand#deutschland

#maga#clinton

#uk

#europesai

#islamist

london

#muslimgtchristian

#orlando

week

merkel

war

belov

mohamad

europ

#america

#pegidaworldwid

#migrant

#tcot

onlirape

obam

a

dai

#england

ar

religion

armi

#merkel's

follow

#britain

ago

@trobinsonnewera

amp

#trump

#nyc

#euref

#leaveeu

#britainfirst

#france

#trump2016

#london

#migrants

#calais

britain

invas

children

#muslims

watch

#ccot

countri#pjnet

#australia

burn

world

#paris

women

thei

#usa

citizen

#germany

koran

#refugee#wakeupamerica

#dc

#islam

muslim

#cameron

#christian#welfare

#merkel

#obama

islam

#eu

#rednationrising

#british

#austria

refuge

stop

#banislam

#auspol back

......

#islamic

#ramadan

evil

proar

jewmovi

riotfulles

tablish

voter

isibritish

muslim pleas

blamefraud

sadiq

votebbc

nworacist

turkei

watch

#ukip

race

governopen

@prisonplanet

killnothwar

childrenpaul

truthmass

dai woman

ahead

elit

reason

khanrothschild

film

#jocoxmpjoseph

befor

cox'fals

victori

speechwomen black

peoplmigrant

rapefarag

divers

@breitbartnews

left

britain

videohere'

soro

counter

agenda

#whitegenocide

they'rviolent

thei

polit

bilderberg

thi

#jocox

england europ

anti

@mailonline

eu

amp

lead

world

white

@realalexjones

show

jo @sargon_of_akkad

referendum

collaps

london

watson

hate

alreadi

stop

happen

israel

mediadeath

sabotag

coxtalk

Immigration, Islam Jo Cox and Xenophobia

STM Results (example of remain topics)

vote

jeremi

@jeremycorbyn

#ids

toriamp

rt

peoplli lie

antimediagovt

chang

@carolinelucas

cut

rupertcase

murdoch

britain

hijack

climat

realgreen

healthukip

fraud

labour#nhs

remain

supportttip

#greenerin

public paul

week

servic

#marr

pai

350m

save

nhparti

auster

agre

iain

man

#ttip

#bbcqt

#labourin

democraci

leaderthei

votertrust

campaign

poor

cultur

#tories

job

sai

patel

backduncan

debat

reason

environ

social

left

sun

smith

wing

fight

#euref

fake

bori

priti

corbyn

work

worker

#toryelectionfraud

disabl

paid

protectar

class

johngovernjohnson

care

gove

elit

mp

eu

risk

europ

michael

ha

blam

e

monei

councilonli

borderpost

euamp

callunit

pm

alreadi partiremainit'

supporteurop

back

make

great

féin

haconfer

dure #scotland

@thesnp

irishref

membership #indyref2

govern

@alexsalmond

edinburghbelfast

poll

implic

wa

indyref

scottish

voteindepend

ireland

fein

scotland

#eurefcampaign

european

good

give

voter

nicolapart

morn

kingdom

sai

result

place

thi

event

uk

england

referendum

everi

northern

lead

#snp ye

reunif

welcom

union

case

scotland'clean

anoth

walebbc

ni north

respons

favour

#indyref

salmond

glasgowwelsh

tori

@nicolasturgeon

clear

sturgeon

statement

ireland'

#voteremain

time indi

english

#snpin

sinn

begun

tabl

2nd

snphighliuniti

Labour, NHS Scotland, Northern Ireland

STM Results (examples of remain economic topics)

#gbpusd

vote#tradingvolatileurobankyear

chanc

@reutersdata

amp

uk

fear

#auspolbigamid

investor

lower

ralli

earli

remain

daiboe

dow

impact

ar

british

morn

biggest

sterl

riskthi

#gbp

time

drop

bigger

asia

central

poll

currenc

#stocks

ftserate

higher

usd

bet

yellen

plung

bond

traderbui

sai back

fx

financi

global

sinc#fx

gold

marketshort

#gold

yen

fall

@wsj ha

low

eur

post

todai

latest

#ausvotes

forex

referendum

#forex

exchang

high

oil

updat

u.k

befor

gbpusd

gbp chart

mai

watch

#pound

stock

show

u.

@business

record

lead

fed

week

aheaddollar

#fed

june

triggerchiefarstaff

long

citi

london'

econom

ist

shockha

global

london

growth

propertiftse bu

dget

affect potenti

bank

deal

market

badcaus

financi

lead

high

pricelow

effect

term

doerise

lose

biggestmai

oecd

leav

aheadeuropean

sector

plan

economipostuk'

britain'video

recess

uncertaintivoteosborn

cut

pound ft

britaingovernor

billion

carnei

trade

warnjob

referendum

threattop

year

crisi

world

europ

uk

imf

casedamag

big

foreign british

leader

investor

sai

industri

cost

face

expat

riskbusi

neg impact

eu

sterl

ceo

share fear

major

hurt

german

fall

rate englandinvest

firm

treasuri

put

Brexit and Market Brexit and Economy

Topics by sideTable 6: Detected TopicsSide Topic Title

Remain Brexit ideologuesEconomic consequence of BrexitEncourage participationExchange rateFinancial riskGlobalization and migrationIrelandObama in LondonStock market riskStrongerInTabloid (Trump, Queen, Jo Cox)Talking about articles on BrexitThe city, and big businessVoter registration

Leave Brexit MovieBrexit, UKIP, Leave EUDavid Cameron and BrusselsDavid Camerons liesDebate and discussionsEconomic/Financial SovereigntyFree trade and migrantsGeneral leave argumentJobs and social securityLeave campaignNews discussionReasons to Leave EUTake back controlUndemocratic EUVote Leave

Neutral Fiscal burden of EU membershipFootball, nationalism, racismJohn OliverOpinion pollsParties and leaders (Scotland)Uncertainty

18

Table 6: Detected TopicsSide Topic Title

Remain Brexit ideologuesEconomic consequence of BrexitEncourage participationExchange rateFinancial riskGlobalization and migrationIrelandObama in LondonStock market riskStrongerInTabloid (Trump, Queen, Jo Cox)Talking about articles on BrexitThe city, and big businessVoter registration

Leave Brexit MovieBrexit, UKIP, Leave EUDavid Cameron and BrusselsDavid Camerons liesDebate and discussionsEconomic/Financial SovereigntyFree trade and migrantsGeneral leave argumentJobs and social securityLeave campaignNews discussionReasons to Leave EUTake back controlUndemocratic EUVote Leave

Neutral Fiscal burden of EU membershipFootball, nationalism, racismJohn OliverOpinion pollsParties and leaders (Scotland)Uncertainty

18

Networks of topicsFinancial risk

Vote Leave

Brexit, UKIP, Leave EU

Stock market risk

News discussion

Encourage participation

Debate and discussions

Voter registration

Parties and leaders (Scotland)

StrongerInEconomic/Financial Sovereignty

Tabloid (Trump, Queen, Jo Cox)

Brexit ideologues

General leave argument

David Cameron's lies

Brexit Movie

Take back control

Fiscal burden of EU membership

Undemocratic EU

Jobs and social security

Economic consequence of Brexit

John Oliver

Obama in London

Talking about articles on Brexit

David Cameron and Brussels

Globalization and migration

Opinion polls

Football, nationalism, racism

Reasons to Leave EU

The city, and big business

Free trade and migrants

Leave campaign

Ireland

Exchange rate

Uncertainty

Figure 6: Network of topics. The red circles are Remain topics (judged from the estimated propor-tions of topics, described above), and blue circles are Remain topics. The widths of the lines thatconnect two topics indicate proximity between two topics measured by the strength of correlations.The distance of two topics in the plot also reflects the proximity.

Table 7: Sides by Source.

Twitter YouGov

Leave 2515 2030Remain 2485 2170

19

Summary: Text analysis was used to

• predict the side of the user • determining the most common hashtags by side • using followership networks to estimate “ideology” • measuring sentiment using dictionaries • analyzing the topics that were discussed • mapping connections between topics via networks

How? Using software written in R (and C++ and Python)