Including the Human in Humanitarian Big Data

Sara-Jayne Farmer@bodaceacat

The UN’s Development Reversal Problem

2

Where we’d like to be

3

How bad is it? 2008 – 2011 information lags

4

2008: Not knowing what, who, where

5

2011: New Data Sources

6

But new data only gets us so far…

7

Bots

Humans

Humans don’t have all the info, machines don’t have all the smarts

Hunchworks: hypothesis management

» System = people + processes + tech

» Break down the siloes: Engage and connect the global network of UN and international development field staff with analysts and decision makers

» Use the machines better: use agents to detect weak signals

» Use the people better: Encourage ‘hunches’ (hypotheses) about anomalous or interesting data and observations

» Encourage cooperation: Rapidly collect information about emerging situations

» Combine strengths: mix human insight with data analysis results

8

Hunch Mechanism

» Detect an interesting “weak signal”

» Hypothesize (make a hunch) about the nature of the signal

» Share the hunch with your social graph

» Attract related signals and evidence

» Verify hunch by engaging communities of practice

» Act

» Use confirmed hunch to refine trust within the system

9

Issues: Community, Complementary skills

10

Issues: Encouraging use, encouraging Trust

11

Issues: Encouraging Action

12

“is a hunch true enough to take action?”

Issues: Security, localisation, bandwidth

13

Issues: fun problems we’re working on

14

» Hunch set management• Clustering: hunches are related• Merging: hunches are the same thing• Splitting: hunch has changed significantly

» Mixing human and bot users• Autonomy: how to share information, decisions, responsibility• Managing culture differences between humans and bots

» Managing credibility• Rating hunches, users, data• Overall and friends’ ratings

» Hunch network as a reasoning system • vs “a hunch is the discussion surrounding it”