tweet tracking app design document

Runninghead:TwitterTrackingandAnalysisWebAppDesignDocument 1

Twitter Tracking and Analysis Web App Design Document

Bessie Chu

University of Southern California

TwitterTrackingandAnalysisWebAppDesignDocument 2

Abstract

This is a design document to providing a product roadmap for web

application to track Twitter conversations during live events, such as a television

show or a sports game, and output parsable data for post-analysis. This

document serves to complete the requirements of IML 502: Techniques of

Information Visualization.

The web-based application would ideally be used by non-technical

personnel at an advertising agency, research group, academic institution, and

other organizations to track conversations around a certain topic, such as

individual users or a brand. The potential design and visualization functionality I

detail in this document works in conjunction with accepted design principles

drawing from theoretical frameworks and best practices. A basic rundown of

needed personnel, estimated timeline, and tools is also included with the

caveat that a specialized technical team may change these steps and

timelines depending on resources.

This document should accomplish the goal of working as a roadmap for

development in terms of design, functionality, and process to a functional alpha

version of this software for testing before a wider beta rollout to a limited number

of clients.


Expanded Project Background:

I am working on long-term research project researching international

sports fandom and brands using Twitter conversations during the UEFA

Champion’s League and the World Cup as subjects of study as a part of the

Annenberg Innovation Lab. We are working in conjunction with IBM’s jStart

Team, an emerging technologies group in the company, and international

communications conglomerate HAVAS Sports and Entertainment. I have been

working with IBM to develop a platform for the type of research we’re trying to

accomplish, namely a 1) web-based application for collecting live tweets

during a soccer game to be used by non-technical personnel, in this case

account planners at advertising agencies such at HAVAS SE. 2) a platform to

directly plug into collected tweets for limited analysis using data visualization as

an exploratory tool.

They have currently developed a Java-based desktop app that is not

scalable for distributed use. We have proposed a browser-based web

application that they are currently beginning to build.

This design document moves beyond the more basic feature set I’ve

been working with IBM. What I’m attempting to accomplish here with this

design doc is an expansion of that idea into an idealized platform.

I choose this project as I eventually want to go into Product Management,

and this is a good exercise for me to get feedback on how to organize different

moving parts, how to communicate them visually, and how to integrate


functionality with design. Part of the feature set of this idealized web

application includes plug and play data visualizations, similar to how Tableau

has pre-set options, with four specific presets in the application.

Objective

Build web-based solution to track and analyze tweets for a non-technical

audience.

Current Datasets and Collection Method:

Twitter conversations are tracked in real time using Twitter’s Public

Streaming API. The current collection methodology is a Python-based script run

through a shell script pre-set with access tokens and variable fields for each

data collection run. The script collects metadata about each Tweet, such as

User ID, handle name, text of Tweet, etc. and compiles them into dictionaries.

We then run another script to parse the dictionaries out into csv files.


Current Process:


This methodology is currently not scalable to non-technical staff,

especially given that we are tasked with having an application to be used by

staff scattering in various international markets with varying technical capability

and system configurations. Working with spreadsheets is not scalable due data

exceeding Excel’s capabilities. The data needs to be moved into a MongoDB

or PostGreSQL database with connectors and software in place to be accessed

by non-technical staff.

Data has already been collected, with nearly a million tweets on servers

from the UEFA Champion’s League Series with plans to collect data in several

key games about HAVAS SE client brands during the FIFA World Cup this

summer. This web app will allow us to scale this data collection to non-technical

staff worldwide in HAVAS SE’s offices and among non-technical USC

researchers.

Informed Design Principles: Theoretical Framework

In creating this web application, beautiful and functional design is

paramount. Working with an international audience of marketers and getting

their buy into this project will require an intuitive and aesthetically pleasing portal

to establish the credibility of our team. In this process, I must insist on using

design principles and acknowledge some perils in creating essentially preset


data visualization outputs as well as the need for an iterative process rather than

a single platform release set in stone.

Design Guidelines

1. Simple Design Principles

The current set-up with IBM’s proprietary (hence screenshots not included)

platform is not friendly for non-technical users and is desktop based. Part of that

is because it uses more than a dozen colors on the interface that resembles

more of a Windows desktop application than the browser-based applications

that access a cloud stack that our target user-base would be more familiar with.

Hence, the web app I’m proposing uses design principles in color using what

William Lidwell, Krstina Holden, and Jill Butler (2003) call “using colors

conservatively” (Color section, para. 2). I’ve done so on the wireframes that

the site colors will evolve from. The eye can only process about five colors in

one glance (Color section, para. 2). With that in mind, the website should be

created using opposing colors on the color wheel and use pure hues to attract

attention to functional elements, such as buttons to save and move data (Color

section, para. 3 & 4).

I also took care to keep elements balanced on the wireframes and will

continue to do so in the design of the web application, particularly paying

attention to symmetry. This website will be used by a relatively discerning and

design savvy yet not technical crowd, having an application pleasing to the

eye paired with functionality will go a long way. Ahamed Altaboli and Yingzi Lin


(2012) found during their experiments that at “high levels of symmetry, unity of

form… has significant effects on perceived visual aesthetics of website interface

design” (p. 732). The design should focus on coherent layout and symmetric

layout for user-friendly experience. Large conglomerates often forget these

principles that start-ups have embraced in the analytics space to their success.

I’ve kept this in mind as I’ve wireframed this app.

2. Acknowledging Perils

Alexander Galloway argues that, “Data have no necessary form” (p. 82)

meaning “any visualization of data requires a contingent leap from the mode of

the mathematical to the mode of the visual. This does not mean that

aestheticization cannot be achieved” (p. 82) thus “every interface must try to

overcome its own unworkability” (p. 85). In other words, data should be made

beautiful and understandable. This is the challenge of how to strip down the

interface to be usable by laypersons but not remove too many potential

features for research, being mindful of not to “sacrifice the aesthetic for the

algorithmic” nor “sacrifice the algorithmic in favor of the aesthetic” (p. 97).

Neither the art nor the science should be shortchanged.

In both interface design for the site and the data visualization outputs, I

chose the simple visual outputs to acknowledge these issues. This audience

should be able to understand planned outputs from the naked eye: namely a

network graph, bar charts, and spreadsheets. This could be a particular issue

with the network visualization outputs. Development will have to pay careful


attention to the network mapping and making sure that connections can be

zoomed in and out of and panned without loosing the density provides deep

insides into what Julie Steele & Noah Illiinsky (2010) call “social intimacy” (p. 23)

in the connections among users. The outputs should allow us to see the social

landscape easily.

3. Iterative Design

One challenge and design with this idealized app is to make it a constant

work in progress. I concur with Jer Thorp (2003) describing taking “small

visualization steps” as “sketch points” that don’t initially consider aesthetics.

The way this design doc is initially designed, as well as the alpha and beta

versions could be taken as adhering to that process. Some companies want to

deploy everything done at once, but I believe in the iterative approach and

allowing the end users to weigh in their input and going back and making

incremental changes before a large release. A large release should not

constrain future improvements.


Design Document

Key Functionality

• Login page

o Twitter API key configurations tied to logins

• Simple navigation page

• Tracking page connected to Twitter Public API Stream

o Field to copy and paste keywords and hashtags to track

o Field to copy and paste user IDs to track

• Analysis Dashboard

o Network analysis showing how users are interconnected with links

and nodes

o Summary Bar Chart analysis on which URLs or web links were

tweeted the most

o Summary Bar Chart showing which handles had the most retweets

o Tweet and User Search

! Look up individual users and tweets by metadata elements

! Look up tweets containing keyword, hashtag, or other search

value


Wireframes

Site mockups created with Balsamiq

Core functionality outline:

Index Page:

• Login page that will load a post login page

• It is expected that users will be pre-generated accounts from company

without a registration step


Post Login Page:

• Simple navigation page


Tracking Page

• Stripped down functionality as much as possible as a motivation

• Current IBM tool is extremely cluttered with too many options, which has

proved frustrating for even technical staff at lab

• Once tracking starts the “Tracking Output Page” will appear

• Users will be able to stop and add more keywords or user ids, which will

automatically saved in the fields

• When user clicks “Finished,” values will clear


User Configuration Page:

• Simple configuration page to copy over Twitter credentials

• Field order and text mimics Twitter Developer Account layout

• This step has proven one of the most difficult in beta testing

• Duplicated link to documentation with the text “Help” since there will likely

be confusion here


Tracking Output Page:

• Streaming page with just with text output from Twitter stream to watch

tracking


Analysis Dashboard

• Search for file (files available to all members of team) and select method

of analysis


Network Graph

• Example graph taken from Gephi

• Ideal scenario where users can zoom into nodes and click individual

nodes for user metadata

• See how users might be linked to certain influencers and social eco-

systems on Twitter


Retweet Bar Chart

• Bar chart of most Retweeted users. Reasonable magnitude algorithms will

need to be built to cut-off too values too small to add to analysis


URL Breakdown Bar Chart

• Bar chart of most Tweeted URLs. Certain television shows have Instagram

heavy audiences, for example, that could be interesting for advertisers


Spreadsheet Browse and Search

• Search by individual user or terms in Tweet text to load into interface for

csv export


Production Timeline and Personnel Needed

Personnel

• Backend Developer with knowledge of EC2 Amazon Cloud Servers

• Frontend Developer with knowledge of moving data from site inputs to

back-end smoothly

• Product Manager gathering requiresment and feedback

Development Timeline

Next Steps

After the completion of this course, I plan to deliver a stripped down

version of this design document for our purposes to IBM to build the tool that will

only include the login and collection functionality as an initial version.

The rest of the functionality roadmap will be cleaned up and expanded

upon accordingly with feedback to be loaded on a personal online portfolio of

my graduate work.


UPDATE: This application was built and used in six international offices at HAVAS

(Brazil, France, Italy, Mexico, Russia, United Kingdom) and by three USC

researchers as part of a data collection of nearly 10 million tweets.


References

Altaboi, A., & Lin, Y. (2012). Effects of unity of form and symmetry on visual aesthetics of

website interface design. Proceedings of the Human Factors and Ergonomics

Society Annual Meeting, 56(1), 728-732. doi:10.1177/1071181312561152

Galloway, A. R. (2013). The interface effect. Hoboken: Polity.

Lidwell, W., Holden, K., & Butler, J. (2003). Universal principles of design: 100 ways to

enhance usability, influence perception, increase appeal, make better design

decisions, and teach through design [Kindle Edition]. Gloucester, Mass: Rockport.

Steele, J., & Iliinsky, N. P. N. (2010). Beautiful visualization: Looking at data through the

eyes of experts. Sebastopol, CA: O'Reilly.

Thorp, J. (2013). Visualization as a process., 2014, from

http://blogs.hbr.org/2013/04/visualization-as-process/

Twitter Developer Documentation. (2014). Frequently asked questions., 2014, from

https://dev.twitter.com/docs/faq#6861

tweet tracking app design document

Data & Analytics