effective temporal graph layout: a comparative study of animation

http://ivi.sagepub.com/Information Visualization

http://ivi.sagepub.com/content/10/1/47The online version of this article can be found at:

DOI: 10.1057/ivs.2010.10

2011 10: 47Information VisualizationMichael Farrugia and Aaron Quigley

Effective Temporal Graph Layout: A Comparative Study of Animation versus Static Display Methods

Published by:

http://www.sagepublications.com

can be found at:Information VisualizationAdditional services and information for

http://ivi.sagepub.com/cgi/alertsEmail Alerts:

http://ivi.sagepub.com/subscriptionsSubscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

http://ivi.sagepub.com/content/10/1/47.refs.htmlCitations:

by guest on May 22, 2011ivi.sagepub.comDownloaded from

http://ivi.sagepub.com/

http://ivi.sagepub.com/content/10/1/47

http://www.sagepublications.com

http://ivi.sagepub.com/cgi/alerts

http://ivi.sagepub.com/subscriptions

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.com/journalsPermissions.nav

http://ivi.sagepub.com/content/10/1/47.refs.html


Original Article

Effective temporal graph layout:A comparative study of animationversus static display methods

Michael Farrugiaa,∗ andAaron Quigleyb

aUniversity College Dublin, 8 Talbot Down,Dublin15, Ireland.E-mail: [email protected] of Human Computer Interaction,School of Computer Science, The Universityof St. Andrews, UK.E-mail: [email protected]

∗Corresponding author.

Received: 27 August 2009Revised: 23 August 2010Accepted: 13 September 2010

Abstract Graph drawing algorithms have classically addressed the layoutof static graphs. However, the need to draw evolving or dynamic graphshas brought into question many of the assumptions, conventions and layoutmethods designed to date. For example, social scientists studying evolvingsocial networks have created a demand for visual representations of graphschanging over time. Two common approaches to represent temporal informa-tion in graphs include animation of the network and use of static snapshotsof the network at different points in time. Here, we report on two experi-ments, one in a laboratory environment and another using an asynchronousremote web-based platform, Mechanical Turk, to compare the efficiency ofanimated displays versus static displays. Four tasks are studied with each visualrepresentation, where two characterise overview level information presenta-tion, and two characterise micro level analytical tasks. For the tasks studied inthese experiments and within the limits of the experimental system, the resultsof this study indicate that static representations are generally more effectiveparticularly in terms of time performance, when compared to fully animatedmovie representations of dynamic networks.Information Visualization (2011) 10, 47--64. doi:10.1057/ivs.2010.10

Keywords: graph drawing; dynamic social networks; information visualisation;evaluation; animation

Introduction

Real-world networks are not static, they are in a state of constant evolu-tion. Networks, such as social science networks, biological networks andcomputer networks, are constantly changing with new nodes added andnew relations forming, while old relations may either persist or end. Inthis article, we focus on dynamic social networks and their effective visu-alisation. Despite the lack of widespread research publications on dynamicsocial networks, Doreian and Stokman1 report that research on dynamicnetworks is increasing. Recently, Volume 325 (July 2009) of the ScienceJournal featured a special edition on Complex Systems and Networks. Inthis feature, written by the most prominent authors in network science,the study of dynamic networks was identified as one of the current mainchallenges in network theory.It is recognised that collecting longitudinal data is a time consuming andchallenging task. This explains why network data is typically only collectedonce for analysis. Owing to this, the field of social network analysis hasoften concentrated on single snapshot instances of a network. Thanks inpart to the proliferation of online technologies and social media, the datacollection bottleneck has been partially overcome and dynamic networkdata sources have become more accessible.

© SAGE Publications, 2011. 1473-8716 Information Visualization Vol. 10, 1, 47–64



Farrugia and Quigley

Dynamic social network analysis on longitudinaldata can be useful for investigating long-term humanbehaviour. Christakis and Fowler2 studied the dynamicsof smoking behaviour over a period of 32 years. Smokingdecreased throughout the data collection period and inter-estingly connected groups of smokers stopped smokingtogether suggesting a network effect. In another study,van Duijn et al,3 studied the evolution of friendshipsin University freshmen. Their results show that phys-ical proximity of actors and the individual similaritiesbetween actors, are only contributors to the beginningof friendship formation. Dynamic network analysis canalso be important in criminal investigations. Xu et al4

describe the processes and measures applicable whenusing dynamic network analysis in a criminal contextand also make effective use of visualisation, includinganimation, to present the evolution process of criminalnetworks.

From the earliest research in social network analysis,visualisation has been a central tool. Moreno, the inventorof sociometry,5 devised the first graphical representationof a relationship between two entities and called it thesociogram. Since Moreno’s days there has been a signif-icant increase in complexity and improvement in themethods for how social networks are drawn.6 Howeverin network visualisation, the temporal aspects of thedata within the network have often been ‘flattened out’or ignored. The realisation that the study of differentversions of a network over time can lead to insights notpossible through static display alone has given rise to anincreasing interest in dynamic graphs within the visual-isation community. A testament to this, is the fact thatat least three major conference competitions, GD’98,7

InfoVis’048 and VAST’089 contained tasks requiring thevisualisation and characterisation of dynamic networks.

Beyond the problem of developing either a graphlayout technique or a new visual design for dynamicnetworks lies the presentation of the visualisation. Clas-sically, there are three ways in which dynamic networks,visualised as node-link diagrams, can be presented. Themost common presentation of dynamic graphs is toanimate the different layouts for each time point, andproduce a movie of the dynamic network.6,10–13 Alter-natively, the individual layouts can be displayed adjacentto each other in one screen akin to a photo album withthumbnail images.14,15 In a variant of this presentation,the node link diagrams can be displayed as static imagesin sequence, similar to a slideshow.16,17

When it comes to evaluating the mode of presenta-tion of dynamic networks, studies usually either rely onthe general intuition that animation is beneficial, or elseask for experts to judge the usefulness of animation.4 Inreality, no empirical studies have been conducted on theactual benefits of animation when visualising dynamicsocial networks. In this study, we attempt to empiricallyevaluate the benefits of animation and understand thetasks and cases for which animation is beneficial overstatic images.

As part of our evaluation, we conduct two experimentsusing different experimentation platforms. The first exper-iment is a laboratory user study with a group of computerscience students. In the second experiment, we attemptto generalise the results obtained in the laboratory byextending the study to anonymous users, who might nothave experience with social network visualisations, in aremote experiment using Mechanical Turk (MTurk). Aspart of our results, and discussion we detail our experi-ences using a remote experimentation platform and theimplications it has on visualisation experiments.

The rest of the article is organised as follows. InSection ‘Related work’, we review related work in thearea of dynamic social network visualisation, animationand visual perception, including approaches to visuali-sation evaluation with an emphasis on remote experi-ments for visualisation. In Section ‘Experiment setup andmaterials’, we describe our experiments’ design, materialsand methodology used to compare two visual represen-tations. In Sections ‘Results’ and ‘Discussion’, we reporton the results and then discuss them. Finally, we deriveconclusions based on the results and discuss areas forfurther work, both in the evaluation and implementationof such visual systems.

Related Work

This work draws on three principal areas of visualisationresearch; graph drawing, visual perception and visual-isation evaluation methods. Dynamic graph drawingprovides the technical background to visualise networksfollowing desirable criteria of graph layouts with atemporal component. As the aim of the study is tocompare animated and static modes of presentation,research on the efficiency of both mediums as a percep-tual aid in different visualisation applications, is reviewed.Part of this research includes a user study, where a rela-tively new platform that takes advantage of a web-basedmicro-market to engage a user study population. Earlystudies this platform, report promising results if thenecessary care is take to interpret the results, in the appro-priate context. Recent work on the applicability of thisplatform for the purpose of resourcing participants andconducting online experiments is also reviewed.

Dynamic graph drawing

In its most basic form a graph consists of a number ofnodes, representing elements of information and theinterrelationships between these elements, or edges.Graph drawing is the challenge of determining ageometric position for every node and a route for everyedge, so that a picture of the graph can be rendered.Dynamic graph drawing is an area within graph drawingthat studies the layout of graphs with a temporal compo-nent. This dynamic component can come about from new

48



Effective temporal graph layout

nodes or edges being added to the network or existingones being removed or updated.

For the purposes of this research, we focus specifi-cally on social networks. Graph theory provides a solidframework onto which the concepts and ideas fromsuch networks can be mapped. This brings a vocabularywhich can be employed to denote structural proper-ties and a set of primitive concepts and mathematicaloperators to measure these structural properties that arerepresenting social structures.18 Given the natural repre-sentation of a social network as a graph, graph drawingitself is the natural approach to the visualisation of socialnetworks. Graph drawing is a well-researched area incomputer science and several good references exist on thesubject.19–22 In addition, Freeman6 provides a detailedreference on which of these algorithmic approaches arebest suited to social network visualisation.

In traditional graph drawing, a single static version ofthe network is drawn. However, when drawing ‘dynamicnetworks’ several related versions of the same basicnetwork need to be drawn. Here, the time dimensionadds an additional constraint beyond the traditional opti-misation of aesthetic criteria20,23,24 with static graphs.When drawing temporal dynamic graphs it is generallyconsidered beneficial to keep node movement betweentime periods at a minimum. This is to preserve the mentalmap between different related layouts.25,26

Where dynamic social network visualisation has beenstudied, the focus has been predominantly on the forma-tion of network structures. The most common approachto the dynamic graph drawing problem builds upon theflexible nature of force directed layout algorithm27 tocontrol node movement. These layout algorithms28,29

often include a stiffness parameter to dampen node move-ment between successive time-steps to help maintain asmooth change to the layout.

When the sequence of graph versions are known beforethe drawing, the complete layout information can beexploited to improve the node placement. These class ofgraph drawing techniques are called offline layout algo-rithms. In one of these techniques, Diehl and Görg30

use the supergraph of all graph iterations to calculate thenode positions. These positions are then used to informthe individual node locations at the drawing of each timestep.

Lee et al31 extend layout heuristics first applied to staticgraphs, that of simulated annealing by Davidson andHarel.32 They formulate the dynamic graph drawing ques-tion as an optimisation problem, whereby they attempt tominimise a cost function of graph changes between iter-ations. Although encoding different graph criteria sepa-rately gives the user more control over the layout, thesealgorithms are typically quite slow to execute. Dwyer andGallagher33 employ a two and a half-dimensional layoutwhere time is mapped to the third dimension providinga 3D layered view of the changes in the data over time.

In this study, we fix the parameters related to graphlayout. We employ the Social Network Image Animator

(SoNIA)34 software framework whose focus is on the opti-misation of dynamic graph layouts and use it to generatethe graph images and videos for our studies. Our approachis to vary the mode of presentation to study the impacton graph understanding.

When evaluating dynamic graph drawing algorithmsone needs to compare the difference between layoutsacross different time steps and measure the change tothe layout over time.35 Friedrich and Eades36 provide acomprehensible list of criteria and measures of a goodanimation. Frishman and Tal29 use the distance betweennodes in successive layouts to calculate the averagedisplacement of the nodes. They complement this withmeasures of the energy (stress) of the graph at successivetime points as a measure of the ‘niceness’ of the layout.

While several good attempts have been made to under-stand the impact of graph layouts and presentation on thegraph comprehension,23,24,37 work on this is still in itsformative stages. It is difficult, if not impossible, to claimthat a group of measures can be used as an absolute indi-cator of optimal layout or presentation. Instead, quanti-tative claims from a new algorithm provide an indicationthat it improves upon certain criteria, that are currentlybelieved to impact upon perception.

Mode of presentation

Animation is essentially a sequence of images displayedin rapid succession to give the illusion of movement.Using animation, one can embed time within thenetwork structure itself without changing any additionalattributes. Animation is a natural way of presentingtemporal information,38 therefore it is hardly surprisingthat animation is the most popular manner of presentingdynamic networks.

One of the most concrete applications of dynamicnetwork visualisation is the study by Moody et al17 thatuses dynamic network visualisation to characterise tworeal and one synthetic dynamic network. Both anima-tion and static images are used to present the results.The animation presentation is used to create movies ofthe networks, whereas static images are presented usingflipbooks (slideshows). In the animations, nodes are freeand can change position between different time periods,however in flipbooks all the nodes are fixed between timeperiods. In flipbooks only one common graph layout isused, an approach that the authors claim is appropriatefor relatively sparse networks.

The authors also distinguish between continuous anddiscrete time in networks. Continuous network data ismade up of streaming data with known start and end timesfor each relationship, for example, a network extractedfrom a log of phone calls. Discrete networks, on the otherhand, collect data at specific time intervals. Two classroominteractions from the McFarland classroom network39 areused as an example of a continuous time network. Thisdata consists of a stream of interactions of conversionturns in classrooms.

49




In their article, the patterns identified in the threenetworks are presented convincingly using animation.It is clear that animation was the preferred presenta-tion method for analysis and presentation. Perhaps, onereason for this is the fact that animation provided lessinteractive burden owing to the number of time periodsin the networks (for instance, 200 iterations for thesyntactic network). Animation has the ability to condensemany images in a short space of time and to be executedwith one play action. To visualise this data with flipbooksone has to manually browse through all the images insequence. In addition, the fact that nodes were fixed inflipbooks, does not provide a comparable alternative toanimation were the nodes are moving.

Animated visualisations of node link diagrams arealso popular and successful in visualisation competi-tions where dynamic network data needs to be analysed.The best two submissions in the Graph Drawing ’98competition40 were both presented as recorded anima-tions. In Info Vis ’04 the winning submissions analysedthe temporal data set of visualisation publications usingmainly static network images41 or alternative visualisa-tions to node link diagrams.42 In VAST 2008, there wasno single winner but a number of different rewards forsignificant contributions were awarded. Among thesewas an award for the best animation which clearly high-lighted the structural change occurring in the evolvingnetwork.43 The submission explains how the animatedvisualisations were used to support the analysis andassisted in finding the solution to the problem scenario.Most of the submissions in VAST 2008 used multiplestatic images of node-link diagrams to show the visualisa-tion. Few of the more advanced systems such as44 addedinteractive features to allow image scrolling or viewingmultiple images in a single window. In addition, somesystems extracted the time element from the graph data,and used the time-dimension in a traditional time-basedline chart. A notable visualisation in this case was thecontribution by the University of Maryland45 that usedthe Social Action platform46 to visualise the activity ofeach node throughout the time period as a stacked linechart.

Animation studies in node link diagrams

Although animation is an intuitive and almost naturalway of presenting temporal information, there were veryfew efforts to evaluate animation as a way of presentingnode link diagrams. Most of the evaluations of dynamicnetworks visualised as node link diagrams study theimportance of retaining the mental map in for the graphlayout.26 When animation is used for presentation, it isoften assumed that animation is the logical and henceonly way to represent such information.

One of the few studies in this area was conducted byMcGrath and Blythe47 who study the impact of motionon viewers’ perception of graph structure. The authorsconduct a user study based on an analytical scenario,

rather than a purely syntactical task-based experiment,to understand the impact of both motion and layout inunderstanding the network. Motion is used to facilitatethe tracking of the nodes’ between two time periods inthe network. The results show that motion has a positiveeffect when used to understand change in the status ofa node. A change to the network layout alone does nothave a statistically significant impact unless motion isadded to the visualisation.

Ware and Bobrow48 investigate motion in node linkdiagrams, however in this study motion is only usedas a highlighting mechanism. In this context, the pre-attentive property of motion is exploited and used as avisual coding attribute of nodes, such as colour and shape.The three different experiments conducted all suggestthat motion is effective as a means of highlighting innode link diagrams.

Animation for perception

Animation as a mode of presentation, has been studiedin different fields with mixed results. Most of this workstudies animation in a learning environment.38 Incomputer science, this was reflected in using animation toexplain the internal workings of complex algorithms.49

Similar to the case with dynamic networks, the first devel-opers of algorithm animations believed that animationsare beneficial as a learning tool, without empirically evalu-ating the perceived positive effect. When scientists startedto validate this claim by running perception experimentsto understand the benefits of animation, animation wasnot found to perform significantly better or worse thanstatic images.50 Interestingly, followup studies in thisarea51 attempted to find instances where animation ishelpful. In these studies, the benefits of animation wereseen when students were allowed to use it in a less rigidenvironment supplemented with other material. Anima-tion was also found to be useful when the users wereallowed to actively interact with the animation by usingtheir own data to be visualised with the animation.52

Inspired by the favourable audience response to HansRosling’s presentations at TED 200653 and TED 2007,54

Robertson et al55 addressed the effectiveness of animationin trend visualisation. The authors differentiate animationas used for analysis (when the results are not known), asopposed to when animation as used for the presentationof results (when the results are known). In their study,animation is compared to two static image representa-tions, small multiples and a slideshow view with tracesof movement shown. In this study, on multivariate dataanimation some positive effects of animation are identi-fied in the presentation tasks, yet even for presentationthe accuracy in the animated visualisations was lowerthan the static representations. For the analytical tasks,the two static image representations had better perfor-mance in both accuracy and time measures. Interestingly,in the informal survey at the end of the experiment, the

50




participants thought that animation was easier and moreenjoyable than their static counterparts. The appeal ofusing animation in competition submission40,43 and inexplaining network patterns in research publications2,17

can be partially explained by the positive effect animationhas when presenting data as described by Robertson.

When describing animation as a mode of presen-tation (in Section ‘Mode of presentation’), we distin-guish between ‘a presentation mode’ as opposed to‘for presentation’. In the context of this article, modefor presentation refers to the way the visualisation ispresented, whether it is presented using static images oranimation. The graph layout and the node link diagramare the visualisation, whereas animation and static imagesare two ways that visualisation can be presented.

Other work36 has investigated aspects of animationespecially the interpolation between keyframes in anima-tions. Yee et al56 explore the case of interpolating betweenradial-based images and from the informal usabilitystudies find that interpolating the polar coordinates of thenodes radial layout, makes the transition between screenseasier to follow. Animation can also be used to assistwith transitioning between different visualisations.56–58

When animation is used as a transition the animation isnot the main component of the visualisation but anima-tion is used to assist with a shift of viewpoints to makethe retention of context or the ‘mental map’ easier. Boththe formal evaluation by Heer and Robertson and theinformal study by Yee et al report positive results in favourof animation. In this case, animation is used as a tool toassist with transitions rather than as the mode of presen-tation itself. This application of animation however isquite distinct from animation as a means of presenta-tion, which is the way that animation is being studiedin this article. In these experiments, we are investigatinganimation as a tool in it’s own right.

Current systems support for dynamic networkvisualisation

The number of software packages that can be easily usedby social scientists to visualise dynamic social networksis limited when compared to the number of packagesavailable for studying static networks. Social scientistswho study dynamic networks tend to employ strictlymathematical tools such as the Simulation Investigationfor Empirical Network Analysis (SIENA) package,59 whichdo not support visualisation. If we look at the benefitsof information visualisation,60 we can see that dynamicSocial Network Analysis can clearly benefit from visualisa-tion. However, a simplistic approach to this problem mayfurther obscure the data and reduce the utility of the visualdisplay with clutter and gratuitous glyph movement.

Few visualisation packages support a time element toallow for the network to change over time. For example,Netdraw, which comes packaged with the popularUCINET61 package does not support dynamic networks.

Pajek62 supports time-based networks and can generateimages of the network at different points in time. Pajekpresents the different networks snapshots as single imagesin a slideshow requiring the user to manually click on asequence of images.

The biggest contribution to date, to the visualisation ofdynamic network data, has been from Skye Bender-deMolland Daniel McFarland,34 who developed an open sourcesoftware framework called SoNIA to visualise dynamicnetworks. The main focus of SoNIA is to provide a plat-form for testing and comparing different graph layouttechniques, using dynamic instead of static network data.The network animation in SoNIA is created by joiningtogether a sequence of images of the network, and inter-polating the position of nodes between keyframes.

The authors define a framework for representing time-based network events. They categorise different timesequences in typical network data sources, and suggestways how time can be represented in the source datafor use in the network visualisations. They introduce theconcept of ‘slicing’ event data, as a metaphor to describea network at a point in time. The slice can be either a thinslice or a thick slice. In a thin slice, the network is extractedfor an exact point in time. Thin slices are good to querynetwork data that contains a duration element in thenetwork events. In a thick slice, all the events in particulartime window are considered. This is good when networkevents don’t have duration, or have a small duration, anda time window is used to group multiple events in thenetwork. The framework used for defining time-basedevents in network data is scalable to various data sources.

Recently, Loet Leydesdorff et al12 have developed a newlayout technique based on multidimensional scaling fordynamic network layouts. Unlike previous approaches,such as SoNIA, they do not use linear interpolationbetween independent static frames. In the new layoutalgorithm proposed, the authors specifically add a param-eter to control the change and movement of nodes overtime. This parameter in turn controls how much thelayout changes between subsequent frames and can betuned to retain the mental model of the layout betweendifferent frames. This method is implemented in Visone63

a publicly available software system for visualising andanalysing social networks.

Despite the interest and advances from the visualisa-tion community in dynamic social networks, the numberof available systems used by social network scientists thathandle dynamic networks is still lagging behind the devel-opments reported from visualisation. Perhaps, the mainreason for this is that these advances are developed inprototype systems and not extended into comprehensivegeneral purpose systems that are widely available.

Crowdsourcing experiments

As part of our experiment, we employ a service providedby Amazon called Mechanical Turk (Mturk).64 MTurk

51




allows requestors (the employers) to outsource, or bettercrowdsource, small, stand alone tasks to a pool of workers.The tasks are typically small enough to be completed in afew seconds or minutes, and each task is usually paid inUS cents. In the site’s vernacular, the tasks are called HITs– Human Intelligence Tasks. On the basis of sample usageexamples in the service’s documentation, the site wasenvisaged to help with tasks such as image categorisation,survey responses and website filtering.

The site also provides a facility to filter the workers thatcan participate on tasks either by measures of past perfor-mance or general demographics such as the worker’s loca-tion. Users can also be requested to qualify for certain HITsby first completing a qualification task designed to testthe suitability of the worker to complete those tasks. Thisprovides a good way to filter different workers based ontheir skill set and knowledge.

Studies on the demographics of Mechanical Turkworkers65,66 report an evolving trend in the worker popu-lation. Currently the highest concentration of workers isaround 56 per cent US based, about 30 per cent Indianand a minority from other countries. The gender hasbecome more balanced at almost 48 per cent male, 52per cent female, as opposed to a 60 per cent female popu-lation in the early years of the service. The average ageof workers is 31.6 and the education level is high with59 per cent of workers having a Bachelor degree or higher.The reasons survey respondents give for working onthe site are predominantly for making money (espe-cially among the Indian population), for fun and to passtime.

Such a platform is attractive to researchers as it providesa set of participants that are easily available, willing toconduct small tasks such as those involved in experi-ments, and usually involves a lower payment and timeoverhead in the process of conducting the experiments.The platform has already been used with promisingsuccess in user studies67 and to validate the quality ofmachine translation68 where Mechanical Turk workerswere asked to judge the quality of a translation.

Recently, Heer and Bostock69 studied the suitabilityof MTurk as an experimentation platform for visualperception, and found that the platform has potential forconducting viable visualisation experiments at a lowercost, faster completion time and with more participantsthan typical laboratory studies. The seminal study ongraphical perception by Cleveland and McGill70 wasreplicated on MTurk with identical results to the orig-inal study. In the replicated study on contrast,71 theauthors find that web-based experiments might, in fact,be more representative of the general usage in day to daylife, rather than results in a laboratory based on a singledisplay. Among their findings, Heer and Bostock claimthat long tasks are not suited to MTurk as they are moreprone to ‘gaming’. On the basis of this insight, we splitthe whole experiment in smaller chunks, yet promotethe completion of the entire set of experimental tasks byoffering a bonus for a complete set of answers.

Experiment Setup and Materials

In order to study the impact of the mode of presentationon understanding dynamic networks, we conducted twodistinct yet related experiments. Before the actual exper-iments, the experiment design and material was testedwith a pilot group of social scientists, as distinct fromour subsequent computer science students and onlinetest subjects. Following the observations from the pilotstudy, our first experiment is a group laboratory experi-ment with computer science students familiar with nodelink diagrams. In the second experiment, we expandthe audience of the first experiment to unspecialisedparticipants by conducting the same experiment on theMTurk platform as introduced in Section ‘Crowdsourcingexperiments’.

A within subjects factorial design was employed in bothexperiments. The independent variables tested were 2×visualisation Type (Animation versus Static Images), 2 × 2Tasks (Global Network overview versus Local Individualnode versus Specified time period versus Unspecified timeperiod) 2× Density (Low density versus High density) and2× questions. The total number of questions for eachparticipant were 2×4×2×2=32. In addition to this, fourquestions were presented to each participant in the begin-ning of the experiment to help them familiarise them-selves with the system and question types. This data wassubsequently discarded from the analysis.

Tasks

When studying dynamic networks, scientists are inter-ested in change that occurs at different time periods inthe network. Change in the network can occur at twolevels; at the global network overview level and the localindividual node level. Overview level tasks, are tasks thatrequire the analysis of the entire network, whereas indi-vidual actor tasks focus an individual actor or a smallgroup of actors. On the basis of prior experience withanalysing dynamic networks,13 an analyst typically startsby identifying the overall change in the network beforedrilling down to investigate the actors that are responsiblefor that change. This task distinction is also applicableto static social network analysis and is synonymous withoverview and detailed tasks when analysing attribute data.

The temporal search space of a dynamic network isanother important variable when studying dynamicnetworks. In a similar manner in which a network can bestudied from an overview perspective and a local nodeperspective, the temporal search space can either be globalacross the entire time period or else localised to focus on aspecified time period. Entire time period tasks question thecomplete data collection period of the network whereasspecified time period tasks, question the network (orindividual actor) at a specific point in time, for examplein weeks 2, 3 and 4. Each of the first two task variablescan be combined with tasks from the second category to

52




Table 1: Examples of experiment questions for each task

Network overview – Specified time period (Connection) Find a node that has only one connection on weeks 3, 4 and 5?Network overview – Entire time period (Density) Which week has the least number of connections?Actor detail – Specified time period (Degree) How do the number of connections of Node 1 change in weeks 2, 3 and 4?Actor detail – Unspecified time period (Connection) For how many weeks does the connection between Node 1 and Node 6 last?

obtain network level tasks within a specified time period,network tasks throughout the entire time period, indi-vidual tasks with a specified time period and individualtasks throughout the entire time period.

To formulate the user questions, we selected a number oftasks related to graph visualisation72 and formulated themin a dynamic context. For the individual actor tasks wedeveloped questions on actor degree, path length betweenactors and general questions about the creation of newlinks, and the removal of old links. As a purely dynamictask for individual actors, we employed tasks on transitiverelations between actors.

For the overall network tasks we asked questions aboutnetwork density and the number of groups observedthroughout the entire network. An important networkwide observation in dynamic networks characterises howactors form new ties, or change their linking preferencesover time.15 As a pure dynamic concept question in thiscase, we included questions on homophilious patterns ofconnection between actors in the network. The principleof homophily states that new ties are more likely to formwith other actors that resemble oneself.73 An example of aquestion on homophily is ‘Throughout the 6-week period,green nodes tend to form more connections with bluenodes, than with other green nodes?’. Other examples ofquestions for each task type are described in Table 1.

The pilot questions asked to the social science studentsand subsequently the computer science students werequestions that assumed a prior knowledge in interpretingsocial network or graph drawings. As MTurk workersmight not necessarily be familiar with node link diagramsand graph concepts, questions about paths were removedand reformulated into other forms of questions. Also,based on the results of the pilot study and the first labo-ratory user study, questions on transitivity proved to bequite challenging, therefore they were removed from theset of question types for the MTurk. The four differenttypes of question types used for MTurk included questionsabout connections, density, degree and homophily. Thequestions did not use any technical terms and were allphrased in terms of the connection patterns of the nodes.

Hypotheses

In this work, we ground our hypotheses on previous workon animation and the effect of motion on perception andapply it to social network visualisation.

Hypothesis 1: Network overview tasks and no specified timeperiod – Based on the work on the benefits of motion

in overall tasks by McGrath and Blythe,47 wehypothesise that animation will be beneficial, there-fore (a) faster and (b) more accurate, in analysingnetwork overview task without a constraint on thetime period.

Hypothesis 2: Individual Actors and no specified timeperiod – The positive results of animation used fortransitions57 suggests that animation can be useful tofacilitate the retention of context between differentvisualisations. If this notion is extrapolated a stepfurther, one can hypothesise that animation maybe beneficial, therefore (a) faster and (b) more accu-rate, for tasks that require the user to follow a nodethroughout the network.

Hypothesis 3: Specific time period tasks – For localisedtasks constrained by a time period, the overheadof interaction with the video for searching, stop-ping and pausing might result in animated presenta-tions constrained by time being (a) slower than staticversions. The interaction overhead however shouldnot impact the (b) accuracy of the results.

Hypothesis 4: Network density – Following Robertsonet al,55 we hypothesise that the average time torespond will be (a) faster and (b) more accuratein lower density networks than in higher densitynetworks. We are unsure about the effect of densityand representation on each other, to this effect, wewould like to investigate if the performance in thedifferent representations is effected by the density ofthe network.

Data sets

Two data sets with different densities were used in thisstudy. The first network (shown in Figure 1) is a lowdensity, manually created network containing nine nodesacross six different time periods. In this network, eachnode has one attribute encoded by shape and colour. Thesecond network (shown in Figure 2) has a higher densityand consists of 32 nodes, 2 node attributes, 2 types ofedges and 6 time periods. In the visualisations, one of thenode attributes is represented by colour and the other byshape. The second network is adapted from van Duijnet al74 who studied the evolution of friendship among aclass of freshmen. We extracted part of this network thatincluded only relationships of strength 1, 2 and 5 andremoved edge direction. The reason for this simplificationis to make the network more visually legible by removingedges of low semantic significance. Throughout, the

53




Figure 1: Low density network week 4.

Figure 2: High density network week 4.

experiment no reference was made to the source of thedata in order to avoid any bias by participants who mighthave been familiar with the data set.

Network images and animation

The two data sets in the study are presented to the partic-ipants as static network images and as animated movies.The two representations have an identical graph layout,colours and labels. In fact, the static images are taken fromthe six keyframes in the movie, one at each time pointin the network. The movies were created by interpolatingthe positions of the nodes between the six keyframesused as static images. The movies for each network weregenerated at three speeds, normal (182 frames), slow(242 frames) and slower (302 frames), according to thefeedback received during the pilot study.

The algorithm used for generating the layoutsis the Kamada-Kawai algorithm.75 To minimise thedisplacement of node position, chain-based anchoring isused between the successive layouts. In this approach,

the first frame of each network is generated using randomnode coordinates and each subsequent frame uses thenode coordinates of the previous frame as a startingpoint. This technique was used as it has been found tobe the most effective to generate these type of networksin previous studies.17 The layouts generated by the algo-rithm were improved by manually positioning someof the nodes to increase readability and retain contextbetween time frames. SoNIA34 was used to generate allthe visualisations and movies.

Web-based experiment system

An online web-based system was specifically designed torun the experiment. The system is compatible with allmajor browsers with Flash and Javascript enabled to watchthe movies and collect time measurements, respectively.

The visual representations were displayed on the left,while the question was displayed on the same page to theright. The static images were laid out in a 3 × 2 grid, witheach of the static images having the same size as the video(see Figure 3). Participants answered questions either byinputting a number in a text box, or using radio buttonsfor Likert scale questions. An option to skip a questionwas available if the participant was unable to answer. Forany skipped questions participants were asked to supplya reason for skipping the question. The questions of theexperiment were ordered using a Latin square design.

The interaction techniques of the video animation werelimited to play, pause and search for specific time posi-tion on the time bar. Pausing the video and clicking onthe timeline enabled participants to move directly to aspecific point in time. Participants were made aware ofthis functionality but no effort was made to promote thismode of interaction. Participants could also select theirspeed preference for the movie playback by clicking onthe speed mode under the movie (see Figure 4).

At the end of the experiment, each participant waspresented with a list of adjectives and asked to select thetop five words applicable to each representation. The listcontained 100 adjectives and was equally split betweenpositive adjectives and negative adjectives. The lists ofwords were randomised for each participant but kept thesame for each representation.

This qualitative data was gathered in an effort to under-stand the sentiments and impressions the participantsexperienced while conducting the experiment. Althougha representation mode might be more effective in termsof measurable quantities such as accuracy and timeliness,there are other less easily quantifiable aspects that mightprovide a reason for using alternative presentation modes.Furthermore, a checklist of terms tends to encouragefeedback more than simple open-ended questions onuser preferences and this was reflected in the number ofanswers received for the adjective list as opposed to theopen-ended question on the reason for representationpreference. The list of adjectives also attempts to under-stand more subtle reasons for a preference towards a

54




Figure 3: Web-based experiment system showing static image questions.

representation by prompting the users with a list ofemotions that they might not have necessarily thoughtabout expressing but know that they have experienced.

Adaptation of the lab System to MTurk

The original system that was developed for the labora-tory group study was adapted for MTurk. All the primarymaterials used in this experiment, images, videos, data setand experimentation system were kept the same. A minoraddition was made to the experiment system to allow theusers to pause the experiment. In a laboratory setting,the surrounding environment is designed to minimiseexternal distractions and users are visually monitored.When an experiment is being run over the internet in aremote location, the experimenter loses all control of thesurrounding environment and distractions. To minimisethe effect of distractions on the timing of the results inperforming the experiment, a pause button was added tothe experimental system to allow the user to pause thetimer. When the pause button is pressed the experimentmaterials are not displayed until the user presses theresume button to continue the experiment. The experi-ment still timed out after 30 min of inactivity, therefore ifsomeone abandoned the experiment for a long time thedata was discarded.

Participants

In the experiments different sets of participants were used.In the first experiment, the participants were seven males(average age: 29, min: 23) postgraduate research studentsin computer science, all of whom were familiar with nodelink diagrams. The student background was known andall the participants were previously screened before theexperiment in terms of suitability for the tasks.

In the second experiment, the participants were anony-mous workers on Mechanical Turk. From the collectedgender and age information, there were 26 females(average age: 29, min: 19, max: 57) and 33 males (averageage: 30, min: 17, max: 58). In this case, there was nocontrol over the participants and participants did notnecessarily have knowledge of node link diagrams, socialnetwork or computer graphs. As reported in other studiesa large proportion of MTurk workers have a high level ofeduction,66 therefore understanding the basic principlesbehind node link diagrams was probably not difficult.

The HITs submitted on MTurk were entered in fourbatches of 24 questions each, over a period of 11 days.Out of the 96 requests issued, 12 were rejected outrightand reissued for a total of 108 submissions. From the 108submissions, 48 submissions were discarded because theywere completed by the same person more than once.

55




Figure 4: Web-based experiment system showing animated image questions.

These multiple submissions, identified by the uniqueworker id, were completed by seven workers. In thiscase, only the first submission was taken into consider-ation and the others were discarded. From the numberof submissions only three people did not complete theexperiment in full. In order to have standardised resultsand retain the question ordering and within subjectsdesign, these three submissions were discarded. All thesubmissions were screened for obvious signs of gaming ofthe system such as answering quickly and inaccurately.After cleaning the data, the total number of participantsfor the second experiment were 57.

Laboratory group experimental method

The experiments for the group of computer scientistswere held in a classroom environment with each of theseven participants conducting the experiment simul-taneously. Before each experiment, a brief tutorial waspresented to explain the use of the system. A set of four

test questions, each with a different representation andtask type were given to make the participants familiarwith the system. Participants were encouraged to experi-ment with the system controls during these four examplequestions.

MTurk experimental method

The actual experiment platform was hosted on ourservers to ensure maximum control on the presentationof content, collecting accurate timing data per questionand control the question order to conform to the orig-inal Latin Square design. The MTurk HIT consisted of adescription of the task with a link pointing to the exper-iment system and a passcode to uniquely identify eachHIT. Upon finishing the experiment each participant wasgiven a code to input in MTurk to finish the HIT.

In order to keep in spirit with the small discernibletasks forming HITs on MTurk, we provided the optionfor the participants to finish the experiment early after

56




completing each batch of four tasks. Each batch of fourquestions was worth 0.10c, and there was a total of ninebatches. After each batch of four tasks was completed bythe participant, a screen showing the monetary rewardup to that point and asking if the user wants to continuewas displayed. In order to incentivise the participantto complete the entire experiment, the participant wasnotified of a US $1 bonus if the entire experiment wascompleted. If the participant opted out of the experimenta question asking the reason for opting out was presentedfollowed by the final qualitative questions. After theparticipant finished the experiment (or decided to stopprematurely), a passcode reflecting the level of comple-tion was provided. We did this to prevent publishing theentire experiment as one long task with a high rewardpossibly attracting workers that want to ‘game’ the systemand gain a ‘large’ some of money for a little reward.69

The MTurk platform does not provide a way to bundle agroup of HITs together as a HIT can only be either rejectedor approved. For this reason, all the HITs were publishedwith the minimum amount of money for each HIT (0.10c)and not with the total amount, as otherwise people whodo not complete the entire experiment will still get themaximum amount of money. The monetary balance basedon the completion of the experiment was transferred tothe participant in the form of a bonus in MTurk.

When conducting an experiment remotely the experi-menter has no control over the selection of participants,the surrounding environment and the computer setup ofthe participants. In order to ensure that the participantshad a computer setup that conforms to the minimumrequirements of the experiment, each remote participanthad to pass a screening task before being able to partic-ipate. The screening task text provided the basic infor-mation necessary to interpret node link diagrams andanswer the experiment questions. Each participant thenhad to pass a simple test that tested both the basic under-standing of node link diagrams and also ensured that eachparticipant had the minimum requirements to conductthe experiment (that is, watch a movie using Flash).

Results

In this section, we report on error rates, time to completetask, and descriptive preferences of the participants. Thepaired samples t-test was used to analyse both measures.

General observations

In the two experiments with the two different user groupsthe main results obtained are generally consistent. Interms of results, the first laboratory experiment with thesmaller computer science students user group acted asa prequel to the larger experiment conducted with ananonymous audience. Most of the results of the firstexperiment are indicative of the general trend, however

because of the small number of participants, these resultsare not statistically significant. In the second experimentthe same trends can be observed, but in this case most ofthe results are statistically significant. The performanceof both groups of different users suggests that throughoutmost of the experiments, questions answered using staticpresentation were faster and sometimes more accuratethan questions answered using animation. In general,this difference is stronger in measures related to timethan measures of accuracy.

In the laboratory experiment, questions with a staticmode of presentation were answered faster (M = 56.45 s)than those with an animated presentation (M = 62.22 s).This difference however is not significant. In the secondexperiment, the t-test did reveal a significant differencein time to complete tasks between static and animateddisplays, t(904) = 9.154, (P <0.001). This indicates thatthe average time to complete tasks using static repre-sentations (M = 35.16 s) was significantly lower thanthe average time to complete tasks using animation(M=47.55 s). The average time to complete is lower in thesecond experiment because the questions were simplifiedand in general were easier to answer. The experiment wasdesigned this way due to the remote experimentationplatform use and the anonymous possibly non specialisedparticipants, taking part in the second experiment.

The total accuracy in the first experiment was 85 percent, whereas the accuracy of the second experiment was86 per cent. The accuracy is sufficiently high in both casesto indicate that the participants were generally able toanswer correctly and the questions were not too difficult.Participants in the second experiment who might havehad no previous experience with node link diagrams werestill able to answer most of the questions correctly. Thesimilar accuracy rating of the participants suggests thatthe simplification of the questions for the second exper-iment using a non-specialised audience was appropriateand adequate. The number of errors made when animatedpresentation was used were higher (59 per cent of the totalin the first experiment and 56 per cent of the total in thesecond), and the difference was significant t(904)=2.660,(P <0.01).

Task differences

The 2 × 2 tasks for each group were cross tabulated andanalysed in terms of average number of errors and averagecompletion time. Once again both experiments reportedsimilar results, in that static images were more effective inboth time and accuracy. Figures 5 and 6 give an overviewof the results in graphical format and Tables 2–5 give theresults in tabular format.

Network overview tasks with no specified time periodThroughout the conditions tested in both experiments,the only measure that reported a positive difference for

57




Figure 5: Experiment 1 result overview.

Figure 6: Experiment 2 result overview.

Table 2: Average time to complete task – experiment 1

Average time to Time Representationcomplete Animated Static

Actor detail No specified time 41.10 36.31Specified time 56.28 53.15

Network overview No specified time 71.01 70.66Specified time 77.42 67.66

*Significant differences between averages.

Table 3: Average error per task – experiment 1

Average no of errors Time RepresentationAnimated Static


Network overview No specified time 0.21 0.28Specified time 0.31 0.13

animation was the accuracy measure for network overviewtasks with no specified time period (Hypothesis 1b) in thefirst experiment. In the first experiment, animated imagesM = 0.21 reported a lower error rate than static images

Table 4: Average time to complete task – experiment 2

Average time to complete Time RepresentationAnimated Static

Actor detail No specified time 62.83* 44.81*

Specified time 37.99* 32.37*

Network overview No specified time 48.14* 35.39*

Specified time 42.88* 29.65*

∗Significant differences between averages.

Table 5: Average error per task – experiment 2

Average no of errors Time RepresentationAnimated Static


Network overview No Specified Time 0.16* 0.11*

Specified time 0.06 0.03

∗Significant differences between averages.

M = 0.28, yet this difference is not significant. For thetime to respond, the means are almost equal (M=71.01 s)for animated and (M = 70.66 s) for static images and thedifference is also not statistically significant.

58




The only time the results between the experiments showdifferent trends is in this task. Network wide questionswithout a specified time period were answered faster usingstatic images (M = 35.39) than using animated movies(M = 48.14), t(226) = 3.815, P <0.001. Here, questionsanswered using animated movies had a higher error rateM=0.16 than those answered with static images M=0.11.In this case, the difference is also significant t(226)=1.95,P <0.05. The results obtained from the second experimentcontradict our first Hypothesis (1a and 1b), where anima-tion is believed to be faster and more accurate in tasks onnetwork overviews and no specified time.

Individual actors and no specified time periodIn the first experiment, the results show that static imageswere both faster (M = 36.31 s for static, M = 41.10 s foranimated) and had a smaller error rate (0.03 for static,0.04 for animated) than animated movies, however bothdifferences are not statistically significant.

In the second experiment, a significant differenceconfirming that static images (M = 44.81s) are faster thananimated movies (M = 62.83) is reported t(226) = 6.017,P <0.001. The difference in terms of error rate is notstatistically significant P <0.286. This result also contra-dicts our initial hypothesis related to speed (2a), but isinconclusive on the second part of the hypothesis relatedto accuracy (2b).

For tasks querying individual actors where the timecondition is specified, the difference is significant for thetiming measurement t(226)= 3.493, P <0.01) but not forthe error measurement.

Specific time period tasksIn the first experiment, both the response time (M=60.41sfor static, M = 66.85s for animated) and the error rate ofstatic images (M = 0.08 for static, M = 0.21 for animated)have better performance than the animated representa-tion, for tasks with a specific time period specified. Again,the results in this experiment are not significant.

In the second experiment, the result mirror the resultsin the first experiment but have different significancevalues. The tests revealed a lower time to respond in thecase of static visualisation (M = 30.19 s) when comparedto animated visualisations (M = 40.41 s), t(451) = 6.264,P <0.001. For the error rate measurement, the differ-ences between static (M = 0.07) and animated (M = 0.10)representations is statistically significant t(451) = 1.636,P <0.103. This result confirms our third Hypothesis 3aand 3b.

Network densityIn Hypothesis 4, we postulate that denser networkslead to more errors and take a longer time to complete.This hypothesis is supported in the second experimentfor the time to respond but not for the number oferrors. In the first experiment, both differences are not

statistically significant. In the second experiment, theaverage time to reply in the dense networks is M=47.57 swhereas in the lesser dense network it is M = 35.70 s,t(903) = −8.559′, P <0.001. In case of errors, the differ-ence is 0.13 in the lesser dense network and 0.14 in thedense network.

We also conduct an RM-ANOVA analysis on the timemeasure of density data of the second experiment, tostudy the relationship between density and representa-tion. From this analysis, we observe an effect of bothdensity F(1448) = 70.48, P <0.001 and representationF(1448) = 64.49, P <0.001, but there was no significantinteraction between density and representation.

Participants’ comments and preferences

At the end of the exercise, participants were asked tocomment on both visual representations and indicatetheir preference between the static and animated displays.From the participants who answered that question, sixpreferred animations and 22 preferred static images.Perhaps, the most informative comment on this canbe expressed by one of the participants who wrote ‘Ireally liked the videos but they were difficult to use ifyou are being timed, I felt a little rushed with them andturned them to the slowest speed’. One other participantremarked that he preferred ‘Static mostly, for accuracy.But the video was useful for following a single nodearound more easily’. This was in line with our originalintuition, yet the quantitative results did not confirm it.

The main reasons in favour of static displays included;‘easier to access information selectively’, ‘easier compar-isons over time periods’, ‘easy to verify results’. Someparticipants reported that they had to stop the animationand manually browse through the static frames to registerthe change. This behaviour was also noticed during therunning of the laboratory experiment. In the words ofone participant, ‘when trying to compare the transitionbetween two weeks, it was very useful to be able to flipback and forth between two frames in the timeline andhave them presented one on top of the other as in theanimation.’

Some MTurk workers also commented positively onthe nature of the HIT and expressed encouragementtowards publishing other similar HITs. One participantalso remarked on the novel structure of the HIT; ‘Hit wasfairly enjoyable to complete. I like how it was broken upinto four parts each, and I was able to stop and collectthe money I earned without having to do all the ques-tions if I didn’t want to. More hits broken up like thatwould be great!’. Some workers were even interested inthe experiment and asked questions on the purpose ofthe experiment and the outcome (‘I wonder what youwere really testing. It was interesting stuff anyway.’).It would probably be a good idea for future experiments toprovide more information on the publications resultingfrom the experiment to create a stronger bond with theparticipants.

59




Figure 7: Word cloud for adjectives selected for static repre-sentations.

Figure 8: Word cloud for adjectives selected for static repre-sentations.

In order to get a general impression of the senti-ments that the visualisations elicited from the partic-ipants, each participant was asked to choose the topfive words from a list of 100 words equally dividedbetween positive or negative sentiments. Figures 7 and8 show the list of static and animated words chosen bythe participants respectively. In the figures, the size ofthe words corresponds with the number of times theword was chosen, whereas the locations of the words arerandom.

The sentiments expressed for the respective repre-sentations are evident from the two diagrams. Mostparticipants felt that animated images were harder andmore difficult to use than the static alternatives. Partic-ipants also remarked on the time consuming natureof animation, a measure which was clearly reflected inthe quantitative results. Time is of essence to workerson Mechanical Turk, since the longer a task takes theless money per HIT they can make, therefore thissentiment is well understood. Conversely, when onelooks at the sentiments expressed for static images,

opposite words to the ones used for describing anima-tion can be observed. The visualisation also clearly illus-trates that participants preferred static images for theirefficiency.

Discussion

Similarly to other experimenters who were the first to testanimation in a field that considered animation to be thenatural way of representing temporal information,50 wewere surprised at the overwhelming evidence in favourof static images. From the results, it is clear that for theseexperiments the static representations allowed partici-pants to perform the tasks faster, and in some cases moreaccurately than animated representations. While for sometasks this was expected from the original hypotheses,most of the hypotheses on the benefits of animationwere contradicted. In this discussion, we shall attempt tooutline and understand some possible reasons for this.

One possible reason is that interaction with animateddisplays was very limited, therefore the full potentialof using animation was not exploited. Previous studiesparticularly in algorithm animations52 and our previousexperiences43 have observed that interaction withanimated displays is crucial during analysis. In the exper-iments, animation interaction was limited to pause, seekand play operations on the movie, rather than the interac-tions typically available in a complete interactive system.Time constraints and a desire to limit confounding factorsin our experiments limited the use of such a complexsystem in the experiments.

When animating one is effectively interpolatingbetween many different frames creating many in-betweenor artificial graph layouts which in reality do not exist. Ifa user is interacting with these animations by starting andstopping the animations the search space is much largerbecause of the added interpolated frames. In this case,when a user is searching a video it takes much longerthan searching a static image that quickly provides accessto the final solution of the graph. A possible suggestionin this case, considering that some participants admittedthat they used the animation like a static image bybrowsing the video, would be to let the user flip betweenkeyframes and only animate when the play button ison. This means adding an additional interaction param-eter to allow the user to browse between timeframes,slideshow style. Alternatively, keyframe locations can behighlighted on the video time line.

Another possible reason is that most of the tasks in theexperiment were low level topological tasks that requirecareful analysis of the network instead of a general overallperspective. In our first hypothesis, we believed thatanimation will be beneficial for general overview tasks,however this hypothesis was contradicted in both timeand accuracy measures. While the network tasks werereal overview tasks the questions might have been toospecific, requiring the participants to look carefully for

60




answers instead of trying to identify a general trend.General overview questions might also benefit from alayout scheme that emphasises the trend with respect tothe position of the nodes.

One of the participants commented that questionsanswered using animation felt like trying to hit a movingtarget. This comment might give some insight why oursecond hypothesis, where we thought that animation willbe beneficial in tracking a single node, was contradicted.In our animation all the nodes were moving throughoutthe whole animation, thus possibly distracting the atten-tion of the participants. One needs to be very carefulwhen using animation, and it is probably better if anima-tion is used sparingly rather than throughout a wholevisualisation. Selectively animating parts of the networkto exploit the pre-attentive nature of motion can proveto be more effective then simply animating the wholenetwork.

Interestingly, in our tests on density, the number oferrors in the denser network was not significantly higherthan the number of errors in the lower density network.One aspect of density that was not tested in this experi-ment was time density. Both networks spanned a periodof six weeks, which resulted in 6 individual images andsix key frames of video. It will be interesting to test anima-tion on a higher time density network, say for examplespanning a period of 20–200 time points. In such a case,looking at 20–200 different images of the network mightbe impractical as static images consume plenty of screenspace, whereas animation or a flipbook may be easierto control as they can be contained in one screen. Oneproblem with this sort of experiment would be the timerequired to run the experiment and the necessary precau-tion not to cognitively overload participants with a taskthat is too difficult. In this respect, we believe that a moreethnological type of study is probably a more realisticapproach to study the analyst’s toolbox.

The quantitative results from the measurements supportthe comments written by the participants in the finalsurvey. This result is noteworthy because it informallyvalidates the positive benefits of static images. The majorcomplaint for animation, as can be clearly seen from thegraphic in Figure 8, is that animation is hard to use. Inour laboratory-based studies, some of the participants feltthat animation took more time to become familiar with.From our word analysis, we found that participants in thelaboratory tended to be more careful in selecting negativesentiments than anonymous participants, especially withrespect to animation.

Although for most tasks static representations weregenerally more accurate, the results were for the mostpart not statistically significant. We believe that thereare still possibilities for using animation as an analyticaltool, especially if animation is used selectively and withcare. For example, animation can be useful to illustratedynamics when presenting solutions, aided by addi-tional voice over explanations of the network dynamics.Participants used positive sentiments such as ‘exciting’,

‘engaging’ and ‘fun’ when describing animated represen-tations.

Using remote web-based services for experimentation

Web-based experimentation and web-based workers givethe researcher the opportunity to reach a diverse partic-ipant group that extends beyond the typical sample ofstudents typically available within a university context.This setup is a good stepping stone to start generalisingcertain types of experiments in a possibly more authenticsetup than a laboratory space. Although we are still ata very early stage of understanding the benefits, draw-backs and possibilities that these web-based platformscan provide for research, we are encouraged by the resultsobtained and the quality of work obtained from theparticipants using the site. We caution other researchersto structure their experiments carefully and to considerany novelty effect in future uses of such a platform inexperimental design.

From our experience, we found that people who startedthe experiment were willing to complete the whole exper-iment, even if they were given the option to opt out andclaim money at regular intervals. It will be interesting infuture studies to measure the effect of the size of the bonuson the completion rate of the experiments. In some cases,the same participants completed the experiment severaltimes in a row, giving the impression that workers on theopposite side are working in a production line. Workerson the site are accustomed to the repetitive nature of thetasks typically found on the site, which makes runninglonger experiments practical. In hindsight, the number oftasks run in the experiment could have been extended forthe web-based experiment.

One area that we were sceptical about during the designof the experiment with Mechanical Turk was in the collec-tion of timing data from a remote experiment withoutthe ability to control distractions in the environment.In reality, the timing data collected was well within theexpected range of task time completion and there wereno obvious instances where the time kept on running fora long time that was disproportionate to the task length.We can conjecture that the fact that the participants weremade aware of the importance of timing and the addi-tion of the pause button helped to ensure more accuraterecords.

Apart from a few users who completed the experimentmultiple times we did not find any instances of trying togame the system by answering quickly and inaccurately.We were careful to take into consideration the recommen-dations by other researchers who detected instances ofgaming, such as keeping the initial reward relatively lownot to attract undesirable participants and pre-screeningthe workers. It could also be the case that workers arelearning that if they try to abuse the system in this waytheir employees can easily detect their ploy and reject thetasks accordingly.

61




We believe that there is promise in web-based experi-ments and that this experiment platform and traditionallaboratory experiments can complement each other. Theobvious benefits of participant resourcing, low cost andrelatively fast data collection times make remote websystems attractive for the researcher. Naturally, the resultsobtained from this platform need to be interpreted inthe context of the platform and care must be exercisedin ensuring that the data is clean before being used foranalysis.

Future Work and Conclusions

When interpreting networks it is impossible to becompletely independent of the graph layout used togenerate the images. As previous studies have shown23,37

the graph layouts have a significant effect on the inter-pretation and understanding of the graph structure. Forthis experiment, we use a layout algorithm that was orig-inally designed for static networks, yet there are still veryfew graph layout algorithms designed specifically fordynamic networks, that are available in social networkanalysis toolkits. Furthermore, in these experiments thedynamic network visualisation was restricted to nodelink diagrams. There is further scope to look at differentvisual representations for dynamic network data thatextends beyond the current paradigms of network visual-isation, perhaps even being designed with an eye towardsexploiting the power of motion in subtler ways.

As we have discussed, there is scope for extendingthis study to test other criteria such as time density andperhaps investigate the difference between structural andanalytical question formulation for tasks. Furthermore,more interactive features can be added to the animateddisplays to study the effect of interaction on networkanimation. Improvement in dynamic network layoutswill undoubtedly be of great help to dynamic networkvisualisation irrespective of the representation mediumused to visualise the network.

The aim of this user study was to understand thestrength of static images and animated movies to analysedynamic social networks. We found that static imagesprovide better performance than their animated counter-parts. The results obtained though do not justify givingup further studies on animated representations. Never-theless, improvements and further studies are requiredto understand better the beneficial uses of animation inapplied social network analysis.

References

1 Doreian, P. and Stokman, F. (1997) Evolution of Social Networks.Amsterdam: Gordon and Breach Publishers.

2 Christakis, N. and Fowler, J. (2008) The collective dynamics ofsmoking in a large social network. New England Journal of Medicine358(21): 2249.

3 van Duijn, M., Zeggelink, E., Huisman, M., Stokman, F. andWasseur, F. (2003) Evolution of sociology freshmen into a friend-ship network. Journal of Mathematical Sociology 27(2): 153–191.

4 Xu, J., Marshall, B., Kaza, S. and Chen, H. (2004) Analyzing andvisualizing criminal network dynamics: A case study. Proceedingsof the Second Symposium on Intelligence and Security InformaticsSpringer.

5 Moreno, J.L. (1941) Foundations of sociometry: An introduction.Sociometry 4(1): 15–35.

6 Freeman, L.C. (2000) Visualizing social networks. Journal of SocialStructure 1.

7 Eades, P., Marks, J., Mutzel, P. and North, S. (1997) Graph-drawingcontest report. In: G. Dibattista (ed.), Graph Drawing. Berlin,Heidelberg: Springer, pp. 423–435.

8 Fekete, J., Grinstein, G. and Plaisant, C. (2004) InfoVis 2004Contest: The History of InfoVis. IEEE, http://www.cs.umd.edu/heil/iv04contest/.

9 Grinstein, G., Plaisant, C., Laskowski, S., O’ Connell, T., Scholtz,T., and Whiting, M. (2008) Vast 2008 Challenge: IntroducingMini-challenges. Visual Analytics Science and Technology, 2008.VAST ’08. IEEE Symposium.

10 Erten, C., Harding, P., Kobourov, S., Wampler, K. and Yee,G. (2003) GraphAEL: Graph animations with evolving layouts.Lecture Notes in Computer Science 2912: 98–110.

11 Bender-deMoll, S. and McFarland, D. (2006) The art and scienceof dynamic network visualization. Journal of Social Structure 7(2).

12 Leydesdorff, L., Schank, T., Scharnhorst, A. and De Nooy, W.(2008) Animating the development of Social Networks overtime using a dynamic extension of multidimensional scaling. ElProfesional de Informacion 17(6): 611–626.

13 Farrugia, M. and Quigley, A. (2009) TGD: Visual data explorationof Temporal Graph Data. In: K. Borner and J. Park (eds.)Visualization and Data Analysis 2009, Vol. 7243, No.1, San Jose,CA: SPIE, p.11.

14 Archambault, D. (2009) Structural differences between two graphsthrough hierarchies. Proceedings of Graphics Interface 2009.

15 Powell, W., White, D., Koput, K. and Owen-Smith, J.(2005) Network dynamics and field evolution: The growth ofinterorganizational collaboration in the life sciences. AmericanJournal of Sociology 110(4): 1132–1205.

16 Pajek. http://pajek.imfm.si/doku.php.17 Moody, J., McFarland, D. and Bender-deMoll, S. (2005) Dynamic

network visualization 1. American Journal of Sociology 110(4):1206–1241.

18 Wasserman, S. and Faust, K. (1994) Social Network Analysis.Cambridge UK.

19 Herman, I., Melançon, G. and Marshall, M. (2000) Graphvisualization and navigation in information visualization: Asurvey. IEEE Transactions on Visualization and Computer Graphics6(1): 24–43.

20 Di Battista, G., Eades, P., Tamassia, R. and Tollis, I. (1999) GraphDrawing; Algorithms for the Visualization of Graphs. Upper SaddleRiver, NJ: Prentice Hall.

21 Kaufmann, M. and Wagner, D. (2001) Drawing Graphs: Methodsand Models. Springer-Verlag: London, UK.

22 Di Battista, G., Eades, P., Tamassia, R. and Tollis, I. (1994)Algorithms for drawing graphs: An annotated bibliography.Computational Geometry: Theory and Applications 4(5): 235–282.

23 Purchase, H. (1997) Which aesthetic has the greatest effect onhuman understanding? In: G. Di Battista (ed.) Graph Drawing.Lecture notes in Computer Science, Vol. 1353. Berlin; Heidelberg:Springer, pp. 248–261.

24 Ware, C., Purchase, H.C., Colpoys, L. and McGill, M.(2002) Cognitive measurements of graph aesthetics. InformationVisualization 1(2): 103–110.

25 Misue, K., Eades, P., Lai, W. and Sugiyama, K. (1995) Layoutadjustment and the mental map. Journal of Visual Languages andComputing 6(2): 183–210.

26 Purchase, H., Hoggan, E. and Görg, C. (2007). How importantis the ‘mental map’? – An empirical investigation of a dynamicgraph layout algorithm. Lecture Notes in Computer Science 4372:184.

62




27 Brandes, U. (2001). Drawing on physical analogies. Lecture Notesin Computer Science 2025: 71–86.

28 Brandes, U. and Wagner, D. (1997) A bayesian paradigm fordynamic graph layout. Lecture Notes in Computer Science 1353:236–247.

29 Frishman, Y. and Tal, A. (2008) Online dynamic graph drawing.IEEE Transactions on Visualization and Computer Graphics 14:727–740.

30 Diehl, S., Görg, C. and Kerren, A. (2001) Preserving the MentalMap Using Foresighted Layout. Proceedings of the Joint Eurographics-IEEE TCVG Symposium on Visualization VisSym ’01, Ascona,Switzerland, Wien, Austria: Springer Verlag, pp. 175–184.

31 Lee, Y., Lin, C. and Yen, H. (2006) Mental Map Preserving GraphDrawing Using Simulated Annealing. Proceedings of the 2006Asia-Pacific Symposium on Information Visualisation-Volume 60,Australian Computer Society, p. 188.

32 Davidson, R. and Harel, D. (1996) Drawing graphs nicelyusing simulated annealing. ACM Transactions on Graphics 15(4):301–331.

33 Dwyer, T. and Gallagher, D. (2004) Visualising changes in fundmanager holdings in two and a half-dimensions. InformationVisualization 3(4): 227–244.

34 Bender-deMoll, S. and McFarland, D. (2006) The art and scienceof dynamic network visualization. Journal of Social Structure 7(2)

35 Bridgeman, S. and Tamassia, R. (2002) A user study in similaritymeasures for graph drawing. Lecture Notes in Computer Science 6(3):225–254.

36 Friedrich, C. and Eades, P. (2002) Graph drawing in motion.Journal of Graph Algorithms and Applications 6(3): 353–370.

37 Blythe, J., McGrath, C. and Krackhardt, D. (1996) The effectof graph layout on inference from social network data.Proceedings of the Symposium on graph Drawing GD ’95. London:Springer-Verlag, pp. 40–51.

38 Tversky, B., Morrison, J. and Betrancourt, M. (2002) Animation:can it facilitate?: International Journal of Human-Computer Studies57(4): 247–262.

39 McFarland, D. (2001) Student resistance: How the formal andinformal organization of classrooms facilitate everyday formsof student defiance1. American Journal of Sociology 107(3):612–678.

40 Brandes, U., Kääb, V., Löh, A., Wagner, D. and Willhalm, T. (2000)Dynamic WWW Structures in 3D. Journal of Graph Algorithms andApplications 4(3): 183–191.

41 Ke, W., Börner, K. and Viswanath, L. (2004) Major informationvisualization authors, papers and topics in the ACM library.Proceedings of the IEEE Symposium on Information Visualization.Washington DC: IEEE Computer Society, p. 216.1.

42 Lee, B., Czerwinski, M., Robertson, G. and Bederson, B. (2004)Understanding Eight Years of Info Vis Conferences Using PaperLens.Washington DC: IEEE Computer Society, p. 216.3.

43 Farrugia, M. and Quigley, A. (2008) Cell phone mini challenge:Node-link animation award animating multivariate dynamicsocial networks. IEEE Symposium on Visual Analytics Science andTechnology, 2008. VAST ’08; October, IEEE, pp. 215–216.

44 Correa, C.D., et al. (2008) Cell phone mini challenge award:intuitive social network graphs visual analytics of cell phonedata using Mobivis and Ontovis. IEEE Symposium on VisualAnalytics Science and Technology, 2008. VAST ’08; October. IEEE,pp. 211–212.

45 Perer, A. (2008) Using socialaction to uncover structure in socialnetworks over time. IEEE Symposium on Visual Analytics Scienceand Technology, 2008. VAST ’08; October. IEEE, pp. 213–214.

46 Perer, A. and Shneiderman, B. (2006) Balancing systematic andflexible exploration of social networks. IEEE Transactions onVisualization and Computer Graphics, 12: 693–700.

47 McGrath, C. and Blythe, J. (2004) Do you see what i want youto see? the effects of motion and spatial layout on viewers’perceptions of graph structure. Journal of Social Structure 5(2).

48 Ware, C. and Bobrow, R. (1985) Motion to support rapidinteractive queries on node-link diagrams. ACM Transactions onApplied Perception (TAP) 1(1): 3–18.

49 Brown, M. and Sedgewick, R. (1985) Techniques for algorithmanimation. IEEE Software 2(1): 28–39.

50 Stasko, J., Badre, A. and Lewis, C. (1993) Do AlgorithmAnimations Assist Learning?: An Empirical Study and Analysis.Proceedings of the INTERACT’93 and CHI’93 Conference on HumanFactors in Computing Systems ACM CHI ’93; Amsterdam; TheNetherlands; New York: ACM, pp. 61–66.

51 Kehoe, C., Stasko, J. and Taylor, A. (2001) Rethinking theevaluation of algorithm animations as learning aids: Anobservational study. International Journal of Human ComputerStudies 54(2): 265.

52 Lawrence, A., Badre, A. and Stasko, J. (1994) EmpiricallyEvaluating the Use of Animations to Teach Algorithms. Proceedingsof the 1994 IEEE Symposium on Visual Languages, Citeseer,pp. 48–54.

53 Rosling, H. (2006) Debunking myths about the ‘third world’,.http://www.gapminder.org/videos /ted-talks/hans-rosling - ted-2006-debunking-myths-about-the-third-world/.

54 Rosling, H. (2007) The seemingly impossible is possible,http://www.gapminder.org/videos/ted-talks/hans-rosling-ted-talk-2007-seemingly-impossible-is-possible/

55 Robertson, G., Fernandez, R., Fisher, D., Lee, B. and Stasko,J. (2008) Effectiveness of animation in trend visualization.IEEE Transactions on Visualization and Computer Graphics 14(6):1325–1332.

56 Yee, K.-P., Fisher ,D., Dhamija, R. and Hearst, M. (2001) AnimatedExploration of Dynamic Graphs with Radial Layout. Proceedingsof the IEEE Symposium on Information Visualization. INFOVIS ’01.Washington DC: IEEE Computer society, p. 43.

57 Heer, J. and Robertson, G. (2007) Animated transitions instatistical data graphics. IEEE Transactions on Visualization andComputer Graphics 13(6): 1240–1247.

58 Elmqvist, N., Dragicevic, P. and Fekete, J.-D. (2008) Rollingthe dice: Multidimensional visual exploration using scatterplotmatrix navigation. IEEE Transactions on Visualization and ComputerGraphics 14(6): 1148–1539.

59 Snijders, T., Steglich, C., Schweinberger, M. and Huisman,M.(2007) Manual for SIENA, version 3.1.

60 Ware, C. (2004) Information Visualization: Perception for Design.San Francisco, CA: Morgan Kaufmann.

61 UCINET. http://www.analytictech.com/ucinet/ucinet.htm.62 Batagelj, V. and Mrvar, A. (1998) Pajek-program for large network

analysis. Connections 21(2): 47–57.63 Brandes, U. and Wagner, D. (2003) Visone: Analysis and

visualization of social networks. Graph Drawing Software. Secaucus,NJ: Springer-Verlag New York, pp. 321–340.

64 Turk, M. http://www.mturk.com.65 Ipeirotis, P. (2010) Demographics of Mechanical Turk, http://behind-

the-enemy-lines.blogspot.com/2010/03/new-demographics-of-mechanical-turk.html.

66 Ross, J., Irani, L., Silberman, M., Zaldivar, A. and Tomlinson, B.(2010) Who are the Crowd-workers?: Shifting Demographicsin Mechanical Turk. Proceedings of the 28th of the InternationalConference Extended Abstracts on Human Factors in ComputingSystems ACM, CHI EA’10; Atlanta, GA. New York: ACM, pp.2863–2872.

67 Kittur, A., Chi, E., and Suh, B. (2008) Crowdsourcing user studieswith Mechanical Turk. Proceeding of the Twenty-Sixth AnnualSIGCHI Conference on Human Factors in Computing Systems ACM,CHI ’08; Florence, Italy. New York: ACM, pp. 453–456.

68 Callison-Burch, C. (2009) Fast, cheap, and creative: EvaluatingTranslation quality using Amazon’s Mechanical Turk. Proceedingsof the 2009 Conference on Empirical Methods in NaturalLanguage Processing; Association for Computational Linguistics,pp. 286–295.

69 Heer, J. and Bostock, M. (2010) Crowdsourcing graphicalperception: Using mechanical Turk to assess visualization design.Proceedings of the 28th of the International Conference ExtendedAbstracts on Human Factors in Computing Systems, CHI ’10; Atlanta,GA; New York: ACK, pp. 203–212.

70 Cleveland, W. and McGill, R. (1984) Graphical perception:Theory, experimentation, and application to the development ofgraphical methods. Journal of the American Statistical Association79(387): 531–554.

63




71 Stone, M., Bartram, L. and Consulting, S. (2009) Alpha, contrastand the perception of visual Metadata. Sixteenth Color ImagingConference: Color Science and Engineering Systems, Technologies andApplications, Vol. 16, November 2008, Portland, OR: The Societyfor Imaging Science and Technology.

72 Plaisant, C., Lee, B., Parr, C., Fekete, J. and Henry, N. (2006) Tasktaxonomy for graph visualization. Beyond Time and Errors: NovelEvaluation Methods for Information Visualization (BELIV 06): 82–86.

73 McPherson, M., Smith-Lovin, L. and Cook, J. (2001) Birds of afeather: Homophily in social networks. Annual Review of Sociology27(1): 415–444.

74 van Duijn, M., Zeggelink, E., Huisman, M., Stokman, F. andWasseur, F. (2003) Evolution of sociology freshmen into a friend-ship network. Journal of Mathematical Sociology 27(2): 153–191.

75 Kamada, T. and Kawai, S. (1989) An algorithm for drawing generalundirected graphs. Information Processing Letters 31: 7–15.

64



effective temporal graph layout: a comparative study of animation

Documents