A survey on Information Visualization in light of Vision

and Cognitive sciences

Corresponding author: Jose F Rodrigues-Jr, junio@icmc.usp.br

Luciana A M Zaina+,

Maria C F de Oliveira,

Bruno Brandoli,

and Agma J M Traina

May 7, 2018

Abstract

Information Visualization techniques are built on a context with many factors related

to both vision and cognition, making it difficult to draw a clear picture of how data

visually turns into comprehension. In the intent of promoting a better picture, here,

we survey concepts on vision, cognition, and Information Visualization organized in a

theorization named Visual Expression Process. Our theorization organizes the basis of

visualization techniques with a reduced level of complexity; still, it is complete enough

to foster discussions related to design and analytical tasks. Our work introduces the fol-

lowing contributions: (1) a Theoretical compilation of vision, cognition, and Information

Visualization; (2) Discussions supported by vast literature; and (3) Reflections on visual-

cognitive aspects concerning use and design. We expect our contributions will provide

further clarification about how users and designers think about InfoVis, leveraging the

potential of systems and techniques.

Keywords— Information Visualization, InfoVis, Vision Science, Cognitive Science

1

arX

iv:1

505.

0707

9v2

[cs

.GR

] 1

3 M

ay 2

016

1 Introduction

Understanding why and how visual representations work is an important issue addressed in

many works on Information Visualization (Dill et al. 2012)(Thomas & Cook 2005). On the track

of this issue, in this work, we assume that well-designed visualizations must stimulate visual

and cognitive processes in a way that reasoning is amplified. The effective promotion of such

reasoning depends on principles mastered in the sciences of vision and cognition: vision is the

gate through which information derived from computer graphics reaches the brain; cognition

refers to the processing that is induced by such graphics. Vision and cognition are closely

intertwined, a fact to be considered in the design of visualizations. Accordingly, understanding

Information Visualization (InfoVis for short) in light of these sciences may improve design

principles that, usually, are performed intuitively. To this end, we review the steps that take

place during vision-cognition phenomena when the goal is data analysis. In survey fashion, we

compile the literature introducing the following contributions:

• Theoretical compendium: we draw a descriptive relationship between vision, cognition,

and Information Visualization;

• Discussions: we debate our rationalizations over extensive literature;

• Reflections: we provide study cases to revisit design practices.

We organize visualization concepts aiming at the needs pointed out by Johnson et al. (2006),

who recommend the characterization of how and why visualizations work, and by Scaife &

Rogers (1996), who stress the importance of the cognitive aspects underlying visualizations.

Furthermore, we review principles for visual representations that, according to Card et al.

(1999), are an initial step towards more effective visualization techniques, a demand defended

by Wong et al. (2012).

We note that, while we build an association between Information Visualization, vision, and

cognitive sciences, we do not reach a definitive settlement. This is because neither vision nor

cognition are yet fully understood. Rather, constrained to the current state of the art, we

introduce an organizational process that discusses use and design.

2

2 Related work

The literature presents several models that lay the basis to Information Visualization. Bertin

(1977/1981) introduced the concept of deriving visual structures from a set of fundamental

components. Cleveland & McGill (1984), and Mackinlay (1986), conducted studies on the use-

fulness of visual patterns in the form of frameworks for design. Card et al. (1999) follow Bertin

by discussing the importance of the spatial substrate. Keim (2002) suggests a taxonomical

space for quick referencing; Shneiderman (1996) reasons about the possibilities of visualization

and interaction; and Chi (2000) focuses on data transformations. In the realm of design, Buga-

jska (2003) deals with spatial and abstract visualizations considering the design guidelines of

Tweedie (1997). In the line of works that reflect about the visualization field, van Wijk (2006)

systematically discusses visualizations based on a cost-oriented analysis; and Green et al. (2009)

research many cognitive and perceptual aspects (Rensink 2000) to build a model and a set of

guidelines for design. Patterson et al. (2014) introduces a framework based on vision and cog-

nition sciences, similar to ours, focusing on top-down processes. Comparatively, we conduct a

complementary bottom-up approach; we translate vision-cognition phenomena into an original

vocabulary that may bring such science closer to the design practice.

To organize our compilation on vision and cognitive sciences, we depart from the Visualiza-

tion Pipeline of Card et al. (1999) to draw the Visual Expression Process, a sequence of events

delineated by the possibilities of the visual-cognitive interplay. According to our organization,

(1) vision phenomena (pre-attentive stimuli) determines a map of potential interesting objects.

Then, attentive selection concentrates on one single element, part of a set of (2) analytical

perceptions. Such perceptions go through a pattern-matching process that turns them into

(3) abstract patterns that, in working memory, support cognition in combination with domain

knowledge originated from long-term memory. Finally, leading to (4) cognitive decision sup-

port. Nevertheless, our process does not explain the intercourse between vision, cognition, and

Information Visualization; this is not feasible considering the current knowledge, and neither

it would fit in a single article.

The rest of the paper is organized as follows. Section 3 draws a detailed panorama of

vision and cognition in the realm of Information Visualization. Section 4 introduces the Visual

Expression Process, our organizational theorization; and the last section presents conclusive

remarks.

3

3 Concepts on Vision, Cognition, and Visualization

According to Vision Science, the visual process has two stages, namely, the parallel extraction

of low-level properties, called pre-attentive processing, followed by a slower detailed scan. The

first stage promotes the major benefit of visualizations, that is, improved data comprehension

(Triesman 1985). Meanwhile, the second stage addresses conventional reading practices that

do not contribute towards faster cognition, but that are necessary for further analysis. In fact,

Ware (2004) states that understanding what is processed pre-attentively is probably the most

important contribution that Vision Science can bring to visualization.

This two-stage process might be the underlying motivation for the broadly referenced “visu-

alization mantra”: overview first, zoom and filter, then details on demand (Shneiderman 1996)

– Schneiderman came to this conclusion by means of intuition and empirical experiments, never

drawing the vision/cognitive reasons of why this is the case. We discuss this process in the

following sections according to our process, presented in section 4.

3.1 Maps of Saliences

In the early stages of vision, the brain deals with the problem of casting potential elements of

interest; regions of the scene that should be considered for cognition. To cope with that, a com-

plex process takes place so that certain characteristics pop out to the eyes. These characteristics

appear as saliences over the scene. They define the so-called maps of saliences, exemplified in

Figure 1; the first element of our theoretical organization.

Figure 1: (a) Example of a map of saliences over the Gapminder tool. In the scene, the bubbleswith outstanding features stand for saliences corresponding to countries of interest. (b) Theresult is a map of potential targets that pop out due to their color, or their shape.

The principle of salience is to reinforce the perception of the areas in the scene whose visual

4

properties contrast with those of their surroundings (Itti & Koch 2001). It is a process that

considers different visual aspects, such as position and color, and that ends as it combines them

onto a single scene (Nothdurft 2000).

The occurrence of salient features stems from their interplay with other stimuli, depending

on a context that favors conspicuousness. The brain is highly trained to detect such configura-

tions, being able to track salient features in parallel, in real-time, and covering the entire visual

field. There is also evidence that this ability is strongly influenced by the task-at-hand, in a

top-down (cognition-to-vision) process (Navalpakkam & Itti 2007) (Patterson et al. 2014).

3.2 Attention and Attentive Selection

Although the brain perceives visual targets simultaneously in a map of saliences, it cannot

process all of them in parallel. This is considered a prohibitively computational task even

to the most sophisticated brains (Tsotsos 1991). Primates and other animals handle this by

restricting the consideration of the objects presented to their eyes: their vision concentrates

on small regions considering objects one after the other. This is a serialization process ruled

by what is called attention. In other words, once a map of candidates is ready, it is necessary

to “filter out” one of these candidates for attention. Attention, here, occurs in accordance

with the task that the user is performing; the task determines the information demands and,

consequently, what visual stimuli should be extracted from the scene. Trial and error is part of

the process, which is iterative.

This process of filtering has been modeled as a pyramidal neuronal structure – named

selective tuning model (Cutzu & Tsotsos 2003) (Essen et al. 1992). This theoretical model

predicts a broad layer of neurons in its first level, narrowing down as it advances to upper layers.

The layers intercommunicate through feed-forward and feedback connections and, according to

Cutzu & Tsotsos (2003) (Tsotsos et al. 1995), a pyramid of neurons successively performs three

stages of processing, illustrated in Figure 2: (a) bottom-up feed-forward, (b) top-down winner-

take-all feedback (Lippmann 1987), and (c) bottom-up straight path. This process explains

what is broadly referenced as filtering or attentive selection.

Following this widely-accepted model, it is worthy to note that, as described, filtering occurs

according to both bottom-up and top-down processes. The bottom-up process depends on the

visual stimuli, while the top-down process depends on the task at hand and is prioritized (Wolfe

1994). Each process influences the other iteratively. Actually, according to Patterson (2012),

the entire visual-cognition interplay in influenced by top-down mechanisms that emanate from

5

working memory, but that may be highly influenced by long-term memory (Woodman et al.

2013). In this work, we focus on the bottom-up process, but there are works that explore

higher-level factors related to the top-down process (Patterson et al. 2014).

Figure 2: Three-stage pyramidal visual selection: (a) bottom-up feed-forward, (b) top-downwinner-take-all feedback (Lippmann 1987), and (c) bottom-up straight path.

3.3 Cognition, Memory and Vision

After a target is selected, it is potentially useful for “details on demand”, or cognition. In

general terms, cognition refers to the acquisition or use of knowledge Cohen (1985), and may

occur analytically (conscious and slower) or intuitively (automatic and fast), as defended by the

dual system theory (Evans 2008)(Evans & Stanovich 2013). Cognition is of special importance

in data visualization as it supports analogical reasoning (Patterson et al. 2009). That is,

the transfer of inferences from a relationship of elements in one domain (the analogue) to a

relationship of elements in another domain (the target). In any case, cognition is intermediated

by the memory system. The relationship between memory and cognition is studied by works

on Cognitive Architectures, such as ACT-R (Buttner 2010) and Soar (Young & Lewis 1999).

In Soar and other theories, the structural configuration of memory roughly reflects the model

of Baddeley & Hitch (1974). Working memory includes three components: the central executive

module, the phonological loop, and the sketchpad. The central executive module determines

the attention focus, guiding the visual system, for example, by top-down biasing the pyramidal

selection mechanism. The phonological loop stores information related to sound. The sketchpad

(also known as Visual Short-Term Memory – VSTM 1) is associated with the maps of saliences

1although, this is not a consensus

6

discussed in Section 3.1, storing information related to space and to visual features.

Following these lines, memory comes to be the main element in supporting cognition as it

allows complex mental operations. Consequently, it also supports the practice of data visual-

ization. Miller’s Law (Miller 1956) states that the memory is limited to 7+/-2 elements; more

recent, and accepted, works state that this limit is at 4 chunks of elements – a chunk refers

to the grouping of elements into larger units based on their meaning (Cowan 2010). Alvarez

& Cavanagh (2004) discuss memory limits considering the nature of the elements to be re-

membered, demonstrating that, despite the consensus of a (very) limited resource, there is not

an ultimate conclusion about the topic. In any case, these limitations are severe because the

greater the capacity of an individual’s memory, the more information she/he has available for

solving problems (Just & Carpenter 1992).

The memory system supports cognition in two ways: by retaining a list of elements for

quick referencing, and by assisting in the construction of mental models (Johnson-Laird 1983).

According to Johnson-Laird (2010), mental models preserve the relationship between entities

by defining analogies that save on logical reasoning, one of the principles behind complex

visualization techniques and interaction (Liu & Stasko 2010a). Following the study of Logie

(1995), mental models are created in the visuo-spatial sketchpad, a specialization of VSTM.

Stimuli encoded into VSTM, ruled by cognition, may activate enduring information known as

long-term memory, including facts, meanings, relationships, skills, and procedures (Patterson

et al. 2010) – for a deeper discussion on working memory and long-term memory, refer to the

work of Rose et al. (2010).

The concepts presented in the former sections rely on ideas posed by widely accepted the-

ories, among several others, for the visual-cognitive process. The choice for this specific line of

thought has been motivated by its intuitive coherence and scope of influence in the literature.

Notwithstanding, other theories are widely referenced, such as the Spotlight (Eriksen & Hoff-

man 1973) and the Gradient (Cheal et al. 1994) models. For an ample discussion, refer to the

work of Squire (2009).

4 The Visual Expression Process

In this section, we review the practice of visualizing data by considering the concepts presented

so far. We organize the relationship among visualization, vision, and cognition according to

a framework named Visual Expression Process – Figure 3. Our theoretical organization has

7

four constituents: (1) pre-attentive stimuli, (2) analytical perceptions, (3) abstract patterns,

and (4) decision support. For completeness of our survey, Table 1 summarizes previous models

found in the literature. Our theorization is inspired by these previous works putting together

concepts in a complementary point of view.

Table 1: Summary of previous models on InfoVis.

Model Summary

Lohse (1993) An algebraic model to estimate the effort to answer

questions based on a visualization.

Card et al.

(1999)

A pipeline of how to conduct the visualization

practice.

van Wijk (2005) An economic model stated to evaluate efficacy and

efficiency.

(Ware 2005) A top-down (problem-solving) model that states that

what we see in a visualization depends on what we

are seeking for.

(Hegarty 2011) A descriptive discussion of how visualizations work

that culminates into a compilation of principles and

perspectives.

(Patterson et al.

2014)

A vision-cognition model that explains visualization

and how to improve it from a top-down perspective.

According to our theorization, (1) pre-attentive stimuli come from the neuronal reaction

to light, determining a map of potential interesting objects, or saliences. Then, attention

concentrates on one single element, part of a limited taxonomic vocabulary of (2) analytical

perceptions. Such perceptions go through a pattern-matching process that turns them into

(3) abstract patterns. Finally, abstract patterns in working memory support cognition in

combination with domain knowledge originated from long-term memory, leading to (4) decision

support. Notice, in the figure, that the arrow in between each pair of the four constituents is

bidirectional to indicate that the interplay occurs both bottom-up and top-down, notably in

response to the task at hand. Also, notice that after cognition, there might be new information

demands, what leads to the redefinition of the visualization by means of new parameters for

interaction – as depicted in blue in Figure 3. Following, we present further details.

8

Figure 3: The Visual Expression Process for Information Visualization specified with fourconstituents. The black bidirectional arrows refer to bottom-up/top-down visual-cognitive pro-cesses. In blue, new parameters of interaction alter the scene iteratively.

4.1 Pre-attentive Stimuli - channels for data encoding

Pre-attentive stimuli impel maps of saliences, as highlighted on the leftmost side of Figure 3.

In the work of Rodrigues et al. (2008), the authors verified that such stimuli manifest through

position, shape, color, and time. Here, we refer to these factors as channels for data encoding,

in the sense that they encode data into visual stimuli.

Although the consideration of four channels is a reductionist classification, it is supported

by the literature. About color and texture, Watt (1995) affirms that, just like texture, color

is the psychological response to the spectral characteristics of a surface; and that, different

surfaces are perceived as having different colors. Furthermore, Motter (1994) observes that,

early in visual processing, the incoming information is sorted and grouped according to the

similarity of simple shape features, such as orientation or size, and of surface features, such as

color, luminance, or texture.

The features of each channel span to a large set, but Card et al. (1999) observe that just a

limited number of the many existing graphical properties are used for Information Visualization.

Healey & Enns (2012) present an extensive survey on pre-attentive features and visualization.

A non-exhaustive list of such features is presented in Table 2.

9

Table 2: Classes/channels of pre-attentive features from the perspective of data representation.Pre-attentive class

(data encoding channel)

Features

Position 1D/2D/3D position, stereoscopic depth;

Shape line, area, volume, form, orientation, length, width,

collinearity, size, curvature, marks, numerosity, con-

vex/concave;

Color hue, saturation, brightness and texture;

Time movement, morphing, blinking, color/light intermit-

tence.

Discussion

A well-designed visual representation must present a high overlap between its map of saliences

and its (implicit) map of semantic relevance. That is, the design of visual representations is

supposed to maximize pre-attentive effects. However, such maximization may not be possible

without flexible human intervention over the channels of data encoding. In other words, inter-

action, the active redefinition of the channels, is a mandatory need not fully explored in many

designs.

Liu et al. (2008) point out that many visualization systems are not sufficiently flexible to

support user customization and appropriation. In fact, it is not difficult to find visualization

tools that are limited in allowing the user to determine how to encode data. In these circum-

stances, a user may ask “may I change the positioning order of the elements?”, “can I have

each year represented with a different shape?”, “can I color the left group in red?”, or “can I

see that animated?”. Each of these examples refers to a particular pre-attentive feature or, as

we propose, to a data encoding channel.

For instance, consider the seminal system GGobi (Cook & Swayne 2007), which introduced

a large set of features if compared to its former version, system XGobi. Still, a brief analysis

reveals that there is much to be improved: positioning of views is limited, except for Parallel

Coordinates; shape is not an option for coding in the same way that color is; and animation

is restricted to scatter plots through touring techniques. Those design issues contrast to what

is observed in commercial systems like TIBCO’s Spotfire (http://spotfire.tibco.com/) and

Google’s Gapminder (http://www.gapminder.org/), which present higher levels of freedom

for each encoding channel, but that, still, do not achieve full appropriation of the scene the way

we discuss.

Accordingly, the design of visualizations must not only allow users to alter visual attributes,

10

they must incorporate mechanisms to emphasize each channel. This is necessary because a given

stimulus (position, shape, color, or time) may fail to capture attention when it is surrounded by

other stimuli – this is called inattentional blindness. In such circumstance, this given stimulus

fails in competing with other more interesting stimuli or, worse, it is ignored (Mack A 1998)(DJ

2000). “Blindness” occurs especially when the viewer has some previous expectation on the

scene; in such situations, top-down processes can strongly influence on what is noticed. This

issue has been noticed by Patterson et al. (2014), who claim that the design of a visualization

technique must provide means to capture attention alerting users of changes.

Furthermore, according to the limitations of the memory system, as presented in Section 3.3,

memory overloading is a problem that might constrain problem-solving. In fact, according to

Wickens (2008), multiple tasks in one same cognitive dimension might decrease performance.

One possibility to lessen this drawback is to selectively turn the visual channels on and off.

For example, it is possible to conceive a dispersion plot in which size is simply turned off,

so to temporarily avoid overlapping – see Figure 4 (b). Similarly, it is possible to turn off

color, presenting all the information with black marks, emphasizing the role shape contours

– Figure 4(c). It is also possible to include intermittence, or movement, much the way that

cyclists do with lights in urban traffic. This way, specific graphical elements can start to blink

– Figure 4(e). We conjecture that this kind of selective constraining of channels not only

alters attention mechanisms, as there will be fewer candidates for attention, it also reduces the

memory load, as there would be fewer targets. Furthermore, it has the potential of affecting top-

down processes by de-emphasizing features that are expected by the user – and that prevent

her/him from noticing unexpected traits due to “blindness”. In these cases, algorithms to

assist visualizations may be helpful, as advocated by Chen (2006). Such algorithms might

monitor statistical features or mine characteristics of interest to suggest new encodings “on the

fly”, enabling more intense appropriation of the scene. However, not much research has been

conducted on this issue, which has potential for further developments.

4.2 Analytical Perceptions

Data encoding channels provide maps of potential targets for attention. Now, following vision

theory, the next mechanism is attentive selection – see Figure 3. Biased by user intention, a

subset of the prominent entities in a visualization will reach working memory. Once selected,

the chosen visual stimuli will be the basis of the analogies that lead to mental models, see

Section 3.3. Here, one question comes up – what perceptions are produced by the targets of

11

Figure 4: (a) Dispersion plot using channels of position, shape contour, shape size, and color.(b) Shape size turned off. (c) Color turned off. (d) Shape contour turned off. (e) Timemovement turned on for automatically detected outliers. (f) Shape size turned on.

attention in an information visualization design?

To answer this question, we have extensively inspected the literature tracking the ways in

which visual manifestation occurs when the intent is data analysis. We have found a limited set

of possibilities, defining a visual taxonomic vocabulary whose elements appear recurrently. We

refer to these elements analytical perceptions, depicted in the second part of Figure 3. In the

realm of interaction, Yi et al. (2007) followed a similar procedure based on extensive literature

inspection; they achieved results concerning the user intent, producing a limited set of recurrent

possibilities.

Analytical perceptions are the traits that any user attentively seeks for in a visual repre-

sentation. Following the dual system theory presented in Section 3.3, such perceptions occur

intuitively. Our investigation indicates that such elements include correspondence, differentia-

tion, recognition, connectivity, arrangement, and variation in time. The most verified of these

phenomena, correspondence and differentiation, are noted by Bertin (1977/1981) and by Card

et al. (1999). The third analytical perception is presented by Mackinlay (1986) who states

that the notion of relationship among graphical entities comes from the perception of connec-

tivity. Meanwhile, arrangement arises from group positional configurations, largely studied by

12

the Gestalt psychology (Koffka 1935) as observed, for example, in graph layouts (Dwyer et al.

2009)(Tollis et al. 1998). Recognition, in turn, takes place as a resemblance to previous knowl-

edge and/or expertise, a concept studied in psychological models and Information Visualization

models. As Liu & Stasko (2010b) point out, the concept of internalization involves the encod-

ing of perceived information into long-term memory (enduring information or pattern). Lastly,

variation, manifests only along time – not necessarily for temporal data – and in combination

to the other five perceptions.

The notion of analytical perceptions becomes evident when they are not found and the

pipeline outlined in Figure 3 is broken, preventing Visual Expression – if none of the aforemen-

tioned perceptions occur, the user is unable to make sense. As depicted in our theorization,

analytical perceptions occur after pre-attention (Section 4.1) and before abstract patterns (Sec-

tion 4.3), independently of the data domain. Hence, they bridge vision and data interpretation.

Specifically, we discuss the analytical perceptions and how they relate to the data encoding

channels in the following.

• correspondence: each position/shape/color has a direct correspondence to a referential

map – discrete or continuous – that is part of the scene (explicit) or that is mental (im-

plicit), defining analogical reasoning as explained in Section 3.3. Explicit maps include

axes, geographical maps, shape/color dictionaries, and position/shape/color ranges. Im-

plicit maps include known orderings and shape metaphors;

• differentiation: each position/shape/color discriminates graphical items. Differentiation

is a correspondence achieved by the user, who creates a referential map in memory. Such

map is limited in the number of elements (or differentiations) according to the limitations

of memory;

• recognition: positions/shapes/colors whose decoding comes from the expertise of the user

or from previous knowledge – recognition is a correspondence established from visual

entities to concepts retained/learned in long-term memory;

• connectivity: shapes, mainly edges, that convey information about relationships among

entities in memory;

• arrangement: Gestalt principles of organization – positional placements (closure, proxim-

ity, and symmetry) that convey perception about group properties, for example, clusters

and structural cues;

13

• variation in time: obtained when the parameters of position/shape/color are altered along

time, inducing new perceptions for each of these channels.

Discussion

Our set of analytical perceptions is not an exhaustive listing, but a first reference that suggests

that visualization designs tend to resort to the same basic set. For instance, the design of

Google’s Gapminder tool, although contemporaneous, reproduces the same dispersion plots of

statistical books a hundred years old. Nevertheless, as observed by Liu et al. (2014), works on

new designs and on the evaluation of existing ones have not considered that there is a limited

set of elements to instantiate in data representations. This fact could fruitfully support the def-

inition of design languages and frameworks, which would benefit from recurrent constructs that

lead to a limited set of analytical perceptions. Differently, current languages and frameworks

rely on graphical patterns and it is up to the user to build the desired analytical perception;

see Table 3 for a representative set.

Table 3: Previous works on languages and frameworks for visualization design.

Work Approach Elements

Protovis

(Bostock &

Heer 2009)(Heer

& Bostock 2010)

Graphical Marks, shapes and layout

D3 (Bostock

et al. 2011)

Visualization

pipeline

Selection, operation, join, layout and

transformation

Improvise

(Weaver 2004)

Link and

coordinate

Variable, function and view

Prefuse (Heer

et al. 2005)

High-level API Filter, layout, interaction, color and

size

ggplot2

(Wickham

2009)

Domain specific Layer, scale, coordinate system and

facet

Flexible Linked

Axes (Claessen

& van Wijk

2011)

Linked axes Axes mapping, interaction, line and

point

This work Visual-cognitive Pre-attentive encoding channels and

analytical perceptions

By considering the concepts reviewed so far, it is possible to conceive a design language

14

whose approach is based on cognition, and whose elements are interactive encoding channels,

and analytical perceptions – refer to Table 3 for comparison. For example, in this design

language, one would be able to state a visualization by choosing channel color and perception of

differentiation. This same visualization would demand a few more elements, as channel position

and perception of correspondence. As in any language, these elements would receive parameters

according to an extensible library of data-to-marks mapping. In the case of relationships,

one would be able to choose channel position with perception of arrangement, together with

channel shape and perception of connectivity. Underlying algorithms would abstract undesired

complexity, as an algorithm for outlier detection to support color differentiation; and a force-

directed algorithm to support arrangement. As a design language, this approach would bring

the benefit of discriminating the recurring elements of visualization techniques and having

them in libraries for composition. This alternative approach contrasts with the usual practice

of combining recurrent elements in ensembles assumed as new techniques.

4.3 Abstract Patterns

According to Hutchins (1996), tools – or externalizations (Hegarty 2004) – transform difficult

tasks into in-mind manipulations of physical systems, or into pattern-matching problems (Giere

2002). Pattern matching, or pattern-recognition (Patterson et al. 2014), the association of a

given stimulus to information retrieved from memory (Eysenck & Keane 2003), is one of the

principles of data visualization and the second step of the Visual Expression Process – see

Figure 3.

According to our organization, once a user focuses on an analytical perception, she/he

proceeds to match this perception to an abstract pattern. Based on the theory of vision – seen

in Section 3.3, the generation of patterns is supported by memory, which is filled with data

from the visual-sensorial system or from long-term memory. The efficiency of this intercourse is

explained by the fact that the visual-sensorial system provides spatial information to memory

at rates higher than that of long-term memory (Ware 2004), in a time ranging from 100 to

250 ms (Kieras & Meyer 1997). Hence, analytical perceptions in the visual-sensorial system

work similarly to the images stored in long-term memory, providing efficient pattern-matching.

According to the dual system theory, Section 3.3, this process is conscious and slow; but,

with practice and experience, a given analyst can become a specialist. For specialists, pattern

matching becomes intuitive (Evans 2008) – refer to the work of Patterson et al. (2010) for a

thorough discussion. It is a straight conclusion, then, that a period of practice and experience

15

is necessary for most visualization techniques, and that this is an obstacle for use.

A suggestive set of the patterns – third part of Figure 3 – that arise from the perception-to-

pattern matching, includes: correlation, tendency, classification, relationship, order, summa-

rization, outlier, cluster, structure, and reading. Tufte (1997) and Amar et al. (2005) provide

more exhaustive listings that follow different rationalizations.

Discussion

In our literature review, we noticed that many systems disregard the fact that visualization

techniques converge through abstract patterns. According to current practices, instead of a

set of familiar patterns, the user has to choose among a set of visualization designs that,

quite often, they have little experience with (Chen 2005). Thomas & Cook (2005) reinforce

this notion; they state that abstract patterns correspond to the second factor of their four-

steps analytical-reasoning process. Following their process – a pattern-to-construct sequence,

users have constraints in relation to what they can search for in face of a given pattern. For

example, suspiciousness tends to appear by means of tracking for outliers; while evidence of

illegal lobbying practices emerge from clusters; and community detection in graphs is a task for

relationship. Still, users are often offered a menu whose options are, for instance, dimensional

stacking (LeBlanc et al. 1990), star coordinates (Kandogan 2000), and table lens (Rao & Card

1994); alternatives far from the pattern-to-construct task they have in their minds. A similar

notion dates back to the 1990’s, in the work of Wehrend & Lewis (1990), who advocates in favor

of more specific problem-oriented choices. We suspect that, because design-oriented interfaces

neglect the more natural notion of abstract patterns, this is possibly one of the reasons why

advanced visualization techniques have struggled to achieve a wider commercial dissemination,

e.g, in office suites and in everyday spreadsheets.

Take, for instance, the visualization technique Treemap (Shneiderman 1992), introduced

for visualizing hierarchical data in general. In two decades, Treemap gained popularity at

the academy, but, as a general hierarchical tool, it has failed in reaching a wider use. This is,

possibly, because it has been criticized since its introduction (Barlow & Neville 2001, Cawthon &

Moere 2007, Fabrikant & Skupin 2005), being accused of lacking cognitive plausibility, having

poorly perceived aesthetic qualities, and presenting poor task-driven performance (Wood &

Dykes 2008). Despite all, however, an especial design of the Treemap has remarkably succeeded.

The SequoiaView system (www.win.tue.nl/sequoiaview) has achieved wide dissemination

(check (van Wijk 2006) for some impressive numbers), far beyond the academic walls. But,

16

how can we have two flavors of the same technique evolve in different ways? Certainly, not

one single aspect explains everything, but an outstanding factor comes up: the SequoiaView

is domain and pattern-oriented, it is distributed to visualize the structure and the sizes of the

files in your hard-drive, specifically. In accordance, users do not have to discover that the tool

is good at doing this; instead, when they have this specific problem at hand, they are guided

to SequoiaView, a more natural process.

In designing systems, an alternative course of action would be to initially present the user

with a set of patterns to choose from. After that, she/he would be offered a set of visualization

techniques that better suit the pattern they are seeking for. Users may know what to look for

by means of previous knowledge of the data domain, by means of known problems to be solved,

by means of suspicious clues perceived along the data usage, and also by means of previous

visual exploration of the data.

4.4 Cognitive Decision Support

After producing patterns, vision is no longer an active agent, neither pre-attentively nor atten-

tively. Now, the analysis follows the widely accepted pattern-then-cognition process (Margolis

1990) (MacEachren 2004) to achieve decision support. Indeed, according to the analytical rea-

soning of Thomas & Cook (2005), Information Visualization cannot ultimately provide decision

support, which can only be achieved after interpreting the patterns in light of the data domain

– rightmost side of Figure 3. That is, even though users can come up with patterns without

considering the underlying data, these patterns are not of great use if the domain is not deeply

understood.

Insufficient domain knowledge leads to unsatisfied expectations in relation to InfoVis, cre-

ating situations in which a user is presented to a supposedly insightful visualization but, then,

everything one hears is “so what”? A disappointment that happens due to the enthusiasm

according to which one can solve a wide range of problems just by looking at the data. How-

ever, visualization tools can do little if the analyst is not well-prepared to assess what the data

ultimately describes and potentially carries within.

Discussion

To prevent the aforementioned problems, InfoVis systems should define systematic means to

aid the user in recording and accessing the domain knowledge related to the problems at hand.

An interesting approach to overcome the gap of domain knowledge is to use annotations (or

17

analytic provenance (Xu et al. 2015)), either automatic or manual. As pointed out by Hullman

et al. (2011), annotations help to direct the user’s attention and foreground particular insights,

supporting the most efficient inferences. The work of Hullman et al. (2013) exemplifies this

issue. Their work focuses on stock-price time series, which are hard to understand if one is

not aware of the facts that influenced the behavior of the market. Their system solves this

problem by identifying news that happened contemporaneously to outstanding patterns found

in the plots, presenting them on demand – illustrated in Figure 5. Similar approaches (Dennis

et al. 2003, Conesa et al. 2005) have been proposed for genomic data, an extreme case in which

domain knowledge is necessary, otherwise no reading of the data (either visual or textual) will

make sense.

Figure 5: Example of an annotated visualization produced with Contextifier – in the figure,labels provide domain knowledge. Reproduced from the work of Hullman et al. (2011).

An alternative is to use technique storytelling, as surveyed by Segel & Heer (2010). Plaisant

(2005) defends that advanced interfaces need to address the long-term process of analysis that

may require annotation, history keeping, collaboration with peers, and the dissemination of

results and procedures used. Storytelling not only attacks the problem of lacking domain

knowledge, it also provides knowledge about new findings in the form of further “story chapters”

interactively created. The visual analysis, potentially, becomes an incremental set of insights

from multiple experts in the form of bookmarks, keyword tags, text comments, and audio

18

annotations.

By bringing existing (factual) or produced (annotated) domain knowledge to the stage,

abstract patterns can support reasoning by supporting conditional inferences (Johnson-Laird

& Byrne 2002), possibly influenced by statistical regularities (Patterson et al. 2012). The

result is the provision of inferences for decision making, notably, predictions, alternatives, and

evaluations as depicted in Figure 3.

4.5 Remarks

In Sections 4.1 through 4.4, we have drawn conclusions by surveying accepted concepts of

vision and cognition. Notwithstanding, we state these conclusions as conjectures with theo-

retical evidence and with examples. This is because the validation of these hypotheses would

encompass vast experimentation; producing material enough to spam a few papers or a book,

to be conservative. Therefore, we leave our discussions both as contributions – to guide new

systematizations; and, as future work – to drive further refinements and discoveries.

Our conclusions also point to a challenging systematization. Rendering all the recommended

aspects, together with multiple techniques and data domains, might involve a development ef-

fort similar to that of huge software pieces, as office suites for instance. This is academically

non-attractive, and economically risky. Possibly, the solution is to set a well-defined devel-

opment framework for collaborative work, with ample acceptation, and standard interfacing.

In the realm of machine learning, software Weka (Hall et al. 2009) has achieved great success

in a similar endeavor. However, the graphical nature of InfoVis, and its data preprocessing

techniques, imposes big challenges.

5 Conclusions

We reviewed concepts on vision, cognition, and Information Visualization by introducing the

organizational theorization named Visual Expression Process, which proposes a course of action

to explain visual data analysis. Over an extensive literature survey, the theorization provides

comprehension and science for data graphical presentation. It proposes a new perspective to

discuss and characterize techniques. From this perspective, for instance, it would be possible

to “dissect” a given technique in terms of the channels, the analytical perceptions and the

patterns that it supports, potentially revealing strengths and weaknesses. Our contributions

are as follows:

19

• Theoretical compendium: we plotted the Visual Expression Process to interrelate vision,

cognition, and Information Visualization;

• Discussions: we provided an extensive survey from different fields of science to serve as

basis for relevant debate;

• Reflections: we revisited design practices supported by examples and study cases.

Overall, we have put together key concepts to introduce an insightful consideration of vi-

sualizations. The reductionist perspective of our organization translates non-familiar concepts

of vision and cognition into their corresponding effects with respect to design. In the form

of a vocabulary taxonomically organized, we proposed a simplified comprehension of the fac-

tors that define techniques and systems. With our contributions, we expect to foster a more

comprehensive, accessible, and applied science of visualization.

References

Alvarez, G. A. & Cavanagh, P. (2004), ‘The capacity of visual short-term memory is set both

by visual information load and by number of objects’, Psychological science 15(2), 106–111.

Amar, R., Eagan, J. & Stasko, J. (2005), Low-level components of analytic activity in infor-

mation visualization, in ‘Proceedings of the Proceedings of the 2005 IEEE Symposium on

Information Visualization’, INFOVIS ’05, IEEE Computer Society, Washington, DC, USA,

pp. 15–.

URL: http://dx.doi.org/10.1109/INFOVIS.2005.24

Baddeley, A. & Hitch, G. (1974), ‘Working memory’, The psychology of learning and motivation:

advances in research and theory 8, 47–89.

Barlow, T. & Neville, P. (2001), A comparison of 2-d visualizations of hierarchies, in ‘Informa-

tion Visualization, 2001. INFOVIS 2001. IEEE Symposium on’, pp. 131–138.

Bertin, J. (1977/1981), Graphics and Graphic Information-Processing, Walter de Gruyter,

Berlin. 273 pages.

Bostock, M. & Heer, J. (2009), ‘Protovis: A graphical toolkit for visualization’, Visualization

and Computer Graphics, IEEE Transactions on 15(6), 1121 –1128.

20

Bostock, M., Ogievetsky, V. & Heer, J. (2011), ‘D3 data-driven documents’, IEEE Transactions

on Visualization and Computer Graphics 17(12), 2301–2309.

URL: http://dx.doi.org/10.1109/TVCG.2011.185

Bugajska, M. (2003), Spatial Visualization of Abstract Information, PhD thesis, ETH Zurich.

197 pp.

Buttner, P. (2010), Hello java: Linking act-r 6 with a java simulation, in ‘Proceedings of the

10th International Conference on Cognitive Modeling’, pp. 289–290.

Card, S., Mackinlay, J. & Shneiderman, B. (1999), Readings in Information Visualization:

Using Vision to Think, San Francisco, CA, EUA: Morgan Kaufmann.

Cawthon, N. & Moere, A. (2007), The effect of aesthetic on the usability of data visualization,

in ‘Information Visualization, 2007. IV ’07. 11th International Conference’, pp. 637–648.

Cheal, M., Lyon, D. & Gottlob, L. (1994), ‘A framework for understanding the allocation of

attention in location-precued discrimination’, Quarterly Journal of Experimental Psychology:

Human Experimental Psychology 47, 699–739.

Chen, C. (2005), ‘Top 10 unsolved information visualization problems’, IEEE Computer Graph-

ics and Applications 25(4), 12–16.

Chen, C. (2006), Information Visualization: Beyond the Horizon, Springer Science.

Chi, E. H. (2000), A taxonomy of visualization techniques using the data state reference model,

in ‘IEEE Symposium on Information Visualization’, IEEE Computer Society Press, Salt Lake

City, UT, USA, pp. 69–75.

Claessen, J. & van Wijk, J. (2011), ‘Flexible linked axes for multivariate data visualization’,

Visualization and Computer Graphics, IEEE Transactions on 17(12), 2310–2316.

Cleveland, W. S. & McGill, R. (1984), ‘Graphical perception: Theory, experimentation and

application to the development of graphical methods’, J. Am. Stat. Assoc. 79, 387 5, 531–

554.

Cohen, R. (1985), The Development of Spatial Cognition, Psychology Press.

21

Conesa, A., Gotz, S., Garcia-Gomez, J. M., Terol, J., Talon, M. & Robles, M. (2005), ‘Blast2go:

a universal tool for annotation, visualization and analysis in functional genomics research’,

Bioinformatics 21(18), 3674–6.

Cook, D. & Swayne, D. F. (2007), Interactive and Dynamic Graphics for Data Analysis,

Springer-Verlag.

Cowan, N. (2010), ‘The magical mystery four: How is working memory capacity limited, and

why?’, Current directions in psychological science 19(1), 51–57.

Cutzu, F. & Tsotsos, J. (2003), ‘The selective tuning model of attention: psychophysical evi-

dence for a suppressive annulus around an attended item’, Vision Research 43, 205–219.

Dennis, G., Sherman, B., Hosack, D., Yang, J., Gao, W., Lane, H. C. & Lempicki, R. (2003),

‘DAVID: Database for Annotation, Visualization, and Integrated Discovery’, Genome Biology

4(5), P3+.

Dill, J., Earnshaw, R., Kasik, D., Vince, J. & Wong, P. (2012), Expanding the Frontiers of

Visual Analytics and Visualization, Springer-Verlag.

DJ, S. (2000), ‘Attentional capture and inattentional blindness’, Trends Cognitive Science

4(4), 147155.

Dwyer, T., Lee, B., Fisher, D., Quinn, K., Isenberg, P., Robertson, G. & North, C. (2009),

‘A comparison of user-generated and automatic graph layouts’, Visualization and Computer

Graphics, IEEE Transactions on 15(6), 961 –968.

Eriksen, C. W. & Hoffman, J. E. (1973), ‘The extent of processing of noise elements during

selective encoding from visual displays’, Perception and Psychophysics 14, 155–160.

Essen, D. V., Anderson, C. & Fellowman, D. (1992), ‘Information processing in the primate

visual system: An integrated systems perspective’, Science 255(5043), 419–422.

Evans, J. S. B. T. (2008), ‘Dual-processing accounts of reasoning, judgment, and social cogni-

tion’, Annual Review of Psychology 59(1), 255–278.

Evans, J. S. B. T. & Stanovich, K. E. (2013), ‘Dual-process theories of higher cognition: Ad-

vancing the debate’, Perspectives on Psychological Science 8(3), 223–241.

URL: http://pps.sagepub.com/content/8/3/223.abstract

22

Eysenck, M. W. & Keane, M. T. (2003), Cognitive Psychology: A Student’s Handbook, Taylor

& Francis.

Fabrikant, S. & Skupin, A. (2005), Cognitively plausible information visualization, in ‘Amster-

dam’, Elsevier Science.

Giere, R. N. (2002), Models as parts of distributed cognitive systems, in ‘Model-Based Reason-

ing’, Springer US, pp. 227–241.

Green, T. M., Ribarsky, W. & Fisher, B. (2009), ‘Building and applying a human cognition

model for visual analytics’, Information Visualization 8(1), 1–13.

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. & Witten, I. H. (2009), ‘The

weka data mining software: An update’, SIGKDD Explor. Newsl. 11(1), 10–18.

URL: http://doi.acm.org/10.1145/1656274.1656278

Healey, C. G. & Enns, J. T. (2012), ‘Attention and visual memory in visualization and computer

graphics’, Visualization and Computer Graphics, IEEE Transactions on 18(7), 1170–1188.

Heer, J. & Bostock, M. (2010), ‘Declarative language design for interactive visualization’, IEEE

Transactions on Visualization and Computer Graphics 16, 1149–1156.

Heer, J., Card, S. K. & Landay, J. A. (2005), Prefuse: A toolkit for interactive information

visualization, in ‘Proceedings of the SIGCHI Conference on Human Factors in Computing

Systems’, CHI ’05, ACM, New York, NY, USA, pp. 421–430.

URL: http://doi.acm.org/10.1145/1054972.1055031

Hegarty, M. (2004), Diagrams in the mind and in the world: Relations between internal and

external visualizations, in ‘Diagrams’, pp. 1–13.

Hegarty, M. (2011), ‘The cognitive science of visual-spatial displays: Implications for design’,

Topics in Cognitive Science 3(3), 446–474.

Hullman, J., Adar, E. & Shah, P. (2011), ‘Benefitting infovis with visual difficulties’, IEEE

Transactions on Visualization and Computer Graphics 17(12), 2213–2222.

URL: http://dx.doi.org/10.1109/TVCG.2011.175

Hullman, J., Diakopoulos, N. & Adar, E. (2013), Contextifier: Automatic generation of anno-

tated stock visualizations, in ‘Proceedings of the SIGCHI Conference on Human Factors in

23

Computing Systems’, CHI ’13, ACM, New York, NY, USA, pp. 2707–2716.

URL: http://doi.acm.org/10.1145/2470654.2481374

Hutchins, E. (1996), Cognition in the Wild, The MIT Press.

Itti, L. & Koch, C. (2001), ‘Computational modeling of visual attention’, Nature Reviews

Neuroscience 2(3), 194–203.

Johnson, C., Moorhead, R., Munzner, T., Pfister, H., Rheingans, P. & Yoo, T. (2006), NIH/NSF

Visualization Research Challenges, Los Alamitos, CA, EUA: IEEE Press. 36 pages.

Johnson-Laird, P. N. (1983), Mental models: towards a cognitive science of language, inference,

and consciousness, Harvard University Press.

Johnson-Laird, P. N. (2010), ‘Mental models and human reasoning’, Proceedings of the National

Academy of Sciences 107(43), 18243–18250.

Johnson-Laird, P. N. & Byrne, R. M. (2002), ‘Conditionals: a theory of meaning, pragmatics,

and inference.’, Psychological review 109(4), 646.

Just, M. & Carpenter, P. (1992), ‘A capacity theory of comprehension: individual differences

in working memory’, Psychological Review 99, 122–149.

Kandogan, E. (2000), Star coordinates: A multi-dimensional visualization technique with uni-

form treatment of dimensions, in ‘IEEE Symposium on Information Visualization’, IEEE

Press, Salt Lake City, pp. 4–8.

Keim, D. (2002), ‘Information visualization and visual data mining’, IEEE Trans. Visualization

and Computer Graphics 8(1), 1–8.

Kieras, D. & Meyer, D. (1997), ‘An overview of the epic architecture for cognition and perfor-

mance with application to human-computer interaction’, HCI 12, 391–438.

Koffka, K. (1935), Principles of Gestalt psychology, International library of psychology, philos-

ophy and scientific method, New York : Harcourt, Brace and Company.

LeBlanc, J., Ward, M. & Wittels, N. (1990), Exploring n-dimensional databases, in ‘Visu-

alization, 1990. Visualization ’90., Proceedings of the First IEEE Conference on’, pp. 230

–237.

24

Lippmann, R. P. (1987), ‘An introduction to computing with neural nets’, IEEE ASSP Maga-

zine 4(2), 4–22.

Liu, S., Cui, W., Wu, Y. & Liu, M. (2014), ‘A survey on information visualization: recent

advances and challenges’, The Visual Computer 30(12), 1373–1393.

Liu, Z., Nersessian, N. & Stasko, J. (2008), ‘Distributed cognition as a theoretical framework

for information visualization’, Visualization and Computer Graphics, IEEE Transactions on

14(6), 1173 –1180.

Liu, Z. & Stasko, J. (2010a), ‘Mental models, visual reasoning and interaction in information

visualization: A top-down perspective’, IEEE Transactions on Visualization and Computer

Graphics 16(6), 999–1008.

URL: http://dx.doi.org/10.1109/TVCG.2010.177

Liu, Z. & Stasko, J. (2010b), ‘Mental models, visual reasoning and interaction in informa-

tion visualization: A top-down perspective’, Visualization and Computer Graphics, IEEE

Transactions on 16(6), 999 –1008.

Logie, R. H. (1995), Visual-spatial working memory, Lawrence Erlbaum Associates.

Lohse, G. L. (1993), ‘A cognitive model for understanding graphical perception’, Hum.-Comput.

Interact. 8(4), 353–388.

MacEachren, A. M. (2004), How Maps Work: Representation, Visualization, and Design, The

Guilford Press.

Mack A, R. I. (1998), ‘Inattentional blindness’, Psyche 5(3).

Mackinlay, J. (1986), ‘Automating the design of graphical presentations of relational informa-

tion’, ACM Transactions on Graphics 5(2), 110–141.

Margolis, H. (1990), Patterns, Thinking, and Cognition: A Theory of Judgment, University Of

Chicago Press.

Miller, G. A. (1956), ‘The magical number seven, plus or minus two: Some limits on our

capacity for processing information’, Psychological Review 2(63), 34335.

Motter, B. C. (1994), ‘Neural correlates of attentive selection for color or luminance in extras-

triate area v4’, The Journal of Neuroscience 14(4), 2178–2189.

25

Navalpakkam, V. & Itti, L. (2007), ‘Search goal tunes visual features optimally’, Neuron

53(4), 605–617.

Nothdurft, H. (2000), ‘Salience from feature contrast: additivity across dimensions’, Vision

Research 40(10-12), 1183–1201.

Patterson, R. E. (2012), ‘Cognitive engineering, cognitive augmentation, and information dis-

play’, Journal of the Society for Information Display 20(4), 208–213.

URL: http://dx.doi.org/10.1889/JSID20.4.208

Patterson, R. E., Blaha, L. M., Grinstein, G. G., Liggett, K. K., Kaveney, D. E., Sheldon,

K. C., Havig, P. R. & Moore, J. A. (2014), ‘A human cognition framework for information

visualization’, Computers & Graphics 42(0), 42 – 58.

URL: http://www.sciencedirect.com/science/article/pii/S0097849314000387

Patterson, R. E., Pierce, B., Bell, H. H., Andrews, D. & Winterbottom, M. (2009), ‘Train-

ing robust decision making in immersive environments’, Cognitive Engineering and Decision

Making 3(4), 331–361.

Patterson, R. E., Pierce, B. J., Bell, H. H. & Klein, G. (2010), ‘Implicit learning, tacit knowl-

edge, expertise development, and naturalistic decision making’, Journal of Cognitive Engi-

neering and Decision Making 4(4), 289–303.

Patterson, R. E., Pierce, B. J., Boydstun, A. S., Ramsey, L. M., Shannan, J., Tripp, L. & Bell,

H. (2012), ‘Training intuitive decision making in a simulated real-world environment’, Human

Factors: The Journal of the Human Factors and Ergonomics Society p. 0018720812454432.

Plaisant, C. (2005), Chapter 3 - information visualization and the challenge of universal usabil-

ity, in ‘Exploring Geovisualization’, International Cartographic Association, Elsevier, Oxford,

pp. 53 – 82.

Rao, R. & Card, S. (1994), The table lens: Merging graphical and symbolic representation

in an interactive focus+context visualization for tabular information, in ‘Human Factors in

Computing Systems’, pp. 318–322.

Rensink, R. A. (2000), ‘The dynamic representation of scenes’, Visual Cognition 7(1-3), 17–42.

26

Rodrigues, J., Traina, A., Oliveira, M. & Traina, C. (2008), ‘The spatial/perceptual design

space: a new comprehension for data visualization’, Palgrave Information Visualization

6, 261–279.

Rose, N. S., Myerson, J., Roediger III, H. L. & Hale, S. (2010), ‘Similarities and differences

between working memory and long-term memory: evidence from the levels-of-processing span

task.’, Journal of Experimental Psychology: Learning, Memory, and Cognition 36(2), 471.

Scaife, M. & Rogers, Y. (1996), ‘External cognition: how do graphical representations work?’,

Int. J. Hum.-Comput. Stud. 45(2), 185–213.

Segel, E. & Heer, J. (2010), ‘Narrative visualization: Telling stories with data’, IEEE Transac-

tions on Visualization and Computer Graphics 16(6), 1139–1148.

URL: http://dx.doi.org/10.1109/TVCG.2010.179

Shneiderman, B. (1992), ‘Tree visualization with treemaps: A 2d space-filling approach’, ACM

Trans. on Graphics 11(1), 92–99.

Shneiderman, B. (1996), The eyes have it: A task by data type taxonomy of information

visualizations, in ‘IEEE Symp. Visual Languages’, IEEE CS Press, pp. 336–343.

Squire, L. R. (2009), ‘Memory and brain systems: 1969–2009’, The Journal of Neuroscience

29(41), 12711–12716.

Thomas, J. & Cook, K., eds (2005), Illuminating the Path: The Research and Development

Agenda for Visual Analytics, IEEE Press.

Tollis, I. G., Di Battista, G., Eades, P. & Tamassia, R. (1998), Graph Drawing: Algorithms for

the Visualization of Graphs, Prentice Hall.

Triesman, A. (1985), ‘Preattentive processing in vision’, Computer Vision, Graphics and Image

Processing 31, 156–177.

Tsotsos, J. (1991), ‘Is complexity theory appropriate for analysing biological systems?’, Behav-

ioral and Brain Sciences 14(4), 770–773.

Tsotsos, J., Culhane, S., Wai, W., Lai, Y., Davis, N. & Nuflo, F. (1995), ‘Modeling visual

attention via selective tuning’, Artificial Intelligence 78(1-2), 507–547.

27

Tufte, E. R. (1997), Visual Explanations: Images and Quantities, Evidence and Narrative,

Graphics Press.

Tweedie, L. (1997), Characterizing interactive externalizations, in ‘ACM SIGCHI conference

on Human factors in computing systems’, ACM Press, Atlanta, Georgia, USA, pp. 375–382.

van Wijk, J. (2005), The value of visualization, in ‘Visualization, 2005. VIS 05. IEEE’, pp. 79–

86.

van Wijk, J. (2006), ‘Views on visualization’, Visualization and Computer Graphics, IEEE

Transactions on 12(4), 1000 –433.

Ware, C. (2004), Information Visualization: Perception for design, San Francisco, CA: Morgan

Kaufman. 486 pages.

Ware, C. (2005), Knowledge and Information Visualization: Searching for Synergies, Springer

Berlin Heidelberg, Berlin, Heidelberg, chapter Visual Queries: The Foundation of Visual

Thinking, pp. 27–35.

Watt, R. J. (1995), Some speculations on the role of texture processing in visual perception,

in ‘Early vision and beyond’, Massachusetts Institute of Technology, Cambridge, MA, USA,

pp. 59–67.

Weaver, C. (2004), Building highly-coordinated visualizations in improvise, in ‘Information

Visualization, 2004. INFOVIS 2004. IEEE Symposium on’, pp. 159–166.

Wehrend, S. & Lewis, C. (1990), A problem-oriented classification of visualization techniques,

in ‘Visualization, 1990. Visualization ’90., Proceedings of the First IEEE Conference on’,

pp. 139–143, 469.

Wickens, C. D. (2008), ‘Multiple resources and mental workload’, Human Factors: The Journal

of the Human Factors and Ergonomics Society 50(3), 449–455.

URL: http://hfs.sagepub.com/content/50/3/449.abstract

Wickham, H. (2009), ggplot2: Elegant Graphics for Data Analysis, Springer, New York.

Wolfe, J. M. (1994), ‘Guided search 2.0 a revised model of visual search’, Psychonomic Bulletin

& Review 1(2), 202–238.

28

Wong, P. C., Shen, H.-W., Johnson, C., Chen, C. & Ross, R. B. (2012), ‘The top 10 challenges

in extreme-scale visual analytics’, Computer Graphics and Applications, IEEE 32(4), 63–67.

Wood, J. & Dykes, J. (2008), ‘Spatially ordered treemaps’, IEEE Transactions on Visualization

and Computer Graphics 14(6), 1348–1355.

Woodman, G. F., Carlisle, N. B. & Reinhart, R. M. (2013), ‘Where do we store the memory

representations that guide attention?’, Journal of vision 13(3), 1.

Xu, K., Attfield, S., Jankun-Kelly, T., Wheat, A., Nguyen, P. & Selvaraj, N. (2015), ‘Analytic

provenance for sensemaking: A research agenda’, Computer Graphics and Applications, IEEE

35(3), 56–64.

Yi, J. S., ah Kang, Y., Stasko, J. T. & Jacko, J. A. (2007), ‘Toward a deeper understanding of

the role of interaction in information visualization’, Visualization and Computer Graphics,

IEEE Transactions on 13(6), 1224–1231.

Young, R. M. & Lewis, R. L. (1999), The soar cognitive architecture and human working

memory, in A. Miyake & P. Shah, eds, ‘Models of Working Memory: Mechanisms of Active

Maintenance and Executive Control’, Cambridge University Press, pp. 224–256.

29