usability of planning support systems: analysing adoption and use in planning practice ·...
TRANSCRIPT
Usability of Planning Support Systems: Analysing Adoption and Use
in Planning Practice
Patrizia Russo
Submitted in total fulfilment of the requirements of the degree of Doctor of Philosophy
August 2017
Faculty of Architecture, Building and Planning The University of Melbourne
iii
Abstract
Planning Support Systems (PSS) are software tools designed for assisting planners in making
better decisions about current and future land uses. Despite their potential to support planners
in the context of strategic planning tasks, PSS adoption in planning practice is low. The literature
suggests that a major factor limiting the adoption of PSS is their low usability, i.e. they do not
allow planners to satisfactorily perform their activities. To better understand to what extent
planners were satisfied with functionality provided by PSS and its use, the author conducted an
in-depth investigation into PSS usability. Using a primarily qualitative approach, it employs two
user studies consisting of a user test and a series of interviews. In the user test, planners were
observed while interacting with a set of PSS in order to identify any usability problems arising
and to assess their experiences. General software adoption by planners and factors influencing
it were investigated in the interview study involving people from the planning discipline from
three countries, i.e. Australia, Italy and Switzerland. Interviews were also conducted with
academic planners in the respective countries who provided information about the education
and software training which planning students are provided with by university curricula. The aim
of this research was to provide the planning discipline with recommendations and contributions
to improve PSS usability and adoption. Moreover, developers are provided with effective
guidance to create more usable PSS.
iv
Declaration
This is to certify that:
(1) the thesis comprises only my original work towards the degree of Doctor of Philosophy
except where indicated in the Preface,
(2) due acknowledgement has been made in the text to all other material used,
(3) the thesis is fewer than 100,000 words in length, exclusive of tables, maps,
bibliographies and appendices.
Patrizia Russo
v
Acknowledgments
I am very grateful to my supervisor Chris Pettit for having given me the opportunity to do this
research, for his support, trust and constant encouragement. I also would like to thank Andy
Krause for reading my chapters, providing support and prompt feedback. Advice was also given
if needed by Martin Tomko, Claudia Pelizaro, Susanne Bleisch, Ray Wyatt, Arzu Coltekin and the
AURIN technical team.
I express my gratitude for the collaboration and support to Maria Francesca Costabile and Rosa
Lanzilotti as well as the whole Interaction Visualization Usability & UX (IVU) group at the
Department of Computer Science of the University of Bari Aldo Moro in Italy. They devoted their
time to my research and took responsibility for my academic education.
The support of the Cooperative Research Centre for Spatial Information (CRC-SI) is
acknowledged, whose activities are funded by the Business Cooperative Research Centres
Programme.
vi
Preface
Three publications resulted from this thesis. These publications were undertaken in
collaboration with other researchers. Parts of these publications have appeared in different
chapters of this thesis. The bibliographic details of these publications are as follows:
• Russo, P., Lanzilotti, R., Costabile, M.F. & Pettit, C.J., 2018. Towards satisfying practitioners
in using Planning Support Systems. Computers, Environment and Urban Systems, 67, pp.9-
20.
• Russo, P., Lanzilotti, R., Costabile, M.F. & Pettit, C.J., 2017. Adoption and use of software
in land use planning practice: A multiple-country study. International Journal of Human-
Computer Interaction, pp.1-16.
• Russo, P., Costabile, M.F., Lanzilotti, R. & Pettit, C.J., 2015. Usability of Planning Support
Systems: an evaluation framework. In S. Geertman et al., eds. Planning Support Systems
and Smart Cities, pp.337–353.
vii
Table of contents
1. Introduction ......................................................................................... 1
1.1. Research Questions (RQs) ................................................................... 3
1.2. Main contributions ............................................................................. 4
1.3. Methodological approach ................................................................... 6
1.4. Scope of the study .............................................................................. 8
1.5. Outline of the thesis............................................................................ 10
2. Literature review .......................................................................................... 11
2.1. Planning practice ................................................................................ 11
2.2. Information and Communication Technologies (ICT) in planning
practice ............................................................................................... 13
2.3. Evolution of planning theory and the use of ICT in planning .............. 14
2.4. Spatial Information Systems (SIS) in planning practice ...................... 15
2.5. Planning Support Systems (PSS) ......................................................... 17
2.6. PSS repositories and review ................................................................ 20
2.7. Bottlenecks for low adoption of PSS ................................................... 21
2.8. Usability .............................................................................................. 24
2.9. User eXperience (UX) .......................................................................... 27
2.10. System-centred vs. user-centred design............................................. 28
2.11. Usability evaluation ............................................................................ 32
2.11.1 Inspection methods ................................................................. 33
2.11.2 User-based methods ................................................................ 34
2.12. Previous PSS development approaches and evaluation studies ........ 35
2.13. Conclusion ........................................................................................... 37
3. Research design .................................................................................... 39
3.1. Research Questions (RQs) ................................................................... 39
3.2. Theoretical framework ....................................................................... 40
3.3. Methodological approach ................................................................... 47
3.3.1 User test ..................................................................................... 48
3.3.2 Interviews .................................................................................. 51
3.4. Answering the research questions ..................................................... 53
3.5. Conclusion ........................................................................................... 54
viii
4. The Online Planning Support Systems (PSS) Resource ............................ 55
4.1. Introduction ........................................................................................ 55
4.2. Methods .............................................................................................. 57
4.3. Results and discussion ........................................................................ 60
4.4. Conclusion ........................................................................................... 62
5. The Planning Support Systems (PSS) Evaluation Framework ................... 63
5.1. Introduction and motivation .............................................................. 63
5.2. The PSS Evaluation Framework .......................................................... 64
5.2.1 Determine the evaluation goals ................................................ 64
5.2.2 Explore the questions ................................................................ 65
5.2.3 Choose the evaluation and data collection methods ................ 65
5.2.4 Identify the practical issues ....................................................... 66
5.2.5 Decide how to deal with the ethical issues ............................... 69
5.2.6 Evaluate, analyse, interpret and present the data .................... 69
5.3. Conclusion ........................................................................................... 70
6. A PSS usability evaluation study ............................................................ 72
6.1. Introduction and motivation............................................................... 72
6.2. An overview of the selected Planning Support Systems (PSS) .............. 74
6.3. The evaluation study ........................................................................... 79
6.3.1 Participants ................................................................................ 79
6.3.2 Design and procedure ................................................................ 80
6.3.3 Apparatus ................................................................................... 82
6.3.4 Data collection and analysis techniques .................................... 83
6.4. Results ................................................................................................. 84
6.4.1 Goal 1: Usability problems ......................................................... 84
6.4.2 Goal 2: Users’ expectations ....................................................... 86
6.5. Discussion and design indications....................................................... 87
6.6. Conclusion and future work ................................................................ 92
7. A tri-country interview study ................................................................. 95
7.1. Introduction and motivation .............................................................. 95
ix
7.2. The interview study: design and execution ........................................ 96
7.2.1 Goals .......................................................................................... 97
7.2.2 Participants ................................................................................ 97
7.2.3 The interviews ............................................................................ 99
7.2.4 Interview execution and data coding ........................................ 99
7.3. The interview study: results................................................................ 100
7.3.1 Goal 1: Software used by planners ............................................ 101
7.3.1.1 Software tools ................................................................ 101
7.3.1.2 Software training courses at universities and other
tertiary institutions ........................................................ 103
7.3.1.3 Planners’ training in software use ................................. 104
7.3.1.4 Functionality of planning software highlighted by the
interviewees .................................................................. 105
7.3.1.5 Frequency of software use by planners ......................... 108
7.3.2 Goal 2: Factors affecting the adoption of planning software .... 109
7.3.2.1 System-related factors ................................................... 109
7.3.2.2 Non-system-related factors ........................................... 111
7.4. Focus group ......................................................................................... 112
7.4.1 Participants ................................................................................ 113
7.4.2 Procedure and data collection ................................................... 113
7.4.3 Results ........................................................................................ 113
7.5. Discussion ........................................................................................... 115
7.5.1 Recommendations for the planning discipline .......................... 115
7.5.2 Recommendations for the design and development of
planning software ...................................................................... 117
7.6. Conclusion ........................................................................................... 119
8. Discussion and conclusion ..................................................................... 122
8.1. Contributions and key results ............................................................. 122
8.1.1 Theoretical contribution ............................................................ 125
8.1.1.1 Factors influencing the PSS adoption process ............... 125
8.1.1.2 Factors affecting UX ....................................................... 125
8.2. Findings: answers to the research questions ..................................... 126
8.3. Limitations of this research ................................................................ 134
x
8.4. Future work......................................................................................... 135
Bibliography ......................................................................................... 137
Appendices ........................................................................................... 155
A. System Usability Scale (SUS)
B. Coding scheme for observation, screen recording and thinking-aloud
C. PSS included in the online resource
D. Content of the Online PSS Resource
E. Questionnaire of the developer survey
F. Identification method of the PSS included in the online resource
G. The PSS Evaluation Framework - check-list of activities
H. List of potential participants
I. Examples of free software for recording
J. Running sheet
K. List of participants
L. Documents required during the user test
M. What to do and not do during a user test
N. Plain language statement for user tests
O. Consent form for user tests
P. Evaluation of participants’ performance
Q. Evaluation of responses to the SUS questionnaire
R. Explanation of parameters and their acronyms used in Envision
S. Research questions
T. UX questionnaire
U. Expertise questionnaire
V. Questions of interview with professional planners
W. Questions of interview with other planning actors
X. Questions of interview with academic planners
xi
List of figures
Figure 1.1: Outline of the thesis ........................................................................................... 10 Figure 2.1: Conceptual model of the three PSS notions ...................................................... 19 Figure 2.2: Nielsen’s [1993] model on system acceptability ................................................ 25 Figure 2.3: The waterfall model ........................................................................................... 29 Figure 2.4: The Human-Centred Design (HCD) process for interactive systems .................. 31 Figure 2.5: The star life cycle model .................................................................................... 32 Figure 3.1: System-related factors (green box) and non-system-related factors (red boxes) illustrated in Vonk’s et al. [2005] theoretical framework of the PSS adoption process ......................................................................................... 43 Figure 3.2: The three main factors affecting UX and the RQs addressing them in this research .............................................................................................................. 46 Figure 4.1: Screenshot of the Online PSS Resource ............................................................. 58 Figure 5.1: The six activities of the PSS Evaluation Framework ........................................... 65 Figure 6.1: Parameters, slider bars and the suitability layer provided in CommunityViz ... 76 Figure 6.2: The thirty-four parameters and slider bars provided in Envision for selecting their weights (right) and the suitability layer and overlays in QuantumGIS (left) .............................................................................................. 78 Figure 6.3: Parameters and spin boxes provided in Online What if? to assign the weights (left) and the suitability layer (right) ..................................................... 78 Figure 6.4: A participant (left) and the facilitator (right) during the test ............................. 83 Figure 6.5: Screenshot of CommunityViz online help .......................................................... 87 Figure 7.1: Software type ordered by descending percentage of use .................................. 103 Figure 7.2: System-related factors influencing the adoption of planning software. The values indicate the percentages of interviewees who mentioned the factors .. 110 Figure 7.3: Non-system-related factors affecting the adoption of planning software. The values indicate the percentages of interviewees who mentioned the factors .. 112 Figure 7.4: The participants during the focus group ............................................................ 114
xii
List of tables
Table 2.1: The three usability definitions and their attributes in comparison .................... 26 Table 3.1: Characteristics of nine PSS for assessing their suitability to be evaluated in the user test ........................................................................................................ 49 Table 3.2: The methods and activities used for answering the research questions ........... 54 Table 4.1: Number of PSS identified and the methods adopted ......................................... 60 Table 4.2: Number of PSS and available information .......................................................... 60 Table 6.1: The counterbalanced order in which the participants (P1,…,P6) used CommunityViz (CViz), Envision (Env) and Online What if? (OWI) ....................... 80 Table 6.2: The parameters in each of the three PSS as within the scope of the user test .. 81 Table 6.3: Actions highlighting usability problems in CommunityViz (CViz), Envision (Env) and Online What if? (OWI) ......................................................................... 84 Table 6.4: Frequency and percentage of the participants’ negative behaviour during the interaction with the PSS ................................................................................ 85 Table 6.5: Planners’ expectations of PSS functionality specifically for LSA and for PSS in general................................................................................................................. 89 Table 7.1: Numbers and percentages of participants by job and country .......................... 98 Table 7.2: Software tools used by the planners, as indicated by the interviewees ............ 102 Table 7.3: Strengths of the software types as evidenced in the interviews ........................ 106 Table 8.1: Planners’ expectations of PSS functionality emerged in the user test and/or interviews ............................................................................................................ 129
xiii
Glossary
ABS Australian Bureau of Statistics
AURIN Australian Urban Research Infrastructure Network
CAD Computer-Aided Design
CViz CommunityViz
DSS Decision Support Systems
eLSE e-Learning Systematic Evaluation
ETH Swiss Federal Institute of Technology
EUD End-User Development
GIS Geographic Information Systems
GUI Graphical User Interface
HCD Human-Centred Design
HCI Human-Computer Interaction
HSR University of Applied Sciences Rapperswil
ICT Information and Communication Technologies
IEC International Electrotechnical Commission
ISO International Organization for Standardization
LSA Land Suitability Analysis
LSUM Large Scale Urban Models
MCDA Multi-Criteria Decision Analysis
OWI Online What if?
PSS Planning Support Systems
SDSS Spatial Decision Support Systems
SIS Spatial Information Systems
SIT Spatial Information Tools
SMCE Spatial Multi-Criteria Evaluation
SUS System Usability Scale
U.S. EPA United States Environmental Protection Agency
UX User eXperience
1
Chapter 1: Introduction
Land is a basic resource for providing people with services, infrastructures, and living and work
spaces. It is, however, also a limited resource. This is why it needs to be used effectively and
sustainably. Planning is a field in public policy that manages the disposition of land with a view
to creating efficient and attractive environments for present and future generations. It is a
challenging task considering that it has to reconcile requirements from communities,
environment, politics, economic and demographic developments such as population growth
[Houghton et al. 2014; Biermann 2011]. In fact, many cities in the world are growing rapidly (e.g.
Izmir in Turkey) while some are entirely new greenfield developments (e.g. Ordos in China).
In Australia, urban growth and the coordination of land to meet the demand from
increasing population are important planning issues [United Nations 2012; Australian Bureau of
Statistics (ABS) 2003]. By addressing such issues, planning practice affects the long term
development of built and natural environments and therefore social well-being [Gurran 2007].
The complexity of the task leads planners to systematically assess information from multiple
sources as generated through a myriad of disciplines, in order to generate comprehensive land
use scenarios. Information required by planners is, however, not always available or not in the
required form. Planners use software tools to convert data into information that provides them
with knowledge and an ability to make informed decisions.
Planners use a variety of software. Excluding generic software tools such as word
processing and spreadsheet programs, planners use software tools that have not specifically
been developed for their activities but rather for geomatic engineers, architects, urban
designers and graphic designers. These are, for example, Geographic Information Systems (GIS),
Computer-Aided Design (CAD), three-dimensional (3D) visualisation and graphic design software
tools [Zeile et al. 2007].
An attempt to provide planners with specialised software tools has been made with
Planning Support Systems (PSS) dating from the early 90s [Harris & Batty 1993]. PSS refer to
software tools that in addition to data visualisation and analysis have the capability to forecast
land use scenarios through modelling techniques [Krause 2013; Densham 1991; Harris 1989].
PSS assist planners in generating and evaluating multiple land use scenarios and so in making
better decisions [Arciniegas et al. 2013; Klosterman 2001].
Although PSS have been available for more than two decades, their adoption by
planners is low [Brömmelstroet 2013; Williamson & McFarland 2012; Vonk & Geertman 2008;
Klosterman & Pettit 2005]. Existing research has shown that instrumental, human,
2
organisational and institutional factors such as low instrument quality, low awareness by
planners and low diffusion to and within planning organisations hamper the adoption of PSS.
One of the most important factors, or bottlenecks as they are referred to in the literature, that
has been identified is the low usability of PSS [Brömmelstroet 2010a; Vonk et al. 2005].
Usability is a quality factor that describes how well people can use provided functionality
[Nielsen 1993]. According to most widely acknowledged definitions by the Human-Computer
Interaction (HCI) and the Software Engineering discipline, i.e. Nielsen's [1993], ISO 9126-1 [ISO
9126 1998b] and ISO 9241 [ISO 9241 2010a], it is mainly characterised by the following five
attributes: i) effectiveness, i.e. the accuracy and completeness of the goals achieved; ii)
efficiency, i.e. the resources utilised in relation to the accuracy and completeness of the goals
achieved; iii) understandability, i.e. the capability of the software to show what tasks can be
performed with it and in which context of use, iv) learnability, i.e. the ease of learning the
functionality and the behaviour of the system; and v) satisfaction, i.e. the measure of how much
the users like the system.
In recent years, usability has been extended by the concept of User eXperience (UX)
[Albert & Tullis 2013]. This goes beyond the traditional usability attributes and emphasises
hedonistic aspects of software systems as well as experiences that people have when interacting
with them. Today’s systems should, in addition to providing usable functionality, involve users
in pleasant and engaging experiences.
The development process is fundamental for creating usable software systems
[Costabile 2001]. In particular, developers need to understand user needs and consider them
when designing software. To do this, the HCI discipline suggests an early involvement of users
in the development process [ISO 9241 2010b]. Specifically, it stresses an iterative process that
consists of specifying user requirements of system functionality, producing prototypes and
evaluating them possibly with users to check if their requirements are met. Without properly
considering user requirements, it is not possible to create software that is used with satisfaction
by professionals in their work practice. Professionals want software tools that provide the right
functionality and ‘speak a familiar language’ so that they can perform their tasks according to
their habits without being forced by the software to change their method of working. One of
the consequences of not properly addressing requirements of professionals and their
organisations is that software tools are at most adopted for a while, then soon discarded and
no longer used.
Many PSS have been developed to date and extensive resources have been put into
3
their development [Hughes & Heckbert 2012; United States Environmental Protection Agency
(U.S. EPA) 2000]. Past research reported that most PSS do not undergo well-considered design
processes and evaluations (e.g. Arciniegas et al. [2013], Vonk & Ligtenberg [2010], Geertman &
Stillwell [2003b]). In particular, PSS have been stated to be “far too generic, complex, inflexible,
incompatible with most planning tasks, oriented towards technology rather than problems, and
too focused on strict rationality” [Vonk et al. 2005; Batty 2003; Uran & Janssen 2003; Bishop
1998; Nedovic-Budic 1998; Couclelis 1991].
Experts in the field argue that in-depth research on PSS usability is required. The
evaluation and improvement of PSS usability have been identified in the research as a specific
priority [Williamson 2012; Pelizaro et al. 2009; Couclelis 2005]. In fact, evaluation of software
enables identifying usability problems and understanding the impact on users [Costabile 2001].
As long as the use of PSS connotes frustration and negative experiences, it is likely the adoption
of PSS will remain relatively low.
This thesis aims to counteract this tendency. It is intended to communicate knowledge
acknowledged by HCI research, in order to assist in creating more usable PSS. The main message
of this thesis is that user interfaces do not have to be considered only at the very end of system
development. Indeed, as noted by Raskin [2000], creating an interface is much like building a
house: if you do not get the foundations right, no amount of decorating can fix the resulting
structure. In essence, by conveying indications that are likely to improve PSS usability and UX,
this thesis endeavours to contribute towards advancing PSS adoption.
1.1 Research Questions (RQs)
This research focuses on PSS usability and adoption. It provides a comprehensive study in that
it firstly aims to understand usability problems that emerge when interacting with PSS. Secondly,
it examines from the perspectives of planners their experiences when interacting with PSS and
their expectations of system’s functionality. Based on this, as a further step it endeavours to
define recommendations for improving PSS usability and adoption. Specifically, Research
Question (RQ) 1 addresses ‘What are identified usability problems and UX when interacting with
PSS?’. This is a basis question in that it provides information required to understand the severity
of usability problems and what needs to be improved. It assists in understanding whether
usability problems are the cause of non-rigorous software development or developers’
negligence that can be easily overcome.
A strength of this research is the close involvement of planners in the study. Specifically,
4
with RQ 2: ‘What are planners’ expectations of PSS functionality?’, this research aims to gather
information from planners themselves on what they expect when interacting with PSS. This
information is relevant in that it provides indications on what functionality developers should
implement, in order to create PSS that are more tailored to planners.
RQ 1 and 2 are geared to clearly identify the functionality of a system that can be
improved. PSS adoption is, however, not only determined by system-related factors such as the
functionality and usability of PSS. System-related factors are a key determinant but there are
also other, non-system-related factors of a personal, organisational and institutional nature,
such as lack of skills and experience, lack of management support, and regulations, that hamper
PSS adoption [Vonk et al. 2005]. As such, non-system-related factors are examined in this
research and in particular, how they can positively affect PSS usability and adoption. In essence,
with RQ 3: ‘How can planners’ context contribute to improving PSS usability and adoption in
planning practice?’ this thesis aims to define indications and pathways that, if followed by PSS
developers and academic planners, can influence non-system-related factors, in order to
improve PSS usability and adoption. To stimulate following these pathways, recommendations
for PSS developers and academic planners are formulated.
1.2 Main contributions
A key contribution of this research is the description of the current situation of software
adoption by planners and factors influencing it. The latter was extensively studied by Vonk et al.
[2008; 2007b; 2006; 2005]. About a decade later, this research examines whether these factors
have changed.
This also leads to the theoretical contribution of this thesis. Vonk et al. [2005] provided
a theoretical framework that describes the PSS adoption process. This theoretical framework is
examined in this thesis. Specifically, the results that emerge in this research are discussed as to
whether and to what extent they are in line with the theoretical framework. Commonalities and
differences were stressed, in order to advance theory around PSS adoption.
The theory of this thesis also built upon a framework of factors (user, system, context)
that according to Roto et al. [2011] influences UX with software in general. By adopting this
framework, observations are made and described in this research specifically on experiences of
planners with PSS, which contribute to the theory at the intersection between the research
fields of HCI and PSS.
A further contribution is recommendations for PSS developers that foster improving the
5
usability and adoption of PSS. Software development and evaluation methods are
recommended that help create more usable software systems as acknowledged by HCI research.
PSS developers are provided with indications of what to do or not do for creating more usable
PSS.
As an innovative contribution, this research has developed a guide, called the PSS
Evaluation Framework, that provides support for performing usability evaluation of PSS. This
guide consists of six activities to be taken into account when evaluating PSS:
(1) determine the evaluation goals,
(2) explore the questions,
(3) choose the evaluation and data collection methods,
(4) identify the practical issues,
(5) decide how to deal with the ethical issues,
(6) evaluate, analyse, interpret and present the data.
Evaluations have to be carefully planned. This guide provides information about the activities to
consider in order to achieve the expected results from evaluations. Through providing cost-
effective methods the framework not only supports novice evaluators but fosters the inclusion
of evaluations as a standard work process in PSS development. The proposed PSS Evaluation
Framework emphasises that it is only through encouraging developers to perform evaluations
that the usability of PSS they create can be improved.
A series of recommendations is also provided to academic planners that inform how the
transfer of PSS from academia to practice can be improved. This is important as in the current
state of play many PSS are developed by academic planners [Hughes & Heckbert 2012; U.S. EPA
2000]. Recommendations include increased partnerships with planning organisations, better
preparation of planning students for PSS adoption through appropriate courses, and the
provision of PSS repositories.
As an example of a PSS repository and as a practical contribution of this research, the
Online PSS Resource (http://docs.aurin.org.au/projects/planning-support-systems/, accessed
on July 17, 2016) has been formulated. This repository comprises i) over one-hundred PSS
developed and applied internationally and ii) practical and technical information on a
comprehensive number of the available PSS. It is a platform that enables developers to publish
PSS they create and informs planners about PSS potential through the provision of key metrics.
This repository is an example of how planners’ context, and specifically academic planners, can
contribute to improving PSS adoption (RQ 3).
6
1.3 Methodological approach
This section describes the methods used in this research. Predominantly a qualitative approach
and low-constraint research were adopted which enabled insights into a specific context, i.e. the
PSS adoption process, and identifying contingent relationships among context variables (e.g.
planners’ skills and UX with PSS). As exploratory research in a rather new field of research, that
is planners’ UX with PSS, planning actors, i.e. planners and people working with them such as
specialists in GIS, modelling and data management, were directly involved in the two main
studies of this research. These are an interview and an evaluation study, as described here.
Interviews were conducted with thirty planning actors. The interviews were conducted
in three countries; Australia, Italy and Switzerland, in order to investigate software adoption in
planning practice and factors influencing it. Interviews were also conducted with five academic
planners of four higher education institutions who provided information about planning
education, in order to understand what type of software training, if any, planning students get
in university curricula or at other tertiary institutions. Through interviewing academic and
professional planners, the transfer of PSS from academia to practice was examined. The
interviews were mainly conducted face-to-face and one-on-one. They were undertaken in three
countries, in order to increase the international relevance and representativeness of the study
[Miles & Huberman 1994]. Identified similarities between countries increased confidence in the
findings [Yin 2003].
In order to identify any usability problems and UX when interacting with PSS, this
research conducted a thorough evaluation of three PSS; CommunityViz, Envision and Online
What if?, identified in the Online PSS Resource. This evaluation study was designed following
the PSS Evaluation Framework. It involved six planners from planning organisations in the
Melbourne metropolitan area participating in testing the PSS individually. During the test, the
participants had to perform a Land Suitability Analysis (LSA). This included selecting factors and
assigning them importance through weights. The PSS compute the suitability of land units for a
certain use and visualise them on a map. LSA is one of four spatial planning tasks as defined by
Pullar and McDonald [1999]. According to the literature [Seewald & Hassenzahl 2004], tasks that
participants have to perform during user tests do not have to be too easy but must be possible
to solve within a reasonable amount of time. The selected PSS allowed the participants to go
through a whole workflow process from data input to data output. Furthermore, they offered a
level of guidance so that participants did not require step-by-step instruction.
The participants were observed and, as part of the adopted thinking-aloud method, they
7
were asked to speak out loud during the interaction with the PSS. After the interaction with each
PSS, they were asked to complete a questionnaire in which positive and negative aspects of the
PSS as well as information on their experience were gathered. Through testing three PSS
different functionality and its impact on planners was investigated which provided indications
on what functionality to implement/not implement in order to create more usable PSS.
Expectations of functionality were explicitly and implicitly expressed by planners both
in the interview and evaluation study. Taking these into account in PSS design and development
could assist developers in creating more usable PSS. Similarly, recommendations for PSS
developers and academic planners that foster improving PSS usability and adoption were
defined based on the outcomes of both studies.
Both studies were conducted with small user samples which enabled an in-depth
investigation into PSS usability and adoption. Due to the small samples, however, this research
allows only limited generalisation of the findings. In particular, it paves the way for further and
quantitative research.
The formulation of the PSS Evaluation Framework has been inspired by a more general
framework presented in Preece et al. [2015] which provides a structure for performing
evaluations of interactive systems. To customise this framework for the evaluation of PSS, i.e.
to formulate the PSS Evaluation Framework, a collaboration with experts in the field of usability
and UX has been undertaken. The PSS Evaluation Framework has been formulated because PSS
evaluations have been found to be rather rare and there is also a need for them to be performed
more rigorously [Pelzer et al. 2016; Brömmelstroet 2015; Brömmelstroet 2013; Allen 2008]. As
the PSS Evaluation Framework assists in identifying usability problems, the framework was used
to design the evaluation study performed in this research.
Methods such as literature search and a survey with PSS developers were adopted to
formulate the Online PSS Resource. Specifically, the literature search has been undertaken to
identify the PSS added to the repository. The developers of the identified PSS were contacted
and asked to complete a survey that gathered practical and technical information on the PSS
they developed. The survey was formulated in order to gather information as provided in other
repositories and reviews [Hughes & Heckbert 2012; The Redlands Institute 2012; Geertman &
Stillwell 2003a; U.S. EPA 2000].
8
1.4 Scope of the study
Due to time and resource constraints, this research conducted a small part of a much larger
potential research agenda, which is understanding the PSS adoption process. It presents, a
decade after the studies of Vonk et al. [2007b; 2006; 2005], some initial evidence of the factors
impacting on the adoption of PSS with particular focus on PSS usability. However, due to the
qualitative approach and the small user samples adopted, this research did not allow the
examination of any correlation between usability and actual adoption of PSS.
This study focused on the usability and adoption of PSS. It combined two fields of
research that are firstly planning and secondly HCI. Planning as a field in public policy includes
several divisions such as land use, environmental and transportation planning. According to
Geertman and Stillwell [2003a], PSS have primarily been developed for assisting activities
performed in land use planning and strategic planning. The latter is concerned with establishing
plans and policy for achieving sustainable and effective land use. This involves exploring and
evaluating possible land use development scenarios considering economic, environmental and
social conditions and trends.
In Australia, professional planners dealing with these tasks are known as strategic
planners. Their tasks differ from those of statutory planners, i.e. professionals who control
development to ensure that it is compatible with current regulation. They assess proposals of
changes to land use against planning policy.
This research concentrated on land use planning and specifically on strategic planning
independent of the spatial scale (urban and regional). It used the term planners for referring to
professionals working in strategic planning. If not explicitly stated otherwise, the term planning
actors included, besides the aforementioned planners, people involved in the field of planning,
for example, academic planners, and people working in planning organisations, for example, GIS
and data specialists. The term planning actors was used by Williamson [2012]. However, in
contrast to this research, Williamson [2012: 78] not only referred to humans but also to “texts,
graphical representations, and technical systems” involved in planning practice.
This research has taken an Australian focus because it has been funded by the
Cooperative Research Centre for Spatial Information (CRC-SI) [CRC-SI 2016]. However, the
planning process in Australia is inherently similar to what is undertaken in the US and in a
number of countries in Europe including the UK. The international relevance of this research has
been further increased through interviews conducted by the author with planning actors in Italy
and Switzerland.
9
The term PSS emerged from early academic leaders in the field in the late 1980s.
Specifically, it was Harris [1989] who coined the term as a varied set of computer-based tools
that support most of the stages of the technical planning process [Batty 2007]. The definition
evolved since. This thesis defines PSS as software tools that use simple or complex mathematical
models for analysing and forecasting development of urban or regional land use [Klosterman
1999]. Through different types of modelling paradigm, such as agent-based modelling (e.g. used
in UrbanSim [Waddell 2002]), rule-based modelling (e.g. used in What if? [Klosterman 1999])
and cellular automata (e.g. used in SLEUTH [Clarke et al. 2007]), these systems provide abstract
representations of land use and assist planners through the formulation of various land use
scenarios.
This research focuses on the usability of PSS as a factor hampering its adoption. In
particular, the focus is on the system’s user interface and functionality which are the most
important components during its use and from the point of view of users because that is what
users interact with [Costabile 2001]. The term functionality often used in this thesis refers to
system’s functions (i.e. activities that can be performed with the system such as creating a buffer
around an area) and features (i.e. any attribute of the system such as the legend). However, this
research does not neglect factors hampering the adoption of PSS that are not directly related to
the interaction phase (e.g. system installation, data preparation) or to the system (e.g. training).
This research concentrates on the use of PSS by planners as individuals, not when used in a
group situation. Previous studies mainly analysed PSS use in collaborative settings and through
participatory modelling [McIntosh et al. 2011] (see Section 2.12), possibly because PSS are
complex systems and difficult to use without prior training. Whereas this study supports that
planning is a collaborative exercise [Healey 1992], to the author’s knowledge, planners’
interaction with software occurs most of the time individually.
The terms adoption and use are central to the research in this thesis. Vonk’s et al. [2005]
framework distinguishes between the two terms in that it assumes that adoption precedes
continued use. The former involves activities such as installation and data preparation that make
it possible to use a PSS. The term use presupposes that a PSS is ready to use and is regularly
used. Although this research supports Vonk’s et al. [2005] notion, it uses the term adoption as
synonymous with use when talking about PSS. In other words, if not otherwise stated, this
research assumes that once a PSS is adopted, it is also continuously used by planners. This
decision has been made because to the authors’ knowledge, there is no research that showed
the usage rate of PSS to significantly differ from its adoption rate.
10
1.5 Outline of the thesis
The thesis is organised as follows (see Figure 1.1):
• Chapter 2 provides the research background with a focus on planning and usability.
Literature is presented that reported on PSS adoption barriers, previous PSS development
approaches and evaluation studies.
• Chapter 3 documents the design of this research including the theoretical framework and
methodological approach adopted.
• Chapter 4 describes the Online PSS Resource and the methods adopted for formulating it.
Conclusions drawn from its formulation are discussed.
• Chapter 5 presents the PSS Evaluation Framework including a detailed description of the
six activities that it proposes.
• Chapter 6 describes in detail the evaluation study designed according to the PSS
Evaluation Framework.
• Chapter 7 reports the interview study and its results in relation to previous studies.
• Chapter 8 summarises the contributions and findings of this research and discusses future
work.
Figure 1.1: Outline of the thesis
11
Chapter 2: Literature review
This chapter provides a brief overview of the major elements of planning practice with the goal
of showing how and why Information and Communication Technologies (ICT) have become
important in this field. To begin, the evolution of ICT use is described in the context of planning
theory. This is followed by a discussion of Planning Support Systems (PSS), which are a subgroup
of ICT and the primary focus of this thesis. Background on the definition and availability of PSS
is provided. Importantly the chapter reviews a number of previous studies which focused on the
low adoption of PSS in planning practice. Another significant element of this thesis relating to
usability of PSS lies in the field of Human-Computer Interaction (HCI), which offers promising
opportunities. Hence, the state of the art in the field of HCI is provided in this chapter with a
focus on including usability, development and evaluation of software tools. Following this is a
discussion of the application of HCI practices in PSS development and evaluation. The chapter
concludes with identifying current research gaps and how this PhD research attempts to bridge
them.
2.1 Planning practice
Land is becoming a limited resource especially in urban areas where half of the world’s
population, and 87% of Australians, now lives [United Nations 2012; Australian Bureau of
Statistics (ABS) 2003]. Ensuring that land is used effectively and sustainably is fundamental for
providing present and future generations with convenient, efficient and attractive
environments. This task of managing the disposition of land to allow people to benefit from
services, infrastructures, living and work spaces is addressed by planning. It is a complex task as
it has to consider the interplay of population needs, political pressures, economic demands,
demographic and climatic changes [Bonner 2002]. Given its complexity a systematic approach is
needed to address these challenges [Houghton et al. 2014]. Two main activities, namely
statutory and strategic planning, can be distinguished in planning.
Statutory planning refers to the part of the planning process that controls development
to ensure that it is compatible with current regulation. It assesses proposals of changes to land
use against planning policy, principles and plans. Depending on whether it complies with plans
and policy, proposals are approved or rejected. Statutory planning is faced with more
standardised tasks and work processes at the end of which clear-cut answers are given.
Strategic planning, on the other hand, is concerned with establishing plans and policy
for achieving sustainable and effective land use. Strategic planning has been encapsulated in a
12
number of frameworks over the years including the rationalist approach [Yehezkel 1963],
incrementalist [Lindblom 1965], mixed scanning [Etzioni 1967] and collaborative planning
[Healey 1992]. However, it has been argued that the planning process does not follow any
standardised procedure but it has a dynamic and non-uniform character [Brömmelstroet 2013].
Despite this, some basic activities can be recognised that occur commonly in the planning
frameworks mentioned above. For instance, Vonk et al. [2007a] described the planning process
as having seven stages which do not necessarily occur in order but in varying sequence and
iteratively:
(1) problem definition,
(2) problem exploration and analysis,
(3) change exploration and analysis,
(4) consultation,
(5) decision,
(6) implementation,
(7) monitoring and evaluation.
The first stage involves indicating the existence of a problem to management and
authorities. This stage also includes scheduling a plan of tasks and a timeframe for solving the
problem. In a second stage, strategic planners explore and analyse the planning problem
through investigating policy, community needs and economic, environmental and social
conditions. Once planners understand the situation, they perform more targeted and advanced
analysis such as generating and evaluating goals, options and scenarios, conducting impact
assessment and developing plans. The fourth stage involves discussing and negotiating options
and implementation modes with other experts, in order to decide, in the following fifth stage,
which of those to choose. In the sixth stage, the implementation of the decision takes place.
Effects of the implementation are monitored and evaluated in the seventh stage.
The following scenario provides an example of the planning process. The City of Canning,
a municipality in the south-east of Western Australia, faced the problem of rapid population
growth in recent years. As a consequence, the planning department of the State of Western
Australia addressed the following critical planning task (stage 1): to identify areas within the City
of Canning where residential redevelopment could be allocated. As part of stage 2 of the
planning process, the department analysed conditions such as the size of the area required and
parameters important for precinct redevelopment. Then, by using appropriate software they
13
performed a more in-depth analysis (stage 3) which involved conducting a suitability analysis
based on the set of parameters identified in stage 2. The outcome of the analysis, i.e. possible
growth scenarios, was the focus of several consultations and discussions amongst senior
planners in the department, as well as policy makers from the State of Western Australia, to
assist in arriving at a good decision (stages 4 and 5). Once a final plan was agreed upon, it was
implemented (stage 6). Finally, the implemented plan was monitored and evaluated over time
(stage 7).
The previous scenario description is simple. In reality, the strategic planning process
constitutes a complex and challenging task, considering that requirements amongst the
community might diverge and strategic planning must reconcile them [Kunze et al. 2012].
Decisions made by strategic planning will rarely fully satisfy all groups in the community: it is
more about finding compromises to alleviate land use conflicts and addressing issues that have
priority.
2.2 Information and Communication Technologies (ICT) in planning practice
Data and information availability has dramatically increased in recent decades. Observation of
changes and analysis and interpretation of data is essential in the planning process [Branch
1998]. Planners use information in different forms and types in most of their activities, from
problem definition to plan presentation to stakeholder engagement [Krause 2013]. Information
required by planners is, however, not always available or not in the required form. In Australia,
for example, as in many countries, there are no national building footprint datasets or consistent
data on land use, including vacancies at the level of resolution of land parcel. Due to this lack of
information, some planning activities are not always satisfactorily performed and planners must
do with the best available data [Laurian et al. 2010; Seasons 2003].
One alternative for planners is to gather required information through the use of
software tools and applications, for example, the use of Nearmap [Nearmap 2016] for
determining the location of swimming pools and other fine scale land details required to support
planning tasks. In fact, the increasing availability of data and information has been accompanied
by a rapid development in ICT that has also reached planning practice. People from the planning
discipline use ICT to convert data into information for gathering knowledge that enables them
to make informed decisions.
Today, ICT and their capability to enhance data and information handling and processing
have taken an important role in planners’ activities [Krause 2013; Geertman & Stillwell 2009b].
14
However, how ICT, including information and quantitative methods, have been used and seen
in planning practice has evolved in recent decades. This ICT revolution has some direct linkages
to the evolution of planning theory as reported in the literature [Williamson 2012; Klosterman
2001]. These variations and changes have also been influenced by evolving theories in planning
[Klosterman 2001]. Silva [2010] suggests discussing ICT use in relation to planning theories, in
order to better understand past, present and future developments. The following section
provides an insight predominantly into planning theories since 1960 as this is when computers
began being used to support planning and so influenced the use of information, methods and
ICT in planning.
2.3 Evolution of planning theory and the use of ICT in planning
The 1960s were characterised by the system and the rationalist planning theories [Klosterman
2001]. System theory saw urban spaces as complex and dynamic constructs that are difficult to
plan with design methods. It saw quantitative modelling as a better method for capturing
dependences between components of urban systems and the impact of planning decisions [Silva
2010].
The rationalist theory found many supporters as it underlined planning processes with
a rational procedure consisting of three steps [Pettit & Pullar 1999]. These included: i) defining
the planning problem, goals and conditions; ii) generating and evaluating planning options; and
iii) choosing the option that best met the goals. Linked with these two theories was a shift from
a view of planning as design to planning as applied science. Design methods were increasingly
substituted by quantitative methods. Transportation and large scale urban models (LSUM)
[Stimson et al. 2012] gained more interest in planning education and practice. Quantitative
methods and ICT were seen as providing information that was free of political interests and
optimised the planning of cities [Klosterman 2001].
The 1970s saw criticism of the rational planning theory and a conviction that planning
was driven by politics. Responsible for this was the use of digital information and ICT. In fact,
theorists posited the idea that choices of analysis and information, and how such data-driven
and modelling outputs are presented and communicated, can be considered inherently
influenced by political interests which can be hidden from stakeholders who are less familiar
with technical methods [Klosterman 2001]. Planners also realised that digital information is
more ubiquitously accessible and subject to alterations than paper-based information. Planners’
increasing awareness stressed the threats in relation to the use of digital information and ICT
15
which are a reinforcement of existing structures of influence and the alteration of the plan- and
policy-making process. As a result of this, there was the demand for more security, transparency
and responsibility in the use of ICT [Williamson 2012; Klosterman 2001].
The 1980s and 90s were characterised by the recognition that communication and
participation play an essential role in planning practice, also known as the communicative turn
in planning theory [Healey 1992]. Thus, it was conceived and emphasised that planning is more
than plan and policy making and that many activities require communication. In the 1980s in
particular, the transmission of information to stakeholders was emphasised and often
considered more important than the ICT and analytical techniques that planners used.
A decade later, the significance of communication amongst planners and stakeholders
for exchanging ideas on common issues and for finding consensus prior to decision-making was
stressed. This approach was also known as ‘planning through consensus building’ [Innes 1996]
in which planning was seen as a process of reasoning together. In this theory, ICT and the
information that it provided had been expected to become the basis for improving social
interaction and communication. Despite the development and advances of ICT since, this
expectation has only partly been met. ICT is indeed used in planning practice; however, there is
also the awareness that its use, and particularly that of software specific for planning, is
somehow limited and could be improved [Klosterman 1997].
Quite recently a moving back to a view of planning as design with the advent of
Geodesign has appeared [Steinitz 2012]. This approach to spatial planning is characterised by
extensive use of methods and tools that deal with spatial data [Goodchild 2010]. Due to the
widespread availability of such data, this approach is relevant to people both in academia and
industry [Campagna 2014]. Geodesign is based on data- and information-rich methods for
enabling sustainable planning and responsible decision-making. Its concept and value have been
discussed by the planning discipline in recent literature [Wilson 2015; Campagna & Matta 2014;
Currier & Couclelis 2014; Lee et al. 2014].
2.4 Spatial Information Systems (SIS) in planning practice
Planners particularly deal with data with a spatial component. Through spatial data, they
attempt to gain information on conditions at locations to speculate the impact of the past and
the present and what is likely to happen in the future [CRC-SI 2016]. Dealing with spatial data,
however, requires specific software functionality. This is provided by a subcategory of ICT,
namely Spatial Information Systems (SIS). SIS consist of a set of components such as hardware,
16
software, methods, techniques and data that allow acquiring, managing, analysing and
communicating spatial data [CRC-SI 2016].
To refer to the software component of SIS, the term Spatial Information Tools (SIT) is
used in this study hereafter. Basically, any software that allows entering, visualising, analysing
and presenting spatial data can be considered a SIT. However, specific examples are Geographic
Information Systems (GIS), Computer-Aided Design (CAD), three-dimensional (3D) visualisation
and graphic design software tools. Despite their fairly wide application range, these software
tools have primarily been developed for architects, urban designers, engineers and other
professionals who create, present, visually explore and analyse spatial designs [Lee & Wu 2005].
SIT can, but do not have to, support users in the decision-making process. If they do,
they can also be seen as Spatial Decision Support Systems (SDSS). These systems provide
computerised assistance to people who have to make decisions on complex spatial problems
[Arciniegas et al. 2013]. To reduce the risk of human error, SDSS assist users with an evidence-
based approach that is based on a framework including data, “analytical models, graphical
display and tabular reporting capabilities, and the expert knowledge of decision-makers”
[Densham 1991: 404]. SDSS itself is a branch of Decision Support Systems (DSS), whose
assistance focus on decision-making processes but is not limited to spatial problems [Pettit et
al. 2008; Geertman & Stillwell 2003b].
Previous literature [Stevens et al. 2007; Zeile et al. 2007] reported that SIT, and
specifically GIS, CAD, 3D visualisation and graphic design software tools, are also used by
planners, even if they have not been primarily designed for addressing planning tasks. In general,
planners use SIT for visualising and analysing spatial data, in order to gather information that
supports planners and stakeholders in decision-making. Geertman and Stillwell [2009a] and
Pettit et al. [2014] specified that how analysis is performed, does not follow any routine but
depends on the task planners have to carry out, the functionality provided by the software tool
and on planners’ skills and experience with the software tool.
Among the various SIT, GIS appears to be the most used by planners. In fact, there is a
substantial amount of literature on the use of GIS in planning practice [Wit et al. 2009; Pettit &
Pullar 2008; Carsjens & Ligtenberg 2007; Bell et al. 2000; Koeninger & Bartel 1998]. Besides SIT
functionality, GIS is characteristic for offering a wide array of analytical functionality that is
applicable to geo-referenced data. Some functions include overlaying and joining datasets,
performing spatial queries and spatial modelling [Stillwell et al. 1999]. GIS represents a generic
tool given its application in different fields such as engineering, hydrology and planning.
17
The literature shows that amongst those planners using GIS, the majority use a very
small part of GIS functionality, such as for visualising spatial data or performing spatial queries,
and only a small community of planners use more advanced functions such as buffering, overlay
and union [Klosterman 2013; Merry et al. 2008; Geertman & Stillwell 2003a; Nedovic-Budic
1998]. The research community agrees upon GIS being an essential software tool for planners
as it assists them in many activities [Arciniegas et al. 2013; Vonk & Ligtenberg 2010]. However,
there is also consensus that GIS has not been fully embraced by planners [Geertman & Stillwell
2003a]. Another dimension to this discussion is that, according to some researchers [Batty 2007;
Klosterman 1999], GIS does not allow for performance of the whole range of tasks required by
planners such as plan generation and evaluation (see planning stage 3 by Vonk et al. [2007a] in
Section 2.1). This is associated with GIS providing descriptive rather than predictive modelling
as argued by Batty [2007].
For improving the predictive capability of GIS, Batty [2007] suggested linking GIS to
urban models and tools. For instance, Malczewski [1999] reported the linkage of GIS with Multi-
Criteria Decision Analysis (MCDA) tools for allowing planners to perform site selection tasks. In
that case, GIS was used for preparing input data to perform the MCDA and for mapping its
output [Malczewski 1999]. Some scholars define the products resulting from linking GIS and
predictive models as Planning Support Systems (PSS), i.e. software tools specifically designed to
assist planners in their activities [Brits et al. 2014; Vonk 2006; Klosterman 1999]. A detailed
discussion on what constitute PSS is provided in the following section.
2.5 Planning Support Systems (PSS)
In order to support planners and their activities, particularly strategic planning activities,
Planning Support Systems (PSS) have been developed. These software tools are intended to help
planners address one or multiple tasks and stages of the planning process [Geertman & Stillwell
2009b; Klosterman & Pettit 2005; Brail & Klosterman 2001]. As with many terms, a unique
definition of PSS does not exist. In the literature, PSS are seen as both, conceptual frameworks,
including planning data, information, knowledge, theory and tasks, and software tools for
planners [Geertman & Stillwell 2003a; Brail & Klosterman 2001; Klosterman 1997; Harris & Batty
1993]. This study considers PSS as software tools. Like many software tools, PSS have continued
to evolve over the years [Krause 2013]. Indeed, three main notions of PSS have been identified
in the literature. The first two notions, which are outlined below, are along more traditional
lines, while the third is the most recent and broadest notion.
18
The first notion sees PSS as synonymous with modelling software (also known as land
use models, urban models or urban modelling tools) [Couclelis 2005]. Effectively, modelling
systems have existed in the planning scene since the 1960s and were known as so-called Large
Scale Urban Models (LSUM) [Stimson et al. 2012]. Besides their ability to predict growth and
land use development, the characteristics of LSUM involved spatial disaggregated modelling
across metropolitan areas [Lee 1973]. Similarly to PSS, LSUM were intended to support planners;
indicating that the concept of PSS existed well before the term PSS arose in the late 80s and
early 90s [Vonk 2006; Geertman & Stillwell 2003a].
The second notion is the one adopted throughout this entire thesis (see Section 1.4). It
is based on the first notion and suggests the need for software tools to be model-based, i.e.
“include the use of techniques such as agent-based modelling (e.g. UrbanSim), rule-based
modelling (e.g. What if?) and cellular automata (e.g. SLEUTH)” [Brits et al. 2014: 2], in order to
be designated as PSS. More experienced scholars of the field support this definition [Vonk 2006;
Klosterman 1999]. Modelling and forecasting capability is also a central functionality in this
notion of PSS [Krause 2013; Densham 1991; Harris 1989]. However, these PSS have been
complemented with GIS functionality and visualisation capability [Geertman & Stillwell 2009a;
Vonk 2006; Pettit 2005; Klosterman 1999], and therefore constitute a subset of Spatial
Information Systems (SIS). Modelling and GIS capabilities mainly contribute to transforming
spatial data and information in development scenarios that assist planners in making better
decisions about current and future land uses [Geertman & Stillwell 2009b; Pettit et al. 2008].
This computerised and evidence-based assistance provided to planners in the process of making
decisions related to complex spatial problems is why these PSS are also seen as a subset of SDSS
[Arciniegas et al. 2013; Pettit et al. 2008]. With reference to the planning stages as indicated by
Vonk et al. [2007a] and reported in Section 2.1, these PSS are mainly designed to support
planners in the stages 3-6, i.e. advanced analysis (e.g. scenario generation, impact assessment),
plan choice and implementation.
In respect to the development stage of such PSS, Geertman and Stillwell [2004]
emphasised that a minority of these PSS has reached a level of maturity that allows them to be
sold as off-the-shelf software tools. Rather, most of them have been developed for research
purposes and have not passed the prototype stage. Some of the more popular PSS are
CommunityViz [Walker & Daniels 2011], SLEUTH [Silva & Clarke 2002], UrbanSim [Waddell 2002]
and WhatIf? [Klosterman 1999].
Building upon the previous notion is the third notion of what constitutes PSS [Krause
19
2013]. It has been stated that, to the fullest possible extent, any computer-based tool that
contributes to planners’ work process can be considered a PSS, from spreadsheets to websites
[Couclelis 2005]. This broader notion takes into account that planners use tools other than the
modelling software of the more traditional notions, which are essential for performing their
tasks. It does not exclude the modelling software tools but it includes tools that are more
generic, more practical and of daily use such as GIS, graphics software and word processors.
Another difference is that while modelling software tools are exclusively for strategic planning
tasks, tools of the broader notion are also used by statutory planners. In comparison with the
more traditional notions, the broader notion is hardly promoted in the literature. Also absent in
the literature is, to the author’s knowledge, a discussion that compares and analyses these
different notions.
Figure 2.1: Conceptual model of the three PSS notions
Figure 2.1 illustrates a conceptual model of how the three PSS notions and their
relationships are understood in this thesis. It shows that the third notion of PSS is the only one
that does not necessarily deal with spatial information. No matter what notion is adopted, a
wide range of PSS can be identified in publications of scholarly articles and edited volumes on
PSS (e.g. Geertman et al. [2015; 2013; 2009b], Brail [2008], Geertman & Stillwell [2003b], Brail
& Klosterman [2001]). PSS vary substantially in terms of, for example:
• tasks they address (e.g. site selection, land use allocation),
• capabilities they possess (e.g. evaluation, communication),
• outputs they provide (e.g. digital map, table displays),
• modelling techniques they adopt (e.g. rule-based, cellular automata),
20
• spatial scale of analysis (e.g. national, regional, subregional),
• form in which they occur (e.g. standalone software, GIS module, web application),
• access (e.g. open source, proprietary),
• development stage (e.g. early, advanced).
According to the literature, this is the result of the rapid development in computer systems and
ICT in recent decades [Geertman & Stillwell 2009b] and a lack of standards in PSS development
[Pettit & Wyatt 2009]. It appears that PSS and their use in planning has gained in dynamics over
the years. In the past, there was increasingly the notion that PSS had to provide the solution to
a planning problem. For achieving this, PSS performing comprehensive analysis were developed.
Due to their complexity, such PSS were often more cumbersome to use. Nowadays, there is a
rather rapid development of more generic software tools and applications. Planners eventually
use them in their activities, whereas, depending on the definition, they represent PSS. In
comparison to the past, planners also increasingly expect PSS to provide information on what
should be further analysed instead of the solution.
2.6 PSS repositories and reviews
It is difficult to demarcate a clear boundary which encapsulates a complete overview of the wide
range of existing PSS and their functionality. Individuals and groups based at scientific or
research institutions have created repositories and reviews of PSS to facilitate systematic
analysis, compare functionality and select appropriate PSS. Some of these repositories and
reviews are presented as follows.
Between 2000 and 2001, Geertman and Stillwell [2003a] created an online repository
(www.nexpri.nl, accessed March 14, 2013), including not only PSS but also data, methods and
theories for the planning discipline. The authors identified a high diversity among the PSS and
their functionality. They explained this diversity with the diverse provenance of the PSS and their
functionality being adapted to planning processes and problems of the country and region
where they were developed.
Hughes and Heckbert [2012] created a review of nineteen PSS. The review was
developed based on the need of the Western Australian Department of Planning to identify
appropriate software tools. The nineteen PSS were included in the review because they fulfilled
initial requirements of the Planning Department staff. To facilitate identification of appropriate
software tools, results of an assessment which examined the nineteen PSS against a set of
variables such as cost, user friendliness, data and technical requirements, were also included in
21
the review.
The Spatial Decision Support Knowledge Portal [The Redlands Institute 2012] includes
information on PSS as well as on software tools not primarily designed for planners.
Furthermore, it provides data, methods and literature relevant for people dealing with spatial
issues and decision-making in various domains such as Environment, Forestry, Climatology and
Geology.
The United States Environmental Protection Agency (U.S. EPA) [2000] undertook a
review of twenty-two PSS and particularly on models for assessing the effects of community
growth and land use change. The review provides information on functionality, strengths and
weaknesses of the models. The aim was to support potential users such as planners, citizens and
decision-makers in choosing appropriate models.
Despite the relatively large number of available PSS, the high potential value PSS offer
for increasing the effectiveness of planning activities, and extensive resources put into their
development and research, the adoption of PSS in planning practice has been very limited
[Brömmelstroet 2013; Williamson & McFarland 2012; Brömmelstroet 2010b; Vonk & Geertman
2008; Klosterman & Pettit 2005]. Past studies examined factors hampering PSS adoption. The
results of those studies are reported in the following section.
2.7 Bottlenecks for low adoption of PSS
For decades, factors that obstruct the adoption of PSS in planning practice have been
investigated. Back in 1973, Lee [1973] examined factors that limited the adoption of LSUM. He
claimed LSUM to be data hungry, expensive, complicated, to be causing more uncertainties than
benefits and that such models did not provide resultant outputs at an appropriate level of detail.
Some of these limitations have continued to affect LSUM as stated by the same author in a
subsequent paper [Lee 1994] and PSS in more recent literature (e.g. Brits et al. [2014], Wang
[2013], Brömmelstroet [2010a]).
Data hungriness is not the only issue with data that hampers PSS adoption. Data
collection and preparation can be very time-consuming activities. These activities have been
further revealed to be among two of the most important challenges in the process of PSS
adoption [Brits et al. 2014; Klosterman 2013; Waddell 2010; Bishop & Foerster 2007; Harding
2007; Timmermans 2003].
A number of researchers stressed that model complexity hampers the adoption of PSS
due to decreased transparency and learnability [Brits et al. 2014; Klosterman 2013;
22
Brömmelstroet 2010b]. The more complex a model the larger the number of assumptions that
underlie the analysis [Klosterman 2013]. Planners require assumptions and functions to be
transparent for relying on model outcomes and for making decisions based on them without
having to simply trust them [Brits et al. 2014; Bishop & Foerster 2007]. Especially LSUM but also
PSS are known as black boxes in that methods and processes they use are not always clear to
users. Past literature reported that models should be validated and limitations and methods
explained, in order to increase users’ reliability in models [Brits et al. 2014; Bishop & Foerster
2007].
The willingness to invest in training in the use of complex systems is limited by constant
resource constraints in the planning profession [Houghton et al. 2014; Krause 2013]. Research
showed that GIS training had a positive effect on its adoption in planning practice. A number of
researchers argue that more training of PSS in planning education is required to increase
planners’ awareness and experience with PSS and therefore PSS adoption in planning practice
[Brits et al. 2014; Houghton et al. 2014; Krause 2013].
Additionally to educational systems, Houghton et al. [2014: 29] showed that regulations,
i.e. systems of rules imposed by authorities that regulate planners’ activities, limit planners’
scope of software adoption by stating: “Planners felt that they were tightly controlled by
regulation and process. This limited what new and innovative approaches they could use …There
was a feeling that the planner is given a specific set of guidelines for how they operate and that,
beyond that, they had little influence”. Particularly, regulations lag behind technological
developments hampering adoption of new technology in Australian planning organisations.
Houghton et al. [2014] also suggested more dialogue with research disciplines,
particularly with Human-Computer Interaction (HCI) and software designers. The authors
recommend projects that involve partnering of planners with researchers and designers as a
way of reciprocal understanding of situations and experiences, in order to foster ICT to be part
of planners’ daily work and foster innovation in planning practice.
Additional empirical studies show that factors or bottlenecks, as they are also referred
to, hindering the adoption of PSS can relate to technical, human, organisational and institutional
aspects and are of different importance. Vonk et al. [2005] asked eight-hundred people from
the planning discipline (consultants, researchers, PSS developers and users) throughout the
world to participate in an online survey to identify the relative importance of potential
bottlenecks. About one-hundred respondents completed the survey, predominantly from North
America and Europe. The results showed that potential users are reluctant to use PSS because
23
they consider their experience with PSS inappropriate and because they have low awareness of
existing PSS. Low usability of those systems was revealed to be a more technical bottleneck.
For a more focused study on bottlenecks related to instrumental quality, user
acceptance and diffusion of instruments to user, Vonk et al. [2007b] conducted interviews and
a literature review. The interviews were conducted in Dutch planning organisations with forty-
three geo-information specialists, planners and managers. Poor fit of PSS to planners’
expectations was revealed to be the most important bottleneck. More precisely, planners
demanded simple systems while developers supplied advanced systems. The adoption of PSS is
further obstructed by planning organisations and the limited intention by planners to actually
use those systems.
In a more recent study, Brömmelstroet [2010a] conducted a web-based survey which
had its focus on the Netherlands and land use-transportation PSS. The survey was conducted
with sixty land use and sixty-two transportation planners. In this study, the respondents
expressed that PSS lacked transparency, usability and flexibility to adapt the systems to their
needs. Developers should take into account that PSS users want to tailor the system they use,
and should create PSS capable of being adapted to the differing needs and preferences of
planners [Geertman & Stillwell 2004]. In order to satisfy this basic user requirement, they should
apply methodologies that allow making changes to the system for creating PSS with a wide
applicability [Brits et al. 2014].
Poor attention to PSS design - development as well as lack of design standards - appear
to be a main reason for many bottlenecks that limit the adoption of PSS [Brits et al. 2014; Vonk
et al. 2007a; Vonk et al. 2006; Geertman & Stillwell 2004]. Specifically, developers do not know
and/or do not take into proper consideration planners’ needs and expectations during the
design process of PSS [Brits et al. 2014]. There is a clear mismatch between PSS functionality and
the way it is provided and what planners expect, which causes PSS not being attractive to
planners [Wang 2013; Williamson 2012]. In fact, PSS development has been very much
technology-oriented [Vonk & Geertman 2008]. In other words, developers have predominantly
adopted a system-centred design approach, i.e. they focused on the system and design systems
that meet their expectations rather than the targeted needs of the end user.
The literature suggested that both functionality and usability of PSS have to be improved
[Brömmelstroet 2013; Williamson 2012; Vonk & Ligtenberg 2010; Pelizaro et al. 2009; Couclelis
2005]. In order to improve these, the collaboration of software engineers and planners during
PSS design and development is required [Pettit et al. 2014; Krause 2013]. Planners should be
24
involved, in order to allow developers to perform a better analysis of planners’ requirements
and to evaluate if PSS meet their requirements. In fact, Allen [2008] suggested rigorously
conducting evaluations of PSS which are fairly rare to date as well (e.g. Arciniegas et al. [2013],
Vonk & Ligtenberg [2010]). Only PSS that comply with planners’ requirements can effectively
support their work practice and are more likely to be more widely used [Vonk 2006; Geertman
& Stillwell 2004].
2.8 Usability
Users experience software through its interface and, as a result, users regard the interface as
the most important element of a software product. Its quality affects users’ experiences when
performing tasks. Low-quality interfaces can develop frustration when they are hard to use. The
community of the HCI research field has identified factors of the overall software quality [McCall
1994]. A fundamental quality factor is, as widely acknowledged by HCI research, usability.
Several definitions of usability can be found in the literature. The three definitions
provided by i) Nielsen [1993], ii) the standard International Organization for Standardization
(ISO) and International Electrotechnical Commission (IEC) 9126-1 [ISO 9126 1998b] and iii) 9241
[ISO 9241 2010a] are reported as follows. They include the most significant aspects of usability
and are widely acknowledged by the HCI and the Software Engineering communities [Costabile
2001].
Nielsen [1993] defines usability as how well people can use the provided functionality.
Specifically, he characterised usability by five attributes:
(1) learnability, i.e. how easy users learn to use the system,
(2) efficiency, i.e. how quick users perform tasks once the users have learned to use the
system,
(3) memorability, i.e. how easily users remember to use the system after a period when
they have not interacted with it,
(4) errors, i.e. how easily users make errors, how severe are these errors and how easily
users recover from the errors,
(5) satisfaction, i.e. how pleasant it is to use the system.
According to Nielsen, usability influences the acceptability of interactive systems by end users.
He created a model (see Figure 2.2) that conceptualises acceptability of interactive systems by
end users through several attributes such as cost, reliability, compatibility with existing systems,
25
usefulness. Usefulness reflects whether the system allows people to achieve their desired goals
easily and with satisfaction. It is decomposed in two dimensions which are usability and utility.
The latter reflecting whether the functionality provided by a system can do what is needed.
Figure 2.2: Nielsen’s [1993] model on system acceptability
The standard for software product evaluation ISO 9126 provides the view of software
engineering on software quality. It describes usability as the “capability of the software product
to be understood, learned, used and attractive to the user, when used under specified
conditions” [ISO 9126 1998a]. Usability is furthermore characterised by five attributes:
(1) understandability, i.e. the capability of the software product to show what tasks can be
performed with it and in which context of use,
(2) learnability, i.e. the capability of the software product to help the user learn its
functionality,
(3) operability, i.e. the capability of the software product to enable the user to operate
and control its functionality,
(4) attractiveness, i.e. the capability of the software product to be pleasant for the user,
(5) compliance, i.e. the capability of the software product to support standards,
conventions, style guides about usability [ISO 9126 1998a].
Building upon the previous standard, ISO 9126-1 (Information-Technology Software
Product Quality) stresses the notion of designing for quality [ISO 9126 1998b]. Specifically, it
describes the overall software product quality in a model which identifies three components,
i.e. internal and external quality and quality in use. Each component is characterised by
attributes that can be measured. The first component focuses on static properties of the
software product (e.g. path length of the code), the second addresses the behaviour of the
software product when combined with computer systems (e.g. response time when code is
executed), and the third concerns the interaction between user and the software product in the
real-world context (e.g. number of tasks performed in a specific time period) [Bevan 1999]. More
26
precisely, the component quality in use considers software quality as seen by the user and is
characterised by four attributes, i.e. effectiveness, productivity, safety and satisfaction.
Two of these attributes, namely effectiveness and satisfaction, can also be found in the
usability definition in the standard of Ergonomic Requirements for Office Work with Visual
Display Terminals ISO 9241 [ISO 9241 2010a]. Specifically, ISO 9241-210 defines usability as “the
extent to which a product can be used by specified users to achieve specified goals with
effectiveness, efficiency and satisfaction in a specified context of use” [ISO 9241 2010b], where
effectiveness refers to how accurately and completely users can perform specified goals in a
certain environment; efficiency is defined as the resources utilised in relation to the accuracy
and completeness of the goals achieved, and satisfaction refers to how comfortable and
acceptable the product is for its users.
Table 2.1: The three usability definitions and their attributes in comparison
Nielsen [1993] ISO 9126 [ISO 9126 1998a] ISO 9241-210 [ISO 9241 2010b]
Definition how well people can use the provided functionality
the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions
the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use
Attribute
Learnability: how easy users learn to use the system
Learnability: the capability of the software product to help the user learn its functionality
Efficiency: how quickly users perform tasks once the users have learned to use the system
Efficiency: the resources utilised in relation to the accuracy and completeness of the goals achieved
Satisfaction: how pleasant it is to use the system
Attractiveness: the capability of the software product to be pleasant for the user
Satisfaction: how comfortable and acceptable the product is for its users
Errors: how easily users make errors, how severe are these errors and how easily users recover from the errors
Memorability: how easily users remember to use the system after a period when they have not interacted with it
Understandability: the capability of the software product to show what tasks can be performed with it and in which context of use
Operability: the capability of the software product to enable the user to operate and control its functionality
Effectiveness: how accurately and completely users can perform specified goals in a certain environment
Compliance: the capability of the software product to support standards, conventions, style guides about usability
27
The usability definitions and attributes outlined above are summarised in Table 2.1. The
two ISO definitions stress that usability depends on the “conditions” [ISO 9126 1998a] and
“context” [ISO 9241 2010b] in which a software tool is used, i.e. on the users, the tasks and the
physical and social environments. Efficiency, as defined by Nielsen [1993], is restricted to time
required for performing a task while ISO 9241-210 [ISO 9241 2010b] also refers to financial
expenses with “resources”. Attractiveness, as defined by ISO 9126 [ISO 9126 1998a], can likely
be seen as a sub-attribute of satisfaction. Errors, memorability, understandability, operability,
effectiveness are related in that they all provide indications on how easy a software tool can be
used and how well a task can be performed whether in terms of error rate or task accuracy.
Finally, compliance appears to be a quite specific usability attribute of ISO 9126 [ISO 9126
1998a].
It is worth remarking that the attributes of usability have a different importance
depending on the type of system. For example, learnability plays a significant role for the
usability of websites because they are used by occasional users browsing on the web, who have
to easily understand what the website is about.
2.9 User eXperience (UX)
The objectives of Information and Communication Technology (ICT) development have changed
over the years [Costabile & Buono 2013]. In the past, ICT aimed to provide useful, and in
particular usable, functionality; today, there is more of a focus on involving users in positive and
engaging experiences. This is reflected in that ICT designers put increasing emphasis on
aesthetics and attractiveness which, according to the literature, help improve users’ experiences
[Costabile & Buono 2013; Law et al. 2009]. In particular, mobile systems and the Internet, which
accompany people everywhere, have generated great attention on user experiences, going
beyond the traditional usability attributes [Roto et al. 2008]. Depending on the system, different
user experiences, for example, pleasure and fun with video games or reliability with medical
software, are expected.
The importance of people’s overall experience when interacting with software systems
has also been recognised by the HCI discipline [Law et al. 2009], which established a field of
study, called User eXperiences (UX). UX focuses on the user and deals with studying and
evaluating UX [Albert & Tullis 2013; Roto et al. 2011]. By asking the user questions such as
‘Would you recommend the system?’, ‘How do you feel about the system and yourself after
using it?’, information on UX can be gathered [Albert & Tullis 2013]. UX is influenced by different
28
factors such as the properties of the system that is used (e.g. functionality, aesthetics,
responsiveness, brand, image), the state of the user (e.g. motivation, mood, expectations), the
context of use (e.g. task, environment) [Roto et al. 2011]. It is also a powerful tool to evaluate,
improve and design systems, in order to provide particular UX. With respect to the user
interface, a good layout with appropriate fonts and colours and engaging interaction techniques
are likely to contribute to a better UX [Balzarini et al. 2013], which is also referred to as UX design
[Roto et al. 2011].
Although UX and usability are two different concepts, they relate to each other. For
instance, user satisfaction which is a typical aspect of UX, is also a usability attribute. UX data
can help identify any usability problems during evaluations [Albert & Tullis 2013]. On the other
hand, usability influences UX [Roto et al. 2011]. In this research, both usability problems and
planners’ experiences when interacting with PSS were investigated, in order to draw more
comprehensive conclusions on how usability and UX affect PSS adoption.
Since the origins of HCI in the 80s, researchers have been debating how to develop
computer systems that are better suited to the needs and expectations of people who use them.
Models, methodologies and techniques have been defined by academic researchers, in order to
create systems that are not only usable, but also able to generate a pleasant UX. However,
despite the whole body of HCI knowledge generated in these years, the goal of creating systems
that people appreciate and enjoy using has not been reached in many cases. Indeed, the
literature reported various studies (e.g. Ardito et al. [2014], Bak et al. [2008], Ji & Yu [2006],
Vredenburg et al. [2002], Rosenbaum et al. [2000]), which showed that too many software
development companies continue to either neglect or not properly address usability and UX.
Some contributions of HCI research for developing usable and engaging software systems are
reported in the following subsections.
2.10 System-centred vs. user-centred design
The essence of software design processes is to build software [Ghezzi et al. 1991]. Software
functionality and user interfaces are products of design processes. Poor design can lead to low
software quality and usability. This is why designers have to take quality and usability into
account during software development [Madsen 1999; Mayhew 1999; 1992]. Usability problems
and design modifications needed to solve them, are more difficult to make and costly at the end
of the development cycle rather than during [Bias & Mayhew 2005; Hackos & Redish 1998;
Nielsen & Landauer 1993]. In particular, developers should design software according to what
29
users need. This is the only way to ensure that system functionality complies with user
requirements.
One reason why software ends up to be hard to use is that there is a mismatch between
what and how functionality is provided by the system and what users require. It is a mistake to
assume that users adapt to functionality once the software has been completed [Costabile
2001]. In order to avoid such a mismatch, software engineers should know user requirements
and take them into account during software development [Rubin 1994]. Instead, too many
developers continue to adopt a so-called system-centred design approach which focuses on the
system, its functional aspects and engineering goals such as code efficiency. They put little focus
on user needs and so do little for developing software that is supportive and usable for target
users [Ardito et al. 2014; Bak et al. 2008; Cajander et al. 2006; Venturi & Troost 2004; Boivie et
al. 2003].
An example of a system-centred design process is the waterfall model (see Figure 2.3).
It involves a uni-directional process of activities beginning with the collection and specification
of requirements, in order to create a general design. Based on this, a more detailed design is
specified which is coded, integrated and tested. The last activity comprises finalisation and
ongoing maintenance of the product [Sommerville 1996].
Figure 2.3: The waterfall model [Sommerville 1996]
30
There are some specific limitations of the waterfall model in relation to HCI
recommendations for designing usable systems. For instance, requirements are not primarily
collected with target users but with people who work in the same organisation as developers
and are responsible for software acquisition such as IT specialists and managers. The problem is
that they are likely to have different requirements, for example, due to diverse skills and
knowledge, and they may not have precise knowledge of target users’ requirements. In addition,
requirements focus more on what the system should provide and less on how it does this. As a
result, design aspects such as easy to remember, easy to use, etc. are neglected. Other problems
of the waterfall model are related to system testing. Specifically, testing the system towards the
end of the cycle, which is what the model calls for, does not allow radical changes to be made
to the design and so it limits the possibility to best adapt the system to requirements.
Furthermore, developers adopting the waterfall model focus more on testing engineering goals
such as code efficiency or innovative functionality which reflect their goals and interests rather
than the real need of target users. By doing so, the system is barely tested against user needs
and usability which is essential for developing software that supports target users in their
activities.
In contrast, attention is given to users’ needs and usability testing in the so-called user-
centred design process. Indeed, it is a principle of this process to involve target users from the
beginning of the development process. Their characteristics (who will use the system?), their
tasks they want to complete with the system (what for?) and the context of use (where and
how?) are identified, in order to specify their requirements. These requirements are
implemented into system prototypes. Prototypes allow exploration of design solutions and
functionality. Early involvement of users and their requirements limits the risk of critical design
mistakes and allows a focus on functionality that is really needed.
The second principle consists of implementing the overall system iteratively through
prototypes of increasing complexity. However, before a more complex prototype is designed, it
must be ensured that the current prototype meets users’ requirements. This is achieved by
evaluating the prototype with target users, which is also the third and last principle. The
evaluation of prototypes is fundamental, in order to avoid testing a system only at the end of its
development. It allows detecting the impact of a prototype on target users and possible
interaction and usability problems that are much cheaper to solve at the early phase of the
design and development [Ardito et al. 2014; Larusdottir 2012; Bias & Mayhew 2005].
The user-centred design process, whose importance for developing high-quality user
31
interfaces has been acknowledged by the HCI discipline [Dix et al. 1998; Preece et al. 1994; Rubin
1994], is reflected in the ISO standard Human-Centred Design (HCD) process [ISO 9241 2010b]
(see Figure 2.4). The numbered rectangles stress key activities to be performed during the
iterative process that consists of:
(1) understanding and specifying the context of use of the interactive system to be created,
(2) specifying user requirements,
(3) producing prototypes in accordance with the identified requirements,
(4) evaluating them.
Figure 2.4: The Human-Centred Design (HCD) process for interactive systems [ISO 9241 2010b]
If the outcome of the evaluation does not meet the specified requirements, it might be
necessary to iterate the process, in order to re-investigate context of use and user requirements
and/or to re-design the prototype. The HCD process takes an important role in this research
because it contributes to improved usability and UX [Roto et al. 2011].
A well-known method for gathering information on users and their context of use is
contextual inquiry [Preece et al. 2015]. This method involves observing users while they are
completing their tasks at their workplace, and subsequently asking the users questions in the
form of interviews. The observer can see with her/his own eyes how software is used in the field
and how software use is influenced by procedures within the organisation, for example, social
interactions of a user with colleagues, that can hardly be simulated in usability labs. The
interviews serve to ask the user questions that emerge during the observation as well as
predefined questions about opinions, motivations and skills that cannot be observed [Karlsson
32
et al. 2007; Ackerman 2000; Sutcliffe 2000; Seaman 1999].
Prototype evaluation is recognised as a key activity of system development processes
also in the star life cycle model (see Figure 2.5) by Hartson and Hix [1993]. The model shows that
system evaluation follows after any system design activity, i.e. implementation, task and user
analyses, requirement and usability specifications, conceptual and formal design, has been
performed. Based on the results of the evaluation, those activities are performed again, leading
to continuous evolvement of the system. The process is stopped when the developer is satisfied
with the obtained product design. The issue with this model is that developers do not have to
identify any user requirements before beginning software design. This goes against
recommendations by the HCI discipline and as reflected in the HCD process.
Figure 2.5: The star life cycle model [Hartson & Hix 1993]
2.11 Usability evaluation
Usability of user interfaces heavily impacts systems’ use. It is not just an abstract concept but it
can also be evaluated and improved [Nielsen 1993]. Thus, systems’ evaluation is one of the most
important steps in the development of user interfaces [Somervell & McCrickard 2004].
Evaluations can be performed at different development stages of a system. If a system is
evaluated during its development, it is referred to as formative evaluation, while evaluations
carried out after a system has been developed are called summative. Formative evaluations are
carried out to check whether developers are implementing users’ requirements well, and to
obtain feedback on design choices. Summative evaluations are performed under real conditions
33
to detect whether design revisions have to be made. There are multiple methods for evaluating
systems’ usability and UX (e.g. Lanzilotti et al. [2011], Preece et al. [2015]). The most commonly
adopted are inspection methods and user-based methods.
2.11.1 Inspection methods
Inspection methods involve evaluators who assess a system and provide judgments based on
their expertise on usability and a list of usability principles [Mack & Nielsen 1994]. Evaluators
are typically usability experts who have a good understanding of usability principles and of
critical functionality that could cause problems to users. The great advantage of inspection
methods is that they are cost-effective, since they do not require users nor special equipment
or lab facilities. In a limited amount of time and with only three to five evaluators, it is possible
to detect a wide range of a system’s problems [Lanzilotti 2006; Vetere et al. 2003; Nielsen &
Landauer 1993].
The main disadvantage is that the evaluation is subjective as it depends on the
evaluators’ skills and experience. Furthermore, usability problems identified by usability experts
could differ from those that real users encounter under real conditions. As no real users are
involved, inspection methods allow evaluators only to hypothesise what could be users’
experiences. Heuristic evaluation and cognitive walkthrough are examples of inspection
methods [Larusdottir 2012; Mack & Nielsen 1994].
Heuristic evaluation involves a small set of experts who inspect the system and evaluate
the interface against a list of heuristics [Nielsen 1993]. The main drawback of this technique is
that heuristics are often generic and non-specific [Lanzilotti et al. 2011; Doubleday et al. 1997].
Evaluation depends therefore on the evaluators’ skills and experience, i.e. their understanding
of usability principles and their ability to apply them.
To facilitate system evaluation for novice evaluators with no or little usability expertise,
system-specific heuristics have been established, typically by people who have knowledge of
both usability and the system application domain. Specific heuristics provide more support and
guidance than generic ones in that they call the evaluators’ attention to functionality that likely
causes usability problems for the specific system. Specific heuristics are usually formulated at a
lower level of abstraction than more generic heuristics. Specific heuristics often stem from
existing heuristics [Ling & Salvendy 2005] and are validated before they are used [Baker et al.
2002].
In cognitive walkthrough, the evaluators simulate steps and actions required for
34
completing users’ tasks. After each step or subtask, the evaluators answer a set of questions
which reveal whether the user would be able to complete the task and whether and what
problems he/she could encounter [Wharton et al. 1994; Polson et al. 1992].
2.11.2 User-based methods
In user-based methods, system evaluation is performed by involving system target users. They
are generally considered a complete form of evaluation because it occurs through samples of
real users [Lanzilotti 2006; Costabile 2001]. A common method is user testing, i.e. users interact
and perform tasks with a system [Preece et al. 2015; Dix 2009]. During the interaction, data on
user performance, such as execution time, number of errors, and on UX can be collected. If the
user tests are carried out in a laboratory, it is difficult to reproduce realistic situations [Preece
et al. 2015; Lim et al. 1996]. For instance, conducting user tests of mobile systems in a lab is
critical because their ubiquitous nature is not entirely considered. Furthermore, complex and
specialised systems are often only used by a small group of professionals who are difficult to
recruit for a user test session. Consequently, recruitment also includes training participants, in
order to allow the testing of sophisticated system functionality [Costabile 2001].
Observation is a common method during user tests. The evaluator usually takes notes
of observations about user behaviour. It is a precise and reliable method because data gathered
corresponds to what the evaluator has seen. Because the evaluator has to take time to observe
users individually, it is also a time and cost-expensive method [Costabile 2001].
A cost-effective technique in user testing is thinking-aloud, which requires users to
speak out loud their thoughts while performing tasks. This technique offers a window into the
users’ mental models, allowing detection of their perceptions and emotions, as well as any
misconceptions about the system and the interface elements which cause them. It provides
useful results even with a small number of users [Nielsen & Landauer 1993; Virzi 1992].
Other more controlled techniques for gathering data about users’ experiences and their
attitude to the system can be obtained through survey techniques, such as questionnaires and
interviews [Shneiderman & Plaisant 2010; Lanzilotti 2006]. Questionnaires and interviews are
especially valuable to assess user satisfaction and other hedonic qualities of UX. They are often
executed after participants interact with the software.
Depending on the questions, qualitative or quantitative information can be gathered
through questionnaires [Preece et al. 2015]. As formulating questionnaires is a delicate task, it
is recommended to use questionnaires that have been tested. A standard questionnaire in
35
usability evaluations is the System Usability Scale (SUS) [Brooke 1996] (see Appendix A).
Although low response rate is a well-known issue, questionnaires allow reaching a relatively
large number of people and gathering data at low cost [Preece et al. 2015]. This is particularly
important for evaluators who intend to carry out statistical analyses.
Interviews have been seen as “conversation with a purpose” in past years by Bingham
and Moore [1924: 3]. The purpose determines whether a structured or unstructured interview
is conducted. The former consists of predefined questions in a specific order and is more
appropriate for gathering quantitative information as the interview can be easily repeated.
Unstructured interviews are open-ended conversations which allow the interviewer to ask
spontaneous questions if something is unclear as well as giving the interviewee the opportunity
to provide in-depth responses [Karlsson et al. 2007].
Interviews can be conducted one-on-one or in groups. Focus group can be seen as an
example of a group interview if the session is guided by one or more questions that participants
should answer. Focus group is a technique that can be used for multiple purposes due to its
several advantages [Choe et al. 2006]. A general advantage is that it allows a relatively large
sample size which gives the opportunity to gather experiences of multiple people at once
without dramatically increasing time investment of the interviewer. However, the disadvantage
is that the level of participation in the discussion might greatly differ amongst participants. In
HCI, focus groups have been used for improving software quality through the identification of
user requirements and usability problems [Choe et al. 2006].
2.12 Previous PSS development approaches and evaluation studies
There is a significant body of literature on PSS applications in specific contexts (e.g. Sharma et
al. [2011], Hoeven et al. [2009], Hopkins et al. [2004]), also in edited volumes (e.g. Geertman et
al. [2015; 2013], Geertman & Stillwell [2009b], Brail [2008], Geertman & Stillwell [2003b], Brail
& Klosterman [2001], Stillwell et al. [1999]). From this comprehensive literature review, there
has been no direct application of the HCD process in the design and development of PSS to date.
However, the review of the literature has brought to the forefront two approaches, i.e. the
socio-technical and the co-design approaches, which have some commonalities with the HCD
process, in so much as they involve close collaboration with end users during software
development.
The socio-technical approach was adopted in Vonk and Ligtenberg [2010] to develop a
PSS prototype for collaborative sketch planning implemented on a touch table. During
36
development, planning practitioners and developers met five times. In each meeting, the
practitioners tested the latest version of the PSS and discussed with the developers their
experiences, their work process and system requirements. After each meeting, the developers
implemented users’ requirements in system functionality to improve the PSS. Subsequent to the
development, the PSS and another PSS for collaborative sketch planning implemented on a
touch table and developed through a traditional design process, were tested. The evaluation,
that involved planners, showed that the traditionally developed PSS was rejected by planners
because of their poor functionality and usability, while the other PSS was much more accepted.
The co-design approach [Sanders & Stappers 2008] was adopted by Pettit et al. [2014]
for the development of a PSS for precinct planning, the Envision Scenario Planner. The
developers created a prototype of the PSS including design options for some system attributes.
Planners were involved in the development in that they commented on the prototype and the
design options during workshops. Based on the workshop outcomes, the PSS prototype was
modified and evaluated by end users. This process was repeated multiple times until the
developers answered their design questions.
The value of user involvement and the HCD process has been acknowledged in the
design of a wide range of software tools which currently does not extend to the category of PSS
software. Of particular relevance to this study, a considerable body of literature called for
understanding users’ needs and applying the HCD process in the development of GIS-based
applications [Delikostidis et al. 2015; Haklay 2010; Haklay & Zafiri 2008; Pucher 2008; Tsou &
Curran 2008; Haklay & Tobon 2003]. Such studies revealed that adopting the HCD process
overcame usability issues identified in the past systems.
The literature showed that most PSS evaluations have been conducted within
collaborative settings, i.e. workshops in which domain experts (e.g. planners, environmentalists,
transport engineers) were involved, and in the Netherlands [Pelzer et al. 2016; 2014; Arciniegas
et al. 2013; Salter et al. 2009]. The analysed PSS were implemented on a touch table, i.e.
hardware with a horizontal screen, which can be surrounded by groups of people in order to
perform collaborative tasks. In many cases, the domain experts did not interact with the PSS
themselves but often the developers of the PSS, acting as intermediaries, carried out the actions
requested by the domain experts (e.g. specifying indicators, uploading data and running
analysis) (e.g. Brömmelstroet [in press; 2012], Pelzer et al. [2013], Goodspeed [2013],
Brömmelstroet & Schrijnen [2010]). The intermediaries not only helped to operate the PSS but
also explained the input and how to interpret the output [Pelzer in press; Brömmelstroet &
37
Schrijnen 2010]. This approach is also known as participatory modelling [McIntosh et al. 2011].
The focus of these evaluations, mainly using methods such as observation, interview and
questionnaire, was primarily on the impact the use of PSS in groups had on social and planning
processes, such as communication, shared language, decision-making, consensus (on problem,
goals, strategies) [Brömmelstroet in press; Pelzer in press; Pelzer et al. 2014; Goodspeed 2013;
Brömmelstroet & Schrijnen 2010; Salter et al. 2009].
Further evaluations have been conducted of software tools that have not been
specifically referred to as PSS but are potentially useful to assist planners [Pettit et al. 2011;
Meng & Malczewski 2009; Sidlar & Rinner 2007; Nyerges et al. 2006]. Houghton et al. [2014]
actually involved twelve urban planning practitioners of different sectors in Queensland
(Australia) in interviews and a focus group in order to investigate planners’ experiences with ICT
and the role of ICT in planning practice.
Some scholars [Pelzer et al. 2016; 2015; Brömmelstroet 2015; 2013] called for more
standardisation of PSS evaluations. The main reasons for this are to facilitate execution and
comparison of evaluations. In fact, Pelzer et al. [2016] reported difficulties in the organisation
and execution of a PSS evaluation due to the many issues, such as access of PSS, setting up
questionnaires and validity of procedure, that had to be considered. Lack of standardisation also
leads to a different conceptualisation of the quality factors of usefulness, usability and utility.
For instance, Arciniegas et al. [2013] investigated the effectiveness of PSS based on their
usefulness, the clarity of their outputs and their impact on the decision-making process, while
Salter et al. [2009] related PSS effectiveness as to how helpful the software tools were during a
planning task.
Some efforts have been made towards more standardisation of PSS evaluations,
however, much work has still to be done. Recent literature indicated sets of usefulness and
usability attributes that should be considered, forming conceptual frameworks for PSS
evaluation [Pelzer in press; Brömmelstroet in press]. A more practical framework has been
developed whose aim is to assist developers in organising and performing PSS evaluation, such
as defining goals, identifying appropriate evaluation techniques, identifying data gathering
methods [Russo et al. 2015] (see also Chapter 5).
2.13 Conclusion
Considerable effort has been made by the research community to improve PSS adoption. This
includes development and reviews of PSS, understanding bottlenecks hampering PSS adoption
38
as well as conducting workshops for evaluating PSS. The acknowledgement that PSS are not
widely used is a long-standing issue, dating back three decades.
This thesis builds upon previous research and attempts to extend work performed by
researchers in the past. For instance, existing PSS repositories and reviews are not up to date
and do not provide a comprehensive picture of existing PSS. For this reason and to contribute to
improving planners’ awareness of PSS, a comprehensive PSS repository has been developed
within the scope of this thesis and is presented in Chapter 4.
Increasing emphasis has been placed on improving usability of PSS, including PSS
development and evaluation. However, the literature review shows that there is potential for
improving these processes. People involved in developing and evaluating PSS appear not to be
aware and familiar with best practices in HCI. Despite the huge amount of literature on HCI, it is
often not straightforward and not simple for non-experts to keep an overview of procedures,
methods and terms to use. In other words, more appropriate guidance is needed.
This research aims to contribute to this by providing pathways on how to design and
evaluate PSS. For instance, the value of the HCD process in software development has been
demonstrated by years of HCI research [Dix et al. 1998; Preece et al. 1994; Rubin 1994]. This
process is presented in this research and recommended for future development of PSS. In
addition, to assist people in evaluating PSS, a framework that provides guidelines for carrying
out evaluations has been defined [Russo et al. 2015] as reported in Chapter 5.
Due to the lack of empirical PSS evaluations within individual use settings, a user test
with six planners who individually tested three PSS was conducted in this study as reported in
Chapter 6. This allowed observing the planners and their experiences when using PSS and
therefore gaining an overview of what problems planners encounter. Besides recognising that
PSS functionality does not align with planners’ expectations, it emerged that planners are
actually not familiar with PSS and use other software. The latter links to discussions in current
literature on the definition of PSS and on what software PSS actually constitute. To better
understand what, how and why some software is more used than others, this study conducted
a series of interviews with people from the planning discipline. The results of the interviews are
reported in Chapter 7.
Lack of usability is not the only bottleneck hampering PSS adoption. As such, this thesis
examined, in addition to system functionality for improving PSS usability, pathways such as in
education that can contribute to improving PSS adoption. The design of the thesis and a more
detailed description of the methodology used in this study are presented in the next chapter.
39
Chapter 3: Research design
This chapter presents the theoretical and methodological structure of the thesis. First, the
research questions and the theoretical framework underpinning the research are reported.
Next, the main methods used and how they support answering the research questions are
described. The aim of this chapter is to show the relationship between research questions,
theory and adopted methods as well as the link to previous research and knowledge.
3.1 Research Questions (RQs)
This thesis investigates the usability of Planning Support Systems (PSS) and provides
recommendations for improving their adoption by planners. It examines system functionality as
well as other factors that have an influence on whether and to what extent PSS are seen as
usable by planners and consequently can have obstructive or supportive effects on PSS
adoption. Based on this, it provides planning actors, specifically PSS developers and academic
planners, with indications and recommendations on how system functionality can be improved
and other factors can contribute to improving PSS usability and adoption. Specifically, this thesis
aims to answer the following research questions (RQs).
RQ 1: What are identified usability problems and User eXperiences (UX) when interacting with
PSS?
Previous research showed that the usability of PSS is low [Brits et al. 2014; Brömmelstroet
2010a; Vonk et al. 2005]. However, the literature indicates that no matter how complex software
might be it has to be usable [Costabile 2001]. In order to investigate why PSS usability is low and
provide contributions for improving it, a first step in this research is to identify any usability
problems that planners encounter, and their experiences when interacting with PSS. This
information builds the basis for this research. It also aims to increase developers’ awareness of
what impact poorly designed PSS user interfaces can have during interaction and what to pay
attention to in the design process for creating more usable PSS.
RQ 2: What are planners’ expectations of PSS functionality?
People have expectations of software they choose to adopt. These relate to its functionality such
as software platform, data analysis and visualisation techniques. Whether and to what extent
software meets these expectations has been a known issue in the literature. The term
compatibility describes “the degree to which an innovation is perceived as consistent with the
40
existing values, past experiences, and needs of potential adopters” [Rogers 1995: 224].
Furthermore, Rogers [1995: 234] indicates compatibility to be “positively related to its rate of
adoption”. Low compatibility has been identified for PSS and planners in the literature.
Specifically, Vonk [2006] described a mismatch between PSS and planners’ requirements and
their tasks. This suggests that PSS functionality does not allow planners to satisfactorily perform
their activities. In order to improve PSS compatibility, planners’ expectations of PSS functionality
have to be known and implemented by system developers. Through documenting planners’
expectations of functionality, this research results in new findings which are essential in the
development of PSS that are more usable and will lead to greater adoption in planning practice.
RQ 3: How can planners’ context contribute to improving PSS usability and adoption in
planning practice?
The two previous research questions address PSS adoption by planners as influenced by the
system itself and its functionality, i.e. so-called system-related factors. There are, however, also
factors that are non-system-related and of personal, organisational and institutional nature,
such as planners’ skills, management support and regulations, that affect PSS adoption [Brits et
al. 2014; Vonk et al. 2005]. This research is based on the notion that non-system-related factors
can influence whether and to what extent PSS are seen as usable by planners. For instance,
planners’ skills and experience, as an example of a non-system-related factor, affects planners’
ability to use functionality provided by PSS and therefore their satisfaction of PSS usability. As
such, this research examines non-system-related factors and in particular, how they can
positively affect PSS usability and adoption. It investigates activities that, if undertaken by
planning actors, specifically PSS developers and academic planners, influence non-system-
related factors, in order to improve PSS usability and adoption. With the objective that these
activities are performed, this research defines recommendations for PSS developers and
academic planners.
3.2 Theoretical framework
This research draws on the following two theoretical contributions of the literature:
(1) a framework that describes the PSS adoption process as a combination of technical,
individual, social and organisational factors (see Figure 3.1) [Vonk et al. 2005]. The factors
are organised along the following main dimensions:
41
• persuasion influences (e.g. support, marketing and product improvement efforts by
provider),
• adopter characteristics (e.g. culture of organisation, attitude of management and
employees, experience of the planner with technology),
• social influences (e.g. community of practice, social pressure, persuasion by colleagues),
• perceived innovation characteristics (e.g. applicability, usability of the system, system fit
to tasks, relative advantage from using the system, capability of the system to handle
data, accessibility of the data and the system),
• external conditions.
Perceived innovation characteristics are in part conditioned by social influences, adopter
characteristics and persuasion influences. Furthermore, the framework indicates that
continued use of PSS is preceded by four steps, where potential users i) are aware of the
existence of the PSS, ii) consider using it, iii) intend to use it, and iv) decide to adopt it.
The origin of the framework goes back to the willingness of Frambach and
Schillewaert [2002] to combine research of the domains of innovation and management
science to describe the process of adoption and implementation of Information and
Communications Technology (ICT). In fact, they combined Rogers’ diffusion of innovations
[2010] and Davis’ technology acceptance model [1989]. The former saw innovation
adoption and implementation as a five-stage process consisting of:
(1) knowledge, i.e. the individual is aware of the existence of the innovation and gains
an understanding of its functionality,
(2) persuasion, i.e. the individual forms a positive or negative attitude towards the
innovation,
(3) decision, i.e. the individual makes a decision whether to adopt or reject the
innovation,
(4) implementation, i.e. the individual starts using the innovation with some uncertainty
and
(5) confirmation, i.e. the individual decides whether to continue to use or reject the
innovation.
Davis [1989] described in his technology acceptance model that continued use of a
technology is preceded by two steps, i.e. potential users form an attitude towards the
technology and after that, they decide whether to accept it or not. According to the
42
model, ‘perceived usefulness’, ‘perceived ease of use’ and ‘external conditions’ (e.g.
output quality and job fit) are relevant for potential users’ attitude.
Vonk et al. [2005] specialised the framework by Frambach and Schillewaert [2002]
for PSS. The influence factors ‘hardware issues’ and ‘data issues’ have been added.
Furthermore, Vonk et al. [2005] considered the process of PSS adoption to begin at both
an individual and organisational level while originally Frambach and Schillewaert [2002]
considered technology adoption to start at an organisational level.
By investigating factors influencing the PSS adoption process, this research provides
further information on the starting point of this process. It is expected that organisational
factors will dominate individual factors, in particular, that decisions made by the
managements of planning organisations determine whether software is adopted or not.
However, management may not have the skills required to judge software and whether it
is worthwhile to adopt it. In other words, they are committed to opinions of employees
who actually use specific software. This shows that employees can influence the
management and that individual and organisational factors are difficult to separate on the
choice of software.
This research places more emphasis at the individual level because usability and UX
focus on users [Hassenzahl & Tractinsky 2006]. It does not, however, neglect
organisational and other factors. In fact, a user test performed in this research
concentrated on the interaction of potential users with PSS while interviews were
undertaken to investigate procedures within planning organisations (e.g. work division
such as data preparation amongst planning actors) and at universities (PSS development
and transfer to practice). The interviews were conducted with planners at their workplace
which provided an adequate venue to investigate the context of software use in planning
practice. The user test was carried out in a usability lab, i.e. in an unknown environment
for participants. Although the focus is on the user and the PSS, it is likely that participants
do not completely exclude procedures as occurring at their workplaces but involve them
while interacting with the PSS. Influencing factors of the PSS adoption process, that are or
are not studied in this research, are discussed in relation to Figure 3.1 as follows.
In Vonk’s et al. [2005] framework both system-related and non-system-related
factors (see green and red boxes respectively in Figure 3.1) can be identified that influence
PSS adoption. The framework divides perceived innovation characteristics in problems
related to its ease of use and usefulness. Furthermore, Vonk et al. [2005] includes a series
43
Figure 3.1: System-related factors (green box) and non-system-related factors (red boxes) illustrated in Vonk’s et al. [2005] theoretical framework of the PSS adoption process
of attributes to specify ease of use (i.e. complexity, trialability, observability, hardware
and data issues) and usefulness (i.e. relative advantage, compatibility, uncertainty). These
specifications show that, as many scholars in the field of PSS research (e.g. Pelzer [in
press], Arciniegas et al. [2013], Broemmelstroet [2012]), Vonk et al. [2005] does not refer
to ISO standards acknowledged by the Human-Computer Interaction (HCI) discipline (see
Section 2.8) for defining concepts such as usability and usefulness.
This research associates system-related factors hampering PSS adoption with low
usability of PSS functionality. In particular, it claims low usability to stem from a mismatch
between what functionality planners desire and what is actually provided by the PSS. This
explains why as a first step, usability problems or mismatches are investigated (RQ 1) and
after that, functionality, as expected by planners, is examined (RQ 2), in order to
overcome these mismatches. Examples of mismatches are: the PSS requires a GIS for
operation and the planner prefers web-based software systems or the PSS allows
changing the colours of the legend but not the classes as required by the planner. This
research claims that planners’ expectations of PSS functionality are determined by non-
44
system-related factors. For instance, planners are bound by regulations of governmental
planning departments such as submitting plans with a certain layout and in a specific
format. To comply with such regulations they need and expect specific PSS functionality.
These expectations might not be their personal choice but are determined by external
pressures. For this reason, this research supports Vonk’s et al. [2005] framework that
proposes perceived innovation characteristics to be affected by personal adopter
characteristics, social influences and persuasion influences (illustrated by arrows from red
boxes to green box in Figure 3.1). This research claims that innovation characteristics are
not perceived as stated in this framework but can be objectively evaluated. For instance,
whether a system has data issues or meets user requirements, is a fact and has little to do
with perception.
Social influences refer to the impact people have on planners and their decisions
whether or not to adopt PSS. These can be colleagues in another branch or people of
other planning organisations. In any case, it is expected that planners are aware of what
software other people use and might think that they need to use the same or similar
software. This social influences factor is considered in the interview study of this research.
Specifically, to the question ‘how do you choose what software to acquire?’, it is expected
that some interviewees’ answers will refer to this factor.
Persuasion influences involve activities undertaken by providers to enhance the
supply of PSS. This factor has been given considerable emphasis in this research. Many of
the interviewees are people who develop and supply PSS. Particularly, academic planners
are asked ‘have you developed any software for planning?’. If they answer yes, further
questioning gathers information on understanding efforts they make for improving the
product such as ‘have you involved users when developing it?’, ‘has any evaluation been
conducted?’, ‘has it been applied in practice?’. In particular, by interviewing academic
planners this research expected to gather information on activities undertaken by them
to transfer PSS from research to practice. Suppliers of PSS are not only academic planners
but include other planning actors. In fact, technical specialists working with planners are
asked whether they develop PSS for planners. If the answer is yes, questions on the
adopted development process and on provided support are asked.
Adopter characteristics can refer to both planners (e.g. experience level with
technology) or planning organisations (e.g. size in terms of number of employees).
Information on these characteristics and how they influence PSS adoption are specifically
45
gathered in the interview study. By interviewing planners and other planning actors of the
same organisation, a good overview of the culture of the organisation and attitude of the
management and employees is expected to be gained. Questions asked in both the
evaluation and interview study, in order to gather information on adopter characteristics,
are: ‘what is your job function?’, ‘how long have you been working in planning practice?’,
‘what have you studied?’, ‘what is your background?’ and ‘what are planners’ technical
skills level within the organisation?’.
External conditions cannot be more closely discussed in this chapter because it is
not clear what it refers to. In Vonk et al. [2005: 917] it is merely described as a “bottleneck
indicator related to the role of the provider”. This description furthermore does not make
clear how external conditions differ from provider marketing efforts.
Awareness of the existence of PSS is a basic factor for PSS adoption. If a planner
does not know about the existence of a PSS, he/she will never adopt it. As an example of
how planners’ context, and specifically academic planners, can contribute to improving
planners’ awareness and PSS adoption (RQ 3), this research formulates an Online PSS
Resource (Chapter 4) that presents over one-hundred PSS developed and applied
internationally. By providing technical and practical details, this PSS repository assists
planners in finding out information on available PSS and increasing their awareness of PSS
potential. Vonk’s et al. [2005] results showed that low awareness is a major bottleneck
for PSS adoption. Whether it is still a bottleneck is investigated in the interview study.
Indeed, this research examines if planners are familiar with the term PSS. Furthermore,
planning actors are asked ‘how do you choose what software to acquire?’. This question
may allow identification of other influencing factors as those reported in Vonk’s et al.
[2005] framework. Whilst this research takes the framework as an underlying basis, it
does not exclude that other factors not included in the framework, such as regulations,
determine PSS adoption by planners and planning organisations.
(2) a categorisation of factors affecting UX as defined by a group of experts in HCI research
[Roto et al. 2011]. UX is a quite recent research field which still contains some open
questions on its conceptualisation [Hassenzahl & Tractinsky 2006]. Roto et al. [2011: 1]
have attempted “to bringing clarity to the concept of user experience”, as the subtitle of
their white paper reports. According to the authors, factors influencing UX can be
classified into three categories:
46
• Context refers to everything that is around the user and system. Examples are: people
working with the user, working environment of the user (e.g. using the system on a desk
or on a bus), network connection, regulations, planning education.
• User concerns the state of the user when interacting with the system. Examples are:
his/her motivation, mood, mental and physical resources, expectations.
• System refers to the properties of the system. Examples are: functionality, interactive
behaviour, responsiveness.
Figure 3.2: The three main factors affecting UX [Roto et al. 2011] and
the RQs addressing them in this research
This research has formulated the three RQs (see Section 3.1) around these factors
(see Figure 3.2), in order to explore UX of planners with PSS. RQ 1 focuses on the factor
system as its aim is to identify any usability problems of PSS functionality. The expectation
is that most problems could be quite easily avoided with a thorough working process. This
includes planning enough resources for PSS development and involving users in PSS
development following the Human-Centred Design (HCD) process (see Figure 2.4). RQ 2
concentrates on the factor user, in order to explore what planners’ expectations of PSS
functionality are. It is expected that considerable functionality provided by PSS is not
wanted by planners who might have other expectations. RQ 3 addresses the factor
context. By doing this, this research aims to understand the influence planners’ context
has on their UX with PSS and their PSS adoption. In fact, this research builds on the notion
that PSS usability and UX of planners with PSS are influenced by non-system-related
factors such as planning policy, planners’ working environment, planning education,
47
which are part of planners’ context. By following Roto’s et al. [2011] categorisation, this
research attempts to better understand UX of planners with PSS and the reasons for their
UX. The next section describes the methodological approach adopted in this research to
answer the RQs.
3.3 Methodological approach
This thesis is mainly based on a qualitative approach. Qualitative research is very much
performed in psychology, social sciences, human-computer interaction, and more recently in
software engineering, since it helps researchers exploring more in-depth the practice of
software engineering [Dittrich et al. 2007; Seaman 1999]. Traditionally, quantitative methods
have been considered better than qualitative ones, because they provide objective
measurements and permit the replication of the study [Coleman & O’Connor 2007]. However,
the validity of qualitative studies is questioned by some researchers, because they are based on
subjective interpretation [Dittrich et al. 2007]. This is not true if qualitative data is analysed with
methods that ensure study objectivity and soundness [Preece et al. 2015], as done in this thesis.
An advantage of qualitative research is the flexibility of the research design and the opportunity
to involve practitioners more closely in the process, enabling researchers to discuss with them
pros and cons of their practices. Once an insight into an unexplored field has been gained
through qualitative research, it is possible to frame the design of a quantitative study to provide
better indications of the magnitude of the researched phenomenon [Karlsson et al. 2007]. There
is confidence that the research presented in this thesis will stimulate further work towards
improving PSS usability and adoption by professional planners.
Specifically, this thesis adopted low-constraint research [Graziano & Raulin 2012]. This
method consists of studying behaviour as occurring in natural settings. To achieve this, typical
methods used are observation, questionnaires and interviews. The researcher intervenes as
little as possible, in order to impose minimal constraint on participants. Low-constraint research
has been adopted in this research because, similar to UX research, its focus is on individuals,
their behaviour and context. These aspects are also central in UX research. Two of them, i.e.
individuals and context, influence UX as described in Roto et al. [2011] and illustrated in Figure
3.2. The third aspect, behaviour, can provide an explanation of UX and consequences of UX.
Low-constraint research has also been used because it allows exploratory research of
unexplored research fields such as PSS usability and UX with PSS. Its aim is to recognise events
occurring in a specific context and identify contingent relationships among context variables. An
48
example of a context in this research is the PSS adoption process, whereas possible variables
influencing it are planners’ skills and UX of planners with PSS. Low-constraint research, however,
allows little or no generalisation of findings. As such, higher-constraint research, i.e. controlled
experiment, has to be performed in future works, in order to study in-depth contingencies
among context variables.
This thesis mainly builds upon two studies. These are an evaluation study, also referred
to as user test, and an interview study. User tests and interviews are so-called user-based
methods, i.e. involve potential or real users of the product, and are therefore effective for
obtaining significant results. Due to these methods, this research involved mainly planners
because they are the people for whom PSS are designed. However, planning actors as defined
in Section 1.4 were also included.
User tests consist of observing potential or real users while they are using a software
system. They are often combined with the thinking-aloud technique [Nielsen 1993]. Gathering
participants’ thoughts while they are using the system offers an insight into users’ mental
models. Mental models can relate to users’ reasoning strategies, procedures, the manner in
which they carry out tasks in daily practice, languages and notations they are familiar with,
functionality they need and formalism in the field of the users.
Interviews enable discussions with potential or real users to gather feedback and
information about their experiences with software systems. It is a valuable method for assessing
user satisfaction and hedonic qualities of UX. Interviews are also appropriate for analysing users’
requirements, tasks and skills if they are conducted with people who are experts in the system
application domain [Ackerman 2000; Sutcliffe 2000]. The adoption of both methods in this
research as well as how they provided answers to the RQs is reported as follows.
3.3.1 User test
This research aims to understand usability problems and UX of planners when interacting with
PSS (RQ 1). To gather this information, planners were observed during their interaction with PSS.
As PSS adoption is low in planning practice [Brits et al. 2014; Brömmelstroet 2010a; Vonk et al.
2005], no observation at planners’ workplace was possible. Instead, a user test in which planners
were asked to use PSS was organised. Preparation of user tests is not simple as many issues have
to be considered [Preece et al. 2015]. As first, the PSS to be tested had to be identified.
A comprehensive search in academic literature of PSS developed and applied
internationally was performed. As many, over one-hundred, PSS were identified, it was decided
49
to create a PSS repository. This repository, called the Online PSS Resource
(http://docs.aurin.org.au/projects/planning-support-systems/, accessed on June 5, 2015 and
see Chapter 4), could ultimately assist in increasing planners’ awareness level of existing PSS
which is, according to the literature, low and a barrier to PSS adoption [Vonk et al. 2005].
Furthermore, an online survey with the developers of the identified PSS was undertaken to
gather practical and technical information on the PSS they developed. This information, together
with the PSS, was provided in the Online PSS Resource in order to facilitate the selection of
appropriate PSS for planners intending to adopt PSS. In this sense, the Online PSS Resource is a
practical contribution of this research and an example of how planners’ context, and specifically
academic planners, can contribute to improving PSS adoption (RQ 3).
Table 3.1: Characteristics of nine PSS for assessing their suitability to be evaluated in the user test
The repository was used to search for PSS to evaluate in the user test. Nine PSS (see
Table 3.1) that were easy to access and to install and allowed performing any spatial planning
task as defined in the literature [Balzarini et al. 2013; Pullar & McDonald 1999], such as impact
assessment, Land Suitability Analysis (LSA), land use demand and allocation analysis, were
thoroughly studied. Their suitability to be used within a user test setting had to be checked. Six
PSS were excluded due to different reasons. The housing development tool and Online Envision
provided sample data for Australia; however, their Graphical User Interfaces (GUI) did not work
Complex and steep learning curve
GUI not working well and lack of assistance
PSS of 3rd notion
Sample data available for Australia
Tutorial (user guide, example)
CommunityViz x
Envision x
Housing development tool x x
MCASS-S x x x
Online Envision x x
Online What if? x
TRANUS x x
UPlan x x
UrbanSim x x
50
properly. MCASS-S was excluded because it was limited to visualisation capabilities and
therefore it did not represent a PSS as defined in this research (see Section 2.5). The author also
tried to use TRANUS, UPlan and UrbanSim with the provided tutorials. Understanding how to
use these PSS required extensive training and learning (weeks to months), which was beyond
the time schedule of the research. Finally, the LSA module of CommunityViz, Envision and Online
What if? were considered most suitable for a user test setting because:
• their use did not require step-by-step instruction,
• the whole workflow process from data input to data output could be tested within a
reasonable amount of time.
These two conditions are based on the literature [Seewald & Hassenzahl 2004], which prescribes
that tasks during user tests do not have to be too easy but must be possible to solve.
LSA is a common activity undertaken by land use planners when performing site
selection or strategic planning tasks as illustrated by the considerable amount of literature on it
(e.g. Pettit et al. [2013], Klosterman [1999], Pullar & McDonald [1999]). It determines the
suitability of land units for a specific purpose, based on a set of parameters that planners choose.
The three PSS allowed users to give different importance to parameters by assigning weights.
This technique, also known as Spatial Multi-Criteria Evaluation (SMCE), is well-supported in
decision-making processes [Arciniegas et al. 2013]. The output of the three PSS was a map
showing the suitability score for each land parcel through colour-coding. LSA has been used
extensively in a considerable number of studies both in Australia and in other locations, such as
Perth [Pettit et al. 2015b], Mitchell Shire [Pettit et al. 2008] and Hervey Bay [Pettit 2005] in
Australia; Ohio [Klosterman et al. 2003] and Washington [Jankowski & Richard 1994] in the US;
and in the Netherlands [Janssen & Rietveld 1990].
Applying the LSA modules allowed testing of important characteristics of PSS such as
modelling capabilities, spatial visualisation and geoprocessing functionality. Despite the three
PSS adopting the same technique, they differed in functionality they offered and how they
provided it through the user interface. For examining commonalities and differences, Miles and
Huberman [1994] suggest testing two or more cases and hence three PSS were selected. The
three PSS were evaluated to gather information on strengths and weaknesses of functionality.
This information should help system developers to identify desired and unwanted functionality,
in order to design more usable PSS.
Once the PSS to evaluate were identified, other issues such as participant recruitment
51
and facilities had to be organised. In order to achieve an effective organisation, a framework
that assists in planning and performing evaluation of interactive systems, called DECIDE [Preece
et al. 2015], was adopted. The framework indicates six main activities that are related to
important aspects to consider during the evaluation process. These are:
(1) determine the goals,
(2) explore the questions,
(3) choose the evaluation paradigm and techniques,
(4) identify the practical issues,
(5) decide how to deal with the ethical issues,
(6) evaluate, interpret and present the data.
This framework was designed for interactive systems in general. Within the scope of this thesis,
it was specialised for PSS evaluation (see Chapter 5 and Russo et al. [2015]). The resulting PSS
Evaluation Framework, as it is called, has been developed for supporting evaluators of PSS and
for designing the user test in this thesis.
The user test involved six planners, working in private and governmental planning
organisations in the Melbourne metropolitan area in Australia. They individually came to a
usability lab at The University of Melbourne, where each participant used all three PSS in
sequence but in a different order to avoid learning effects [Graziano & Raulin 2012]. As
suggested by the PSS Evaluation Framework, cost-effective methods such as thinking-aloud
technique and questionnaires were adopted [Lanzilotti 2006]. The questions were of qualitative
nature, in order to give participants the opportunity to better describe their experience with the
PSS. The thinking-aloud technique provides useful results even with a small number of users
[Nielsen & Landauer 1993; Virzi 1992]. Additionally, it would have been difficult to organise a
quantitative study with a significant number of users because planners operate under constant
resource constraints [Houghton et al. 2014; Krause 2013]. Sections 6.2 - 6.5 provide details on
the three selected PSS and the user test, performed according to the six activities of the PSS
Evaluation Framework. How the data gathered during the user test contributed to answering
the research questions is described in Section 3.4.
3.3.2 Interviews
The other main study of this research directly involving planning actors was an interview study.
Overall, thirty-five planning actors were interviewed in three countries; Australia, Italy and
52
Switzerland. The interviews were conducted in different countries in order to investigate the
effect of possible differences in terms of planning concepts, terms, policy, etc. on software
adoption in planning practice. Interviewing planning actors from three countries and of different
cultural backgrounds increased the international relevance and representativeness of this
research. To the author’s knowledge, besides Vonk et al. [2005], most studies on PSS adoption
have been conducted within the context of a particular country [Goodspeed 2013; Williamson
2012; Brömmelstroet 2010b; Vonk & Geertman 2008; Vonk et al. 2007; Uran & Janssen 2003].
Thirty of the interviewees were planners and people working with them, such as
specialists in GIS, modelling and data management. Questions were formulated in order to
gather information on planners’ software adoption and factors influencing it.
Interviews were also conducted with five academic planners, in order to get an overview
of technical and software courses provided in planning education at university. Through
interviewing academic and professional planners, the transfer of PSS from academia to practice
was examined.
The interviews were mainly conducted face-to-face and one-on-one. All interviews were
semi-structured. This design prescribes a set of questions that is the same for all planning actors;
however, it also allows other questions that could spontaneously emerge during the interviews.
Predefined questions allow gathering quantitative information and confirmation of what is
already known. On the other hand, spontaneous questions stimulate discussion, in order to get
explanation for unclear or unexpected answers [Preece et al. 2015; Karlsson et al. 2007].
It is worth remarking that interviews have some limitations when trying to gather
information about a certain practice [Preece et al. 2015]: what interviewees say is not always
what they do, know or think. One of the reasons for this is that “practitioners exhibit a kind of
knowing in practice, most of which is tacit” [Schoen 1983: 8]; in other words, some knowledge
that they use in their activities is tacit and they are not able to explain it in words [Costabile et
al. 2007]. Moreover, interviews are affected by a ‘please the researcher’ bias, i.e. interviewees
are not always objective and answer in a way designed to please the researcher.
To avoid these limitations, the literature suggests also getting the opinions and
experiences of work colleagues and people working in the same field of the interviewees. A
further solution is to triangulate the interviews with other methods, such as observation (see
Section 2.11.2). This research followed these suggestions by interviewing not only professional
planners but also other planning actors including academic planners. Furthermore, in order to
verify the results of the interviews, a focus group with six professional and academic planners
53
was held. Details on the design and execution of the interview study and the focus group are
provided in Sections 7.2 and 7.4.
3.4 Answering the research questions
Table 3.2 summarises the methods and activities used in this thesis for answering the research
questions. According to the PSS Evaluation Framework, the first activity when designing an
evaluation study is to define the goals or, in other words, what should be achieved once the
evaluation has been carried out. The goal of the user test was to answer RQ 1 and RQ 2, i.e.
identifying usability problems and UX of planners when interacting with PSS as well as their
expectations of functionality (see Chapter 6).
In order to achieve this goal, participants’ comments as part of the thinking-aloud
technique and answers in the questionnaire were examined. Specifically, their comments and
answers were analysed against indications of positive and negative experiences and
functionality. A scheme inspired by Vermeeren et al. [2002] (see Appendix B) was used that
allows identifying such indications. Usability problems and expectations of functionality were
explicitly and implicitly expressed through planners’ comments. For instance, functionality
defined by the participants as negative, not only referred to a possible usability problem but also
provided information on what functionality is not desired in PSS. Similarly, planners’ suggestions
for design improvements stressed functionality they expect and at the same time indicated
usability problems and dissatisfaction with the current system’s functionality.
The interviews revealed factors related to system functionality that determine whether
a software tool is adopted and used by planners. This information also indicated expectations of
functionality that planners have. Further information gathered through the interviews was on
processes such as PSS development and transfer from academia to planning practice as well as
software courses provided in university curricula. How these processes and planning education
could be improved was analysed, in order to enhance PSS usability and adoption. Activities were
studied that if undertaken by planning actors, in particular PSS developers and academic
planners, could lead to these improvements. This research defined recommendations for
planning actors with the objective that these activities are performed.
54
Table 3.2: The methods and activities used for answering the research questions
3.5 Conclusion
This chapter presented the research design of this thesis, including the research questions,
theoretical framework and methods. In particular, factors influencing the PSS adoption process
were rigorously discussed and conceptualised in the context of the study and in line with the
research questions. Then the motivations for the chosen research design was reported followed
by how the studies in this research and the adopted methods addressed the research questions.
In the following chapters, the three individual studies are presented including their results.
RQ Method Activity
1 What are identified usability problems and UX when interacting with PSS? User test
(thinking-aloud, questionnaire)
-Analysing planners’ positive and negative experiences
-Investigating positive and negative functionality of the three tested PSS
-Identifying planners’ design improvement suggestions
2 What are planners’ expectations of PSS functionality?
Semi-structured interviews
-Investigating factors of PSS adoption
3
How can planners’ context contribute to improving PSS usability and adoption in planning practice?
User test (thinking-aloud, questionnaire),
Semi-structured interviews
Gathering information on:
-software courses in university curricula,
-development processes adopted by PSS designers,
-transfer of PSS from academia to planning practice
55
Chapter 4: The Online Planning Support Systems (PSS) Resource
Existing Planning Support Systems (PSS) repositories and reviews have been presented in
Chapter 2. This chapter introduces a further repository, called the Online PSS Resource,
developed within the scope of this thesis. It includes PSS developed and applied worldwide as
well as technical and practical details of the PSS. The formulation of this repository is a practical
contribution to assist planners in finding out information on available PSS and complements
existing PSS repositories. It is also an example of how planners’ context, and specifically
academic planners, can contribute to improving PSS adoption (research question 3). In essence,
this chapter is organised as follows. First, the benefits of PSS repositories in general and the
motivations for creating the Online PSS Resource are introduced. The chapter continues with
the process and methods, mainly literature search and questionnaire, adopted to identify PSS
and their technical and practical details. The results of the data gathering process are then
discussed with reference to how it shaped the Online PSS Resource as well as to how it provided
insights into reasons for low adoption of PSS. The chapter concludes with challenges
encountered during the development process and how future work could address limitations of
the Online PSS Resource.
4.1 Introduction
Planners deal with a complex mixture of tasks as they have to consider a wide range of factors
that influence and shape the urban system. Economic, social and environmental considerations
are fundamental inputs into planners’ choices and decisions and therefore policy- and plan-
making [Bonner 2002]. Alongside the rapid growth of our cities, the responsibility and the
demand to adopt a holistic approach to their activities put substantial pressure on planners. This
explains why planners consistently request software that supports their analytical, problem-
solving and decision-making capabilities [Hughes & Heckbert 2012; Geertman & Stillwell 2003a].
To date, a plethora of PSS has been developed around the world by researchers and
software developers. They differ in many ways, occurring as prototypes or as commercial
products, using different analysis techniques and having been designed to address different
tasks. Scholars remark that all PSS have their strengths and weaknesses in terms of functionality
and adoption. Significantly there is no PSS that addresses all land use planning issues [Hughes &
Heckbert 2012; Klosterman & Pettit 2005].
Given the absence of an all-purpose PSS capable of dealing with the complex nature of
planning activities, it might be necessary to access a range of PSS. The existence of so many PSS
56
provides a challenge for planners to know what is in fact available and accessible. The literature
showed that planners’ awareness of PSS existence and potential is low and that this strongly
hinders PSS adoption in planning practice [Vonk et al. 2005]. Some researchers [Pettit et al. 2011;
Batty 2007] stressed the need to provide planners with an overview of available PSS and
information on their functionality, strengths and limitations “through a well-publicised and well-
formatted website” [Pettit et al. 2011: 239]. Such repositories may not only increase planners’
awareness of existing PSS but they may also support the planning discipline, including
developers and researchers, in multiple ways through:
• informing of the state-of-the-art and innovations of PSS,
• helping to identify gaps in PSS development,
• facilitating technical comparison and assessment of PSS,
• supporting selection of appropriate PSS,
• increasing access and correct application of PSS.
Identifying existing PSS and creating a comprehensive PSS repository is challenging
because there is a large number of publications including books, journal articles, conference
papers and government reports on PSS. Some repositories and reviews of PSS have been
presented in Chapter 2. However, they do not (and some of them were actually not designed to)
provide a comprehensive picture of available PSS. For instance, Geertman and Stillwell [2003a]
and The Redlands Institute [2012] included not only PSS but also planning methods and data as
well as software for professionals other than planners in their repositories. This is a useful
resource but not comprehensive in relation to inclusion of available PSS. The reviews by the
United States Environmental Protection Agency (U.S. EPA) [2000] and Hughes and Heckbert
[2012] have been developed with the objective to provide planning organisations, again with a
selective summary of PSS. Therefore, they concentrated on framing the functionality of the
presented PSS rather than the breadth of existing PSS.
Given the absence of a comprehensive repository to support people from the planning
discipline in accessing PSS and, in order to show the broad spectrum of PSS, the Online PSS
Resource (http://docs.aurin.org.au/projects/planning-support-systems/, accessed on June 5,
2015) has been developed and is presented in this chapter. This repository was created by the
author on the 5th of June 2015 and comprises 108 PSS (see Appendix C) developed and applied
all over the world, and information for most of the PSS (see Appendix D for the content of the
Online PSS Resource). Specifically, information entailed:
57
(1) the URL to the website of the PSS where further system information can be found,
(2) sites where the PSS has been applied,
(3) technical and practical aspects of the PSS relevant to making a decision on whether or
not to adopt the PSS.
The latter is listed in the left column of the resource under the heading ‘PSS with additional
information’, while all other PSS can be found in the right column named ‘Further PSS’ (see
Figure 4.1). The PSS in the two columns are ordered alphabetically.
The Online PSS Resource has been created to support planners’ identification and
selection of appropriate PSS that assist their planning endeavours. For accessing a wide range of
planners and contributing to improved planning practice, it has been integrated as part of the
Australian Urban Research Infrastructure Network (AURIN) initiative, which supports planning
and decision-making across Australian cities [Sinnott et al. 2014]. The methods used to create
the Online PSS Resource and the obtained results are reported in the following sections.
4.2 Methods
The goal of this study was to identify developed and applied PSS as well as to provide information
on their technical and practical aspects relevant for adoption. To achieve this goal multiple
methods were adopted. An extensive literature search was carried out using the search engine
Google Scholar [Google 2013b]. Its main advantage is that it holds the largest database of
scientific papers [Ardito et al. 2015]. The papers retrieved included journal articles, books, the
grey literature, i.e. PhD theses, industrial and technical reports, working papers and white
papers. They were screened by reading the title and abstract. As a result of this first screening,
relevant papers were selected, i.e. those reporting on software tools for addressing land use
planning issues. A second screening, this time of the full paper, was carried out to determine
whether the software tool met a set of criteria and therefore could be considered a PSS. The
criteria were:
(1) the software tool incorporated a modelling technique, i.e. “include[d] the use of
techniques such as agent-based modelling (e.g. UrbanSim), rule-based modelling (e.g.
What if?) and cellular automata (e.g. SLEUTH)” [Brits et al. 2014: 2],
(2) it focused on land use planning issues. If it additionally addressed other aspects such as
transportation, environment, vegetation, etc., it was also included. In other words,
hybrid software tools such as integrated land use-transportation models were included.
58
Figure 4.1: Screenshot of the Online PSS Resource (Source: http://docs.aurin.org.au/projects/planning-support-systems/, accessed on June 5, 2015)
59
If these criteria were met, the screening also included identifying, where available, the
supplier of the PSS, the email address of the supplier, the URL of the system website and sites
where the PSS was applied. System websites often contain information such as requirements,
strengths and costs needed by planners for making a decision on whether or not to adopt a PSS.
Information on application sites of PSS is important for planners as previous literature [Vonk et
al. 2005] showed that their decision to adopt a PSS can also depend on whether the PSS is used
by their networks in surrounding areas. If this information could not be retrieved in the paper,
the name of the PSS was used as string to carry out a query in Google [Google 2013a] and so to
search for the information on the web.
The references of the paper were analysed. If a referred paper appeared relevant, the
same screening process was applied to it. If there were multiple papers on the same PSS, only
the most recent one was reviewed. Papers not written in English were excluded. The identified
PSS and information were added to the Online PSS Resource.
In order to identify any unintentional omissions of PSS in the online resource a survey
with eight experts from the planning field based in Australia was conducted (hereafter referred
to as expert survey). An email was sent to them in which they were asked whether any PSS
should be included. Responding to the experts’ feedback, the Online PSS Resource was updated.
As a next step, in order to collect additional information on the PSS a survey with the
developers was conducted (hereafter referred to as developer survey). Specifically, they were
contacted via email and asked to complete an online questionnaire through a link provided in
the email. The questionnaire gathered information concerning technical and practical aspects of
the PSS that are likely to be considered by planners who intend to adopt a PSS. More precisely,
thirty-one questions were comprised in the questionnaire related to user assistance, skill and
software requirements, analysis functionality and access. The last question (Question 31) asked
whether any further PSS should be included in the online resource.
Most questions were based on previous planning literature, PSS repositories and
reviews [Hughes & Heckbert 2012; The Redlands Institute 2012; Geertman & Stillwell 2004;
Geertman & Stillwell 2003a; Agarwal et al. 2000; U.S. EPA 2000; Pullar & McDonald 1999]. The
remaining questions were suggestions of four people, comprising a GIS-specialist with
experience in planning and PSS development, a researcher with experience in user-based
methods and two academic planners, who reviewed the questionnaire. After each question in
Appendix E, the reference and whether the question was a suggestion of a reviewer is indicated
in square brackets.
60
Overall, ninety-one emails to PSS developers were sent on the 5th of May 2015: an
additional seventeen developers could not be contacted due to invalid or unknown email
addresses. Three weeks later, another email reminder to complete the questionnaire was sent
to the developers who had not participated up to that point in time.
4.3 Results and discussion
This section first summarises the results of the adopted methods and how these framed the
Online PSS Resource. Subsequently, the results are briefly discussed in relation to low adoption
of PSS.
A total of one-hundred-and-eight PSS have been identified, being ninety-one through
the literature search, fourteen through the expert survey, and three through the developer
survey (see Table 4.1 and Appendix F for the method used to identify each PSS, including
literature reference). For most PSS some information is provided; the URL to the system website
has been identified for fifty PSS while application sites have been retrieved for seventy-three
PSS. Overall, nineteen of the ninety-one contacted developers completed the questionnaire
providing additional information related to technical and practical aspects of the PSS (see Table
4.2). This corresponds to a response rate of 21% which is a common return for online surveys
with large and remote populations [Preece et al. 2015]. The percentages of respondents for each
question and answer have been determined (see Appendix E). Some of them have been used
for a preliminary analysis and discussion related to PSS adoption as follows.
Table 4.1: Number of PSS identified and the methods adopted
Method Number of PSS Literature search 91 Expert survey 14 Developer survey 3 Total 108
Table 4.2: Number of PSS and available information
Information Number of PSS URL 50 Application site 73 Technical and practical aspects 19
61
The responses to Question 7, 8, 12, 25 and 27 raises some questions about the actual
availability of PSS for planners. It appears that from the nineteen respondents, not all have
developed PSS to be supplied and used by professional planners. In fact, the results show that
26% of the PSS are intended for academic purposes only, 11% of the PSS are for adoption in a
specific area (i.e. are not readily adaptable for other geographical areas), for 37% of the PSS the
developers have to be contacted for questions on system access and for 37% of the PSS the
developers have to be contacted for questions on system cost, or the PSS are not priced. The
average of these respondents’ percentages is 28%. In other words, it appears that almost a third
of the nineteen PSS have not been designed for professional use.
There are also PSS that, based on the responses, have been intended to be used by
professionals. Fifty-eight percent of the PSS were supported, for 58% of the PSS assistance was
provided, 74% of the PSS could be applied to any area, 47% of the PSS were available online or
after purchasing and 53% were either available at no cost or at a price. The average of these
respondents’ percentages is 58%. The interpretation here is that almost two-thirds of the
nineteen PSS have been developed with the intention of making them available.
Specific skills are required to use PSS. The results relating to Question 10 indicate that
for using most of the nineteen PSS (about 70%) GIS and system modelling skills are required
while programming skills, mainly at a basic level, are only necessary for using a minority of the
PSS (about 30%).
Previous research stressed that for improving adoption PSS have to be validated1,
evaluated and fitted to planners’ tasks [Brits et al. 2014; Allen 2008; Vonk et al. 2006]. Although
no specification is available on the methodology used, 68% and 47% of the respondents
indicated in Question 26 they had validated and evaluated their PSS, respectively. This is,
however, a real weakness considering that model validation and usability evaluation should be
a key activity during PSS development [Russo et al. 2017]. The fit of PSS to planners’ tasks can
be improved if the PSS is customisable, for example, through scripting or open API access. The
results to Question 22 revealed that about half of the nineteen PSS are customisable whilst the
other half is not.
1 Refers here and as follows to validation of the model used by the PSS. It is the process that aims to demonstrate whether the model and its output reproduce the real and represented system reasonably and satisfactorily well [Hillston 2003].
62
4.4 Conclusion
The development of the Online PSS Resource was not without challenges. On the contrary,
diligent work was required for gathering information, defining the questionnaire and reaching
potential respondents. Significant efforts were made to develop a comprehensive PSS
repository. All information in the Online PSS Resource was considered correct at the time of
development. However, due to the large and constantly changing population of PSS, errors and
omissions are possible yet unintentional.
Limitations of the Online PSS Resource are the static webpage which makes it difficult
for developers to add their own PSS. It is current as of June 2015 and requires someone to
maintain its currency by adding new PSS as they become available.
Future work could involve keeping the Online PSS Resource up-to-date and gathering
technical and practical information for at least currently supported PSS. Further potential for
improvement includes enhancing the website with visualisation and interaction tools that allow
filtering, categorising and comparing the PSS for facilitating their assessment and selection.
The creation of the Online PSS Resource provided an international overview of PSS. The
survey conducted with the developers of the PSS allowed performance of a preliminary analysis
and gaining some insights into possible reasons for low adoption of PSS. These are: i) many PSS
are actually not designed to be used by professional planners, ii) the adoption of PSS requires
specific skills and knowledge, and iii) PSS do not always undergo proper development including
model validation and user-based evaluations that contribute to improving the fit of PSS to
planners’ requirements.
The results obtained in this chapter are not meant to be exhaustive: reasons for low
adoption of PSS are investigated in more depth in the following chapters through testing three
PSS (Chapter 6) and through interviewing people from the planning discipline (Chapter 7).
Additionally, indications for proper development and evaluation of PSS are provided.
Specifically, a framework that supports people in carrying out PSS evaluations is proposed. The
framework and a detailed description of the activities to take into account when evaluating PSS
are presented in the next chapter. In concluding this discussion of the Online PSS Resource it is
noted that the purpose in creating this repository is two-fold, firstly to understand the current
state of play of PSS and secondly, to provide a practical resource which can be used by planners
in locating and identifying available PSS to adopt in practice.
63
Chapter 5: The Planning Support Systems (PSS) Evaluation Framework
This chapter presents a guide for performing usability evaluations of Planning Support Systems
(PSS), the so-called PSS Evaluation Framework. The framework informs those intending to
conduct PSS evaluations of activities to take into account, from the planning stage up to
performing an evaluation. These can be PSS developers themselves who want to check whether
they implemented users’ requirements well. However, evaluators can also be other planning
actors, in particular technical specialists or academic planners, who want to test a PSS before
adopting it within the planning organisation or in a planning course. Through providing support
to evaluators, it is a tool for increasing systematic and standardised evaluations of PSS, which in
turn will facilitate comparison of evaluation studies whose outcomes should be used for
improving PSS development and usability. This chapter begins with the motivation and basis for
this framework. Next, the framework and its proposed activities are described in detail. The
chapter ends with possible future work for addressing limitations of the framework and the role
of the framework as part of this thesis.
5.1 Introduction and motivation
In Human-Computer Interaction (HCI), evaluation consists of testing software systems and their
functionality against quality factors and attributes. This provides evaluators with feedback on
implemented functionality and design choices. Obtaining feedback is an important reason for
conducting evaluations as it checks whether developers are implementing user requirements
well, determining the need for design revisions, testing legibility of font, etc.
In the software design process, evaluation is a central activity in that it contributes to
the development of products that are more satisfying for users [Somervell & McCrickard 2004].
Outcomes and feedback of evaluations which include information on the impact of software on
users, possible interaction and usability problems as well as user needs and expectations, are
used by designers to undertake changes to software and improve it.
Evaluations of PSS are quite rare. Indeed, usability of PSS could be improved if more
evaluations were performed during software development [Allen 2008]. However, evaluations
have to be carefully planned, in order to obtain reliable and valid results. Decisions have to be
made in relation to when to perform evaluations and how to achieve evaluation goals, i.e.
procedures and methods to adopt. Such decisions require some expertise and may differ among
products. For instance, evaluations can indeed be performed at different development stages of
a system. However, it is important to highlight that the often neglected interaction and usability
64
problems are much cheaper to solve at an early phase of the design and development [Ardito et
al. 2014; Larusdottir 2012; Bias & Mayhew 2005]. This indicates that novice evaluators need
more guidance on how to plan and perform the overall evaluation.
The DECIDE framework proposed by Preece et al. [2015] addresses this need by
providing a structure for performing evaluation studies, focusing on issues that have to be taken
into account. Its name derives from the initial of the name of each of the six main activities
proposed by the framework: Determine, Explore, Choose, Identify, Decide, Evaluate. More
precisely, the activities range from planning evaluation up to analysing gathered data and
include tasks such as defining evaluation goals, getting ethics approval, recruiting participants.
The DECIDE framework has been designed for evaluation of software systems in general. In
order to encourage and support planning actors, in particular researchers and PSS developers,
to perform usability and UX evaluation of PSS, a more specific framework has been developed
within the scope of this research. The structure of this so-called PSS Evaluation Framework is
presented in the following section.
5.2 The PSS Evaluation Framework
The PSS Evaluation Framework is a guide that provides support for evaluating PSS usability. It
has been inspired by the DECIDE framework. Specifically, its six activities have been customised
for PSS evaluation as described in the following subsections and summarised in Appendix G and
Figure 5.1, where the dashed arrows indicate the possibility of going back to a previous activity
for some refinement.
5.2.1 Determine the evaluation goals
The first activity when planning a usability evaluation is to determine what the goals of the
evaluation are. This helps to define the scope of the evaluation and what should be achieved
once the evaluation has been carried out. For determining goals, it might be helpful to question
who and what the evaluation is for. For example, the following questions could help to
determine the goals: are we going to evaluate the ease of learning of the PSS because we are
using it for a planning course at university? Are we interested in the ease of use of the PSS
because it will be adopted by a planning organisation in practice? Are we comparing the usability
of similar functionality of two or more PSS? Are we evaluating the PSS to define the design of its
next version? Are we testing the PSS as part of a needs assessment activity with clients
[Lethbridge 2000]?
65
Figure 5.1: The six activities of the PSS Evaluation Framework
5.2.2 Explore the questions
This activity includes the formulation of specific questions that underpin the goals and should
be answered through the evaluation. Evaluations are performed because there are questions
about system usability which need to be answered. The creation of a hierarchy of questions and
sub-questions enables focusing on issues that need to be addressed and so planning the
evaluation more in detail. For example, the question “is the PSS usable?” can be decomposed in
sub-questions such as: Is the user interface easy to navigate? Is the terminology confusing
because it is inconsistent? Is the feedback provided to users sufficient? Is the response time too
slow in displaying the results?
In relation to the goals presented in 5.2.1, possible questions are: which steps are
difficult to understand by planning students? Can students use the PSS efficiently up to the end
of the course? What are the skills required by planning professionals to adopt the PSS? Which
PSS provides the most efficient functionality? What could potentially be improved in the current
version? Should the next version be web-based or implemented as a plugin to a GIS?
5.2.3 Choose the evaluation and data collection methods
Methods for achieving the goals and answering the questions have to be chosen. Many
evaluation methods are proposed in the literature (e.g. Lanzilotti et al. [2011], Preece et al.
66
[2015]). The most commonly adopted are i) inspection methods and ii) user-based methods
which are discussed in detail in Chapter 2.
Studies have outlined that these two methods are complementary and can be
effectively coupled to obtain a reliable evaluation process [Lanzilotti et al. 2011]. Some
evaluation methodologies have been proposed that rely on these methods. For example, the
eLSE (e-Learning Systematic Evaluation) methodology prescribes a structured flow of activities
for evaluating e-learning systems [Lanzilotti 2006]. Specifically, eLSE suggests coupling
inspection activities and user testing, and more precisely indicates how to combine them to
make evaluation more reliable and still cost-effective.
The PSS Evaluation Framework proposes a similar approach which suggests inspection
first. User testing will be performed only in specific cases, when the evaluator feels the need for
a more objective evaluation that can be obtained through user involvement. In particular, the
framework suggests using the thinking-aloud technique; it can be complemented with other
methods involving users, namely questionnaires and interviews, which are especially valuable
to assess user satisfaction and other hedonic qualities of UX.
Observation, screen recording and measures of user performance are other often
adopted user-based methods. Video and screen recordings are recommended for complex tests
as they allow reviewing participant’s interaction and behaviour if something is unclear, for
example, during data analysis. Overall, it is recommended to use multiple data collection
methods for a variety of reasons:
• the results of measured usability (gathered through measures of user performance during
the interaction) and perceived usability (users indicate their experience, for example, in
questionnaires and interviews after the interaction) can differ,
• they provide rich datasets which are most valuable in relatively unexplored research fields
such as PSS usability evaluation,
• the convergence of information from different sources increases the validity of results
[Hilbert & Redmiles 2001].
5.2.4 Identify the practical issues
Many practical issues have to be considered when conducting an evaluation such as evaluators’
expertise and resources required. Indeed, one of the reasons why most developers neglect
usability evaluation is because they think that they do not have the right expertise and that too
many resources are required [Ardito et al. 2014]. As stated above, one of the goals of the PSS
67
Evaluation Framework is to provide enough guidance to planning actors, suggesting methods
appropriate for specific situations and indicating required resources.
Usability inspection should be performed by at least four usability experts [Nielsen
1993]. If only one expert is available, more novice evaluators (e.g. students of usability courses)
should be involved. Novice evaluators might consider using Nielsen’s heuristics [Nielsen 1993].
After the inspection, the evaluators should compare and discuss the results.
In user-based methods, several issues have to be addressed, including:
(1) choice and recruitment of the participants; participants of user tests are ideally end
users. At least four to five participants should engage with the software [Nielsen 1993].
As PSS is specialised software with a focused application on improving the efficiency
and/or effectiveness of planning tasks, planning professionals who are familiar with
computer applications are recommended as participants of user-based PSS evaluations.
Based on the evaluation goals, participants might have to meet specific requirements,
in order to get reliable results. It is recommended to clearly specify requirements that
they have to meet when recruiting participants.
As in a number of professions, planners are time poor and it is difficult to have them
agree to participate in a user test, especially if it is not carried out at their workplace. As
such, recruitment of participants should be well planned. This task can take some time
to complete and so considerable time and effort should be given to this task. It is
therefore recommended to make a list of existing planning organisations and to define
the order according to which these are contacted (see Appendix H). For example,
planning organisations which are closer to the venue where the evaluation will be
carried out, should be contacted first. Carrying out the user test at their workplaces
should be considered as this will likely increase their willingness to participate. An
alternative to planning professionals might be students of planning courses. Particularly,
students of higher degrees often work in planning organisations concurrently with their
studies and might have the required experience in planning practice for participating in
PSS evaluations. Depending on available resources, providing some incentives can help
in the process of recruiting participants.
(2) choice of the experimental design; if two or more products are evaluated through user
testing, a decision has to be made whether a between- or within-subjects design is
adopted. In a between-subjects design, each participant is randomly assigned to each of
68
the various experimental conditions, i.e. he/she is going to use only one product; in the
within-subjects design, each participant is tested under each experimental conditions,
i.e. he/she is using all products [Graziano & Raulin 2012].
Furthermore, with respect to the activities that participants are asked to perform during
the user test, so-called task-centred and scenario-centred tests can be differed [Polillo
2010]. In task-centred tests, participants execute single tasks which allow testing specific
functions of a software system and their usability in detail. These tests can be performed
both when system development is not fully completed as well as when development is
quite advanced.
In scenario-centred tests, a scenario is provided to participants that includes a goal to
be achieved through the interaction with the system. Scenario-centred tests expect
participants to use the system in relation to their requirements, preferences and habits.
This allows gathering information on the overall usability of the system such as whether
the system allowed participants to adopt their workflow process or whether the system
lacked functionality; thus, scenario-centred tests are more comprehensive. To avoid
having major defects identified late in the development process when they are much
more expensive to solve, these tests should be carried out as early as possible in the
development process. Independent of which test is carried out, the tasks or the scenario
must be provided in written form, in order to provide all participants with the same
conditions.
(3) choice of the venue, facilities and equipment; equipment required for performing the
user test and for data gathering has to be identified. If the think-aloud technique is
adopted, it is recommended to record the audio, in order to have the opportunity to
listen again to comments if something is unclear during data analysis. Appendix I
provides two examples of free software for recording. If web-based PSS is evaluated
ensure the availability of good internet connection. Usability labs usually provide
standard equipment and facilities for user tests, which is also a guarantee for ensuring
the same conditions for all participants. Hiring usability labs can, however, be expensive
and also has the downside that the user test is carried out at participants’ workplace.
The equipment should be tested to ensure the quality of the data.
It is recommended to conduct a few pilot studies, for example, with planning students,
in order to test the feasibility of the user test. These can give answers on whether
69
participants’ tasks are clear and on whether changes have to be undertaken to the
experimental design. Ideally, two persons are present during the user test; a facilitator
that interacts with the participant and one that manages the equipment and takes notes
of the observation. As a first step, the facilitator introduces the user test to the
participant. A running sheet (see Appendix J) is recommended, in order not to forget any
information that has to be provided to the participant. Furthermore, it must be ensured
that nobody can disrupt the user test and that all electronic devices with sound alerts
are disabled. Allow enough time (approximately one hour) between two participants
(see Appendix K) for revising notes and preparing the next test session. A summary of
required documents and advice of what to do or not do during user tests are provided
in Appendix L and M.
5.2.5 Decide how to deal with the ethical issues
Evaluations involving people have to address ethical issues, in order to protect participants and
their privacy. Participants have to be informed about data that is gathered and how this is used.
This is often done through a plain language statement. Universities typically provide these for
standard human ethics applications. Participants are commonly also provided with a consent
form, again which is usually provided by universities and other research institutions. A plain
language statement and consent form developed specifically for PSS evaluation are included in
Appendices N and O. Participants need to sign this, in order to state that they know about the
procedure and agree to participate. To guarantee that the evaluation is carried out in alignment
with ethical codes, the evaluation study usually needs to be presented to the responsible
authority which has to approve it. However, ethical issues are treated differently among
institutions; thus, it is recommended to consult their policy as to how to obtain ethics approval.
As the evaluation cannot be conducted prior to approval, enough time should be allowed for
getting ethics clearance.
5.2.6 Evaluate, analyse, interpret and present the data
Before conducting the evaluation, decisions have to be made about how data is analysed and
presented. For more details, the reader may refer to Graziano and Raulin [2012]. Another critical
decision is whether qualitative data versus quantitative data is considered and the identification
of proper methods and metrics to evaluate it. Several examples for measuring UX attributes are
in Albert and Tullis [2013].
70
It is most important to define how the quality of the collected data can be demonstrated
[Graziano & Raulin 2012]. To this aim, important factors such as reliability and validity have to
be considered. Reliability is an index of the consistency of the measures applied to the data.
Good measures give consistent results, regardless of who does the measuring. A measure is not
wholly reliable or unreliable, but varies in its degree of reliability. A correlation index can be used
to quantify the degree of reliability. An example of a correlation index is the coefficient alpha,
although much more complicated ones exist. Validity of the measure is its effectiveness in
reflecting the characteristic measured. Validity, like reliability, varies in its degree. Once again, a
correlation index is typically used to quantify the degree of validity [Graziano & Raulin 2012].
Data collected during the evaluation must be analysed, interpreted and properly
presented. This framework presents some tools for data analysis: a table for reporting
participants’ performance (see Appendix P), a scheme for coding observation, screen recording
and thinking-aloud (see Appendix B), steps for evaluating responses to the System Usability Scale
(SUS) questionnaire (see Appendix Q).
Statistics are powerful tools for analysing and understanding data. Specifically,
descriptive statistics summarise, simplify and describe a large number of data. Examples of
descriptive statistics are: mode, median, mean, average deviation, variance, standard deviation.
They are a basis for later analyses in which inferential statistics could be used. Inferential
statistics help researchers to interpret what data means. Examples of inferential statistics are:
T-Test (this applies to independent groups, i.e. when samples are different, as for between-
subjects design), correlated T-Test (used for within-subjects design), analysis of Variance or
ANOVA (when more than two groups are involved), repeated ANOVA measures (used for within-
subjects design involving more than two groups).
Besides video recording, which provides further qualitative data, it is suggested to take
pictures during evaluation (e.g. of participants during an interview or a user test), because they
are also useful for documentation purposes. Data is better presented if organised in tables,
histograms, pie diagrams, etc., whenever possible.
5.3 Conclusion
The PSS Evaluation Framework provides a possible structure and activities to undertake when
planning and executing evaluations. However, the framework is not meant to be exhaustive,
rather it provides a guide for those who embark on the PSS development and evaluation path.
Users might be required to undertake other activities as described in the framework depending
71
on the evaluation context. Indeed, the framework provides indications of literature where
further information can be found and a series of templates included in Appendices G to Q.
As part of future work, the PSS Evaluation Framework could be further specialised, for
example, on user-based or heuristic evaluations, or complemented with more specific examples
of methods such as for analysing and presenting data. Within the scope of this thesis, the PSS
Evaluation Framework was used for planning a user test of three PSS; Online What if?,
CommunityViz and Envision. The application of the framework and to what extent activities
were adopted for carrying out the user test is described in the next chapter.
72
Chapter 6: A PSS usability evaluation study
This chapter describes a user test conducted with six planners, each using three Planning
Support Systems (PSS). This evaluation study was carried out to examine possible usability
problems that planners encounter when interacting with PSS and User eXperiences (UX) that
these cause. Additionally, some system functionality planners expect when using PSS emerged.
This should be considered by developers when designing PSS, as it is one of the other indications
provided in this chapter for creating more satisfying PSS for planners. The user test was designed
using the PSS Evaluation Framework presented in the preceding chapter. After providing the
motivation for this user test and the description of the three tested PSS, the chapter is structured
following the activities proposed in the PSS Evaluation Framework. These involve presenting the
evaluation goals and the overall study including methods used for data gathering, analysis and
presentation. The chapter concludes with the findings of this study for designing PSS capable of
meeting planners’ requirements.
6.1 Introduction and motivation
Previous literature as reported in Section 2.7 shows that a basic problem for low adoption of PSS
is a mismatch between PSS functionality, and the way it is provided through the user interface,
and what planners expect. For instance, Vonk et al. [2007b] indicated that planners demanded
quite simple PSS but developers supplied advanced systems. Furthermore, despite the literature
reporting that planners are under constant resource constraints [Houghton et al. 2014], PSS are
characterised by requiring considerable data for their operation, the collection and preparation
of which are very time-consuming activities [Brits et al. 2014].
Ways to decrease this mismatch are to know user requirements and take them into
account during PSS development, as well as to regularly evaluate PSS during development to
test if user requirements are well implemented [Rubin 1994]. These methods, which are
supported by the Human-Computer Interaction (HCI) research through the user-centred design
process (see Section 2.10), are relatively neglected in PSS development. Specifically, Allen [2008]
stated that usability evaluations of PSS are quite rare. Only a small number of PSS evaluations
are reported in the literature [Arciniegas et al. 2013; Brömmelstroet 2012; Geertman 2002].
These mainly involved PSS implemented on touch tables tested in workshop settings. However,
in such collaborative settings users do not always interact with the PSS themselves. Rather, PSS
capabilities are often shown by facilitators and discussed in groups of stakeholders. Indeed, the
evaluations mainly gathered participants’ feedback on the potential value of the PSS for
73
improving communication and collaboration, but did not focus on testing the PSS in-depth for
identifying usability problems and expected functionality, which requires more systematic and
rigorous evaluation.
A development process that considered user requirements was used for creating a PSS
in Vonk and Ligtenberg [2010]. More precisely, they developed and tested two PSS prototypes
for collaborative sketch planning. One prototype was developed by following a traditional
system engineering method, while for the second prototype a socio-technical approach was
adopted [Ackerman 2000; Sutcliffe 2000], which requires the development of the system in close
collaboration with users. In the HCI discipline, this approach is called “Participatory Design”
[Schuler & Namioka 1993]. Due to the involvement of users, the socio-technical approach takes
into account what users consider really important for them. There is the opportunity to discuss
with users appropriate functionality of the system and its implementation according to their
view of how the system should work, in order to effectively support their work practice and the
way they are used to performing their tasks.
When considering participatory design teams, some authors talk about “symmetry of
ignorance” [Rittel 1984]; in other words, the usual situation in such teams is that system
developers are expert in technology and know less about the application domain, while users
and other stakeholders do not know about technology but are expert in the application domain.
Thus, in socio-technical development, developers and users compensate for the weaknesses of
each other and, together, are able to perform improved task analysis. Moreover, users’ skills
and expectations become more explicit during the process, which results in a better usability of
the final system. Indeed, Vonk and Ligtenberg [2010] showed that the traditionally developed
PSS was not well regarded compared to the PSS developed through the socio-technical approach
because of their poor functionality and usability.
Despite such studies, little has changed in the development of PSS. Pettit et al. [2014]
propose a co-design approach that emphasises collaboration of software engineers and planners
during PSS design and development. However, there is a need for more research on these topics
to convince developers that PSS they create have to comply with planners’ actual expectations
in order to become more widely used.
This chapter provides a contribution in this direction by reporting an evaluation study
with six professional planners, whose results provide indications for creating PSS that planners
are willing to use in their daily work. The study was designed according to the PSS Evaluation
Framework described in Chapter 5 and in Russo et al. [2015] and involved the six planners as
74
participants of a user test. Each planner interacted with three PSS individually which allowed
investigating the adoption of PSS by practitioners and the extent to which lack of proper
functionality and low usability still occur in current PSS. During the user test, the participants
were asked to perform a Land Suitability Analysis (LSA) with three PSS.
LSA is one of the common activities undertaken by land use planners when performing
site selection or strategic planning tasks as illustrated by the considerable amount of literature
on it and the various reported case studies (e.g. Klosterman [1999], Pettit & Pullar [1999], Pullar
& McDonald [1999], Jankowski & Richard [1994]). LSA determines the suitability of each land
unit for a specific purpose, based on a set of parameters that planners or actors in the planning
process have to set, in order to calculate the output. Specifically, the performed study had two
goals. The first goal was to identify any usability problems of the tested PSS. The second goal
was to better understand planners’ expectations when interacting with PSS, in order to provide
PSS developers with insights for creating systems that practitioners find satisfactory for
supporting their activities.
The structure of this chapter is as follows: key characteristics of the three PSS evaluated
in the user study are described in Section 6.2. The overall study is reported in Section 6.3. On
the basis of the study results (see Section 6.4), Section 6.5 discusses the findings in relation to
the design of PSS capable of meeting planning professionals’ requirements. Section 6.6 provides
conclusions.
6.2 An overview of the selected Planning Support Systems (PSS)
In the evaluation study, six professional planners were required to test the LSA module of three
PSS. This was implemented in different ways as studying multiple implementations increases
understanding and explanation of similarities and differences between the systems [Miles &
Huberman 1994]. In order to identify the three PSS to be used in the test, the Online PSS
Resource presented in Chapter 4 was analysed to determine the most suitable PSS to be
selected. Nine PSS that were easy to access and to install, and allowed performing any spatial
planning task as defined in the literature [Balzarini et al. 2013; Pullar & McDonald 1999], such
as impact assessment, LSA, land use demand and allocation analysis, were shortlisted and
thoroughly inspected. The inspection results are presented in Table 3.1. The three PSS that best
complied with the scope of the user test were selected. Specifically, they had to satisfy the
following two conditions:
(1) provide a task that allows participants to go through the whole workflow process from
75
data input to data output within a reasonable amount of time,
(2) offer a certain level of guidance so that participants would be able to work without
requiring step-by-step instruction during the test.
Condition (1) was taken into account, in order to investigate properties that planners
consider very significant, like PSS transparency (i.e. “the extent to which the underlying models
and variables of the PSS are accessible and understandable to users”) and reliability (i.e. “the
extent to which the outcomes of the tool are perceived to be valid”) [Pelzer 2015: 134].
Condition (2) was considered because according to the literature [Seewald & Hassenzahl 2004],
tasks during user tests do not have to be too easy but must be possible to solve. The other six
inspected PSS were excluded because they did not fulfil the two conditions. More precisely, their
workflow process from data input to output was too long. The consequence was that too much
time was required for task completion (contradiction to condition (2)) and it would have been
impossible for participants to perform all necessary steps during the user test. As an alternative
to the latter, only parts of the whole workflow process could have been tested, which however
is in contradiction to condition (1).
Finally, the evaluation focused on the PSS modules devoted to LSA, which determines
the suitability of each land unit (usually at a land parcel level) for a specific purpose and based
on a set of parameters. Through the assignment of weights, users can attribute different
importance to parameters. All three PSS draw on Spatial Multi-Criteria Evaluation, a well-
supported methodology in decision-making processes [Arciniegas et al. 2013] that combines
multiple datasets with the same spatial extension into one single dataset (output). The output
includes a map, also referred to as a land suitability layer, which displays the suitability score for
each land parcel through colour-coding. LSA functionality differs in the three PSS, requiring
different weighting scales or parameter processing. The three PSS are briefly described as
follows, illustrating the main operations for performing LSA.
CommunityViz (http://placeways.com/communityviz, accessed on May 24, 2017) is a
commercial desktop extension of the ESRI ArcGIS (GIS software). The user performing LSA has
first to define a series of settings, in particular suitability criteria for each parameter of interest.
For example, “is the suitability of a land parcel positively or negatively associated with increasing
distance to flooding risk zone?” or “is the suitability of a land parcel positively or negatively
associated with increasing overlap with industrial development zone?”. After that, the user
assigns to each parameter a weight in the range of zero to ten through slider bars. The suitability
layer obtained as output can be analysed through spatial analysis functionality available in
76
ArcGIS (see Figure 6.1).
Besides LSA CommunityViz allows performing build-out analysis and impact assessment.
The build-out analysis supports the user in identifying the holding capacity of land in terms of
buildings. Specifically, the analysis provides answers on where in a larger area future
development could be allocated. The impact assessment tool draws on a rule-based model and
focusses on impacts of land use change at urban precinct to city scale [Holway et al. 2012; Pettit
& Klosterman 2005]. CommunityViz is one of the most notable PSS [Vonk et al. 2005] and counts
numerous applications in the United States, probably also due to the extensive user support
provided [Vonk & Geertman 2008]. For instance, hyperlinks in the user interface represent help
documentation that explains provided functions.
Figure 6.1: Parameters, slider bars and the suitability layer provided in CommunityViz
Envision is a software tool that has been implemented as a plugin in QuantumGIS (i.e. a
Geographic Information Systems (GIS) software tool) initially for research purposes with the
broader aim of being made available and used by practitioners. The objective of the design was
to develop a tool that supports sustainable redevelopment of precincts in Australia [Newton &
Glackin 2013]. LSA is based on thirty-four predefined parameters, related to property,
demographics and location (see Appendix R). The user selects the parameters that he/she wants
to consider and assigns to each parameter a weight in the range of zero to twenty through slider
bars (see Figure 6.2). As an option, through a tick box, the user can choose to display overlays
and aerial imageries that provide current zoning information and basemaps. The output, i.e. the
77
suitability layer, is displayed in QuantumGIS. Spatial analysis functionality available in
QuantumGIS can be used to further analyse this suitability layer. Two test cases have been
implemented in Envision by the developers. These involve two cities in Australia, namely
Manningham in Melbourne and Canning in Perth.
Overall, Envision offers, besides LSA, one module related to redevelopment potential,
one module dealing with housing capacity and one module considering housing supply impacts.
The redevelopment potential module strives to highlight properties that due to their age, zoning
or other factors are ripe for redevelopment in the near future. Indicators of redevelopment
potential are defined through binary logistic analysis as well as user input. The housing capacity
module helps users to identify areas with maximal dwelling capacity. This can be through
applying new residential density on selected areas or rezoning non-residential land to
residential. Existing conditions such as housing supply are taken into account for comprehensive
analysis. The housing supply module produces estimates of lot amalgamation impacts. Changes
can occur through consolidation of blocks and creation of larger lots which enable more
flexibility in design for achieving greater housing affordability and sustainability. The
computation is made possible through geoprocessing functionality of Envision to buffer and join
adjacent lots. Envision incorporates various analysis methods and data at cadastral basis such as
of geographical, demographic nature, property values, distance measures. As such, it should
reach various stakeholders engaged at urban, local and precinct level with the aim to increase
their interaction and discussion for better urban growth management.
Online What if? [Pettit et al. 2015b] is an open source, web-based PSS, which actually is
the modified version of the well-known standalone PSS What if? [Klosterman 1999]. To perform
LSA, the user first has to define a series of settings, for example, developable and convertible
land. After that, the user selects the parameters that he/she wants to consider and assigns to
each parameter a weight in the range of zero to one-hundred through spin boxes (see Figure
6.3). As a result, land use parcels are classified as not developable, not convertible, not suitable,
or suitable. In the case of suitable parcels, in the resulting suitability layer, a value on a five-point
scale from low to high is indicated. For exploring the suitability layer, basic navigation functions
such as panning and zooming are provided. The LSA module in Online What if? is based on
McHarg’s [1969] sieve overlay approach.
Other modules in Online What if? address land use demand and allocation. The land use
demand module creates projections of demands for different land uses. These are driven by
demographic trends and policies such as requirements for open space. If such information is not
78
Figure 6.2: The thirty-four parameters and slider bars provided in Envision for selecting their weights (right) and creating the suitability layer and overlays in QuantumGIS (left)
Figure 6.3: Parameters and spin boxes provided in Online What if? to assign the weights (left) and create the suitability layer (right)
available linear extrapolations of population and employment are taken into account. The
resulting land use demands are provided in table format and used for proceeding with the next
module. This is the land use allocation module. It builds upon both the LSA and land use demand
modules and is a rule-based urban growth model [Pettit & Klosterman 2005]. The allocation
module assigns land use demand to suitable sites. Once the demand has been covered the
79
engine deals with the allocation of the next land use demand. Diverse scenarios are projected
that take into consideration different time intervals and policies such as zoning and growth
controls, preservation and infrastructure services. Online What if? is one of the most notable
PSS [Vonk et al. 2005; Pettit & Klosterman 2005] and counts numerous applications in the United
States (e.g. Ohio [Klosterman et al. 2003]) and Australia (e.g. Perth [Pettit et al. 2015b], Mitchell
Shire [Pettit et al. 2008], Hervey Bay [Pettit 2005]).
6.3 The evaluation study
The study was performed according to the PSS Evaluation Framework outlined in Chapter 5 and
in Russo et al. [2015]. The first activity of the framework is to determine the evaluation goals. In
particular, two goals were defined: i) to identify any usability problems encountered, and ii) to
better understand participants’ expectations of functionality when interacting with a PSS, in
order to provide PSS developers with indications for designing more usable PSS.
A set of questions, underpinning the goals, was also defined, as indicated by the second
activity of the PSS Evaluation Framework. For the first goal, the questions identified included: Is
the user interface easy to navigate? Is the terminology confusing? Is the feedback provided to
users clear? For the second goal, the questions included: Which steps are difficult to
understand? What functionality should PSS provide in order to be accepted by practitioners?
Which PSS provides efficient functionality? The complete set of questions is reported in
Appendix S.
The third activity was the choice of the evaluation method. A user test with the thinking-
aloud technique was chosen because it offers a window into the users’ mental models, allowing
evaluators to detect any misconceptions about the system and the interface elements that cause
them. In addition, it provides useful results even with a small number of users [Nielsen 1993]. In
the following sub-sections, details on the user test, organised according to the fourth, the fifth,
and the sixth activity of the PSS Evaluation Framework, are reported.
6.3.1 Participants
A total of twenty-one people from the planning discipline in the State of Victoria in Australia
were contacted, primarily via e-mail, and informed of the purpose of the user test. Six people
agreed to participate. The participants were professional land use planners (age between
twenty-five and forty-five years old, three female). One was from local government and five
were from the private sector. The local government planner worked for a city council in the
80
south-west of Melbourne, while the other five planners were employed in three different
consultancy and service companies located in Melbourne. Two of the five planners operating in
the private sector worked for a consultancy that specialised in strategic planning and urban
design. The other three worked in globally operating enterprises that provided services in
planning, architecture, environment, engineering and other domains. Participants had to meet
the following criteria: i) be strategic planners, ii) be familiar with basic GIS functionality such as
layer (de)activation, map zooming and panning and iii) have not used the three PSS before. The
latter was to ensure that the participants had the same expertise of the PSS under investigation.
This is a prerequisite for user test participants according to the HCI literature [Preece et al. 2015].
6.3.2 Design and procedure
The evaluation study of the three PSS was carried out following a within-subject design. This
means that each participant used all three PSS in sequence but in a different order by
considering all permutations of the three PSS (see Table 6.1), in order to avoid potential learning
effects [Graziano & Raulin 2012]. A technical problem prevented Participant 5 from beginning
with CommunityViz. Thus, he used the PSS in the order shown in Table 6.1. The problem did not
affect the evaluation in that the participant did not notice the problem.
Table 6.1: The counterbalanced order in which the participants (P1,…,P6) used CommunityViz (CViz),
Envision (Env) and Online What if? (OWI)
No time restriction was imposed on the duration of the test, which was instead
determined by participants’ task completion with all three PSS. Ethics clearance was provided
by The University of Melbourne: each participant read and signed a consent form. The
evaluation was carried out over two consecutive days. Three participants per day came to a
usability lab at The University of Melbourne. The facilitator firstly informed the participant of
the procedure and the task to perform during the test with each PSS. The LSA task was:
Participant 1st PSS 2nd PSS 3rd PSS
P1 Env CViz OWI
P2 CViz OWI Env
P3 OWI Env CViz
P4 Env OWI CViz
P5 Env CViz OWI
P6 OWI CViz Env
81
Identify an area within the City of Canning (Western Australia) where residential redevelop-
ment might be most suitable based on a set of parameters that you regard as important.
The City of Canning is a municipality in the south-east of Western Australia. In recent decades,
it has experienced continuous growth, developing from a district to a city and becoming part of
the greater metropolitan area of Western Australia’s capital city of Perth. In particular, large
population growth motivates the need for residential redevelopment.
Table 6.2: The parameters in each of the three PSS as within the scope of the user test.
CViz Env OWI 1 Heritage restriction RPI Heritage restriction 2 Future industrial land use Age of dwelling Future industrial land use 3 Flooding zone Area Flooding zone 4 Distance to primary school Frontage Distance to primary school
5 Protected area and national park Development efficiency Protected area and
national park 6 Development planned Lot squareness Development planned 7 Zoning 8 Strata titled 9 Vacant land
10 LGA owned 11 Sensitive area 12 Age 0 - 19 13 Age 20 - 29 14 Age 30 - 54 15 Age 55 - 74 16 Age 75+ 17 SEIFA 18 Distance to primary centre 19 Distance to neighbourhood 20 Distance to local centre 21 Distance to train station 22 Distance to bus stop 23 Distance to main road 24 Distance to park 25 Walkability 26 Relative extra land 27 Distance to primary school 28 Distance to secondary school 29 Distance to tertiary school 30 Recent nearby demolitions 31 Relative density 32 Net increase 33 PTAL/SNAMUTS 34 Slope
82
While Envision focuses on LSA specifically for residential redevelopment within a
municipality, CommunityViz and Online What if? support LSA for any use, including but not
limited to redevelopment. Envision already provided a test case on the City of Canning, including
thirty-four parameters. The facilitator had to prepare a data set for CommunityViz and Online
What if?. This included a geographical base map of the City of Canning and the parameters. Only
a set of six parameters was provided to the participants (see Table 6.2). This set was determined
by: i) data availability as provided by the Australian Bureau of Statistics (ABS) [2003] and ii) the
aim to examine if a smaller set of parameters than provided in Envision had a different impact
(e.g. complexity, transparency) on the participants.
As the participants had never used the three PSS, a short introduction was given to each
tool prior to the interaction, providing information required for completing the task that
participants could not be aware of. The introduction was longer for CommunityViz and Online
What if? than for Envision because the latter does not have a project setup module. According
to the thinking-aloud technique, during the interaction with the PSS, the facilitator intervened
to elicit information, to clarify unclear verbal statements, to provide help if the participants
could not continue. As it was expected that some parameters and their acronyms used in
Envision could be unclear, a list was provided to the participants that explained the meanings
(see Appendix R).
After engaging with each PSS, the participants were asked to complete an intermediate
questionnaire about their UX which is reported in Appendix T. At the end, a final questionnaire
gathered participants’ planning experience, expertise in GIS and educational level (see Appendix
U). Prior to execution of the evaluation study, a pilot study involving two planning students was
carried out to check and refine the overall procedure.
6.3.3 Apparatus
The technical equipment used by the participants in the usability lab included a screen (twenty-
three inches widescreen, resolution: 1366 x 768), a keyboard and a mouse. To decrease the level
of surveillance and make the participant feel more comfortable, the facilitator followed the
session on a laptop next to the participant. During the user test, video, audio and screen
recording was conducted. A camera was positioned at the ceiling of the lab (see Figure 6.4).
Later in the study, the recordings helped to analyse user actions during the interaction.
83
Figure 6.4: A participant (left) and the facilitator (right) during the test
6.3.4 Data collection and analysis techniques
Data collected through the video recording (including audio), screen recording and
questionnaires was triangulated. The questions were of qualitative nature, in order to give
participants the opportunity to better describe their experience with the PSS.
The audio recording data was transcribed by the author. In order to gather information
related to the first goal, i.e. to identify any usability problems, the transcription, the recordings
(video and screen) and participants’ answers in the questionnaires were assessed against
behaviour, actions and statements that according to a coding scheme inspired by Vermeeren et
al. [2002] (see Appendix B) reveal difficulties in interacting with a system; thus, indicate usability
problems.
Actions highlighting usability problems were: i) random actions, i.e. actions that did not
belong to the correct sequence to perform the task or the user randomly clicked on interface
objects and moved the mouse on different widgets; ii) puzzled actions, i.e. actions that indicated
that the user either did not know how to perform the task or was not sure whether an action
was needed; iii) uncomfortable actions, i.e. actions which the user indicated to be difficult or
uncomfortable to execute.
Behaviour and statements highlighting usability problems were: i) uncertain behaviour,
i.e. the user showed uncertainty about an aspect or a content of the interface or he/she did not
understand an action effect; ii) dissatisfied behaviour, i.e. the user indicated disliking something
or the effect of an action was unsatisfactory or frustrating.
84
The author coded the random, puzzled or uncomfortable actions in the data as well as
the statements and behaviour that indicated uncertainty or dissatisfaction. The transcription of
the audio recording data and participants’ answers in the questionnaires were also analysed, in
order to gather information related to the second goal of the study, i.e. to understand
participants’ expectations of PSS functionality. The author adopted an open coding approach.
Specifically, as a first step, i.e. code data, statements were coded that indicated functionality
appreciated or disliked by the participants and that suggested design improvements and
explained the importance of specific functionality from the point of view of the participants. As
it is required in this kind of analysis, the codes were double-checked for reliability by another
researcher. Discrepancies were solved through discussion in a consensus meeting. It was
followed by the second step, translate codes into themes, in which the author condensed the
codes into two themes: i) what are participants’ expectations of functionality and ii) what do
they want to achieve with this functionality. The latter is to investigate why the participants
expect specific functionality and what this is determined by (e.g. work habits, regulations). The
results of this analyses are reported as follows.
6.4 Results
All participants were able to complete the task, i.e. identify an area in the output where
residential redevelopment might be most suitable based on a set of parameters. However,
several problems emerged. As follows, the results are reported according to the two goals of the
usability study.
6.4.1 Goal 1: Usability problems
Usability problems of the PSS were identified by analysing participants’ behaviour, actions and
statements as described in Section 6.3.4. Table 6.3 reports the total number of the random,
puzzled and uncomfortable actions, performed by the users while interacting with the three PSS.
Table 6.3: Actions highlighting usability problems in CommunityViz (CViz), Envision (Env) and Online
What if? (OWI)
CViz Env OWI
Random actions (N) 3 1 14
Puzzled actions (N) 4 2 15
Uncomfortable actions (N) 2 3 9
85
Specifically, while interacting with CommunityViz, the participants carried out a total of
three random actions. Only one random action was observed during the interaction with
Envision and fourteen with Online What if?. CommunityViz prevented participants from
performing actions out of the correct sequence and therefore, provided a form of guidance. In
fact, a participant said: “it’s all greyed out so far. I can’t select anything else here except for that”.
Four puzzled actions were observed during the interaction with CommunityViz, while
two and fifteen puzzled actions were identified with Envision and Online What if?, respectively.
“Should I take it out?” or “How do you actually run it?” are examples of statements highlighting
puzzled actions. The terminology used in Online What if? and Envision user interface appeared
to be a cause of puzzled actions. Two users said respectively: “I do not know what to do, the
terms are confusing” and “development efficiency … perhaps that’s a Western Australia thing,
it’s not something we usually have in Victoria”. In many situations in which puzzled actions
occurred, the intervention of the facilitator was required either because the participants
requested it or to stop actions that prevented task completion (e.g. cancel input data).
Finally, nine uncomfortable actions were identified during participants’ interaction with
Online What if?. A smaller number of such actions were recognised for the other two PSS, i.e.
two for CommunityViz and three for Envision. “It is difficult to use this [spin box]” or “inputting
all the info is a bit tedious” are examples of statements highlighting uncomfortable actions.
Table 6.4: Frequency and percentage of the participants’ negative behaviour during the interaction with
the PSS
CViz Env OWI f % f % f %
Uncertain 28 76 18 78 39 71 Dissatisfied 9 24 5 22 16 29 Total 37 100 23 100 55 100
Frequency and percentage of the participants’ negative behaviour are reported in Table
6.4. Envision appeared the least problematic PSS, followed by CommunityViz and Online What
if?. Participants indicated to be uncertain eighteen times with Envision, twenty-eight and thirty-
nine times with CommunityViz and Online What if?, respectively. Examples of statements
showing uncertainty were: “I would just assume that green is good and red is bad” and “I don't
really understand these widgets”. For all three PSS, the participants showed uncertainty:
specifically on how to interpret the weighting scales and suitability scores in the case of
CommunityViz and Envision (i.e. how does a weighting of thirteen differs from sixteen or a
86
suitability score of 2.5 from 2.7) and on the basis on which suitability classes in Online What if?
are established. Examples of participants’ statements in relation to this are: “I don’t know what
the different weightings mean”, “the numbers do not mean too much to me at this point” and “I
don't know what the low, medium, high suitability means in the legend”.
Only five statements were related to dissatisfaction during the interaction with Envision.
The same behaviour was recognised nine and sixteen times when interacting with
CommunityViz and Online What if?, respectively. Examples of statements indicating
dissatisfaction were: “I do not want to continue with this activity” and “I don't like the scale”.
6.4.2 Goal 2: Users’ expectations
Users’ expectations were identified through participants’ indications for design improvements
including those which were made unconsciously, in order to make the PSS more compliant with
their expectations. A total of thirty-three indications were collected. Only one of these was
related to CommunityViz: “You need to have the assumptions outlined”. Twelve re-design
indications addressed Envision and twenty Online What if?.
At different stages of the interaction with Online What if?, the participants emphasised
the wish for a map display (Online What if? provides a map only at the end for displaying the
suitability layer). An example of a statement was “I want a map”.
Participants also indicated the importance to have contextual information pertaining to
the study area by stating: “I would start with the big picture”, “we first look at the existing
conditions” and “it is good to have them [the overlays] because you can see the context”.
Participants’ willingness to zoom-in on items of interest and filter out uninteresting
items was revealed by statements such as “we've got far too much information in there to start
stripping it out” and “it would be good to exclude existing residential so that’s a bit more targeted
and you could just focus on new areas”.
Participants appreciated the spatial analysis functionality provided in the GIS-based PSS,
i.e. CommunityViz and Envision, which was illustrated through statements such as: “It provides
all the tools and customisation options I would expect”.
A participant showed hesitance with data input in two ways:
(1) the participant stated that she wanted to make sure that she was entering a valid input
in Online What if? based on an existing planning policy that was not available during the
user test: “I probably would not do that, unless I knew there was a written
report…because that would be a political suicide”,
87
(2) the participant suggested another data type than required by the three PSS for assigning
the weights by noting “quite often we just go low, medium and high important”.
A participant voiced the need for outputs in different formats such as PDF or
spreadsheet and their export for being able to use them in planning reports and presentations.
This is provided by CommunityViz and Online What if? but not by Envision.
Some participants used the help documentation in CommunityViz accessible through
hyperlinks but complained about the difficulties in understanding the suggestions. The help
button was not selected in Envision, while Online What if? does not provide one.
6.5 Discussion and design indications
The evaluation study was carried out, in order to i) identify what types of usability problems
emerged, and ii) better understand participants’ expectations during the interactions with the
PSS so that useful indications could be provided to developers, in order to create PSS that users
can find satisfactory.
It soon emerged that tedious interaction mechanisms and lack of guidance make PSS
cumbersome to use and are likely to overshadow functionality and capabilities of PSS. An
example of a tedious interaction mechanism was the spin boxes used in Online What if? for
selecting a weight to be assigned to a parameter; participants found them more difficult to use
and inefficient compared to the slider bars provided by both CommunityViz and Envision.
Figure 6.5: Screenshot of CommunityViz online help (Source: http://placeways.com/communityviz/s360webhelp/, accessed on February 8, 2017)
88
Regarding guidance during the adoption of the PSS, participants often looked for a help
function, which was not available in Online What if? and was not working in Envision. The online
help of CommunityViz was not well structured for being easily readable on the web: too much
text, which cannot be read quickly, little provision of useful examples (see Figure 6.5). Because
PSS are quite complex systems, it is recommended to provide help documentation, possibly fully
integrated, which includes examples and short demos of system use. In addition, PSS should
offer more interaction mechanisms to guide planners. For example, ‘back’ and ‘next’ buttons to
move back and forward or greying out items to make them not selectable facilitate user
interaction.
It is evident that much more attention has to be devoted to PSS usability, which plays a
major role in systems that have to satisfy their users. The wide body of research in HCI shows
that the Human-Centred Design (HCD) process, focusing on user aspects and evaluating early
prototypes of the user interface at the initial stages of the software life cycle, is the model to
consider, in order to create usable systems [ISO 9241 2010b]. Most usability problems found
during the PSS test could have been easily avoided if usability evaluation had been performed
during software design and development. Evaluation is a key point of the approach since it
allows understanding the impact of a prototype on users and identifying appropriate
functionality for them, while revealing possible problems that are much cheaper to solve at the
early phase of the design and development [Ardito et al. 2014]. Thus, this research highly
recommends careful application of the HCD process when developing PSS.
Table 6.5 shows expectations of system functionality as indicated by the participants or
as they emerged from the usability problems they encountered. The expectations in the table
have been ordered as follows. First, what provides information for setting up and using PSS was
listed (i.e. help documentation). After that, desired visualisation and interaction tools (i.e. from
contextual data to feedback techniques) were mentioned. This is followed by functionality
expected by the planners for performing analysis (i.e. spatial and multi-disciplinary). Then,
expectations, which show capabilities that PSS should have (i.e. from transparency to efficiency
of operations), are shown. Depending on the required capabilities, PSS implemented as GIS
module or web application can be more suitable as discussed later in this section. The table
shows that although the participants interacted with PSS specifically for undertaking LSA, most
of these expectations are relevant for PSS in general.
Participants remarked that a map display, in which the case study area is clearly visible,
should be provided during the whole workflow process. The value of using visualisation and
89
feedback techniques, such as moving over or brushing, in order to support planners in their work,
is also reported in Widjaja et al. [2015]. Providing effective visualisations is not easy. A great
body of research on information visualisation is currently available (e.g. Ward et al. [2015]).
Several strategies for supporting users to understand large amounts of data have been
proposed. An example is the visual-information-seeking mantra, summarised as Overview, Zoom
and filter, Details on demand [Shneiderman 1996]. It states that a good visualisation firstly
requires an overview of the base data, in order to help users making their mental model about
the overall data; then, it should provide zoom and filter mechanisms to let users concentrate on
a portion of data of interest; finally, it should give details of specific data, when requested by
users. PSS developers need to take into account available knowledge about information
visualisation methods and tools.
Table 6.5: Planners’ expectations of PSS functionality specifically for LSA and for PSS in general
A predefined set of parameters, as provided in Envision, which is specifically designed
for specific LSA tasks (based on redevelopment potential), requires fewer actions by the user, in
order to perform the required task. This resulted in increased ease of learning of the PSS
functionality as well as ease of use and time saving (efficiency). The participants did not
comment on the relatively high number of provided parameters (thirty-four). Instead, they were
only partly happy with the provided set. Indeed, it appeared that there may not be a set of
Expectation PSS specifically for LSA
PSS in general
Help documentation ✔ Clear layout elements and interaction mechanisms (e.g. ‘back’ and ‘next’ buttons) ✔
Contextual spatial data of study area ✔ Predefined set of parameters ✔ Slider bars ✔ Map display ✔ Data and information visualisation (e.g. graph, 3D visualisation, visual-information-seeking mantra) ✔
Feedback techniques (e.g. moving over, brushing) ✔ Georeferencing and -processing functionality ✔ Comprehensive, multi-disciplinary analysis ✔ Transparency of operations and workflow process ✔ Diverse output formats (pdf, spreadsheet) ✔ Adaptability and customisation of PSS ✔ Efficiency and stability of operations ✔ GIS module ✔ Web application ✔
90
parameters that will satisfy all planners and be regarded as complete. Planners want to have the
opportunity to choose parameters individually based on their work process, their planning
problem and context. This is provided by CommunityViz and Online What if?. Specifically,
parameters have to be defined and related data prepared prior to using the two PSS.
Additional functions and settings, as provided in CommunityViz and Online What if?,
especially if they meet planners’ requirements, increase the flexibility of the system and allow
users to adapt the analysis more closely to the planning problem as occurring in the real world.
Thus, developers face a trade-off: to provide specific functionality that limits the actions the user
can perform, in order to cause fewer usability problems (as occurred with Envision), or to
provide additional functions and settings (as done by CommunityViz and Online What if?) that
allow planners to consider various aspects and therefore to perform a multi-disciplinary and
more comprehensive analysis, which is one of the most important PSS functionality according to
planners. Additional functions and settings, however, also require more input data that might
not be available for the investigation area. In many countries, including Australia, there is a move
towards open government data, which assists in overcoming some of the data access barriers,
in order to run such PSS. However, even with the shift towards open data, required data may
still not be available in the required form or may never have been collected so is not available
to support desired additional functions and settings. On this basis, the geographical context is
an important consideration and PSS developers should be aware that planners’ requirements
and data availability might differ depending on the regions and countries in which planners
work. Thus, functionality might, in turn, need to be adapted to the geographical area the PSS is
used for. As with functionality, terminology should be common to the region and country where
the PSS is used. In fact, the participants’ confusion about the terminology used by the PSS might
emerge from i) developers not having good knowledge of the terms used in planning and ii)
location-based differences of planning terminology.
The above concerns about the planners’ need of location-based functionality and
terminology highlight that the slogan “one size fits all” does not apply to PSS, as it happens with
many systems in other contexts [Cabitza et al. 2014b]. Users of the same interactive system are
often diverse, being characterised by specific cultures, goals, tasks and context of activities.
Different users may need different interfaces that provide them adequate support. People
experience many difficulties when interacting with a system that has been designed without
taking into account their cultural background, their reasoning strategies, the way they carry out
tasks in daily practices, the languages and notations they are familiar with. They do not want to
91
be constrained by formalisms unfamiliar to their culture [Ardito et al. 2012].
In the evaluation study, it clearly emerged that PSS terminology and functionality did
not actually satisfy most participants. Indeed, terminology and tasks to be performed vary
depending on countries and even regions where the PSS are used. For example, due to high
urbanisation in city centres, residential density analyses might be more important for planning
organisations in city councils than in regional councils. People wish to use software tools that
are easily accessible and usable, but which can also be tailored to their needs, tasks and habits.
A solution may come from current research on End-User Development (EUD), whose aim is to
create systems that support people to tailor software according to their own needs and
preferences (e.g. Diaz et al. [2015], Costabile et al. [2007]). Fischer [2010] pointed out that EUD
is a necessity and not a luxury because, besides people being very diverse, systems modelling
some particular ‘world’ are never complete. Indeed, new requirements emerge over time
because either the world changes or skilled domain professionals change their work practices.
Thus, developers should consider methodologies created by EUD specialists and apply them for
creating future PSS [Cabitza et al. 2014b; Costabile et al. 2007].
One of the main complaints by participants was that the PSS did not provide enough
feedback on how the resulting outputs were generated. In other words, they felt a lack of
transparency, and even distrust, in the operations performed by the PSS. This was already
highlighted in previous literature [Pelzer in press; Goodspeed 2013; Brömmelstroet & Schrijnen
2010] and is the reason why PSS are also referred to as black boxes [Lee 1973; 1994; Klosterman
2013] (see Section 2.7). In this study, lack of transparency was primarily related to the weighting
scales and suitability scores of CommunityViz and Envision and to the suitability classes of Online
What if?. For example, for the former two PSS, it was not clear how a weighting of thirteen
differs from sixteen or a suitability score of 2.5 differs from 2.7, while for Online What if?
participants did not understand on what basis suitability classes were defined. Thus,
explanations about assumptions made by the system, calculation, meaning and interpretation
of the outputs should be provided.Planners generally also use results of analyses in planning
reports and presentations. Therefore, PSS should provide outputs in other formats, for example,
PDF or CSV.
PSS can also be GIS-based. Both CommunityViz and the version of Envision evaluated in
this research were GIS-based, meaning that they were extensions of existing GIS, ArcGIS and
QuantumGIS respectively. GIS typically provides a platform with a rich set of geoprocessing and
spatial analysis functionality and customisation potential of cartographic elements (e.g. of the
92
colours and classes in a legend considered) and facilitates data management capabilities such as
the ability to manage multiple layers of data and assign metadata to support data provenance.
The PSS test confirmed that planners’ expectation of GIS functionalities differ, also depending
on their familiarity with GIS and whether they use and have access to GIS at their workplace. It
is worth remarking that most planners are familiar with GIS; thus, they expect to have GIS
functionality in PSS. Online What if? is the only one of the three tested PSS which is web-based.
Thus, it is easily accessible and does not require the installation of any software. However, it is
noted that a web-based version of Envision is currently being developed.
Independent of whether the PSS was GIS- or web-based, users complained about the
slow speed of some operations. It is highly recommended to use appropriate methods and
optimisation techniques, in order to speed up PSS operations.
6.6 Conclusion and future work
Despite the potential of PSS as tools for supporting planning professionals in making decisions
in the context of strategic planning tasks, their uptake in planning practice is still very low.
Usability problems of PSS and the mismatch between their functionality and planners’
expectations, as emerged in this chapter and confirming previous literature [Wang 2013;
Williamson 2012; Vonk 2006], certainly do not help to increase PSS adoption.
The mismatch was demonstrated in that the planners participating in the user test
voiced the desire for other functionality and interaction mechanisms than those provided by the
three tested PSS. This mismatch affected all parts of the PSS and their workflow processes,
including the help documentation, data visualisation, information gathering, analysis
functionality, general capabilities such as efficiency of operations, and implementation and
hardware issues. That the mismatch occurred throughout the whole interaction of the
participants with the PSS reveals that the developers did not satisfactorily investigate and
consider planners’ requirements during PSS design.
The objective of this study was not to compare the three evaluated PSS but rather to
identify different functionality each provides and the extent to which this met planners’
expectations. CommunityViz and the required GIS platform for operating it, i.e. ArcGIS, are
commercial. Envision and Online What if? are open source software tools. Furthermore, Envision
which is also GIS-based, uses an open source GIS, namely QuantumGIS. Generally, more
resources are expected to be invested in the development and usability of commercial than
open source software tools. Whether this is the case for the three PSS evaluated in this study,
93
and for PSS in general, could be the object of further work. In any case, CommunityViz was used
in this research because it complied with the scope of the user test, its academic license was
cost-effective and ArcGIS required for operating it was provided by the academic institution of
the author.
From the discussion of the study results, indications emerged for designing PSS that
better meet users’ requirements. One indication is to consider the identified planners’
expectations of functionality (see Table 6.5). They are meant as guidance and do not substitute
performing a rigorous user requirement analysis by developers, not least because user
expectations vary depending on the context of use. It is the task of developers and a possible
object of future research to further examine in more detail how the expected functionality
should be provided and what other functionality planners desire.
A main indication is that PSS have to be created by following the HCD process, whose
value has been demonstrated by years of HCI research. In this process, a key role is played by
the iterative approach of developing system prototypes of increasing complexity. Such
prototypes have to be carefully evaluated, in order to check if they comply with the identified
user and context requirements. Evaluation is the only way to understand the impact of a
prototype on users; moreover, it enables the identification of possible problems, whose solution
requires limited resources when performed in the early phases of the design. To the author’s
knowledge, the HCD process has not been adopted for Envision and Online What if?. Whether
it was applied for CommunityViz is unknown to the author. Generally, a comprehensive study
investigating development processes adopted by PSS designers should be the object of further
work, which would potentially explain the low usability of PSS and the need to adopt the HCD
process.
Finally, another important finding of the study is the need, expressed by planners, of
PSS capable of being adapted to their specific needs and preferences. PSS users are very diverse
not only in different countries but also in specific regions, they are characterised by specific
cultures, tasks to perform, the context of activities, governmental policies, etc. Thus, developers
should take into account that PSS users want to tailor the system they use. In order to satisfy
this basic users’ requirement, they should consider methodologies developed in the field of EUD
and apply them for creating PSS of a wide applicability. Specific research has still to be
undertaken to address these issues.
The study reported in this chapter was characterised by a small sample of participants,
an Australian focus and a particular attention to PSS specifically for LSA. In order to examine and
94
validate the emerged findings, a further study with a larger sample of participants (including
different planning actors) and concentrating on software used in planning practice and in other
countries than Australia, i.e. Italy and Switzerland, was conducted and is presented in the next
chapter.
95
Chapter 7: A tri-country interview study
The study presented in this chapter investigated in-depth the current situation of the adoption
of Planning Support Systems (PSS) and factors hampering it, devoting particular attention to PSS
usability from the point of view of planning actors. PSS functionality expected by planners was
also identified. The main part of the study consisted of thirty-five interviews with planning
actors. The results of the interviews were reviewed in a focus group with six planning actors.
This chapter is organised as follows. Firstly, previous empirical studies on factors influencing PSS
adoption are introduced. Then, the design and results of the interviews and the focus group are
presented. A discussion follows in which the results of this study are compared with past results.
Recommendations and pathways are provided for improving PSS usability and adoption before
concluding this chapter.
7.1 Introduction and motivation
During 2005 – 2008, Vonk and co-authors performed some remarkable empirical studies [Vonk
et al. 2008; Vonk et al. 2007b; Vonk 2006; Vonk et al. 2005], which aimed to investigate factors
hampering the widespread adoption of PSS. In Vonk et al. [2005], eight-hundred people from
the planning discipline (consultants, researchers, PSS developers and users) all over the world
were asked to participate in an online survey for determining the importance of sixty-two
factors, related to human, organisational, institutional and technical issues, in hampering
adoption of PSS. About one-hundred people, primarily from North America and Europe,
completed the survey. For each factor, participants had to state whether and to what degree it
prevented widespread adoption of PSS. The results showed that most of the sixty-two factors
represented obstacles to adoption of PSS. However, the three most important bottlenecks were
lack of planners’ experience with PSS (70% of the respondents), low planners’ awareness of PSS
potential (66% of the respondents) and low user friendliness of PSS (66% of the respondents).
Following the survey, in order to gather more detailed information on PSS adoption,
another more qualitative study was performed [Vonk et al. 2007b]. The study consisted of a
series of interviews conducted at regional government organisations in the Netherlands with
forty-three participants: fifteen geo-information specialists, twelve planners, three managers
and thirteen other actors involved in the planning process. The interviews were performed in
groups during twelve sessions of several hours each and aimed to identify patterns of PSS
diffusion within planning organisations. Specifically, the participants were asked to describe the
circumstances of PSS diffusion, when and why diffusion failed or succeeded at their workplace.
96
The results showed, in comparison to the online survey, a slight increase in the significance of
organisational bottlenecks and a slight decrease in importance of technical bottlenecks. In fact,
user friendliness of the system (53%) was the fifth most important bottleneck preceded by
attitude of management (71%) towards the adoption of PSS in the organisation, social
organisation of users (70%) in planning networks, awareness of potentials (66%) of PSS and
implementation support (64%) by geo-information specialists within the organisation. Low
intention to use PSS by planning organisations as well as by planners was explained by poor fit
of PSS to planners’ expectations.
Poor fit of PSS to planners’ expectations also emerged in Vonk [2006], when
investigating why some PSS are more widely used than others; it was noted that planners ask,
in general, for less complex PSS, while researchers are interested in developing sophisticated
PSS, disregarding whether these actually meet planners’ demand. From a planners’ view,
adopting and using more sophisticated PSS is associated with higher costs (e.g. for installation
and training) and risks. Instead, planners ask for incremental innovations, which are less invasive
in planners’ work routines, require less effort to adapt to them and are driven by planning
practice and planners’ demands.
A decade after such studies, there is still a situation in which PSS are challenged with
low adoption rates. In order to analyse in depth the current situation and get updated
information on factors hampering a wider adoption of PSS, the following study was undertaken.
It involved thirty-five planning actors from three countries, i.e. Australia, Switzerland and Italy,
who were interviewed to provide information related to the following goals. Goal 1 focuses on
understanding the current situation of the adoption of PSS in planning activities. Goal 2 aims to
determine factors that affect PSS adoption, whilst also identifying planners’ expectations in
order to determine functionality that PSS should provide.
The chapter is organised as follows. Sections 7.2 and 7.3 present the design and results
of the interview study. The focus group and its results are reported in Section 7.4. Section 7.5
provides a discussion of the obtained results and recommendations for creating better PSS and
mitigating the barriers to PSS adoption in planning practice. Section 7.6 concludes the chapter.
7.2 The interview study: design and execution
The study was undertaken in order to gain further insights into PSS adoption in current planning
practice and to identify specific factors that should be taken into account to create PSS that users
appreciate and enjoy using. It involved planners and other actors of the planning process in
97
Australia, Italy and Switzerland, who were interviewed to shed light on planners’ mental models
and expectations from different perspectives, as well as to explore the extent to which these
were understood by technical people and implemented in available software. The study also
aimed to gain information about planning education, in order to understand what type of
software training, if any, planning students receive in university or other tertiary institution
curricula.
This section reports how the study has been performed, describing the goals,
participants, the design and execution of the interviews and data coding. The results are
presented in Section 7.3.
7.2.1 Goals
Two goals drove this study.
Goal 1: What is the current situation of PSS adoption in planning activities?
PSS are not the only software tools to offer support in planning activities. The availability and
accessibility of such tools differ: some are proprietary software, others are free software,
available online. In order to fully understand the current situation, it is important to know the
current state of software that planners actually use.
Goal 2: What factors affect PSS adoption?
This goal aims to determine factors that affect software adoption by planners. Being aware of
the influence of the various factors will advance the body of knowledge for creating more
satisfying PSS as well as identifying pathways on how to improve PSS adoption.
7.2.2 Participants
A total of forty people from the planning discipline in Australia, Italy and Switzerland were
contacted, primarily via email, and informed of the purpose and questions of the interview. They
were identified through the snowball sampling approach [Patton 1990]. In this approach, the
researcher searching for participants of a study approaches people who might refer him/her to
potential participants. The latter are contacted and asked by the researcher whether they would
like to participate in the study under the condition that they meet the eligibility criteria for
inclusion.
Of the forty people who were contacted in this study, thirty-five agreed to participate:
sixteen were professional planners working for private and government organisations (hereafter
98
also referred to as planners), five were academic planners, i.e. university professors or
researchers, and fourteen were other actors involved in the planning process, for example,
planning organisation managers and technical specialists. Table 7.1 reports the participants; for
each type of participants, the number of males (M) and females (F) is indicated in brackets.
Table 7.1: Numbers and percentages of participants by job type and country
Professional planners Academic
planners Other actors Total per country Private organisations
Governmental organisations
N N N N N % Australia 3 (3 M) 5 (4 M, 1 F) 2 (2 M) 8 (4 M, 4 F) 18 (13 M, 5 F) 52 Italy 2 (2 M) 0 1 (1 F) 2 (2 M) 5 (4 M, 1 F) 14 Switzerland 5 (1 M, 4 F) 1 (1 M) 2 (2 M) 4 (2 M, 2 F) 12 (6 M, 6 F) 34 Total per job type
N 10 (6 M, 4 F) 6 (5 M, 1 F) 5 (4 M, 1 F) 14 (8 M, 6 F) 35 (23 M, 12 F) % 46 14 40 100
Out of the sixteen professional planners (46%), ten (3 in Australia, two in Italy and five
in Switzerland) worked in private planning organisations; some of them had a background in
related fields such as transport planning and urban design. The remaining six planners (five in
Australia, one in Switzerland) worked in governmental organisations, which comprised local
governments of different scale as well as organisations addressing planning at regional and
metropolitan scale.
Out of the five academic planners, two worked at The University of Melbourne in
Australia. One of them had significant experience in the United States but less in the Australian
planning system since he had only recently arrived in Australia. Two academic planners were
interviewed in Switzerland, at the Swiss Federal Institute of Technology (ETH) in Zurich and at
the University of Applied Sciences Rapperswil (HSR). Besides their academic positions, they were
also working part-time in private planning organisations. An academic planner was interviewed
at the Polytechnic of Bari in Italy; she had also been responsible for the environment and
planning division of the regional government of the Apulia region for ten years.
The fourteen other actors (eight in Australia, two in Italy and four in Switzerland)
included planning organisation managers, a scientific officer, an environmental engineer, a PhD
student, specialists in GIS, three-dimensional (3D) modelling and data management. Not all
actors worked in the same organisations as the planners. For instance, the scientific officer
worked for the Institute for Environment and Sustainability within the European Commission.
The engineer worked on environment and planning at the regional government of Apulia in Italy.
99
The PhD student was involved in a project supported by the Cooperative Research Centre for
Spatial Information that developed and applied a software tool for an Australian planning case
study. The GIS specialist had expertise in data processing for planning software and was part of
a project team in Australia, which was involved in the development of PSS.
7.2.3 The interviews
The interviews were semi-structured: they included a set of questions prepared in advance as
well as questions that emerged during the interview. This type of interview allows checking what
is already known but also provides the opportunity for learning new things. Often information
obtained from semi-structured interviews provides not just answers, but also reasons
motivating the answers [Preece et al. 2015].
After initial questions regarding job function, work experience and educational
background, the questions for professional planners focused on the adoption of software, their
strengths and weaknesses (see Appendix V). In addition, the planners were asked about their
procedures when choosing software and when needing assistance, as well as about software
courses that they attended during their study and/or working periods.
The fourteen other actors were interviewed, in order to gain a different perspective with
respect to planners, on the adoption of software in planning practice. The questions defined for
professional planners were used, but were formulated so that the interviewees would answer
by referring to the planners they work with (see Appendix W). For example, a question for
professional planners was ‘What planning task do you use the software for?’. This question was
presented to the other actors as ‘What planning tasks do planners use the software for?’.
The interviews with the five academic planners primarily aimed to provide indications
of software courses in planning curricula and trends in development and adoption of software
for planning (see Appendix X).
7.2.4 Interview execution and data coding
The interviews took place in the interviewees’ workplace. They were performed face-to-face and
one-on-one, with a few exceptions: in one case, the workplace of an interviewee was very
distant and therefore the interview was conducted via phone; in four cases, the interview was
conducted with two people, who worked in the same planning organisation.
The interviews were conducted in order, first in Australia, then Switzerland and lastly in
Italy. The international relevance of the study was additionally increased due to the fact that
100
some interviewees in Australia and Switzerland were not Australian or Swiss, respectively, but
from other origins; thus, they had different cultural backgrounds and international work
experience. The interviewer had to consider differences between countries in terms of planning
concepts, terms, policy, etc. to avoid interviewees becoming confused. For instance, the term
strategic planning was used in Australia and Italy but not in Switzerland where the interviewees
were familiar with the term spatial planning. Identifying these differences and whether they
influenced software adoption by planners was a further motivation for conducting the
interviews in three countries.
The interviews lasted on average twenty-eight minutes (excluding warm-up and cool-
off periods) and were audio-recorded with permission. A plain language statement explained
how information would be used and a consent form was signed by the interviewees.
The interviews were transcribed and analysed by the interviewer following the
grounded theory approach [Charmaz 2014] and the recommended steps for thematic synthesis
[Cruzes & Dybå 2011]. Specifically, the first step, i.e. extract data, was already started during the
transcription, and initial ideas and identification of possible patterns in the data were shaped.
As a second step, code data, the transcriptions were analysed, in order to code significant data.
As it is required in this kind of analysis, the codes were double-checked for reliability by another
researcher. Discrepancies were solved through discussion and consensus decision-making. This
was followed by the third step, translate codes into themes, in which the interviewer condensed
the codes into themes. The identified themes were refined in multiple iteration steps and
discussed with another researcher in a consensus meeting. Afterwards, a final set of themes was
condensed into an overview of the results reported in this chapter.
The analysis of the interviews performed in Australia and Switzerland was carried out
in Italy, where five more interviews were also conducted. The analysis indicated that the number
of interviews was enough to comply with “theoretical saturation” [Strauss & Corbin 1998]. This
concept suggests researchers carry on sampling until no new or relevant data seems to be
emerging. This was indeed the case in this research: the interviews performed in Italy did not
provide any new relevant findings, indicating that data was saturated. Thus, data gathering was
stopped.
7.3 The interview study: results
The results obtained by the analysis of the interview transcripts are reported, with respect to
the two goals.
101
7.3.1 Goal 1: Software used by planners
Goal 1 addressed analysing software tools used by professionals in their planning activities.
Additionally, the main functionality of the tools and the frequency of use by the planners, as
highlighted in the interviews, are reported.
7.3.1.1 Software tools
In the interviews, it emerged that professional planners use various software tools. Table 7.2
reports the name of the tools mentioned in the interviews. Because land use planning addresses
spatial problems only tools that process and visualise spatial data, were considered. Other tools,
such as word processors and spreadsheets, were excluded.
It is worth noting that the interviewer used the term PSS in her questions. However, in
the first interview, the professional planner had never heard this term and did not know its
meaning. This was consistent with all professional planners and with most of the other actors.
Even the Australian planners who reported that they use Online What if?, CommunityViz, which
are in fact PSS, did not call them PSS. Thus, in the interviews, the generic term software was
employed rather than PSS. Only the academic planners were familiar with PSS. This is a
significant finding which will be discussed later.
All interviewees knew Geographic Information Systems (GIS) and Computer-Aided
Design (CAD). GIS refer to software tools as presented in Section 2.4, i.e. for managing, analysing
and visualising geographical data. They offer a wide array of functionality applicable to geo-
referenced data ranging from performing analytics and spatial queries to spatial modelling.
CAD is software for creating, modifying and analysing technical drawings. It allows depicting
object shapes such as curves and figures in two-dimensional (2D) space as well as the overall
appearance of objects including surfaces and solids in 3D space. It is used by planners for
drawing plans at small scales; they found such drawings useful for detailed analyses but also for
documentation purposes.
3D modelling and visualisation software refers to systems for creating digital
representations of 3D objects or so-called 3D models. Some planners use them to provide
detailed realistic visual representations of urban environments through 3D modelling, also
based on augmented and virtual reality technology. The latter allows users to virtually immerge
in urban environments and to interact with them to improve their perception of reality.
102
Table 7.2: Software tools used by the planners, as indicated by the interviewees
Type Name
Geographic Information Systems (GIS)
ESRI ArcGIS Global Mapper Intergraph GeoMedia MapInfo QuantumGIS
Computer-Aided Design (CAD) software Autodesk AutoCAD Graphisoft ArchiCAD Vectorworks
3D modelling and visualisation software
Autodesk Infraworks Autodesk 3Ds Max ESRI City Engine Sketch up Urban Circus The Urban Engine
Graphics software
Adobe Illustrator Adobe InDesign Adobe Photoshop Microsoft Visio
Planning Support Systems (PSS) Online What if? Placeways CommunityViz
Graphics software is used for creating and manipulating graphical elements (e.g.
headings, backgrounds, images) and for page layout and formatting, for example, organising
elements (text and images) on a page, defining their style and colour. Graphics software is
primarily used to improve presentation and communication of information; thus, for finalising
products.
Planning Support Systems are intended as defined in Section 1.4. Hereafter, to refer to
the various software types used by planners for their activities, the generic term planning
software is used.
The most widely used software type, as resulting from the interviews, is GIS, which is
used by 96% of the planners (see Figure 7.1). 35% of the planners used CAD software. Its use
was higher in Switzerland (17%) than in Australia (9%). Different software tools for 3D modelling
and visualisation were used by 26% of the planners. 13% of the planners used graphics and PSS.
The latter was only used by interviewees in Australia.
The interviews highlighted that software functionality is not mutually exclusive, i. e.
software tools categorised as a specific type also perform functions that are more characteristic
of another software type as the following statement shows: “AutoCAD was more for engineering
drawings and now you can do a lot of GIS stuff”. Furthermore, software tools of the same type
can differ, for example, in terms of complexity, functionality, performance, as the following
103
statements demonstrate: “ArcGIS is really stable but really slow, MapInfo is unstable but a lot
faster” and “Vectorworks is probably less technical and it does not provide as many construction
possibilities as other CAD software, for example, AutoCAD”. Software performance is important
in relation to the planning stages 2 and 3 (see Section 2.1), i.e. problem exploration and
advanced analysis. In fact, in these stages planners might need to explore and analyse large
amounts of data, depending on the planning problem and the area under investigation. This can
substantially slow down the software and challenge its capacity.
Figure 7.1: Software type ordered by descending percentage of use
7.3.1.2 Software training courses at universities and other tertiary institutions
The interviews indicated that GIS and CAD systems are the types of software that are primarily
taught in university courses as well as in technical training courses provided by other institutions.
For example, the University of Applied Sciences Rapperswil in Switzerland provides courses on
ArcGIS (GIS) and Vectorworks (CAD) for the bachelor degree curriculum on planning. In contrast,
at the Faculty of Architecture, Building and Planning of The University of Melbourne, in addition
to a GIS course called ‘GIS in Planning, Design and Development’, a course called ‘Urban
Informatics’ is offered in some years and semesters; it provides an introduction to PSS, teaching
also how to use tools such as Online What if? [Pettit et al. 2015b] and the Australian Urban
Research Infrastructure Network (AURIN) portal; the latter supports planning and decision-
making across Australian Cities [Pettit et al. 2015a; Sinnott et al. 2014].
An academic planner of the ETH in Zurich said that in the master degree a specific course
was offered that required planning students to develop their own web-based software
prototypes; prerequisite for students to attend that course was to have GIS knowledge while
104
programming skills were acquired during the course. The interviewee said that generally at the
ETH courses would go beyond merely applying software and are more oriented towards doing
research on software that can be re-designed, preparing planning students for critical evaluation
and creation of software. The following statements illustrate the objectives: “We don’t want
that students get out of here as users, but as novice developers or as someone who can take part
in a discussion about software development”.
The interviewee of the Polytechnic of Bari in Italy reported that planning students
acquire much basic knowledge on maths, physics, etc.; they are also trained “To apply models,
maybe too many, and often without a critical attitude… nobody explains students the strengths,
weaknesses, opportunities and threats of applying them”.
7.3.1.3 Planners’ training in software use
Planners’ attendance of software training courses during their studies differed greatly. Some
interviewees indicated that the universities and other institutions they attended did not provide
any software training course as the following statements show: “when studying planning, I
haven’t really learnt to use any software. It would have been very useful if I had learnt how to
use GIS but I did not get any chance” and ”I used AutoCAD during my study but it was more auto-
didactic because there was no course”. In other cases, where software training courses were
provided, they were not always compulsory and sufficiently emphasised. As a consequence,
planning students were not always aware of the existence and importance of those courses. For
instance, an interviewee indicated that he had been one of very few among 30 fellow students
who had attended the GIS course offered in planning curricula.
Those interviewees who were exposed to software training during their studies
described the courses as rather inadequate preparation for software use in planning practice.
Examples of statements are: “I think that GIS training should go a step further and should be
taught interdisciplinary”, “The GIS lecture I attended at university in Great Britain was rather an
introduction and not very instructive” and “I think that with what I learned during the course of
informatics at the university in Venezuela I would not have been able to do what I’m doing
today”. However, an interviewee stated to have gained satisfactory knowledge during a CAD
course as reported in the following statement: “I got good Vectorworks training and knowledge
during 2 semesters at university”.
Some interviewees indicated that, due to lack of training, software use in planning
organisations is often inadequate as the following statements exemplify: “they do not use the
105
full potential of GIS and 3D modelling tools” and “unfortunately I used Modelbuilder (an ArcMap
application) only a few times and with moderate satisfaction because it also requires knowledge
in informatics such as dealing with databases”.
It appears that planners overcome some of their limitations in software use once they
are out of university as the following statement shows: “When I finished my degree, the first
thing I did was to go to the IT Department and work with computers and software and on data
processing”. Some planning organisations provide in-house, e.g. by GIS specialists, or external
software (in particular GIS) training for complementing planners’ self-learning with more formal
education. For instance, an interviewee decided to attend a GIS master course beside his job as
planner while another interviewee learned to use a PSS on her own because the management
of the planning organisation that she worked for did not allocate resources for formal training.
7.3.1.4 Functionality of planning software highlighted by the interviewees
The strengths of planning software appreciated by the interviewees are reported in Table 7.3.
GIS functionality for managing, georeferencing, geoprocessing and mapping spatial data was
seen by many interviewees as fundamental because it provides support to all planning stages
(see Section 2.1). However, several interviewees indicated that the support could be improved
in that the functionality was either too complex or not very specific for planning purposes. GIS
were also found to be not very intuitive, difficult to learn and to remember, as the following
statements show: “It is too technical for a casual user”, “What you think is very simple can be
really complicated”, “It is sometimes not intuitive”, “Maybe it is possible but we haven’t found
out yet how to do it” and “I often need some assistance”.
The strength of CAD tools was attributed to the many possibilities and techniques to
create line and polygon constructions of the built environment at small scale. Some planners
prefer doing the layout of plans with CAD tools due to their numerous presentation possibilities.
Examples of statements that reveal this are: “It is less difficult to do constructions in CAD than in
other software and there are plenty of techniques” and “Especially important for us are
georeferenced drawings”.
Some interviewees indicated that being able to see the third dimension of the built
environment, such as through 3D modelling and visualisation software, is useful for performing
specific analysis tasks (see planning stages 2 and 3 in Section 2.1), such as dwelling density
analyses, and for representation and communication of the built environment from different
perspectives. Particularly, for stakeholders who inspect planning proposals and who are less
106
familiar with 2D maps, 3D visualisations appear to be easier to understand. Statements that
exemplify this are: “If there is the intention to change the utilisation number of an area from 120
to 140, then it is also for the planner difficult to understand what this means in terms of height
or width by means of 2D visualisation”, “If it is about high-density construction as currently in
the city of Zurich, then it is easier to understand how the city is developing with 3D visualisation”
and “Some people understand 3D objects better than plans”.
Two interviewees indicated that planners and stakeholders value high-resolution
imagery and are generally impressed by the immersion effect provided by augmented and
virtual reality technology which allows an experience of planning proposals which is very close
to reality, as the following statements show: “Users are fascinated and report about great
experiences when using it” and “It provides planners with the opportunity to experience one to
one their planning proposals”.
Table 7.3: Strengths of the software types as evidenced in the interviews
Software type Strength
GIS - Functionality for managing, georeferencing, geoprocessing and mapping spatial data
CAD - Possibilities and techniques for creating constructions at small scale
- Presentation possibilities and layout of plans
3D modelling and visualisation software
- Visualisation of the third dimension - Different views of the built environment - High-resolution imagery - More real experience of planning proposal (immersion effect) - Easy communication amongst stakeholders
Graphics software - Appropriate drawing functionality - Appealing presentation and layout of plans
PSS
- Comprehensive analysis Further strengths of GIS-based tools: - Improved learnability for people familiar with GIS - Direct access to GIS capabilities - Increased standardisation
The opportunity to create appealing presentations and layouts of plans was claimed to
be the strength of graphics software and its provided drawing functionality; for example,
interviewees said: “You have more options, you can insert various textures and sort of Photoshop
elements” and “You can use it even without any knowledge and after only 1-2 hours of training”.
The strength of PSS is that it primarily allows planners to perform advanced analysis (see
planning stage 3 by Vonk et al. [2007a] in Section 2.1), like comprehensive forecasting tasks, by
including many land use factors and variables from different domains, as demonstrated by the
following statement: “When you talk about land use, you will have sewerage, terrain analysis,
107
transport analysis, you will have all these analyses involved”. Some PSS are GIS-based, i.e. they
constitute a module of a GIS. An example is CommunityViz [Walker & Daniels 2011].
Interviewees remarked the following strengths of GIS-based software:
(1) improved learnability (“I learned to use CommunityViz because it is an add-on of ArcGIS
and I have GIS knowledge”),
(2) direct access to GIS capabilities (“GIS capabilities are at hand and can be used in parallel,
for example, for data processing”),
(3) increased standardisation: (“GIS has done a great job in forcing everyone in using
shapefiles”).
Some weaknesses also emerged for PSS. Interviewees mentioned that it is not easy to
understand how such tools process data. Lack of transparency (i.e. “the extent to which the
underlying models and variables of the software tool are accessible and understandable to
users” [Pelzer 2015: 134]) and reliability (i.e. “the extent to which the outcomes of the tool are
perceived to be valid” [Pelzer 2015: 134]) was remarked, as the following statements show:
“Plan A is scoring 5.4 and plan B 4.9 you got no means what that means. Planners cannot see
any difference between them” and “It gives me an answer but I don’t know how the answer came
and I don’t really trust it”.
Low fit of PSS to planners’ tasks and needs also emerged. According to some
interviewees, an important factor for low PSS adoption is that PSS outputs do not sufficiently
provide information on social dynamics and possible implications of plan implementation on the
local community. This information is important for stages 5 and 6 of the planning process (see
Section 2.1), i.e. decision and implementation. In fact, prior to the decision of what plan to
choose for implementation, planners would like to know what effects implementation of
available plans would have on the local community. These results, which were discussed in the
focus group (see Section 7.4), emerged from the following statements: “At the end, we make
lots of choices about social dynamics that I don’t think that they have been accurately modelled”,
“These tools give answers but it is very hard to visualise what effect on people’s daily’s life the
answers will have” and “The implementation of plans is a very important step within the planning
process which is often ignored and which doesn’t get much support by PSS”.
Possible trends in planning software development and adoption also emerged that
mainly contribute to improving communication amongst planning actors. According to some
interviewees, a transition is occurring from standalone desktop to web-based tools (including
108
apps, blogs) that can be easily accessed. In addition, web applications are considered by some
interviewees the basis for multi-user collaborative software, in order to allow users,
independently of their location, to make real-time changes to the same file (e.g. a planning
proposal) and therefore to collaboratively develop a planning solution, as the following
statement shows: “The next step is, even if already sparingly happening, the collaboration
among different stakeholders through web applications”.
7.3.1.5 Frequency of software use by planners
The interviews indicated that the use of planning software has increased and improved over the
years. The frequency of software use varies a lot as demonstrated by the following statements:
“I use GeoMedia ever day”, “I’m using ArcGIS maybe not once a week but I’m going through
periods of time where I’m using it quite heavily”, “It can often go weeks or months without using
GeoMedia”.
The use of planning software appeared to differ substantially amongst the interviewed
planners. For instance, GIS use ranged from performing basic tasks such as visualising data (“A
lot of work we do in MapInfo is purely visual mapping”) to completing more advanced spatial
analysis tasks (“We check where possibilities exist that more population actually generate more
employees”). Furthermore, some planners indicated using CAD for visualising data while others
used it for creating complex constructions and 3D models.
The use of planning software also varies amongst planners within the same planning
organisation as the following statements show: “Four of the nine planners in our organisation
are able to actually create maps in ArcGIS” and “My colleague deals with 3D modelling and
rendering”.
It emerged that planners do not use only one but multiple planning software tools of
different types together for performing their activities, as the following statement shows: “We
are using these programs together, InDesign connected to ArcGIS and then we export the
basemap into Illustrator” and “What I do is I export the map from GIS into CAD, I do the
construction in CAD and then I re-export into GIS”.
As already remarked, the various types of planning software share some functionality.
For certain activities, planners might, over time, change the tools they use, as the following
statement shows: “While I did everything in ArcGIS in the past, now I export all bits and pieces
into Photoshop and put them together for plan-making”.
109
7.3.2 Goal 2: Factors affecting the adoption of planning software
Goal 2 addresses factors that affect the adoption of planning software. Both factors that are
related to the software systems, and others that are not strictly related to the software, are
reported.
7.3.2.1 System-related factors
An analysis of the factors that planners consider when choosing planning software was carried
out. Figure 7.2 shows these factors and the percentage of the interviewees who indicated them.
The factors were identified during the data analysis based on the grounded theory approach
(see Section 7.2.4). Where possible and in order to facilitate comparison of the results, the same
names as used in previous research [Vonk et al. 2007b; Vonk 2006; Vonk et al. 2005] were used.
This was, for example, the case for the factors fit to tasks and users and transparency. Other
factors were purposely differently named from previous literature, even if they meant the same.
For instance, relative advantage as in Vonk [2006], was called efficiency in this study because it
is acknowledged by the Human-Computer Interaction (HCI) discipline.
The results showed that 43% of the interviewees mentioned fit to tasks and users as a
factor for adoption of planning software. Statements related to this factor are: “We use the tool
because it reflects the planning process we adopt”, “We did not adopt AutoCAD because it was
too technical and complex for their requirements” and “For the plans we develop and the
functions we need, Vectorworks was the most suitable”. Moreover, when planners use software
that lacks some functionality, they ask developers for modifications, in order to better satisfy
their needs, as the following statement shows “Based on planners’ feedback, the button
‘transferring’ was added”.
Cost is an important factor for software adoption for 28% of the interviewees. This does
not only refer to the actual cost of the software but also to additional costs, such as training of
users. In particular, web-based software was valued for being easily accessible and generally not
requiring any installation. For example, an interviewee referring to a web-based PSS said: “OK,
let’s try this if it is not too much trouble and there is no cost involved”. However, it is somewhat
interesting that this interviewee expected ubiquitous software access at no cost.
About 20% of the interviewees reported that they adopt certain software because
planning organisations are committed to software compatibility. For instance, statements
demonstrating this are: “We adopted the same software as our client to facilitate the exchange
of data” and “Data transfer from one tool to another one is important for taking advantage of
110
different software functionalities”.
Another relevant factor for 17% of the interviewees is learnability, i.e. whether the
amount of time required for learning how to use it is acceptable. Some interviewees commented
that they adopt and use modules of software they already use because they have only to learn
Figure 7.2: System-related factors influencing the adoption of planning software. The values indicate the percentages of interviewees who mentioned the factors
the functionality of the module and not of the overall software. Other statements that remark
the importance of learnability are: “In extreme cases, if I cannot figure out in three clicks then
I’m not doing it” and “If you have got nine thousand things to do then it is very hard to learn to
use new software”.
Efficiency, i.e. time gain when completing a task, is an important factor for 11% of the
interviewees. For example, statements demonstrating this are: “Probably first is performance
and whether it makes projects quicker and more efficient” and “I don’t want to do everything
manually because there is not enough time for that basically”.
Transparency and reliability were further factors indicated by 9% of the interviewees.
These are previously mentioned in Table 7.3 as weaknesses of PSS.
The opinion of 9% of the interviewees is that visualisation capabilities are necessary for
adopting planning software as they are important for improving communication between
planners and stakeholders: “You might have done a lot of work in the background but if you
111
cannot communicate it then your work could go to waste”.
7.3.2.2 Non-system-related factors
Figure 7.3 shows factors that are not related to the functionality of software systems but
nonetheless have an impact on planners’ adoption of software. These non-system-related
factors were, where possible, named very similarly to that of previous research [Vonk 2006]. For
example, a factor in Vonk [2006] was awareness of potentials, while in this study awareness was
used. The latter was chosen because it includes not only planners’ awareness of software
potential but also and in particular of software existence influences whether a software tool is
adopted or not.
Planners’ awareness, i.e. knowledge on the existence and potential of planning software
tools, was indicated by 17% of the interviewees. The following statement: “There was no interest
to adopt other software also because I don’t know what is out there” exemplifies this finding.
Planners’ skills and experience have an influence for 17% of the interviewees, as the
following statements show: “I have intermediate and not advanced GIS skills and so I cannot do
all the GIS tasks by myself” and “So we solve the problem with CAD because there is someone
who knows how to do it, whereas nobody really knows QGIS”. In particular, some interviewees
indicated that the organisations they were working for adopted planning software because
some planners in the organisation had either learned or used it during their studies. For example,
they said: “We adopted Vectorworks because some of the planners, who had studied at the
University of Applied Sciences Rapperswil, were taught how to use it” and “Some of them use
sometimes a variety of CAD because quite a few of them have a background in architecture for
whatever reason, it must have to do with their degree or training”.
Some interviewees indicated that they are influenced by what others use, as shown by
the following statements: “We adopted Vectorworks because it is currently popular” and “Once
in the workforce, I noticed that all people were using AutoCAD so I did”. According to Vonk et al.
[2005], this factor is categorised as social organisation of users; it was indicated by 12% of the
interviewees.
Planning software has to take into account regulations, i.e. systems of rules imposed by
authorities that regulate planners’ activities, depending on the regions and countries in which
planners work. The following statements exemplify this: “Planning is about policy and so
rigorous methods are not important”, “They are not widely used as planning decisions are more
political than evidence-based” and “The reason for developing this tool is because we have to
112
Figure 7.3: Non-system-related factors affecting the adoption of planning software. The values indicate the percentages of interviewees who mentioned the factors
provide specific data, which is imposed by the Region”. This factor was mentioned by 9% of the
interviewees.
Small percentages of the interviewees indicated data availability and management
support as further factors. About data availability, 6% of the interviewees pointed out that
planning software designed to be applied in a specific area might require data that is not
available or difficult to get elsewhere. Examples of statements are: “The application of models
depends very much on the existence and availability of data”, “Data requirement is tailored to
data available in the UK as that is where the tool has been developed and designed for” and “If
you run the model for another country there will be lack of data and information and it will be
very hard to get an output”.
About management support, 6% of the interviewees reported that the management of
the organisation has to be convinced that it is a worthwhile investment, as the following
statement shows: “They want to know what do you gain and what’s the investment”.
In order to examine whether the interviews had been correctly interpreted and
reported, the results were reviewed in a focus group with planning actors documented as
follows.
7.4 Focus Group
Focus group is a technique can be used for multiple purposes given that people are gathered
together and debate on the same issue (see Section 2.11.2 and e.g. Choe et al. [2006]). In this
113
research, a focus group was undertaken in order to discuss the results of the interview study
with a group of planning actors. The objective of the discussion was to review the results and
gain some further insights into the current situation of the adoption of planning software in
practice and factors affecting it.
7.4.1 Participants
Six participants took part. Three of them had already participated in the interviews, i.e. an
academic planner who had also been responsible for the planning division of the Apulia region
and two professional planners; one working in a private and one in a governmental organisation.
The other three participants had not been involved in the interview study. They worked at the
planning department of the Polytechnic of Bari in Italy; specifically, one was an assistant
professor, one was a research fellow, and the third was a PhD student with work experience in
a governmental organisation.
7.4.2 Procedure and data collection
The focus group was held at the Department of Computer Science of the University of Bari in
Italy and lasted ninety minutes. As shown in Figure 7.4, the participants were sitting at a table.
Each participant was provided with a tag including his/her name and affiliation. The participants
signed a consent form that allowed taking photos and recording the audio during the session.
After that, a short presentation was given that informed the participants about the interview
study, its purpose as well as the outcomes. These were subsequently discussed with the
participants. Most of the time, an open discussion was held in order to gather as much
information and feedback as possible. However, the participants were also asked whether and
to what extent the planning process and stages as defined by Vonk et al. [2007] (see Section 2.1)
reflect planners’ activities in real life. The reason for this question was to ensure that all
participants would have the same understanding and would refer to the same planning process
during the discussion. All participants actively participated in the discussion. The results are
reported in the next subsection.
7.4.3 Results
The participants agreed that the planning process and stages as defined by Vonk et al. [2007]
cover planners’ activities in real life. They added that the process should ideally be iterative. I.e.
114
Figure 7.4: The participants during the focus group
after the seventh stage, the first and the subsequent stages should be performed again and
again. This is because conditions and requirements change over time.
The participants stated that, to the best of their knowledge, GIS is the software type
most used by themselves and by their colleagues. In addition, they use it for performing all seven
planning stages (see Section 2.1). A reason for the supremacy of GIS provided by participants
was that the Apulia region, where they mostly operate, requires planning organisations to use
GIS. This result confirmed that authorities can affect planners’ software adoption, as also
outlined in the non-system-related factor regulations in the interview study results.
With reference to the system-related factors, fit to tasks and users and visualisation
capabilities revealed to be important. Regarding the former factor, the focus group confirmed
the results of the interview study in that PSS are deficient in adequately supporting planners in
making decisions about what plan to choose. For achieving effective plan choice, planners need
to understand how specific plans, once they are implemented, might influence people’s
experiences and activities. This is important as the focus group participants reported that,
nowadays, planners are constantly dealing with problems that citizens experience, either
through data provided by citizens via applications, such as SeeClickFix, Ushahidi, or social media,
for example, Twitter, Facebook, Instagram, or through direct notification by citizens who
complain about issues such as footpaths that are too narrow, overhanging tree branches,
garbage bins blocking passageways, transportation and criminality issues. These (daily) issues
and their impact on citizens’ life get little consideration in PSS and their outputs. This is why,
according to the participants, PSS outputs do not provide adequate support to planners for plan
115
choice and implementation.
As in the interview study, the participants voiced their willingness to adapt software
tools to specific tasks. However, they complained that making modifications to software tools
requires programming skills that planners do not have.
What emerged in the interviews about visualisation capabilities was confirmed, with
strong views that lack of appropriate visualisation makes it difficult for planners to communicate
their planning intention to stakeholders. The participants provided the example of surveyors and
decision-makers who, according to them, are used to hard copies and have difficulty in using
digital outputs, in making even small changes and updates.
7.5 Discussion
This study, conducted to analyse the current situation of the adoption of PSS in planning practice
showed, very surprisingly, that professional planners do not know the term PSS and, in most
cases, do not use those tools. This finding was reinforced by the results of the focus group. It
was only in Australia that some interviewees said that they used the tools categorised as PSS in
Table 7.2, however, they identified them generically as software for planning, rather than PSS.
The interviewees pointed out that planners use multiple software types and tools (i.e.
GIS, CAD, 3D modelling and visualisation software, graphics software), in order to benefit from
the diverse functionality they provide. In fact, most of these tools have not specifically been
designed for planning and therefore do not perfectly fit planners’ expectations and their tasks.
GIS is, due to its large set of functionality, the most used type of tool, as emerged from both the
interviews and the focus group.
The discussion on the factors affecting the adoption of planning software in practice is
divided into two parts: Section 7.5.1 reports recommendations specific for the planning
discipline, while Section 7.5.2 provides recommendations for the design and development of
planning software.
7.5.1 Recommendations for the planning discipline
As noted above, results indicated GIS as the most used type of tool. Professional planners in
Switzerland said that they equally choose GIS and CAD systems as their basic software tools. It
is of interest to observe that, according to the interviewees, the planning software most used in
university courses are GIS, and also CAD systems in Switzerland. This could explain why these
software types are the most used in planning practice: planners are inclined to use software
116
tools they already know from their university studies. Consequently, an important
recommendation resulting from this study is that university curricula should include PSS
training. This would produce several advantages. As reported in Brits et al. [2014: 5], “planners
in general do not understand models and modelling process”. Models are an important
component of PSS (see Section 2.5) thus, studying PSS at university will increase students’
modelling abilities. Houghton et al. [2014] analysed the importance of making ICT part of
professional planners’ daily practice, in order to foster innovation in planning activities. To this
aim, technical and software courses should be offered more extensively to planning students.
Edwards and Bates [2011] analysed planning’s core curriculum within master’s degree programs
of thirty planning schools in USA and Canada. They showed that little attention is devoted to GIS
and spatial analysis in comparison with other skills. This research agrees with Dawkins [2016] on
the benefits of providing a solid basic knowledge, but emphasises that students have also to be
informed about ICT advances that can have a positive impact on their future jobs. For example,
planners, once in the workplace, could attend online courses offered through MOOC
(https://www.edx.org/course, accessed on January 15, 2017) and Planetizen
(https://courses.planetizen.com/, accessed on January 15, 2017), in order to get a more in-depth
knowledge of ICT.
Planning skills were analysed in Greenlee et al. [2015], where both practitioners and
educators were asked to rank skill areas. Data analysis and visual communication skills were
both ranked very high. By data analysis they mean “the ability to use data and compute
numerical summaries of information, creation of projections, forecasts, and scenarios”
[Greenlee et al. 2015: 166]. Since by definition PSS provide support to such advanced data
analysis (see Section 2.5), knowledge on PSS is thus very important. Visual communication is
defined as “the ability to visualize complex information using GIS maps, tables, charts, and other
illustrations” [Greenlee et al. 2015: 166]. PSS should actually provide visualisations like maps,
tables, charts, etc. Summarising, the findings from this study indicate that it is essential to train
planning students to work with PSS so that they are prepared to adopt and use them in their
work practices.
Out of the six non-system-related factors influencing the adoption of planning software,
reported in Figure 7.3, five were already identified in the studies by Vonk and co-authors [Vonk
et al. 2007b; Vonk 2006; Vonk et al. 2005]. Among these, low awareness and insufficient skills
and experience of planners resulted as the most important.
The further factor that resulted from both the interviews and the focus group is
117
regulations. Its influence on the adoption of planning software was also identified in Brits et al.
[2014] and emerged in Russo et al. [2017]. The latter reported how during a user test (see also
Chapter 6) a participant was reluctant to enter data into the Online What if? PSS because she
was not sure whether it conformed to current planning regulations. Specifically, she refused to
define ‘current land uses to be converted into residential land use’ because she did not know
current rules on this issue. Software developers should take this factor into serious consideration
and work towards developing PSS that should be easily adapted to the planning regulations of a
specific area.
7.5.2 Recommendations for the design and development of planning software
With reference to the system-related factors hampering the adoption of planning software, the
results of this study confirmed to some extent what other authors reported in the literature, but
two new factors, namely learnability and visualisation capabilities, emerged. The other five
system-related factors shown in Figure 7.2 were already reported in previous articles [Vonk et
al. 2007b; Vonk 2006; Vonk et al. 2005]. Fit to tasks and users is by far the most important
system-related factor influencing the adoption of planning software. However, the interviews
showed that planners have different needs, influenced by different national policies and the
individual’s background; thus, they have different expectations of what planning software
should provide.
It is well known that user needs evolve over time [Fogli & Piccinno 2013; Fischer et al.
2008], and so too do planning conditions and requirements, as was pointed out in the focus
group. Thus, planners ask for software tools that can be easily tailored to their needs, tasks,
habits and the planning context. Existing planning software is not flexible and does not enable
planners to find answers to their specific and evolving requirements. Currently, as emerged both
in the interviews and the focus group, modifications to planning software are performed (e.g.
new functionality is implemented) by technicians, in order to adapt the software to what
planners wish to do. This is a costly procedure. To avoid this issue, planning software should be
developed according to the End-User Development (EUD) approach [Diaz et al. 2015; Cabitza et
al. 2014a, 2014b; Ardito et al. 2012]. EUD refers to the concept that non-technical end users, i.e.
people who are not programmers, are enabled to develop or modify software artefacts, in order
to adapt them to their own needs and preferences. Indeed, recent technology offers end users
the opportunity to perform various activities ranging from simple parameter customisation to
more complex activities, such as variation and assemblage of components [Ardito et al. 2012;
118
Lieberman et al. 2006]. EUD is generating considerable attention amongst researchers,
representing a great challenge to software design. It is not an easy task to create environments
and tools that empower end users to tailor software they use, without being obliged to become
programmers.
In line with Vonk [2006], further but less important system-related factors were costs
(of software, user training, etc.), software compatibility, efficiency, and transparency and
reliability. In particular, transparency and reliability were mentioned only for PSS. Some
interviewees stressed that planners are and should be even more critical of the underlying
assumptions and methods used by PSS in their analyses: they want to be informed by the system
on how data are analysed, models are created and outputs are generated, in order to better
understand the obtained results. This also emerged in the study reported in Russo et al. [2017].
This research strongly recommends taking into account transparency and reliability as important
requirements of future PSS.
A further recommendation for developers, emerging both in the interviews and the
focus group, is that PSS should provide planners with information on how the proposed output,
once implemented, impacts on social dynamics and people daily’s life. PSS developers should
take data provided by citizens through applications, such as SeeClickFix, Ushahidi, or social
media, for example, Twitter, Facebook, Instagram, as examples of information that planners
expect for achieving more effective plan choice and implementation.
The new factors emerging from this study are learnability and visualisation capabilities.
According to Nielsen [1993], learnability is one of the five dimensions of usability (see Section
2.8). Easy to use, which is another usability dimension, considerably affects efficiency, which was
mentioned as an important factor in a previous study [Vonk 2006]. The interviewees explicitly
mentioned learnability, possibly because there is increasing awareness amongst users that
software has to be usable, no matter how complex it might be.
As reported in Chapter 2, the concept of usability has been evolving in the last years.
The emphasis is now on User eXperience (UX), which includes subjective attributes such as
aesthetic, emotions, and social involvement. Until recently, a primary goal of software design
was to provide useful and usable functionality to allow people to perform their tasks. These
goals are still important but it is an objective to make sure that the tools people use are
pleasurable as well, that they generate positive feelings and well-being. To the designers of
future planning software, it is recommended to take into account usability and develop software
that, beyond being easy to learn and use, also considers the attributes related to the overall UX.
119
Regarding visualisation capabilities, both the interviews and the focus group indicated
that lack of appropriate visualisations makes communication between planners and
stakeholders more difficult, despite it being an important activity in planning practice. In order
to increase planners’ acceptance of PSS and facilitate communication with stakeholders, PSS
should provide mechanisms for dynamically linking maps to other visualisations, such as graphs,
charts and tables, i.e. multiple visualisations of data should be available. Previous articles [Vonk
et al. 2007b; Vonk 2006; Vonk et al. 2005] have not identified visualisation capabilities as a factor
influencing PSS adoption by planners. However, the strengths of visualisation capabilities and in
particular of coordinated multiple views were previously documented in Pettit et al. [2012].
Some interviewees, in particular software developers, reported that, in general, PSS are
developed without a proper software development methodology: user requirement analysis is
very poor and system functionality is implemented without considering what planners actually
need. Once planners use the system and complain about system functionality, developers
intervene in order to fix the identified problems. Scant attention is devoted to the user interface
which is often considered only at the very end of the system development, despite Raskin’s
[2000] observations: creating an interface is much like building a house: if you do not get the
foundations right, no amount of decorating can fix the resulting structure.
As emphasised in the HCI literature, and also in Russo et al. [2017] with reference to PSS,
it is recommended to follow the Human-Centred Design (HCD) process [ISO 9241 2010b], in
order to create systems that can meet users’ requirements. It prescribes focusing on users, their
tasks and the context in which they operate and the adoption of an iterative approach of
developing system prototypes of increasing complexity, which have to be carefully evaluated, in
order to check if they comply with user and context requirements. These basic HCD principles
and methods may sound a naïve recommendation; however, it is an important one, as the
literature (e.g. Ardito et al. [2014], Bak et al. [2008], Ji & Yu [2006], Vredenburg et al. [2002],
Rosenbaum et al. [2000]) showed that one obstacle to the creation of systems meeting their
users’ expectations is that too many software developers continue to either neglect or not
properly address HCD principles and methods.
7.6 Conclusion
The study reported in this chapter was performed in order to investigate why PSS are not widely
used in work practice and to identify relevant functionality that PSS should provide for meeting
planners’ expectations. The study consisted of 35 interviews with planning actors from three
120
countries and a follow-up focus group involving six Italian planners.
The first surprising result of the study is that only researchers in the field use the term
PSS. Most professional planners do not know this term and do not use these tools. In order to
be supported in their work, they use different types of software, including GIS and CAD. With
respect to further study results, factors that affect the adoption of planning software in practice
were indicated; some of them are related to the functionality of software systems, others are
not. Moreover, some were also identified by previous studies. Relevant non-system-related
factors are awareness, i.e. knowledge on the existence and potential of planning software tools,
skills and experience on planning software tools. Certainly, it emerged that planners are inclined
to adopt and use software tools if they have already learned, used and understood them during
their studies. Thus, a key recommendation, which was another significant outcome of this study,
is that planning in university curricula should include PSS training, also emphasising the
importance of making ICT part of planners’ daily work, in order to foster innovation in planning
practice.
For the interviewees, the most important system-related factor was fit to tasks and
users, which was already indicated in previous studies [Vonk 2006]. The new factors learnability
and visualisation capabilities emerged which should be the subject of further studies.
Learnability is an important dimension of system usability. Although PSS have been developed
for about thirty years, their usability and adoption is still very poor. This study provided
further evidence that, despite a large amount of research has been performed, there is a gap
between HCI principles and methods proposed in academia and what is actually adopted in work
practice. This showed that the HCI research community is still failing in transferring relevant
knowledge developed over the years to practitioners. Based on these results, recommendations
have been provided for the design of PSS that better support planners in their work practice.
In an era of digitisation, smart cities and smart devices, people are more attuned to all
aspects of usability. In order to include usability, and even UX attributes, as system quality
factors, the most important recommendation is to develop PSS following the HCD process [ISO
9241 2010b], which prescribes focusing on user aspects and evaluating prototypes of the user
interface iteratively. Another recommendation is to create PSS as flexible systems that can be
easily adapted by users themselves, at use time, to their diverse and evolving requirements. To
this aim, PSS developers should consider methodologies and techniques proposed by the
research on EUD [Diaz et al. 2015; Cabitza et al. 2014a; 2014b; Ardito et al. 2012]. These
recommendations were also provided in Chapter 6 and Russo et al. [2018].
121
To conclude it is worth remarking that the results presented in this chapter are based
on a qualitative study. To ensure study objectivity and soundness, the collected data was
carefully analysed according to content analysis, which allows identifying, analysing and
reporting themes in a systematic way [Kim et al. 2014]. In addition, the interpretative power of
content analysis was increased by the use of the grounded theory theoretical framework.
The study involved thirty-five interviewees. This number can be considered adequate
with reference to the theoretical saturation concept [Strauss & Corbin 1998] (see also Section
7.2.4); however, it is too small for permitting a broad generalisation. The study provided insights
into the PSS adoption process, factors influencing it and the development of more usable PSS.
Multi-national commonalities and differences in planning education and practice were also
identified. For analysing if this information is significant and can be generalised, further work is
needed that focuses on a multi-national comparison with targeted research questions and
methodology. There is confidence that this study will stimulate such further work for examining
more in depth the insights gained in this chapter, in order to foster improved usability and
widespread adoption of PSS.
122
8. Discussion and conclusion
This thesis focused on investigating the usability and adoption of Planning Support Systems
(PSS). Chapters 1-3 provided the introduction, the research background and the design of this
research. The novel contribution of this thesis is articulated across Chapters 4-7. In each of these
chapters, the contributions and results are discussed in relation to PSS adoption. This chapter
provides a summary of these results and related discussions and directly addresses how this
work answered the original research questions (RQs). This chapter concludes with final remarks
and recommended future work.
8.1 Contributions and key results
This thesis made several contributions to the research on PSS and supporting their adoption by
planners. A key practical contribution of this thesis is the PSS resource described in Chapter 4
and available online (http://docs.aurin.org.au/projects/planning-support-systems/, accessed on
July 17, 2016). This repository comprises i) PSS developed and applied internationally and ii)
practical and technical information on a comprehensive number of the available PSS. Methods
such as literature search and a survey with the developers of the PSS were adopted to gather
the information. The survey showed that PSS are in general not easy to access and not well
publicised by developers. This certainly does not help to improve planners’ low awareness of
PSS existence and potential which has been indicated as a major barrier to PSS adoption in
previous literature [Vonk et al. 2005]. Hence, this repository indirectly contributes to improving
PSS adoption by providing a platform that i) enables developers to publish PSS they create, and
ii) communicates to planners the available PSS and supports them in the selection of appropriate
PSS through the provision of key metrics.
This thesis supported getting more standardisation of PSS usability evaluations by
providing some knowledge and practice of Human-Computer Interaction (HCI) research.
Nielsen’s model (see Figure 2.2) was reported in order to present the original concept of usability
and to clarify terms such as usefulness, usability and utility, which are sometimes confused in
the literature on PSS. Another significant contribution in this regard is the PSS Evaluation
Framework described in Chapter 5. It provides researchers and PSS developers with guidelines
and activities to take into account when planning and performing PSS evaluation. It is certainly
not an easy task as demonstrated by the literature in HCI research [Ardito et al. 2011; Costabile
2001] and by Pelzer et al. [2016] who reported on difficulties in performing PSS evaluations due
to the many issues that have to be considered.
123
The framework not only supports novice evaluators but fosters the practice of making
evaluations a standard work process in PSS development through proposing the use of cost-
effective evaluation methods acknowledged by HCI research. In fact, evaluation is a central
activity in the software design process in that it assists in identifying usability problems and in
developing a product that satisfies users’ requirements [Costabile 2001]. Low usability of PSS
and their poor fit to planners’ requirements have been identified as a major barrier to PSS
adoption [Vonk 2006; Vonk et al. 2005]. The proposed PSS Evaluation Framework is a
contribution for encouraging PSS developers to perform evaluations and so to improve the
usability of PSS they create.
Rigorous PSS evaluations are rare as shown in the literature [Allen 2008] and as part of
the findings of Chapter 4. In view of that, this research conducted a thorough evaluation of three
PSS; CommunityViz, Envision and Online What if?, identified in the Online PSS Resource reported
in Chapter 4. This evaluation study, described in detail in Chapter 6, was designed following the
PSS Evaluation Framework and focused on six planners participating in testing the PSS
individually. The participants were observed and, as part of the adopted thinking-aloud method,
they were asked to speak out loud during the interaction with the PSS. After the interaction with
each PSS, they were asked to complete a questionnaire in which positive and negative aspects
of the PSS, as well as information on their experience, were gathered. The analysis of their
answers and comments gave the following results, which developers should take into account
for designing more usable PSS:
(1) usability issues and User eXperiences (UX) need to be considered (see RQ 1 in Section
8.2),
(2) system functionality does not satisfactorily match planners’ expectations (see RQ 2 in
Section 8.2).
This mismatch was also identified in previous literature and indicated as a barrier to PSS
adoption [Vonk 2006]. To mitigate this mismatch and create more usable PSS, recommendations
for developers based on HCI research have been developed and are provided in RQ 3 in Section
8.2.
Chapter 7 describes interviews conducted with thirty people from planning practice, for
example, planners, specialists in GIS, modelling and data management, from three countries;
Australia, Italy and Switzerland, in which planners’ adoption of software and factors influencing
it were investigated. Interviews were also conducted with five academic planners who provided
124
information about planning education, in order to understand what type of software training, if
any, planning students receive in university or other tertiary institution curricula.
The interviews showed that little has changed in the adoption of PSS in planning
practice. In fact, as Vonk [2006] published more than a decade ago, this research identified that
professional planners are still not commonly aware of the term PSS, do not know such software
tools and rarely use them. In particular, this study showed that they use other software tools for
performing activities that require processing and visualising spatial data. GIS was by far the most
used software, followed by CAD, three-dimensional (3D) modelling and visualisation software,
and computer graphics software. By using multiple software, planners benefit from their
different functionalities and strengths, yet there is not a core piece of software which specifically
meets the needs of planners, unlike, for example, architects who commonly use CAD to perform
core design tasks. Based on the planners’ answers on what software they use and why, some
expectations they have of system functionality were identified. These are provided under RQ 2
in Section 8.2.
The interview study confirmed past research [Vonk et al. 2007b; Vonk 2006; Vonk et al.
2005] regarding factors that influence software adoption by planners. The most important
factors were software fit to tasks and users, i.e. the extent to which software functionality
complies with planners’ tasks and expectations, awareness, i.e. planners’ knowledge of the
existence and capabilities of a software tool, and skills and experience of a software tool, i.e.
planners’ ability to use a software tool. That planners’ skills and experience influence what
software they adopt has also been shown in that GIS and CAD, i.e. the most used software by
the planners in the interview study, are also those that, according to the interviewed academic
planners, are primarily taught at universities and other tertiary institutions. This is a further
indication that planners tend to use tools they got to know during their studies.
Some new factors influencing planners’ adoption of software were also identified. These
are: learnability, i.e. whether the amount of time required for learning how to use a software
tool is acceptable, regulations, i.e. systems of rules imposed by authorities that regulate
planners’ activities, and visualisation capabilities, i.e. techniques provided by a software tool for
representing data graphically. Based on the identified factors, recommendations for improving
PSS usability and adoption were defined which are provided under RQ 3 in Section 8.2.
125
8.1.1 Theoretical contribution
The results of this research are discussed in relation to the theory reported in Section 3.2, which
involved factors influencing the PSS adoption process and factors affecting UX.
8.1.1.1 Factors influencing the PSS adoption process
This research identified both system-related and non-system-related factors influencing PSS
adoption as in Vonk’s et al. [2005] framework (see Figure 3.1).
Among the non-system-related factors, some were of personal (e.g. skills and
experiences) and some of organisational (e.g. management support) nature. Based on this, this
research claims that PSS adoption can start at both individual and organisational level as
indicated in Vonk’s et al. [2005] framework. Non-system-related factors, as presented in Vonk’s
et al. [2005] framework, were identified in this research. For instance, awareness has been
shown to be low and therefore an important barrier to PSS adoption. Furthermore, the factor
skills and experiences can be associated with adopter characteristics, social organisation with
social influences and management support with persuasion influences.
Seven system-related factors were mentioned by the interviewees. In particular,
attributes that Vonk et al. [2005] used to specify ease of use and usefulness were identified in
this research. Complexity emerged in the evaluation study as participants defined some PSS
functionality as complex. Hardware issues were addressed with discussing planners’ expectation
of GIS- and web-based PSS. Data issues were discussed in relation to the problem of data
availability that interviewees reported. Furthermore, the term relative advantage was
substituted with efficiency in this research because the latter is acknowledged in Nielsen’s [1993]
definition of usability (see Section 2.8). Reliability describes what in Vonk’s et al. [2005]
framework is called uncertainty.
This research also identified several new factors not specified in Vonk’s et al. [2005]
framework, i.e. learnability, regulations, and visualisation capabilities. It also emerged that these
factors influenced planners’ expectations of functionality and therefore whether they saw
software functionality as usable. This confirms that system-related factors are affected by non-
system-related factors as indicated by Vonk’s et al. [2005] framework.
8.1.1.2 Factors affecting UX
In relation to the three factors influencing UX as described by Roto et al. [2011], i.e. context,
system, user, the following observations were made in this research. The three factors influence
126
i) each other, ii) planners’ UX with software and iii) planners’ software adoption process.
For instance, it was observed that technical specialists within planning organisations,
who do not use planning software themselves and do represent the factor context, can affect
the factor system. This emerged as some technical specialists noted in the interviews that they
make changes to software systems for planners in the organisation. More precisely, they
adapted software functionality to planners’ requirements.
Usability problems of PSS identified in the evaluation study resulted in a negative UX
(see RQ 1 in Section 8.2). In particular, it appeared that good PSS functionality and capabilities
were overshadowed by negative UX. Most participants wanted the user test to finish quickly
because they were frustrated and anxious that they would not perform well. These observations
are indications that frustration and negative UX have negative influences on adoption of PSS and
that the latter will remain relatively low as long as users connote frustration and negative
experiences with PSS. Many of the identified usability problems (e.g. inadequate terminology,
cumbersome interaction mechanism) were due to not planning enough resources for PSS
development and not involving users in PSS development and evaluation as recommended by
the Human-Centred Design (HCD) process (see Figure 2.4).
Finally, the interview study (see Figure 7.3) showed that different aspects that can be
attributed to the factor context (e.g. social organisation, education in planning curricula,
regulations, data availability, management support) affect software adoption in planning
practice.
8.2 Findings: answers to the research questions
The three research questions (RQs) of this thesis are discussed as follows:
RQ 1: What are identified usability problems and UX when interacting with PSS?
Six main usability problems were identified in the evaluation study:
(1) unclear terminology and layout elements,
(2) inappropriate and/or lack of interaction mechanism,
(3) lack of guidance,
(4) low learnability,
(5) lack of transparency and reliability,
(6) mismatch between functionality and planners’ expectations.
Overall, the participants, consisting of six planners, had difficulties interacting with the PSS. They
127
experienced difficulty in executing a correct sequence of actions. The fact that users could not
“rapidly start getting some work done” indicates according to Nielsen [1993: 26], that the PSS
were not easy to learn. Unclear layout elements and terminology were the reasons why
participants indicated not knowing how to proceed. Thereupon, participants reacted differently;
some got frustrated and clicked randomly on interface objects, others were anxious about
performing incorrect actions and asked the facilitator for assistance or for confirmation prior to
execution of an action.
Participants found some actions tedious and when having to deal with these they did
not show much perseverance. For instance, if an action required utilising a user interface
element that was cumbersome to use, participants did not complete the action. Similarly,
participants did not use the help documentation provided by one of the three PSS because it
contained too much text. This is in line with previous literature which established that people
spend limited time on help documentation [Lim & Dey 2009].
A mismatch between functionality as provided by the PSS and how participants
expected it, was observed to cause uncertainty and confusion amongst the participants. Indeed,
the participants sometimes stopped interacting with the PSS to express their doubts about the
provided functionality and to describe how they usually carry out the task and what their usual
procedures are. Furthermore, the terms used in the user interface such as ‘New Presrv.’ or
‘Development efficiency’ confused participants. These were inappropriate, non-technical or just
not what the participants were used to. As a consequence, some participants began to guess
possible meanings, in part because, according to some participants different terms are
sometimes used, depending on locations, for referring to the same planning issue.
Participants were further dissatisfied with not understanding the operations and their
effects. This is a clear indication of lack of transparency which is caused by not providing the
user with enough explanations and feedback on what the system does [Lim & Dey 2009; Lim et
al. 2009]. A consequence of lack of transparency is lack of reliability. In other words, the
participants did not trust the outputs because they did not know how they were generated. This
problem is related to PSS being referred to as black boxes and as a major obstacle to PSS
adoption as reported in previous literature [Lee 1973; 1994; Klosterman 2013] and in Section
2.7.
Low reliability was also identified, as one of the three PSS sometimes stopped working
while participants were interacting with it and had to be restarted. This breakdown was quite
significant because data input up to that point was not always saved by the system and
128
participants did not know how to recover without help from the facilitator [Nielsen 1993].
Indeed, a data set previously prepared by the facilitator had to be used to continue with the
evaluation. This usability problem is a clear indication that the particular PSS lacked stability and
maturity at the time of the evaluation study.
RQ 2: What are planners’ expectations of system functionality?
In total, twenty-one planners’ expectations of system functionality were identified in the
evaluation and interview study (see Table 8.1): taking them into account in PSS design and
development could assist developers in creating more usable PSS. The expectations in Table 8.1
have been ordered as follows. First, what is needed to set up and adopt PSS was listed (i.e. help
documentation and data requirement). After that, desired visualisation and interaction tools
(i.e. from contextual data to feedback techniques) were mentioned. This is followed by
functionality expected by the planners for performing analysis (i.e. spatial and multi-disciplinary)
and creating outputs (i.e. drawings and plans). Then, expectations were listed which show
capabilities that PSS should have (i.e. from transparency of operations to compatibility with
other software). Depending on the required capabilities, PSS implemented as GIS module or web
application can be more suitable as discussed later in this section. The expectation table
concludes with what is a general expectation of products, i.e. that their cost is minimal. The
expectations were either explicitly or implicitly suggested through planners’ statements and
answers. As shown in Table 8.1, not all expectations emerged in both studies. This is because
the studies had different designs and objectives. In the evaluation study, participants were using
PSS and so more detailed functionality that was expected when interacting with PSS emerged.
On the other hand, expectations not directly related to the interaction were discussed in the
interviews.
High data requirement of PSS, or data hungriness as it is referred to in Lee [1994; 1973],
is a known obstacle to PSS adoption in the literature [Brits et al. 2014; Klosterman 2013; Waddell
2010; Timmermans 2003]. This emerged also in this thesis, where the interviewees called for
minimal data requirement and preparation. Data preparation was not part of the user test, i.e.
participants did not have to enter data in the databases of the three PSS. Similarly, participants
were not faced with cost issues during the user test, while some interviewees stated high costs
as one of the main obstacles to planning software adoption.
Participants expressed the need for having contextual spatial data (e.g. economic, demographic
indicators) of the study area, in order to gain an overview of the conditions before performing
129
more in-depth analysis. Specifically, planners called for PSS that allow performing a
comprehensive and multi-disciplinary analysis. Indeed, land use planning is a complex field just
because it has to take into account considerations from different disciplines [Bonner 2002].
Particularly, mobility is a key factor which is why many integrated land use-transportation PSS
are available [Geertman 2016; Waddell 2010; Timmermans 2003]. It is worth noting, that system
developers might face a trade-off in the attempt to implement PSS that allow comprehensive
analysis. In fact, comprehensive analysis likely implies tool complexity and heavy data
requirements which create barriers to adoption for planners.
Table 8.1: Planners’ expectations of PSS functionality emerged in the user test and/or interviews
The interviewees expected PSS to provide information on how the proposed output,
once implemented, impacts on social dynamics and people daily’s life. This intervention is critical
for supporting planners in plan choice and implementation as well as for increasing PSS
acceptance.
Planners are inclined to visual representation and exploration of data. They expect an
Expectation Evaluation study
Interview study
Minimal data requirement and preparation ✔ Help documentation ✔ ✔ Clear layout elements and interaction mechanisms (e.g. ‘back’ and ‘next’ buttons) ✔ ✔
Contextual spatial data of study area ✔ Map display ✔ ✔ High-resolution imagery ✔ Data and information visualisation (e.g. graph, 3D visualisation, visual-information-seeking mantra) ✔ ✔
Feedback techniques (e.g. moving over, brushing) ✔ Georeferencing and -processing functionality ✔ ✔ Viewing and creating site plan drawings ✔ Creating appealing plans ✔ Comprehensive, multi-disciplinary analysis (including transportation) ✔ ✔
Transparency of operations and workflow process ✔ ✔ Information on effects of plan implementation on social dynamics ✔ Diverse output formats (pdf, spreadsheet) ✔ Adaptability and customisation of PSS ✔ ✔ Efficiency and stability of operations ✔ ✔ GIS module ✔ ✔ Web application ✔ ✔ Data interoperability and software compatibility ✔ Minimal cost ✔
130
intuitive map display and high-resolution imagery. Moreover, for analysis purposes, the
interviewees expressed the desire to have functionality for viewing and creating site plan
drawings as can be found in CAD and graphics software. The interviewees emphasised the
importance of providing stakeholders with high-quality plans. Thus, they stated their
expectation of PSS to provide appropriate functionality for creating appealing plans. Planners
also need to provide decision- and policy-makers with data and information visualisations, such
as graphs, charts and tables, as well as textual outputs, such as spreadsheets, because they are
generally not very familiar with maps.
PSS developers should evaluate whether to implement GIS-based or web-based PSS.
Both have their strengths as remarked by the planners in the evaluation and interview study.
PSS implemented as a GIS module or a GIS-based PSS provide access to the GIS platform. The
latter comprises a rich set of geoprocessing and spatial analysis functionality, which as shown in
Chapter 7 and in the literature [Arciniegas et al. 2013; Vonk & Ligtenberg 2010] also supports
planning activities. Furthermore, if planners are familiar with the underpinning GIS, they also
benefit from improved learnability in that they do not have to learn to use the core software but
only additional modules. PSS implemented as web applications or web-based PSS are easily
accessible because they generally do not require any software installation and can run on any
computer with a modern Internet browser. For these reasons, web-based PSS also facilitate the
exchange of information and collaboration with stakeholders, which is an important activity for
planners. To ensure that this activity can also be satisfactorily performed with PSS that are
implemented as GIS modules or standalone software, developers should consider aspects such
as data interoperability and software compatibility in PSS design.
Planners in the evaluation and interview study expressed the desire to make changes to
system functionality. Due to diverse habits, tasks and context of planning activities where the
PSS are used, planners have different needs which require a broad array of functionality. Data
availability might also differ depending on the regions and countries in which planners work. To
create PSS that are satisfying for practising planners, PSS need to be adaptable and customisable
to their needs.
As expected, the evaluation and interview study showed that planners search for help
documentation when needing assistance for proceeding with the interaction. Participants of the
user test had difficulties understanding the workflow process underpinning the PSS. Indeed, PSS
are quite complex systems [Brail 2006]. Nonetheless, they have to be usable [Costabile 2001].
For supporting planners with adequate assistance and improving transparency of the workflow
131
process, help documentation should be provided that includes explanations about assumptions,
computations, meaning and interpretation of outputs. Somewhat surprisingly, the existing PSS
reviewed did not have comprehensive help material: this is considered a significant barrier to
adoption that existing and future PSS developers should address.
Expectations were identified in the evaluation and interview study that are standard for
ensuring good user interface usability and UX. For instance, feedback techniques are
instrumental for improved user guidance, transparency of operations and trust towards the
system [Lim & Dey 2009; Lim et al. 2009]. Clear interface elements, engaging interaction
mechanisms and efficiency and stability are also elementary for positive UX. Yet a number of
these elements were found lacking in the evaluation of the three PSS.
RQ 3: How can planners’ context contribute to improving PSS usability and adoption in
planning practice?
Various pathways for improving PSS usability and adoption in planning practice emerged in this
research. To encourage following these pathways, recommendations for planning actors,
specifically PSS developers and academic planners, have been developed. Although they are not
mutually exclusive, they are discussed separately, as follows:
Recommendations for PSS developers
A main recommendation is to develop PSS following the HCD process [ISO 9241 2010b] (see
Figure 2.4), whose value in creating usable software systems has been demonstrated through
years of HCI research [Dix 2009; Costabile 2001; Preece et al. 1994; Rubin 1994]. The process
prescribes focusing on users from the very beginning of the system design and development
process. Developers closely collaborate with users to identify their requirements and the context
in which the system will be used. A further key role is played by the iterative development of
the system through prototypes of increasing complexity. Such prototypes have to be carefully
evaluated, in order to check if they comply with the identified user and context requirements.
Evaluation is the only way to understand the impact of a prototype on users and to identify
appropriate functionality for them and usability problems that are much cheaper to solve at the
early phase of the design and development [Ardito et al. 2014]. It can be performed by using
one or more of the many evaluation methods developed in the last three decades of research
on HCD [ISO 9241 2010b]. For people who are not confident with performing evaluations, this
research recommends the use of the PSS Evaluation Framework presented in Chapter 5 of this
132
thesis and in Russo et al. [2015]. Summarising, this research recommends the application of the
HCD process and its suggested iterative design-implementation-evaluation cycle for effectively
developing PSS that are satisfactory for planners.
An expectation of planners is to have flexible and customisable PSS (see Table 8.1) that
allow tailoring functionality to their needs. A minority of current PSS are customisable, as shown
in the findings of Chapter 4. Currently, modifications and new functionality are often
implemented by software engineers, in order to adapt software to what planners want. This
exercise requires significant financial resources. A recommendation of this research is to create
PSS as flexible systems that can be easily adapted by users themselves, at use time, to their
diverse and evolving requirements. Specifically, PSS developers should consider methodologies
and techniques proposed by research on End-User Development (EUD) whose aim is to create
systems that enable users themselves to tailor software according to their needs and
preferences [Diaz et al. 2015; Cabitza et al. 2014a, 2014b; Ardito et al. 2012].
Developments towards EUD can actually be observed for GIS applications. However, in
most cases programming skills are still required. As part of a so-called GIS revolution, Batty et
al. [2010; 2007] described how some open source GIS provide developer tools to extend the
platform through specialised modules. The Envision PSS implemented as a plugin in the GIS
software QuantumGIS [Newton & Glackin 2013] and evaluated in this thesis, is an example of
such a module. A more recent pathway is provided by GIS toolboxes, generally moving online
into clouds and cyberinfrastructure [Batty et al. 2010], that allow professional users and
programmers to integrate GIS functionality in more generic systems for enabling more
specialised tasks. This pathway has been taken by some PSS developers. The Online What if? PSS
evaluated in this thesis has been implemented as part of the AURIN workbench which also
comprises a suite of visualisation and statistical tools [Pettit et al. 2015a; 2015b]. Similarly, the
UrbanSim PSS [Waddell 2002] has been connected with scenario-generation and visualisation
tools within the UrbanSim Cloud Platform (www.urbansim.com/platform, accessed on
November 16, 2016). This research postulates that EUD will also be relevant for the future
generation of GIS applications. In fact, according to Batty, this will not be fully but “almost
entirely automated” [Batty et al. 2010: 12]. This suggests that there will still be steps that users
will have to perform which ideally should be adaptable to users’ needs based on EUD.
133
Recommendations for academic planners
The findings of Chapter 4 clearly showed that many PSS are developed in research projects but
for their access and adoption there is limited or no support, i.e. their access requires contacting
the developer, help documentation is not available or the PSS have not undergone rigorous
software development. The issue is that innovation and its potential remain underexploited in
planning practice. For bridging the gap in the transfer of PSS from academia to the planning
profession this thesis recommends that researchers develop partnerships with the private and
government sector or open source community to ensure there is a pathway for robust software
development and supporting community of practice in PSS adoption. While this approach has
already been proposed by Houghton et al. [2014], findings of the interview study in this thesis
showed that it has actually been adopted with success by researchers in Australia. In fact,
researchers of the AURIN project (http://aurin.org.au, accessed on November 16, 2016)
supported the Western Australia Department of Planning with implementing and adopting
Online What if? [Pettit et al. 2015b]. Developers of the Envision Scenario Planner [Pettit et al.
2014] presented the PSS to councils in the Melbourne Metropolitan Region. At least one council
agreed to adopt the PSS. The approach is promising as planning organisations do not have to
struggle with data preparation and can count on support from the research team and
developers.
The results of Chapter 7 showed that planners’ skills, experience and awareness of PSS
need to be improved, in order to realise a widespread adoption of PSS in the future. In order to
provide planners with skills required for using PSS, this research recommends that university
curricula should more extensively offer courses that increase planners’ technical and modelling
abilities. This conclusion has been reached as models are a central functionality of PSS [Krause
2013; Harris 1989] and, as outlined in Chapter 4, developers indicated GIS and modelling skills
to be required for using PSS they create. Moreover, this conclusion is supported by the literature
which reports that planners’ understanding of models and modelling process is low [Brits et al.
2014]. Thus, increasing students’ modelling abilities at university is instrumental in preparing
them for PSS adoption and overcoming this significant barrier. Learning pathways into higher
degree Masters courses in the newly emerged domain of city (urban) science [Batty 2013] could
also assist. Furthermore, it is suggested that just-in-time web courses offered through new
generation online learning platforms such as MOOC (https://www.edx.org/course, accessed on
January 15, 2017) and Planetizen (https://courses.planetizen.com/, accessed on January 15,
2017) could assist in increasing planners’ skills and providing them with appropriate teaching
134
and learning material.
In order to improve planners’ experiences and awareness of PSS and paving the way to
PSS adoption in later work practice, planners should study them within their university curricula.
This is indicated by the findings from the interview study which clearly show that planners are
inclined to adopt and use software tools they used during their studies. As such, this research
recommends that university curricula should explicitly include PSS training. In particular, courses
should train planning students to develop a critical attitude when applying PSS. This includes
being able to justify why they use a specific PSS and perform a certain analysis. In this way, such
training would prepare planning students for adoption of PSS in their subsequent work practice.
A further recommendation is to ensure that PSS reviews and repositories, such as the
Online PSS Resource formulated as part of this research and other such resources as the Spatial
Decision Support Knowledge Portal [The Redlands Institute 2012], are properly communicated,
disseminated amongst the planning discipline, and kept up-to-date. This will contribute to
improving planners’ awareness of PSS potential and ensure they choose PSS suitable to their
needs. Overall there seems to be a paucity of such PSS resources available online.
8.3 Limitations of this research
Due to time and resource constraints, this research conducted a small part of a potentially much
larger research agenda. While PSS adoption is determined by multiple factors, this research
focused on one factor, i.e. PSS usability and UX with PSS. However, due to the adopted
qualitative approach, this research did not allow for the examination of any correlation between
usability and actual adoption of PSS.
The evaluation and interview study have been conducted with quite small samples. This
is less of a limitation for the evaluation study, where a main goal of the user test was to identify
any usability problems of three PSS. These can be identified even with small samples of
participants as indicated in the literature [Nielsen & Landauer 1993; Virzi 1992]. A consequence
of the small sample of interviewees from three very diverse countries in the interview study
means that only limited generalisation of the findings could be made.
Both studies were of explorative nature and illustrate specific contexts. Despite the large
number of available PSS addressing different planning tasks, the evaluation study focused on
the usability of three PSS specifically for LSA, i.e. CommunityViz, Envision and Online What if?,
as well as on the UX of the participants when interacting with the three PSS.
The Online PSS Resource presented in this thesis also contains some limitations. It is
135
current as of June 2015 and the static nature of the webpage makes it difficult for developers to
add new PSS.
The PSS Evaluation Framework provides a possible structure and activities for those who
embark on the PSS development and evaluation path. The framework is not exhaustive: users
may be required to undertake other activities as described in the framework depending on the
evaluation context.
8.4 Future work
This research is a first step towards better understanding the PSS adoption process. Factors
affecting this process, in particular PSS usability, were investigated in the interview study.
Whether any direct correlation between usability and actual adoption of PSS exists, should be
the object of further work.
The influence of planners’ skills on PSS adoption has been discussed (see Section 7.5.1)
and compared with previous literature. Future work could involve comparing all influencing
factors identified in this study with previous literature, in order to examine the magnitude of the
findings and increase their generalisation. In general, further quantitative work is needed to
better understand PSS adoption. For instance, recent literature reported on planning software
training provided at US universities [Dawkins 2016; Greenlee et al. 2015; Edwards & Bates 2011].
A comprehensive international university curriculum evaluation is needed to gather information
about the state of the art in planning education and define appropriate pathways for preparing
planning students for adoption of PSS later in work practice.
Several multi-national commonalities and differences in planning education and
practice were identified in this research. For analysing if this information is significant and can
be generalised, further work is needed that focuses on a multi-national comparison with
targeted research questions and methodology.
Next steps also include assessing how technological and conceptual evolutions
mentioned in this thesis, such as the GIS revolution [Batty et al. 2010], the move of software
systems online into clouds and cyberinfrastructure [Batty et al. 2010], and the Geodesign
approach [Steinitz 2012], will influence the development of new PSS. This is important as these
evolutions are likely to impact the usability and therefore the adoption of new PSS.
Most results of the user test are not limited to PSS specifically for LSA but can be
transferred to PSS in general (see Table 6.5). Nonetheless, evaluation studies should be
performed on PSS other than those specifically for LSA to gain an actual overview of PSS
136
usability. Testing the LSA module of three PSS within a user test setting was possible because
they offered a certain level of guidance so that participants did not require any step-by-step
instruction. Other PSS, for example, for impact assessment, are likely to be too complex for being
tested by people without any previous training. To evaluate such PSS, inspection methods such
as heuristic evaluation and cognitive walkthrough are recommended (see Section 2.11.1).
Generally, more resources are expected to be invested in the development of
commercial than open source software tools. Whether this was the case for the three PSS
evaluated in the user test and for PSS in general, could be the object of further work.
Future work could also involve keeping the Online PSS Resource up-to-date and
gathering technical and practical information for at least currently supported PSS. Furthermore,
enhancing the website with visualisation and interaction tools would support users in the
selection of appropriate PSS.
The PSS Evaluation Framework as well as the presented planners’ expectations of
system functionality provide first guidance for PSS evaluation and design; however, they have
to be further specified, in order to improve their applicability. Specifically, the PSS Evaluation
Framework could be extended, for example, with more indications on how to perform
inspection methods and analyse data gathered from these methods. With these possible next
steps of research, this thesis stimulates and encourages further work towards fostering
improved usability and widespread adoption of PSS.
137
Bibliography
Ackerman, M.S., 2000. The intellectual challenge of CSCW: the gap between social requirements and technical feasibility. Journal of Human-Computer Interaction, 15(2), pp.179–203.
Agarwal, C., Green, G.L., Grove, M., Evans, T. & Schweik, C., 2000. A Review and Assessment of Land- Use Change Models Dynamics of Space, Time, and Human Choice, pp.812–855.
Albert, W. & Tullis, T., 2013. Measuring the user experience: collecting, analyzing, and presenting usability metrics. Second Ed., Newnes: Morgan Kaufmann.
Allen, E., 2008. Clicking toward better outcomes: Experience with INDEX. In R. K. Brail, ed. Planning support systems for cities and regions. Cambridge, Massachusetts: Lincoln Institute of Land Policy, pp.139–166.
Allen, E., 2001. INDEX: software for community indicators. In R. Brail & R. Klosterman, eds. Planning Support Systems: Integrating Geographic Information Systems, Models and Visualization Tools. Redlands, CA: Esri Press, pp.229–262.
Anas, A., 1998. The NYMTC Land Use Model, New York, USA.
Arciniegas, G., Janssen, R. & Rietveld, P., 2013. Effectiveness of collaborative map-based decision support tools: Results of an experiment. Environmental Modelling & Software, 39, pp.159–175.
Arciniegas, G. & Janssen, R., 2012. Spatial decision support for collaborative land use planning workshops. Landscape and Urban Planning, 107, pp.332–342.
Arciniegas, G., 2012. Map-based decision support tools for collaborative land use planning. Free University Amsterdam.
Ardito, C., Buono, P., Costabile, M.F. & Desolda, G., 2015. Interaction with large displays: a survey. ACM Computing Surveys, 47(3), pp.1–47.
Ardito, C., Buono, P., Caivano, D., Costabile, M.F. & Lanzilotti, R., 2014. Investigating and promoting UX practice in industry: an experimental study. In Elsevier, ed. International Journal of Human- Computer Studies. Missouri, USA: Maryland Heights, pp.542–551.
Ardito, C., Buono, P., Costabile, M.F., Lanzilotti, R. & Piccinno, A., 2012. End users as co-designers of their own tools and products. Journal of Visual Languages and Computing2, 23(2), pp.78–90.
Ardito, C., Buono, P., Caivano, D., Costabile, M. F., Lanzilotti, R., Bruun, A. & Stage, J., 2011. Usability evaluation: a survey of software development organizations. In International Conference on Software Engineering and Knowledge Engineering (SEKE 2011). Skokie, Illinois, USA: Knowledge Systems Institute, pp.282–287.
Australian Bureau of Statistics (ABS), 2003. Census of Population and Housing: Population Growth and Distribution, Australia. Australian Bureau of Statistics Census.
Bailey, S., 1982. Book Review: Inner City Regeneration by Robert K. Home. Urban Studies, 19(4), pp.425–426.
Bak, J.O., Nguyen, K., Risgaard, P. & Stage, J., 2008. Obstacles to usability evaluation in practice: a survey of software development organizations. In Proceedings of 5th Nordic conference on Human-computer interaction (NordiCHI ’08). New York, USA: ACM Press, pp.23–32.
Baker, K., Greenberg, S. & Gutwin, C., 2002. Empirical Development of a Heuristic Evaluation
138
Methodology for Shared Workspace Groupware. In Proceedings of the 2002 ACM conference on Computer supported cooperative work. ACM, pp.96–105.
Balzarini, R., Davoine, P.-A. & Ney, M., 2013. GIS-based land-use suitability mapping: cognitive processes and designing instructions that lead to expertise. In International Cartographic Conference, pp.411.
Bates, J. & Oosterhaven, J., 1999. Review of Land-Use/Transport Interaction Models, London.
Batty, M., 2013. The new science of city, MIT Press.
Batty, M., Hudson-Smith, A., Milton, R. & Crooks, A., 2010. Map mashups, Web 2.0 and the GIS revolution. Annals of GIS, 16(1), pp.1–13.
Batty, M., 2007. Planning Support Systems: Progress, Predictions and Speculations on the Shape of Things to Come. In Paper presented to the seminar on Planning Support Systems for Urban and Regional Analysis. Cambridge, pp.0–25.
Batty, M., 2003. Planning support systems: technologies that are driving planning. In S. Geertman & J. Stillwell, eds. Planning Support Systems in Practice. Springer Berlin Heidelberg, pp.v–viii.
Bell, M., Dean, C. & Blake, M., 2000. A model for forecasting the location of fringe urbanisation with GIS and 3D visualisation. Computers, Environment and Urban Systems, 24, pp.559–581.
Bevan, N., 1999. Quality in Use: Meeting User Needs for Quality. Journal of System and Software, pp.1–14.
Bias, R. & Mayhew, D., 2005. Cost-Justifying Usability. An Update for the Internet Age. 2nd Edition, Morgan Kaufmann.
Biermann, S., 2011. Planning Support Systems in a Multi-Dualistic Spatial Planning Context. Journal of Urban Technology, 18(4), pp.5–37.
Bingham, W. & Moore, B., 1924. Improvement of the quality of responses to factual survey questions by interviewer training. Public Opinion Quarterly, 52, pp.190–211.
Bishop, I. & Foerster, R., 2007. Where is the vision? Developing systems to enhance adoption of technology for public decision-making. In CUPUM07 10th International Conference on Computers in Urban Planning and Urban Management. Brasil, pp.1–13.
Bishop, I., 1998. Planning Support: hardware and software in search of a system. Computers, Environment and Urban Systems, 22, pp.189–202.
Boivie, I., Aaborg, C., Persson, J. & Lofberg, M., 2003. Why usability gets lost or usability in in-house software development. Interacting with Computers, 15(4), pp.623–639.
Bonner, K., 2002. Understanding Placemaking: Economics, Politics and Everyday Life in the Culture of Cities. Canadian Journal of Urban Research, 11(1), pp.1-16.
Brail, R., 2008. Planning support systems for cities and regions, Cambridge, MA: Lincoln Institute of Land Policy.
Brail, R., 2006. Planning Support Systems Evolving: When the Rubber Hits the Road. In Complex artificial environments. Springer Berlin Heidelberg, pp.307–317.
Brail, R. & Klosterman, R., 2001. Planning Support Systems: Integrating Geographical Information Systems, models and visualization tools, Redlands, CA: Esri Press.
139
Branch, M., 1998. Comprehensive Planning for the 21st Century, Praeger, Westport, CT.
Briassoulis, H., 2003. Analysis of land use change: theoretical and modeling approaches. The Web Book of Regional Science.
Brits, A., Burke, M. & Li, T., 2014. Improved modelling for urban sustainability assessment and strategic planning: local government planner and modeller perspectives on the key challenges. Australian Planner, 51(1), pp.76–86.
Brömmelstroet, M.T., 2016. PSS are more user-friendly, but are they also increasingly useful? Transportation Research Part A (in press).
Brömmelstroet, M.T., 2015. A critical reflection on the experimental method for planning research: Testing the added value of PSS in a controlled environment. Planning Practice and Research, 30(2), pp.179–201.
Brömmelstroet, M.T., 2014. Usability of usefulness: critical reflections on PSS solutionism. In COST/CITTA congress. Porto.
Brömmelstroet, M.T., 2013. Performance of Planning Support Systems: What is it, and how do we report on it? Computers, Environment and Urban Systems, (41), pp.299–308.
Brömmelstroet, M.T., 2012. Transparency, flexibility, simplicity: From buzzwords to strategies for real PSS improvement. Computers, Environment and Urban Systems, 36(1), pp.96–104.
Brömmelstroet, M.T. & Schrijnen, P., 2010. From planning support systems to mediated planning support: a structured dialogue to overcome the implementation gap. Environment and Planning B: Planning and Design, 37, pp.3-20.
Brömmelstroet, M.T., 2010a. Equip the Warrior instead of Manning the Equipment: Land Use and Transport Planning Support in the Netherlands. The Journal of Transport and Land Use, 3(1), pp.1–17.
Brömmelstroet, M.T., 2010b. Making planning support systems matter: improving the use of planning support systems for integrated land use and transport strategy-making. University of Amsterdam.
Brooke, J., 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry, 189(194), pp.4–7.
Cabitza, F. & Simone, C., 2015. Building socially embedded technologies: Implications about design. In V. Wulf, K. Schmidt, & D. Randall, eds. Designing socially embedded technologies in the real- world. Springer London, pp.217–270.
Cabitza, F., Fogli, D. & Piccinno, A., 2014a. “Each to His Own”: Distinguishing Activities, Roles and Artifacts in EUD Practices. In L. Caporarello, B. Di Martino, & M. Martinez, eds. Smart Organizations and Smart Artifacts. Switzerland: Springer International Publishing, pp.193–205.
Cabitza, F., Fogli, D. & Piccinno, A., 2014b. Fostering participation and co-evolution in sentient multimedia systems. Journal of Visual Languages and Computing, 25(6), pp.684–694.
Cajander, A., Gulliksen, J. & Boivie, I., 2006. Management perspectives on usability in a public authority: a case study. In Proceedings of the 4th Nordic conference on HCI (NordiCHI 06). New York, USA: ACM, pp.38–47.
Campagna, M., 2014. Geodesign from theory to practice. In TeMA Journal of Land Use, Mobility and Environment, Eight International Conference INPUT, Naples, pp.15-25.
140
Campagna, M. & Matta, A., 2014. Geoinformation technologies in sustainable spatial planning: a Geodesign approach to local land use planning. In Proceedings of the 2nd International Conference on Remote Sensing and Geoinformation of Environment, Cyprus.
Carsjens, G.J. & Ligtenberg, A., 2007. A GIS-based support tool for sustainable spatial planning in metropolitan areas, Landscape and Urban Planning, 80, pp.72–83.
Charmaz, K., 2014. Constructing grounded theory. Sage.
Choe, P., Kim, C., Lehto, M., Lehto, X. & Allebach, J., 2006. Evaluating and Improving a Self-Help Technical Support Web Site: Use of Focus Group Interviews, International Journal of Human–Computer Interaction, 21(3), pp.333-354.
Clarke, K., Gazulis, N., Dietzel, C. & Goldstein, N., 2007. A decade of SLEUTHing: Lessons learned from applications of a cellular automation land use change model. In P. Fisher, ed. Classics from IJGIS. Twenty Years of the International Journal of Geographical Information Systems and Science. Boca Raton, FL: Taylor and Francis, CRC, pp.413–425.
Coleman, G. & O’Connor, R., 2007. Using grounded theory to understand software process improvement: A study of Irish software product companies. Information Software Technologies, 49(6), pp.654-667.
Costabile, M.F. & Buono, P., 2013. Principles for Human-Centred Design of IR Interfaces. In A. M. Ferro et al., eds. Information retrieval meets information visualization. Berlin, Heidelberg: Springer, pp.28–47.
Costabile, M.F., Fogli, D., Mussio, P. & Piccinno, A., 2007. Visual Interactive Systems for End-User Development: a Model-based Design Methodology. IEEE Transactions On Systems Man And Cybernetics Part A-Systems And Humans, 37(6), pp.1029–1046.
Costabile, M.F., 2001. USABILITY IN THE SOFTWARE LIFE CYCLE. In Handbook of Software Engineering and Knowledge Engineering. World Scientific Publishing Company, pp.179–192.
Couclelis, H., 2005. “Where has the future gone?’’ Rethinking the role of integrated land-use models in spatial planning. Environment and Planning A, 37(8), pp.1353–1371.
Couclelis, H., 1991. Geographically informed planning: requirements for planning relevant GIS. Papers in Regional Science, 70, pp.9–20.
CRC-SI, 2016. Cooperative Research Centre for Spatial Information. www.crcsi.com.au/about/what-is-spatial-information/, accessed on January 18, 2017.
Cruzes, D.F. & Dybå, T., 2011. Recommended Steps for Thematic Synthesis in Software Engineering. In International Symposium on Empirical Software Engineering and Measurement (ESEM 2011), IEEE.
Currier, K. & Couclelis, H., 2014. Geodesigning ‘From the Inside Out”. In Geodesign by integrating design and geospatial sciences, Springer, pp.287-298.
Davis, F.D., 1989. Information Technology Introduction, 13(3), pp.319-340.
Dawkins, C., 2016. Preparing Planners: The Role of Graduate Planning Education. Journal of Planning Education and Research2, pp.1–13.
De la Barra, T., 2001. Integrated land use and transport modeling: the TRANUS experience. In R. Brail & R. E. Klosterman, eds. Planning Support Systems: Integrating Geographic Information Systems, Models and Visualization Tools2. Redlands, CA: Esri Press, pp.129–
141
156.
Delikostidis, I., Van Elzakker, C. & Kraak, M.-J., 2015. Overcoming challenges in developing more usable pedestrian navigation systems. Cartogrphy and Geographic Information Science, pp.1– 19.
Densham, P., 1991. Spatial decision support systems. In D. Maguire, M. Goodchild, & D. Rhind, eds. Geographical Information Systems: principles and applications. London: Longman Group.
Diaz, P., Pipek, V., Ardito, C., Jensen, C., Aedo, I. & Boden, A., 2015. End-User Development. In Proceedings of the 5th International Symposium, (IS-EUD 2015) (Vol. 9083). Springer Berlin Heidelberg.
Dittrich, Y., John, M., Singer, J. & Tessem, B., 2007. For the special issue on Qualitative Software Engineering Research Information and Software Technology2, 49(6), pp.531-539.
Dix, A., 2009. Human-computer interaction, Springer US.
Dix, J., Finlay, G., Abowd, G. & Beale, R., 1998. Human Computer Interaction, Prentice Hall.
Doubleday, A., Ryan, M., Springett, M. & Sutcliffe, A., 1997. A comparison of usability techniques for evaluating design. In Proceedings of Designing Interactive Systems. Amsterdam, The Netherlands: Springer Verlag, pp.101–110.
Edwards, M. & Bates, L., 2011. Planning’s Core Curriculum: Knowledge, Practice, and Implementation. Journal of Planning Education and Research, 31(2), pp.172–183.
Etzioni, A., 1967. Mixed scanning: a “third” approach to decision making. Public Administration Review.
Fischer, G. 2010. End User Development and Meta-Design: Foundations for Cultures of Participation. Journal of Organizational and End User Computing, 22(1), pp.52–82.
Fischer, G., Piccinno, A. & Ye, Y., 2008. The Ecology of Participants in Co-evolving Socio-technical Environments. In P. Forbrig & F. Paterno’, eds. Engineering Interactive Systems. Springer Berlin Heidelberg, pp.279–286.
Fogli, D. & Piccinno, A., 2013. Co-evolution of End-User Developers and Systems in Multi-tiered Proxy Design Problems. In Y. Dittrich et al., eds. End-User Development. Springer Berlin Heidelberg, pp.153–168.
Frambach, R. & Schillewaert, N., 2002. Organizational innovation adoption, a multilevel framework of determinants and opportunities for future research. Journal of Business Research, 55, pp.163–176.
Geertman, S. 2016. Beyond the implementation gap. Transportation Research Part A: Policy & Practice.
Geertman, S., Ferreira, J., Goodspeed, R. & Stillwell, J., 2015. Planning Support Systems and Smart Cities, Springer.
Geertman, S., Stillwell, J. & Toppen, F., 2013. Planning Support Systems for sustainable urban development, Springer Berlin Heidelberg.
Geertman, S. & Stillwell, J., 2009a. Planning Support Systems: Content, Issues and Trends. In S. Geertman & J. Stillwell, eds. Planning Support Systems Best Practice and New Methods. Springer Berlin Heidelberg, pp.1–26.
142
Geertman, S. & Stillwell, J., 2009b. Planning support systems: Best practices and new methods, Springer Berlin Heidelberg.
Geertman, S. & Stillwell, J., 2004. Planning support systems: an inventory of current practice. Computers, Environment and Urban Systems, 28, pp.291–310.
Geertman, S. & Stillwell, J., 2003a. Planning Support Systems: An Introduction. In S. Geertman & J. Stillwell, eds. Planning Support Systems in Practice. Springer Berlin Heidelberg, pp.3–22.
Geertman, S. & Stillwell, J., 2003b. Planning Support Systems in practice, Berlin: Springer Berlin Heidelberg.
Geertman, S., De Jong, T. & Wessels, C., 2003c. Flowmap: A Support Tool for Strategic Network Analysis. In S. Geertman & J. Stillwell, eds. Planning Support Systems in Practice. Berlin: Springer, pp.155–175.
Geertman, S. 2002. Participatory planning and GIS: a PSS to bridge the gap. Environment and Planning B: Planning and Design, 29, pp.21–35.
Ghezzi, C., Jazayeri, M. & Mandrioli, D., 1991. Fundamentals of Software Engineering, Prentice Hall International.
Glackin, S., 2013. REDEVELOPING THE GREYFIELDS WITH ENVISION: USING PARTICIPATORY SUPPORT SYSTEMS TO REDUCE URBAN SPRAWL IN AUSTRALIA. European Journal of Geography, 3(3), pp.6–22.
Goodchild, M., 2010. Towards GeoDesign: Repurposing cartography and GIS? Cartographic Perspectives, 66, pp.7-22.
Goodspeed, R., 2013. Planning Support Systems for Spatial Planning Through Social Learning. MIT, PhD thesis.
Google, 2013a. Google. https://www.google.com/, accessed on April 5, 2015.
Google, 2013b. Google Scholar. http://scholar.google.com/, accessed on April 5, 2015.
Graziano, A.M. & Raulin, M.L., 2012. Research methods, a process of inquiry. 8th Edition, New York, USA: Pearson.
Greenlee, A., Edwards, M. & Anthony, J., 2015. Planning Skills: An Examination of Supply and Local Government Demand. Journal of Planning Education and Research, 35(2), pp.161–173.
Gregor, B., 2007. Land use scenario developer: Practical land use model using a stochastic microsimulation framework. Transportation Research Record: Journal of the Transportation Research Board.
Gurran, N., 2007. Australian urban land use planning: Introducing statutory planning practice in New South Wales, Sydney University Press.
Hackos, J. & Redish, J., 1998. User and Task Analysis for Interface Design, John Wiley & Sons. Haklay, M., 2010. Interacting with geospatial technologies, Chichester, UK: Wiley-Blackwell.
Haklay, M., 2010. Interacting with geospatial technologies, Chichester, UK: Wiley-Blackwell.
Haklay, M. & Zafiri, A., 2008. Usability Engineering for GIS: Learning from a Screenshot. Cartographic Journal, The, 45(2), pp.87–97.
Haklay, M. & Tobon, C., 2003. Usability Evaluation and PPGIS: Towards a User-Centred Approach.
143
International Journal of Geographical Information Science, 17(6), pp.577–592.
Harding, A., 2007. Challenges and Opportunities of Dynamic Microsimulation Modelling. In The 1st General Conference of the International Microsimulation Association. Vienna.
Harris, B. & Batty, M., 1993. Locational models, geographical information, and planning support systems. Journal of Planning Education and Research, 12, pp.184–198.
Harris, B., 1989. Beyond geographic information systems. Journal of the American Planning Association1, 55(1), pp.85–90.
Hartson, H.R. & Hix, D., 1993. Developing User Interfaces, New York, USA: John Wiley.
Hassenzahl, M. & Tractinsky, N., 2006. User experience-a research agenda. Behaviour & information technology, 25(2), pp.91-97.
Healey, P., 1992. Planning through Debate: the communicative Turn in Planning Theory. Town Planning Review, 63(2), pp.143–162.
Hilbert, D.M. & Redmiles, D.F., 2001. Extracting Usability Information from User Interface Events. ACM Computing Surveys, 32(4), pp.384–421.
Hilferink, M. & Rietveld, P., 1999. LAND USE SCANNER: An integrated GIS based model for long term projections of land use in urban and rural areas. Journal of Geographical Systems, 1(2), pp.155– 177.
Hillston, J., 2003. Model validation and verification. University of Edinburgh, pp.102-109.
Hoeven, E., Van der Aarts, J., Van der Klis, H. & Koomen, E., 2009. An integrated discussion support system for new Dutch flood risk management strategies. In S. Geertman & J. Stillwell, eds. Planning support systems: Best practices and new methods. Springer Berlin Heidelberg, pp.159–174.
Holway, J., Gabbe, C., Hebbert, F., Lally, J., Matthews, R. & Quay, R., 2012. Opening Access to Scenario Planning Tools, Cambridge, MA.
Hopkins, L., Ramanathan, R. & Pallathucheril, V., 2004. Interface for a sketchplanning workbench. Computers Environment and Urban Systems, 28, pp.653–666.
Houghton, K., Miller, E. & Foth, M., 2014. Integrating ICT into the planning process: impacts, opportunities and challenges. Australian Planner2, 51(1), pp.24–33.
Hughes, M. & Heckbert, S., 2012. Scenario Planning Software Tool Review. A report for the Department of Planning WA, Perth, Western Australia.
Hunt, J. & Abraham, J., 2005. Design and implementation of PECAS: A generalised system for allocating economic production, exchange and consumption quantities. In M. Lee-Gosselin & S. Doherty, eds. Integrated land-use and transportation models: Behavioural foundations. Amsterdam, The Netherlands: Elsevier, pp.253–273.
Hunt, J.D., Kriger, D.S. & Miller, E.J., 2005. Current operational urban land-use–transport modelling frameworks: A review. Transport Reviews, 25(3), pp.329–376.
Iacono, M., Levinson, D. & El-Geneidy, a., 2008. Models of Transportation and Land Use Change: A Guide to the Territory. Journal of Planning Literature, 22(4), pp.323–340.
Innes, J., 1996. Planning Through Consensus Building: A new View of the Comprehensive Planning Ideal. Journal of the American Planning Association1, 62(4), pp.460–472.
ISO 9126, 1998a. ISO/IEC 9126: Information technology - Software Product Evaluation.
144
ISO 9126, 1998b. ISO/IEC 9126-1: Information technology - Software Product Quality.
ISO 9241, 2010a. ISO/IEC 9241: Ergonomic Requirements for Office Work with Visual Display Terminals.
ISO 9241, 2010b. ISO/IEC 9241-210: Ergonomics of Human-System Interaction - Part 210: Human-Centred Design for Interactive Systems.
Jankowski, P. & Richard, L., 1994. Integration of GIS-Based Suitability Analysis and Multicriteria Evaluation in a Spatial Decision Support System for Route Selection. Environment and Planning B: Planning and Design, (21), pp.323–340.
Janssen, R. & Rietveld, P., 1990. Multicriteria analysis and geographical information systems: an application to agricultural land use in the Netherlands. In H. Scholten & J. Stillwell, eds. Geographical information systems for urban and regional planning. Springer Netherlands, pp.129–139.
Ji, Y.G. & Yun, M.H., 2006. Enhancing the Minority Discipline in the IT Industry: A Survey of Usability and User-Centered Design Practice. International Journal of Human–Computer Interaction, (20)2, pp.117-134.
Karlsson, L., Dahlstedt, A., Regnell, B., Natt och Dag, J. & Persson, A., 2007. Requirements engineering challenges in market-driven software development – an interview study with practitioners. Information and Software Technology2, 49(6), pp.588–604.
Kim, J., Giacomin, R. & Macredie, R., 2014. A qualitative Study of Stakeholders’ Perspectives on the Social Network Service Environment. International Journal of Human-Computer Interaction, 30(12), pp.965-976.
Klosterman, R., 2013. Lessons Learned About Planning, pp.1–37.
Klosterman, R.E. & Pettit, C.J., 2005. An update on planning support systems. Environment and Planning B: Planning and Design, 32(4), pp.477–484.
Klosterman, R.E., Siebert, L., Hoque, M. A., Kim, J. W. & Parveen, A., 2003. Using an operational planning support system to evaluate farmland preservation policies. In S. Geertman & J. Stillwell, eds. Planning Support Systems in Practice. Springer Berlin Heidelberg, pp.391–407.
Klosterman, R.E., 2001. Planning Support Systems: A New Perspective on Computer-aided Planning R. E. Klosterman & R. K. Brail, eds., Esri Press.
Klosterman, R.E., 1999. The What if? Collaborative Planning Support System. Environment and Planning B: Planning and Design, (26), pp.393–408.
Klosterman, R.E., 1997. Planning support systems: A new perspective on computer-aided planning. Journal of Education and Research, 17(1), pp.45–54.
Koeninger, A. & Bartel, S., 1998. 3D-GIS for Urban Purposes. GeoInformatica, 2(1), pp.79–103.
Koomen, E. & Stillwell, J., 2007. Modelling Land-Use Change. In E. Koomen & J. Stillwell, eds. Modelling Land-Use Change. Netherlands: Springer, pp.1–22.
Krause, A., 2013. Urban Intensification in Seattle: A Data System, Policy Evaluation and Market Analysis. University of Washington.
Kunze, A., Burkhard, R., Gebhardt, S. & Tuncer, B., 2012. Visualization and Decision Support Tools in Urban Planning. Digital Urban Modeling and Simulation. Springer Berlin Heidelberg, pp.279–298.
145
Kwartler, M. & Bernard, R.N., 2001. CommunityViz: an integrated planning support system. In Planning Support Systems: integrating geographic information systems, models, and visualization tools, pp.279-298.
Landis, J., Monzon, J., Reilly, M. & Cogan, C., 1998. Development and pilot application of the California urban and biodiversity analysis (CURBA) Model, Berkeley, CA.
Landis, J., 1994. The California Urban Futures Model: a new generation of metropolitan simulation models. Environment and Planning B: Planning and Design1, 21(4), pp.399–420.
Lantman, J.V.S., Verburg, P.H., Bregt, A. & Geertman, S., 2011. Core Principles and Concepts in Land-Use Modelling: A Literature Review, pp.35–57.
Lanzilotti, R., De Angeli, A., Ardito, C. & Costabile, M.F., 2011. Do patterns help novice evaluators? A comparative study. International Journal of Human-Computer Studies2, 69(1–2), pp.52–69.
Lanzilotti, R., 2006. A HOLISTIC APPROACH TO DESIGNING AND EVALUATING E-LEARNING SYSTEMS QUALITY: USABILITY AND EDUCATIONAL EFFECTIVENESS. Universita’ degli Studi di Bari.
Larsen, K., Cser, J., Conder, S. & Planner, P., 2000. MetroScope: Simulating Future Urban Landscapes at the Parcel Level. In Proceedings of the Twentieth ESRI International User Conference.
Larusdottir, M.K., 2012. User Centred Evaluation in Experimental and Practical Settings. KTH Royal Institute of Technology, Stockholm, Sweden.
Laurian, L., Crawford, J., Day, M., Kouwenhoven, P., Mason, G., Ericksen, N. & Beattie, L., 2010. Evaluating the outcomes of plans: theory, practice and methodology. Environment and Planning B: Planning and Design, 37, pp.740–757.
Lautso, K., 2003. The SPARTACUS system for defining and analyzing sustainable land use and transport policies. In S. Geertman & J. Stillwell, eds. Planning Support Systems in Practice. Springer Berlin Heidelberg, pp.453–463.
Lavalle, C., Baranzelli, C., Batista, F., Mubareka, S., Gomes, C., Koomen, E. & Hilferink, M., 2011. A High Resolution Land Use/Cover Modelling Framework for Europe: Introducing the EU- ClueScanner100 Model. In International Conference on Computational Science and its Applications. Berlin, Heidelberg: Springer, pp.60–75.
Law, E., Roto, V., Hassenzahl, M., Vermeeren, A. & Kort, J., 2009. Understanding, scoping and defining user experience: a survey approach. In Proceedings of the SIGCHI Conference on Human factors in Computing Systems. New York, USA, pp.719–728.
Lee, D., Dias, E. & Scholten, H. 2014. Geodesign by integrating design and geospatial sciences, Vol. 111, Springer.
Lee, A. & Wu, S., 2005. THE UTILISATION OF BUILDING INFORMATION MODELS IN MODELLING: A STUDY OF DATA INTERFACING AND ADOPTION BARRIERS, 10(February), pp.85–110.
Lee, D., 1994. Retrospective on Large-Scale Urban Models. Journal of the American Planning Association1, 60(1), pp.35–40.
Lee, D., 1973. Requiem for large-scale models. Journal of the American Institute of Planners, 39(3), pp.163–178.
146
Lei, Z., Pijanowski, B.C., Alexandridis, K.T. & Olson, J., 2005. Distributed Modeling Architecture of a Multi-Agent-Based Behavioral Economic Landscape (MABEL) Model. Simulation, 81(7), pp.503– 515.
Le Page, C., Bousquet, F., Bakam, I., Bah, A. & Baron, C., 2000. CORMAS: A multiagent simulation toolkit to model natural and social dynamics at multiple scales. In The ecology of scales. Wageningen.
Lethbridge, T.C., 2000. What knowledge is important to a software professional? IEEE Computer, 33(5), pp.44–50.
Lieberman, H., Paternò, F. & Wulf W. 2006. End User Development. Dordrecht, The Netherlands: Springer.
Lim, B. & Dey, A., 2009. Assessing Demand for Intelligibility in Context-aware Applications. In Proceedings of the 11th International Conference on Ubiquitous Computing (UbiComp ’09). New York, USA: ACM, pp.195–204.
Lim, B., Dey, A. & Avrahami, D., 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp.2119–2128.
Lim, K.H., Bembasat, I. & Tood, P.A., 1996. An experimental investigation of the interactive effects of interactive style, instructions, and task familiarity on user performance. ACM Transaction on Computer-Human Interaction, 3(1), pp.1–37.
Lindblom, C.E., 1965. The intelligence of democracy, New York, USA: Free Press.
Ling, C. & Salvendy, G., 2005. Extension of heuristic evaluation method: a review and reappraisal. Ergonomia - An International Journal of Ergonomics and Human Factors, 27(3), pp.179–197.
McCall, J., 1994. Quality factors. In J. Marciniak, ed. Encyclopedia of Software Engineering. New York, USA: Wiley & Sons, pp.958–969.
Mack, R. & Nielsen, J., 1994. Usability inspection methods, New York, USA: Wiley & Sons.
MacLean, A., Carter, K., Loevstrand, L. & Moran, T., 1990. User-tailorable systems: pressing the issues with buttons. In SIGCHI Conference on Human Factors in Computing Systems: Empowering People. Seattle, WA, United States: ACM, pp.175–182.
Madsen, K., 1999. Special Issue on “The Diversity of Usability.” Communication of ACM, 42(5).
Malczewski, J., 1999. GIS and multicriteria decision analysis, John Wiley & Sons.
Mayfield, C., 2015. Automating the Classification of Thematic Rasters for Weighted Overlay Analysis in GeoPlanner for ArcGIS. University of Redlands.
Mayhew, D., 1992. Principles and Guidelines in Software User Interface Design, Prentice Hall: Englewood Cliffs.
Mayhew, D., 1999. The Usability Engineering Lifecycle: a Practitioner’s Handbook for User Interface Design, California, USA: Morgan Kaufmann.
McHarg, I.L., 1969. Design with Nature. Doubleday-Natural History Press, New York.
Meng, Y. & Malczewski, J., 2009. Usability evaluation for a web-based public participation GIS: a case study in Canmore, Alberta. Cybergeo: European Journal of Geography.
Merry, K., Bettinger, P. & Hubbard, W., 2008. Back to the future part I: Surveying geospatial
147
technology needs of Georgia land use planners. Journal of Extension, 46(3).
Miles, M.B. & Huberman, A.M., 1994. Qualitative data analysis: An expanded sourcebook. 2nd Edition, Thousand Oaks, California: SAGE Publications.
MOOC, 2017. MOOC courses. https://www.edx.org/course, accessed on January 15, 2017.
Nearmap, 2016. Nearmap Pty Ltd. www.nearmap.com.au, accessed on January 2, 2017.
Nedovic-Budic, Z., 1998. The impact of GIS technology. Environment and Planning B: Planning and Design, 25, pp.681–692.
Newton, P. & Glackin, S. 2013. Using geo-spatial technologies as stakeholder engagement tools in urban planning and development. Built Environment, 39(4), pp.473–501.
Nielsen, J., 1993. Usability Engineering, Boston: Academic Press.
Nielsen, J. & Landauer, T., 1993. A Mathematical Model of The Finding of Usability Problems. In International Conference on Human Factors in Computing Systems (ACM INTERCHI’93). Amsterdam, The Netherlands: ACM Press, pp.296–313.
Nijkamp, P. & Delft, A. 1977. Multi-Criteria Analysis and Regional Decision-Making, Springer Science+Business Media B.V.
Nyerges, T., Jankowski, P., Tuthill, D. & Ramsey, K., 2006. Collaborative water resource decision support: results of a field experiment. Annals of the Association of American Geographers2, 96(4), pp.699–725.
Norman, D. 1983. Some Observations on Mental Models. In D. Gentner & A. Stevens, eds. Mental Models. Lawrence Erlbaum Associates, Inc., pp.7–14.
Ottensmann, J., 2005. Accessibility in the Luci2 Urban Simulation model and the importance of accessibility for urban development. In D. Levinson & K. Krizek, eds. Access to destinations. Amsterdam, The Netherlands: Elsevier, pp.297–324.
Patton, M., 1990. Qualitative evaluation and research methods, Sage Publications, Newbury Park, California.
Pelizaro, C., Arentze, T. & Timmermans, H., 2009. GRAS: A Spatial Decision Support System for Green Space Planning. In S. Geertman & J. Stillwell, eds. Planning Support Systems Best Practice and New Methods. Springer Berlin Heidelberg, pp.191–208.
Pelzer, P., 2016. Usefulness of planning support systems: A conceptual framework and an empirical illustration. Transportation Research Part A (in press).
Pelzer, P., Geertman, S. & Van der Heijden, R., 2016. A comparison of the perceived added value of PSS applications in group settings. Computers, Environment and Urban Systems2, 56, pp.25–35.
Pelzer, P., Arciniegas, G., Geertman, S. & Lenferink, S., 2015. Planning Support Systems and Task Technology Fit: a Comparative Case Study. Applied Spatial Analysis and Policy, pp.1–21.
Pelzer, P., Goodspeed, R. & Brömmelstroet, M.T., 2015. Facilitating PSS workshops: A conceptual frameworks and findings from interviews with facilitators. In S. Geertman et al., eds. Planning Support Systems and Smart Cities. Springer International Publishing, pp.355–369.
Pelzer, P., 2015. Usefulness of Planning Support Systems: Conceptual perspectives and practitioners’ experiences, Utrecht University: Department of Human Geography and
148
Spatial Planning, PhD thesis. Available at: https://dspace.library.uu.nl/bitstream/handle/1874/312867/Pelzer.pdf%3Bsequence=1, accessed on March 17, 2017.
Pelzer, P., Geertman, S., Van der Heijden, R. & Rouwette, E., 2014. The added value of Planning Support Systems: A practitioner’s perspective. Computers, Environment and Urban Systems, 48, pp.16–27.
Pelzer, P., Arciniegas, G., Geertman, S. & De Kroes, J., 2013. Using MapTable to learn about sustainable urban development. In S. Geertman, F. Toppen, & J. Stillwell, eds. Planning Support Systems for Sustaninable Urban Development. Lecture Notes in Geoinformation and Cartography. Berlin, Heidelberg: Springer, pp.167–186.
Pettit, C.J., Barton, J., Goldie, X., Sinnott, R., Stimson, R. & Kvan, T., 2015a. The Australian Urban Intelligence Network supporting Smart Cities. In S. Geertman et al., eds. Planning Support Systems and Smart Cities. Springer, pp.243–259.
Pettit, C.J., Klosterman, R. E., Delaney, P., Whitehead, A. L., Kujala, H., Bromage, A. & Nino-Ruiz, M., 2015b. The Online What if? Planning Support System: A Land Suitability Application in Western Australia. Applied Spatial Analysis and Policy, 8(2), pp.93–112.
Pettit, C.J., Glackin, S., Trubka, R., Ngo, T., Lade, O., Newton, P. & Newman, P., 2014. A Co-Design prototyping approach for building a Precinct Planning Tool. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2(2), pp.47.
Pettit, C.J., Klosterman, R.E., Nino-Ruiz, M., Widjaja, I., Russo, P., Tomko, M. & Sinnott, R., 2013. The Online What if? Planning Support System. In S. Geertman, F. Toppen, & J. Stillwell, eds. Planning Support Systems for Sustaninable Urban Development. Lecture Notes in Geoinformation and Cartography. Berlin, Heidelberg: Springer, pp.349–362.
Pettit, C.J., Widjaja, I., Russo, P., Sinnott, R., Stimson, R. & Tomko, M., 2012. Visualisation support for exploring urban space and place. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, I-2, pp.153–158.
Pettit, C.J., Raymond, C.M., Bryan, B. a. & Lewis, H., 2011. Identifying strengths and weaknesses of landscape visualisation for effective communication of future alternatives. Landscape and Urban Planning, 100(3), pp.231–241.
Pettit, C.J. & Wyatt, R., 2009. A Planning Support System Toolkit Approach for Formulating and Evaluating Land-use Change Scenarios. In S. Geertman & J. Stillwell, eds. Planning Support Systems Best Practice and New Methods. Springer Berlin Heidelberg, pp.69–90.
Pettit, C., Keysers, J., Bishop, I. & Klosterman, R., 2008. Applying the What if? Planning Support System for Better Planning at the Urban Fringe. In Landscape Analysis and Visualisation. Springer Berlin Heidelberg, pp.435–454.
Pettit, C.J. & Pullar, D., 2008. An Online Course Introducing GIS to Urban and Regional Planners. Applied Spatial Analysis and Policy, 2(1), pp.1–21.
Pettit, C.J., 2005. Use of a collaborative GIS-based planning-support system to assist in formulating a sustainable-development scenario for Hervey Bay, Australia. Environment and Planning B: Planning and Design, 32(1998), pp.523–546.
Pettit, C. & Pullar, D., 1999. An integrated planning tool based upon multiple criteria evaluation of spatial information. Computers, Environment and Urban Systems, 23(5), pp.339–357.
Planetizen, 2017. Planetizen courses. https://courses.planetizen.com/, accessed on January 15,
149
2017.
Polillo, R., 2010. Facile da usare. Una moderna introduzione alla ingegneria della usabilita’, Milano: Apogeo.
Polson, P.G., Lewis, C., Rieman, J. & Wharton, C., 1992. Cognitive walkthroughs: a method for theory- based evaluation of user interfaces. International Journal of Man-Machine Studies, 36(5), pp.741–773.
Pontius, R., Cornell, J. & Hall, C., 2001. Modeling the spatial pattern of land-use change with GEOMOD2: Application and validation for Costa Rica. Agriculture, Ecosystems & Environment, 85(1–3), pp.191–203.
Pozoukidou, G., 2006. Planning Support Systems’ Application Bottlenecks. In ERSA conference papers. European Regional Science Association, pp.1–17.
Protocollo eGLU, 2015. Glossario dell’ usabilita’, Ministry of the Public Administration, Italian Government, pp.1-114.
Preece, J., Sharp, H. & Rogers, Y., 2015. Interaction design: beyond human-computer, West Sussex: John Wiley & Sons Ltd.
Preece, J., Rogers, Y., Sharp, H., Beyon, D., Holland, S. & Carey, T., 1994. Human-Computer Interaction, Addison-Wesley Professional.
Pucher, A., 2008. Use and Users of the OEROK-Atlas online. The Cartographic Journal2, 45(2), pp.108–116.
Pullar, D. & McDonald, G., 1999. Geographical Information Systems. Australian Planner, 36(4), pp.216–222.
Putman, S., 2001. The METROPILUS planning support system: Urban models and GIS. In R. Brail & R. E. Klosterman, eds. Planning support systems: Integrating geographic information systems, models and visualization tools. Redlands, CA: Esri Press, pp.99–128.
Raskin, J., 2000. The humane interface: new directions for designing interactive systems. New York, USA: ACM Press/Addison-Wesley Publishing Co.
Rittel, H. 1984. Second-Generation Design Methods. In N. Cross, ed. Developments in Design Methodology. New York, USA: John Wiley & Sons, pp.317–327.
Rodier, C. & Spiller, M., 2012. Model-based Transportation Performance: A Comparative Framework and Literature Synthesis, San Jose’, California, USA.
Rogers, E., 2010. Diffusion Of Innovations, Simon and Schuster.
Rogers, E., 1995. Diffusion Of Innovations, Simon and Schuster.
Rosenbaum, S., Rohn, J.A. & Humburg, J., 2000. A toolkit for strategic usability: results from workshops, panels, and surveys. In Proceedings of SIGCHI conference on Human Factors in Computing Systems (CHI 00). New York, USA: ACM, pp.337-344.
Roto, L., Law, E., Vermeeren, A. & Hoonhout, J., 2011. User Experience White Paper - Bringing clarity to the concept of user experience, Dagstuhl, Germany.
Roto, V., Ketola, P. & Huotari, S., 2008. User Experience Evaluation in Nokia. In Proceedings of UXEM: User Experience Evaluation Methods in Product Development Workshop (CHI 08).
Rubin, J., 1994. Handbook of Usability Testing, John Wiley & Sons.
Russo, P., Lanzilotti, R., Costabile, M.F. & Pettit, C.J., 2017. Adoption and use of software in land
150
use planning practice: A multiple-country study. International Journal of Human-Computer Interaction, pp.1-16.
Russo, P., Lanzilotti, R., Costabile, M.F. & Pettit, C.J., 2018. Towards satisfying practitioners in using Planning Support Systems. Computers, Environment and Urban Systems, 67, pp.9-20.
Russo, P., Costabile, M.F., Lanzilotti, R. & Pettit, C.J., 2015. Usability of Planning Support Systems: an evaluation framework. In S. Geertman et al., eds. Planning Support Systems and Smart Cities. Springer International Publishing, pp.337–353.
Salter, J.D., Campbell, C., Journeay, M. & Sheppard, S.R.J., 2009. The digital workshop: Exploring the use of interactive and immersive visualisation tools in participatory planning. Journal of Environmental Management, 90(6), pp.2090–2101.
Salvani, P. & Miller, E., 2005. ILUTE: An operational prototype of a comprehensive microsimulation model of urban systems. Networks and Spatial Economics, 5(2), pp.217–234.
Sanders, E. & Stappers, P., 2008. Co-creation and the new landscapes of design. Co-design, 4(1), pp.5–18.
Schoen, D.A., 1983. The Reflective Practitioner: How Professionals Think in Action, New York, USA: Basic Books.
Schuler, D. & Namioka, A. 1993. Participatory Design: Principles and Practices, Hillsdale, NJ, USA: L. Erlbaum Associates Inc.
Seaman, C.B., 1999. Qualitative Methods in Empirical Studies of Software Engineering. Software Engineering, IEEE Transactions, 25(4), pp.557–572.
Seasons, M., 2003. Monitoring and evaluation in municipal planning: Considering the realities. Journal of the American Planning Association, 69(4), pp.430–440.
Seewald, F. & Hassenzahl, M., 2004. Vom kritischen Ereignis zum Nutzungsproblem: Die qualitative Analyse in diagnostischen Usability Tests. In: M. Hassenzahl & M. Peissner, eds. Tagungsband UP04. Stuttgart: Fraunhofer Verlag, pp.142-148.
Sharma, S., Pettit, C., Bishop, I., Chan, P. & Sheth, F., 2011. An online landscape object library to support interactive landscape planning. Future Internet, 3(4), pp.319–343.
Shneiderman, B. & Plaisant, C., 2010. Designing the User Interface: Strategies for Effective Human- Computer Interaction. 5th Edition, Reading, MA: Addison-Wesley Professional.
Shneiderman, B. 1996. The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. In IEEE Symposium on Visual Languages. pp.336–343.
Sidlar, C.L. & Rinner, C., 2007. Analyzing the usability of an argumentation map as a participatory spatial decision support tool. Journal of the Urban and Regional Information Systems Association, 19, pp.47–55.
Silva, C.N., 2010. Handbook of Research on E-Planning: ICTs for Urban Development and Monitoring, IGI Global, Hershey.
Silva, E. & Clarke, K., 2002. Calibration of the SLEUTH urban growth model for Lisbon and Porto. Computers Environment and Urban Systems2, 26(6), pp.525–552.
Sinnott, R., Bayliss, C., Bromage, A., Galang, G., Grazioli, G., Greenwood, P., Macauley, G., Mannix, D., Morandini, L., Nino-Ruiz, M., Pettit, C.J., Tomko, M., Sarwar, M., Stimson, R.,
151
Voorsluys, W. & Widjaja, I., 2014. The Australia Urban Research Gateway. Concurrency and Computation: Practice and Experience, 27(2), pp.358–375.
Somervell, J. & McCrickard, S., 2004. Comparing generic vs. specific heuristics: Illustrating a new UEM comparison technique. In Proceedings of the Human Factors and Ergonomics Society. SAGE Publications, pp.2480–2484.
Sommerville, I., 1996. Software process models. ACM Computing Surveys1, 28(1), pp.269–271.
Stevens, D., Dragicevic, S. & Rothley, K., 2007. iCity: A GIS–CA modelling tool for urban planning and decision making. Environmental Modelling & Software, 22(6), pp.761–773.
Steinitz, C., 2012. A framework for Geodesign. ESRI Press, Redlands, CA, United States.
Stillwell, J., Geertman, S. & Openshaw, S., 1999. Geographical information and planning: European perspectives, Springer Berlin Heidelberg.
Stimson, R., Bell, M., Corcoran, J. & Pullar, D., 2012. Using a large scale urban model to test planning scenarios in the Brisbane-South East Queensland Region. Regional Science Policy & Practice, 4(4), pp.373–392.
Strauss, A. & Corbin, J., 1998. Basics of Qualitative Research Techniques and Procedures for Developing Grounded Theory. 2nd Edition, London: Sage Publications.
Sun, Z., Deal, B. & Pallathucheril, V., 2009. The land-use evolution and impact assessment model: A comprehensive urban planning support system. URISA Journal, 21(1), p.57.
Sunter, P. & Wigan, M., 2011. Enhancing community participation in metropolitan strategic transport planning through shared analysis. Road and Transport Research, 20(1), pp.2–4.
Sutcliffe, A., 2000. Requirements analysis for socio-technical system design. Information Systems, 25(3), pp.213–233.
The Redlands Institute, 2012. Spatial Decision Support Knowledge Portal. University of Redlands & the Spatial Decision Support Consortium. http://www.spatial.redlands.edu/sds/, accessed on April 5, 2015.
Timmermans, H., 2003. The Saga of Integrated Land Use- Transport Modelling: How Many More Dreams Before We Wake Up? In The 10th International Conference on Travel Behaviour Research. Lucerne, Switzerland.
Tsou, M.-H. & Curran, J., 2008. User-Centered Design Approaches for Web Mapping Applications: A Case Study with USGS Hydrological Data in the United States. In International perspectives on maps and the internet. Springer Berlin Heidelberg, pp.301–321.
United Nations, 2012. Urban Population 1950-2050. Available at: http://esa.un.org/unup/, accessed on November 20, 2012.
Uran, O. & Janssen, R., 2003. Why are spatial decision support systems not used? Some experiences from the Netherlands. Computers, Environment and Urban Systems, 27(5), pp.511–526.
U.S. EPA, 2000. Projecting Land-Use Change: A Summary of Models for Assessing the Effects of Community Growth and Change on Land-Use Patterns. EPA/600/R-00/098. U.S. Environmental Protection Agency, Office of Research and Development, Cincinnati, OH, pp.1–260.
Van Rees, E., 2014. Esri CityEngine 2013. GeoInformatics, 17(2), p.6.
152
Veldhuisen, K., Timmermans, H. & Kapoen, L., 2000. Ramblas: A regional planning model based on the microsimulation of daily travel patterns. Environment and Planning A, 32(3), pp.427–443.
Venturi, G. & Troost, J., 2004. Survey on the UCD integration in the industry. In Proceedings of the 3rd Nordic conference on Human-computer interaction (NordiCHI04). Tampere, Finland: ACM, pp.449–452.
Verburg, P.H. & Overmars, K.P., 2007. DYNAMIC SIMULATION OF LAND-USE CHANGE TRAJECTORIES WITH THE CLUE-s MODEL, pp.321–335.
Verburg, P., Schot, P., Dijst, M. & Veldkamp, A., 2004. Land use change modelling: current practice and research priorities. GeoJournal, 61, pp.309–324.
Vermeeren, A., den Bouwmeester, K., Aasman, J. & de Ridder, H., 2002. DEVAN: A tool for detailed video analysis of user test data. Behaviour & Information Technology, 21(6), pp.403–423.
Vetere, F., Howard, S., Pedell, S. & Balbo, S., 2003. Walking through mobile use: novel heuristics and their application. In Proceedings of OzCHI.
Virzi, R.A., 1992. Refining the Test Phase of Usability Evaluation: How Many Subject Is Enough? Human Factors, 34(4), pp.457-458.
Vonk, G. & Ligtenberg, A., 2010. Socio-technical PSS development to improve functionality and usability — Sketch planning using a Maptable. Landscape and Urban Planning, 94, pp.166–174.
Vonk, G. & Geertman, S., 2008. Improving the Adoption and Use of Planning Support Systems in Practice. Applied Spatial Analysis and Policy, 1(3), pp.153–173.
Vonk, G., Geertman, S. & Schot, P., 2007a. A SWOT analysis of planning support systems. Environment and Planning A, 39, pp.1699–1714.
Vonk, G., Geertman, S. & Schot, P., 2007b. New technologies stuck in old hierarchies: The diffusion of geo-information technologies in Dutch public organizations. Public Administration Review, 67(4), pp.745–756.
Vonk, G., Geertman, S. & Schot, P., 2006. Usage of Planning Support Systems: Combining three approaches. In J. Van Leeuwen & H. Timmermans, eds. Innovations in design and decision support systems in architecture and urban planning. Dordrecht: Springer Netherlands, pp.263– 274.
Vonk, G., 2006. Improving Planning Support. The use of Planning Support Systems for spatial planning. Faculty of Geosciences, University of Utrecht. Available at: https://dspace.library.uu.nl/handle/1874/8576, accessed on March 17, 2017.
Vonk, G., Geertman, S. & Schot, P., 2005. Bottlenecks blocking widespread usage of planning support systems. Environment and Planning A, 37(5), pp.909–924.
Voogd, H., 1983. Multicriteria evaluation for urban and regional planning, Pion Ltd.
Vredenburg, K., Mao, J.-Y., Smith, P.W. & Carey, T., 2002. A survey of user-centered design practice. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 02). New York, USA: ACM, pp.471-478.
Waddell, P., 2010. Integrated Land Use and Transportation Planning and Modelling: Addressing the Challenges in Research and Practice. Transport Reviews2, 31(2), pp.209–229.
153
Waddell, P. & Ulfarsson, G., 2004. INTRODUCTION TO URBAN SIMULATION: DESIGN AND DEVELOPMENT OF OPERATIONAL MODELS. Handbook in Transport2, 5, pp.203–236.
Waddell, P., 2002. UrbanSim: Modeling urban development for land use, transportation, and environmental planning. Journal of the American Planning Association, 68(3), pp.297–314.
Wagner, P. & Wegener, M., 2007. Urban land use, transport and environment models: Experiences with an integrated microscopic approach. disP-The Planning Review, 43(170), pp.45–56.
Walker, D. & Daniels, T.L., 2011. The planners guide to CommunityViz: The essential tool for a new generation of planning. American Planning Association. Chicago, USA: Planners Press.
Wang, H., 2013. A GIS-based Framework for Supporting Sustainable Land Use Planning in Urban Renewal Projects. The Hong Kong Polytechnic University.
Ward, M.O., Grinstein, G. & Keim, D. 2015. Interactive Data Visualization: Foundations, Techniques, and Applications 2nd Ed., Peters/CRC Press.
Wegener, M., 2004. Overview of Land-use Transport Models. In D. A. Hensher & K. Button, eds. Transport Geography and Spatial Systems. Kidlington, UK: Pergamon/Elsevier Science, pp.127– 146.
Wegener, M., 2000. A new ISGLUTI: the SPARTACUS and PROPOLIS Projects. In Second Oregon Symposium on Integrated Land Use and Transport Models. Portland, Oregon, pp.1–32.
Wegener, M., 1994. Operational Urban Models State of the Art. Journal of the American Planning Association, 60(1), pp.17–29.
Wharton, C., Rieman, J., Lewis, C. & Polson, P., 1994. The Cognitive Walkthrough Method: A Practitioner’s Guide. In J. Nielsen & R. Mack, eds. Usability Inspection Methods.
Williamson, W. & McFarland, P., 2012. Investigating the Role of Electronic Planning within Planning Reform. International Journal of E-Planning Research, 1(2), pp.65–78.
Williamson, W.E., 2012. Information and communication technology adoption and use in the New South Wales planning system: a socio-technical approach. The University of New South Wales, Sydney, Australia.
Widjaja, I., Russo, P., Pettit, C.J., Sinnott, R. & Tomko, M., 2015. Modeling coordinated multiple views of heterogeneous data cubes for urban visual analytics. International Journal of Digital Earth, pp.1–21.
Wilson, M., 2014. On the criticality of mapping practices: Geodesign as critical GIS?. Landscape and Urban Planning, 142, pp.226-234.
Wit, A. De, Brink, A. Van Den, Bregt, A.K. & Velde, R. Van De, 2009. Spatial Planning and Geo-ICT: How Spatial Planners Invented GIS and Are Still Learning How to Use It, pp.163–185.
Xie, Y., 1996. A generalized model for cellular urban dynamics. Geographical Analysis1, 284, pp.350– 37.
Yehezkel, D., 1963. The planning process: a facet design. International review of administrative sciences, 29(1), pp.46–58.
Yin, R., 2003. Case study Research: Design and Methods. 3rd Edition, Thousand Oaks: SAGE Publications.
154
Zeile, P., Farnoudi, F. & Streich, B., 2007. LANDSCAPE DESIGN. In Fascination Google Earth–Use In Urban And Landscape Design. Alexandria.
155
Appendices
Appendix A. System Usability Scale (SUS) [Brooke 1986]
156
Appendix B. Coding scheme for observation, screen recording and thinking-aloud [Vermeeren et al. 2002]
Code Short description Definition
Breakdown indication types based on observed actions on the products ACT Wrong action An action does not belong in the correct sequence of actions
An action is omitted from the sequence An action within a sequence is replaced by another action Actions within the sequence are performed in reversed order
DISC Discontinues action User points at function as if to start executing it, but then does not User stops executing action, before it is finished
EXE Execution problem Execution of action not done correctly or optimally REP Repeated action An action is repeated with the same effect CORR Corrective action An action is corrected with a subsequent action (or sequence of actions)
An action is undone STOP Task stopped Starts new task, before having successfully finished the current task
Breakdown indication types based on verbal utterances or on non-verbal behaviour GOAL Wrong goal User formulates a goal that cannot be achieved with the product or that does not contribute to achieving the task goal PUZZ Puzzled User indicates:
not to know how to perform the task or what function is needed for it not to be sure whether a specific action is needed or not
RAND Random actions User indicates: that the current action(s) are chosen randomly
SEARCH Searches for function User indicates: not being able to locate a specific function to be searching for a function of which the analyst knows it does not exist
DIFF Execution difficulty User indicates: having physical problems in executing an action that executing the action is difficult or uncomfortable
DSF Doubt, surprise, frustration User indicates: not to be sure whether an action was executed properly not to understand an action’s effect to be surprised by an action’s effect the effect of an action was unsatisfactory or frustrated the user
REC Recognition of error or User indicates: misunderstanding to recognize a preceding error
to understand something previously not understood QUIT Quits task User indicates:
to recognize that the current task was not finished successfully, but continues with a subsequent task.
157
Appendix C. PSS included in the online resource
PSS
Id Name
1 ALCES
2 Amersfoort
3 Brisbane Urban Growth (BUG) Model
4 California Urban & Biodiversity Analysis (Curba)
5 California Urban Futures I/II Model (CUF-1/2)
6 Chicago Area Transportation Land use Analysis System (CATLAS)
7 CityEngine
8 CLUMondo
9 Common-pool Resources & Multi-Agent Systems (CORMAS)
10 CommunityViz (suite)
11 Computer-Aided Land-Use Transport Analysis System (CALUTAS)
12 Constrained Cellular Automata model
13 Conversion Land Use & its Effects (CLUE/-s)
14 CorPlan
15 Criem/GIS
16 Cube Land
17 DELTA
18 Disaggregated Residential Allocation Model of Household Location & the Employment Allocation Model (DRAM/EMPAL)
19 DSCMODE
20 DSSM
21 Dyna-CLUE model
22 Dynamic Urban Evolutionary Model (DUEM)
23 Environment Explorer
24 Envision
158
25 Envision Scenario Planner
26 Envision Tomorrow
27 EU-ClueScanner (EUCS)
28 EZ-IMPACT
29 FLOWMAP
30 GeneticLand
31 GEOMOD2
32 Geoplanner
33 Growth Simulation Model (GSM)
34 Harvard Urban Development (HUDS)
35 Hedonic pricing model
36 Housing development tool
37 IFS model
38 ILUMASS
39 IMAT
40 INDEX
41 Integrated Infrastructure Planning Tool (IIPT)
42 Integrated Land Use Transportation Environment (ILUTE) Model
43 Integrated Model of Residential & Employment Location (IMREL)
44 Integrated Model to Predict European Landuse (IMPEL) model
45 Integrated Transportation & Land Use Package (ITLUP)
46 I-Place3S
47 IRPUD
48 Irregular City (iCity) & Agent iCity
49 KIM
50 LAND
51 Land Change Modeler
52 Land Transformation Model (LTM)
159
53 Land Use Change Analysis System (LUCAS)
54 Land Use Change (LUC) Model
55 Land Use-based Integrated Sustainability Assessment modelling platform (LUISA)
56 Land Use Scanner Model
57 Land Use Scenario DevelopeR (LUSDR)
58 Landuse Evolution & Impact Assessment Model (LEAM)
59 Large Scale Urban Model
60 LILT
61 LUCI2 Urban Simulation Model
62 Lustre
63 Luti
64 MARXAN
65 MENTOR
66 MEPLAN
67 Metronamica
68 Metropolitan Integrated Land Use System (Metropilus)
69 MetroScope
70 METROSIM
71 MODULUS
72 Moland
73 Multi-Agent-based Behavioral Economic Landscape (MABEL)
74 MUSSA
75 Mutopia
76 NBER
77 NYMTC-LUM
78 Online Envision
79 Online What if?
80 Osaka
160
81 Predicting Urban Population (PUP) model
82 Production, Exchange & Consumption Allocation System (PECAS)
83 Projected Land Use Model (PLUM)
84 Projective Optimization Landuse Information System (POLIS)
85 PUMA
86 Ramblas
87 Random-Utility URBAN (RURBAN) model
88 RapidFire model
89 SALOC
90 SIMLUCIA
91 Slope, Land use, Exclusion, Urban extent, Transportation & Hill shade (SLEUTH)
92 Smart Growth INDEX
93 Smart Places
94 Spartacus
95 Spatial and Transport Emissions Assessment Module (STEAM)
96 Spatial Vision's peri-urban model
97 Sub-Area Allocation Model-Improved Method (SAM-IM)
98 Sustainable Urban Structure & Interaction Networks (SUSTAIN)
99 Technique for Optimal Placement of Activities in Zones (TOPAZ)
100 Transportation Economic & Land Use System (TELUS)
101 Transportation & Land Use System (TRANUS)
102 UGrow
103 UPlan
104 UrbanCanvas
105 Urban Footprint model
106 Urban Housing Growth Model
107 UrbanSim
108 Vacancy Chain Models for Housing Needs & Impact Assessment
161
Appendix D. Content of the Online PSS Resource
PSS Id URL Additional information Application site
1 www.alces.ca -
Australia, Western Australia, Kimberley Region Canada, Alberta India South America
2 - - - 3 - - Australia, Brisbane 4 www.-dcrp.ced.berkeley.edu - USA, California Region 5 www.-dcrp.ced.berkeley.edu - USA, California Bay Region 6 - - USA, Chicago 7 www.esri.com/software/cityengine - Switzerland, Zurich
8 -
Developer Peter Verburg, Sanneke van Asselen, David Eitelberg, VU University Amsterdam, the Netherlands
Contact person Peter Verburg, [email protected]
Description A national to global scale simulation model of land system changes. Instead of representing land cover it is able to simulate conversions of land systems in response to changing demands for agricultural commodities, residential space, but also accounting for demands for ecosystem services. The model is able to simulate both regional and global scale scenarios.
Publication Asselen S, Verburg PH. 2013. Land cover change or land use intensification: simulating
162
land system change with a global-scale land change model. Global Change Biology 19(12): 3648-3667.
State of the PSS The PSS is currently supported and intended for academic purposes.
Available assistance Training material and user manuals are in preparation.
Target user group Planners
Required skills Geographic Information Systems: Advanced Computer programming: No System modelling: Advanced
Application Global scale Laos national scale
Planning task(s) that it targets Strategic planning Impact assessment Global scale assessments/policy
Aspects included in the analysis Land use Environment Ecosystem services
Geographical analysis scale National State/territorial
163
Required data input and format Detailed data on land use, land management and its location factors.
Output and format Grid files
Underlying assumptions See documentation in publication
Methods and techniques used Uncertainty method: sensitivity analysis
Is the PSS customisable? No
Requirements No specific, standalone software
Accessibility The supplier has to be contacted.
PSS evaluation undertaken Validation
Cost Free
Strengths of the PSS A flexible land systems approach for scenario analysis.
Weaknesses of the PSS Data demanding and requires experts to parameterize the model.
9 - - -
10 Developer Placeways, USA
164
Contact person Doug Walker, [email protected]
URL http://placeways.com/communityviz
Description Analyze. Visualize. Engage. These are fundamental tasks of urban planners as they seek to promote informed, collaborative decision-making about the future of cities and regions, large and small. CommunityViz® software is here to help. Working as a seamless ArcGIS® extension, CommunityViz provides an advanced-yet-accessible framework for planners and citizens to learn and make choices about the future of the places they love. Feature-rich, versatile, well established, widely used, award-winning, and famously people-friendly, CommunityViz is the software no planner should be without.
Publication Walker, D., and T. L. Daniels. 2011. The planners guide to CommunityViz: The essential tool for a new generation of planning. Chicago: Planners Press, American Planning Association.
Scott N. Lieske and Jeffrey D. Hamerlinck. Integrating Planning Support Systems and Multicriteria Evaluation for Energy Facility Site Suitability Evaluation. URISA Journal Vol. 26 No. 1.
Peter Pelzer & Gustavo Arciniegas & Stan Geertman & Sander Lenferink. Planning Support Systems and Task-Technology Fit: a Comparative Case Study. Applied Spatial Analysis. Springer.
State of the PSS The PSS is currently supported.
Available assistance A full spectrum of support including built-in help, online help, online tutorials, videos, samples, etc. is provided. A wide variety of online and in-person training options are available. Commercial quality technical support via email, phone and web is available.
Target user group Planners
Required skills Geographic Information Systems: Intermediate
165
Computer programming: No System modelling: Basic
Available case studies USA
See also http://placeways.com/communityviz/casestudies.html
Application It can be applied to any area.
Number of applications 5000
Planning task(s) that it targets Urban management Site selection Strategic planning Impact assessment Build-out and capacity analysis Growth allocation Public engagement
Aspects included in the analysis Land use Transportation Population Employment Environment Economy
Geographical analysis scale National State/territorial Regional and metropolitan
166
Local government Neighbourhood/precinct Parcel
Required data input and format Varies by analysis but typically land use and demographic data in any standard GIS format.
Maximal size of data file No limit
Output and format Live on-screen visual displays GIS feature classes Tabular reports Optional viewer formats for 3D and online apps
Underlying assumptions Varies by analysis. Assumptions are clearly identified and the analyses are designed to be as transparent as possible.
Methods and techniques used Simulation: time series analysis, what-if analysis
Uncertainty method: sensitivity analysis
Commensurate scale generation method: linear scale transformation
Criterion weighting methods: criterion weights aggregation methods, pairwise comparison, ranking, rating, trade-off analysis
Multi-attribute combination method: weighted linear combination
Multi-objective combination method: interactive programming
Is the PSS customisable? Yes
Requirements Windows, Esri ArcGIS Desktop, ArcGIS Online optional
167
Compatibility with other software Exchanges data in standard formats with other models capable of generating tables, databases, geodatabases, etc. Specialized connections to Excel and Hazus.
Accessibility It is available after purchasing.
PSS evaluation undertaken Validation and usability
Cost USD 501 – 3000
Strengths of the PSS Widely used, well established, versatile and feature rich.
11 - - Japan, Tokyo 12 - - -
13 - -
China Costa Rica Ecuador Indonesia, Java Malaysia Netherlands
14
Developer Renaissance Planning Group, USA
Contact person Chris Sinclair, [email protected]
URL www.citiesthatwork.com
Description CorPlan is allows users to develop and allocate place types to create future development scenarios from smaller areas to regions. Place types are the basic building block, and each
-
168
quantifies the amount of building and parking areas by use and the number of people and jobs in those buildings. Place types can reflect any development type. Once composed, place types are allocated to polygons using the ArcGIS select tool. CorPlan maintains a running inventory of the allocated building areas and socioeconomic information.
Target user group Planners
Required skills Geographic Information Systems: Intermediate Computer programming: No System modelling: Basic
Planning task(s) that it targets Site selection Strategic planning
Aspects included in the analysis Land use Transportation Population Employment Environment
Geographical analysis scale Regional and metropolitan Local government Neighbourhood/precinct
Required data input and format A virtual present is created, which includes parcel level land use data, census population, housing and jobs information and environmental layers, such as wetlands.
169
Output and format Building and socioeconomic information by polygon and summarized at differing levels, such as traffic analysis zones for transportation modeling.
Is the PSS customisable (through scripting or open API access)? Yes
Requirements ESRI ArcGIS.
Accessibility The supplier has to be contacted.
PSS evaluation undertaken Validation
Cost Free
Strengths of the PSS Effective and efficient methods of developing land use scenarios
Weaknesses of the PSS Requires ability to conceptualize future plans
15 - - USA, Chicago
16
Developer Dr. Francisco Martinez and researchers at the University of Chile
Contact person Heejoo Ham, [email protected]
URL http://citilabs.com/software/products/cube/cube-land
Description Cube Land forecasts land use and land price by simulating the real estate market under different economic conditions. For a user- defined scenario, Cube Land forecasts the supply and the demand for different types of properties, and estimates the location of
170
households and non-residential activities. Cube Land is an economic land-use forecasting software designed especially for interaction with transportation models and is based upon the MUSSA II framework.
Publication Martínez, F., & Donoso, P. (2004). MUSSA: a behavioural land use equilibrium model with location externalities, planning regulations and pricing policies. Santiago: University of Chile.
Martínez, Francisco; Donoso, Pedro. (2004). MUSSA: a behavioural land use equilibrium model with location externalities, planning regulations and pricing policies. Santiago: University of Chile.
Sujeet Kumar Modi (2015) Development of a Land-Use and Transport Integration Demo-Model in Cube Land for the Munich Region. Master Thesis in Technische Universität München (Germany)
Guglielmo Barè (2013) UN MODELLO DI USO DEL SUOLO ORIENTATO AI TRASPORTI: UN’APPLICAZIONE AL COMUNE DI MILANO. Master Thesis in Politecnico di Milano (Italy)
State of the PSS The PSS is currently supported.
Available assistance User guides Technical support Webinars Standard and bespoke training courses
Target user group Planners
Required skills Geographic Information Systems: No Computer programming: No System modelling: Intermediate
Available case studies Phitsanulok, Thailand Bakersfield, US
171
Twin Cities, US Louisville, US Boston, US Paris, France (only non-residential model) Berlin, Germany (Prototype) City of Panama, Panama (under development) Munich, Germany (Student dissertation) Milan, Italy (Student dissertation)
Application It can be applied to any area.
Number of applications The PSS is being used around the world for both small and large scale applications
Planning task(s) that it targets Urban management Site selection Strategic planning Impact assessment
Aspects included in the analysis Land use Transportation Population Employment Transportation through the direct and internal links with Cube Voyager
Geographical analysis scale National State/territorial Regional and metropolitan Local government
172
Neighbourhood/precinct Parcel
Required data input and format Cube Land processes supply, demand, and space in a disaggregated manner, based on the characteristics that describe: – Activities to be localized – Real estate supply – Location of said activities at real estate properties – Rent values of the resulting land uses
With input and configuration files, you specify the required information and the elements that define the city and market that Cube Land simulates. Specifically, you define and categorize the agents, properties, and zones involved.
Cube Land input files are grouped into three sets: 1- Input files that are used for predicting scenarios: files which generate changes in the city that will lead to changes in real estate market. Files containing total demand, subsidies, supply restrictions, location restrictions, and accessibility are the part of this set. 2- Input files that define market attributes as attributes of the zones, classification of the agents, and classification of the real estate. Cube Land’s demand model allows the inclusion of a large number of variables representing the most relevant attributes of consumers (socioeconomic characteristics), real estate (property types), and the locations (neighbourhoods). 3- Input files that define the Cube Land models: The files of demand model, supply model, rent model, cost adjustment, rent adjustment, and bid adjustment are part of this set. The files belonging to this subset contains all the calibration parameters that are obtained during calibration.
Inputs format are TEXT Files and DBF (Cube binary MAT format can also be used).
Maximal size of data file No limit
Output and format The outputs of the model are bids, real estate values, agent’s location, rent values and housing supply: – zonal endogenous attributes at equilibrium (location externalities) – how many agents are located at equilibrium by agent category, real estate type and zone – rents for each type of property located in each zone
173
– bid for each consumer (households and firms) for each type of real estate and zone – occupied supply of real estate, reached at equilibrium
The outputs are DBF files that can be easily post-processed within the Cube environment to obtain further information in DBF or text format.
Underlying assumptions Main hypothesis are the ones behind the bid-rent theory and the market equilibrium process: – The model used by Cube Land is based on microeconomic consumer theory, which assumes that the system’s agents, whether they are households, firms, or real estate, are rational beings that maximize their benefit. – The auction theory is based on the assumption that each property is allocated to the highest bidding consumer. This buying and selling mechanism is justified in urban economics because location is quasi-unique; that is, suppliers cannot produce more of a location in identical quality to satisfy an increase in demand (as is the case with other goods). Auction theory guarantees that maximum utility is reached when the consumer is the highest bidder. – The supply model is based on maximizing profits. The process consists of supply agents deciding on the amount of each type of real estate to offer in each area so as to optimize their own profits. – The rent model links the supply models and demand models, and assumes that rents are endogenous variables in the Cube Land model, and they are derived from the real estate auction process, representing the highest bid for each property, so that consumer agents are located in the property where their willingness to pay exceeds that of all those bidding on said property.
Methods and techniques used Simulation: agent-based modelling, time series analysis, what-if analysis Uncertainty method: sensitivity analysis
Is the PSS customisable (through scripting or open API access)? The process is not directly customisable but there is high flexibility in terms of configuration of the input data set, parameters, output post processing and interaction with transportation models.
Requirements To run Cube Land, the Cube Base software is required to access the interface. Cube Base comes with a complete transportation GIS built on ESRI’s leading GIS technology.
Operating System: Windows 7 SP1 x64 Professional, Enterprise, or Ultimate; Windows 8 x64 Pro or Enterprise
174
Compatibility with other software Cube Land is fully compatible with other Cube products, and particularly Cube Voyager for Land-Use-Transport-Interaction models. In addition, Cube allows the user to run external programs from within the Cube interface and can therefore link text files directly between external programs and Cube Land.
Generally, Inputs/Outputs from other software can be easily processed in Cube to obtain the Cube Land required formats, allowing high compatibility.
Accessibility It is available online after purchasing it and contacting the supplier. A small demo version is available to download with 30-day trial licence.
PSS evaluation undertaken Validation and usability
Cost > USD 3000
Strengths of the PSS – Commercial support from Citilabs, a software firm with expertise in travel modelling, land use, and GIS – Strong econometric bid-rent formulation simulates the auction of properties to the highest-valued use – Flexible data requirements allow users to design and specify the model best suited to local conditions – Scalable geography and market segmentation can be matched to the resolution of an existing travel model – Transparent estimation process results in robust parameter sensitivities that remain intact after calibration – Automated calibration process is virtually guaranteed to fit base year targets with relatively little effort – Proven equilibrium solution readily lends itself to making valid comparisons between scenario alternatives – Integrated ArcGIS and reporting results in attractive maps, charts, and other output data visualization graphics – Open scripting platform makes it easy to integrate Cube Land with other software and third-party tools – Easy to integrate with existing models/software
Weaknesses of the PSS The complexity level of Cube Land is high because Cube Land is based on a solid microeconomic theory and includes complex components like supply and location externalities and handles multiple types of constraints: system constraints (for example spatial regulations, land capacities) and individual constraints (budget constraints on consumers).
175
17
Developer David Simmonds Consultancy (DSC), UK
Contact person David Simmonds, [email protected]
URL http://www.davidsimmonds.com/index.php?section=33
Description The DELTA package has been developed by DSC since 1995. It allows a range of models to be implemented for a city or region. The models can focus on change within one city, across a region or group of regions, or on a combination of both levels.
DELTA itself is a land-use/economic model, designed to interact with any appropriate transport model in order to create a full model of interactions between land-use, economy and transport (usually known as a land-use/transport interaction or LUTI model). Because land uses and economic activities take time to change, these interactions are modelled over time. DELTA provides land-use or economic inputs to the transport model, which generate demands for transport. The transport model (which may be very elaborate or very simple) provides inputs on travel and transport to DELTA, which influence subsequent changes in the location of households, production and jobs.
DELTA represents a number of distinct processes of urban and regional change, such as household change, migration, business location, etc. Each process is generally the subject of research in economics, urban geography, demography etc . Different processes are modelled at urban and regional levels, reflecting (for example) the differences between the variables affecting the choice of which city region to locate in, and the choice of where within that city region to locate.
Publication See references and links at http://www.davidsimmonds.com/index.php?section=4
State of the PSS The PSS is currently supported.
-
176
Available assistance Projects generally require at least some consultancy input from DSC to advise on the model implementation and calibration.
Target user group Planners
Required skills Geographic Information Systems: Basic Computer programming: No System modelling: No
Available case studies See references and links at http://www.davidsimmonds.com/index.php?section=4
Application Any city or region. The modelled area needs to be defined with care, and should generally be larger than the area in which policies are to be tested.
Number of applications 24
Planning task(s) that it targets Urban management Strategic planning Impact assessment Investigating impacts of transport investment and other policies
Aspects included in the analysis Land use Transportation Population Employment Environment Economy
177
Geographical analysis scale National State/territorial Regional and metropolitan Local government Neighbourhood/precinct
Required data input and format An initial database of households and population, employment, the floorspace they occupy and the rents they pay needs to be compiled, together with matrices of travel costs and times (usually but not always obtained from a transport model). The data needs to be prepared as ASCII files in DELTA-specific formats.
Maximal size of data file DELTA has been applied to models of up to 1300 zones
Output and format The main outputs are tables of CSV files which are input to spreadsheets or mapping/GIS software for analysis and interpretation.
Underlying assumptions The general assumptions are that urban systems need to be analysed as dynamic systems of distinct but interacting processes, the processes representing different kinds of choices made by residents, by firms and by developers – all of them influenced by government interventions. The details within this approach depend on the particular model implemented within DELTA.
Methods and techniques used Simulation: what-if analysis
Multi-attribute combination method: value/utility function method
Multi-objective combination method: value/utility function method
178
Is the PSS customisable? Yes, via definition files that are themselves input to the package
Requirements DELTA itself runs in DOS under Microsoft Windows (XP or later)
Compatibility with other software The CSV output files can be readily used in a wide range of other software.
Accessibility It is available after purchasing
Cost Contact developer
Strengths of the PSS The DELTA package provides the platform for a range of sophisticated models. The strengths of any particular application of DELTA depend very largely on the effort and skill applied to implementing, calibrating and testing it.
18 http://dolphin.upenn.edu/~yongmin/intro.html - USA
19 -
Developer David Simmonds Consultancy (DSC), UK
Contact person David Simmonds, [email protected]
Description DSCMOD was a relatively simple design of land-use model created in 1990-91 and used mainly to test the impact of major transport changes.
-
179
State of the PSS The PSS is no longer supported, but the design can if necessary be implemented in the DELTA package, described elsewhere in this database.
20 - - Thailand, Chiang Mai 21 - - -
22 www.bartlett.ucl.ac.uk/casa/latest/software/duem-ca - USA, Detroit
23 www.lumos.info/environmentexplorer.php -
Canada Indonesia Netherlands
24 - - Australia, Western Australia, City of Canning Australia, Victoria, City of Manningham
25 - - Australia, Melbourne
26
Developer Fregonese Associates Inc., USA
Contact person Alex Steinberger, [email protected]
URL www.envisiontomorrow.org
Description Envision Tomorrow is an open source, scenario planning platform that enables cities to better understand interactions between land use, transportation, housing, energy and water use and public health, to name just a few of the evaluation measures.
State of the PSS The PSS is currently supported.
Available assistance see www.envisiontomorrow.org
180
Target user group Planners
Required skills Geographic Information Systems: Intermediate Computer programming: No System modelling: No Excel: Basic
Available case studies Large-scale regional planning: Salt Lake City UT, Austin TX, Denver CO, Kansas City KS, and across Southern California. Small-scale downtown and district-level plans: South Shore in Austin TX, the Brady District in Tulsa OK, Ogden UT, Bend OR, Portland OR.
Application The PSS can be applied to any area.
Number of applications over 20
Planning task(s) that it targets Urban management Site selection Strategic planning Impact assessment
Aspects included in the analysis Land use Transportation Population Employment Environment Economy
Geographical analysis scale Regional and metropolitan
181
Local government Neighbourhood/precinct Parcel
Required data input and format Envision Tomorrow relies heavily on GIS data, and specifically County Assessor data to understand what areas are built, vacant or constrained in some way. On the built parcels, we need to know what type of development is there, the amount of it and (ideally) the value of the development for redevelopment analysis.
Maximal size of data file Limited by ESRI’s Geodatabase limits, not necessarily by Envision Tomorrow.
Output and format ESRI Geodatabase and Excel
Underlying assumptions Countless, modifiable assumptions exist in any PSS tool. Envision Tomorrow does not bury them in code, but rather allows user to adjust them to calibrate to their unique market or geography within GIS or the Excel sheets associated with Envision Tomorrow.
Methods and techniques used Simulation: time series analysis, what-if analysis
Uncertainty method: sensitivity analysis
Criterion weighting method: ranking, trade-off analysis
Requirements ESRI ArcGIS, Microsoft Excel
Compatibility with other software No
Accessibility It is available online
PSS evaluation undertaken Validation and usability
182
Cost Free
Strengths of the PSS Free and open access. Assumptions are visible and changeable.
Weaknesses of the PSS Not built on a relational database so models interact between ArcGIS Geodatabases and Excel, which can slow down when doing large-scale processes on very large datasets (over 1 million features).
27 - - Europe
28 www.spatial.redlands.edu/sds/ontology/?n=SDSSTool:EZ-IMPACT - -
29
Developer Dept Human Geography and Planning, Faculty Geosciences, Untracht University, the Netherlands
Contact person Dr. Tom de Jong, [email protected]
URL http://flowmap.geo.uu.nl
Description See website
Publication See website
State of the PSS The PSS is currently supported.
Available assistance Manual available on the website.
Target user group Wider public
-
183
Required skills Geographic Information Systems: No Computer programming: No System modelling: No Common sense at academic level.
Available case studies See website
Application It can be applied to any area.
Planning task(s) that it targets Site selection Strategic planning Impact assessment
Aspects included in the analysis Land use Transportation Population Employment Economy
Geographical analysis scale National State/territorial Regional and metropolitan
Required data input and format Shapefiles of activity locations and transport networks.
Maximal size of data file Up to 2Gb
184
Output and format Attribute data in DBF format, vectordata in BNA or MapInfo Export Format (MifMid).
Underlying assumptions Spatial rationality
Methods and techniques used Simulation: what-if analysis
Uncertainty method: sensitivity analysis
Optimisation methods: heuristic algorithms, network optimisation
Is the PSS customisable? Yes, professsional version only
Requirements Windows XP, 7, 8
Compatibility with other software Yes, MapInfo & ArcGIS
Accessibility It is available online.
Cost Free
30 - - Portugal 31 - - - 32 http://doc.arcgis.com/en/geoplanner/ - - 33 www.mdp.state.md.us - USA 34 - - - 35 - - - 36 - - Australia, North-west Melbourne Region 37 - - Global scale 38 www.transport-research.info/web - Germany, Dortmund
185
/projects/project_details.cfm?id=34012 39 - - Australia, Victoria, Brimbank 40 www.crit.com - USA 41 http://vimeo.com/88019947 - -
42 www.civ.utoronto.ca/sect/traeng/ilute/ilute_the_model.htm - Canada, Toronto
43 - - - 44 - - Europe
45 http://people.hofstra.edu/geotrans/eng/methods/flowitlup.html - USA, Austin
46 www.sacog.org/services/I-PLACE3S/ www.sacog.org/services/scenario-planning/
- -
47 www.raumplanung.tu- dortmund.de/irpud/pro/mod/mod_e.htm - Germany, Dortmund
48
Developer Spatial Analysis and Modeling (SAM) Research Laboratory, Simon Fraser University
Contact person Dr. Suzana Dragicevic, [email protected]
URL http://www.sfu.ca/dragicevic/iCity/
Description The novel irregular city iCity series of models are geosimulation approaches and tools developed to represent urban growth processes occurring at a fine cadastral scale using complex systems theory and geographic information systems (GIS). The Agent iCity model with interacting agent components that mimics some human drivers of urban development and having the capability to automatically subdivide land parcels to cadastral lots and roads. The iCity models were developed to potentially assist urban
-
186
planners, land-use managers and policy-makers to generate urban growth outcomes and ‘what-if’ scenarios that can facilitate planning needs.
Publication Stevens, D., Dragicevic, S. and Rothley, K. (2007). iCity: A GIS-CA modelling tool for urban planning and decision making. Environmental Modelling & Software, 22(6):761-773.
Stevens, D., Dragicevic, S. (2007). A GIS-based irregular cellular automata model of land-use change. Environment and Planning B, 34(4):708–724.
Jjumba, A. and Dragicevic, S. (2012). High resolution urban land-use change modeling: Agent iCity Approach. Applied Spatial Analysis and Policy, 5(4):291-315.
State of the PSS The PSS is intended for academic purposes.
Available assistance None
Target user group Planners Geosimulation modelers
Required skills Geographic Information Systems: Advanced Computer programming: Intermediate System modelling: Advanced
Available case studies Please see publications
Planning task(s) that it targets Site selection Strategic planning
Aspects included in the analysis Land use
187
Transportation Population Environment
Geographical analysis scale Neighbourhood/precinct Parcel
Required data input and format GIS data files
Output and format Simulation output maps
Methods and techniques used Simulation: agent-based modelling, cellular automata
Requirements ESRI ArcGIS
PSS evaluation undertaken Validation and usability
Cost Not priced
49 - - - 50 - - -
51
Developer Clark Labs, Clark University, USA
Contact person Stefano Crema, [email protected]
URL http://www.clarklabs.org/
188
Description Land Change Modeler is part of a constellation of tools that are part of TerrSet software. Fully integrated into the TerrSet system, Land Change Modeler is an innovative land planning and decision support software tool. With an automated, user-friendly workflow, Land Change Modeler simplifies the complexities of change analysis. Land Change Modeler allows you to rapidly analyze land cover change, empirically model relationships to explanatory variables, and simulate future land change scenarios. Land Change Modeler also includes special tools for the assessment of REDD (Reducing Emissions from Deforestation and forest Degradation) climate change mitigation strategies. Land Change Modeler provides a start-to-finish solution for your land change analysis needs.
Publication Aguejdad, Rahim and Thomas Houet. “Modeling the Urban Sprawl Using Land Change Modeler on a French Metropolitan Area (Rennes): Forsee the Unpredictable”. Symposium “Spatial Landscape Modelling; From Dynamic Approaches to Functional Evaluations”. Toulouse, France. June 3-5 2008.
Sangermano, F., J.R. Eastman, and H. Zhu. “Similarity weighted instance based learning for the generation of transition potentials in land change modeling.” Transactions in GIS 14, 5 (2010): 569-580.
Wakode, Hemant Balwant, Klaus Baier, Ramakar Jha, and Raffig Azzam. “Analysis of Urban Growth Using Landsat TM/ETM Data and GIS—a Case Study of Hyderabad, India.” Arabian Journal of Geosciences (January 2013).
State of the PSS The PSS is currently supported.
Target user group Planners Wider public
Required skills Geographic Information Systems: Intermediate Computer programming: No System modelling: Intermediate
Available case studies France
India
189
See list of publications.
Application Broad international user community: USA, Central America, South America, Africa, Asia and Australia.
Planning task(s) that it targets Urban management Site selection Strategic planning Impact assessment Reducing emissions from deforestation and forest degradation (REDD)
Aspects included in the analysis Land use Transportation Population Environment Economy
Geographical analysis scale National State/territorial Regional and metropolitan Local government Neighbourhood/precinct Parcel
Required data input and format The inputs are in IDRISI raster format (.rst) but the TerrSet software has a suite of import routines that covers most data formats.
Maximal size of data file Unlimited depending on computer resources.
Output and format The TerrSet software has a suite of export routines that covers most data formats.
190
Underlying assumptions The Land Change modeler is an empirical model where it it takes the historical information and project it to the future.
Is the PSS customisable (through scripting or open API access)? No
Requirements Land Change Modeler in TerrSet: TerrSet is an object-oriented system designed for professional-level use on platforms employing the Microsoft Windows operating environment.
Windows 7 and above, or Windows Server 2003 and above Microsoft ACE 2010 or Microsoft Office 2010 or later 1.3 GB hard drive space for application 7.4 GB for Tutorial data 4 GB RAM, 8 GB or more recommended HD display (1920×1080) or greater recommended
Land Change Modeler for ArcGIS: The Land Change Modeler software is intended for professional-level planning on platforms employing the Microsoft® Windows operating system and the ESRI® ArcGIS® software. Any Windows system that supports ArcGIS 10.2 or later can run Land Change Modeler, although Windows 7 or above recommended. 500 MB of hard disk space is required.
Compatibility with other software There is an extension for ArcGIS 10.2 or later.
Accessibility It is available online after purchase and after contacting the supplier. Trial versions are available.
PSS evaluation undertaken Validation and usability.
Cost USD 501 – 3000
191
Strengths of the PSS Land Change Analysis: Quickly generate graphs and maps of land change, including gains and losses, net change, and persistence of specific transitions. Uncover underlying trends of complex land change with a change abstraction tool.
Land Transition Potential Modeling: Model land cover transition potentials that express the likelihood that land will transition in the future using one of three methodologies-a multi-layer perceptron neural network with full reporting on the explanatory power of driver variables, logistic regression, and SimWeight, a modified machine-learning procedure. Incorporate dynamic variables that drive or explain change.
Change Prediction: Incorporate planning interventions, incentives and constraints, such as reserve areas and infrastructural changes that may alter the course of development when modeling future scenarios. Conduct scenario mapping by creating either a hard prediction map based on a multi-objective land competition model with a single realization or a soft prediction map that is a continuous map of vulnerability to change. Validate the quality of the predicted land cover map in relation to a map of reality through a 3-way crosstabulation. Hits, misses and false alarms are reported.
REDD Analysis: Evaluate REDD related forest conservation strategies and carbon impact scenarios with full GHG emission impact accounting. Assess additionality of REDD projects and business-as-usual projection scenarios.
52 www.ltm.msu.edu - USA 53 www.cs.utk.edu/~lucas - USA
54 - - China Northeast Asia
55
Developer European Commission – DG Joint Research Centre (European Union)
Contact person Carlo Lavalle, [email protected]
192
URL https://ec.europa.eu/jrc/en/luisa http://sa.jrc.ec.europa.eu/?page_id=763
Description The ‘Land-Use-based Integrated Sustainability Assessment’ modelling platform (LUISA) is primarily used for the ex-ante evaluation of EC policies that have a direct or indirect territorial impact. It is based on the concept of ‘land function’ for cross-sector integration and for the representation of complex system dynamics. Beyond a traditional land use model, LUISA adopts a new approach towards activity-based modelling based upon the endogenous dynamic allocation of population, services and activities.
Publication https://ec.europa.eu/jrc/en/publications-list/?f[0]=im_field_identities%3A570
State of the PSS The PSS is currently supported.
Target user group Policy makers
Required skills Geographic Information Systems: Advanced Computer programming: Advanced System modelling: Advanced
Available case studies LUISA is applied to the 28 Member States of the European Union.
Application It can be applied to any area, provided the necessary data is available.
Planning task(s) that it targets Urban management Site selection Strategic planning Impact assessment
193
Aspects included in the analysis Land use Transportation Population Employment Environment Economy Ecosystem services Energy
Geographical analysis scale National State/Territorial Regional and metropolitan
Required data input and format LUISA includes a set of procedures that capture top-down or macro drivers of land-use change (taken from a set of upstream models) and transform them into actual regional quantities of the modelled land-use types. Regional land demands for agricultural commodities are taken from the CAPRI (Common Agricultural Policy Regionalised Impact) model (Britz and Witzke, 2008), which simulates market dynamics using nonlinear regional programming techniques to forecast the consequences of the Common Agricultural Policy. Demographic projections from Eurostat and tourism projections from the United Nations World Tourism Organization (UNWTO) are used to derive future demand for urban areas in each region; land demand for industrial and commercial areas are driven primarily by the economic growth as projected by the Directorate-General for Economic and Financial Affairs of the European Commission (DG ECFIN); and the demand for forest is determined by extrapolating observed trends of afforestation and deforestation rates reported under the scheme of the United Nations Framework Convention on Climate Change (UNFCCC). The demand for the different land-use types is ultimately expressed in terms of acreage and defined yearly and regionally (NUTS2). GIS format
Maximal size of data file No limit
Output and format The final output of LUISA is in the form of a set of spatially explicit indicators that can be grouped according to specific themes (bio-
194
physical, ecological, economic, and social) which as referred to as ‘land function’. The indicators are projected in time until typically year 2030 or 2050, and can be represented at various levels (national, regional or other). GIS format or tabular
Underlying assumptions Depends on the policy case. In the Reference scenario 2014, the economic and demographic assumptions are consistent with the 2012 Ageing Report (EC, 2012). The demographic projections, hereinafter referred as EUROPOP2010, were produced by Eurostat, whereas the long-term economic outlook was undertaken by DG ECFIN and the Economic Policy Committee. The actual economic figures used in LUISA were taken from the GEM-E3 model, which modelled the sector composition of future economy (GVA per sector) consistently with the DG ECFIN’s projections (EC, 2014). Both projections are mutually consistent in terms of scenario assumptions.
Methods and techniques used Simulation: cellular automata, what-if analysis
Uncertainty methods: sensitivity analysis
Criterion weighting method: criterion weights aggregation methods, ranking, rating, trade-off analysis
Multi-attribute combination method: value/utility function method, weighted linear combination
Optimisation method: value/utility function method
Requirements Windows, GeoDMS
PSS evaluation undertaken Validation
Strengths of the PSS EU policy coverage
Weaknesses of the PSS Assumptions and exogenous dependencies
56 www.lumos.info/landusescanner.php - Netherlands
57 www.oregon.gov/ODOT/TD/TP/pages/landuse.aspx - USA, Oregon
195
58 www.leam.illinois.edu/leam www.spatial.redlands.edu/sds/ontology/?n=SDSSTool:LEAM
- USA, City of St.Louis and Preoria Tri-County
59 - - Australia, Queensland Region
60 - - Germany, Dortmund Japan, Tokyo UK
61 -
Developer John R. Ottensmann, United States
Contact person John Ottensmann, [email protected]
Description The luci2 model provides non-specialist users the ability to create and compare scenarios reflecting the effects of alternative assumptions and policy choices on urban development. The model simulates new urban development in grid cells as a function of accessibility to employment, availability of infrastructure, and other factors.
Publication John R. Ottensmann. LUCI: Land Use in Central Indiana Model and the relationships of public infrastructure to urban development. Public Works, Management & Policy 8, 1 (July 2003): 62-76.
John R. Ottensmann. luci2 urban simulation model for generating alternative scenarios. Urban Design and Planning 161, 3 (September 2008): 131-140.
John R. Ottensmann. Accessibility in the luci2 Urban Simulation Model and the importance of accessibility for urban development. In Access to Destinations: Rethinking the Transportation Future, David M Levinson and Kevin J. Krizek, eds. Amsterdam: Elsevier, 2005, pp. 297-324.
John R. Ottensmann, Laurence Brown, Jon Fricker, and Li Jin. Incorporating a land consumption model with a statewide travel model. Proceedings of the 12th TRB National
196
Transportation Planning Applications Conference. Transportation Research Board, Washington, DC, 2009, at http://www.trb-appcon.org/.
John R. Ottensmann and Don Reitz. luci2, scenarios, and the Hendricks County, Indiana, USA, comprehensive plan. In Future Cities and Regions: Simulation, Scenario and Visioning, Governance and Scales, Liliana Bazzanella, Luca Caneparo, Franco Corsico, and Giuseppe Roccasalva, eds. New York: Springer, 2012, pp. 125-146.
John R. Ottensmann and Jamie Palmer. New Model Predicts Growth Patterns in Central Indiana. Indianapolis: Center for Urban Policy and the Environment, 2003.
John R. Ottensmann and Jamie Palmer. LUCI Model Aids Planning for Transportation and Other Infrastructure. Indianapolis: Center for Urban Policy and the Environment, 2004.
Available assistance Model includes comprehensive help system
Complete documentation
Contact John Ottensmann for further information
Target user group Planners Wider public
Required skills Geographic Information Systems: No Computer programming: No System modelling: No
Available case studies See list of publications
Application The PSS can be applied to any area.
197
Number of applications 3 (Central Indiana, State of Indiana, Indianapolis metropolitan area)
Planning task(s) that it targets Strategic planning
Aspects included in the analysis Land use Population Employment
Geographical analysis scale State/territorial Regional and metropolitan
Required data input and format land use at two points in time for model estimation, aggregate amounts for grid cells population employment
Output and format Results displayed on-screen, with two scenarios side-by-side Results optionally exported to csv file which may be joined to shapefile for grid cells (provided)
Underlying assumptions New development, coversion of land from nonurban to urban use depends on forecast population growth for entire area
Factors affecting location of new development will continue as in recent past (as estimated from data)
Methods and techniques used Simulation: what-if analysis
Multi-objective combination method: value/utility function method
198
Is the PSS customisable (through scripting or open API access)? No
Requirements Windows (tested on versions through Windows 7)
Version being developed for Mac OS X
Compatibility with other software No
Accessibility The supplier has to be contacted
Cost Free
Strengths of the PSS Usable by anyone with no prior experience
Allows direct comparison of scenarios reflecting alternative assumptions and policy choices
Relationships estimated using historical data
Weaknesses of the PSS Relatively simple model based on simple assumptions
Limited to the simulation of new urban development, conversion of land from nonurban to urban use
Relatively high degree of aggregation (at least compared with some models) 62 - - USA, Washington, DC. 63 - - UK 64 www.uq.edu.au/marxan - - 65 - - -
199
66 www.meap.co.uk -
Brasil, Sao Paolo Chile Colombia Finland, Helsinki Italy Japan Spain Sweden UK USA, Sacramento and Salt Lake City Venezuela
67 www.metronamica.nl - - 68 - - USA
69 http://metroscope-psu.wikispaces.com /home - -
70 - - USA
71 - - Canada Indonesia Netherlands
72 - - Urban areas across Europe 73 - - - 74 - - Chile, Santiago City
75 www.mutopia.unimelb.edu.au/spatial- platform.html - -
76 - - - 77 - - USA, New York 78 - - Australia
79 Developer Dick Klosterman, USA
200
Contact person Chris Pettit, [email protected]
URL http://aurin.org.au/projects/portal-and-infrastructure/what-if/
Description The online What if? PSS tool has been designed to assist cities and regions across Australia in understanding land use supply, demand and likely future land use change scenarios.
Publication Pettit, C. J., Klosterman, R. E., Nino-ruiz, M., Widjaja, I., Russo, P., Tomko, M., & Sinnott, R. (2013). The online what if? Planning support system. In S. Geertman, F. Toppen, & J. Stillwell (Eds.), Planning support systems for sustainable urban development, Vol. 195, pp. 349–362. Berlin: Springer.
State of the PSS The PSS is currently supported.
Available assistance User manual, training and assistance available on request.
Target user group Planners Wider public Researchers in urban planning
Required skills Geographic Information Systems: Advanced Computer programming: No System modelling: No
Available case studies On request
Application It can be applied to any area.
201
Number of applications 2 (Metropolitan areas of Melbourne and Perth, Australia)
Planning task(s) that it targets Strategic planning Population and employment projection Future residential land demand Future employment-related land demand
Aspects included in the analysis Land use Population Employment Planning policies and strategies
Geographical analysis scale State/territorial Regional and metropolitan Local government Neighbourhood/precinct Parcel
Required data input and format Vector base GIS data – Esri Shapefile format (zip compressed)
Maximal size of data file Tested up to 750,000 polygons
Output and format Future land use, same format as input data
Underlying assumptions Assumptions related to factors and weights in land suitability analysis (MCE), population growth trends, land use densities (residential and employment related), employment sectors growth and spatial growth patterns (not compulsory).
202
Methods and techniques used Simulation: what-if analysis
Alternatives screening methods: compensatory and non-compensatory screening
Criterion weighting methods: ranking and rating
Multi-attribute combination method: weighted linear combination
Is the PSS customisable? Not at the moment, however, it has been developed using open source technologies.
Requirements Internet browser
Compatibility with other software Not applicable
Accessibility It is available online and the supplier has to be contacted.
PSS evaluation undertaken Validation
Cost Contact developer
Strengths of the PSS Project can be accessed by the members of a team, anywhere and anytime.
Weakness of the PSS Relies on heavy data pre-processing.
80 - - -
81 - - Australia, Adelaide and in the eastern part of the South East Queensland Region
82 - - USA, Sacramento 83 - - -
203
84 - - USA, Oakland 85 - - Netherlands, Randstad 86 - - - 87 - - -
88 http://calthorpeanalytics.com/scenario_modeling_tools - -
89 - - -
90 - - Canada Indonesia Netherlands
91
Developer Keith Clarke and colleagues, University of California, Santa Barbara, USA
Contact person Keith Clarke, [email protected]
URL http://www.ncgia.ucsb.edu/projects/gig/
Description SLEUTH is a cellular automaton-based urban growth and land use change model.
Publication Chaudhuri, G. and Clarke, K. C. (2013) The SLEUTH Land Use Change Model: A Review. International Journal of Environmental Resources Research, 1, 1, 88-104.
State of the PSS The PSS is currently supported and intended for academic purposes.
Available assistance See the website and discussion forum.
Target user group Planners
204
Wider public Academics
Required skills Geographic Information Systems: Basic Computer programming: Basic System modelling: Basic Unix
Available case studies See the cited paper, one of three comprehensive reviews.
Application It can be applied to any area.
Number of applications There are over 100 documented applications on all continents except Antarctica.
Planning task(s) that it targets Urban management Site selection Strategic planning Impact assessment
Aspects included in the analysis Land use Transportation Environment Topography
Geographical analysis scale National State/territorial Regional and metropolitan Local government
205
Neighbourhood/precinct Parcel
Required data input and format 8-bit GIF files for slope, land use, exclusions, urban extent and topography.
Maximal size of data file No limit. Supports parallel processing with MPI.
Output and format Statistics, log files, graphic files and animations.
Underlying assumptions Land use change is impacted by slope, urban extent and roads.
Methods and techniques used Simulation: cellular automata, time series analysis
Uncertainty methods: monte carlo simulation, sensitivity analysis
Multi-objective combination methods: genetic algorithm, heuristic algorithms
Is the PSS customisable? Yes
Requirements Any unix, linux or emultor system
Compatibility with other software Giles are simple GIF images, transferable to many GIS packages.
Accessibility It is available online.
PSS evaluation undertaken Validation and usability
206
Cost Free
Strengths of the PSS Works well, free, adaptable.
Weakness of the PSS No social or economic inputs.
92 www.crit.com - USA
93 www.epri.com www.smartplaces.com - USA
94 www.fhwa.dot.gov/planning/toolbox/spartacus_overview.htm -
Finland, Helsinki Italy, Naples Spain, Bilbao
95 -
Developer CSIRO, Australia
Contact person Leorey Marquez, [email protected]
Target user group Planners
Required skills Geographic Information Systems: No Computer programming: No System modelling: Basic
Application Area-specific, it cannot be applied to any area.
Number of applications 1
-
207
Planning task(s) that it targets Impact assessment
Aspects included in the analysis Land use Transportation Population
Geographical analysis scale Regional and metropolitan Local government Neighbourhood/precinct
Required data input and format Land use, transport data Excel files
Maximal size of data file No limit
Output and format Excel files
Methods and techniques used Optimisation methods: heuristic algorithms, linear programming
Is the PSS customisable (through scripting or open API access)? Yes
Requirements Excel for Windows
Cost Not priced
208
96 http://spatialvision.com.au/html/IA/ - - 97 www.all4gis.com - USA
98 -
Developer CSIRO, Australia
Contact person Leorey Marquez, [email protected]
Target user group Planners
Required skills Geographic Information Systems: Basic Computer programming: Basic System modelling: Intermediate
Available case studies None
Application Area-specific, it cannot be applied to any area.
Number of applications 1
Planning task(s) that it targets Strategic planning
Aspects included in the analysis Land use Transportation Population
Geographical analysis scale Regional and metropolitan
-
209
Local government Neighbourhood/precinct
Required data input and format Land use, transport data Text files
Maximal size of data file No limit
Output and format Text files
Underlying assumption Geometric simplification of urban form
Methods and techniques used Multi-attribute combination method: weighted linear combination
Multi-objective combination method: heuristic algorithms
Optimisation methods: linear programming, network optimisation
Is the PSS customisable (through scripting or open API access)? Yes
Requirements Windows
Cost Not priced
210
Strength of the PSS Simplified geometry of urban form
Weakness of the PSS Simplified geometry of urban form
99 -
Developer CSIRO, Australia
Contact person Leorey Marquez, [email protected]
Target user group Planners
Required skills Geographic Information Systems: Basic Computer programming: Basic System modelling: Basic
Application The PSS can be applied to any area.
Number of applications 5 (Australia)
Planning task(s) that it targets Urban management Site selection Strategic planning Impact assessment
Geographical analysis scale Regional and metropolitan Local government Neighbourhood/precinct
211
Required data input and format Land use, transport data Text files
Maximal size of data file No limit
Output and format Text files
Methods and techniques used Simulation: what-if analysis
Multi-objective combination method: heuristic algorithms
Optimisation methods: linear programming, network optimisation
Is the PSS customisable (through scripting or open API access)? Yes
Requirements Windows
PSS evaluation undertaken Validation and usability
Cost Not priced
100 www.telus- national.org/products/telus.htm - USA
101 www.modelistica.com/tranus/ -
Colombia, Bogota Belgium, Brussel Spain, Valencia USA Venezuela, Caracas and La Victoria and island of Curacao
212
102 - - USA
103
Developer University of California, Davis, USA
Contact person Nathaniel Roth, [email protected]
URL uplan.readthedocs.org
Description UPlan is a rule based growth allocation mode. Based on locally defined opportunities and constraints including planning policy, accessibility, environmental conditions, and expert knowledge, new land uses are allocated to the landscape.
Publication Beardsley, Karen, James H. Thorne, Nathaniel Roth, Shengyi Gao, and Michael C. McCoy. “Assessing the Influences of Rapid Urban Growth and Regional Policies on Biological Resources.” Landscape and Urban Planning 93, no. 3–4 (2009): 172–83. doi:10.1016/j.landurbplan.2009.07.003.
Byrd, Kristin B., Adena R. Rissman, and Adina M. Merenlender. “Impacts of Conservation Easements for Threat Abatement and Fire Management in a Rural Oak Woodland Landscape.” Landscape and Urban Planning 92, no. 2 (2009): 106–16.
Gerrard, R., P. Stine, R. Church, and M. Gilpin. “Habitat Evaluation Using GIS – A Case Study Applied to the San Joaquin Kit Fox.” Landsc. Urban Plan. 52, no. 4 (2001): 239–55.
Huber, Patrick R, James H. Thorne, Nathaniel E. Roth, and Michael M. McCoy. “Assessing Ecological Condition, Vulnerability, and Restorability of a Conservation Network Under Alternative Urban Growth Policies.” Natural Areas Journal 31 (July 2011): 234–45. doi:10.3375/043.031.0306.
Johnston, R. A., M. McCoy, M. Kirn, and M. Fell. “Streamlining the National Environmental Policy Act Process through Cooperative Local-State-Federal Transportation and Land Use Planning.” Transportation Research Record 1880 (March 3, 2004): 135–43.
Johnston, R. A., D. R. Shabazian, and S. Y. Gao. “UPlan – A Versatile Urban Growth Model for Transportation Planning.” Transportation Research Record: Journal of the Transportation Research Board, Transportation Research Record, 1831 (2003): 202–9.
213
Johnston, Robert A., Nathaniel Roth, and Jackie Bjorkman. “Adapting Travel Models and Urban Models to Forecast Greenhouse Gasses in California.” Transportation Research Record 2133 (2009): 23–32.
Merenlender, Adina M., Colin Brooks, David Shabazian, Shengyi Gao, and Robert Johnston. “Forecasting Exurban Development to Evaluate the Influence of Land-Use Policies on Wildland and Farmland Conservation.” Journal of Conservation Planning 1, no. 1 (2005): 40–57.
Roth, Nathaniel, James Thorne, Robert Johnston, James Quinn, and Michael McCoy. “Modeling Impacts to Agricultural Revenue and Government Service Costs from Urban Growth.” Journal of Agriculture, Food Systems, and Community Development 2, no. 4 (August 29, 2012): 43–62. doi:10.5304/jafscd.2012.024.008.
Thorne, James H., Maria J. Santos, and Jacquelyn H. Bjorkman. “Regional Assessment of Urban Impacts on Landcover and Open Space Finds a Smart Urban Growth Policy Performs Little Better than Business as Usual.” Edited by Matteo Convertino. PLoS ONE 8, no. 6 (June 5, 2013): e65258. doi:10.1371/journal.pone.0065258.
Thorne, J. H., S. Y. Gao, A. D. Hollander, J. A. Kennedy, M. McCoy, R. A. Johnston, and J. F. Quinn. “Modeling Potential Species Richness and Urban Buildout to Identify Mitigation Sites along a California Highway.” Transport. Res. Part D-Transport. Environ. 11, no. 4 (2006): 277–91.
Walker, T., S. Gao, and R. Johnston. “UPlan: Geographic Information System as a Framework for Integrated Land Use Planning Model.” Transportation Research Record: Journal of the Transportation Research Board 1994 (2007): 117–27
State of the PSS The PSS is currently supported and is intended for academic purposes.
Available assistance User manual Email assistance Training
Target user group Planners
Required skills Geographic Information Systems: Intermediate Computer programming: Basic
214
System modelling: Basic Land use planning
Available case studies San Joaquin Valley Blueprint Calaveras County Amador County Tuolumne County Shasta County Lake County Mendocino County Santa Barbara County
Application It can be applied to any area.
Number of applications approximately 30
Planning task(s) that it targets Strategic planning
Aspects included in the analysis Land use Transportation Population Employment Environment Economy Many are handled through exporting results to other tools such as a travel demand model.
Geographical analysis scale Regional and metropolitan
215
Local government Neighbourhood/precinct
Required data input and format General/comprehensive plans Road networks Environmental constraints Utility service areas City boundaries UPlan is open to the inclusion of any spatial data that the user believes influences the locations of growth. All data in the current version is stored in ESRI GRID format.
Maximal size of data file Limited by ESRI file constraints
Output and format A projection of new land use growth
Underlying assumptions That growth is responsive to land use policy and the proximity to attractive or discouraging features of the landscape.
Methods and techniques used Simulation: what-if analysis Criterion weighting method: rating Multi-attribute combination method: weighted linear combination
Is the PSS customisable (through scripting or open API access)? Yes
Requirements ESRI ArcGIS. The current version of UPlan is a raster-based model using VBA with the Spatial Analyst Extension. It is currently being rewritten into Python with support for polygon based (commonly parcel) tracking of space conversion.
Compatibility with other software Export tools are available to standard interchange formats either built into UPlan, or through ArcGIS.
216
Accessibility It is available online.
PSS evaluation undertaken Validation and usability
Cost Free
Strengths of the PSS Flexible, can be used in data poor areas, and can grow with organization’s capacity.
Weaknesses of the PSS Complex economic interactions, cross-boundary influences on growth. Redevelopment and infill in compact areas.
104 - - - 105 http://calthorpeanalytics.com/ - USA, California 106 www.lesterfranks.com.au/gis.html - Australia, Tansania, Hobart 107 www.urbansim.org - USA
108 -
Developer Philip Emmi (USA) and Lena Magnusson (Sweden)
Contact person Philip Emmi, [email protected]
Description Data on residential mobility is used to calibrate vacancy chain models for (1) urban housing needs assessments and (2) residential development impact assessments. Applications in the USA and Sweden show that the model parameters are stable through relevant forecast periods and produce highly accurate simulations of residential mobility in response to housing demographic and residential inventory changes.
Publication Emmi, P. C. and L. Magnusson. 1995. Opportunity and mobility in urban housing markets. Progress in Planning, 43(1): 1-88.
217
Emmi, P. C. 1995. Further evidence on the accuracy of residential vacancy chain models, Urban Studies, 32(8): 1361-1367.
Emmi, P. C. and L. Magnusson. 1994. The accuracy of residential vacancy chain models, Urban Studies, 31(7): 1117-1131.
Emmi, P. C. and L. Magnusson. 1988. Vacancy chain models of an urban housing market: exercises in impact and needs assessment. Scandinavian Planning and Housing Research, 5(3): 129-145, http://www.researchgate.net/profile/Philip_C_Emmi/publication/232923677.
Emmi, P C. 1990. Testing the assumptions underlying residential vacancy chain models – a comment, Scandinavian Planning and Housing Research, 7: 55-56.
State of the PSS The PSS is intended for academic purposes.
Available assistance Email assistance: [email protected]
Target user group Planners Large residential developers
Required skills Geographic Information Systems: No Computer programming: No System modelling: Intermediate
Available case studies See list of publications, especially Emmi, P. C. and L. Magnusson. 1995. Opportunity and mobility in urban housing markets. Progress in Planning, 43(1): 1-88.
Application It can be applied to any metropolitan housing market area.
218
Number of applications 4 (Sweden, USA)
Planning task(s) that it targets Strategic planning Impact assessment Metropolitan housing needs assessment
Aspects included in the analysis Land use Population
Required data input and format A matrix of residential mobility by characteristics of origin and destination plus a vector of newly built dwellings net demolitions and conversions or vectors of net household migration and new family formation.
Maximal size of data file No limit
Output and format (1) A vector of housing vacancies to be taken up by in-migration, newly formed families, demolitions and conversions. (2) Or a vector of new construction (by generalized housing type/location) required to accommodate expected housing needs.
Underlying assumptions The model assumes that the metropolitan housing market can be intelligently divided in a small number of sub-markets, that the coefficients for inter-sub-market transfer of residential vacancies remains stable for the forecast period (5-15 years) as tests show they do and that either new construction plus outmigration or in-migration plus family formation can be accurately forecast.
Methods and techniques used Simulation: what-if analysis
219
Is the PSS customisable? Since the model relies on matrix inversion, yes, it can be semi-scripted using a spreadsheet.
Requirements A spreadsheet with matrix algebraic capacity.
Compatibility with other software No
Accessibility The supplier has to be contacted.
PSS evaluation undertaken Validation and usability.
Cost Contact developer
Strengths of the PSS Simplicity, accuracy, manageable data inputs and useful outputs.
Weaknesses of the PSS Model accuracy declines with an excessive number of sub-market delineations.
220
Appendix E. Questionnaire of the developer survey
Questionnaire Percentage of respondents
1. What is the name of the PSS you developed [Agarwal et al. 2000]?
100
2. Who developed the PSS [U.S. EPA 2000]? 100
3. Please provide the name and email address of the key contact person [Hughes & Heckbert 2012].
100
4. Please provide the URL of any website related to the PSS [U.S. EPA 2000].
74
5. Please provide a short description of the PSS [U.S. EPA 2000].
79
6. Please provide any publications related to the PSS [Agarwal et al. 2000].
53
7. The PSS is ... (please tick all that apply) [Geertman & Stillwell 2004].
□ ... currently supported. 58
□ ... is intended for academic purposes. 26
8. If any assistance (e.g. user manual, email assistance, training, etc.) is provided, please specify [Hughes & Heckbert 2012].
58
9. Who is the target user group (please tick all that apply) [The Redlands Institute 2012]?
□ Planners 84
□ Wider public 26
□ other (please specify): 26
10. Which of the following types and levels of skills are required for using the PSS [Geertman & Stillwell 2004]?
No
Basic
Intermediate
Advanced 95
Geographic Information Systems (GIS) ☐ ☐ ☐ ☐ 26 21 26 21
Computer programming ☐ ☐ ☐ ☐ 63 21 5 5
221
System modelling ☐ ☐ ☐ ☐ 26 32 21 16
other (please specify): ☐ ☐ ☐ ☐ 5 0 5 0
11. Can you provide information on available case studies [Hughes & Heckbert 2012]?
32
12. Is the PSS area-specific or can it be applied to any area [Geertman & Stillwell 2004]?
area-specific: 11 non-area-specific: 74
13. If the PSS is applicable to any area, what is the approximate number of areas to which the PSS has been applied [U.S. EPA 2000]?
63
14. Which planning task(s) does the PSS target (please tick all that apply) [Pullar & McDonald 1999]?
□ Urban management (including monitoring, controlling and evaluating developments)
42
□ Site selection (i.e. finding the best location for an activity)
58
□ Strategic planning (i.e. exploring and evaluating future options including scenario generation and
l i )
89
□ Impact assessment (i.e. investigating effects of a new development)
63
□ other (please specify): 26
15. Which aspects are included in the analysis functionality of the PSS (please tick all that apply) [The Redlands Institute 2012]?
□ Land use 89
□ Transportation 74
□ Population 74
□ Employment 53
□ Environment 53
□ Economy 37
□ other (please specify)? 37
16. What is the geographical analysis scale (please tick all that apply) [The Redlands Institute 2012]?
□ National 42
□ State/Territorial 53
□ Regional and metropolitan 79
222
□ Local government 63
□ Neighbourhood/Precinct 68
□ Parcel 37 17. What data input is required and what is its format [Geertman & Stillwell 2004]?
95
18. What is the size of data files that the PSS supports [Reviewer]?
74
19. What is the output and its format [U.S. EPA 2000]? 95
20. What assumptions underlie the analysis [Reviewer]?
68
21. Please indicate the methods and techniques used by the PSS (please tick all that apply) [The Redlands Institute 2012]?
Simulation
□ Agent-based modelling 11
□ Cellular automata 16
□ Time series analysis 21
□ What-if analysis 58 Uncertainty methods
□ Error propagation analysis 0
□ Monte carlo simulation 5
□ Sensitivity analysis 37 Alternative screening methods
□ Compensatory screening 5
□ Fuzzy screening 0
□ Non-compensatory screening 5 Commensurate scale generation methods
□ Fuzzy set membership approach 0
□ Linear scale transformation 5 Criterion weighting methods
□ Criterion weights aggregation methods 11
223
□ Pairwise comparison 5
□ Ranking 21
□ Rating 21
□ Trade-off analysis 16 Multi-attribute combination methods
□ Analytical hierarchy process 0
□ Concordance methods 0
□ Preference ranking organisation method for enrichment evaluations
0
□ Fuzzy aggregation operation 0
□ Ideal/Reference point method 0
□ Value/Utility function method 11
□ Weighted linear combination 26 Multi-objective combination methods
□ Compromise programming 0
□ Data envelopment analysis 0
□ Genetic algorithm 5
□ Goal programming 0
□ Heuristic algorithms 16
□ Interactive programming 5
□ Value/Utility function method 11 Optimisation methods
□ Heuristic algorithms 11
□ Linear programming 16
□ Multi-objective combination methods 0
□ Network optimisation 16
□ Value/Utility function method 5
224
22. Is the PSS customisable (e.g. through scripting or open API access) [Reviewer]?
no: 42 yes: 47
23. What platform, software and database (e.g. operation system other than Microsoft Windows, internet browser, GIS) are required for running the PSS [Geertman & Stillwell 2003]?
47
24. Is the PSS compatible with other PSS? If yes, which one [Geertman & Stillwell 2003]?
0
25. How can the PSS be accessed [Hughes & Heckbert 2012]?
□ It is available online (download or web application) 26
□ The supplier has to be contacted 37
□ It is available after purchasing 21
□ other (please specify): 16
26. Has any evaluation of the PSS been done (please tick all that apply) [Reviewer]?
□ Validation 68
□ Usability 47
27. What is the cost of the PSS [Hughes & Heckbert 2012]?
□ Free 37
☐ < USD 500 0
☐ USD 501 - 3000 11
☐ > USD 3000 5
□ Not priced 21
□ Contact developer 16 28. What are the strengths of the PSS [Reviewer]? 74 29. What are the weaknesses of the PSS [Reviewer]? 58 30. If you have any remarks you can provide them here [Reviewer].
0
31. Please specify here other PSS that should be added to the Online PSS Resource [Reviewer].
16
225
Appendix F. Identification method of the PSS included in the online resource
PSS Id Identification method of PSS
Literature search Expert survey Developer survey
1 Hughes & Heckbert 2012
2 Wegener 2000
3
4 Landis et al. 1998
5 Landis 1994
6 Iacono et al. 2008
7 Van Rees 2014
8
9 Le Page et al. 2000
10 Kwartler & Bernard 2001
11 Hunt et al. 2005
12 Hilferink & Rietveld 1999
13 Verburg & Overmars 2007
14 Goodspeed 2013
15 Rodier & Spiller 2012
16 Hughes & Heckbert 2012
17 Bates & Oosterhaven 1999
18 Hunt et al. 2005
19 Bates & Oosterhaven 1999
20 Koomen & Stillwell 2007
21 Lavalle et al. 2011
22 Xie 1996
23 Lantman et al. 2011
24 Glackin 2013
226
25
26 Holway et al. 2012
27 Lavalle et al. 2011
28 Bailey 1982
29 Geertman et al. 2003c
30 Koomen & Stillwell 2007
31 Pontius et al. 2001
32 Mayfield 2015
33 U.S. EPA 2000
34 Wegener 1994
35
36
37 Briassoulis 2003
38 Wagner & Wegener 2007
39
40 Allen 2001
41 Expert survey
42 Salvani & Miller 2005
43 Bates & Oosterhaven 1999
44 Briassoulis 2003
45 Briassoulis 2003
46 Holway et al. 2012
47 Sunter & Wigan 2011
48 Stevens et al. 2007
49 Wegener 2004
50 Wegener 1994
51
52 Lantman et al. 2011
53 U.S. EPA 2000
227
54 Briassoulis 2003
55 Lavalle et al. 2011
56 Hilferink & Rietveld 1999
57 Gregor 2007
58 Sun et al. 2009
59 Stimson et al. 2012
60 Wegener 2004
61 Ottensmann 2005
62 Rodier & Spiller 2012
63 Rodier & Spiller 2012
64 Hughes & Heckbert 2012
65 Bates & Oosterhaven 1999
66 Wegener 2000
67
68 Putman 2001
69 Larsen et al. 2000
70 U.S. EPA 2000
71 Verburg et al. 2004
72 Koomen & Stillwell 2007
73 Lei et al. 2005
74 Hunt et al. 2005
75
76 Wegener 1994
77 Anas 1998
78
79 Pettit et al. 2013
80 Wegener 2000
81 Bell et al. 2000
228
82 Hunt & Abraham 2005
83 Briassoulis 2003
84 Waddell & Ulfarsson 2004
85 Koomen & Stillwell 2007
86 Veldhuisen et al. 2000
87 Wegener 1994
88 Holway et al. 2012
89 Wegener 2000
90 Verburg et al. 2004
91 Clarke et al. 2007
92 U.S. EPA 2000
93 U.S. EPA 2000
94 Lautso 2003
95
96
97 U.S. EPA 2000
98
99
100 Pozoukidou 2006
101 De la Barra 2001
102 U.S. EPA 2000
103 Goodspeed 2013
104
105 Holway et al. 2012
106 Hughes & Heckbert 2012
107 Waddell 2002
108
229
Appendix G. The PSS Evaluation Framework - check-list of activities
PSS Evaluation Framework ✔/✘
1. Determine the evaluation goals 2. Explore the questions and sub-questions
3. Choose the evaluation and data collection methods Inspection methods
Heuristic evaluation User-based methods
User testing Thinking-aloud Questionnaire (e.g. System Usability Scale (SUS) Interviews Observation Screen recording Measures of user performance
4. Identify the practical issues Resources (cost and time)
PSS modul(es) or aspect(s) to evaluate Specific for inspection methods
Inspectors (expert and novice) Inspection criteria (e.g. Nielsen’s heuristics)
Specific for user-based methods Criteria for participants
Number of participants Incentive Between- versus within- subjects design Task- versus scenario-centred test Venue, facilities and equipment
5. Decide how to deal with the ethical issues Plain language statement
Consent form Ethics clearance
6. Evaluate, analyse, interpret and present the data Qualitative versus quantitative data
Evaluation methods and metrics Demonstrate data quality
Reliability Validity
Analyse data Coding scheme Participants’ performance Descriptive statistics Inferential statistics
Present data Pictures Graphics
230
Appendix H. List of potential participants
No
Planning organisation and contact person
(name, address)
Potential participant
(name, email address)
Invitation send
(date)
Responded (yes/no)
Meets criteria
(yes/no)
Accepted invitation (yes/no)
1
2
3
4
5
6
…
231
Appendix I. Examples of free software for recording [Protocollo eGLU 2015]
• The function “screen recording” provided by Apple Quick Time in Mac user system for recording of the screen and of the participant through webcam,
• Screencast-O-Matic (for Windows, Mac and Linux) which can be downloaded from http://www.screencast-o-matic.com.
Other free available screen recording software exists. Attention has to be paid before installing it as some of it might have limited recording capacity, install spyware and plug-ins and alter browser settings.
232
Appendix J. Running sheet
Thanks for your availability to take part in this user test. Your opinion is important to us. The
purpose of this test is to …
The PSS that you will use, allows performing … [short description of the PSS].
You will first … After …, I will ask you to … [specify the procedure of the user test].
I will sit next to you. Do not worry if you do not know how to move on. The purpose of this user
test is to identify problems and solutions to improve the usability of the PSS. It is not about
testing you but the PSS.
The data you will see, covers … [geographical area]. However, this is not relevant for completing
the task(s) / achieving the goal(s).
Please use the PSS as you usually would in your daily work and say out loud what you are
thinking. Audio, video and screen recording will be conducted for enabling later data inspection
and analysis.
If you agree to participate in the user test, please read the plain language statement and sign
the Consent Form.
Here are the tasks / scenarios [indicate on sheet on desk]. Please let me know once you think
that you have completed the task / achieved the goal.
Do you have any questions? [If any, give simple answers. Emphasise that everything will be
clearer once that he/she will interact with the PSS].
At the end: Thank you again for participating. Do you wish that I send you a copy of the findings?
Here is a voucher for your participation.
233
Appendix K. List of participants
No Date Time Participant name Planning organisation
1 2 3 4 5 6 …
234
Appendix L. Documents required during the user test
• a running sheet (for the facilitator), • a consent form (a copy for each participant),
• a plain language statement (only a copy),
• a sheet that lists the task(s) and scenario(s) for the participants (only a copy),
• a sheet for recording participants’ success or failure in completing each task and
achieving the goals as well as for taking notes during the observation as, for example, below:
Observation sheet
No participant: Task/Scenario 1:
Criteria for success:
Task has been completed/goal has been achieved: □ yes □ no Remarks:
Task/Scenario 2:
Criteria for success:
Task has been completed/goal has been achieved: □ yes □ no Remarks:
235
Appendix M. What to do and not do during a user test [Protocollo eGLU 2015]
• do not tell participants what they have to do or not do to complete the task or achieve the goal. Do not answer participants’ questions but ask counterquestions.
• pretend not to know how to use the PSS and ask even simple question to get further explanations such as “what did you do to display the map?” and “what information do you usually display?”.
• ask a question if a participant’s comment is not clear, for example, by paraphrasing what the participant said.
• do not ask questions that can be answered with “yes” or “no”. • after you asked a question, give the participant time to answer.
• try to understand participants’ behaviour and causes of problems.
• if task completion is taking too much time, adopt strategies to make the participant move
on.
236
Appendix N. Plain language statement for user tests
[Name(s) and affiliation(s) of person(s) who conduct the test]
Introduction You have been asked to participate in testing … as you … The aim of this user test is to … This user test has been approved by the Human Ethics Committee and does not involve any risks to you.
What will I be asked to do? [Explain the procedure of the user test] You will be given one goal/multiple goals to achieve with the Planning Support System (PSS) under investigation. With your permission, data collection methods such as … will be applied while you use the PSS. After … you will be asked to ... We estimate that the total time commitment will not exceed … [add time requirement].
How will my confidentiality be protected? We intend to protect your anonymity and the confidentiality of the information provided to the fullest possible extent, within the limits of the law. Your name and contact details will be kept in a separate, password-protected computer file from any data that you supply. This will only be able to be linked to your responses by the test leader(s), for example, in order to know where we should send you a copy of the findings. However, it is possible for data to be subject to subpoena, freedom of information request or mandated reporting by some professions. Depending on the proposal you may need to specifically state these limitations. We will remove any references to personal information that might allow someone to guess your identity; however, due to the small number of participants, it is possible that someone may still be able to identify you through contextual information. The data will be kept securely in … for [number of years].
How will I receive feedback? A summary of the findings will be made available by the test leader(s) upon application. It is also possible that the results will be presented at conferences and meetings.
Will participation prejudice me in any way? Please be advised that your participation in this test is completely voluntary. Should you wish to withdraw at any stage, or to withdraw any unprocessed data you have supplied, you are free to do so without prejudice.
Where can I get further information? Should you require any further information, or have any concerns, please do not hesitate to contact either the test leader(s) on the number given above. Should you have any concerns about the conduct of the test, you are welcome to contact the Executive Officer, Human Ethics of … [provide name of the institution/organisation], on phone: …, or fax: …
How do I agree to participate? If you would like to participate, please indicate that you have read and understood this information by signing the accompanying consent form.
237
Appendix O. Consent form for user tests
Test conducted by:
Name of participant:
1. I consent to participate in this user test, the details of which have been explained to me, and I have been provided with a written plain language statement to keep.
2. I understand that after I sign and return this consent form it will be retained.
3. I understand that this test will involve data collection through [add data collection methods] and I agree that the results may be used as described in the plain language statement.
4. I acknowledge that:
(a) the possible effects of participating in the user test have been explained to my satisfaction;
(b) I have been informed that I am free to withdraw from participating at any time without explanation or prejudice and to withdraw any unprocessed data that I have provided;
(c) I have been informed that the confidentiality of the information that I provide will be safeguarded subject to any legal requirements;
(d) I have been informed that with my consent the data will be stored at … and will be destroyed after … [number of years];
(e) my name will be referred to by a pseudonym; however, due to the small number of participants, it is possible that someone may still be able to identify me through contextual information;
(f) I have been informed that a copy of the findings will be forwarded to me, should I agree to this.
I wish to receive a copy of the findings (please tick) □ yes □ no
Participant signature: Date:
238
Appendix P: Evaluation of participants’ performance
The following table provides an example for evaluating participants’ performance in a user test
[Protocollo eGLU 2015]. It illustrates participants’ overall success, their success per task and the
mean success rate. “1” indicates tasks that have been successfully completed and “0” not
completed respectively.
Task 1 Task 2 Task 3
Participant 1 1 0 0 33%
Success per participant
Participant 2 0 1 0 33%
Participant 3 1 0 1 66%
Participant 4 1 0 1 66%
Participant 5 0 0 0 0%
Participant 6 1 0 1 66%
66% 10% 50% 44% Mean success rate
Success per task
239
Appendix Q. Evaluation of responses to the System Usability Scale (SUS) questionnaire
For calculating the level of participants’ satisfaction the following steps are required
[Protocollo eGLU 2015]:
(1) for the even question numbers (i.e. 2, 4, 6, 8, 10) calculate: 5 – (minus) score assigned
by the participant,
(2) sum up the calculated scores,
(3) multiply the obtained score with 2.5,
(4) repeat the above steps for the questionnaires of all participants,
(5) calculate the mean score of all questionnaires.
The calculation will result in a score between 0 and 100 which will represent the level of
participants’ satisfaction. Based on more than 500 applications the mean score in System
Usability Scale (SUS) questionnaires is 68. Only in 10% of SUS questionnaires a mean score of 80
is achieved. In case of small samples of participants, the result of such an evaluation does allow
little or no generalisation but indicates what could be possible usability problems.
240
Appendix R. Explanation of parameters and their acronyms used in Envision [Glackin 2013]
Property factors Sliderbar
Age of dwelling: the age of dwelling using 2012 as the base year. Positively weights older buildings
Area: size of the lot. Positively weights the size of the lot
Development Efficiency: The ratio of the number of dwellings on a property compared to the number that could be accommodated given the associated R-Code (The maximum dwelling densities that can be achieved on zoned land).
Weights how efficiently the land is currently achieving its set densities
Frontage: The length of street access to a property. Positively weights the amount of frontage that a lot has
LGA Owned: A Boolean indicator of whether or not a property is owned by the local government.
Positively weights dwellings owned by a municipality
Lot Squareness: How square a property is. Positively weights squarer lots
RPI: The Redevelopment Potential Index, calculated by dividing a property’s unimproved land value (ULV) by its capital improved value (CIV). As the index’s value approaches 1.0 it indicates that little value is left in the dwelling and the majority of the value resides in the land itself.
Sensitive Area: A Boolean indicator of whether or not a property is on environmentally sensitive land or occupied by a heritage listed structure.
Negatively weights environmental or culturally sensitive areas
Strata Titled: A Boolean indicator of whether or not a property is strata titled. Negatively weights strata titled lots
Vacant Land: A Boolean indicator of whether or not a lot is vacant. Positively weights vacant land
Zoning: In the Western Australian context, this indicator is represented as the number of dwellings allowed per hectare on a property.
How important zoning is for redevelopment
Demographics Sliderbar
Age 0-19 Positively weights areas with high amounts of young people
Age 20-29 Positively weights areas with high amounts of young adults
Age 30-54 Positively weights areas with high amounts of mature adults
Age 55-74 Positively weights areas with high amounts of middle aged
241
Age 75+ Positively weights areas with high amounts of old people
SEIFA Negatively weights wealthier areas (due to greater resistance)
Location Sliderbar
Nearby Demolitions: The number of demolitions that have occurred within 200 metres of a property within recent years (an indicator of change)
Positively weights the amount of nearby demolitions
Net Increase: The dwelling increase that has occurred within 200 metres of a property within recent years (an indicator of densification)
Positively weights an increase in dwellings within 200m of the property
Proximity to Bus Stop: Linear distance to the nearest bus stop. Positively weights distance
Proximity to Centres: Three indicators measuring the linear distance to the nearest neighbourhood, district and strategic metropolitan centre (or CBD).
Positively weights distance
Proximity to Main Road: Linear distance to the nearest main road. Positively weights distance
Proximity to Park: Linear distance to the nearest local park. Positively weights distance
Proximity to Schools: Three indicators measuring the linear distance to the nearest primary, secondary and tertiary education institution.
Positively weights distance
Proximity to Train Station: Linear distance to the nearest train station. Positively weights distance
PTAL / SNAMUTS: Access level to public transport as indicated by the Public Transport Accessibility Level (PTAL) indicator for CCDs in Victoria and the Spatial Network Multimodal Urban Transport Systems (SNAMUTS) (Scheurer and Silva 2010) indicator for CCDs in WA. Both represent a composite index of walking distance, service frequencies and network connectivity pertaining to all public transport modes. Redeveloping near areas of good public transport access can improve living affordability and sustainability.
Positively weights access to public transport
Relative Density: A ratio of a property’s achieved density per net hectare relative to the average density of other residential properties within 200 metres.
Positively weights underdeveloped lots relative to those around them
242
Relative Extra Land: A ratio of the amount of extra land on a property, in square meters, relative to the average amount of extra land on other residential properties within 200 metres.
Adds importance to the amount of extra (undeveloped) land on lots
Walkability: A composite indicator consisting of a land- use mix (LUM) index, street network connectivity index, and net residential density calculation for street network catchments extending 1600 metres from each property.
Positively weights the walkability of areas
243
Appendix S. Research questions
Goal Question Sub-question
Identify any usability problems of the three PSS
What were the most relevant participants’ experiences (positive/negative) during task execution with the PSS?
At which point of the test execution did these experiences mainly occur?
Which actions/steps caused problems?
Was the user interface easy to navigate?
Was the terminology confusing? Was the feedback provided to users clear?
Are links and buttons clearly visible? Were appropriate icons provided?
Better understand planners’ expectations when interacting with PSS, in order to provide PSS developers with recommendations for designing more usable PSS.
To what extent was the PSS functionality satisfactory? Did the provided functionality make sense to the participants? Did the participants understand the outcome? Did the outcome meet participants’ expectations?
Did the participant know what he/she was actually doing by changing the weighting? Which PSS functionality allowed the participants to perform the task efficiently? Which PSS functionality was well accepted by the participants?
244
Appendix T. UX questionnaire
What were your most relevant experiences while engaging with the PSS?
At which point of the test execution did these experiences mainly occur?
To what extent are the PSS features and functions satisfactory for identifying land suitable for redevelopment?
List the 3 most positive aspects
i.
ii.
iii.
List the 3 most negative aspects
i.
ii.
iii.
245
Appendix U. Expertise questionnaire
Participants’ background and expertise
What is your job function?
In what sector do you usually work? Local Gov’t State Gov‘t Federal Gov’t
Private Sector Other (please specify):
In which state is your work location situated?
VIC NSW SA TAS QLD WA NT
What was the motivation for participating in this test?
I have been given the task by the management
The management stated to require participants and I volunteered We discussed and selected the participant in the group according to workload and
Other (please specify):
Experience
How long have you been working in planning practice?
On a scale of 1-5, with 1 being novice and 5 being expert, … 1 2 3 4 5
… how would you rate your expertise in planning practice?
… how would you rate your expertise level on GIS?
… how would you rate your expertise level on PSS?
Education
What have you studied? What is your background?
General
Gender (m/f)
Do you have any visual impairment? (If yes, please describe the constraints)
246
Appendix V. Questions of interview with professional planners
1. What is your job function?
2. How long have you been working in planning practice?
3. What have you studied? What is your background?
4. What software do you use? Who else does it use it in your organisation?
5. What planning tasks do you use the software for?
6. What are the strengths and weaknesses of the software you use?
7. How do you proceed if you get stuck in using a software tool?
8. Have you attended any courses or training on learning how to use any software tool
during your studies and afterwards?
9. How do you choose what software to acquire?
247
Appendix W. Questions of interview with other planning actors
1. What is your job function?
2. How long have you been doing this for?
3. What have you studied? What is your background?
4. What software do planners use?
5. What planning tasks do planners use the software for?
6. What are the strengths and weaknesses of the software planners use?
7. What is planners’ technical skills level?
8. How do planners proceed if they get stuck in using a software tool?
9. Do planners get any software training?
10. How do you choose what software to acquire?
248
Appendix X. Questions of interview with academic planners
1. Are any courses provided that train planning students to use software?
2. If yes, what is the course about and what are the requirements for attending it?
3. If there are no courses, why? Should software courses be provided in your opinion?
4. Have you developed any software for planning?
5. If yes, what is the software about? Have you involved users when developing it?
Has any evaluation been conducted? Has it been applied in practice?
6. What are trends in PSS development?
249
Minerva Access is the Institutional Repository of The University of Melbourne
Author/s:
Russo, Patrizia
Title:
Usability of Planning Support Systems: analysing adoption and use in planning practice
Date:
2017
Persistent Link:
http://hdl.handle.net/11343/197777
File Description:
Usability of Planning Support Systems: analysing adoption and use in planning practice
Terms and Conditions:
Terms and Conditions: Copyright in works deposited in Minerva Access is retained by the
copyright owner. The work may not be altered without permission from the copyright owner.
Readers may only download, print and save electronic copies of whole works for their own
personal non-commercial use. Any use that exceeds these limits requires permission from
the copyright owner. Attribution is essential when quoting or paraphrasing from these works.