epubs.surrey.ac.ukepubs.surrey.ac.uk/841324/1/20170606 paca_garden_d… · web viewunearthing the...

RUNNING HEAD: Thinking Processes during Garden Design

Unearthing the creative thinking process: fresh insights

from a think aloud study of garden design

Andrew Pringle, Paul T. Sowden

Department of Psychology, University of Surrey, United Kingdom

In press Psychology of Aesthetics, Creativity & the Arts

Author note:

Andrew Pringle, School of Psychology, University of Surrey, Guildford, United Kingdom and the Insight Centre for Data Analytics, University College Dublin. Paul Sowden, School of Psychology, University of Surrey, Guildford, United Kingdom. Both authors contributed to writing this paper with the first draft produced by the first author based on their doctoral dissertation at the University of Surrey. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. We wish to thank Matthew Peacock for his dedicated work as the second coder of verbal protocols and his advice and expertise. We also wish to thank Adrian Banks and Ken Gilhooly for their advice and expertise on the use of think-aloud protocols and Markov chain models. Correspondence concerning this article should be addressed to Andrew Pringle, Insight Centre for Data Analytics, University College Dublin, O’Brien Centre for Science. Belfield, Dublin 4, Dublin, Ireland. Tel.: +353 017162313. E-mail address: [email protected].

1


Abstract

A number of theories of creativity have converged on the idea that creative thinking

entails shifting between different processes. We attempt to build on recent theoretical

developments through empirical work to examine creativity in the everyday

environment of a garden designer. We asked designers with different levels of

expertise, a matched group of fine artists and a non-designer, non-artist control group

to work on a garden design. We asked them to ‘think aloud’ as they designed and we

recorded audio and video. We coded resultant verbal segments as indicating the

operation of different types of underlying thinking process identified in recent

theoretical work. We then mapped these segments to the video of the designs and

conducted Markov chain analysis to explore how thinking processes shifted as the

design evolved. Finally, we examined the extent to which different types of thinking

process shifts predicted the creativity of the final garden designs as determined by

experts. We found that shifts between associative and analytic thinking processes

predicted design creativity, but only when the operation of these two processes were

tightly coupled in time. The positive association between shifting and creativity was

strongest when analytic thinking processed affective content. These types of shifting

were also elevated at times when a subset of participants switched between working

on different designs; a strategy that positively predicted design creativity. Findings

suggest expansion of mode shifting theories of creative thinking to include the

importance of close coupling between different modes of thinking and of an analytic

mode processing affective content.

Keywords: Mode Shifting, Creative Thinking, Design, Think-Aloud Method, Markov

Chain Models

2


A number of theories of creativity have converged on the idea that creative thinking

entails shifting between different processes (e.g. Basadur, Graen & Green, 1982;

Basadur, 1995; Dietrich, 2004; Finke, Ward & Smith, 1992; Gabora & Ranjan, 2013;

Howard-Jones, 2002; Nijstad, De Dreu, Rietzschel & Baas, 2010). These processes

resemble aspects of broader dual-process theories of cognition (Evans & Stanovich,

2013; Frankish, 2010; Stanovich & Toplak, 2012) and recent reviews have critically

examined this similarity and the implications for our understanding of creativity

(Allen & Thomas, 2011; Sowden, Pringle & Gabora, 2015). Most theories of the

creative thinking process propose that creativity requires the generation of ideas that

are then evaluated and/or honed for their intended purpose, with a growing emphasis

that creativity hinges on the ability to shift between different modes of thinking

supporting generative and evaluative activities (Gabora & Ranjan, 2013; Howard-

Jones, 2002; Kaufman, 2011). In fact Kaufman (2011) has argued, “the highest levels

of creativity require…the flexibility to switch modes of thought throughout the

creative process” (p. 458). Further, computational work has developed shifting

algorithms that model the human creative process (Veloz, Gabora, Eyjolfson & Aerts,

2017) and human cultural evolution (Gabora, Chi & Firouzi, 2013). In addition,

laboratory studies (see Vartanian, 2009; Vartanian, Martindale & Kwiatowksi, 2007;

Dorfman, Martindale, Gassimova & Vartanian, 2008; Beaty, Silvia, Nusbaum, Jauk &

Benedek, 2014) and our own recent psychometric work (Pringle & Sowden, 2017)

have provided empirical support for the positive association between creativity and

mode shifting. However, empirical work has yet to look ‘under the hood’ at an

ecologically valid example of human mode shifting to determine if mode shifting

observed in a real-life creative process is linked to the creativity of the product

produced at its conclusion. Thus, the key goal of the present study was to explore this

3


important issue regarding the ecological validity of ideas about mode shifting. To do

so we examined the creative process in garden design.

Looking ‘under the hood’ at the creative process during garden design

We chose garden design as the everyday environment within which to examine the

creative process for a number of reasons. First, design is a recognized area of creative

endeavor requiring mode shifting to generate and evaluate ideas (Cross, 2011; Dorst

& Cross, 2011). Second, designing a garden is a task that can be engaged in by those

without specialist knowledge but where significant expertise and skill can be

developed with training. This was crucial for the present study as both expert and

non-expert groups of participants were included. Third, professional garden designers

are capable of sketching garden designs in a short time period (e.g. within forty-five

minutes) and often have to do so for clients (Fischer-Tomlin, A, personal

communication, 2013). This was important as the design task had to be short enough

that it could be completed within a single session to make it manageable for

participants and those coding the data.

The data were video footage and verbal protocols generated by a ‘think-aloud’

process as participants worked on designing a garden. Further, the finished creative

product produced at the end of the process; that is the individual’s final sketch of their

garden design, was rated by expert judges for its creativity, design quality and for how

closely it met the design brief. This method closely resembles that used in previous

work in this journal examining creativity in visual art (Fayena-Tawil, Kozbelt &

Sitaras, 2011) and related work in the journal Design Studies that commonly makes

4


use of the ‘think-aloud’ method and ‘protocol analysis’ to examine the design process

(e.g. Atman, Chimka, Bursic & Nachtmann, 1999), which we elaborate next.

The ‘Think-aloud’ method and ‘Protocol analysis’

The ‘Think-aloud’ method involves participants continually thinking-aloud their

thoughts as they work on a task, in this case designing a garden. In general, the

‘think-aloud’ method is found not to effect task performance (Ericsson & Simon,

1993). It has been used previously to examine components of creativity, namely

divergent thinking (Gilhooly, Fiortou, Anthony & Wynn, 2007) and insight problem

solving (Fleck & Weisberg, 2004), with no differences in task performance found

between groups completing a task while thinking-aloud compared to a control group

completing a task silently. In the present case, all of a participant’s utterances were

transcribed resulting in a verbal protocol for the entire creative process. The visuals

from the video data were recorded alongside the audio of the verbal protocol. This

verbal-visual protocol was then divided into segments, with a segment defined as

words, phrases or sentences of any length that made up one distinct statement about

something such as an idea or topic (Suwa & Tversky, 1997; Atman et al., 1999;

Gilhooly et al., 2007). Short segments (typically 5 to 10 sec’s) were used to allow a

fine-grained analysis of the timing of shifting between modes of thought. Importantly,

a key feature of the present work was its use of a detailed theoretical framework of

mode shifting to inform the coding scheme that was developed as described next.

5


Theoretical framework of mode shifting and coding scheme

A crucial feature of the present study’s approach is that it allows a critical test

between multiple theories of mode shifting at once to determine which best explains

the empirical data. Theories of mode shifting differ with respect to (1) the number of

different components between which shifts occur (2) the role of affective processing

(3) the degree to which the different component processes are coupled, reflected by

how closely together in time they occur (4) whether the frequency of mode shifting is

important and (5) whether the timing of shifts are important.

In general, dual-process theories of creativity and dual-process theories of cognition

include an associative mode of thinking and an analytic mode (Sowden et al., 2015).

Based on this commonality, the decision was made to pool attributes of thinking

processes across models of creativity (Howard-Jones, 2002; Gabora, 2005) and dual

process models of cognition (Evans & Stanovich, 2013; Frankish, 2010; Kaufman,

2011) in order to identify the operation of associative and analytic modes of thinking

in protocol segments.

Further, most models of creativity that incorporate different modes of thinking only

differentiate between the two modes based on their cognitive characteristics (see

Howard-Jones, 2002; Gabora, 2010; Gabora & Ranjan, 2013; Vartanian, 2009).

However, recent neuroimaging work suggests that affective processes, supported by

default and limbic brain regions, are involved in the evaluation of ideas during a

creative task (Ellamil, Dobson, Beeman & Christoff, 2012) and Dietrich (2004) has

proposed a model of the interplay between different modes of thinking in creativity

6


that does emphasize the need to consider affective processing. Thus, the present work

further coded thinking as with and without affective content in the analyses of mode

shifting. Consequently, coding first identified segments as associative or analytic

before assessing whether each segment contained affective content or not resulting in

four overarching codes for coding the verbal protocols: analytic-cognitive, analytic-

affective, associative-cognitive, associative-affective.

The final coding scheme, with attributes of the different modes of thinking pooled

across the theoretical models discussed (Dietrich, 2004; Evans & Stanovich, 2013;

Frankish, 2010; Gabora, 2005; Howard-Jones, 2002; Kaufman, 2011) is shown in

table 1 with attributes of each mode of thought shown in the ‘segment code’ column

and the theoretical models that attributes are taken from indicated in the ‘source

model(s)’ column. The ‘explanation’ column explains what each attribute is and the

‘example’ column gives an example of this attribute as it appears within verbal

protocols. This approach allows a comparison of the two component models with the

model that additionally separates out affective processes in analytic thinking derived

from Dietrich (2004).

7


Table 1. Displaying the Coding Scheme used to code segments of visual-verbal protocols with either ‘associative’ or ‘analytic’ as well as indicating the further separation of

segments into those with affective content or not.

Associative mode

Segment codeSource model(s) Explanation Example

Generating ideas/concepts 1, 2, 3 Any new idea or elements of new ideas produced ‘what about a stream here’

Developing, thinking through 1 Building new ideas into previous ideas and ‘and I think the stream and path could both meander

& exploring ideas developing existing ideas further and thicken at the apex’

Images, metaphors, analogies 2, 3 Talk concerning visual imagery and use of metaphors ‘the journey through the garden

Linking remote ideas 1, 2 Linking ideas that appear to be disparate ‘a bus makes a journey so I could draw a bus’.

Making associations 1, 2, 4 Making connections between different elements. ‘this is going to be a journey’.

Reasoning based on reference to abstract elements ‘makes me think of drawing into the distance’

Memory retrieval 1 Making associations to knowledge and/or prior experiences ‘this reminds me of the landscape architect

(but not evaluating it/them) George Hargreaves’.

Intuition, instinct, 4, 5, 6 Going with gut instinct/intuition/gut feelings ‘I really feel like this should have a wall to it’.

self-evidently valid

Half-baked/ 1 Things are coming together ‘a journey suggests a flow from one point to another’.

only crudely integrated but it is not clear how they go together

Insight moment 1, 5 Moment of sudden insight ‘Aha I know what I can do here’.

Spontaneous engagement 3 Playfulness and engagement with fantasy ‘I’m playing with shapes, having ideas which are more fantasy

Associative affective 5 Associative thinking that contains affective content ‘I like curvy lines so I’ll put them in’.

Associative cognitive 5 Associative thinking that only contains cognitive content ‘what about a stream here’

Note. Numbers index the following source models: (1) Gabora & Ranjan, 2013; (2) Howard-Jones, 2002; (3) Kaufman, 2011; (4) Evans & Stanovich, 2013; (5) Dietrich, 2004;

(6) Frankish, 2010.

8


Analytic mode

Segment codeSource model(s) Explanation Example

Evaluation of design ideas/concepts 1, 2, 4Evaluating ideas, evaluating in the context of something else ‘that’s working/that’s not working,

(e.g. design brief, expectations) or that’s not going to work within the scale’.

evaluating with reference to reason

Evaluating remembered experiences/ 1, 2 Evaluating remembered info about past design relevant ‘that decision in the past was going against my grain’.

past behaviour experiences

Reasoning justified via logic/evidence 4 Gives evidence/logical argument behind concrete decisions ‘water is a brilliant way in which to unify a site because

it can go on a journey from top to bottom'

Logical deduction 1, 6 Deduction of causal relationships between elements ‘the scale is x metres so this feature will have to be y metres’.

Fixation 2 Adherence to limited set of ideas/stuck in a rut ‘I’m sort of stuck on this idea really’

Planning for future,

with evaluative component 4 Using info from reflection to plan for future ‘this needs further working out, I’d work this out in the future’

Analytic affective 5 Evaluating ideas via affective processing ‘I like/don’t like that’

Analytic cognitive 5 Evaluating ideas via cognitive processing ‘that’s not working’

Note. Numbers index the following source models: (1) Gabora & Ranjan, 2013; (2) Howard-Jones, 2002; (3) Kaufman, 2011; (4) Evans & Stanovich, 2013; (5) Dietrich, 2004;

(6) Frankish, 2010.

9


In addition to coding segments as reflecting the operation of only associative or

analytic thinking, we also identified segments where the operation of different modes

of thinking cannot be clearly distinguished; that is the segment appears to contain

both associative and analytic processes that are tightly coupled together. We labeled

these segments two modes meshed together to reflect the apparent tight coupling

between associative and analytic modes. In meshed segments, analytic evaluations

such as “I can’t…”, “It’s not going to be…” and “it would be really nice…” are often

expressed prior to the evaluated idea actually being introduced suggesting associative-

generative and analytic evaluation are operating closely together in time. In contrast,

in the verbal protocol segments where attributes reflected the operation of only one

mode with adjacent segments coded for a different mode of thinking there was a clear

distinction in the verbal content reflecting a looser coupling between different modes.

The possibility of tight (meshed) coupling between modes is reflected in Nijstad et

al.’s (2010) cognitive flexibility pathway of their dual-pathway model of creativity,

where the operation of an ‘idea monitor’ (an analytic process) continually checks

generated ideas. In contrast, Gabora & Ranjan’s (2013) model implies that shifting

between modes takes more time, with it necessary to disengage one mode of thought

prior to engaging another, or to shift along a continuum between associative and

analytic to enter a different mode. Thus, the additional two modes meshed code

facilitates analysis of the extent to which the different modes of thinking are closely

coupled together in time allowing us to further explore the fit between our data and

different theoretical models. Further, in keeping with Dietrich (2004) and with Ellamil

et al.’s (2012) observation of the role of affective processing when analyzing creative

ideas, we distinguished two modes meshed segments on the basis of whether they

10


contained affective content or not. Examples of ‘two modes meshed together’

segments from participant’s protocols are shown in table 2.

Table 2. Displaying example segments from participants’ verbal protocols coded as ‘two-modes meshed’ with two sets distinguished on the basis of whether or not they contain affective content.

Two modes meshed together Segment code Example

No affective content (i.e. only cognitive)

'Its not going to be curved, because that doesn't work'

'Oooh, it could couldn't it, you could fold the land slightly'

'I don't think I want the terrace or any presumed terrace straight away next to the house'

Containsaffective content

'Ah now I saw something really interesting the other day [hidden hedge] and I think that would be quite fun'

'I really like the idea of this being like a lovely green sort of forest floor underneath this elevated pool'

'I think that would actually be quite nice that we could actually move the water'

Two final points arising from the theories of mode shifting concern the importance of

the frequency and timing of shifts. Nijstad et al.’s (2010) model of creativity

conceptualizes one pathway to creativity as involving cognitive flexibility in the form

of frequent switching between different categories of ideas and approaches with the

concurrent use of an evaluation mechanism, the ‘idea monitor’; to check the

appropriateness of generated responses. Based on this we hypothesized that creativity

would be positively correlated with the frequency of transitions from associative to

analytic and/or the frequency of two modes meshed segments. Models of creativity

also suggest the importance of the timing of shifts between modes during the creative

process, for example to break out from being “stuck in a rut” or overcoming an

impasse (Gabora & Ranjan, 2013; Howard-Jones, 2002; Sowden et al., 2015). Based

on this we examined mode shifting at time points that may be particularly important

to the creative process of designing a garden, namely at points when participants

11


switched between working on different sketches for a design prior to completing their

final design.

The effects of domain specific expertise

A final important issue was to explore the effect of domain specific knowledge and

skill on the creative process and outcomes. Consequently, we included professional

garden designers, student garden designers, fine artists (with therefore highly

developed drawing skills but without garden design specific knowledge) and a group

of university staff who were neither designers nor artists. The latter group was pre-

screened for low levels of creative achievement (Low CAQ group), defined as scoring

low (M= 3.58, SD=2.84) on the creative achievement questionnaire (CAQ) (Carson,

Higgins & Peterson, 2005). All participants were given the same task, to produce a

creative design for a garden based on a short design brief. The rationale for including

different groups was to explore if there were expertise related differences in mode

shifting during the creative process and in the creativity of the designs produced as a

product of this process. Professional garden designers were expected to be most

proficient at mode shifting and to produce the most creative designs followed by

student garden designers and fine artists in the middle, with the low creative

achievement group expected to show the least evidence of shifting and the least

creative designs. It was not clear whether students or fine artists would perform

better, with benefits from the student’s greater expertise in garden design possibly

being balanced out by fine artists having greater expertise in drawing skills.

12


Method

Participants

Forty-eight individuals participated. Twelve were professional garden designers (M=

51.72, SD=7.38, 10 females) recruited from the Society of Garden Designers (SGD),

which is the professional body for garden designers in the United Kingdom. All

twelve were registered members of the SGD, which requires that they pass a strict

accreditation process and have been in business for at least three years. Twelve

participants were student garden designers (M= 39.17, SD=17.21, 8 females),

currently studying on garden design courses or who had graduated from courses in the

year prior to the study’s start date. They were also recruited through the SGD and

colleges running garden design courses. Twelve fine artists (M= 53.50, SD=13.42, 9

females), defined as those who had qualifications in fine art and for which fine art was

currently their profession, were recruited from the Royal Society of British artists

(RBA) and Surrey artists websites and from open studios events held in Surrey in the

UK. The twelve low CAQ group (M= 44.10, SD=13.00, 10 females) were members

of non-academic staff who were recruited in person at the University of Surrey and

one language teacher based outside of the University who was recruited through a

personal contact. The study received approval from the University of Surrey Ethics

Committee. Participants were not compensated financially for their time but were

provided with a summary of the study’s findings once all of the data had been

collected and analyzed.

13


Garden design drawing task

The task required participants to produce a design for a garden on A3 paper within a

period of forty-five minutes. Participants were presented with a brief stating that they

should produce a design for a garden ‘based on a journey and the series of

experiences those who walk around the garden will have on this journey’. The brief

emphasized to make the garden as creative as they could but that it should also be

appropriate and work in the context of the brief (the full brief is available as

supplementary material). The brief was devised with assistance from a lecturer of

garden design at a local college and piloted on a fellow PhD student at the University

of Surrey who was studying on a course of garden design, but was not part of the

sample tested here. This helped ensure that the brief was both clear and had validity as

one that a garden designer might work to. Participants were allowed to sketch the

design for the garden in any way they wished (e.g. plan view, in three-dimensions)

and were allowed to produce as many sketches as they wished. They were given

pencils and equipment if needed or allowed to use their own.

Video recording equipment & video analysis software

A digital Sony high-definition video camera was used to video the process of

designing the garden. The video camera was positioned on a tripod focused on the A3

piece of paper and hands of the designer as they sketched their designs. A software

package called Transana (Woods & Fassnacht, 2012) was used to analyze the audio

and video data captured by the video camera. This package enabled segments in the

video to be linked to segments in the verbal reports produced by participants so that

14


both the video and audio data could be used when coding for attributes of different

modes of thinking within the verbal protocol and for other analyses (e.g. if/when

participants switched between different designs, see results section).

Procedure

Participants completed the garden design task individually in their private design/art

studios, in the studios of design colleges or within a design studio set up within the

School of Psychology at the University of Surrey. Eleven of the twelve members of

the low CAQ group and one member of the group of fine artists completed the session

within the studio at the University. All other participants completed the session in

their own studio or the studio of the design college where they were enrolled. The

session lasted a total of one hour and thirty minutes with the garden design task taking

forty-five minutes and the remainder of the time used for participants to read the

information sheet, give informed consent, practice thinking aloud, set up the video

recording equipment and for de-briefing. After providing informed consent

participants were given instructions to help them to ‘think aloud’ as they worked on

the garden design task (the full instructions are available as supplementary material).

Participants were then given two practice tasks to get them used to thinking aloud.

These were to ‘think-aloud’ while they answered the question “what is the sixth letter

after B?” and to ‘think aloud’ while naming ten animals. Following the ‘think-aloud’

practice participants were presented with the brief for the garden design task and

given 45 minutes to work on it. The experimenter was present in the room while

participants completed the task and answered any questions they had during it. Once

participants had completed the garden design task they were de-briefed about the

15


study. The visual-verbal protocols produced by participants were then coded using

the method described next.

Coding of visual-verbal protocols and inter-coder reliability

One coder, the first author of the paper, coded a total of 13,611 segments across 48

participants in the present study using the coding scheme from tables 1 and 2.

Individual attributes were used as a guide to code segments as either showing the

operation of the associative mode, the analytic mode or both (two-modes meshed

together). If after applying the coding scheme it was still not possible to code a

segment with either one of these three modes then the segment was coded as

‘documentation’. Similarly on occasions participants had to be reminded to continue

‘thinking aloud’ or participants asked questions. For both these and ‘documentation’

segments the mode of thinking operating in that segment was coded as ‘unknown

mode’ to reflect that the mode of thinking operating was unclear from its contents.

A second coder, not involved in the research but with expertise in using coding

schemes, coded 205 segments chosen at random from a range of different participants

from different groups and across different time points of participant’s visual-verbal

protocols. Inter-coder reliability was assessed by Cohen’s kappa, calculated on the

205 segments. Simple agreement was found for 80 % of the segments with the kappa

statistic revealing a level of agreement after adjusting for chance of κ= .62, p < .001,

demonstrating substantial agreement (Landis & Koch, 1977). Disagreements

between coders on the coding categories were discussed with the first coder checking

16


through the coding of modes of thinking across all segments to make sure codes were

applied consistently and any disagreements between coders resolved.

The above reliability check only accounts for the modes of thinking coded based on

two-component models of creativity (e.g. Gabora & Ranjan, 2013; Howard-Jones,

2002). A different strategy was used to check for the reliability of the sub-coding of

whether each segment contained affective content or not. First, one coder coded all

protocol segments as containing affective content or no affective content, labeled as

cognitive content. Words in the associative, analytic and two-modes meshed

segments appearing to reflect affective content were identified. These words were

checked against Warriner, Kuperman & Brysbaert’s (2013) database of norms for the

affective meaning of words in order to provide a validation test of the coder’s

subjective judgment. In this database 13,915 words are rated by individuals on a scale

of 1 to 9 on dimensions of valence, arousal and dominance. On each dimension,

higher ratings indicate more positive affect while lower ratings indicate more negative

affect. Using these ratings, it was possible to determine a mean level of affect, on each

dimension, for the ‘average word’ within the database: valence (M=5.06, SD= 1.68),

arousal (M= 4.21, SD= 2.30) or dominance (M=5.18, SD=2.16). Segments coded as

having affective content only retained these codes if their content included words

rated at least one standard deviation above or below the mean on at least one

dimension of affect. All other segments were classed as only containing cognitive

content and thus coded as associative or analytic cognitive.

17


Garden Design rating task

Three judges, with expertise in garden design and experience in judging at garden

design shows in the UK, rated the designs using Amabile’s (1996) consensual

assessment technique (CAT) on the dimensions of brief, design and creativity/wow

factor. Brief referred to how well the designs met the requirements of the brief, design

referred to the quality of the design that was evident in design sketches. The

creativity/wow factor was the creativity that judges saw evident in the designs. Judges

were asked to keep the criteria of judgment on different dimensions separate. Designs

were rated relative to one another on each dimension rather than against some

absolute standard for garden design. Ratings were given on each dimension on a 1 to

5 point scale with higher numbers indicating higher scores. Judges were presented

with original copies of all sketches of all designs produced by all participants, blind to

which groups produced which designs. Each judge rated designs in a random order,

defined by the experimenter, and was instructed to make full use of the 1 to 5 point

scale when making ratings. Judges were also instructed to go back and review the

ratings they gave to designs that they rated early in the process once they had rated

many of the designs to help ensure consistency of ratings. Agreement between the

three different judges on their ratings of garden designs was assessed using

Cronbach’s alpha (Cronbach, 1951). This analysis showed a high level of agreement

between judges on all dimensions: creativity (α = .80), design quality (α = .87) and

brief (α = .76). In light of this good level of agreement, ratings on each dimension

were averaged across judges and used in subsequent analyses.

18


Results

Between-group differences in the creativity and design quality of garden design

sketches

Prior to examining mode shifting during the creative process, it was first necessary to

establish that the products of this process differed across groups on their rated

creativity and design quality. Given the theorized positive relationship between mode

shifting and design quality it was also important to distinguish groups on design

quality. Even if group differences in mode shifting are found, without concomitant

differences in the creativity of the garden design sketches produced, the argument that

mode shifting during the creative process impacted on creative performance would be

undermined.

One student garden designer was excluded from all subsequent analyses after being

revealed to be an outlier with zero transitions from analytic to associative modes (>

three interquartile range’s (IQR’s) from the bottom of the boxplot on this measure of

shifting for the student garden designer group). Ratings on each dimension were then

compared across groups, with the group means and their associated 95% confidence

intervals displayed in figure 1. Creativity ratings and ratings of design quality were

analyzed using analyses of variance. Ratings on brief were not submitted to further

analyses, as there is no theory concerning how mode shifting is related to how well

creative products meet the requirements of the design brief.

19


Professio

nal Gard

en Des

igners

Student G

arden

Designers

Fine Artis

ts

Low in cr

eativ

e ach

ievem

ent o

n CAQ 0

1

2

3

CreativityDesign QualityBrief

Group

CA

T R

atin

gs

Figure 1. Displaying mean ratings of the creativity, design quality and adherence to brief for garden

design sketches across each of the four groups. Error bars represent 95% confidence intervals.

A one-way independent ANOVA (Group (4) –professional garden designers, fine

artists and student garden designers, low CAQ group) revealed a significant effect of

group on CAT ratings of the creativity of designs (F (3, 43) = 9.91, p < .001,

ηp2= .41). An identical ANOVA conducted on CAT ratings of the design quality of

designs also revealed a significant effect of group (F (3, 43) = 14.51, p < .001, ηp2

= .51). Tukey HSD and Games Howell tests were run to examine group differences

on CAT creativity ratings of creativity and design quality respectively. These

revealed the expected advantage for professional garden designers, with their designs

rated as more creative than those of the low CAQ group (p < .001, r = .77) and

student garden designers (p = .04, r = .45) and having a higher design quality

compared to all three other groups: low CAQ group (p < .001, r = .79), fine artists (p

= .001, r = .69), student garden designers (p = .04, r = .51). The only finding

involving professional garden designers not in line with expectations was that their

designs were only marginally significantly more creative than those produced by fine

20


artists (p = .07, r = .47). Against expectations, student garden designers only

produced designs that received CAT ratings for creativity (p = .07, r = .47) and

design quality (p = .09, r = .49) that were marginally significantly higher than those

designed by the Low CAQ group. In line with expectations, designs produced by fine

artists were rated as more creative than those produced by the low CAQ group (p =

.03, r = .57) but not higher on design quality (p = .20, r = .39). As expected, there

were no differences in CAT ratings on either creativity (p = .99, r = .04) or design

quality between student garden designers and fine artists (p = .60, r = .26), suggesting

their relative strengths in design expertise and drawing ability may have balanced

each other out. Having demonstrated these expected between-group differences in

creativity and design quality, we can now explore whether these differences might be

related to the pattern of mode shifting exhibited by participants in the different

groups.

Although the majority of the most creative and highest quality designs were from the

professional garden designers group (seven out of 12 and eight out of 12 respectively)

some of them came from the student designer and fine artist groups. Thus, group

differences don’t tell the whole story; not all professional garden designers produced

the most creative designs with the highest design quality. Consequently, subsequent

analyses on mode shifting will collapse across groups in addition to examining group

differences in order to fully examine the hypothesized link between mode shifting and

product creativity.

21


Between-group differences in verbal output

It was important to compare the mean length of verbal protocols produced by the four

different groups in order to determine if there were any between group differences in

overall verbal output. Differences in overall verbal output could reflect between

group differences in creative self-efficacy (see discussion section for more details).

The mean length of protocols was calculated for each group in terms of both the total

number of segments coded for in protocols and the length of protocols in minutes.

Professional garden designers produced the longest protocols both in terms of

protocol length, M = 50 minutes, 95% CI [47, 53], and total number of segments, M =

339 segments, 95% CI [304, 374], followed by student garden designers, M= 45

minutes, 95% CI [43, 47]; M= 315 segments, 95% CI [265, 365], fine artists, M= 45

minutes, 95% CI [43, 47]; M= 284 segments, 95% CI [233, 335], with the low

creative achievement group producing the shortest verbal protocols, M= 29 minutes,

95% CI [21, 37]; M= 208 segments, 95% CI [134, 263].

A Group (Professional garden designers, Student garden designers, Fine artists, Low

CAQ) ANOVA conducted on length in minutes revealed a significant effect (F (3, 43)

= 15.06, p < .001, ηp2= .51), as did the identical ANOVA conducted on the total

number of segments (F (3, 43) = 6.84, p = .001, ηp2= .32). Post-hoc Tukey tests

revealed that the mean protocol length of the Low CAQ group was significantly

shorter, both in terms of protocol length and the total number of segments, compared

to the groups of professional and student garden designers (p < .01). There were no

22


other significant between group differences on either measure of protocol length. The

implications of these findings are discussed later (see discussion section).

Evidence for the validity of processes identified in think-aloud transcripts

Prior to the main analyses of mode shifting we sought to ascertain evidence that

processes identified from the think-aloud protocols using the coding scheme reflect

genuine underlying thinking processes. In order to do this we obtained a marker of

the quantity of novel ideas from design sketches that participants produced during the

creative process and correlated this with the frequency of segments coded as

‘generating ideas/concepts’ in participant’s verbal protocols. The marker of the

quantity of novel ideas was the number of additional design sketches a participant

produced that showed the addition of novel features compared to previous sketches.

We would expect the frequency of segments coded as ‘generating ideas/concepts’ in

protocols to positively correlate with the number of design sketches with novel

features, given that both are measures of the quantity of novel features generated. We

did indeed find a positive correlation between the frequency of ‘generating

ideas/concepts’ segments and the number of design sketches containing novel features

(rs= .43, p = .001) thus suggesting genuine underlying thinking processes can be

ascertained from the think-aloud protocols.

Analyzing mode shifting using two-component models of shifting

The coded verbal protocols of participants were first examined for the type of mode

shifting proposed in Gabora and Ranjan’s (2013) and Howard-Jones (2002) two-

23


component models of creativity; that is shifts between associative and analytic modes

of thinking. Shifts were defined as transitions between adjacent segments in a

participant’s verbal protocol, where one segment was coded as associative and the

other coded as analytic. Proficiency in mode shifting was defined here as the

frequency with which a participant transitioned between different adjacent modes of

thinking, from associative to analytic or from analytic to associative, relative to the

frequency with which they did not transition between adjacent modes of thinking, that

is by maintaining an associative (associative to associative) or analytic (analytic to

analytic) mode in adjacent segments. Higher frequencies of transitions between

different modes relative to non-transitions indicate greater shifting proficiency. The

frequency of transitions between adjacent segments where at least one was coded as

‘unknown mode’ was also recorded with these termed ‘unknown transitions’. A

between group (Professional garden designers, Student garden designers, Fine artists,

Low CAQ) ANOVA revealed that there were no systematic differences in unknown

transitions across groups (F (3, 43) = .77, p = .52, ηp2 = .05) and unknown transitions

were also not correlated with either the creativity (rs = - .17, p = .27, N = 47) or

design quality of designs (rs = - .22, p = .14, N = 47). Hence unknown transitions

were excluded from the following Markov chain analyses.

A Markov chain model was used to formally analyze transitions between modes

within a protocol, with modes as categorical events that evolve in a sequence over

time (Kaplan, 2008). An assumption of the model is that the sequence is stochastic,

with the probability of the current categorical event depending only on the categorical

event immediately prior to it (Kaplan, 2008). To illustrate, if events were randomly

distributed then there is a .5 probability that the current mode is associative and a .5

24


probability that it is analytic. There is a .5 probability that the mode immediately

following the current mode is associative and a .5 probability that the mode

immediately following the current mode is analytic. There are thus four possible types

of transition, associative to associative, analytic to analytic, associative to analytic and

analytic to associative. Within the model, the probability of each type of transition

occurring is .25. Transition probabilities thus sum to 1 (Kaplan, 2008). However, in

reality it was expected that the events would not be randomly distributed and that they

would vary between individuals and groups. Thus, taking the example of associative

to analytic transitions:

Transition probability (associative to analytic) = Σ (associative to analytic) / Σ

(associative to analytic + associative to associative)

In words, the transition probability where the mode was associative at time n and

analytic at time n+1 was the ratio of the observed frequency of associative to analytic

transitions out of the total number of transitions in which the start state at time n was

associative. Transition probabilities were calculated separately for shifts between

modes in each direction, from associative to analytic and from analytic to associative

and when the same mode was maintained across consecutive segments: associative to

associative and analytic to analytic. Means for each of the participant groups and

their associated 95% confidence intervals for transition probabilities are shown in

figure 2.

25


Analytic

to Ass

ociativ

e

Associa

tive t

o Analytic

Analytic

to Analy

tic

Associa

tive t

o Associa

tive

00.20.40.60.8

1

Professional GD'sStudent GD'sFine ArtistsLow CAQ

Transition Type

Tran

sitio

n Pr

obab

ility

Figure 2. Displaying group means for the transition probabilities of the four different types of transition. Error bars represent 95% confidence intervals.

Analyses of variance performed to examine group differences in analytic to

associative and associative to analytic transition probabilities failed to reveal any

group differences in mode shifting as did a second set of analyses which collapsed

across groups to examine correlations between participants’ scores for the creativity

and design quality of their designs and each type of transition probability. Thus,

when analyzed through the lens of two component models of the creative thinking

process there was no support for the importance of mode shifting during the creative

process of garden design for the creativity and design quality of designs produced at

the culmination of this process.

Analyzing shifts incorporating the notion of an analytic mode with affective

content

In comparison to two component models of mode shifting, we divided segments into

those containing affective content and those without affective content on the basis that

Dietrich’s (2004) model distinguishes affective and cognitive processing in creativity

26


and that, further, Ellamil et al. (2012) have specifically implicated affective processes

in analytic evaluation during the creative process. Thus we hypothesized that the

frequency of transitions between analytic segments containing affective content

(analytic affective) and associative segments containing no affective content

(associative cognitive) would differ between groups and be associated with the

creativity and design quality of the garden designs. For comparison, we also

conducted the same analyses on the frequency of transitions between analytic

segments containing no affective content (analytic cognitive) and associative

cognitive segments.

Although there was no theoretical basis for predicting differences, we did also

conduct analyses on other types of transition (e.g. associative affective to analytic

affective). Unsurprisingly, these were not significant. We do however also have to

account for these different transitions in the Markov chain model hence all possible

transitions from associative cognitive are included in the denominator of the Markov

chain model (see next).

A Markov chain model was again used to analyze transitions between different

components within the present study’s verbal protocol. Taking the example of

transitions between associative segments with cognitive content, labeled associative

cognitive, to analytic segments with affective content, labeled analytic affective:

Transition probability (associative cognitive to analytic affective) = Σ (associative

cognitive to analytic affective) / Σ (associative cognitive to analytic affective +

associative cognitive to analytic cognitive + associative cognitive to associative

affective + associative cognitive to associative cognitive)

27


In words, the transition probability where the segment was associative cognitive at

time n and analytic affective at time n+1 was the ratio of the observed frequency of

associative cognitive to analytic affective transitions out of the total number of

possible transitions in which the start state at time n was associative cognitive.

Transition probabilities were calculated separately for each of the transitions between

associative cognitive and analytic affective and between associative cognitive and

analytic cognitive in each direction giving a total of four transition types. Means for

each of the participant groups and their associated 95% confidence intervals for the

four types of transition probabilities are shown in figure 3.

Analyt

ic affec

tive t

o Ass

ociat

ive co

gnitiv

e

Assoc

iative c

ognit

ive to

Analy

tic af

fective

Assoc

iative c

ognit

ive to

Analy

tic co

gnitiv

e

Analyt

ic cogn

itive t

o Ass

ociat

ive co

gnitiv

e 0

0.20.40.60.8

1


Transition Type

Tran

sitio

n Pr

obab

ility

Figure 3. Displaying group means for the transition probabilities of the four different types of

transition between associative cognitive and analytic affective (on the left) and between associative

cognitive and analytic cognitive (on the right). Error bars represent 95% confidence intervals.

Comparing group differences across the transition types shown in figure 3 suggests

that the only notable difference between groups was between the low creative

achievement group (Low CAQ) and the three other groups in analytic affective to

associative cognitive transitions. Comparing groups separately on each of the four

28


transition types with Tukey’s HSD tests revealed that the group of professional garden

designers demonstrated elevated Analytic affective to Associative cognitive transition

probabilities compared to the low CAQ group (p = .02) as did the student garden

designers (p = .004). All other differences between groups on Analytic affective to

Associative cognitive transitions and on the other three other transition types shown in

figure 3 were non-significant.

In contrast to the analyses conducted on shifting based on a two component model

these analyses suggest some between group differences in the interplay between

different modes of thinking conceived of as shifts from an analytic mode with

affective content to an associative mode without affective content. Findings revealing

an elevation in this type of shifting within the protocols of professional and student

garden designers compared to the low CAQ group mirror those differences found

between the low CAQ group and both designer groups on the rated design quality of

garden designs and between the low CAQ group and professional designers on the

rated creativity of designs. A final set of analyses, collapsing across groups revealed

that transition probabilities for analytic affective to associative cognitive shifting

positively correlated with garden design ratings of the design quality (rs1 = .28, p =

.03, N = 47) but not the creativity (rs = .15, p = .16, N = 47) of garden designs, thus

mirroring the aforementioned between group analyses showing that expertise in

garden design is positively associated with elevated analytic affective to associative

cognitive shifting.

1 Spearman’s rho correlations were run since the variables were not normally distributed.

29


Meshed Modes of thinking coupled together in time

Thus far, analyses have only examined mode shifting conceptualised as shifts between

adjacent verbal protocol segments coded with one mode or the other; termed shifts in

series. However, an alternative possibility is that the two modes may be tightly

coupled, operating closely together in time. To explore the importance of this

‘meshed’ processing the frequency of two modes meshed together segments was

analysed to examine if there were group differences in the operation of different

meshed modes of thinking. Means for each of three types of two modes meshed

together segments and their associated 95% confidence intervals are shown in figure

4. The type labeled analytic & associative is the measure based on the two-

Analytic

& Associa

tive

Associa

tive c

ognitive &

Analytic

affec

tive

Associa

tive c

ognitive &

Analytic

cogn

itive

0

4

8

12


Type of Two-modes meshed together segment

Two-

mod

es m

eshe

d to

geth

er

segm

ents

Figure 4. Displaying group means for the frequencies of the three different types of meshed segments

within protocols. Error bars represent 95% confidence intervals.

component model of mode shifting and the associative cognitive & analytic affective

and associative cognitive & analytic cognitive are the measures based on

30


distinguishing analytic thinking with affective content from that without (Dietrich,

2004; Ellamil et al., 2012).

The patterns of means show that professional garden designers exhibited a higher

frequency of all three types of two-modes meshed together segments compared to

other groups but there is large variability on each measure within each group. One-

way analyses of variance (Group (4) -Professional garden designers, Student garden

designers, Fine artists, Low CAQ) were performed to examine group differences on

each meshed measure separately. The ANOVA conducted on associative cognitive &

analytic cognitive two-modes meshed together segments only revealed a marginally

significant effect (F (3, 23.36) = 2.40, p = .09, ηp2 = .14) with Tukey HSD tests again

only showing a marginally significant difference between professional garden

designers and fine artists (p = .07). The remaining ANOVA’s were non-significant.

Given the high level of within group variability in two modes meshed together

measures, additional analyses were conducted collapsing across groups with the

frequency of the three measures of two-modes meshed together correlated with the

creativity and design quality of garden designs. Significant positive correlations were

found between associative cognitive & analytic affective two-modes meshed together

segments and creativity (r = .40, p < .01, N = 47) and design quality (r = .34, p < .05,

N = 47) and between associative cognitive & analytic cognitive two-modes meshed

together segments and design quality (r = .27, p < .05, N = 47) but not creativity (r =

.15, p = .10, N = 47).

31


Mode shifting when participants display flexible behavior

The approach taken to examine mode shifting in the present study allows an

examination of the prediction that there may be specific time points during the

creative process when mode shifting is particularly important (Sowden et al., 2015).

We looked to identify an event in the visual-verbal protocol that could signify such

periods of elevated mode shifting. The event we identified was when participant’s

switched between working on different sketches for a design prior to completing their

final design. Such an event would seem to be underpinned by a period of shifting

between modes of thinking, for example in order to move from generating features of

a current design to judging when it is appropriate to start a new design (Sowden et al.,

2015). Participant’s often produced a number of different designs during their

creative process. Such flexibility has been defined in terms of the tendency to switch

between different approaches (Nijstad et al., 2010). The tendency to produce different

designs on the garden design task seems to reflect switching between different

approaches and would therefore appear to be a measure of flexibility (Plucker, J,

personal communication, 2013). It should be noted here that there is evidence from

the visual-verbal protocols (see supplementary material) that participant’s switched to

working on a new design because they reached an impasse or believed they could

come up with a better idea if they switched to working on a different design on

another sheet of paper. The evidence does not suggest they switched between

working on different designs simply because they finished their first design early.

This is perhaps best exemplified by participant, ID 13, commenting when switching to

work on a different design at five minutes into the design session “ah, you know could

be something better, don’t get stuck on one thing to start with”. Further, participant ID

32


3 abandoned working on a new design to return to their first design (see figure 5 and

the supplementary materials) suggesting they reached an impasse. Two measures of

flexibility were obtained: (1) a dichotomous measure of whether participants had

worked on the same or different designs from start to finish and (2) a measure of the

total number of different designs participants produced. The criteria used to define

different designs were that they must be wholly distinct, for example a garden with

curves versus a rectilinear garden. Using similar criteria to those used by Kozbelt

(2008), sketches that merely included the addition of some additional novel features

or attempts to make the designs neater were not coded as new designs.

Prior to examining mode shifting during switches between different designs it was

first important to demonstrate that such instances of switching between different

designs were productive; that is they were positively associated with the creativity of

the final garden designs produced. Correlations performed at the level of the whole

sample (N = 47) revealed a significant positive relationship between the total number

of different designs produced and ratings of both creativity (rs = .43, p = .001) and

design quality (r s = .46, p = .001) for final designs. Additionally, grouping

participants into those who had worked on the same design from start to finish (N =

39) versus those who had worked on different designs (N = 8) revealed that more

creative final designs were produced by the group that switched between working on

different designs (M = 3.10, 95% CI = 2.24, 3.97) compared to the group that

remained working on the same design throughout (M = 1.92, 95% CI = 1.65, 2.20, F

(1, 45) = 11.85, p = .001, ηp2 = .21). Similarly, the group that switched between

working on different designs also produced designs with higher design quality (M =

2.83, 95% CI = 1.97, 3.70) compared to those working on the same design throughout

33


(M = 1.55, 95% CI = 1.30, 1.80, U2= 49, p = .002, r = -.44). In sum, these analyses

show that switching between different designs was linked to superior ratings of the

creativity and design quality of designs produced.

The time windows in verbal protocols when participants, who worked on different

designs, switched from working on a current design to starting a new design were

identified. Each time window was examined for evidence of shifts between modes of

thinking previously coded for in the protocol. Evidence of shifting between modes of

thinking within these windows in the verbal protocol would provide support for the

prediction that the timing of shifts is associated with flexibly switching between

different design approaches and thus in turn with the creativity and design quality of

final designs produced.

An example of one instance of switching between different designs by a professional

garden designer is shown in figure 5 for illustration purposes. The full set of instances

of switching between different designs across all participants is shown in the

supplementary materials. Some participants produced more than two designs and each

instance of switching between designs (e.g. design 1 to design 2, design 2 to design 3

etc.) is displayed in a separate diagram, on a separate page in the supplementary

materials. Figure 5 displays a timeline showing the segments from a participant’s

verbal protocol shortly before, during and after they stopped working on one design

and started working on a new design. The timeline starts at the point of the first

utterance of the first verbal protocol segment within the time window. Displayed on

2 A Mann-Whitney test was used to compare groups on design quality since ratings of design quality were not normally distributed.

34


Figure 5. Displaying one instance of switching between different designs, in this instance performed

by a professional garden designer.

the timeline are time stamps at five-second intervals with increasing time into the

design session/verbal protocol going from left to right. The point at which participants

stop working on one design and the point at which they start work on their next design

are indicated by the arrows on the timeline with the corresponding time stamps in

brackets. Photos of the two designs are displayed above the arrows. In figure 5, the

reason behind labeling the transition from design 1 to design 3 rather than 2 was

because this participant abandoned their second design to return to working and

elaborating on their first design (see also the figure on page 2 of the supplementary

material, which shows design 1 before it was elaborated as well as design 2) after

which they switched to working on a novel third design. The point at which

participants stopped working on a design was defined as the point at which they lifted

35


their pencil from the paper and stopped sketching that design. The point at which they

started working on a design was defined as the point when they put pencil to paper

and began sketching a new design.

Individual segments of the verbal protocol are represented by the colored bars with

the verbal content of each segment shown at the bottom of the figure. The attribute

code or codes given to each segment are shown in the column to the left of the

timeline. The colored bars to the left indicate the relevant modes of thinking. Shifts

between components of thinking are thus indicated by color changes across

consecutive segments. Two-modes meshed together segments are represented by a

pair of different colored bars at the same position in the vertical plane on the timeline

(see figures on pages 11, 19 and 23 of the supplementary material for examples).

The figures in the supplementary material show across the eight participants who

produced different designs there were a total of 24 instances when switches between

different designs were made. On 21 out of the 24 instances when participants

switched designs they evidenced at least one shift between the different thinking

components defined based on the separation out of affective content (Dietrich, 2004;

Ellamil et al., 2012). When the different modes were defined based on the simpler,

two-component model of mode shifting, participants shifted between different modes

on 20 out of the 24 instances when they switched designs.

It was necessary to formally compare the frequency of shifting between modes of

thinking during the periods in the protocol when participants switched between

working on different designs to the frequency of shifting when participants were

working on the same design. In order to do this the frequency of transitions based on

36


the Markov chain model were calculated within a time window when participants

switched between working on different designs. Time windows within verbal

protocols were calculated from 30 seconds downstream of the point when participants

stopped working on one design to 30 seconds upstream of the point when participants

started working on a different design. Thus protocol segments that fell within this

time window were captured. This time window was chosen because it was wide

enough to ensure that all the transitions between different modes of thinking when

participants switched between different designs were captured. Given that previous

work (Ellamil et al., 2012) has emphasized the potential importance of affect in

analytic-evaluative thinking and that this was supported by our earlier analyses, here

we focus on Markov chain analyses that compare shifts between associative cognitive

and analytic affective with those between associative cognitive and analytic cognitive

thinking.

Chi-square tests were run to compare the total frequency of shifting within time

windows to the total frequency of shifting outside of these time windows. The latter

measure was the frequency of shifting displayed by the eight participants who worked

on different designs outside of the time windows summed together with the frequency

of shifting displayed by the remainder of participants (N = 39) who worked on the

same design throughout the garden design task. Within time windows, counts were

produced of the number of each of the two types of transition based on two

component models of mode shifting and transitions based on also coding for analytic

affective content. Transition types that resulted in a count of less than five in any of

the cells were excluded from analysis in order not to violate a key assumption of chi-

square (Field, 2009). For the same reason, transitions were collapsed across

37


direction, for example with transitions from analytic affective to associative cognitive

combined with those from associative cognitive to analytic affective. The number of

two modes meshed together segments within time windows was also calculated.

These measures of the frequency of shifting were each summed to produce totals

across all time windows across all of the participants who switched between working

on different designs. Unknown transitions were also included in the analyses because

we wanted to ensure effects reported here remained after accounting for any

differences in unknown transitions within compared to outside of time windows.

A two (within time windows, outside of time windows) by three (shifts between

analytic affective & associative cognitive, shifts between analytic cognitive and

associative cognitive, unknown transitions) chi-square was run to compare the

frequencies of different transition types within and outside time windows. The chi-

square revealed that there was a significant difference in the pattern of transition

frequencies when they were obtained within compared to outside of time windows (χ2

(2) = 30.67, p <.001; see table 3 for cell counts).

Table 3. Displaying the observed (count) and expected frequency of the different types of Markov

chain transitions based on Dietrich’s (2004) framework within and outside time windows.

Within time windows

Outside timewindows

Transition typeCoun

tExpecte

d % Count Expected %Analytic affective-Associative cognitive 27 11 11 516 532 4Analytic cognitive- Associative cognitive 54 41 21 1908 1921 16Unknown Transitions 174 203 68 9557 9528 80

38


Standardized residuals were used to explore the significant chi-square, with

significant differences between observed and expected counts revealed by z-scores

greater than 1.96/-1.96. Within time windows, there was a significantly higher

observed versus expected frequency of shifts between Analytic affective and

Associative cognitive modes (z = 4.7, p < .001) and between Analytic cognitive and

Associative cognitive modes. (z = 2.1, p < .001). It is also important to note that

within time windows there was a significantly lower observed versus expected

frequency of unknown transitions (z = -2.0, p < .01). There were no significant

differences between any observed and expected counts for any transition type outside

time windows.

Table 4. Displaying the observed and expected frequency of two-modes meshed together segments based on two component models and Dietrich’s (2004) framework within and outside time windows.

Within time windows

Outside timewindows

Type of Meshed segment CountExpecte

d % Count Expected %Associative & Analytic (from two component model) 12 6 5 292 298 2Analytic affective & Associative cognitive 4 2 2 73 76 1Analytic cognitive & Associative cognitive 8 4 3 219 223 2Non meshed segments 249 255 95 13170 13164 98

In addition, to exploring the transition frequencies we also assessed the simple

frequencies of the different types of meshed segments. A two (within time windows,

outside of time windows) by two (meshed segments, non-meshed coded segments)

chi-square was run to compare the frequency of two-modes meshed together segments

based on the two-component model of mode shifting (i.e. associative & analytic)

within and outside time windows. The chi-square revealed a significant difference in

the proportion of two-modes meshed together segments versus non two-modes

39


meshed together segments within compared to outside of time windows (χ2 (1) = 6.97,

p = .008; see Table 4 for cell counts). Within time windows, there was a significantly

higher observed versus expected frequency of two-modes meshed together segments

(z = 2.6, p < .001). There were no significant differences in the observed versus

expected frequency of non-meshed segments either within (z = -.4) or outside (z = .1)

time windows.

A two (within time windows, outside of time windows) by three (analytic affective &

associative cognitive meshed segments, analytic cognitive & associative cognitive

meshed segments, non meshed segments) chi-square was run to compare the

frequency of two-modes meshed together segments based on the model that

distinguishes affective from non-affective content within and outside time windows.

The assumption that at least 25 % of cells in the chi-square should have expected

counts of five or more was violated but cells do reach Everitt’s (1977) criterion of

expected values being greater than 1 giving the analysis credibility. The chi-square

revealed a significant difference in the pattern of frequencies of these different types

of segments when they were obtained within compared to outside of time windows (χ2

(2) = 7.83, p = .02; see Table 4 for cell counts). Within time windows, there was a

significantly higher observed versus expected frequency of analytic affective &

associative cognitive meshed segments (z = 2.1, p < .05). Within time windows, the

difference between observed and expected frequencies of meshed analytic cognitive

& associative cognitive segments was not significant (z = 1.80).

40


Discussion

This was the first work to explore the ecological validity of ideas about mode shifting

by examining if mode shifting during the creative process outside of the laboratory is

linked to the creativity of the product produced at its conclusion. Use of the think-

aloud method in conjunction with video data and a detailed protocol analysis of

participants designing a garden in real-time provide a richer insight into mode shifting

than is typically possible using existing laboratory (e.g. Vartanian, 2009; Vartanian et

al., 2007; Dorfman et al., 2008; Beaty et al., 2014) or psychometric (Pringle &

Sowden, 2017) approaches. Importantly, this study allowed a first test of the

hypothesis that proficiency in mode shifting during the creative process is associated

with the creativity of the products, namely final garden designs, produced at the end

of that process. It is necessary to demonstrate this link in order to show the practical

value of research on mode shifting; namely that shifting could impact on the creativity

of the product produced at the end of the creative process.

Results do suggest a link between proficiency in mode shifting during the creative

process of garden design and final design creativity and design quality, but critically,

this relationship hinges on how proficiency in mode shifting is conceptualized. When

proficiency in mode shifting is conceptualized, according to two component models

of creative thinking (Howard-Jones, 2002; Gabora & Ranjan, 2013; Vartanian, 2009)

as the frequency of shifts between adjacent associative and analytic modes during the

garden design process there was no evidence of any relationship between mode

shifting and final design creativity or design quality. When shifting proficiency was

41


conceptualized according to shifts between Dietrich’s (2004) analytic mode operating

on affective content and an associative mode operating on non-affective, cognitive

content then a relationship between mode shifting and final garden design ratings of

design quality emerged. Further, when proficiency in mode shifting was

conceptualized as the frequency of two modes of thinking meshed together operating

closely together in time (Sowden et al., 2015) then a relationship between mode

shifting and garden design ratings of both creativity and design quality emerged.

Results also revealed elevated mode shifting at key time points in the design process

when participants demonstrated flexibility by switching between working on different

designs; a behavior that appears to be underpinned by mode shifting. Importantly,

switching between different designs was also positively associated with the

production of more creative designs with a higher design quality. Thus mode shifting

during these time points may be another index of shifting proficiency linked to

creative performance and design quality.

The majority of models of the interplay between different modes in creative thinking

conceptualize mode shifting as occurring between two distinct components (Howard-

Jones, 2002; Gabora & Ranjan, 2013; Vartanian, 2009). The finding that these two-

component models of mode shifting are a poor fit to explain the relationship between

shifting proficiency and creativity in the current data suggest the two-component

conceptualization needs to be broadened. Specifically, current findings suggest it

should be expanded in two ways; firstly by introducing the conceptualization that

different modes can act as two modes tightly coupled together in time and secondly

conceptualizing the analytic mode as capable of operating on affective content. Two-

modes meshed together reflects verbal protocol segments where it wasn’t possible to

42


separate idea generation from evaluation, with different modes of thinking appearing

to operate in a tightly coupled fashion. Some accounts of creative thinking such as

Njistad et al.’s (2010) dual-pathway model do suggest a close coupling between idea

generation and evaluation, with the conception of an idea monitor that continuously

checks generated ideas.

Importantly, the strongest relationship between the frequency of two-modes meshed

together and garden design creativity occurred when the meshed mode included the

analytic affective as opposed to the analytic cognitive component, together with the

associative cognitive component. The correlation between the meshed measure

including the analytic affective component and ratings of the creativity of the designs

was almost double the size of those of the meshed measure with the analytic cognitive

component. It may be the case that thinking creatively on the garden design task

involved shifts between an associative mode that underpins cognitive idea generation

and an analytic mode that uses affective information, in order to help with the

evaluation of ideas. Functional neuroimaging (fMRI) has been used previously to

examine the brain networks recruited when participants evaluated the quality of

designs for book covers (Ellamil et al., 2012) finding activation during evaluation in

default network regions including the medial prefrontal cortex and posterior cingulate

cortex, areas involved in processing affective information. In fact, Dietrich (2004)

also proposed that the analytic affective component of his model of creative thinking

is underpinned by similar brain areas namely the ventromedial prefrontal cortex and

cingulate cortex, although he has since argued against the notion that creativity can be

localized in the brain (Dietrich, 2015). Other work has shown that affect can have a

direct influence on higher-order cognition (Blanchette & Richards, 2010), supporting

43


the possibility of the real time influence of affect on judgments, which could help to

explain the close coupling between modes reflected in two-modes meshed together

segments. It’s important to recognize that segments coded with the analytic affective

component in verbal protocols may only show that participants are focusing on

affective “gut reactions” to the ideas they have generated. Affective evaluations

could be taking place at other times during the garden design session but not reported

upon. However, a focus on the affective component of analytic processes seems

important in its own right as a means to monitor progress on the creation of garden

designs (Ellamil et al., 2012). Previous work employing think-aloud protocols in the

creative process of visual artists (Fayena-Tawil et al., 2011) and a psychometric

measure of shifting (Pringle & Sowden, 2017) both suggest an important role for

metacognitive processes in creativity. Future work should examine the link between

affective evaluations and metacognitive judgments in the creative process.

The evidence for mode shifting during key time points when participants switched

between different designs is also valuable for expanding our conception of mode

shifting. Specifically, it supports Sowden et al.’s (2015) suggestion that shifting

proficiency could be conceptualized as performing shifts at the right time. When

participants switched from one design to another they appeared to do so because they

had reached an impasse or come upon a better idea and shifting between modes

allowed them to make this evaluation and generate ideas anew for a better, more

creative design. The finding that adopting this strategy resulted in the production of

more creative garden designs with a higher design quality suggests it was effective.

There was indeed elevated mode shifting when participants switched between

working on different designs compared to periods when participants worked on the

44


same design. The type of mode shifting elevated at these key time points mirrored

that found in analyses conducted on average levels of shifting during the entire verbal

protocol, with elevations in instances of two-modes meshed together segments and

shifts between analytic affective and associative cognitive components.

It is important to consider the present findings in light of several limitations. Firstly,

the findings linking mode shifting during the creative process of garden design to the

creativity and design quality of designs only show correlation: they do not

demonstrate that mode shifting impacted on creative performance. Future work could

attempt to entrain or interfere with the types of shifting, for example meshing of

associative cognitive and analytic affective modes, which was associated with

creativity in the current work, to examine if this impacts on creative performance on

the garden design task. Secondly, there was a large amount of noise in the data,

reflecting a high degree of variability together with a large number of unknown

transitions which were also elevated at time points when participants switched

between working on different designs, and thus could be a confound in the chi-square

analysis. This noise in the data could be the result of modes of thinking being

relatively crudely defined. Ultimately, neural markers for the different modes of

thinking should be identified in order to clearly capture each mode and better define

the time points when each occurs (Sowden et al., 2015). The ‘think-aloud’ method has

been used previously with success (Fleck & Weisberg, 2004; Atman et al., 1999;

Fayena-Tawil, et al., 2011) but it still has some potential drawbacks in that concurrent

verbalization may interfere with aspects of designing such as perception during

sketching activity (Lloyd, Lawson & Scott, 1995). Furthermore, professional garden

designers often explain the thinking behind their designs to their clients, which may

45


present a potential confound as other groups, particularly the fine artists and the low

creative achievement group, are much less likely to have experience in verbalizing

their thinking. Similarly, expertise in garden design likely confers greater creative

self-efficacy (Beghetto, Kaufman, & Baxter, 2011) that could in turn boost the verbal

output of experts who are probably more confident reporting their creative thoughts

than those without expertise in garden design. In support of this our findings show

that mean protocol length, both in terms of the total number of think-aloud protocol

segments and the length in minutes of protocols, was shorter for the low creative

achievement group compared to professional and student garden designers. The

failure to account for differences in creative self-efficacy is a limitation of the present

work. A final limitation concerns the variability in the environment in which

participants worked on the garden design task. While the ecological validity of the

present work is enhanced by the fact that fine artists, professional and student garden

designers worked in their own studios, variability in environment was introduced by

the fact that the low creative achievement group did not have personal studios and

instead worked in a studio within the School of Psychology. This variability in

environment could thus have contributed to differences between the low creative

achievement and the other three groups on the creativity and design quality of final

garden designs and mode shifting during the design process.

In conclusion, this work has taken the study of mode shifting from the laboratory to

the everyday context of producing a creative design for a garden. In doing so it has

provided data to support a broader theoretical conception of proficiency in mode

shifting than previous laboratory and psychometric work has allowed for. In

particular, the findings suggest proficiency at using associative and analytic modes of

46


thinking meshed, operating tightly together in unison, and with an analytic mode of

thinking processing affective content is linked to creative performance. It also

provides evidence to suggest proficiency at mode shifting may depend on the timing

of shifts during the creative process. This expanded conception and new measures of

proficient mode shifting should help future work identify links between mode shifting

during the creative process and the creative product, essential to taking the next step

of entraining proficiency in mode shifting to aid people’s creative performance.

References

Allen, A. P., & Thomas, K. E. (2011). A dual process account of creative thinking.

Creativity Research Journal, 23 (2), 109-118.

Amabile, T. M. (1996). Creativity in context. Oxford, UK: Westview Press.

Atman, C. J., Chimka, J. R., Bursic, K. M., & Nachtmann, H. L. (1999). A

comparison of freshman and senior engineering design processes. Design

Studies, 20 (2), 131-152.

Beaty, R. E., Silvia, P. J., Nusbaum, E. C., Jauk, E., & Benedek, M. (2014). The roles

of associative and executive processes in creative cognition. Memory and

Cognition, 42, 1186-1197.

Beghetto, R. A., Kaufman, J. C., & Baxter, J. (2011). Answering the unexpected

questions: Exploring the relationship between students' creative self-efficacy

and teacher ratings of creativity. Psychology of Aesthetics, Creativity and the

Arts, 5 (4), 342-349.

47


Basadur, M. (1995). Optimal ideation-evaluation ratios. Creativity Research Journal,

8 (1), 63-75.

Basadur, M. S., Graen, G., & Green, S. (1982). Training in creative problem solving:

Effects on ideation and problem finding and solving in an industrial research

organization. Organizational Behavior and Human Performance, 30, 41–70.

Blanchette, I., & Richards, A. (2010). The influence of affect on higher level

cognition: A review of research on interpretation, judgement, decision making

and reasoning. Cognition and Emotion, 24 (4), 561-595.

Carson, S. H., Peterson, J. B., & Higgins, D. M. (2005). Reliability, validity, and

factor structure of the creative achievement questionnaire. Creativity Research

Journal, 17 (1), 37-50.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests.

Psychometrika, 16(3), 297-334.

Cross, N. (2011). Design thinking: Understanding how designers think and work. NY:

Berg.

Dietrich, A. (2004). The cognitive neuroscience of creativity. Psychonomic Bulletin

& Review, 11 (6), 1011-1026.

Dietrich, A. (2015). How Creativity Happens in the Brain. Basingstoke, UK: Palgrave

MacMillan.

Dorfman, L., Martindale, C., Gassimova, V., & Vartanian, O. (2008). Creativity and

speed of information processing: A double dissociation involving elementary

versus inhibitory cognitive tasks. Personality and Individual Differences, 44 (6),

1382-1390.

48


Dorst, K., & Cross, N. (2001). Creativity in the design process: co-evolution of

problem–solution. Design Studies, 22 (5), 425–437.

Ellamil, M., Dobson, C., Beeman, M., & Christoff, K. (2012). Evaluative and

generative modes of thought during the creative process. NeuroImage, 59 (2),

1783-1794.

Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis. Cambridge, MA: MIT

Press.

Evans, J. S. B., & Stanovich, K. E. (2013). Dual-process theories of higher cognition

advancing the debate. Perspectives on Psychological Science, 8 (3), 223-241.

Everitt, B. (1977). The analysis of contingency tables. London, UK: Chapman and

Hall.

Fayena-Tawil, F., Kozbelt, A., & Sitaras, L. (2011). Think global, act local: A

protocol analysis comparison of artists' and non-artists' cognitions,

metacognitions, and evaluations while drawing. Psychology of Aesthetics,

Creativity, and the Arts, 5 (2), 135-145.

Finke, R. A., Ward, T. B., & Smith, S. M. (1992). Creative cognition: Theory,

research, and applications. Cambridge, MA: MIT press.

Fleck, J. I., & Weisberg, R. W. (2004). The use of verbal protocols as data: An

analysis of insight in the candle problem. Memory & Cognition, 32 (6), 990-

1006.

Field, A. (2009). Discovering statistics with SPSS. London, UK: Sage.

Finke, R. A., Ward, T. B., & Smith, S. M. (1992). Creative Cognition: Theory,

Research and Applications. Cambridge, MA: MIT Press.

49


Frankish, K. (2010). Dual-process and dual-system theories of reasoning. Philosophy

Compass, 10, 914–926.

Gabora, L. (2005). Creative thought as a non-Darwinian evolutionary process.

Journal of Creative Behavior, 39, 262–283.

Gabora, L. (2010). Revenge of the “neurds”: Characterizing creative thought in terms

of the structure and dynamics of memory. Creativity Research Journal, 22 (1), 1-

13.

Gabora, L., Chia, W. W., & Firouzi, H. (2013). A computational model of two

cognitive transitions underlying cultural evolution. In M. Knauff, M. Pauen, N.

Sebanz, & I. Wachsmuth (Eds.), Proceedings of the 35th Annual Meeting of the

Cognitive Science Society (pp. 2344-2349). Austin TX: Cognitive Science

Society.

Gabora, L. & Ranjan, A. (2013). How insight emerges in a distributed, content

addressable memory. In A. Bristol, O. Vartanian & J. C. Kaufman (Eds.), The

neuroscience of creativity (pp. 19-44). NY: Oxford University Press

Gilhooly, K., Fioratou, E., Anthony, S., & Wynn, V. (2007). Divergent thinking:

Strategies and executive involvement in generating novel uses for familiar

objects. British Journal of Psychology, 98 (4), 611-625.

Howard-Jones, P. A. (2002). A dual-state model of creative cognition for supporting

strategies that foster creativity in the classroom. International Journal of

Technology and Design Education, 12 (3), 215-226.

50


Kaplan, D. (2008). An overview of Markov chain methods for the study of stage-

sequential developmental processes. Developmental Psychology, 44 (2), 457-

467.

Kaufman, S. B. (2011). Intelligence and the cognitive unconscious. In R. J. Sternberg

& S. B. Kaufman (Ed’s), The Cambridge Handbook of Intelligence (pp. 442-

467). Cambridge, UK: Cambridge University Press.

Kozbelt, A. (2008). Hierarchical linear modeling of creative artists’ problem solving

behaviors. Journal of Creative Behavior, 42, 181–200.

Landis, R., & Koch, G.G. (1977). The measurement of observer agreement for

categorical data. Biometrics, 33 (1), 159-174.

Lloyd, P., Lawson, B., & Scott, P. (1995). Can concurrent verbalization reveal design

cognition? Design Studies, 16 (2), 237-259.

Nijstad, B. A., De Dreu, C. K. W., Rietzschel, E. F., & Baas, M. (2010). The dual

pathway to creativity model: Creative ideation as a function of flexibility and

persistence. European Review of Social Psychology, 21, 34–77.

Pringle, A., Sowden, P.T. (2017). The Mode Shifting Index (MSI): A new measure of

the creative thinking skill of shifting between associative and analytic thinking.

Thinking Skills & Creativity, 23, 17-28.

Sowden, P. T., Pringle, A., Gabora, L. (2015). The Shifting Sands of Creative

Thinking: Connections to Dual Process Theory. Thinking and Reasoning, 21,

40-60.

Stanovich, K. E., & Toplak, M. E. (2012). Defining features versus incidental

correlates of type 1 and type 2 processing. Mind & Society, 11 (1), 3-13.

51


Suwa, M., & Tversky, B. (1997). What do architects and students perceive in their

design sketches? A protocol analysis. Design Studies, 18 (4), 385-403.

Vartanian, O. (2009). Variable attention facilitates creative problem solving.

Psychology of Aesthetics, Creativity, and the Arts, 3 (1), 57-59.

Vartanian, O., Martindale, C., & Kwiatkowski, J. (2007). Creative potential, attention,

and speed of information processing. Personality and Individual Differences, 43

(6), 1470-1480.

Veloz, T., Gabora, L., Eyjolfson, M., & Aerts, D. (2011). Toward a formal model of

the shifting relationship between concepts and contexts in different modes of

thought. In: D. Song, M. Melucci, I. Frommholz, P. Zhang, L. Wang, & S.

Arafat (eds.), Lecture Notes in Computer Science 7052: Proceedings of the Fifth

International Symposium on Quantum Interaction (pp. 25-34). Berlin: Springer.

Warriner, A.B., Kuperman, V. & Brysbaert, M. (2013). Norms of valence, arousal,

and dominance for 13,915 English lemmas. Behavior Research Methods, 45

(4), 1191-207.

Woods, D, and Fassnacht, C. (2012). Transana v 2.51. http://www.transana.org.

Madison, WI: The Board of Regents of the University of Wisconsin System.

Footnotes

1. Spearman’s rho correlations were run since the variables were not normally distributed.

2. A Mann-Whitney test was used to compare groups on design quality since ratings of design quality were not normally distributed.

52

epubs.surrey.ac.ukepubs.surrey.ac.uk/841324/1/20170606 paca_garden_d… · web viewunearthing the...

Documents