lable at ScienceDirect

Journal of Environmental Psychology 31 (2011) 266e269

Contents lists avai

Journal of Environmental Psychology

journal homepage: www.elsevier .com/locate/ jep

A response to: Möser, G., & Bamberg, S. (2008). The effectiveness of soft transportpolicy measures: A critical assessment and meta-analysis of empirical evidence.Journal of Environmental Psychology, 28, 10e26

Rob Wall a,*, Werner Brög b, Erhard Erl b, James Ryle a, Franz Barta c

a Sustrans, 2 Cathedral Square, College Green, Bristol BS1 5DD, United Kingdomb Socialdata, Fürstenrieder Straße 284, 81377 München, Germanyc Socialdata, 37/39 Corn Street, Bristol BS1 1HT, United Kingdom

a r t i c l e i n f o

Article history:Available online 14 May 2011

DOI of original article: 10.1016/j.jenvp.2007.09.00* Corresponding author. Tel.: þ44 (0)117 915 0241.

E-mail address: rob.wall@sustrans.org.uk (R. Wall)

0272-4944/$ e see front matter � 2011 Elsevier Ltd.doi:10.1016/j.jenvp.2011.05.004

Volume 28 of Journal of Environmental Psychology carried anarticle entitled “The effectiveness of soft transport policy measures:A critical assessment and meta-analysis of empirical evidence”(Möser & Bamberg, 2008). It questioned whether narrative reviewsof soft transport measures (e.g., Cairns et al., 2004) “are able todeliver a reliable and valid basis for evidence based decisionmaking” (Möser & Bamberg, 2008, p. 12). We agree that this is animportant issue and that evidence of soft measures’ effectivenessshould be examined, but there are a number of flaws with Möserand Bamberg’s analysis, which we discuss in this response.

Our most serious concerns with Möser and Bamberg’s articlerelate to their Table A1, which presents data from studies used intheir meta-analysis of personalised travel planning (PTP), travelawareness and public transport marketing campaigns. We presentan annotated version in our Table 1, below. This highlights severalerrors in Möser and Bamberg’s reporting of data. Information on theleft of Table 1 (in the columns ‘Primary study’, ‘Data source’, StudyN0

and ‘No car’) is taken directly from Table A1 in the original article.Information on the right of Table 1 (in the columns ‘Corrected no car’and ‘Comments’) has been added for the purposes of this response.

Data under ‘No car’ indicate the share of all trips made by non-car modes. Thus, by way of example, in the second row [‘Enfield TfLpilot (UK)’] Möser and Bamberg (2008) indicate that 74% of tripswere made by non-car modes before the intervention, while 37% oftrips were made by non-car modes after the intervention. Thisraises a question over what sort of intervention would increase the

1.

.

All rights reserved.

share of trips made by car from 26% to 63%. The correct figures areshown to the right of Möser and Bamberg’s figures: 63% of trips bynon-car modes before the intervention and 74% afterwards. Inreality the direction and magnitude of change are much morealigned with what one would expect.

Where we have access to the correct figures we have insertedthem in Table 1. In some instances we have been able to correctboth ‘Before’ and ‘After’ figures (e.g., ‘Bishopsworth/Hartcliffeproject 1 (UK)’ and ‘Frome pilot (UK)’). In other instances the ‘After’figures provided by Möser and Bamberg are correct, so we haveonly inserted the ‘Before’ figures [e.g. ‘Gloucester largescale (UK)’and ‘Gloucester pilot (UK)’].

In many rows we have only been able to comment ‘Both figuresincorrect’. If this begs the question of how we can make thisassertion without knowing the correct figures, the answer lies inknowing the data source used by Möser and Bamberg. This is Ker’s(2003) paper, which examined only interventions designed topromote public transport use. Because of this focus, Ker reportedonly changes in public transport use, not changes in the use of allnon-car modes. To illustrate, consider ‘Christies Beach (Australia)’in Table 1. The ‘No car’ figures are 2.9% ‘Before’ and 3.7% ‘After’.These are actually figures for the share of all trips made by publictransport, which do not include walking, cycling, or any other non-car modes. As such, they are misused here. It is also important tonote that interventions designed to increase public transport usemay have no effect on car use. It is, in principle, quite possible toincrease the former without reducing the latter.

A further problem with data presented in Table 1 is highlightedin the ‘Comments’ column. Rather than providing information on72 separate interventions and associated evaluations, there areactually fewer than this. Eight rows repeat data from studies

Table 1Annotated version of Möser and Bamberg’s (2008) Table A1.

Data presented by Möser and Bamberg (2008) Comments and corrections

Primary study Data source Study N No car Corrected nocar

Comments

Before After Before After

Stepchange pilot (UK) 2 1754 76 25 Not known Details of this project are not actually provided in data source 2.Enfield TfL pilot (UK) 2 235 74 37 63 74 Study N is incorrect.Holland Park (Australia) 3 102 9.3 8.6 Both figures

incorrectDulwich (Australia) 3 515 3.8 3.6 Both figures

incorrectLiverpool (UK) 3 32 20 20 Both figures

incorrectReggio Emilia (Italy) 3 691 17 17 Both figures

incorrectPorto (Portugal) 3 421 22 22 Both figures

incorrectKassel (Germany) 3 13,012 20 20 Both figures

incorrectThe N reported is the number of participants in the intervention,not the number of survey participants.

Luxembourg 3 230 38 39 Both figuresincorrect

Stuttgart-Freiberg (Germany) 3 5330 21 22 Both figuresincorrect

The N reported is the number of participants in the intervention,not the number of survey participants.

Marangaroo (Australia) 2 300/201 40 42 Correct Details of this project are not actually provided in data source 2.South Perth (Australia) 3 1000 6 7 Both figures

incorrectBern A (Swiss) 3 247 31 33 Both figures

incorrectFrome (UK) 3 500 5 6 Both figures

incorrectStudy N wrong, and this is the same project as ‘Frome pilot (UK)’,listed below.

Oslo (Norway) 3 1153 29 31 Both figuresincorrect

Christies Beach (Australia) 3 215 2.9 3.7 Both figuresincorrect

Lisbon (Portugal) 3 548 23 25 Both figuresincorrect

Copenhagen A (Denmark) 3 585 21 23 Both figuresincorrect

York (12 h day) (UK) 2 500 53.8 56.3 CorrectCologne (Germany) 3 235 19 21 Both figures

incorrectLinz (Austria) 3 15,141 19 21 Both figures

incorrectLisbon (Portugal) 3 548 18 20 Both figures

incorrectWiesbaden (Germany) 3 4632 17 19 Both figures

incorrectEast Hampshire 2003 (UK) 2 1000 13 15 Both figures

incorrectBishopsworth/Hartcliffe project 1 (UK) 2 2500 53 56 54 57 The N reported is the number of participants in the intervention,

not the number of survey participants. In addition, this is the same projectas ‘Bristol VIVALDI phase 1 (UK)’, listed below.

Quedgeley pilot (Australia) 2 177 55 58 56 60 Quedgeley is in the UK, not Australia. In addition,this is the same project as ‘Gloucester pilot (UK)’, listed below.

Arnhem (The Netherlands) 3 106 2 3 Both figuresincorrect

Munich (Germany) 3 229 24 27 Both figuresincorrect

Cambridge (UK) 2 529/400 39.5 43 Correct Actually Cambridge (Australia), not ‘Cambridge (UK)’.Bremen (Germany) 3 189 17 20 Both figures

incorrectSalzburg (Austria) 3 5500 16 19 Both figures

incorrectThe N reported is the number of participants in the intervention,not the number of survey participants.

Bishopston (UK) 2 5364 62 66 63 Correct The N reported is the number of participants in the intervention,not the number of survey participants.

Borken (Germany) 3 410 4 6 Both figuresincorrect

Delft/Den Haag (Netherlands) 3 124 4 6 Both figuresincorrect

Halle (Germany) 3 154 19 23 Both figuresincorrect

Armandale (UK) 2 247/210 44 49 45 Correct Should be ‘Armadale’, which is in Australia, not the UK.Details of this project are not actually provided in data source 2.

Frementle (France) 2 476/615 48 53 Correct Fremantle (note spelling) is in Australia, not France.Details of this project are not actually provided in data source 2.

(continued on next page)

R. Wall et al. / Journal of Environmental Psychology 31 (2011) 266e269 267

Table 1 (continued )

Data presented by Möser and Bamberg (2008) Comments and corrections

Primary study Data source Study N No car Corrected nocar

Comments

Before After Before After

Bristol VIVALDI phase 1 (UK) 2 232 52 57 54 Correct This is the same project as ‘Bishopsworth/Hartcliffeproject 1 (UK)’, listed above.

Helsinki (Finland) 3 176 37 42 Both figuresincorrect

Gloucester (UK) 3 445 3 5 Both figuresincorrect

Not clear which project is referred to here, but non-carshare implausible for any UK town or city.

Montpellier (France) 3 411 3 5 Both figuresincorrect

Pinneberg (Germany) 3 501 15 19 Both figuresincorrect

Leipzig (Germany) 3 188 14 18 Both figuresincorrect

Magdeburg (Germany) 3 212 14 18 Both figuresincorrect

Hannover-Südstadt (Germany) 3 40,990 25 30 Both figuresincorrect

The N reported is the number of participants in the intervention,not the number of survey participants.

York (morning peak) (UK) 2 500 57.4 63 CorrectLiverpool (UK) 3 33 12 16 Both figures

incorrectQuedgeley large scale 2004 2 954 75 80 Both figures

incorrectFrome pilot (UK) 2 282 53 59 51 55 This is the same project as ‘Gloucester largescale (UK)’, listed below.Melville (Australia) 2 972/634 34 40 CorrectBern B (Swiss) 3 27 18 23 Both figures

incorrectLudwigshafen (Germany) 3 197 9 13 Both figures

incorrectSubiaco (Australia) 2 400 44 51 CorrectBrisbane (Australia) 3 1100 6 10 Both figures

incorrectNürnberg (Germany) 3 4940 17 23 Both figures

incorrectThe N reported is the number of participants in the intervention,not the number of survey participants.

Vincent (Australia) 2 416/409 44 52 45 50 Details of this project are not actually provided in data source 2.South Perth (Australia) 2 706 40 48 CorrectVenice (Italy) 3 742 37 45 Both figures

incorrectVollmar (Germany) 3 5655 4 8 Both figures

incorrectShould read ‘Vellmar’. The N reported is the number of participantsin the intervention, not the number of survey participants

Bologna (Italy) 3 681 26 34 Both figuresincorrect

Gloucester largescale (UK) 2 2018 46 55 51 Correct Same project as ‘Quedgeley large scale (UK)’, listed above.Gloucester pilot (UK) 2 187 51 60 56 Correct Same project as ‘Quedgeley pilot (UK)’, listed above.Madrid (Spain) 3 382 7 13 Both figures

incorrectBaunatal (Germany) 3 6918 7 13 Both figures

incorrectThe N reported is the number of participants in the intervention,not the number of survey participants.

Bristol Bishopston (UK) 2 5364 56 66 63 Correct Same project as ‘Bishopston (UK)’, listed above.The N reported is the number of participants in the intervention,not the number of survey participants.

Hampshire (UK) 2 162 4 9 Both figuresincorrect

Details of this project are not actually provided in data source 2.

Kingston TfL pilot (UK) 2 793 52 63 58 CorrectTurin (Italy) 3 213 34 47 Both figures

incorrectParma (Italy) 3 721 13 24 Both figures

incorrectSouthwark TfL pilot (UK) 2 257 50 66 59 CorrectBuckinghamshire County (UK) 2 3000 28.7 50.6 Correct This project only involved Buckinghamshire County Council staff.Bike Busters (UK) 2 175 20 55 Not known Details of this project are not actually provided in data source 2.

1. Data sources are: (2) Cairns et al. (2004); (3) Ker (2003).2. Where there are two figures for ‘Study N’ these refer to different sample sizes in ‘Before’ and ‘After’ surveys.

R. Wall et al. / Journal of Environmental Psychology 31 (2011) 266e269268

reported elsewhere in Table 1. Nonetheless, it is notable that wherestudies are repeated, the ‘No car’ figures often differ. For example,‘Quedgeley pilot’ (incorrectly labelled as ‘Australia’ rather than ‘UK’)is the same study as ‘Gloucester pilot (UK)’. However, the ‘No car’figures for Quedgeley are reported as 55% ‘Before’ and 58% ‘After’,while for Gloucester pilot they are reported as 51% ‘Before’ and 60%‘After’. Such inconsistencies raise questions about the extent to

which Möser and Bamberg’s results and conclusions can be reliedupon.

There are also concerns over the origins of some data presentedin Möser and Bamberg’s Table A1. In a number of instances theynote that datawere taken from Cairns et al. (2004) (data source 2 inTable 1) when in fact this publication does not report the studies inquestion. This raises the question of where these data are derived

R. Wall et al. / Journal of Environmental Psychology 31 (2011) 266e269 269

from and how reliable the sources are. It is also notable that inmany instances the ‘Study N’ appears to be the number of partici-pants in the intervention rather than in the evaluation study.

The central point regarding the various errors in Möser andBamberg’s (2008) Table A1 is that incorrect inputs to their meta-analysis will have led to incorrect outputs. For example, they note“a fairly homogeneous distribution of the reported effect sizesaround the mean” (p. 14) for studies evaluating PTP, awareness andinformation campaigns. However, we must surely be cautious inaccepting this homogeneity when there are doubts over many ofthe ‘Before’ and ‘After’ figures for the ‘No car’ variable and even overthe number of unique studies included in the analysis.

Möser and Bamberg state that the reason for relying onsecondary sources to provide data for their analysis was their lackof success in obtaining original research reports from organisationswhich had carried out evaluation studies. On this point, we notethat Socialdata first met one of the authors at a conference inMarch2002, after which Socialdata stated several times in writing and inperson its interest in cooperating with a study of this kind. In June2006 one of the authors specifically requested information aboutPTP projects evaluated by Socialdata. In response to this request,Socialdata provided a document containing detailed informationabout 89 PTP projects (on June 22, 2006).

In addition to the data errors described above, there area number of other problems with Möser and Bamberg’s analysis.First, their treatment of PTP, travel awareness campaigns and publictransport marketing as one type of intervention is questionable. PTPis actually rather different from travel awareness campaigns andpublic transport marketing. It goes much further in engagingparticipants in a dialogue about their travel behaviour and providinginformation, motivation and support specific to individual and/orhousehold circumstances. PTP is based on the premise that everyonehas different transport needs, different levels of awareness anddifferent perceptions of transport modes. The typical features of PTPare well summarised in the UK Department for Transport’s (DfT)recent guide for practitioners in the field (DfT, 2008).

The problem with treating different types of interventiontogether is that we might expect them to yield varying levels ofbehaviour change, yet variations are masked by an analysis thatdoes not differentiate between different types of soft measure. Thisdoes not provide a clear basis for decision making. We may wish toknow, for example, what level of behaviour change could beexpected from different levels of investment. Costs for PTP arelikely to be greater per head of target population than costs for, say,an awareness campaign using posters at public sites across a town.But this extra cost may be worth bearing if it brings sufficientlyenhanced benefits.

Another major concern is that some of Möser and Bamberg’sstatements about others’ work are incorrect. They claim “All 141evaluation studies compiled in our data set use a weak quasi-experimental evaluation design, namely the one-group-pre-post-test design” (Möser & Bamberg, 2008, p. 13). However, it is clearfrom Ker’s (2003) paper e a main data source for Moser andBamberg’s analysis e that fifteen studies he reviewed used controlgroups to evaluate interventions. Furthermore, while we cannotspeak for other organisations, Socialdata’s evaluations of PTPcampaigns (delivered in partnership with Sustrans for UK projects)always use a control-group-pre-post-test-design, where thecontrol group is drawn from a random sample of households nottargeted by the intervention. Several evaluation reports whichdescribe this survey methodology are publicly available (e.g.,Sustrans & Socialdata, 2007).

Möser and Bamberg (2008, p. 14) go on to state that “none of the141 evaluation studies contained in our data set uses any kind ofstatistical significance testing for estimating the probability that

the found pre-post car use differences may only reflect randomfluctuation.” Again, this is incorrect. At least in work conducted bySocialdata and Sustrans, significance tests are conducted. Detailsare provided in several publicly available reports (e.g., Sustrans &Socialdata, 2007).

We must also address the assertion that soft measures evalua-tions routinely use samples that are unrepresentative of largerpopulations in terms of “their socio-demographic background,their location, and car availability but also in their attitudes towardsas well as actual use of transportation means” (Möser & Bamberg,2008, pp. 13e14). Again, Socialdata and Sustrans do correct forsample bias in their PTP evaluations, in an attempt to ensure thattransport behaviour in survey samples is representative of behav-iour in the PTP target population as a whole (e.g., Sustrans &Socialdata, 2007).

As evidence for the supposed lack of correction for sample bias,Möser and Bamberg cite a conference paper by Stopher and Bullock(2003). It is notable, however, that the detailed rebuttal presentedin the same session of the same conference (Roth et al., 2003) is notmentioned. Whether or not Möser and Bamberg accept thisrebuttal, we would argue that coverage of the ongoing debateamong PTP practitioners and researchers would be beneficial toreaders and would allow for a fuller appraisal of the variouspositions.

In conclusion, while we welcome scientifically rigorousapproaches to soft transport measures and their evaluation, wehave serious reservations about Möser and Bamberg’s article. Theserelate to: serious errors in the data used as inputs to meta-analysis;treatment of different types of intervention as one set; and incor-rect assertions about research design and methods. These factorscombine to cast doubt on Möser and Bamberg’s (2008) conclusionthat “the currently available empirical evidence provides no solidbasis for the claim that a broad implementation of soft transportpolicy measures is an effective strategy for reducing car use” (p. 19).While we agree that quantitative meta-analysis can in principleovercome some of the shortcomings of narrative reviews, this isunlikely to be true when quantitative inputs are questionable. Wewould argue, based on the experience of delivering e with a “highdegree of professionality and standardisation in implementing thismeasure type” (p. 20) e a large number of PTP projects in the UKand elsewhere, that there is strong evidence of such projects’effectiveness in promoting modal shift from car use to lesspolluting and healthier modes. Unfortunately this evidence is hardto discern from Möser and Bamberg’s article, for the reasons out-lined in this response.

References

Cairns, S., Sloman, L., Newson, C., Anable, J., Kirkbride, A., & Goodwin, P. (2004).Smarter choices-changing the way we travel. Final report of the research project:The influence of soft factor interventions on travel demand. Research Report.London: Department for Transport. http://www.dft.gov.uk/pgr/sustainable/smarterchoices/ctwwt Retrieved 09/02/09.

Department for Transport. (2008). Making personal travel planning work: Practi-tioners’ guide. London: Department for Transport, ISBN 978 1 906581 25 1.

Ker, I. (2003). Travel demand management: Public transport business case. Victoria:Contract Report RC5051 for Department of Infrastructure.

Möser, G., & Bamberg, S. (2008). The effectiveness of soft transport policy measures:A critical assessment and meta-analysis of empirical evidence. Journal of Envi-ronmental Psychology, 28, 10e26.

Roth, M., Ker, I., James, B., Brög, W., Ashton-Graham, C., Ryle, J., et al. (2003).A dialogue on individualised marketing: Addressing misperceptions. Wellington,New Zealand: 26th Australasian Transportation Research Forum.

Stopher, P. R., & Bullock, P. (2003). Travel behaviour modification: A critical appraisal.Wellington, New Zealand: 26th Australasian Transportation Research Forum.

Sustrans & Socialdata. (2007). Lancashire TravelSmart Programme: Interim evaluation ofStage 2.1 (Torrisholme). Report for Lancashire County Council, September 2007.http://www.sustrans.org.uk/webfiles/travelsmart/Researchreports/Torrisholme%20Interim%20Research%20Report%202007.pdf Retrieved 09/02/09.