what works for whom: the link between process and outcome in effectiveness research

Keywordscommon factors, effectiveness, psychotherapy,

social work research

IntroductionThe following article is a continuation of thedialogue opened by Jill Gibbons in aprevious edition of Australian Social Work onthe tradition of effectiveness research insocial work (Gibbons 2001). The followingdiscussion examines this research tradition

more closely and contrasts it with that of theclosely related field of psychotherapy. Thisreview aims to highlight the key questionsand concerns addressed by researchers inthese fields across the last half-century andidentify the implications of this research forfuture social work effectiveness studies.

Practice effectiveness research has along tradition in both social work andpsychotherapy. Research into theeffectiveness of psychotherapy across thelast five decades can be characterised asoccurring in three generations, with eachgeneration carrying within it distinctresearch concerns and sowing the seedsfor the questions driving the researchactivity of the next generation (Lambert &Bergin 1994, Marziali et al. 1981, Roth et al. 1996, Woolfe & Dryden 1996). To a

What works for whom: the link between process and outcome ineffectiveness researchSerena Smith

To date social work effectiveness research has focused on establishing if socialwork interventions work without much consideration of why interventions work. Thistendency is largely the result of effectiveness research being driven by agendasother than those aimed at enhancing practice. This research tradition is reviewedand contrasted with the similar field of psychotherapy where there has beenconsiderable research into why interventions work. Factors common to a range oftreatment modalities have been shown to have a positive impact on clientimprovement, particularly the therapeutic relationship. Given the central place ofthe ‘helping relationship’ in social work practice and the commonalities betweenthese fields, it is imperative that social work researchers face the challenge ofinvestigating the process factors impacting outcomes. In the absence of thisresearch, effectiveness cannot be honestly attributed to particular interventiontechniques.

Serena Smith, AASW (Victorian branch), 176 AmessStreet, North Carlton, Victoria 3054, Australia. E-mail: [email protected] Smith is currently working in private practice andis convenor of the Psychoanalytic/Psychodynamic SpecialInterest Group.

Australian Social Work/June 2002, Vol. 55, No. 2 147

large extent this tradition has beenparalleled by social work effectivenessresearch, although reviewers of social workeffectiveness research identify two ratherthan three distinct phases in itsdevelopment (Corey et al. 1998,MacDonald et al. 1992, Sheldon 1986,Smith 2000, Videka-Sherman 1988, Wood1978). The research traditions of thesefields, although distinct, have raised andaddressed similar questions and concerns.The following comparison of these researchtraditions provides an opportunity torecognise an important gap in the currentconcerns of social work effectivenessresearch.

Review of the phases andgenerations of effectivenessresearch in social work andpsychotherapyThe first phase of social work effectivenessresearch covers a period roughly between1950 and 1970s. The research of the periodwas focused on a general questionregarding the effectiveness of casework. Inthe early 1970s a landmark effectivenessstudy was conducted by Fischer (1973),who found that casework services have nopositive effects and in a number ofinstances were found to have deleteriouseffects. The result of this rather shockingfinding was to raise a number of questionsregarding the way outcomes were beingdefined and the related question of how tomeasure outcomes in a reliable and validway. This was considered particularlyimportant because of the range ofintervention models used in social work, thediverse client population of social workservices and the varied organisationalsettings in which social workers practiced.

A summary of the key studies and a reviewof this period can be found in Smith (2000).

The first generation of psychotherapyeffectiveness research from the sameperiod, roughly between the 1950s and1970s, was concerned also with a generalquestion regarding the effectiveness oftherapy and how to positively measure it.The period began with the infamous studyconducted by Hans Eysenck (1952)arguing psychotherapy was no moreeffective than spontaneous remission.Eysenck’s study provided the motivation fora great deal of additional research into thequestion of effectiveness. At the end of thisperiod the consensus position held by theresearch community regarding theeffectiveness of psychotherapy, was thattherapy was beneficial to clients but theseeffects were largely attributable to thecommon factors involved in the helpingprocess. This led to the development of anumber of tools to measure these commonfactors, the most prominent being thetherapeutic relationship. Thus a number ofresearch tools were developed to assessboth the therapeutic relationship as definedby Carl Rogers’s facilitative conditions andthe effectiveness of interventions. Asummary of the key studies and a review ofthis period can be found in Smith (2000).

The second phase of social workeffectiveness research between the 1980sand 1990s has continued to tackle thequestion of how to define effectiveness inrelation to social work practice and themethodological problems created by theneed to measure practice in a reliable andvalid way. Social work effectivenessresearch has remained stuck in thesedebates regarding the most appropriatetype of methodology to use whenresearching the outcomes of social work

148 Australian Social Work/June 2002, Vol. 55, No. 2

practice and the ability of global outcomemeasures to capture the complexity of bothsocial work interventions and the context ofsocial work practice. Despite the moreintegrative view of social work researchersin the 1990s, research into the effectivenessof social work practice has largely relied onstudies using global measures of outcomeaimed at establishing only if interventionswork. The tradition has not entered into aphase of research activity directed atestablishing why interventions work. Themore complex question of what factorswithin intervention processes produceoutcomes has been left largely unattendedby social work researchers.

This trend can be recognised in anumber of reviews of social workeffectiveness research undertaken duringthis period. Reid and Hanrahan in theiranalysis of 22 published outcome studiesnoted that a gap can often exist ineffectiveness research between findingsthat are statistically and practicallysignificant. Although they found an overallindication of positive outcomes in the rangeof social work evaluation studies theyreviewed, they did not provide details onthe particular therapeutic processes thatwere associated with these outcomes, asthey had not been reported in the studiesreviewed (Reid & Hanrahan 1982).

In Verdicts on Social Work, Rees andWallace provide some indication of thefactors contributing to outcomes in socialwork practice. In studies using clientreports, they identified a number of factorscontributing to client satisfaction withservices and their sense of being helped.Factors such as the characteristics of thesocial worker, the relationship betweenclients and social workers, the sense that

the social worker cared for them and thegeneral tone of the encounters were allassociated with positive outcomes. Theauthors argue that their review of theresearch clearly indicates that clients valuesocial workers being patient, having theability to listen, being knowledgeable andbeing experienced. On the basis of thesefindings Rees and Wallace argue againstthe push [they identify] within social workevaluation to separate the idea of thehelpful person from a helpful service, as it is clear that clients identify the helpful person as essential (Rees & Wallace1982).

More recently Mary Russell in hersummary of research into the effectivenessof clinical social work practice reported thatsocial work researchers have paid littleattention to the within-session processesthat mediate outcomes (Russell 1990).

The second generation ofpsychotherapy effectiveness research(covering the late 1960s to 1980s) wascharacterised by addressing the questionsraised by appearance of the equivalenceparadox in the previous generation. Thisparadox was the result of the fact thateffectiveness research consistently foundthat different therapeutic approaches to thesame problems showed little difference intheir effectiveness. The appearance of theequivalence paradox was the result of thefinding that factors common topsychotherapy interventions wereinfluencing outcomes. The third generationof research effort in psychotherapy as beeninfluenced by this finding, alongside thefinding that the facilitative conditions(respect, empathy, warmth, congruenceand positive regard) as defined by CarlRogers were necessary but not sufficient


conditions for gaining positive outcomes intherapy. This generation of researchspanning from the late 1980s to the presenthas focused largely on understanding howchange occurs in therapy. The period hasbeen characterised by exploration ofchange pathways, the role played by thetherapeutic alliance in change and the question of how much therapy isenough – the dose/effect relationship. Thekey studies of this period are reviewed inSmith (2000).

In psychotherapy effectiveness researchattention has shifted from addressing thegeneral question of therapy effectiveness tothe factors involved in these interventionsand what makes them work. The consistentfinding in psychotherapy effectivenessresearch is that factors common to theexperience of therapy, particularly thetherapeutic relationship, mediate thetherapeutic experience and impact itseffectiveness. This finding has beenestablished unequivocally.

Social work effectiveness research hasnot yet explored this link in any detail. As aresult, research on the factors common tosocial work practice has remained virtuallyunattended in effectiveness research. MaryRussell in her review of research on clinicalsocial work practice argues, ‘In social workstudies, however, these characteristics(therapy specific, common factors) havebeen studied infrequently in relation tooutcome.’ (Russell 1990). Russell in hercomprehensive review that spanned 18years between 1970 and 1988, cites onlyone study that included a discussion of theimportance of the therapeutic alliance insocial work outcomes. Given the highcorrelation of a number of process variablesin predicting positive outcomes in

psychotherapy and the importance placedon the client/worker relationship in socialwork theory, it would seem essential toaccount for these factors in social workeffectiveness research.

The field of psychotherapy researchprovides a rich and voluminous set offindings to be mined by social workeffectiveness researchers. There areenough similarities between these fields ofpractice, particularly the clinical caseworkend of the social work practice spectrum,for findings to be transferable to social workresearch and to contribute to social workpractice. The findings from this research arewell documented in a number of reviews.

It appears that what can be firmly said isthat factors common across treatment areaccounting for substantial amounts ofimprovement found in psychotherapypatients . . . they loom large as themediators of treatment outcome. Theresearch base for this conclusion issubstantial and multidimensional, and wemust attend to its import. (Bergin & Garfield1994)

This assessment is reinforced by thefindings of an extensive meta-analysis ofprocess and outcome research undertakenby Orlinsky et al. (1994). They concludetheir study by stating:

The quality of the patient’s participation intherapy stands out alone as the mostimportant determinant of outcome. Thetherapeutic bond, especially as perceivedby the patient, is importantly involved inmediating the process–outcome link. Thetherapist’s contribution towards helping thepatient achieve a favourable outcome ismade mainly through empathic, affirmative,collaborative and self congruent


engagement with the patient, and the skilfulapplication of potent interventions . . .These consistent process–outcomerelations, based on literally hundreds ofempirical findings can be considered factsestablished by 40 plus years of research inpsychotherapy . . . [my emphasis]. (Orlinskyet al. 1994; p. 361)

Research into the linkbetween the therapeuticrelationship and outcomesin social work researchThe relationship between the worker andthe client is given a central place in socialwork practice theory. Thus, addressing thelack of systematic research into thequestion of what the therapeutic/helpingrelationship is and its link to interventionoutcomes is essential. Poulin and Young,researchers who have developed a tool forresearching the helping relationship insocial work practice, argue that recentresearch into social work practice hasfocused largely on establishing theoutcomes of social work practice with theprimary aim of establishing some scientificbasis for the profession (Poulin & Young1997). The result of this has been to neglectresearch into the more pertinent questionsregarding the nature of practice. Poulin andYoung argue that although there has been apush to research the outcomes of practice,‘little attention has been paid to the role ofthe helping relationship in delivering theseinterventions effectively’ (Poulin & Young1997).

The most frequent reason given for thelack of attention to these factors in socialwork research is the claim that the majority

of social work practice occurs in settingsvery different from psychotherapy. Social work it is argued occurs in settingsthat deliver short-term interventions withclients who have multiple problems and aredisadvantaged economically. But thiscriticism in the current service climate isclearly too simplistic, given the trend toshort-term interventions across fields, thecomplex and multidimensional problemsexperienced by most individuals or familiesseeking ‘help’, and the oftenmultidisciplinary settings within whichservices are provided. Although thedifferences between the fields are importantto consider, research findings to date donot support the claim that socioeconomicstatus is a significant factor impactingchange once an intervention is in progress(Bergin & Garfield 1994).

In one of the few studies that explicitlyexplores the link between the therapeuticrelationship and outcomes, conducted by asocial worker, the author found that anumber of client and worker characteristicsimpact the development of the therapeuticrelationship and the outcomes achieved bya client. He argued that helping andprotecting behaviours on behalf of theworker were associated with good clientoutcomes and that client trust and relianceestablished early in the treatment wereneeded for the work to proceed and good outcomes to be achieved (Coady1991).

The author of this research considers hisfindings are relevant to general social workpractice and that they confirm social workpractice wisdom regarding the importanceof workers showing clients a consistentlypositive attitude at the outset of the work.He argues workers should involve the client


in the treatment process and use theirinfluence in a mild way, although clients donot necessarily count challenge andexpectation on behalf of the worker asnegative to the progress of the work. Withdifficult clients he argues it is important notto be disillusioned too early in the treatmentand to let the relationship develop. In a noteto the profession Coady argues that, ‘interms of direct practice, it seems likely thatthe association between relationship factorsand client outcomes that have beenestablished in individual counselling can begeneralised to family and group work.’(Coady 1993). This finding was alsoconfirmed in a study of counsellingoutcomes at problem gambling counsellingservices in Victoria, where the therapeuticalliance was positively correlated tosuccessful outcomes in counselling (Smith2000).

The link between processand outcome as establishedin psychotherapy researchThere have been two major meta-analysesconducted into studies researching theprocess–outcome link in psychotherapy.Combined these reviews analysed over 150published researched studies. Horvath andSymonds analysed a number of studies intothe quality of the therapeutic alliance andits impact on outcomes. These studiesincluded reports from the perspective ofclients, therapists and non-participantjudges and a range of outcome measureswere used in the studies. On the basis oftheir analyses the authors argued, ‘Thisreview suggests that the working alliance isa relatively robust variable linking therapy

process to outcome.’ (Horvath & Symonds1991; p. 146).

In their meta-analysis of process andoutcome studies in psychotherapy Orlinskyet al. analysed the findings of 132 studiesthat explored the process–outcome link. Inthe summary of their analyses they highlight11 process variables that are robustlylinked to outcome in psychotherapy. Theystress that the strength of this associationhas been established consistently overmany years of research using a variety ofmethodologies. These variables are:• patient suitability• patient cooperativeness• global therapeutic bond• patient contribution to the bond• patient interactive collaboration• patient expressiveness• patient affirmation of the therapist• reciprocal affirmation• patient openness versus defensiveness• therapeutic realisations• treatment duration.

The authors also stress that: ‘The qualityof the patient’s participation in therapystands out as the most importantdeterminant of outcome.’ (Orlinsky et al.1994, p. 360).

The implications of thesefindings for social workeffectiveness researchOn the basis of the findings already wellestablished in the psychotherapyeffectiveness research literature, social workresearchers need to design future studieswhich account for the effects of processvariables that have been found consistentlyrelated to outcome, before attributing


effectiveness to the intervention type usedin a program. In particular factors such asthe therapeutic relationship, therapist skill,patient openness and treatment duration allneed to be considered as potentiallyimpacting outcomes (Orlinsky et al. 1994).

Social work researchers need toundertake a great deal more systematicinvestigation into the nature of theworker/client relationship and its influenceon outcomes. The fact that practice wisdomtells us that the ‘helping’ relationship isimportant should not lessen our motivationfor rigorous and systematic collection ofevidence to support what we know. Thisresearch is essential to counter the vestedinterests represented by practitioners ofparticular techniques or by funding bodieswho want easily quantifiable techniques tobe performing well enough to justify fundingthese interventions at the expense ofothers. Given the work that has alreadygone into developing a tool for themeasurement of the helping relationship insocial work practice, the Helping AllianceInventory (Poulin & Young 1998), it issurprising and disappointing that this tool isnot being more regularly used in socialwork research and practice.

The active agents of change withinsocial work interventions are not beingcaptured via current research designs andreflected in research findings. Althoughrecently there have been someencouraging trends, a review of social workpractice research published recentlyproposes that the star of change–processresearch is on the rise and that moreattention is now being paid to (little i)interventions and (little o) outcomes ratherthan global measures of inputs andoutcomes (Videka-Sherman & Reid 1990).

Enough research has been conducted inthe psychotherapy field to argue stronglythat attention needs to be turned away fromexclusively measuring interventiontechniques to exploring in a morecomprehensive and focused way theintervention process itself. It is certainlysurprising given the existence of this bodyof effectiveness literature in such a closelyrelated field that researchers of social workpractice are not accounting more rigorouslyfor the impact of common factors onoutcomes. It would appear intellectuallydishonest to attach positive outcomes to aparticular technique when the activeelements in a treatment were theinterpersonal interaction between a clientand worker, and/or some othercharacteristic that is common to theapplication of interventions.

It is essential that researchers andevaluators of clinical social work practiceaccount more rigorously for the range offactors that are involved in providing anintervention to a client and in evaluating theimpact of these factors on outcomes. Thereis a need to develop of a range of researchtools that can assist researchers inachieving this. The impact of such researchon social work practice would befundamental. It would enable social workers to approach their work with clientsconfident of the components of practicethat make interventions work and result inpositive changes for clients. With thisinformation we could meet our ethical duty of providing clients with the bestpossible interventions for their particularneeds.

In essence these arguments arrive at the same point made so well byGibbons.


Social work cannot afford to be complacentabout the role of research in professionalpractice. Social work practice must showaccountability to client groups, demonstratethat practice is based on a body ofknowledge, not just faith [my emphasis],and continue to build on that body ofknowledge. (Gibbons 2001)

Finally it is also important to note that theelements that make interventions work maybe far more universal across the ‘helping’professions than we want to acknowledge.This raises the concern that social workerresearchers are becoming somewhat blindto the values and experiences they sharewith other fields of practice and that thisblindness can be detrimental to expandingthe knowledge base of the profession. Isthis lack of vision also reflected in theagenda of social work effectivenessresearchers narrowly focusing on theexploration of if interventions work ratherthan why they work? This type of focus isclearly promoted in a climate where socialworkers feel the need to assess the value ofthe work they perform for the interests offunding bodies rather than for theadvancement of social work practice. Thechallenge for the social work researchcommunity in the future is to continue tobuild on its tradition of effectivenessresearch in a way that can inform theprofession about the factors influencingchange and producing positive outcomes.Such a shift in focus would ensure theusefulness of this research for the practiceof social work in its various settings.

ReferencesBERGIN AE & GARFIELD SL, (1994), Handbook ofPsychotherapy and Behaviour Change. 4th edn,John Wiley & Sons, New York.

COADY NF (1991), The association between client andtherapist interpersonal processes and outcomes inpsychodynamic psychotherapy. Research on SocialWork Practice, 1, (2), 122–138.

COADY NF (1993), The Worker-Client RelationshipRevisited. Families in Society, 74, 291–300.

EYSENK HJ (1952), The Effects of Psychotherapy: AnEvaluation. Journal of Consulting Psychology, 16, (3),19–324.

FISCHER J (1973), Is casework effective? A review.Social Work, 18 (Jan), 5–20.

GIBBONS J (2001), Effective practice: Social work’slong history of concern about outcomes. AustralianSocial Work, 54, (3), 3–13.

GOREY KM, THYER BA & PAWLUCK DE (1998),Differential effectiveness of prevalent social workpractice models: a meta-analysis. Social Work, 43, (3),269–278.

HORVATH AO & SYMONDS DB (1991), Relationbetween working alliance and outcome inpsychotherapy: a meta-analysis. Journal of CounselingPsychology, 38, (2), 139–149.

LAMBERT MJ & BERGIN AE (1994), The effectivenessof psychotherapy. In: Bergin AE & Garfield SL (eds),Handbook of Psychotherapy and Behaviour Change.John Wiley & Sons, New York.

MACDONALD G, SHELDON B & GILLESPIE J (1992),Contemporary studies of the effectiveness of socialwork. British Journal of Social Work, 22, (6), 615–643.

MARZIALI E, MARMAR C & KRUPNICK J (1981),Therapeutic alliance scales: development andrelationship to psychotherapy outcome. AmericanJournal of Psychiatry, 138, (3), 361–364.

ORLINSKY D, GRAWE K & PARKS BK (1994), Processand outcome in psychotherapy – Noch Einmal. In:Bergin AE & Garfield SL (eds), Handbook ofPsychotherapy and Behaviour Change, John Wiley &Sons, New York.

POULIN J & YOUNG T (1997), Development of aHelping Relationship Inventory for social work practice. Research on Social Work Practice, 7, (4),463–489.

POULIN JE & YOUNG TM (1998), The HelpingRelationship Inventory: A clinical appraisal. Families inSociety: The Journal of Contemporary HumanServices, March/April, 123–133.

REES S & WALLACE A (1982), Verdicts on SocialWork. Edward Arnold, London.

REID W & HANRAHAN P (1982), Recent Evaluations ofsocial work: grounds for optimism. Social Work, 27, (4),328–340.


ROTH A, FONAGY P, PARRY G & TARGET M (1996),What Works for Whom? A Critical Review ofPsychotherapy Research. Guilford Press, London.

RUSSELL M (1990), Clinical Social Work: Researchand Practice. Sage, London.

SHELDON B (1986), Social work effectivenessexperiments: Review and implications. British Journalof Social Work, 16, 223–42.

SMITH SA (2000), Getting to the Heart of the Matter – Client Reported Strength of the TherapeuticAlliance and its Impact on Outcomes in CounsellingPractice – A Study of Clients Using Problem GamblingCounselling Services. MSW. Thesis, School of SocialWork, University of Melbourne.

VIDEKA-SHERMAN L (1988), Meta-analysis ofresearch on social work practice in mental health.Social Work, 33, 325–38.

VIDEKA-SHERMAN L & REID W (eds) (1990),Advances in Clinical Social Work Research. NASWPress, USA.

WOOD KM (1978), Casework effectiveness: A newlook at the research evidence. Social Work, 23November, 437–459.

WOOLFE R & DRYDEN W (1996), Handbook ofCounselling Psychology. Sage Publications, London.

Article accepted for publication February 2001.


what works for whom: the link between process and outcome in effectiveness research

Documents