development and psychometric evaluation of the milwaukee psychotherapy expectations questionnaire

Development and Psychometric Evaluation of the Milwaukee PsychotherapyExpectations Questionnaire

�

Melissa M. Norberg,1 Chad T. Wetterneck,2 Daniel A. Sass,3

and Jonathan W. Kanter4

1National Cannabis Prevention and Information Centre, University of New South Wales2University of Houston-Clear Lake3University of Texas-San Antonio4University of Wisconsin-Milwaukee

The Milwaukee Psychotherapy Expectations Questionnaire (MPEQ) was developed to measure clients’

expectations about the components and effects of therapy. Items were generated rationally based upon

the theoretical literature and existing expectancy measures. An exploratory factor analysis revealed a

2-factor solution, comprised of Process Expectations and Outcome Expectations, which was supported

by confirmatory factor analyses in three additional samples. The measure demonstrated good internal

consistency and test-retest reliability, along with support for convergent, discriminant, and predictive

validity. These results present initial evidence for the utility of the MPEQ in assessing both process and

outcome expectations in therapy. & 2011 Wiley Periodicals, Inc. J Clin Psychol 67:574–590, 2011.

Keywords: expectancy; psychotherapy research; psychotherapy process; measurement

Premature termination, or psychotherapy dropout, significantly reduces the clinical andfinancial effectiveness of mental health services. Community mental health center data(National Institute of Mental Health [NIMH], 1981) and a meta-analysis of 125

psychotherapy studies (Wierzbicki & Pekarik, 1993) indicate that almost half of patientswho start therapy terminate early. Similarly, 25–62% of individuals who initiate therapy maynever show up for their first appointment (Festinger, Lamb, Marlowe, & Kirby, 2002;Livianos-Aldana & Vila-Gomez, 1999; Ritchie, Jenkins, & Cameron, 2000). Discovering what

factors contribute to therapy dropout may help researchers and clinicians to develop methodsfor increasing therapy continuation and ultimately improve treatment outcomes.

Researchers have hypothesized that premature termination may be related to client’s

expectancies about treatment. If these expectancies are not met in therapy, the client maybecome dissatisfied and drop out of treatment (Garfield, 1994). In a review of the early therapyliterature, Garfield (1986) found that clients who withdrew from treatment early often had the

least accurate beliefs about the therapist role. In addition, premature terminators generallyexpected to play a passive role in treatment and to receive specific advice or medical treatment.Later research has found a positive association between the expected length of treatment and

actual length of treatment (Clarkin & Levy, 2004; Jenkins, Fuqua, & Blum, 1986; Mueller &Pekarik, 2000). A recent meta-analytic review found moderate effects sizes for the relationshipbetween therapy dropout and treatment expectations (Sharf & Primavera, unpublishedmanuscript), including expectations about the duration of treatment, therapy tasks, and the

potential outcome. Importantly, no two studies included in the meta-analysis examined thesame type of treatment expectations. In fact, all published reviews on client expectancies have

�This article was reviewed and accepted under the editorship of Beverly E. Thorn.

The authors wish to thank Dr. Mike Hynan for his dedication to training students in process and outcomeresearch and for his assistance with this project.

Correspondence concerning this article should be addressed to: Melissa M. Norberg, National CannabisPrevention and Information Centre, University of New South Wales, PO Box 684, Randwick, NSW, 2031,Australia; e-mail: [email protected]

JOURNAL OF CLINICAL PSYCHOLOGY, Vol. 67(6), 574--590 (2011) & 2011 Wiley Periodicals, Inc.Published online in Wiley Online Library (wileyonlinelibrary.com/journal/jclp). DOI : 10.1002/ jc lp .20781

acknowledged the problem of inconsistent measurement and identified the need to develop acomprehensive, standardized instrument (Duckro, Beal, & George, 1979; Glass, Arnkoff, &

Shapiro, 2001; Greenberg, Constantino, & Bruce, 2006; Nobel, Douglas, & Newman, 2001;Tinsley, Bowman, & Ray, 1988). Therefore, our purpose here is to describe the developmentand psychometric evaluation of a scale that measures clients’ expectations about the

components and effects of psychotherapy. Such a measure may allow for the prediction ofwhich individuals are at risk for dropout, so that interventions that may reduce attrition canbe applied, and ultimately improve treatment outcomes.

Client Expectancy

Researchers have identified two broad classes of expectations: expectations about the process

of therapy and expectations about the outcome of therapy (Glass et al., 2001; Nobel et al.,2001). Expectations about the process of therapy refer to patients’ beliefs about what willhappen during therapy, including the behaviors of the therapist and the client (i.e., role

expectations), the procedures that will occur, and the length of treatment. Outcomeexpectations, sometimes referred to as prognostic expectations, refer to patients’ expectationsfor improvement and the expected helpfulness of therapy.

Lambert and Barley (2001) estimated that 15% of therapeutic improvement is due toexpectancy effects. Although not all studies support Lambert’s conclusion, most studiesdemonstrate a positive correlation between client expectancies and outcome (for reviews, seeArnkoff, Glass, & Shapiro, 2002; Nobel et al., 2001). Research has identified at least three

ways that client expectancies may influence therapy (Tinsley et al., 1988). First, individuals’expectations about therapy may predict who enters therapy (Tinsley, Brown, de St. Aubin, &Lucek, 1984). Second, client expectancies may influence how long someone remains in therapy

(Clarkin et al., 2004; Jenkins et al., 1986; Mueller & Pekarik, 2000). Third, client expectanciesmay moderate the effectiveness of therapy (Joyce, Ogrodniczuk, Piper, & McCallum, 2003;Westra, Dozois, & Marcus, 2007). For example, favorable expectancies about the outcome of

treatment may motivate individuals to follow therapists’ instructions and complete therapeutictasks that may be unpleasant, whereas inconsistent expectancies about the process of therapymay lead individuals to distrust the therapist and exert little effort in therapy (Constantino,

Arrow, Blasey, & Agras, 2005; Joyce et al., 2003; Meyer et al., 2002). Furthermore, clients’expectations about therapy may influence the therapeutic alliance, which in turn, may affecthow long someone remains in therapy and the overall effectiveness of therapy.

Expectancy Measures

As mentioned earlier, reviewers of the expectancy literature have commented that the typical

instruments used in this field have poor or unestablished psychometric properties (Duckroet al., 1979; Glass et al., 2001; Greenberg et al., 2006; Nobel et al., 2001; Tinsley et al., 1988). Ingeneral, most instruments have focused on outcome expectations and have been idiosyncratic

and brief. Researchers typically have used only one (Borkovec & Nau, 1972; Joyce et al., 2003;Sotsky et al., 1991), two (Constantino et al., 2005; Meyer et al., 2002), or three items (Devilly &Borkovec, 2000) to assess outcome expectations, which is psychometrically unadvisable because

such methods decrease the likelihood of obtaining significant results (due to a lack ofsensitivity, specificity, and reliability; Marsh, Hau, Balla, & Grayson, 1998).

The role-expectation literature similarly is limited by poor measurement properties, with theexception of two measures that provide a more comprehensive evaluation of these constructs.

The Psychotherapy Expectancy Inventory-Revised (PEI-R) contains 30 items (24 critical itemsand 6 filler items) that assess clients’ expectations of their own behaviors and the therapists’behaviors during therapy. The PEI-R subscales have high internal consistency and test-retest

reliability with 1- to 4-week intervals. Using exploratory factor analysis, Berzins (1971; as citedin Bleyen, Vertommen, Vander Steene, & Van Audenhove, 2001) identified four factorsmeasured by the PEI-R: approval-seeking, advice seeking, audience-seeking, and relationship-

seeking. Subsequent factor analytic research suggested the possibility of a 5-factor model as

575Psychometric Evaluation of the MPEQ

the original approval-seeking factor appeared to consist of two identifiable factors: approval-seeking and impression (Bleyen et al., 2001). The second identified role expectation

instrument, the Expectations About Counseling Scale (EAC; Tinsley, Workman, & Kass,1980) contains 135 items that measure 17 different types of expectancies. Factor analysisindicated that the EAC has four underlying dimensions: personal commitment, facilitative

conditions, counselor expertise, and nurturance. The internal consistency of the four factorsranged from adequate to excellent (Tinsley et al., 1980). Sixty-six items have been extractedfrom this instrument to create the Expectations About Counseling Scale-Brief Form (EAC-B;Tinsley, 1982). Unfortunately, studies examining the brief version’s factor structure have

produced different solutions. Ægisdottir, Gerstein, and Gridley (2000) conducted confirma-tory factor analyses to examine the various existing models and found that both the 3- and4-factor solutions produced a poor model fit and sizable cross-loadings. Collectively,

psychometric research with these measures demonstrates considerable concern and a need formeasures with better psychometric evidence.

Expanding the Scope of Therapeutic Expectancy

Although the PEI-R and EAC/EAC-B represent advancements in the assessment of client

expectations, they are not without their psychometric and theoretical limitations. Forexample, the PEI-R is restricted to the examination of role expectations, and although theEAC considers outcome expectations, at 135 questions the EAC is too long for clients to

complete routinely in clinical practice. In addition, the unknown factor structure of the EAC-Braises important concerns about its construct validity. Given the problems associated withextremely brief instruments and the problems associated with the existing longer measures, weinitiated the development of the Milwaukee Psychotherapy Expectations Questionnaire

(MPEQ) to curtail both practical and psychometric limitations of the previous measures.Considering a number of theoretical and empirical perspectives (e.g., Austin & Vancouver,

1996; Bordin, 1979; Carver & Scheier, 1998; Devilly & Borkovec, 2000; Glass et al., 2001;

Goldstein, 1962; Kirsch, 1990; Nobel et al., 2001) we conceptualized clients to hold two broadtypes of expectancies: process and outcome expectations. We hypothesized that processexpectations would encompass beliefs about client characteristics, therapist qualities,

treatment structure, change processes, and the therapeutic relationship. In this way,components of each of the five identified common factors were represented (Grencavage &Norcross, 1990). This conceptualization served as a frame of reference for constructing itemsfor the initial version of the MPEQ. Given the interrelated nature of the elements of

psychotherapy, generated items were allowed to target more than one component within eachbroad class. For example, the item ‘‘I expect that I will tell my therapist if I have concernsabout therapy’’ can be seen to represent both client characteristics and the therapeutic

relationship. We hypothesized that outcome expectations would include beliefs about thepotential ways in which personalities and behaviors may change and beliefs about ultimateglobal outcomes.

We conducted four studies to develop and provide preliminary psychometric evidence of theMPEQ. The primary purpose of the first study was to generate the MPEQ items and refine thescale using exploratory factor analysis. The purposes of the second and third studies were to

reevaluate the factor structure of the MPEQ using confirmatory factor analysis and to furtherexamine the psychometric properties of the MPEQ. To this end, we assessed internalconsistency, test-retest reliability, and convergent and discriminant validity in undergraduatesamples. As evidence of convergent validity, we hypothesized that the MPEQ would be

significantly and positively correlated with the PEI-R and with measures of optimism and self-efficacy. On the other hand, we expected a measure of pessimism to be negatively correlatedwith the MPEQ as evidence of discriminant validity. In addition, we did not anticipate that

measures of anxiety and depression would be related to the MPEQ. The primary purpose ofthe final study was to assess the predictive ability of the MPEQ. We hypothesized thatindividuals with stronger expectations for treatment would be more likely to present to

treatment after participating in an intake interview at a university psychology-training clinic.

576 Journal of Clinical Psychology, June 2011

Method

Development of the MPEQ and examination of its psychometric properties were conducted inthree nonclinical samples and a clinical sample at a large university in the Midwest UnitedStates. Participants from the first three samples received extra credit for an undergraduate

psychology course for completing the MPEQ. The fourth sample consisted of clients whoparticipated in an intake interview for services in a psychology-training clinic. Clientsgenerally were community members and not students. The research was approved by the

university’s institutional review board.The first version of the MPEQ contained 28 items. The expectancy items were rationally

chosen by a working group of psychology faculty and graduate students based upon the

theoretical literature related to expectancy (e.g., Lambert, 2004) and previous expectancyscales (e.g., Devilly & Borkovec, 2000). Each expectancy item was presented in an 11-pointresponse-scale format, ranging from ‘‘05 not at all’’ to ‘‘105 very much so,’’ with theexception of four items. Three of the four items inquired about improvement (‘‘How much do

you think you will improve?’’ ‘‘How much do you feel you will improve?’’ ‘‘How satisfied doyou expect to be with treatment results?’’) and required participants to respond usingpercentages ranging from 0 to 100% (e.g., 10%, 20%, 30%, etc.). The fourth item (‘‘What best

describes your expectations about what is likely to happen as a result of your treatment?’’)used unique labels (i.e., 0—I expect to feel worse, 5—I expect to feel about the same, and 10—Iexpect to feel completely better). In addition to questions about expectations, participants also

were asked to nominate whether they had felt dissatisfied with life in the past 2 weeks byanswering ‘‘yes’’ or ‘‘no’’ at the beginning of the survey. Prior to answering the expectancyquestions, nonclinical participants read the following instructions:

Now I want you to imagine that you are experiencing a sufficient amount ofdistress and dissatisfaction with life and are considering seeking therapy for thisdistress and dissatisfaction. If you are currently feeling a lot of distress, then you

do not need to imagine, just focus on how you feel right now. However, if you arenot currently experiencing a lot of distress, imagine that you are and that life isnot going the way you want it to go. Imagine that you are thinking about talking

to a therapist. Given any expectations that you have about therapy, please answerthe following questions. Thank you.

Statistical Analyses

Exploratory factor analysis. Sample 1 data were analyzed using an exploratory factor

analysis (EFA) within Mplus 6 (Muthen & Muthen, 1998–2010). Analyses were performed ona polychoric correlation matrix using a weighted least-squares with mean and variance(WLSMV) estimation procedure with an oblique Geomin rotation. The Geomin rotation was

selected given that it was designed to minimize cross-loading, while reducing the interfactorcorrelation (Browne, 2001; Sass & Schmitt, 2010). A final factor solution that producedunidimensional factors was desired (i.e., a solution with good model fit and minimal cross-loadings), thus items with sizable cross-loadings were targeted for removal.

Due to the number of cross-loading items, Sample 1 responses to expectancy items weresubjected to a number of EFAs. The first EFA determined the number of factors to retain. Thenumber of factors to retain was determined using the following criteria: (a) parallel analysis,

(b) magnitude of eigenvalues, and (c) model fit statistics. After the number of factors wasdetermined, items were removed in the following order: (1) items that did not load on anyfactor (small estimated factor loadings, lr0.40), and (2) items that possessed large cross-

loadings (defined as having factor pattern loadings 40.30 on more than one factor). Itemswere removed independently until an approximate simple structure was obtained.

Confirmatory factor analysis. Several confirmatory factor analyses (CFAs) with

different samples were conducted to replicate the factor structure model proposed by the


Sample 1 results. The CFAs were conducted within Mplus 6 using a polychoric correlationmatrix and a WLSMV estimator. The validity of these models was based on the model fit

statistics and the estimated standardized factor-loading magnitudes. When estimating theseCFA models, the first unstandardized factor loading (i.e., reference indicators) on each factorwas set to 1.00 for latent variable scaling and statistical identification. Each factor loading

estimated is provided in Table 1, with the residuals assumed to be uncorrelated.

Factor analysis sample size. Comrey and Lee (1992) suggested the following samplesize guidelines for factor analyses: 1005 poor, 2005 fair, 3005 good, 5005 very good, 1,000

or more5 excellent. Recent empirical research (MacCallum, Widaman, Zhang, & Hong, 1999)revealed that the adequacy of factor analysis results depends more on the data characteristics(i.e., communalities, defined as the sum of the squared loadings for each item) than on the

sample size employed. Given that such information is unknown for the initial EFA, a largersample size (n5 599) was selected. Using the communalities estimated from Sample 1,the sample sizes for Samples 2 and 3 should be adequate based on the recommendations ofMacCallum et al. (1999). Although the clinical sample (Sample 4) was considerably smaller, it

was evaluated given its clinical significance.

Model fit criteria. The statistics employed to evaluate model fit for the EFA and CFA

were the robust w2, Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and root meansquare error of approximation (RMSEA). The standardized root mean square residual(SRMR) was used for the EFA, whereas the weighted root mean square residual (WRMR)was implemented for the CFAs. Descriptions of these model fit statistics can be obtained from

Hoyle (1995) and Hu and Bentler (1999).Chi-square statistics are nearly always large and statistically significant for complex models,

as well as when the sample size is large and variables are considerably skewed. For these

reasons, less weight was given to the w2 statistics compared to the other model fit statistics(e.g., TLI, CFI, RMSEA, and SRMR or WRMR). In general, CFI and TLI statistics greaterthan 0.90 are considered as ‘‘adequate’’ model fit, whereas values greater than 0.95 are deemed

as ‘‘good’’ model fit (Hu & Bentler, 1999). Hu and Bentler also suggested that fit indexes forRMSEA and SRMR values less than 0.06 and 0.08 are ‘‘good,’’ whereas values between 0.08and 0.10 are ‘‘mediocre.’’ Well-established acceptable values for WRMR are currently

unavailable; however, values closer to 1.00 are desirable (Yu, 2002). Note that the CFI, TLI,RMSEA, and SRMR criterion are only validated for CFA models, as they have not beenformally determined for EFA. Instead, CFI and TLI values closer to 1.00 and SRMR andRMSEA closer to 0.00 are considered better for EFA models.

Missing data and low cell counts. All missing data were treated with full informationmaximum likelihood (FIML) given that this procedure is robust when data are missing

completely at random (MCAR) or missing at random (MAR), or the percentage of missingnessis minimal (Muthen, Kaplan, & Hollis, 1987). For this study, the percentage of missing datawas negligible (0.1–0.2%), except for Sample 2 at Time 2, which had a larger percentage(22.0%) of missing data due to 18% of the sample not completing the MPEQ at Time 2.

Although responses to the MPEQ ranged from ‘‘not at all’’ to ‘‘very much so,’’ very fewparticipants rated items on the lower end of the scale. Due to the low cell count for ‘‘not at all’’responses, this response option was recoded from a zero to a one to create a more symmetrical

distribution, increase model stability, and ensure model convergence. Item distributioncharacteristics also suggested that a response scale with fewer response options might be viable.

Sample 1: Initial Evaluation Using Exploratory Factor Analysis

Participants and Procedure

Participants were 599 students enrolled in an introductory psychology course. Participantsreceived the survey during class, with instructions to complete it outside of class in a quiet

area, and return it to a mailbox in the main office of the Psychology Department. The majority


Table

1FactorAnalysisandInternalConsistency

ResultsAcross

theFourSamplesWithNoCross-Loadings

Tim

e1

Tim

e2

Sample

1/EFA

Sample

1/C

FA

Sample

2/C

FA

Sample

2/C

FA

Sample

3/C

FA

Sample

4/C

FA

Item

F1

F2

F1

F2

F1

F2

F1

F2

F1

F2

F1

F2

i1.68

.01

.69

.72

.77

.69

.49

i5.60

.05

.64

.61

.65

.66

.69

i9.74

�.01

.74

.68

.82

.82

.82

i10

.77

.00

.77

.75

.81

.80

.69

i12

.89

�.08

.82

.87

.84

.89

.84

i13

.80

.02

.82

.79

.81

.82

.72

i14

.66

�.05

.62

.66

.68

.74

.50

i17

.64

.02

.66

.56

.60

.66

.58

i24

.59

.06

.64

.56

.55

.76

.60

i19

.29

.61

.88

.83

.82

.88

.99

i20

.12

.74

.82

.75

.76

.88

.73

i21

�.03

.92

.87

.87

.91

.85

.74

i22

.00

.88

.86

.84

.88

.86

.71

r.67

.73

.74

.75

.78

.52

w2273.38

338.24

295.99

213.26

469.21

138.74

df

53

64

64

64

64

64

CFI

.98

.98

.95

.96

.95

.94

TLI

.97

.97

.94

.96

.94

.93

RMSEA

.08

.09

.13

.11

.15

.12

SRMR/W

RMR

.03

0.94

1.05

0.81

1.16

0.81

Coefficienta

.88

.90

.86

.87

.89

.91

.91

.90

.82

.84

n599

599

219

180

302

89

Notes.

EFA

5Exploratory

factor

analysis;

CFA

5confirm

atory

factor

analysis;

df5degrees

of

freedom;CFI

5Comparative

Fit

Index;TLI

5Tucker-Lew

isIndex;

RMSEA

5rootmeansquare

errorofapproxim

ation;SRMR

5standardized

rootmeansquare

residual;WRMR

5weightedrootmeansquare

residual.Thereliabilityestimates

wereidenticalfortheEFA

andCFA

usingSample1;therefore,theresultswerenotduplicated.F1andF2werenamed

RoleExpectationsandOutcomeExpectations,respectively.


of participants were female (75.8%) and Caucasian (Caucasian5 81.6%, African American5

8.2%, Asian5 3.4%, Native American5 1.8%, Hispanic5 1.6%, and Other5 3.2%).

Participants had an average age of 20.6 years (SD5 4.4). About half of the participants(56.4%) indicated that they were currently dissatisfied with life, and 41% (n5 245) endorsedattending therapy at least once in their life. Although the majority of individuals who had

therapy experience were women (n5 177), an equal proportion of men and women had someprevious therapy experience, w2(1, N5 561)5 0.01, p5 0.92. Thus, the greater number ofwomen with previous therapy experience is simply a reflection of more women in the sample.

Results

The initial EFA with 28-items revealed that a 3-factor solution existed based on the parallelanalysis, although five eigenvalues were greater than one. However, the 3-factor solutionmodel fit was relatively poor, w2(297)5 2688.26, po0.001, CFI5 0.90, TLI5 0.87,

RMSEA5 0.12, SRMR5 0.05, and a considerable number of the items had either smallfactor pattern loadings (lo.40) or large cross-loadings (l4.30). Following this analysis, 15items that did not meet the inclusion criteria above were removed.

After removing these items, a 2-factor solution was supported based on the parallelanalysis, eigenvalues greater than one rule, as well as the model fit statistics (see Table 1). Theeigenvalues were 6.90 and 1.31 for Factors 1 and 2, which explained 53.08 and 10.05 percent of

variance, respectively. As shown in Table 1, the process expectation and outcome expectationfactors contained nine and four items, respectively. The 13 items had a Fleish-Kincaid eighth-grade reading level. The estimated factor pattern loadings were always large (lZ0.40) on onefactor, with smaller (lr0.30) cross-loadings. As seen in Table 1, only one item appeared to

measure aspects of both factors (Item 19). The interfactor correlation was moderate to large(r5 0.67) in magnitude, which suggests these factors share a fair amount of variance(r2 5 0.45). Using item content, and constructs previously identified in the literature, Factor 1

was labeled Process Expectations, while Factor 2 was labeled Outcome Expectations.To supplement the EFA, a CFA also was conducted on this data as a compliment to the

other samples evaluated using CFA. Although this practice is not commonly recommended, it

was done here to provide baseline model fit statistics for the other CFA models and to testwhether the item residuals were correlated. The latter is not tested during EFA, yet correlatedresiduals will influence the CFA model fit and provide an additional assessment of whether

an additional factor exists. Results revealed a good model fit with uncorrelated item residuals(see Table 1), with excellent internal consistency estimates (i.e., coefficient as) and largestandardized factor loadings for both factors.

Table 2 provides the means and standard deviations for both MPEQ factors across gender

using the final MPEQ (see Appendix A). Partially due to differences in statistical power relatedto sample size, gender differences were found in only two of the four samples and meansfavored higher scores for women in all samples with effect sizes ranging from small to medium

based on Cohen’s (1988) tentative standards of small (||d ||5 0.20), medium (||d ||5 0.50), andlarge (||d ||5 0.80).

Samples 2–4: Final Models Tested With Confirmatory Factor Analysisand Validation

Participants and Procedure

Sample 2 consisted of 219 students, 180 of whom completed the questionnaire twice. The majorityof these participants were female (73.1%) and Caucasian (Caucasian585.8%, AfricanAmerican56.9%, Asian52.3%, Hispanic53.7%, and Other51.4%). Participants had an

average age of 20.9 years (SD54.2). A majority of the participants (55.2%) indicated that they werecurrently dissatisfied with life, and approximately half never had participated in therapy (50.2%).

This sample completed the 28-item version of the MPEQ twice, with about a 7-day interval

(M5 7.4, SD5 2.7) between administrations to examine test-retest reliability. The surveys were


completed in a quiet university office. All participants received a reminder phone call the dayprior to their second administration. Participants received the same instructions as the first study,but also completed a number of additional measures to investigate correlations with other

measures of interest. Measures of psychopathology were used to demonstrate discriminantvalidity, thus significant relationships between these measures and the MPEQ were not expected.In addition, a negative correlation between the MPEQ and a measure of pessimism was expectedas an additional check of convergent validity. Measures of role expectations for therapy and

attitudes about optimism were employed to demonstrate convergent validity; thus, significantpositive relations between these measures and the MPEQ were expected.

The third sample contained 302 undergraduate students, mostly females (79.1%),

Caucasian (Caucasian5 82.9%, African American5 7.4%, Asian5 3.3%, Hispanic5 3.7%,and Other5 2.7%), and had an average age of 21.2 years (SD5 4.3). A majority of theparticipants (53.6%) indicated that they were currently dissatisfied with life. This sample was

designed to confirm the 2-factor structure and internal consistency of the MPEQ, in additionto examining its relationship with a measure of self-efficacy. Self-efficacy was hypothesized tobe related to both MPEQ factors because many of its items were thought to be related to aclient’s belief in their ability to perform and complete expected duties in therapy.

The fourth group was a clinical sample consisting of 78 individuals who consecutivelyparticipated in an intake appointment at a university clinical psychology training clinic andwho qualified for services. Eleven of these individuals did not qualify for services because their

presenting issues were inappropriate for a training clinic (e.g., services wanted for pendinglegal issues that may require testifying in court, suicidal intent, or the primary problem wassubstance abuse/dependence) or fit better with specialty clinics that operated outside the

parameters of the general training clinic (e.g., trichotillomania). All 78 individuals were usedfor the factor analyses and reliability analyses, whereas only 71 individuals (the 55 and 16 whodid and did not attend therapy, respectively) were used for the predictive validity analysis.

Participants (n 5 78) included in the fourth sample were mostly females (59.6%) andCaucasian (Caucasian5 83.8%, African American5 14.9%, and Other5 1.4%). Participantswere on average 27.8 years of age (SD5 9.2). Participants’ average Outcome Questionnaire-45(OQ-45; Lambert et al., 2004) score was 76.5 (SD5 23.5), indicative of clinically significant

distress. Approximately 75% of the sample scored above the clinical cutoff (63) on the OQ-45.Sample 4 was used to confirm the factor structure of the MPEQ within a clinically relevant

sample and to examine if preintake MPEQ scores predicted who attended at least one therapy

Table 2Milwaukee Psychotherapy Expectations Questionnaire Mean Scores by Gender

Sample 2 Sample 2

Factor Sample 1 (Time 1) (Time 2) Sample 3 Sample 4

Process Expectations

Men 7.51 (1.52) 7.75 (1.37) 7.62 (1.31) 7.44 (1.53) 7.97 (0.91)

Women 7.94 (1.31) 7.97 (1.23) 7.94 (1.24) 7.98 (1.50) 8.40 (1.11)

t(552)5�3.17�� t(214)5�1.10 t(170)5�1.46 t(296)5�2.50� t(66)5�1.66

d5�0.27 d5�0.15 d5�0.22 d5�0.29 d5�0.41

OP5 0.76 OP5 0.16 OP5 0.25 OP5 0.47 OP5 0.27

Outcome Expectations

Men 6.47 (2.04) 6.81 (1.91) 6.29 (1.13) 6.33 (1.83) 7.17 (1.59)

Women 7.23 (1.64) 7.34 (1.45) 7.41 (1.50) 7.23 (1.87) 7.46 (1.67)

t(558)5�4.41�� t(215)5�2.19� t(175)5�3.91�� t(296)5�3.36�� t(68)5�0.69

d5�0.37 d5�0.30 d5�0.59 d5�0.39 d5�0.17

OP5 0.96 OP5 0.49 OP5 0.96 OP5 0.72 OP5 0.05

Notes. Standard deviations are provided in parentheses and the Cohen’s d is provided below the

t-statistics. OP indicates the observed power for each analysis using a5 .05.�po.05; ��po.01.


session. The MPEQ was completed prior to the intake interviews to ensure that theinterviewing clinician did not alter participants’ preexisting expectations about therapy.

Measures

Beck Anxiety Inventory. The Beck Anxiety Inventory (BAI; Beck, Epstein, Brown, &

Steer, 1988) contains 21 items that measure anxiety severity, with higher scores indicatinggreater levels of anxiety symptoms. The BAI has high internal consistency (a5 0.92) and goodtest-retest reliability over a one-week period (r5 0.75; Beck et al., 1988). The BAI also has

demonstrated adequate convergent and discriminant validity (Beck et al., 1988; Osman,Kopper, Barrios, Osman, & Wade, 1997).

Beck Depression Inventory-II. The Beck Depression Inventory-II (BDI-II; Beck, Steer,

& Brown, 1996) contains 21 items that measure depression severity. High scores indicateelevated levels of depressive symptoms. The BDI-II has displayed excellent internalconsistency (a5 0.92) and test-retest reliability over a one-week period (Pearson’s r5 0.93;

Beck et al., 1996). The measure also has demonstrated convergent and discriminant validity(Beck et al., 1996).

Psychotherapy Expectancy Inventory-Revised. The Psychotherapy ExpectancyInventory-Revised (PEI-R; Berzins, 1971) is a 30-item measure of client’s role expectations

of behavior in psychotherapy. Four factors are generated including: approval-seeking, advice-seeking, audience-seeking, and relationship-seeking. The PEI-R has demonstrated highinternal consistency (as ranging from 0.75 to 0.87) and test-retest reliability (Pearson’s r5 0.54

to r5 0.68 over a one-week interval; Bleyen et al., 2001). The scales have been used to showhow role expectations change over the course of treatment (Tracey & Dundon, 1988).

Life Orientation Test. The Life Orientation Test (LOT; Scheier & Carver, 1985) is a

12-item measure investigating dispositional optimism. There are two individual scales, optimismand pessimism, consisting of four items each. High scores on the former scale indicate greateramounts of optimism and high scores on the latter scale indicate greater amounts of pessimism. The

LOT has demonstrated good internal consistency, temporal stability, convergent validity withmeasures of self-esteem, and discriminant validity from stress and depression (Scheier & Carver,1985).

Generalized Self-Efficacy Scale. The Generalized Self-Efficacy Scale (GSS; Schwarzer& Jerusalem, 1995) is a 10-item measure of self-efficacy with higher scores indicating that aperson strongly believes that one’s actions are responsible for one’s outcome; low scoresindicate the reverse. Studies have found the GSS to have good internal consistency, temporal

stability, and construct validity, including a strong relationship with outcome expectations(Leganger, Kraft, & Røysamb, 2000; Schwarzer, Mueller, & Greenglass, 1999).

Outcome Questionnaire-45. The Outcome Questionnaire-45 (OQ-45; Lambert et al.,

2004) is a 45-item measure of symptom distress, and impairments in interpersonal relationsand social role performance. The OQ-45 has demonstrated good internal consistency and test-retest reliability (Lambert et al., 2004).

Results

Confirmatory factor analyses. Four CFAs were used to evaluate model stability and

factorial validity across three different samples, with one of the samples having both pre- andposttest data. Both nonclinical samples (Samples 2 and 3) and the clinical sample (Sample 4)revealed comparable model fit statistics, estimated factor loadings, interfactor correlations,

and internal consistency estimates (see Table 1). Model fit statistics suggested a reasonablygood fit across the samples, although the RMSEAs were greater than desirable. As might beexpected from Sample 1, the modification indices suggested that Item 19 cross-loaded with

Factor 1. However, allowing this item to cross-load often did not result in a practically (DCFI,


DTLI, DRMSEA, and dWRMR) significant increase in model fit (see Appendix B), althoughone could argue that this model is more statistically correct and would produce a smaller

interfactor correlation. The other modification indices suggested that freeing any additionalparameters would not substantially increase the model fit. Moreover, the residual varianceswere reasonably uncorrelated, which suggests that small secondary dimensions were not being

assessed. However, analyses often revealed rather high interfactor correlations (r40.70)indicating relatively poor discriminate validity for the Process Expectations and OutcomeExpectations factors. In any case, the consistently large standardized factor loadings providedstrong support for factorial and item validity.

Internal consistency. Both factors possessed high internal consistency estimates, withalpha coefficients always larger than 0.85 for Samples 2 and 3. Item analyses indicated that theaverage corrected item-total correlation was 0.64 (SD5 0.10) and 0.77 (SD5 0.06) for the

Process Expectations and Outcome Expectations factors, respectively, across Samples 2(including both time points) and 3. Similar to the factor analysis results, the item analysesindicated that each item was a good measure of the construct of interest and the correcteditem-total correlations were very stable between samples. The internal consistency estimates

also were good for Sample 4, although slightly lower than the other samples. Item analysesindicated that the average corrected item-total correlation was 0.54 (SD5 0.13) and 0.68(SD5 0.04) for the Process Expectations and Outcome Expectations factors, respectively.

Similar to the results from Samples 2 and 3, corrected item-total correlations were large forSample 4 and demonstrated that each item assessed the construct of interest. Removing itemshere would not increase the internal consistency for either factor.

Test-retest reliability. One-week test retest reliability among Sample 2 was good.

Correlation coefficients between Time 1 and Time 2 for the MPEQ Process Expectations factorwas r5 0.83, po0.001, and r5 0.76, po0.001, for the MPEQ Outcome Expectations factor.

Convergent and discriminant validity. Pearson’s correlations were employed to

examine the relationships between the MPEQ factors and measures of role expectations,optimism, pessimism, self-efficacy, and psychopathology (see Table 3). With regard toconvergent validity, both MPEQ factors correlated significantly with each factor on the PEI-R.Visual inspection revealed that the PEI-R Approval-Seeking and Advice-Seeking factors

demonstrated slightly stronger relationships with the MPEQ Outcome Expectations factor,while the MPEQ Process Expectations factor demonstrated stronger relationships with thePEI-R Relationship-Seeking factor. The correlation between the MPEQ Process Expectations

factor and the PEI-R Relationship-Seeking factor was significantly greater than the correlation

Table 3Pearson Correlations Among the Milwaukee Psychotherapy Expectations Questionnaire andHypothesized Convergent and Divergent Measures

Variable Process Expectations Outcome Expectations

BAI �.12 .04

BDI-II �.09 .03

PEI-R Approval-Seeking .19�� .31��

PEI-R Advice-Seeking .25�� .42��

PEI-R Audience-Seeking .26�� .26��

PEI-R Relationship-Seeking .53�� .35��

LOT-Optimism .18�� .08

LOT-Pessimism �.12 .08

GSS .30�� .13�

Notes: BAI5Beck Anxiety Inventory; BDI-II5Beck Depression Inventory-II; PEI-R5Psychotherapy

Expectancy Inventory-Revised; LOT5Life Orientation Test; GSS5Generalized Self-Efficacy Scale.�po.05; ��po.01.


between the MPEQ Outcome Expectations factor and the PEI-R Relationship seeking factor,Z5 2.05, po0.05. No other differences were statistically significant.

Unexpectedly, neither of the MPEQ factors was related to pessimism; however, the MPEQProcess Expectations factor demonstrated a small, but significant relationship with optimism.Both MPEQ factors were significantly positively related to the GSS, but the relationship

between the Process Expectations factor and GSS was significantly stronger than therelationship between the Outcomes Expectations factor and the GSS (Z5 2.15, po0.05). Inaddition, the MPEQ was not related to measures of depression or anxiety.

Predictive validity. Fifty-five of the 71 clients referred for treatment at the clinicalpsychology-training clinic presented for at least one session. Sixteen individuals scheduled a

first therapy session, but failed to show up without rescheduling, or refused to respond to atherapist’s attempt to schedule an initial therapy session. To evaluate predictive validity, alogistic regression was conducted to determine if Process and Outcome Expectations scores

significantly predicted treatment status (attended vs. not attended therapy). Analyses revealeda good model fit, (logit(p) of treatment status5�4.7910.69�Process Expectation10.03�Outcome Expectation), with a Nagelkerke R2 of 0.13. The Hosmer-Lemeshow goodness-of-fittest was not significant, w2 5 10.64 (8), p5 0.22. Overall correct prediction was 78.3%. Based

on these model results, Process Expectation scores significantly predicted treatment status(Wald5 5.20, p5 0.02), whereas Outcome Expectation scores did not predict treatment status(Wald5 0.02, p5 0.89). These conclusions remained even when analyses were conducted

separately for each MPEQ factor, thus the lack of predictive validity for OutcomeExpectations was not associated with possible multicollinearity concerns. Collectively, theseanalyses imply that one could use the Process Expectations scale to predict who will attend

therapy, although the Outcome Expectations scale appears less useful.

Discussion

The present research detailed the development and psychometric validation of a measure ofclient expectancies for psychotherapy. Items were generated rationally by a group of clinicalpsychology faculty and doctoral students based upon the theoretical literature related to

expectancy and previous expectancy scales. Items were chosen to target both the processes andpotential effects of therapy. Initial evaluation of these items identified a 13-item, 2-factor scale:Process Expectations and Outcome Expectations. The 9-item Process Expectations scaleconsisted of items assessing aspects of the therapist, client, therapeutic relationship, and

change processes, whereas the Outcome Expectations scale consisted of items related to howthe client may change as a result of therapy. The 2-factor structure of the MPEQ wasconsistent across three undergraduate samples and one clinical sample. In addition, the MPEQ

demonstrated good to excellent internal consistency across the four samples, and 2-week test-retest reliability was good in a nonclinical sample. Results supported the convergent anddiscriminant validity of the MPEQ. As an initial demonstration of the MPEQ’s predictive

validity, individuals who had lower Process Expectations scores were less likely to present totherapy after participating in an intake interview for service eligibility.

On the other hand, individuals who did and did not present to therapy had equally high

expectations about the outcome of participating in therapy. Theoretically, positive expectanciesare thought to motivate individuals to work towards goals (Austin & Vancouver, 1996; Carver& Scheier, 1998; Kirsch, 1990). Positive expectancies presumably influence a client’s willingnessto engage in a therapeutic relationship, work hard at therapeutic tasks that may be unpleasant,

and create a self-image of improved personal and interpersonal functioning at the conclusion oftherapy. Conversely, negative expectancies may be related to distrust of the therapist, lack oftherapeutic effort, and low motivation to achieve improvement. Because the current study

found that both treatment presenters and nonpresenters had equally high outcomeexpectations, it may be that positive outcome expectations only influence who considerstreatment. In other words, positive outcome expectations may be necessary for treatment

engagement, but not sufficient. Another explanation may be the relatively small sample size and


limited number of people not attending therapy (n5 16). Regardless, the addition of positiveprocess expectations may be sufficient in determining who attends therapy. Likewise, Tinsley

and colleagues (1984) found that respondents’ self-reported tendency to seek help fromcounseling psychologists was related to expectancies about helper attitudes, behaviors, andcharacteristics, but not to expectancies regarding counseling outcomes.

Consistent with research examining outcome expectations (e.g., Connolly Gibbons et al.,2003), the MPEQ was not related to anxiety or depression. These data provide evidence thatclients’ expectations about the process and outcome of therapy are separate from the issues thatmay lead them to seek treatment. Such results, coupled with the finding that lower process

expectations are associated with a reduced likelihood of participating in therapy, indicate thatefforts should be made during the intake interview to routinely evaluate clients’ expectations oftherapy. Individuals who report negative expectations about therapy during the intake interview

may benefit from an intervention aimed at manipulating expectancies. In a review, Tinsley andcolleagues (1988) reported that uncomplicated interventions, such as an audiotape or videotapedescribing therapy, can be effective in improving clients’ expectations about therapy. Assessing

expectations pre- and postintake will help determine if such manipulations are effective.In regards to convergent validity, both MPEQ factors were related to all four of the PEI-R

subscales. The MPEQ Process Expectations factor demonstrated the strongest relationship with

the PEI-R Relationship-Seeking factor. Inspection of item content revealed that three of the nineProcess Expectations items were similar to questions contained on this subscale of the PEI-R(more than the other subscales). Interestingly, the MPEQ Outcome Expectations factor was moststrongly related to the PEI-R Advice-Seeking factor. This suggests that clients’ beliefs regarding

the outcome of therapy may be affected more by their confidence in a therapist to provide adviceduring therapy than by themselves taking charge during the therapy hour or by revealing theirtrue thoughts and feelings during therapy. Intuitively this makes sense, as individuals who do not

believe that a therapist can engender change would be unlikely to seek therapy.Contrary to study hypotheses, indices of optimism and pessimism were not consistently

related to the MPEQ. Despite one item on the MPEQ Outcome Expectations factor assessing

improvement in optimism as a result of therapy, only the MPEQ Process Expectations factorwas related to the LOT optimism subscale. Such findings imply that one’s optimism prior tobeginning therapy may partially shape their beliefs about what will happen during therapy,

but neither their pretreatment tendency to be optimistic or pessimistic influences theirexpectations of improvement. For example, a person could expect that therapy will not bebeneficial even though he or she hopes that it will be beneficial. On the other hand, lack ofrelationships among optimism, pessimism, and Outcome Expectations could imply that other

aspects of outcome that were not assessed by the final MPEQ items, such as global outcomesor behavior change, are necessary to measure. Future research assessing these constructs in aclinical sample is needed as the current results may have been complicated by asking students

to imagine distress. Lastly, there was a stronger relationship between self-efficacy and processexpectations than outcome expectations indicating that self-efficacy is related more to what aclient expects to occur during therapy than how a client expects therapy to turn out.

Inspection of the MPEQ subscale means demonstrated that men consistently scored loweron the MPEQ than women and this difference was statistically significant in two of thesamples. In most cases, men’s subscale scores were less than half a point lower than women’sscores. These findings are consistent with previous research that has indicated that women

may have higher expectations for therapy than men (Bleyen et al., 2001), but that genderdifferences are small to medium at best.

Despite only one item having significant loadings on both MPEQ factors, the factors were

highly correlated (rZ0.52) among most samples (see Appendix B). Bleyen and colleagues(2001) reported similar findings with the PEI-R, although they had considerably more cross-loadings. The interrelated nature of the factors may suggest the need for either a

multidimensional instrument (i.e., an instrument in which items are allowed to load on morethan one factor) or the removal of Item 19. When developing items for the MPEQ we found itdifficult to create items that only targeted one-dimension within process expectations. Thus,

the items that comprise the MPEQ appear to be consistent with Grencavage and Norcross’s


(1990) finding that expectations are highly interrelated. In their review, Grencavage andNorcross found that expectations were consistently represented in three of the five common

factor categories indentified in the literature: client characteristics, therapist qualities, andchange processes. Despite the theoretical support for a partially multidimensional measure, wechose to examine a unidimensional model for ease of interpretation. The current data provide

support for such a model and suggest that the MPEQ may be stronger psychometrically thanthe EAC-B, the only other existing instrument that measures both process and outcomeexpectations. However, before recommendations can be made to routinely use the MPEQ inclinical settings; more research must be done regarding the predictive validity of the MPEQ.

For example, it remains unknown whether the MPEQ can predict the therapeutic alliance ortreatment outcome. In addition, the MPEQ needs to be validated with a larger clinical sample,in clinical settings other than a training clinic, and with individuals with differing cultural

backgrounds as the current samples. Nevertheless, the factorial validity was acceptable usingour clinical sample and promise existed given that greater discriminate validity was obtainedusing a clinical versus nonclinical samples. Despite these limitations, the current study

represents several improvements over previous research and has led to the development of thefirst psychometrically sound brief measure of both the processes and outcome of therapy.

Appendix A

Milwaukee Psychotherapy Expectations Questionnaire (MPEQ)

Below is a list of statements describing expectations about therapy that you may have. Thesestatements cover expectations regarding your own behavior in therapy, your future therapist, andthe therapy setting. Some of these expectations you may not have considered previously, howeverwe would like for you to think about them now. Read each statement carefully and circle thenumber that indicates the strength with which you find yourself expecting what it is described inthe statement.

Not at all Somewhat Very much so

1. I expect my therapist will provide support 0 1 2 3 4 5 6 7 8 9 10

2. My therapist will provide me feedback 0 1 2 3 4 5 6 7 8 9 10

3. I will be able to express my true thoughts

and feelings

0 1 2 3 4 5 6 7 8 9 10

4. I will feel comfortable with my therapist 0 1 2 3 4 5 6 7 8 9 10

5. My therapist will be sincere 0 1 2 3 4 5 6 7 8 9 10

6. My therapist will be interested in what I

have to say

0 1 2 3 4 5 6 7 8 9 10

7. My therapist will be sympathetic 0 1 2 3 4 5 6 7 8 9 10

8. I expect that I will come to every

appointment

0 1 2 3 4 5 6 7 8 9 10

9. Therapy will provide me with an increased

level of self-respect

0 1 2 3 4 5 6 7 8 9 10

10. After therapy, I will have the strength

needed to avoid feelings of distress in the

future

0 1 2 3 4 5 6 7 8 9 10

11. I anticipate being a better person as a

result of therapy

0 1 2 3 4 5 6 7 8 9 10

12. After therapy, I will be a much more

optimistic person

0 1 2 3 4 5 6 7 8 9 10

13. I expect that I will tell my therapist if I

have concerns about therapy

0 1 2 3 4 5 6 7 8 9 10


Appendix B

References

Ægisdottir, S., Gerstein, L.H., & Gridley, B.E. (2000). The factorial structure of the Expectations about

Counseling Questionnaire-Brief Form: Some serious questions. Measurement and Evaluation in

Counseling and Development, 33, 3–20.

Arnkoff, D.B., Glass, C.R., & Shapiro, S.J. (2002). Expectations and preferences. In J.C. Norcross (Ed.),

Psychotherapy relationships that work: Therapists contributions and responsiveness to patients

(pp. 325–346). New York: Oxford University Press.

MPEQ Scoring Key

Item numbers Subscale assessed

1, 2, 3, 4,5, 6, 7, 8, 13 Process Expectations

9, 10, 11, 12 Outcome Expectations

Notes. Subscale scores are calculated by summing the items included on a factor and then dividing by the

number of items included. The total score is the sum of both subscale scores.

Confirmatory Factor Analysis Models Across the Four Samples With Item 19 Cross-Loading

Time 1 Time 2

Sample 1 Sample 2 Sample 2 Sample 3 Sample 4

Item F1 F2 F1 F2 F1 F2 F1 F2 F1 F2

i1 .69 .71 .77 .69 .49

i5 .64 .60 .65 .66 .69

i9 .74 .68 .82 .82 .82

i10 .77 .75 .81 .80 .69

i12 .82 .86 .84 .89 .84

i13 .82 .79 .81 .82 .72

i14 .62 .66 .68 .74 .50

i17 .66 .56 .60 .66 .58

i24 .64 .57 .55 .76 .60

i19 .27 .63 .39 .46 .05 .77 .38 .51 .35 .70

i20 .84 .79 .76 .90 .80

i21 .88 .89 .91 .87 .79

i22 .87 .86 .88 .88 .75

r .68 .66 .74 .73 .40

w2 237.39 235.21 213.46 396.25 110.04

df 63 63 63 63 63

CFI .98 .96 .96 .96 .96

TLI .98 .95 .96 .95 .96

RMSEA .07 .11 .12 .12 .09

SRMR/WRMR 0.77 0.92 0.83 1.04 0.870

Notes. EFA5Exploratory factor analysis; CFA5 confirmatory factor analysis; df5degrees of freedom;

CFI5Comparative Fit Index; TLI5Tucker-Lewis Index; RMSEA5 root mean square error of

approximation; SRMR5 standardized root mean square residual; WRMR5weighted root mean square

residual. The reliability estimates were identical for the EFA and CFA using Sample 1; therefore, the

results were not duplicated. F1 and F2 were named Process Expectations and Outcome Expectations,

respectively.


Austin, J.T., & Vancouver, J.B. (1996). Goal constructs in psychology: Structure, process, and content.

Psychological Bulletin, 120, 338–375.

Beck, A.T., Epstein, N., Brown, G., & Steer, R.A. (1988). An inventory for measuring clinical anxiety:

Psychometric properties. Journal of Consulting and Clinical Psychology, 56, 893–897.

Beck, A.T., Steer, R.A., & Brown, G.K. (1996). Manual for the Beck Depression Inventory (2nd ed.). San

Antonio, TX: The Psychological Corporation.

Berzins, J.I. (1971). Revision of psychotherapy expectancy inventory. Unpublished manuscript.

Bleyen, K., Vertommen, H., Vander Steene, G., & Van Audenhove, C. (2001). Psychometric properties of

the psychotherapy expectancy inventory-revised (PEI-R). Psychotherapy Research, 11, 69–84.

Bordin, E.S. (1979). The generalizability of the psychoanalytic concept of the working alliance.

Psychotherapy: Theory, Research, and Practice, 16, 252–260.

Borkovec, T.D., & Nau, S.D. (1972). Credibility of analogue therapy rationales. Journal of Behavior

Therapy and Experimental Psychiatry, 3, 257–260.

Browne, M.W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate

Behavioral Research, 36, 111–150.

Carver, C.S., & Scheier, W.F. (1998). On the self-regulation of behavior. New York: Cambridge

University Press.

Clarkin, J.F., & Levy, K.N. (2004). The influence of client variables on psychotherapy. In M.J. Lambert

(Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 194–226).

New York: Wiley.

Cohen, P.A. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.

Comrey, A.L., & Lee, H.B. (1992). A first course in factor analysis. Hillsdale, NJ: Erlbaum.

Connolly Gibbons, M.B., Crits-Christoph, P., de la Cruz, C., Barber, J.P., Siqueland, L., & Gladis, M.

(2003). Pretreatment expectations, interpersonal functioning, and symptoms in the prediction of the

therapeutic alliance across supportive-expressive psychotherapy and cognitive therapy. Psychotherapy

Research, 13, 59–76.

Constantino, M.J., Arrow, B.A., Blasey, C., & Agras, W.S. (2005). The association between patient

characteristics and the therapeutic alliance in cognitive-behavioral and interpersonal therapy for

bulimia nervosa. Journal of Consulting and Clinical Psychology, 73, 203–211.

Devilly, G.J., & Borkovec, T.D. (2000). Psychometric properties of the credibility/expectancy

questionnaire. Journal of Behavior Therapy and Experimental Psychiatry, 31, 73–86.

Duckro, P.N., Beal, D., & George, C.A. (1979). Research on the effects of disconfirmed client role

expectations in psychotherapy: A critical review. Psychological Bulletin, 86, 260–275.

Festinger, D.S., Lamb, R.J., Marlowe, D.B., & Kirby, K.C. (2002). From telephone to office: Intake

attendance as a function of appointment delay. Addictive Behaviors, 27, 131–137.

Garfield, S.L. (1986). Research on client variables on psychotherapy. In S.L. Garfield & A.E. Bergin

(Eds.), Handbook of psychotherapy and behavior change (3rd ed., pp. 213–256). New York: Wiley.

Garfield, S.L. (1994). Research on client variables on psychotherapy. In A.E. Bergin & S.L. Garfield

(Eds.), Handbook of psychotherapy and behavior change (4th ed., pp. 190–228). New York: Wiley.

Glass, C.R., Arnkoff, D.B., & Shapiro, S.J. (2001). Expectations and preferences. Psychotherapy, 38,

455–461.

Goldstein, A.P. (1962). Therapist-patient expectancies in psychotherapy. New York: Pergamon Press.

Greenberg, R.P., Constantino, M.J., & Bruce, N. (2006). Are patient expectations still relevant for

psychotherapy process and outcome? Clinical Psychology Review, 26, 657–678.

Grencavage, L.M., & Norcross, J.C. (1990). Where are the commonalities among the therapeutic common

factors? Professional Psychology: Research and Practice, 21, 372–378.

Hoyle, R.H. (1995). Structural equation modeling: Concepts, issues, and applications. Thousand Oaks,

CA: Sage.

Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:

Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

Jenkins, S.J., Fuqua, D.R., & Blum, C.R. (1986). Factors related to duration of counseling in a university

counseling center. Psychological Reports, 58, 467–472.

Joyce, A.S., Ogrodniczuk, J.S., Piper, W.E., & McCalllum, M. (2003). The alliance as mediator of

expectancy effects in short-term individual therapy. Journal of Consulting and Clinical Psychology, 71,

672–679.


Kirsch, I. (1990). Changing expectations: A key to effective psychotherapy. Pacific Grove: Brooks/Cole.

Lambert, M.J. (2004). Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed.).

New York: Wiley.

Lambert, M.J., & Barley, D.E. (2001). Research summary on the therapeutic relationship and

psychotherapy outcome. Psychotherapy, 38, 261–357.

Lambert, M.J., Morton, J.J., Hatfield, D., Harmon, C., Hamilton, S., Reid, R.C., et al. (2004).

Administration and scoring manual for the OQ-45.2 (Outcome Questionnaire). Stevenson, MD:

American Professional Credentialing Services.

Leganger, A., Kraft, P., & Røysamb, E. (2000). Perceived self-efficacy in health behaviour research:

Conceptualisation, measurement and correlates. Psychology & Health, 15, 51–69.

Livianos-Aldana, L., & Vila-Gomez, M. (1999). Patients who miss initial appointments in community

psychiatry? A Spanish community analysis. International Journal of Social Psychiatry, 45, 198–206.

MacCallum, R.C., Widaman, K.F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis.

Psychological Methods, 4, 84–99.

Marsh, H.W., Hau, K.T., Balla, J.R., & Grayson, D. (1998). Is more ever too much? The number of

indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research, 33, 181–220.

Meyer, B., Pilkonis, P.A., Krupnick, J.L., Egan, M.K., Simmens, S.J., & Sotsky, S.M. (2002). Treatment

expectancies, patient alliance, and outcome: Further analyses from the National Institute of Mental

Health Treatment of Depression Collaborative Research Program. Journal of Consulting and Clinical

Psychology, 70, 1051–1055.

Mueller, M., & Pekarik, G. (2000). Treatment duration prediction: Client accuracy and its relationship to

dropout, outcome, and satisfaction. Psychotherapy, 37, 117–123.

Muthen, B.O., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not

missing completely at random. Psychometrika, 55, 107–122.

Muthen, L.K., & Muthen, B.O. (1998–2010). Mplus user’s guide. Los Angeles, CA: Author.

National Institute of Mental Health. (1981). Provisional data on federally funded community mental

health centers, 1978–79. Report prepared by the Survey and Reports Branch, Division of Biometry and

Epidemiology. Washington, DC: U.S. Government Printing Office.

Nobel, L.M., Douglas, B.C., & Newman, S.P. (2001). What do patients expect of psychiatric services? A

systematic and critical review of empirical studies. Social Science and Medicine, 52, 985–998.

Osman, A., Kopper, B.A., Barrios, F.X., Osman, J.R., & Wade, T. (1997). The Beck Anxiety Inventory:

Reexamination of factor structure and psychometric properties. Journal of Clinical Psychology, 53,

7–14.

Ritchie, P.D., Jenkins, M., & Cameron, P.A. (2000). A telephone call reminder to improve outpatient

attendance in patients referred from the emergency department: A randomized controlled trial.

Australian and New Zealand Journal of Medicine, 30, 585–592.

Sass, D.A., & Schmitt, T.A. (2010). A comparative investigation of rotation criteria within exploratory

factor analysis. Multivariate Behavioral Research, 45, 73–103.

Sharf, J., & Primavera, L.H. (2009). Meta-analysis of psychotherapy dropout. Unpublished manuscript,

Adelphi University.

Scheier, M.F., & Carver, C.S. (1985). Optimism, coping, and health: Assessment and implications of

generalized outcome expectancies. Health Psychology, 4, 219–247.

Schwarzer, R., & Jerusalem, M. (1995). Generalized Self-Efficacy Scale. In J. Weinman, S. Wright, &

M. Johnson (Eds.), Measures in health psychology: A user’s portfolio, causal and control beliefs

(pp. 35–37). Windsor England: NFER-NELSON.

Schwarzer, R., Mueller, J., & Greenglass, E. (1999). Assessment of perceived general self-efficacy on the

Internet: Data collection in cyberspace. Anxiety, Stress, and Coping, 12, 145–161.

Sotsky, S.M., Glass, D.R., Shea, M.T., Pilkonis, P.A., Collins, J.F., Watkins, J.T., et al. (1991). Patient

predictors of response to psychotherapy and pharmacotherapy: Findings of the NIMH Treatment of

Depression Collaborative Research Program. American Journal of Psychiatry, 148, 997–1008.

Tinsley, H.E.A. (1982). Expectations about counseling. Unpublished test manual. Carbondale, IL:

Southern Illinois University at Carbondale, Department of Pyschology.

Tinsley, H.E.A., Bowman, S.L., & Ray, S.B. (1988). Manipulation of expectancies about counseling and

psychotherapy: Review and analysis of expectancy manipulation strategies and results. Journal of

Counseling Psychology, 35, 99–108.


Tinsley, H.E.A., Brown, M.T., de St. Aubin, T.M., & Lucek, J. (1984). Relation between expectancies for

a helping relationship and tendency to seek help from a campus help provider. Journal of Counseling

Psychology, 31, 149–160.

Tinsley, H.E.A., Workman, K.R., & Kass, R.A. (1980). Factor analysis of the domain of client

expectancies about counseling. Journal of Counseling Psychology, 6, 561–570.

Tracey, T.J., & Dundon, M. (1988). Role anticipations and preferences over the course of counseling.

Journal of Counseling Psychology, 35, 3–14.

Westra, H.A., Dozois, D.J.A., & Marcus, M. (2007). Expectancy, homework compliance, and initial

change in cognitive-behavioral therapy for anxiety. Journal of Consulting and Clinical Psychology, 75,

363–373.

Wierzbicki, M., & Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional

Psychology: Research and Practice, 24, 190–195.

Yu, C.Y. (2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and

continuous outcomes. Unpublished doctoral dissertation, University of California, Los Angeles.


development and psychometric evaluation of the milwaukee psychotherapy expectations questionnaire

Documents