assessment of psychosocial work environment in primary care-development of a questionnaire

Journal of Medical Systems, Vol. 23, No. 6, 1999

Assessment of Psychosocial Work Environment inPrimary Care—Development of a Questionnaire

Recent studies have indicated deteriorated working conditions of health care person-nel. To have an efficient health care organization requires good working conditionsand the well-being of the personnel. Today there are no "gold-standard" assessmenttools measuring psychosocial working conditions. The aim of this study was todevelop two valid and reliable questionnaires, one generic and one specific, measuringpsychosocial working conditions for general practitioners (GPs) and district nurses(DNs) in Sweden, with a special emphasis on organizational changes. The construc-tion of the questionnaires were made after a stepwise developing phase includingliterature review, interviews, and a pilot study. The pilot study included GPs n = 42and DNs n = 39. The questionnaires were later on used in a main study (GPsn = 465, DNs n = 465). A factor analysis was carried out and showed that therewere fewer items in the main study that had factor loading > 0.40 in more than onefactor, compared to the pilot study. The factors from the main study were easier tolabel and had good correspondence with other studies. After this stepwise developmentphase good construct validity and internal consistency were established for the ques-tionnaire.

KEY WORDS: psychosocial working conditions; general practitioner; district nurses; questionnaire;factor analysis.

INTRODUCTION

The medical system and health care organizations are constantly undergoingchanges in most countries. Modern and efficient health care require personnel who

1Faculty of Health Sciences, Department of Medicine and Care, Division of Nursing Science, LinkopingUniversity, S-581 85 Linkoping, Sweden.

2Division of Primary Care, Linkoping University, Sweden.3Division of Preventive and Social Medicine and Public Health Science, Department of Health andEnvironment, Linkoping University, Sweden.

4Primary Health Care Research and Development Unit, County Council of Ostergotland.5To whom correspondence should be addressed at Department of Medicine and Care, Division ofNursing Science, Faculty of Health Sciences, Linkoping University, SE-581 85 Linkoping, Sweden.Phone: +46 13 221788, Fax: +46 13 123285.

447

0148-5598/99/1200-0447$16.00/0 © 1999 Plenum Publishing Corporation

Susan Wilhelmsson,1,4,5 Ingemar Akerlind,3,4 Tomas Faresjo,2 andAnna-Christina Ek1

have good psychosocial working conditions. These aspects of work environmentamong health service personnel have been in focus during recent years.(1) Severalstudies have indicated that general practitioners (GPs) experience high job stressin primary care today.(2-5) Studies of the working conditions of district nurses (DNs)show that they experience both lower demands than GPs and fewer possibilitiesto influence and control their work.(5)

The research area is comprehensive and full of nuances, and there is at presentno gold-standard assessment tool for the psychosocial work environment. The mostfrequently used method is questionnaires with a combination of generic questionsand specific questions covering the special purpose of the study.(6) Petterson andLevi(7) have compiled different instruments for measuring the psychosocial workenvironment and health in Sweden, and have found a lack of relevant, valid, andreliable instruments.

The theoretical framework used is often influenced by stress theory. The under-lying assumption is that the work environment is experienced by the individual andpossibly leads to stress reactions, which influence health and well-being. However,stress in working life is a multidimensional concept and it is important to analyzemental, social, and physical variables. Karasek and Theorell(8) have presented amodel based on the relationship between psychological demands and decision lati-tude. The model has later been expanded with the concept of social support. Theability to influence working conditions and the access to social support are regardedas vital buffering mechanisms against high working demands. The five areas com-piled in the Job Content Questionnaire (JCQ)(8) are decision latitude, psychologicaljob demands, job insecurity, physical exertion, hazardous exposure, and social sup-port. Cooper et al.(9) investigated job stressors for GPs with a questionnaire con-taining 38 items, which after a factor analysis resulted in six factors: demands,interruptions, practice administration, home interface and social life, dealing withdeath and dying, and medical responsibility for friends and relatives. Petterson andArnetz(10) tested a model existing of the domains subjective work environment,reactions/health, and modifying variables, and included it in questionnaires usedon three large samples.

Studies of psychosocial working conditions require a large set of variables. Toreduce them into more manageable data, factor analysis is commonly used.(11) Theadvantage is that you get a smaller set of new composite dimensions with a minimumloss of information and the results are easier to report and compare with resultsof other studies. Knapp and Campell-Heider(12) maintain that there are too manystudies carried out on too small samples. However, there is not one agreed conven-tion about the relationship between observations and items, but a general rule isto use as large sample as possible.(13) One of the most popular conventions is the"ten to one-rule," which express the relation between the number of observationsand the number of variables.(13,14) Other recommendations are Ackerman andLohnes's,(15) the number of observations divided with the number of variables shouldbe >20. Harris(16) proposes that the number of observations subtracted to the numberof variables >30, Lawley and Maxwell(17) suggested 51 more cases than observations,and Catell(18) 250 subjects or more.

In primary health care in Sweden many organizational changes have occurred

448 Wilhelmsson, Akerlind, Faresjo, and Ek

during the last decade. One example is the personal doctor reform, whose aim,among others things, was to shift the health care system from a hospital-based toa more primary-health-care-based system. These changes have of course influencedthe working conditions of the two key professions, GPs and DNs.(5) They worktogether organizationally with the common responsibility for almost the same pa-tients, who are dependent on their collaboration. Therefore it is very important tofollow and compare their psychosocial work environment, especially in times oforganizational changes. The results depend on valid and reliable instruments.

AIM

The aim of this study was to compile and develop two valid and reliablequestionnaires, one generic and one specific, measuring psychosocial working condi-tions for GPs and DNs with a special emphasis on organizational change.

METHOD

Instrument Development

The construction of the questionnaire was made after a literature review, keyinformant interviews, and after studying other questionnaires used with a similarpurpose. The questionnaire was then tested in a pilot study and used in the mainstudy.

The key informant interviews had a qualitative approach and included fiveDNs and four GPs. The interviewed subjects were selected to represent as manydifferent aspects of the primary health care organization as possible (urban-rural,small-large work place, different work assignments). All interviews were tape-recorded and transcribed, and lasted about 35-60 min each. A general interviewguide approach was used according to Patton.(19) There were only small differencesin the guide between the two professions.

A questionnaire was constructed to cover individual characteristics, objectivework environment, subjective work environment, reactions, and perceived health.Categories generated from the interviews were operationalized and included in thequestionnaire. These categories were cooperation, demands, organizational change,relations, and the experience of working at a surgery without prior appointment.

A questionnaire composed of 81 questions was used in the pilot study. Thefirst 10 questions covered individual characteristics and objective work environment,age, sex, years spent in primary health care, years spent in present workplace,degree of duty, number of employed at workplace, number of GPs and DNs atworkplace, and type of employment. Four questions dealt with two different tasksthat had recently been introduced in primary care, namely, DNs authority to writeout prescriptions and the right for patients to visit a doctor without having a priorappointment. The subjective work environment, reactions, and state of health weremeasured with 27 questions covering data on how the new personal doctor reform

Assessment of Psychosocial Work Environment 449

had influenced their working conditions and 38 items measuring psychosocial work-ing conditions and job satisfaction in common. Finally there were two open-endedquestions concerning the working conditions and feedback on the questionnaire,respectively. Most of the questions were of "Likert-type" with a 5-point scale andthe items ranged, in the organizational change section, from better (score 1) tounchanged (score 3) to worse (score 5). In the psychosocial work environmentsection, the items range from very seldom (1) to sometimes (3) to very often (5).

On the basis of the experiences from the pilot study a revision of the question-naire was made for the main study, which finally contained 77 items. In the partmeasuring individual characteristics and objective work environment, a small revi-sion was made among the response alternatives in the questions. In the subjectivework environment section and modifying varables three questions were excluded(ability to influence staffing, time to talk to patients, ability to take free time atshort notice) and some got a different wording. The final questionnaire contained26 items that dealt with the experience of the personal doctor reform, and 36 itemsabout psychosocial working conditions in common. One item was open ended.

Subjects

The samples were selected from a register held by the company PharmaceuticalStatistics Inc. For the pilot study, GPs (n = 56) and DNs (n = 45) from two countycouncils in Sweden were randomly selected. The questionnaire was mailed to therespondents and after one reminder the questionnaire was returned by 42 GPs and39 DNs. The overall response rate was 80%.

In the main study, GPs (n = 566) and DNs (n = 554) in 4, out of 26, countyand municipalities participated were selected. Two of the county councils hadcarried out a far-reaching implementation of a personal doctor system and two hadonly made minor modifications. The subgroups differed in number. Thus, to makethem relatively equal, a 50% sampling was made from three groups of DNs andone group of GPs. For the remaining GPs and DNs, the entire population in theregister was included. The questionnaire was mailed to 536 GPs and DNs with apersonal doctor system and 574 GPs and DNs with a traditional primary healthcare system. After one reminder the questionnaire was returned from 437 GPs andDNs in the personal doctor system and 493 from the traditional primary care. Fiveadditional questionnaires (four GPs and one DN) were not possible to identifywith respect to which county council they belonged to. The overall participationrate was 83%.

STATISTICAL ANALYSIS

The principal component factor analysis with varimax rotation was calcu-lated.(11) A factor analysis with the criterion of not using eigenvalues > 1.0 wasmainly used, and sometimes the numbers of factors were specified. Two of thereported requirements, for proportions between observations and items in the factor



analysis, were fulfilled in the pilot study and all of them in the main study (TableI). The statistical program StatView for Macintosh was used.

Validity

To investigate construct validity and reduce variables into more manageabledata, a factor analysis was carried out.(11) In studies of psychosocial working condi-tions validity can be hard to achieve, but by asking more than one question thatrefers to similar topics, differences between respondents can be minimized and ahigher validity can be achieved.(20) With respect to the essential content of the itemsfour researchers were asked independently to name the factors. The labeling wascompared and the final names of the factors were established.

Reliability

The internal consistency of each factor derived from the factor analysis wasassessed by Cronbach's alpha.(21)

RESULTS

The Personal Doctor Reform

The factor analysis produced six factors in the pilot study and five in the mainstudy (Table II). In the pilot study the six factors had a total variance of 70%. Thefactors in the pilot study were labeled professionalism, holistic view, stimulation atwork, fellowship at work, influence and control, and demands on the individual.There were 10 items that had a factor loading >0.40 in more than one factor. Theseitems were included in the factor where they had the highest loading. The coefficientalpha varied from 0.70-0.89.

Table I. Comparison Between Proposed Conventions for Factor Analysis in the Pilot Study and in theMain Study

Convention

"Rule of thumb" 10:1

nlv > 20a

n-v > 30

51 more cases

>250 subjects

Source

Marascuilo and Levin (1983),Kerlinger (1986)

Ackerman and Lohnes (1981)

Harris (1985)

Laelwy and Maxwell (1971)

Catell (1978)

Pilot studyn = 81

Number of items26-34

Conventionnot fulfilled


Conventionfulfilled

Conventionfulfilled


Main studyn = 935

Number of items26-32

Conventionfulfilled

Conventionfulfilled

Conventionfulfilled

Conventionfulfilled

Conventionfulfilled

an = number of observations, v = variables.

In the main study the item "ability to influence staffing" had a factor load-ing <0.40 in all factors and was excluded in the main study. The item "cooperationbetween GPs and DNs" got a new instruction in the questionnaire and was included.The factor analysis resulted in five factors with a total variance of 65% and onlythree items with loading >0.40 in more than one factor. Cronbach's alpha variedfrom 0.75-0.89. A comparison between the pilot and the main study shows thatfour factors represent the same conceptual dimension (see Table II).

Psychosocial Working Conditions in General

In the pilot study the part of the questionnaire that measured psychosocialwork environment in common, 35 of the 38 items were included in the factoranalysis. Two with dichotomous variables and the one about salary were excluded.The factor analysis resulted in nine factors. Three of the factors only containedone item each and many items had factor loading >0.40 in several factors. Thereforea new analysis was carried out and seven factors were specified and got an overallvariance of 65.4% (Table III). The factors were labeled workload, social supportat work, physical strain, autonomy, quality of work, job control, and relaxation.The Cronbach's alpha coefficient varied from 0.68-0.88. Five items had factor load-ing >0.40 in more than one factor. The item "possibility to take free time withshort notice" had factor loading <0.40 in all factors and was excluded.

In the main study 32 items were included in the factor analysis. The factoranalysis resulted in six factors. There were still some items that had a factor loading>0.40 in more than one factor, and the coefficient alpha was low in the last factor.Therefore a factor analysis with five factors was calculated. The overall variancewas 56% and the alpha coefficient varied from 0.79-0.86. The factors were labeledstrains and symptoms, professional content, social support at work, work load andjob control. Three of the five factors had the same wording as in the pilot study(see Table III).

DISCUSSION

The main objective of this study was to construct two valid and reliable ques-tionnaires measuring psychosocial factors at work in primary health care. After astepwise development phase including literature and instrument review, key infor-mant interviews with target groups and a pilot study, an extensive final evaluationcould establish good construct validity and internal consistency. Another aim wasto design the instruments for both GPs and DNs. Most studies focus on either ofthe professions, doctors or nurses.(22,23) To make a comparison possible it wasimportant to do the measuring at the same time with the same questionnaire. GPsand DNs work together in primary health care at the same, or almost the same,workplace. However, they represent two different occupational cultures. In timesof organizational changes, it is most valuable to compare the experience from thepersons involved.

Studies of psychosocial working conditions need a multidimensional approach,


which makes great demands on reliable and valid instruments. In this study theinstruments have been accomplished in several steps. It is not always possible touse already developed measurements, because the area is very complex and eachworking situation has its own special prerequisite. It is advisable to use both alreadydeveloped generic questions and more specific questions. The introductory inter-views were valuable to capture new aspects of working conditions from the specialgroups focused on in this study.

To obtain stability in the outcome when multivariate analysis is used it isimportant to have a large sample size. This study illustrates that many of theconventions suggested for factor analysis are too moderate concerning sample size.In the pilot study it was sometimes difficult to establish the labels on the factorsand more items, compared to the main study, had a high factor loading in morethan one factor. Nevertheless, three of the factors in the main study had labels thatcorresponded with the labels in the pilot study. In the literature the relationshipbetween the number of observations and items is not always clearly stated. Lookingat the conventions reported in this study it is clear that some of them are toomoderate in their required number of observations.(15,16) It is very important to usea large number of observations in proportion to items. It is, of course, very costlyto have a large sample size but to follow the Lawley and Maxwell's criterion(17) (51cases more than items) as Mainous(24) recommends, is not recommended either.However, factor analysis is a useful statistical method in studies where a lot ofvariables are essential and when the aim is instrument development. It is possibleto report the results from the analysis both as an index and to use them separately.

It is not always possible to compare results from other studies as work assign-ments differ between countries. The factors generated in this study, however, corre-spond well with factors from other studies(8,9) if allowances are made for the differ-ences between occupations in different countries. Denominating the factorsaccording to the essential content of the items is the subjective part of the factoranalysis. The labels used in other studies can, of course, influence, but by lettingdifferent researchers independently of each other name the factors this risk can becounteracted. It is, however, important that the conformity between studies of thepsychosocial work environment be good.

The content of the questionnaire, developed from a literature survey, keyinformant interviews, studies of other questionnaires and testing in a pilot studyshows high construct validity. Each factor, as well as the whole questionnaire, hashigh internal consistency. The questionnaire is relatively short and easy to use,compared to some other questionnaires, and have both high validity and reliability.

ACKNOWLEDGMENTS

The study was initiated and supported by the Swedish National Board of Healthand Welfare. A special thanks for support to Mats Ribacke, MD, PhD, NationalBoard of Health and Welfare Stockholm, Sweden.


REFERENCES

1. Petterson, I-L., and Arnetz, B., Perceived relevance of psychosocial work site interventions forimproved quality of health care work environment. J. Nurs. Sci. Nordic Countries 18(1):4-10,1997.

2. Sutherland, V.J., and Cooper, C.L., Identifying distress among general practitioners: predictors ofpsychological ill-health and job dissatisfaction. Soc. Sci. Med. 37(5):575-81, 1993.

3. Orozco, P., and Garcia, E., The influence of workload on the mental state of the primary healthcare physician. Fam Pract. 10(3):277-82, 1993.

4. Winefield, H., Murrell, T., and Clifford. J., Sources of occupational stress for Australian GPs, andtheir implications for postgraduate training. Fam. Pract. 11(4):413-17, 1994.

5. Wilhelmsson, S., Faresjo, T., Foldevi, M., and Akerlind, I., The Personal Doctor Reform in Sweden—Perceived changes in working conditions. Fam. Pract. 15(3):192-97, 1998.

6. Michelsen, H., Harenstam, A., and Nordemar, R., Background and method testing of the question-naire: Psychosocial issues. In (M. Hagberg, and C. Hogstedt, eds), Evaluation of Methods Used forthe Measurement of Health and Environmental Exposure in Studies of Locomotor Disorders, MUSICBooks, Stockholm, 1993 (in Swedish).

7. Petterson, I-L., and Levi, L., Measuring Psychosocial Work Environment and Health. Developmentof a Data Base for Psychosocial Work Environment and Health, Karolinska Institutet, Section ofStress Research, Stress Research Reports No 258. Stockholm, 1995.

8. Karasek, R., and Theorell, T., Healthy Work: Stress, Productivity, and the Reconstruction of WorkingLife, Basic Books Inc, New York, 1990.

9. Cooper, L.C., Rout, U., and Faragher, B., Mental health, job satisfaction, and job stress amonggeneral practitioners. BMJ 298:366-370, 1989.

10. Petterson, I-L., and Arnetz, B.B., Measuring psychosocial work quality and health: Developmentof health care measures of measurement. J. Occupat. Health Psychol. 2(3):229-41, 1997.

11. Gorsuch, R.L., Factor Analysis, (second edition), Lawrence Erlbaum Assosiates Inc, Edn. Hillsdale,NJ, 1983.

12. Knapp, R., and Campell-Heider, N., Numbers of observations and variables in multivariate analyses.Western J. Nurs. Res. 11(5):634-41, 1989.

13. Kerlinger, F.N., Foundations of Behavioural Research (third edition), Holt, Rinehart and Winston,Inc, Orlando. Florida, 1986.

14. Marasculio, L.A., and Serlin, R.C., Statistical Methods for the Social and Behavioral Sciences, W.H.Freeman and Company, 1988.

15. Ackerman, W.B., and Lohnes, P.R., Research Methods for Nurses, McGraw-Hill, 1981.16. Harris, R.J., A Primer of Multivariate Statistics (second edition), Academic Press, New York, 1985.17. Lawley, D.N., and Maxwell, A.E., Factor Analysis as a Statistical Method, Butterworth & Co,

London, 1971.18. Catell, R.B., The Scientific Use of Factor Analysis in Behavioral and Life Sciences, Plenum Press,

New York, 1978.19. Patton, M.Q., Qualitative Evaluation and Research Methods (second edition), SAGE, 1991.20. Theorell, T., Michelsen, H., and Nordemar, R., Testing of the validity of the indices in psychosocial

index formations. In (M. Hagberg, and C. Hogstedt, eds.), Evaluations of Methods Used for theMeasurement of Health and Environmental Exposure in Studies of Locomotor Disorders, MUSICBooks, Stockholm, 1993 (in Swedish).

21. Cronbach, L.J., Coefficient alpha and the internal structure of the test. Psychometrika 16:297-334, 1951.

22. Appelton, K., House, A., and Dowell, A., A survey of job satisfaction, sources of stress andpsychological symptoms among general practitioners in Leeds. Br. J. Gen. Pract. 48:1059-63,1998.

23. Doncevic, S., Theroll, T., and Scalia-Tomba, G., The psychosocial work environment of DNs inSweden. Work Stress 2(4):341-51, 1988.

24. Mainous III, A.G., Factor analysis as a tool in primary care research. Fam. Pract. 10(3):330-6,1993.


assessment of psychosocial work environment in primary care-development of a questionnaire

Documents