International Multicenter Psoriasis and PsoriaticArthritis Reliability Trial for the Assessment ofSkin, Joints, Nails, and DactylitisVINOD CHANDRAN,1 ALICE GOTTLIEB,2 RICHARD J. COOK,3 KRISTINA CALLIS DUFFIN,4

AMIT GARG,5 PHILIP HELLIWELL,6 ARTHUR KAVANAUGH,7 GERALD G. KRUEGER,4

RICHARD G. LANGLEY,8 CHARLES LYNDE,9 NEIL MCHUGH,10 PHILIP MEASE,11 IGNAZIO OLIVIERI,12

PROTON RAHMAN,13 CHERYL F. ROSEN,14 CARLO SALVARANI,15 DIAMANT THACI,16

SERGIO M. A. TOLOZA,1 MAXINE YAT WING WONG,1 QIAN M. ZHOU,3 AND DAFNA D. GLADMAN17

Objective. Clinical trials in psoriasis and psoriatic arthritis (PsA) involve assessment of the skin and joints. This study aimedto determine whether assessment of the skin and joints in patients with PsA by rheumatologists and dermatologists isreproducible.Methods. Ten rheumatologists and 9 dermatologists from 7 countries met for a combined physical examination exercise toassess 20 PsA patients (11 men, mean age 51 years, mean PsA duration 11 years). Each physician assessed 10 patientsaccording to a modified Latin square design that enabled the assessment of patient, assessor, and order effect. Tender jointcount (TJC), swollen joint count (SJC), dactylitis, physician’s global assessment (PGA) of PsA disease activity (PGA-PsA),psoriasis body surface area (BSA), Psoriasis Area and Severity Index (PASI), Lattice System Physician’s Global Assessment ofpsoriasis (LS-PGA), National Psoriasis Foundation Psoriasis Score (NPF-PS), modified Nail Psoriasis Severity Index(mNAPSI), number of fingernails with nail changes (NN), and PGA of psoriasis activity (PGA-Ps) were assessed. Variancecomponents analyses were carried out to estimate the intraclass correlation coefficient (ICC), adjusted for the order ofmeasurements.Results. There is excellent agreement (ICC >0.80) on the mNAPSI, substantial agreement (0.6 > ICC < 0.80) on the TJC, PASI,and NN, moderate agreement (0.4 > ICC < 0.60) on the PGA-Ps, LS-PGA, NPF-PS, and BSA, and fair agreement (0.2 > ICC <0.40) on the SJC, dactylitis, and PGA-PsA. The only measure that showed a significant difference between dermatologists andrheumatologists was dactylitis (P � 0.0005).Conclusion. There is substantial to excellent agreement on the TJC, PASI, NN, and mNAPSI among rheumatologists anddermatologists.

INTRODUCTION

Psoriasis is a chronic, immune-mediated, inflammatoryskin disease affecting at least 2.2% of the general popula-

tion in North America (1). Approximately 30% of patientswith psoriasis have psoriatic arthritis (PsA) (2). Compre-hensive assessment of patients with psoriasis and PsAinvolves assessment of the skin, nails, joints, and dactylitis(3–7).

Supported by unrestricted funds from Abbott Canada,Amgen, and Wyeth Canada, the Group for Research andAssessment of Psoriasis and Psoriatic Arthritis, and theKrembil Foundation.

1Vinod Chandran, MBBS, MD, DM, Sergio M. A. Toloza,MD, Maxine Yat Wing Wong, MRCP(UK), FRCPC: TorontoWestern Hospital, Toronto, Ontario, Canada; 2Alice Gottlieb,MD, PhD: Tufts Medical Center, Boston, Massachusetts; 3Rich-ard J. Cook, PhD, Qian M. Zhou, PhD Candidate: University ofWaterloo, Waterloo, Ontario, Canada; 4Kristina Callis Duffin,MD, Gerald G. Krueger, MD: University of Utah, Salt LakeCity; 5Amit Garg, MD: University of Massachusetts MedicalSchool, Boston; 6Philip Helliwell, MD, PhD: University ofLeeds, Leeds, UK; 7Arthur Kavanaugh, MD: University of Cal-ifornia, San Diego; 8Richard G. Langley, MD, FRCPC: Dalhou-sie University, Halifax, Nova Scotia, Canada; 9Charles Lynde,

MD: University of Toronto and Markham-Stouffville Hospi-tal, Toronto, Ontario, Canada; 10Neil McHugh, MD: Univer-sity of Bath, Bath, UK; 11Philip Mease, MD: University ofWashington School of Medicine, Seattle; 12Ignazio Olivieri,MD: San Carlo Hospital, Potenza, and Madonna delle Gra-zie Hospital, Matera, Italy; 13Proton Rahman, MD, FRCPC:Memorial University and Clare’s Mercy Hospital, St. John’s,Newfoundland, Canada; 14Cheryl F. Rosen, MD: Universityof Toronto, Toronto, Ontario, Canada; 15Carlo Salvarani,MD: Arcispedale Santa Maria Nuova, Reggio Emilia, Italy;16Diamant Thaci, MD: J. W. Goethe University, Frankfurt,Germany; 17Dafna D. Gladman, MD, FRCPC: University ofToronto and Toronto Western Hospital, Toronto, Ontario,Canada.

Arthritis & Rheumatism (Arthritis Care & Research)Vol. 61, No. 9, September 15, 2009, pp 1235–1242DOI 10.1002/art.24562© 2009, American College of Rheumatology

ORIGINAL ARTICLE

1235

To assess skin involvement, the most commonly usedmeasure, especially in clinical trials, is the Psoriasis Areaand Severity Index (PASI), which scores the average red-ness, thickness, and scaliness of the lesions (0–4 scale),weighted by the area of involvement. PASI scores rangefrom 0 to 72, where higher scores indicate more severedisease (8). Another frequently used scale is the physi-cian’s global assessment (PGA) of psoriasis activity (PGA-Ps), rated on a scale from clear or inactive to severe or mostactive (5). Although more subjective, this scale moreclosely resembles the assessment used by dermatologistsin clinical practice (9). The Lattice System Physician’sGlobal Assessment (LS-PGA) is a relatively new tool toquantify psoriasis severity that provides an 8-step resultfrom clear to very severe (10). The LS-PGA takes a quan-titative approach to global assessment by integrating theranges of the percentage of body surface area (BSA) in-volved and the overall plaque morphology. The latticeportion of the LS-PGA is performed by computerized al-gorithm. The LS-PGA shows good correlation with boththe PASI and PGA-Ps, and has very good intrarater andinterrater reliability (10,11). The National Psoriasis Foun-dation (NPF) has developed the NPF Psoriasis Score (NPF-PS) (5,12,13), a responder index that includes 6 domains:induration at 2 target sites, current and baseline BSA,PGA, patient’s global assessment, and patient’s assessmentof itch. To improve intrarater and interrater reliability ofthe induration score, the NPF-PS utilizes a reference cardembossed with elevations that increase at 0.25-mm inter-vals. The NPF-PS strongly correlates with the PASI andPGA, and better reflects patient perception (13). The as-sessment of total BSA involved by psoriasis is an impor-tant standalone measure (5). However, there are significantissues with the interrater reliability of the visual measure-ment of BSA (14,15). Nail psoriasis occurs in as many as50% of patients with psoriasis (16) and in more than 80%of patients with PsA (17). The Nail Psoriasis Severity In-dex (NAPSI) is used to evaluate the severity of nail psori-asis and was developed to evaluate the response to thetreatment of psoriatic nails in clinical trials (18). The mod-ified NAPSI (mNAPSI) (19), developed to enhance the facevalidity and feasibility of the NAPSI, demonstrated excel-lent interrater reliability in the original study (19).

The assessment of joints involves counting the numberof tender and swollen joints. Sixty-eight joints are assessedfor tenderness and 66 joints are assessed for swelling (3,4).The assessment of tender joints was found to be reliable inprevious studies conducted by a group of expert rheuma-tologists (20,21). However, the assessment of swollenjoints was found to be less reliable (20,21). Dactylitis,characterized by diffuse swelling of a finger and/or toe, isan important feature of PsA (22,23). Measures of dactylitisinclude the Leeds Dactylitis Index (LDI) (24), the assess-ment tool used in the Infliximab Multinational PsoriaticArthritis Clinical Trial (25), and a simple count of alldactylitic fingers and/or toes (26). The assessment of dac-tylitis based on the count of dactylitic fingers and/or toeswas shown to be moderately reliable (20). PGA of arthritisactivity is one of the core measurements used in clinicaltrials of inflammatory arthritides, especially rheumatoidarthritis (27). In PsA, the PGA of PsA activity (PGA-PsA)was considered to be an important but not mandatorydomain in randomized controlled trials and longitudinalobservational cohorts (7). In a recent multicenter study,the PGA-PsA measured using a 0–100-mm visual analogscale was found to be reliable (28).

The measurements outlined above are now being per-formed by both dermatologists and rheumatologists inclinical trials for psoriasis and PsA. However, it has notbeen determined whether the assessments of arthritis anddactylitis by dermatologists and the assessments of theskin and nails by rheumatologists are reliable. Therefore,the purpose of the present study was to determine whetherthe assessment of arthritis, psoriasis, dactylitis, and nailsin patients with PsA by rheumatologists and dermatolo-gists is reproducible.

PATIENTS AND METHODS

Patients. Twenty patients with PsA with varying de-grees of psoriasis and peripheral joint involvement typi-cally seen in rheumatology clinics were selected from theUniversity of Toronto PsA clinic at the Toronto WesternHospital, Toronto, Ontario, Canada. The patients consistedof 11 men and 9 women with a mean age of 51 years,psoriasis duration of 19 years, and PsA duration of 11years. All of the patients met the CASPAR (ClASsificationcriteria for Psoriatic ARthritis) criteria for the classificationof PsA (29). An attempt was made to include patients witha range of disease activity. When selected to participate inthe study by the study organizers (DDG, VC), the median(range) of these 20 patients for the disease activity mea-sures were as follows: 7.5 (0–41) for tender joint count(TJC), 1.5 (0–8) for swollen joint count (SJC), 0 (0–3) fordactylitic fingers and/or toes, 4.45 (0.4–24.6) for the PASI,3.5% (1–70%) for BSA, and 4.5 (0–10) for fingernails withnail involvement.

Assessors. The assessors consisted of a group of 10rheumatologists and 9 dermatologists who were membersof the Group for Research and Assessment of Psoriasis andPsoriatic Arthritis (GRAPPA). Although 10 dermatologistswere invited, only 9 were able to participate. All of the

Dr. Gottlieb has served on the advisory boards for Amgen,Centocor, Wyeth Pharmaceuticals, Celgene, Bristol-MyersSquibb, Beiersdorf, Abbott Laboratories, Teva, Actelion, UCB,Novo Nordisk, Immune Control, Dermipsor, Incyte, PureTech,Magen Biosciences, and Cytokine Pharmasciences. Dr. Garghas received consultant fees, speaking fees, and/or honoraria(less than $10,000) from Abbott. Dr. Langley has received con-sultant fees, speaking fees, and/or honoraria (less than $10,000each) from Amgen/Wyeth, Abbott, Centocor, and Ortho-Bio-tech. Dr. Thaci has received consultant fees, speaking fees,and/or honoraria (less than $10,000 each) from Abbott, Novar-tis, Leo, Schering-Plough, Janssen-Cilag, Wyeth, and Sanofi-Aventis.

Address correspondence to Dafna D. Gladman, MD,FRCPC, Toronto Western Hospital, 399 Bathurst Street, 1E-410B, Toronto, Ontario, Canada M5T 2S8. E-mail:dafna.gladman@utoronto.ca.

Submitted for publication April 7, 2008; accepted in re-vised form June 1, 2009.

1236 Chandran et al

rheumatologists and dermatologists had extensive experi-ence in the assessment of PsA and psoriasis, respectively.However, there were differences among the assessors intheir experience with individual measures. Among therheumatologists, 70% had �10 years of experience in as-sessing PsA and extensively used joint and dactylitiscounts, and the PGA-PsA. Although the remaining 30%had extensive experience with these measures, their expe-rience was �10 years. With regard to the assessment ofpsoriasis, 50% had extensive experience of �10 years withthe PASI, assessment of nail psoriasis, and PGA-Ps, butonly minimal experience with the BSA, NPF-PS, LS-PGA,and mNAPSI. Among the dermatologists, 50% had �10years of extensive experience with assessing psoriasis us-ing the PASI, BSA, PGA-Ps, and the assessment of nailpsoriasis, and the rest had �10 years of experience. Fiftypercent had �5 years of experience with the LS-PGA,NPF-PS, and mNAPSI. With regard to arthritis assessment,only 1 assessor had extensive experience of �10 yearswith joint counts, and none had experience in assessingdactylitis.

Design. The 20 patients and 19 assessors were dividedinto 2 groups. There were 10 patients in each group. Onegroup included 5 rheumatologists and 5 dermatologists,and the other included 5 rheumatologists and 4 dermatol-ogists. In each group, all of the patients were assessed bythe same assessors according to a Latin square design(which was modified slightly because there was an imbal-ance in the number of subjects and assessors) (30), whichfacilitates an analysis of components of variation due topatient, observer, and error, while controlling for the orderof assessment.

Clinical assessments. The following clinical measure-ments were performed by each assessor on each patient,after a 2.5-hour educational session wherein all of themeasurements were reviewed. The assessors completed allof the assessments within 30 minutes.

The BSA was visually determined using the rule that thepalm of the patient represents 1% of his/her total bodysurface and the rule of nines (14).

The assessors scored erythema, infiltration, scaling, andarea involved for the head, trunk, upper extremity, andlower extremity, and the total PASI score was calculated(8).

The activity of psoriasis was scored on an 11-point (0–10) numerical rating scale, with 0 representing inactivedisease and 10 representing the most active disease. Thismodification of the PGA-Ps was done because the rheuma-tologists were more comfortable with using this type ofscale as opposed to the visual analog scale or a categoricalLikert scale.

The assessors first categorized the BSA involved on ascale from 0 to 6 (category 0 � 0%, 1 � 1–3%, 2 � 4–9%,3 � 10–20%, 4 � 21–29%, 5 � 30–50%, and 6 � 51–100%) (10). They then rated the average plaque qualityacross all of the involved areas from 0 (none) to 3 (marked)for each of the descriptors thickness, erythema, and scale.

Written definitions for each level of severity were pro-vided. The final LS-PGA score was later calculated usingthe lattice algorithm.

The NPF-PS includes an assessment of patients andphysicians (5). In this study, only the reliability of thephysician’s assessment was studied. Because this was asingle time point study, the BSA was assessed as thatcurrently involved by psoriasis, not as a percentage rela-tive to baseline. The first assessor seeing a patient chose 2target lesions, and all subsequent assessors evaluated thesame 2 target lesions for induration using the NPF-PSreference card (none to severe on a 6-point scale [0–5]).The assessors rated the BSA from 0 to 5 (0 � none, 1 �1–20%, 2 � 21–40%, 3 � 41–60%, 4 � 61–80%, and 5 �81–100%) and globally assessed the average induration,erythema, and scaling over all of the lesions on a scalefrom 0 to 5 (where 0 � none and 5 � severe). Writtendefinitions were provided. The final (modified, since pa-tient-reported items were excluded and current BSA wasused) NPF-PS score was then calculated.

Individual fingernails of all of the patients were assessedfor the following features: onycholysis and oil drop dys-chromia (scored from 0 to 3, where 0 � none and 3 ��30% of the nail), pitting (range 0–3, where 0 � 0 pits and3 � �50 pits), nail plate crumbling (range 0–3, where 0 �none and 3 � �50% of the nail), leukonychia (present/absent), splinter hemorrhage (present/absent), nail bed hy-perkeratosis (present/absent), and red spots in the lunula(present/absent). The mNAPSI score was then calculated(19).

A simple count (0–10) of the number of fingernails withnail changes (NN) was performed.

For the peripheral joint assessment, 68 joints (tempero-mandibular, sternoclavicular, acromioclavicular, glenohu-meral, elbows, wrists, metacarpophalangeal, proximalinterphalangeal, distal interphalangeal, hips, knees, tibio-talar and midtarsal joints of the ankles, metatarsophalan-geal, and proximal interphalangeal joints of the toes) wereassessed for tenderness and 66 joints (all of the aboveexcept for hips) were assessed for swelling.

The total number of fingers and/or toes with dactylitiswas also recorded.

For the PGA-PsA, each assessor was asked to assess theactivity of the PsA of the patient on an 11-point numericalrating scale (range 0–10, where 0 � inactive disease and10 � the most active disease).

Statistical analysis. Variance components analyseswere conducted for each measurement based on analysisof variance models with a random observer, random pa-tient, and both, excluding and including fixed-order ef-fects to account for temporal trends in the responses whileassessing the reliability. Estimates of intraclass correlationcoefficients (ICCs) and associated 95% confidence inter-vals (95% CIs) were obtained (31). Sackett et al (32) suggestthat values of kappa in the interval 0.80–1.00 representexcellent agreement beyond chance, in 0.60–0.80 repre-sent substantial agreement, in 0.40–0.60 represent moder-

Reproducibility of Assessment of the Skin and Joints 1237

ate agreement, in 0.20–0.40 represent fair agreement, andin 0.0–0.20 represent poor agreement beyond chance. Forthe purpose of interpreting our results, we adopted thissame classification for the ICC. The overall agreement foreach measure between all of the assessors was first calcu-lated. Subsequently, the agreement for each measurementamong rheumatologists and dermatologists was calculatedand compared. All statistical analyses were performed us-ing SAS statistical software (SAS Institute, Cary, NC).

RESULTS

Table 1 shows the disease characteristics of the 20 patientswith PsA who participated in the study, as well as themedian scores for the measures studied. Table 2 shows theresults of the ICC values of the assessments performed byall of the assessors. The mNAPSI showed excellent agree-ment, the PASI, NN, and TJC showed substantial agree-ment, the BSA, LS-PGA, NPF-PS, and PGA-Ps showedmoderate agreement, and the SJC, number of fingers and/ortoes with dactylitis, and PGA-PsA showed fair agreementbeyond chance. Among dermatologists, there was substan-tial agreement on the BSA, PASI, LS-PGA, NPF-PS,mNAPSI, NN, PGA-Ps, and TJC, moderate agreement onthe PGA-PsA, and fair agreement on the SJC, but no agree-ment on the number of fingers and/or toes with dactylitis.Among rheumatologists, there was excellent agreement onthe mNAPSI and TJC, substantial agreement on the PASI,NN, and number of fingers and/or toes with dactylitis,moderate agreement on the LS-PGA, NPF-PS, PGA-Ps, andSJC, and fair agreement on the BSA and PGA-PsA. Whenthe 2 groups of assessors were compared, the 95% CIs ofthe ICCs for all measures except dactylitis overlapped,indicating that the only measure with a significant differ-ence in the ICC between dermatologists and rheumatolo-gists was dactylitis.

Table 3 shows information on the sources of variation inthe patients seen by rheumatologists and dermatologists,and shows that the majority of the variance was contrib-uted by patients. However, for the 3 measures that provedto be the least reliable among dermatologists (dactylitis,SJC, and PGA-PsA), there was less variation provided bythe patients and a proportionally higher variance due tothe assessors (Table 3). Similarly, for those measures thatproved to be less reliable among rheumatologists (BSA andPGA-PsA), there was less variation provided by the pa-tients and proportionally higher variance provided by theassessors. Order effect was noted for the assessment of

Table 1. Disease characteristics of the study patients(n � 20)*

Characteristics Value

Men, no. 11Age, mean years 51Duration of psoriasis, mean years 19Duration of PsA, mean years 11BSA (range 0–100)† 1.58 (1.0–2.4)PASI score (range 0–72)† 4.16 (3.33–6.61)LS-PGA score (range 1–8)† 2.25 (1.5–3.65)NPF-PS (range 0–20)† 1.88 (1.36–2.45)mNAPSI score (range 0–130)† 0.81 (0.24–1.46)NN (range 0–10)† 6.1 (2.44–7.83)PGA-Ps score (range 0–10)† 2.23 (1.75–2.65)Tender joint count (range 0–68)‡ 9.7 (2.55–23.2)Swollen joint count (range 0–66)‡ 2 (1.2–4.85)Number of fingers and/or toes with

dactylitis (range 0–20)‡0.2 (0–0.7)

PGA-PsA score (range 0–10)‡ 2.8 (1.4–3.6)

* Values are the median (interquartile range) unless otherwise in-dicated. PsA � psoriatic arthritis; BSA � body surface area involvedby psoriasis; PASI � Psoriasis Area and Severity Index; LS-PGA �Lattice System Physician’s Global Assessment; NPF-PS � NationalPsoriasis Foundation Psoriasis Score; mNAPSI � modified NailPsoriasis Severity Index; NN � number of nails with nail changes;PGA-Ps � physician’s global assessment of psoriasis activity; PGA-PsA � physician’s global assessment of PsA activity.† Based on average scores from all dermatologists.‡ Based on average scores from all rheumatologists.

Table 2. Reliability of using ICCs and 95% CIs for each assessment*

Measure Overall† Dermatologists Rheumatologists

Skin assessmentsBSA 0.47 (0.31, 0.68) 0.65 (0.47, 0.82) 0.30 (0.11, 0.54)PASI score 0.66 (0.51, 0.82) 0.74 (0.58, 0.87) 0.70 (0.53, 0.85)LS-PGA 0.58 (0.41, 0.76) 0.73 (0.56, 0.86) 0.42 (0.23, 0.65)NPF-PS 0.58 (0.42, 0.76) 0.66 (0.47, 0.82) 0.52 (0.32, 0.73)mNAPSI 0.80 (0.69, 0.90) 0.78 (0.63, 0.89) 0.85 (0.74, 0.93)NN 0.73 (0.59, 0.86) 0.77 (0.61, 0.89) 0.66 (0.47, 0.82)PGA-Ps 0.54 (0.37, 0.73) 0.66 (0.48, 0.82) 0.49 (0.29, 0.70)

Arthritis assessmentsTender joint count 0.78 (0.65, 0.89) 0.73 (0.56, 0.86) 0.81 (0.68, 0.91)Swollen joint count 0.24 (0.12, 0.45) 0.31 (0.12, 0.57) 0.42 (0.23, 0.65)Dactylitis‡ 0.29 (0.15, 0.51) 0.08 (�0.07, 0.32) 0.69 (0.52, 0.84)PGA-PsA 0.39 (0.23, 0.60) 0.50 (0.29, 0.72) 0.29 (0.11, 0.54)

* ICCs � intraclass correlation coefficients; 95% CIs � 95% confidence intervals; see Table 1 foradditional definitions.† Includes the combined ICCs (95% CIs) using data obtained from all assessors.‡ P � 0.0005 between dermatologists and rheumatologists.

1238 Chandran et al

dactylitis by the dermatologists and for the assessment ofPGA-PsA by the rheumatologists.

DISCUSSION

To our knowledge, this is the first study investigating thereliability of the assessment of the skin, nails, peripheralarthritis, and dactylitis by physicians (dermatologists andrheumatologists) who manage patients and perform clini-cal trials in psoriasis and PsA. Reliable assessment ofdisease is important in patient care, clinical trials, andlongitudinal observational studies, as well as proper phe-notyping of disease for genetic studies. The assessmenttools selected for this study were those currently used inrandomized controlled trials.

Overall, there was substantial to excellent agreementamong dermatologists and rheumatologists on the assess-ment of nails, PASI, and TJC, the most commonly usedmeasures in clinical trials (3). The design of this studyallowed us to efficiently estimate the variability contrib-uted by patients, assessors, and order of assessment. ThePsA patients included provided a spectrum of skin andjoint disease; the majority of the variance was contributedby patients. The random-effects model used for the vari-ance components analysis involved an assumption of nor-mally distributed random effects and error terms. Some ofthe measures studied here had somewhat skewed distri-butions, and some authors have based analyses on log-transformed data to address this. We elected to maintainthe measures on their original scale to ensure that the ICCestimates could be easily clinically interpreted. An alter-native approach would be to discretize all of the responsesand use weighted kappa statistics (33) for these analyses.We did not pursue this approach because of problematicfeatures of the kappa statistic that are being increasinglyrecognized (34).

Among dermatologists, as expected, there was substan-tial agreement on all of the measures used in the assess-ment of the skin and nails. There was also substantialagreement on the number of tender joints. However, therewas no agreement on the number of fingers and/or toes

with dactylitis. Among rheumatologists, as expected, therewas substantial to excellent agreement on the number oftender joints and the number of fingers and/or toes withdactylitis. However, there was only moderate agreementon the number of swollen joints and only fair agreement onthe PGA-PsA. There was substantial agreement on nailassessment. For the assessments of the severity of skinpsoriasis, only the PASI had substantial agreement, withthe LS-PGA, NPF-PS, and PGA-Ps having only moderateagreement.

The PASI has been the gold standard for the assessmentof the severity of psoriasis, and is widely accepted as anoutcome measure in clinical trials (5,9). However, thePASI has a number of drawbacks (5), and the LS-PGA (10)and NPF-PS (12) were developed to overcome some ofthese limitations. PGA provides a subjective overall eval-uation of disease severity (9). In the most recent studycomparing the PASI, LS-PGA, and PGA, the PASI per-formed the best, followed by the LS-PGA and PGA in termsof interrater reliability, although the 95% CI overlapped,indicating that there was no significant difference (11).These measures have not been compared with the NPF-PS.Moreover, to our knowledge, no study has compared theassessment of the skin performed by dermatologists withthat performed by rheumatologists. We show that in thehands of the dermatologists, the PASI, LS-PGA, NPF-PS,and PGA-Ps perform equally well and show substantialinterrater reliability with overlapping 95% CIs. The PASIhad the highest ICC. The interrater reliability among rheu-matologists, however, was lower than that obtained fordermatologists. The PASI performed the best comparedwith the LS-PGA, NPF-PS, and PGA-Ps. However, therewas no significant difference between the measures,among rheumatologists, and between rheumatologists anddermatologists, since the 95% CI overlapped. Overall,when all of the assessments were considered together, thePASI performed the best, with an ICC of 0.66. Therefore,although the PASI does have limitations, it has the highestinterrater reliability and is probably the most reliablemethod that can be used in clinical trials until more validmeasures are developed. The performance of the LS-PGA

Table 3. Variation in patients with PsA*

Measure

Evaluated by dermatologists,% of total variance due

Evaluated by rheumatologists,% of total variance due

Patient Assessor Order† Patient Assessor Order†

BSA 72.1 6.1 0.5 42.7 14.8 0.0PASI score 79.2 5.4 0.1 75.3 2.2 0.0LS-PGA 77.9 7.2 0.2 53.0 19.9 0.0NPF-PS 72.6 11.1 0.0 60.5 14.1 0.1mNAPSI 82.0 6.8 0.1 87.6 2.7 0.0NN 82.8 2.2 0.6 72.6 5.4 0.1PGA-Ps 73.0 6.1 0.0 57.9 18.8 0.1Tender joint count 78.1 3.2 0.1 84.2 2.4 0.0Swollen joint count 44.7 11.4 2.1 52.3 4.3 1.3Dactylitis 25.7 12.8 6.2 74.6 3.4 0.0PGA-PsA 60.3 9.6 0.2 41.6 27.2 2.3

* See Table 1 for abbreviations.† Order of examination (whether seen in the first half or the second half of the day).

Reproducibility of Assessment of the Skin and Joints 1239

and NPF-PS was good, especially in the hands of thedermatologists. The addition of the patients’ rating to theNPF-PS would not have made a difference because theaddition of a constant to all of the physician’s assessmentscores for NPF-PS would not affect the ICC.

Assessment of the BSA affected by psoriasis is crucial. Itinfluences composite scores such as the PASI, LS-PGA,and NPF-PS, and the PGA-Ps. However, visual assessmentof BSA has significant issues with interrater reliability(14,15). In this study, the overall reliability was only mod-erate (ICC 0.47), with the ICC among dermatologists being0.65 and that among rheumatologists being only 0.30. Therelatively poor interrater reliability of BSA has undoubt-edly affected the reliability of the composite scores. Fur-ther training and a more precise definition for visuallyassessing BSA (35) will help improve the reliability of thismeasure and those composite measures that incorporatethe assessment of BSA.

In this study, tools used in the assessment of nail pso-riasis performed well. Overall, the mNAPSI was found tohave excellent agreement among all of the assessors andamong rheumatologists, and substantial agreement amongdermatologists. The mNAPSI was previously shown tohave interrater reliability and construct validity (19). Sen-sitivity to change has to be demonstrated before themNAPSI is fully validated. The NN also showed substan-tial interrater reliability overall (ICC 0.73). Although the95% CI overlapped, the agreement was better among der-matologists (ICC 0.77) compared with rheumatologists(ICC 0.66). This indicates that the reliability of the instru-ments used in assessing nail disease can be further im-proved with training on various manifestations of psoriaticnail disease.

The reliability of peripheral joint assessment in PsA hasbeen demonstrated previously through observers in a sin-gle center (36), from multiple Canadian centers (20), and inan international study (21). The TJC was more reliablethan the SJC. In our study, we have demonstrated substan-tial overall agreement in the TJC (ICC 0.78), which issimilar to what has been shown among rheumatologists inthe previous studies, but only fair agreement on the SJC(ICC 0.24). As expected, there was excellent agreement onthe TJC among rheumatologists (ICC 0.81). Interestingly,there was also substantial agreement among dermatolo-gists (ICC 0.73), which was not significantly different fromthat of rheumatologists. However, assessment of the SJCwas less reliable, with an overall ICC of only 0.24, an ICCamong rheumatologists of only 0.42, and an ICC of 0.31among dermatologists. Previous studies have also demon-strated poor interrater reliability of the SJC in PsA (20,21).Major issues with the definition of joint swelling includedistinction between swelling due to active inflammation,swelling due to damage, and that due to chronic synovialthickening. There is an urgent need to standardize thedefinition of a swollen joint. Further training of assessorsis required to improve reliability.

Dactylitis is a marker of disease severity (22). Althoughthere are at least 3 instruments used in the assessment ofdactylitis (24–26), in this study we chose to assess thereliability of a simple count of the number of fingersand/or toes with dactylitis. The results show that although

there was substantial agreement among rheumatologists(ICC 0.69), there was no agreement among dermatologists(ICC 0.08; 95% CI �0.07, 0.32). In a Canadian study byrheumatologists where the assessment of dactylitis wasbased on the same method that was employed in ourstudy, the ICC was 0.56 (20). In the INSPIRE (INternationalSPondyloarthritis Interobserver Reliability Exercise) study(21), the assessment of dactylitis using the LDI (24)showed a better ICC of 0.70. These results indicate thatthere is agreement among rheumatologists on the assess-ment of dactylitis. The results show that there is very poorreliability of the assessment of dactylitis by dermatolo-gists, and therefore training on the definition and assess-ment of dactylitis needs to be performed. It should benoted that the LDI incorporates a check for the definitionof dactylitis (a 10% difference in circumference and atleast some tenderness on palpation). Therefore, the use ofthis instrument should increase the reliability for non-rheumatology specialists.

The PGA-PsA, although not considered to be a manda-tory domain in randomized controlled trials and longitu-dinal observational studies in PsA (7), is nevertheless animportant measure of disease activity and has been used inthe assessment of response using composite response in-dices such as the American College of Rheumatology cri-teria for 20% improvement in disease activity (37) and thePsoriatic Arthritis Response Criteria (26). However, in thisstudy, the PGA-PsA had only fair agreement among therheumatologists (ICC 0.29), better among the dermatolo-gists (ICC 0.51), and a fair overall agreement (ICC 0.39).This study shows that even among PsA experts, there isonly fair interrater reliability. Interestingly, the reliabilitywas better among dermatologists. Therefore, there againneeds to be further training in this domain, stressing thatthe PGA-PsA is the assessment of disease activity at thetime of examination of the patient and not overall diseaseseverity, which encompasses other factors such as damageand response to treatment.

In summary, our study involving assessors who are in-volved in the care of patients with PsA has demonstratedsubstantial to excellent agreement on the TJC, PASI, NN,and mNAPSI. Measures that involved a global assessmentperformed better in the hands of dermatologists whencompared with rheumatologists, probably reflecting differ-ences in the core training in the respective subspecialties.The only measure that showed a statistically significantdifference between the dermatologists and rheumatolo-gists was the assessment of dactylitis. SJCs performedpoorly even among rheumatologists. Further training inthe assessment of BSA will likely improve reliability of theassessment of BSA and other measures that include BSA,such as the PASI, LS-PGA, and NPF-PS. Further educationand training on the assessment of dactylitis and SJC isimperative because they are considered to be importantmarkers of disease activity and severity. There is also aneed for further training in PGAs for both psoriasis andPsA. It has previously been shown that training and stan-dardization of procedures even among experienced ob-servers can reduce the interobserver variability and reducethe sample size requirements in clinical trials (38,39).Recognizing these deficiencies, GRAPPA is developing ed-

1240 Chandran et al

ucational measures to better define various assessmenttools. There are plans to have dermatologists and rheuma-tologists participate in further training and assessmentexercises in the near future. The assessment of reliability isan important first step in evaluating the utility of clinicalmeasurements. Important next steps in the validation pro-cess include assessment of sensitivity to change (40) andfeasibility. We then plan to develop practical guidelinesregarding the value of these assessments.

AUTHOR CONTRIBUTIONS

All authors were involved in drafting the article or revising itcritically for important intellectual content, and all authors ap-proved the final version to be published. Dr. Gladman had fullaccess to all of the data in the study and takes responsibility forthe integrity of the data and the accuracy of the data analysis.Study conception and design. Chandran, Cook, Duffin, Helliwell,Kavanaugh, Kreuger, Langley, Mease, Rosen, Wong, Gladman.Acquisition of data. Chandran, Gottlieb, Cook, Duffin, Garg, Helli-well, Kavanaugh, Kreuger, Langley, McHugh, Mease, Rahman,Rosen, Salvarani, Thaci, Toloza, Wong, Zhou, Gladman.Analysis and interpretation of data. Chandran, Gottlieb, Cook,Garg, Kavanaugh, Lynde, Mease, Olivieri, Rahman, Rosen, Thaci,Wong, Zhou, Gladman.

ROLE OF THE STUDY SPONSOR

This study was supported by unrestricted educational grantsfrom Abbott Canada, Amgen, and Wyeth Canada. The companieshad no input into the study design, data collection, data analysis,and writing of the manuscript, nor was their approval sought inthe content of the submitted manuscript. The publication of thisarticle was not contingent on the approval of any of the compa-nies.

REFERENCES

1. Stern RS, Nijsten T, Feldman SR, Margolis DJ, Rolstad T.Psoriasis is common, carries a substantial burden even whennot extensive, and is associated with widespread treatmentdissatisfaction. J Investig Dermatol Symp Proc 2004;9:136–9.

2. Gladman DD, Antoni C, Mease P, Clegg DO, Nash P. Psoriaticarthritis: epidemiology, clinical features, course, and out-come. Ann Rheum Dis 2005;64 Suppl 2:ii14–7.

3. Gladman DD, Helliwell P, Mease PJ, Nash P, Ritchlin C, Tay-lor W. Assessment of patients with psoriatic arthritis: a reviewof currently available measures [review]. Arthritis Rheum2004;50:24–35.

4. Mease PJ, Antoni CE, Gladman DD, Taylor WJ. Psoriatic ar-thritis assessment tools in clinical trials. Ann Rheum Dis2005;64 Suppl 2:ii49–54.

5. Feldman SR, Krueger GG. Psoriasis assessment tools in clin-ical trials. Ann Rheum Dis 2005;64 Suppl 2:ii65–73.

6. Gladman DD, Mease PJ, Healy P, Helliwell PS, Fitzgerald O,Cauli A, et al. Outcome measures in psoriatic arthritis. J Rheu-matol 2007;34:1159–66.

7. Gladman DD, Mease PJ, Strand V, Healy P, Helliwell PS,Fitzgerald O, et al. Consensus on a core set of domains forpsoriatic arthritis. J Rheumatol 2007;34:1167–70.

8. Fredriksson T, Pettersson U. Severe psoriasis: oral therapywith a new retinoid. Dermatologica 1978;157:238–44.

9. Feldman SR. A quantitative definition of severe psoriasis foruse in clinical trials. J Dermatolog Treat 2004;15:27–9.

10. Langley RG, Ellis CN. Evaluating psoriasis with Psoriasis Areaand Severity Index, Psoriasis Global Assessment, and LatticeSystem Physician’s Global Assessment. J Am Acad Dermatol2004;51:563–9.

11. Berth-Jones J, Grotzinger K, Rainville C, Pham B, Huang J,Daly S, et al. A study examining inter- and intrarater reliabil-

ity of three scales for measuring severity of psoriasis: PsoriasisArea and Severity Index, Physician’s Global Assessment andLattice System Physician’s Global Assessment. Br J Dermatol2006;155:707–13.

12. Carlin CS, Callis KP, Krueger GG. Efficacy of acitretin andcommercial tanning bed therapy for psoriasis. Arch Dermatol2003;139:436–42.

13. Gottlieb AB, Chaudhari U, Baker DG, Perate M, Dooley LT.The National Psoriasis Foundation Psoriasis Score (NPF-PS)system versus the Psoriasis Area Severity Index (PASI) andPhysician’s Global Assessment (PGA): a comparison. J DrugsDermatol 2003;2:260–6.

14. Ramsay B, Lawrence CM. Measurement of involved surfacearea in patients with psoriasis. Br J Dermatol 1991;124:565–70.

15. Yune YM, Park SY, Oh HS, Kim DJ, Yoo DS, Kim IH, et al.Objective assessment of involved surface area in patients withpsoriasis. Skin Res Technol 2003;9:339–42.

16. Griffiths CE, Camp RD, Barker JN. Psoriasis. In: Burns DA,Breathnach SM, Cox NH, Griffiths CE, editors. Rook’s text-book of dermatology. 7th ed. Oxford: Blackwell Science;2004. p. 35.1–69.

17. Williamson L, Dalbeth N, Dockerty JL, Gee BC, Weatherall R,Wordsworth BP. Extended report: nail disease in psoriaticarthritis—clinically important, potentially treatable and oftenoverlooked. Rheumatology (Oxford) 2004;43:790–4.

18. Rich P, Scher RK. Nail Psoriasis Severity Index: a useful toolfor evaluation of nail psoriasis. J Am Acad Dermatol 2003;49:206–12.

19. Cassell SE, Bieber JD, Rich P, Tutuncu ZN, Lee SJ, KalunianKC, et al. The modified Nail Psoriasis Severity Index: valida-tion of an instrument to assess psoriatic nail involvement inpatients with psoriatic arthritis. J Rheumatol 2007;34:123–9.

20. Gladman DD, Cook RJ, Schentag C, Feletar M, Inman RI,Hitchon C, et al. The clinical assessment of patients withpsoriatic arthritis: results of a reliability study of the Spondy-loarthritis Research Consortium of Canada. J Rheumatol 2004;31:1126–31.

21. Gladman DD, Inman RD, Cook RJ, Maksymowych WP, BraunJ, Davis JC, et al. International spondyloarthritis interobserverreliability exercise: the INSPIRE study. II. Assessment of pe-ripheral joints, enthesitis, and dactylitis. J Rheumatol 2007;34:1740–5.

22. Brockbank JE, Stein M, Schentag CT, Gladman DD. Dactylitisin psoriatic arthritis: a marker for disease severity? AnnRheum Dis 2005;64:188–90.

23. Healy PJ, Groves C, Chandramohan M, Helliwell PS. MRIchanges in psoriatic dactylitis: extent of pathology, relation-ship to tenderness and correlation with clinical indices.Rheumatology (Oxford) 2008;47:92–5.

24. Helliwell PS, Firth J, Ibrahim GH, Melsom RD, Shah I,Turner DE. Development of an assessment tool for dactyli-tis in patients with psoriatic arthritis. J Rheumatol 2005;32:1745–50.

25. Antoni CE, Kavanaugh A, Kirkham B, Tutuncu Z, BurmesterGR, Schneider U, et al. Sustained benefits of infliximab ther-apy for dermatologic and articular manifestations of psoriaticarthritis: results from the Infliximab Multinational PsoriaticArthritis Controlled Trial (IMPACT) [published erratum ap-pears in Arthritis Rheum 2005;52:2951]. Arthritis Rheum2005;52:1227–36.

26. Clegg DO, Reda DJ, Mejias E, Cannon GW, Weisman MH,Taylor T, et al. Comparison of sulfasalazine and placebo inthe treatment of psoriatic arthritis: a Department of Veter-ans Affairs Cooperative Study. Arthritis Rheum 1996;39:2013–20.

27. Felson DT, Anderson JJ, Boers M, Bombardier C, Chernoff M,Fried B, et al. The American College of Rheumatologypreliminary core set of disease activity measures for rheu-matoid arthritis clinical trials. Arthritis Rheum 1993;36:729 – 40.

28. Cauli A, Gladman D, Mathieu A, Olivieri I, Ujfalussy I,Scarpa R, et al. Patient and physician perception of diseasein psoriatic arthritis (PsA): a multicentre GRAPPA and

Reproducibility of Assessment of the Skin and Joints 1241

OMERACT study [abstract]. Arthritis Rheum 2007;56Suppl:S264.

29. Taylor W, Gladman D, Helliwell P, Marchesoni A, Mease P,Mielants H, and the CASPAR Study Group. Classificationcriteria for psoriatic arthritis: development of new criteriafrom a large international study. Arthritis Rheum 2006;54:2665–73.

30. Montgomery DC. Design and analysis of experiments. 3rd ed.New York: John Wiley & Sons; 1991.

31. Burdick RK, Graybill FA. Confidence intervals on variancecomponents. New York: Marcel Dekker; 1992. p. 127.

32. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinicalepidemiology: a basic science for clinical medicine. 2nd ed.Toronto: Little, Brown; 1991.

33. Cook RJ. Kappa. In: Armitage P, Colton T, editors. The ency-clopedia of biostatistics. 1st ed. Hoboken (NJ): Wiley andSons; 1998. p. 2160–6.

34. Cook RJ. Kappa and its dependence on marginal rates. In:Armitage P, Colton T, editors. The encyclopedia of bio-statistics. 1st ed. Hoboken (NJ): Wiley and Sons; 1998.p. 2166–8.

35. Thomas CL, Finlay AY. The ‘handprint’ approximates to 1%

of the total body surface area whereas the ‘palm minus thefingers’ does not [letter]. Br J Dermatol 2007;157:1080–1.

36. Gladman DD, Farewell V, Buskila D, Goodman R, Hamilton L,Langevitz P, et al. Reliability of measurements of active anddamaged joints in psoriatic arthritis. J Rheumatol 1990;17:62–4.

37. Felson DT, Anderson JJ, Boers M, Bombardier C, Furst D,Goldsmith C, et al. American College of Rheumatology pre-liminary definition of improvement in rheumatoid arthritis.Arthritis Rheum 1995;38:727–35.

38. Klinkhoff AV, Bellamy N, Bombardier C, Carette S, ChalmersA, Esdaile JM, et al. An experiment in reducing interobservervariability of the examination for joint tenderness. J Rheuma-tol 1988;15:492–4.

39. Bellamy N, Anastassiades TP, Buchanan WW, Davis P, Lee P,McCain GA, et al. Rheumatoid arthritis antirheumatic drugtrials. I. Effects of standardization procedures on observerdependent outcome measures. J Rheumatol 1991;18:1893–900.

40. Husted J, Gladman DD, Cook RJ, Farewell VT. Responsivenessof health status instruments to changes in articular status andperceived health in patients with psoriatic arthritis. J Rheu-matol 1998;25:2146–55.

1242 Chandran et al