validation of the identification and intervention for

9
RESEARCH ARTICLE Open Access Validation of the Identification and Intervention for Dementia in Elderly Africans (IDEA) cognitive screen in Nigeria and Tanzania Stella-Maria Paddick 1,2 , William K Gray 1 , Luqman Ogunjimi 3 , Bingileki lwezuala 4 , Olaide Olakehinde 3 , Aloyce Kisoli 5 , John Kissima 5 , Godfrey Mbowe 6 , Sarah Mkenda 6 , Catherine L Dotchin 1,7 , Richard W Walker 1,8 , Declare Mushi 6 , Cecilia Collingwood 9 and Adesola Ogunniyi 3* Abstract Background: We have previously described the development of the Identification and Intervention for Dementia in Elderly Africans (IDEA) cognitive screen for use in populations with low levels of formal education. The IDEA cognitive screen was developed and field-tested in an elderly, community-based population in rural Tanzania with a relatively high prevalence of cognitive impairment. The aim of this study was to validate the IDEA cognitive screen as an assessment of major cognitive impairment in hospital settings in Nigeria and Tanzania. Methods: In Nigeria, 121 consecutive elderly medical clinic outpatients reviewed at the University College Hospital, Ibadan were screened using the IDEA cognitive screen. In Tanzania, 97 consecutive inpatients admitted to Mawenzi Regional Hospital (MRH), Moshi, and 108 consecutive medical clinic outpatients attending the geriatric medicine clinic at MRH were screened. Inter-rater reliability was assessed in Tanzanian outpatients attending St Josephs Hospital in Moshi using three raters. A diagnosis of dementia or delirium (DSM-IV criteria) was classified as major cognitive impairment and was provided independently by a physician blinded to the results of the screening assessment. Results: The area under the receiver operating characteristic (AUROC) curve in Nigerian outpatients, Tanzanian outpatients and Tanzanian inpatients was 0.990, 0.919 and 0.917 respectively. Inter-rater reliability was good (intra-class correlation coefficient 0.742 to 0.791). In regression models, the cognitive screen did not appear to be educationally biased. Conclusions: The IDEA cognitive screen performed well in these populations and should prove useful in screening for dementia and delirium in other areas of sub-Saharan Africa. Keywords: Dementia, Delirium, Screening, Nigeria, Tanzania, Africa, Validation Background The prevalence of dementia, alongside other non- communicable diseases (NCDs), is increasing rapidly as populations age globally, with 135.5 million people expected to have dementia by 2050 [1]. The greatest increases are predicted in low- and middle-income countries (LMICs) with 71% of the global total of people with dementia residing in LMICs by 2050 [1]. In sub-Saharan Africa (SSA) the number of cases of de- mentia is expected to increase from 1.31 million in 2013 to 5.05 million by 2050 [1]. Identification of dementia and other major cognitive impairments in LMICs can be problematic, due to the lack of culturally appropriate validated screening tools. The vast major- ity of cognitive screening tools in common use world- wide have been developed and validated in high income countries (HIC) and usefulness in LMIC settings is greatly limited by cultural and educational differences. Illiteracy is highly prevalent in older adults in many LMICs, particularly in rural areas. Minimising educational bias in cognitive screening tools would im- prove their clinical utility. In SSA, perhaps more than in other LMIC settings, this problem is compounded * Correspondence: [email protected] 3 University College Hospital, University of Ibadan, Ibadan, Nigeria Full list of author information is available at the end of the article © 2015 Paddick et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http:// creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Paddick et al. BMC Geriatrics (2015) 15:53 DOI 10.1186/s12877-015-0040-1

Upload: others

Post on 18-Nov-2021

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Validation of the Identification and Intervention for

Paddick et al. BMC Geriatrics (2015) 15:53 DOI 10.1186/s12877-015-0040-1

RESEARCH ARTICLE Open Access

Validation of the Identification and Interventionfor Dementia in Elderly Africans (IDEA) cognitivescreen in Nigeria and TanzaniaStella-Maria Paddick1,2, William K Gray1, Luqman Ogunjimi3, Bingileki lwezuala4, Olaide Olakehinde3, Aloyce Kisoli5,John Kissima5, Godfrey Mbowe6, Sarah Mkenda6, Catherine L Dotchin1,7, Richard W Walker1,8, Declare Mushi6,Cecilia Collingwood9 and Adesola Ogunniyi3*

Abstract

Background: We have previously described the development of the Identification and Intervention for Dementiain Elderly Africans (IDEA) cognitive screen for use in populations with low levels of formal education. The IDEAcognitive screen was developed and field-tested in an elderly, community-based population in rural Tanzania witha relatively high prevalence of cognitive impairment. The aim of this study was to validate the IDEA cognitive screenas an assessment of major cognitive impairment in hospital settings in Nigeria and Tanzania.

Methods: In Nigeria, 121 consecutive elderly medical clinic outpatients reviewed at the University College Hospital,Ibadan were screened using the IDEA cognitive screen. In Tanzania, 97 consecutive inpatients admitted to MawenziRegional Hospital (MRH), Moshi, and 108 consecutive medical clinic outpatients attending the geriatric medicine clinicat MRH were screened. Inter-rater reliability was assessed in Tanzanian outpatients attending St Joseph’s Hospital inMoshi using three raters. A diagnosis of dementia or delirium (DSM-IV criteria) was classified as major cognitiveimpairment and was provided independently by a physician blinded to the results of the screening assessment.

Results: The area under the receiver operating characteristic (AUROC) curve in Nigerian outpatients, Tanzanianoutpatients and Tanzanian inpatients was 0.990, 0.919 and 0.917 respectively. Inter-rater reliability was good(intra-class correlation coefficient 0.742 to 0.791). In regression models, the cognitive screen did not appear tobe educationally biased.

Conclusions: The IDEA cognitive screen performed well in these populations and should prove useful inscreening for dementia and delirium in other areas of sub-Saharan Africa.

Keywords: Dementia, Delirium, Screening, Nigeria, Tanzania, Africa, Validation

BackgroundThe prevalence of dementia, alongside other non-communicable diseases (NCDs), is increasing rapidlyas populations age globally, with 135.5 million peopleexpected to have dementia by 2050 [1]. The greatestincreases are predicted in low- and middle-incomecountries (LMICs) with 71% of the global total ofpeople with dementia residing in LMICs by 2050 [1].In sub-Saharan Africa (SSA) the number of cases of de-mentia is expected to increase from 1.31 million in

* Correspondence: [email protected] College Hospital, University of Ibadan, Ibadan, NigeriaFull list of author information is available at the end of the article

© 2015 Paddick et al. This is an Open Access(http://creativecommons.org/licenses/by/4.0),provided the original work is properly creditedcreativecommons.org/publicdomain/zero/1.0/

2013 to 5.05 million by 2050 [1]. Identification ofdementia and other major cognitive impairments inLMICs can be problematic, due to the lack of culturallyappropriate validated screening tools. The vast major-ity of cognitive screening tools in common use world-wide have been developed and validated in highincome countries (HIC) and usefulness in LMICsettings is greatly limited by cultural and educationaldifferences. Illiteracy is highly prevalent in older adultsin many LMICs, particularly in rural areas. Minimisingeducational bias in cognitive screening tools would im-prove their clinical utility. In SSA, perhaps more thanin other LMIC settings, this problem is compounded

article distributed under the terms of the Creative Commons Attribution Licensewhich permits unrestricted use, distribution, and reproduction in any medium,. The Creative Commons Public Domain Dedication waiver (http://) applies to the data made available in this article, unless otherwise stated.

Page 2: Validation of the Identification and Intervention for

Paddick et al. BMC Geriatrics (2015) 15:53 Page 2 of 9

by a serious shortage of specialist clinicians includingneurologists, geriatricians and psychiatrists [2,3].There are 200 times fewer qualified mental healthworkers per 100,000 population in SSA compared toHICs [4]. One way to overcome this problem is to de-velop and validate cognitive screening tools suitablefor use by non-specialist healthcare workers and clini-cians. These tools must be brief, simple to use and haveexcellent predictive properties. This task-shifting ap-proach is recommended by the World HealthOrganization (WHO) for other mental and neurologicalconditions in low-resource settings [5]. We have re-cently described the development, internal validationand fieldwork testing of the Identification and Interven-tions for Dementia in Elderly Africans (IDEA) studycognitive screening tool [6]. This screening tool was de-veloped using data collected from 1198 older adultsscreened for dementia in rural Tanzania, a low-literacysetting. It was subsequently piloted in a follow up co-hort with relatively high levels of cognitive impairment.The aim of the current study was to externally validatethe IDEA cognitive screen for use in a cohort of olderadults in hospital settings in Nigeria and Tanzania.

MethodsThis current study took place as part of the larger IDEAstudy of dementia in SSA. In Nigeria, the study was ap-proved by the University of Ibadan and Oyo state Minis-try of Health research ethics committees. In Tanzania,the study was approved nationally by the National Insti-tute for Medical Research, and locally by KilimanjaroChristian Medical University College. Written informedconsent was obtained from each participant. We ob-tained a thumbprint for those that could not read orwrite after the purpose and implications of the studywere verbally explained. In cases where patients wereunable to give informed consent due to cognitive deficit,written assent was obtained from a close relative.

Participants and settingNigerian cohortOutpatient sample Participants were geriatric patients,aged 65 years and over, seen at the medical outpatientclinic of University College Hospital Ibadan (UCH)during May 2013. UCH is an 850-bed teaching hospitalin the city of Ibadan, Oyo state, western Nigeria. Thecity has a population of approximately 3 million people.Patients were included if they consented to participateand were 65 years or older.

Tanzanian cohortMawenzi Regional Hospital (MRH) in Moshi is agovernment hospital with approximately 200 beds andprovides care for around 300 outpatients per day. The

hospital serves an urban and rural population of around100,000 people. Those requiring more specialist servicesare referred to Kilimanjaro Christian Medical Centre, atertiary referral hospital in Moshi.

Outpatient sampleOutpatients were recruited from the geriatric medicineoutpatient clinic at MRH. The geriatric clinic offers afree-of-charge service for those able to demonstrate thatthey are aged 60 years or over, usually with a letter fromtheir village committee. All attendees at the clinic aged65 years or over were invited to take part. Screening wasconducted daily for a four-week period during Octoberand November 2013. Screening did not take place onpublic holidays and weekends, as the clinic was closed.Due to resource limitations, a stratified sample of outpa-tients were clinically assessed. A full clinical assessmentof all those scoring ≤ 8 on the IDEA screen was com-pleted. A score of ≤ 7 was considered the optimal cut-offfor detection of major cognitive impairment, but thehigher cut-off was chosen to try to ensure high sensitiv-ity. We aimed to clinically assess a random selection ofat least 40% of those who scored > 8 on the screen. Ran-domisation involved drawing lots.

Inpatient sampleAll admissions to the medical wards of MRH aged65 years and over, from 8th October to 20th December2013 were invited to take part in the study. The wardadmission records were consulted daily and a physicalcheck was made of all wards for new admissions. Patientswere excluded if they refused to participate, or if the asses-sing clinician felt they were too unwell to participate.

Inter-rater reliabilityThis was carried out in the outpatient clinic of StJoseph’s Catholic Mission hospital in Moshi, Tanzania. StJoseph’s was chosen for the assessment of inter-raterreliability to ensure that all patients were previouslyunknown to the raters, thus avoiding the possibility thattheir scores could be influenced by prior information.All outpatients aged 65 years and over were invited totake part. Screening was carried out by three trainedraters (AK, SM or JK), randomly coded A, B and C. Tominimise the confounding influence of a training effect,rater A and rater B were each randomly assigned to seehalf of the patients on the first assessment and the otherhalf on the second assessment. The third assessmentwas completed by rater C. For the third assessment, notall patients could be followed up due to having beendischarged from the clinic. A minimum gap of two dayswas left between consecutive assessments to minimisecarryover effects. Assessments were timed, where pos-sible, to coincide with existing outpatient appointments

Page 3: Validation of the Identification and Intervention for

Paddick et al. BMC Geriatrics (2015) 15:53 Page 3 of 9

in order to avoid additional unnecessary travel for par-ticipants, some of whom were frail.

AssessmentsAt both sites, basic demographic data (age, gender andhighest education level) were collected from each partici-pant. In both countries, birth registration is not universal,and many older people do not know their date of birth.Where age was not accurately known, a validated methodof estimation based on significant past events was used[7]. The method has been shown to have excellent con-cordance during validation work in other SSA populations(intra-class correlation coefficient (ICC) 0.87).

Cognitive screeningThe IDEA cognitive screen has six items derived fromexisting cognitive assessments used in LMIC [6]. Thescreen is shown in Figure 1. Items 1–4 are taken fromthe Community Screening instrument for Dementia(CSI-D) [8]. These involve being able to name a bridgefrom a description of its use, knowing the day of theweek, knowing the name of the village chief/ townmayor/ city governor and naming as many animals aspossible in one minute (score 2 for ≥ 8 animals, score 1for 4–7 animals, score 0 for 0–3 animals). Item 5 is takenfrom Consortium to Establish a Registry for Alzheimer’sDisease (CERAD) 10-word recall test [9], with recall of 10common words after 5 minutes delay (score 1 point foreach word up to a maximum of 5 points). The sixth itemis designed to measure praxis and involves a matchstickdesign test originally developed by Baiyewu et al. [10],with scores ranging from 0 (no matchsticks placed cor-rectly), to 3 (all four matchsticks placed correctly in theshape of a rake). The maximum possible score is 15 andthe minimum 0, with a higher score indicating bettercognitive function. The IDEA screen therefore includesdelayed recall, orientation, two measures of frontal lobefunction, verbal fluency and abstract reasoning, praxis andlong-term memory. An assessment of ability for newlearning is also possible from performance on the 10-word learning list. No items are included requiringreading, writing, drawing or calculation in order toreduce possible educational bias. The screen wasadministered in the local language (Yoruba in Nigeria,Swahili or Chagga in Tanzania) with the words fromthe 10-word list translated into the local equivalent.In Nigeria, the cognitive screen was administered by a

study nurse, prior to formal assessment for cognitive im-pairment by a doctor. The nurse was therefore blind tothe clinical diagnosis at the time of assessment. InTanzanian inpatients and outpatients, the cognitive screenwas completed by one of three assessors: an MSc qualifiednurse (AK), occupational therapist (GM) or assistantmedical officer (JK). As in Nigeria, to ensure blinding,

cognitive screening was conducted prior to clinical as-sessment by a doctor. After administration, all screeningtools were filed and not seen by the doctor completingcognitive and neurological assessment.

Clinical assessmentAt both sites, participants were assessed clinically for majorcognitive impairment. A focused neurological examination,further physical examination (where appropriate) andinformant history including usual level of functioningwere completed wherever possible. Clinical assessmentincluded bedside cognitive screening designed to cover allmajor cognitive domains, including orientation, registra-tion and delayed recall, attention and concentration, re-ceptive and expressive language, praxis and frontal lobefunction using Luria’s three-step hand position test. Aformal mental state examination with completion of thegeriatric depression scale (in order to exclude psychiatricdisorder as a cause of poor cognitive performance), neuro-logical examination and careful questioning of informantson history and instrumental activities of daily living(IADLs) appropriate to the setting in order to assessfunctional impairment were also completed. Due to thelack of validated appropriate cognitive screening toolsin our setting, greater weight was placed on the informanthistory and psychiatric and neurological examination thanthe outcome of bedside cognitive assessment when reach-ing a clinical diagnosis.In inpatients, the Confusion Assessment Method (CAM)

was completed where there was evidence of cognitiveimpairment. Care was taken to ensure that the assessingdoctor remained blinded to the outcome of screeningwhen completing the clinical assessment.

Diagnosis of major cognitive impairmentAfter interview and assessment, a diagnosis of majorcognitive impairment was provided as appropriate bythe study doctor at each site (AO and LO in Nigeria andS-MP in Tanzania). Diagnoses of dementia, delirium andother significant mental illness, where present, werebased on DSM-IV criteria [11]. Informant histories wereextremely useful in attempting to differentiate betweendementia and delirium and these were sought whereverpossible, by telephone if necessary. In cases of diagnosticdifficulty, cases were discussed with a specialist in oldage psychiatry and a consensus on the most likelyclinical diagnosis reached. Anyone with a diagnosis ofdementia or delirium was identified as having majorcognitive impairment.

Sample sizeFor multivariable analysis, a sample size was chosen thatwould avoid over-fitting the model. Although estimatesvary, a minimum of seven cases per predictor was deemed

Page 4: Validation of the Identification and Intervention for

Figure 1 The IDEA cognitive screen in English.

Paddick et al. BMC Geriatrics (2015) 15:53 Page 4 of 9

Page 5: Validation of the Identification and Intervention for

Paddick et al. BMC Geriatrics (2015) 15:53 Page 5 of 9

acceptable. Any model was thought unlikely to containmore than eight predictor variables, and so a minimumsample size for each cohort of 56 was calculated.

Statistical methodsStatistical analysis was conducted using IBM SPSS statis-tics version 21 (IBM corporation, Armonk, NY, USA).All data (including age) were not normally distributedand so non-parametric tests (Mann–Whitney U test andchi-squared test) were used. For data analysis, educationwas dichotomised into some education (attended school)and no education (never attended school). Sensitivity,specificity and likelihood ratio (LR) were calculated.Positive predictive value (PPV) was calculated forNigerian outpatients and Tanzanian inpatients, but not forTanzanian outpatients. Since not all screened Tanzanianoutpatients were clinically assessed, prevalence, and there-fore an accurate PPV, could not be estimated. The areaunder the receiver operating characteristic (AUROC)curve was used as an overall measure of the performanceof the IDEA cognitive screen. Cronbach’s α was calculatedto assess the consistency of the screen.We used regression modeling to investigate whether

the IDEA cognitive screen was educationally biased.Major cognitive impairment becomes more commonwith increasing age and is thought to be more commonin women than men and more common in those withno formal education [12,13]. However, these three vari-ables are also confounded with each other, with womentending to be overrepresented in older age groups and,in many areas of SSA, less likely to have attended schoolthan men. To assess the independent influence of age,gender and education on screening performance, univari-ate and multivariable logistic regression models were de-veloped with screening tool score (dichotomised into ≤ 7and > 7) as the dependent (outcome) variable. Age, gender,education and the presence of major cognitive impairmentwere forced into a multivariable model as independent(predictor) variables. Univariate models were initially in-vestigated within each of the three cohorts separately.Given the similarity in the results of the univariateanalysis, and to increase statistical power, multivariablemodels were constructed using the combined data fromall three cohorts. Education was dichotomised as no for-mal education or some formal education and age was splitinto five-year age bands. Inter-rater reliability was assessedusing the ICC and by comparing the level of agreement interms of clinical decision-making. The significance levelwas set at 5% and two-tailed tests were used throughout.

ResultsIn Tanzania, 97 inpatients were seen, of whom 33 (34.0%)had major cognitive impairment (20 dementia, 13 delir-ium). Of 108 outpatients seen in Tanzania, 16 (14.8%)

scored ≤ 8 and all were clinically assessed. Of theremaining 92 who scored > 8, 43 (46.7%) were randomlyselected for clinical assessment, giving a Tanzanian out-patient cohort of 59, of whom 13 (22.0%) had majorcognitive impairment. All 13 had dementia, though oneperson with dementia was also thought to have deliriumat the time of assessment and was referred for furtherinvestigations. In Nigeria, data were available for 121outpatients, of whom 12 (9.9%) had major cognitive im-pairment (all dementia).Thus, 277 were included in this validation study across

all three settings. The median time taken to completethe screen was 10 minutes (inter quartile range: 8 to12 minutes).

Demographic dataAge, gender and education level data for those with andwithout major cognitive impairment are presented inTable 1. In Nigeria, those with major cognitive im-pairment had significantly higher levels of educationthan those without major cognitive impairment andin Tanzanian outpatients, those with major cognitiveimpairment were significantly older than those with-out major cognitive impairment.

Performance of the IDEA cognitive screenCronbach’s α for the IDEA cognitive screen was 0.807 inTanzanian inpatients, 0.738 in Tanzanian outpatientsand 0.741 in Nigerian outpatients, suggesting it to havean acceptable degree of internal consistency in all threesettings.AUROC curves for each cohort are presented in

Figure 2. Across all three settings no one with majorcognitive impairment scored more than 10, and onlyeight scored greater than the suggested cut off of ≤ 7(three Tanzanian outpatients and one Tanzanian in-patient scored 8, one Tanzanian outpatient scored 9 andone Tanzanian outpatient and two Tanzanian inpatientsscored 10). Of 15 people without major cognitive impair-ment who scored ≤ 7, three had mild cognitive impair-ment (MCI) and five were aphasic or unable to performwell due to physical or mental illness. Sensitivity, specifi-city, PPV, LR and AUROC curve data, are shown inTable 2. The AUROC curve was above 0.9 in all settings.

The influence of age, gender and education on IDEAcognitive screen performanceUnivariate logistic regression models investigating theinfluence of age, gender, education and the presence ofmajor cognitive impairment on screening performance(outcome variable) are summarised in Table 3. Majorcognitive impairment was associated with a cognitivescreening score ≤ 7 in all three cohorts, with age as anadditional correlate in Tanzanian inpatients. In multivariable

Page 6: Validation of the Identification and Intervention for

Table 1 Validation of the Identification and Intervention for Dementia in Elderly Africans (IDEA) cognitive screen in2013: Demographic data

Major cognitive impairment No major cognitive impairment Significance of difference

Outpatients Nigeria

Number of patients 12 109

Median age (IQR) 71 (65.3 to 77.5) 70 (67 to 75.5) U = 619.0, z = −0.305, p = 0.761

Number of females 8 (66.7%) 49 (45.0%) χ2 = 2.045, p = 0.153

Level of education* None: 0 None: 36 (33.0%) χ2 = 5.619, p = 0.018

Some: 11 (91.6%) Some: 67 (61.5%)

Not known: 1 (8.3%) Not known: 6 (5.5%)

Outpatients Tanzania

Number of patients 13 46

Median age (IQR) 79.5 (73.3 to 89.8) 72 (67.3 to 78.8) U = 162.5, z = −2.030, p = 0.042

Number of females 7 (53.8%) 21 (45.7%) χ2 = 2.045, p = 0.153

Level of education None: 5 (38.5%) None: 11 (23.9%) χ2 = 0.273, p = 0.601

Some: 8 (61.5%) Some: 35 (76.1%)

Inpatients Tanzania

Number of patients 33 64

Median age (IQR) 78 (72.5 to 90) 75.5 (70.3 to 81) U = 846.0, z = −1.601, p = 0.109

Number of females 14 (42.4%) 37 (57.8%) χ2 = 2.068, p = 0.150

Level of education None: 15 (45.5%) None: 22 (34.4%) χ2 = 1.407, p = 0.236

Some: 17 (51.5%) Some: 42 (65.6%)

Not known 1 (3.0%)

IQR = interquartile range.U = the test value of the Mann–Whitney U test.* For data analysis education was dichotomised into some education (attended school for at least a year) and no education (never attended school).

Paddick et al. BMC Geriatrics (2015) 15:53 Page 6 of 9

analysis, with data for all three cohorts combined (n = 277),female gender, greater age and the presence of major cog-nitive impairment were independent predictors of lowscreening score, but education was not, see Table 4. Evenafter removing gender and age from the model, educationlevel remained a non-significant predictor.

Figure 2 Validation of the Identification and Intervention for Dementia ineach cohort.

Inter-rater reliabilityFor inter-rater reliability assessment, 30 patients wereseen by raters A and B and 19 by rater C. The mediantime from the first to the second assessment was 3 days(IQR 2 to 4 days) and the median time from the secondto the third assessment was 5 days (IQR 4 to 8 days).

Elderly Africans (IDEA) cognitive screen in 2013: ROC curves for

Page 7: Validation of the Identification and Intervention for

Table 2 Validation of the Identification and Intervention for Dementia in Elderly Africans (IDEA) cognitive screen in2013: Sensitivity, specificity, LR and PPV

Nigerian outpatients Tanzanian outpatients Tanzanian inpatients

AUROC curve 0.990 0.919 0.917

Cut-off of ≤ 7 100% sensitivity 61.5% sensitivity 90.9% sensitivity

96.3% specificity 93.5% specificity 87.5% specificity

27.0 LR 9.5 LR 7.3 LR

75.0% PPV - 78.9% PPV

Cut-off of ≤ 8 100% sensitivity 84.6% sensitivity 93.9% sensitivity

91.7% specificity 89.1% specificity 81.3% specificity

12.0 LR 7.8 LR 5.0 LR

57.1% PPV - 72.1% PPV

AUROC = area under the receiver operating characteristic.PPV = positive predictive value.LR = likelihood ratio.

Paddick et al. BMC Geriatrics (2015) 15:53 Page 7 of 9

The level of agreement between the raters was good.Comparing raters A and B, the ICC was 0.791. For the19 patients seen by rater C, the ICC was 0.787 comparedto rater A and 0.742 compared to rater B. The differ-ences in scores between raters were generally small.Comparing the first two raters, 20 assessments (66.7%)were within one point of each other and 27 (90.0%)within two points of each other. Using ≤ 7 as a cut-off,raters A and B agreed, and would have made the sameclinical decision for 28 (93.3%) cases. Likewise, raters Aand B agreed on 18 (94.7%) cases, with 13 (68.4%) scoreswithin one point of each other and 18 (94.7%) withintwo points. Finally, raters A and C agreed on 17 (89.5%)cases, with 15 (78.9%) within one point of each otherand 17 (89.5%) within two points.

DiscussionThe IDEA cognitive screen performed well in all threesettings, with good internal consistency and inter-raterreliability. The AUROC curve was generally higherthan seen during internal validation and fieldworktesting in Tanzania [6]. The screen appeared to beacceptable and culturally appropriate and no onerefused assessment.The sensitivity in Tanzanian outpatients was relatively

low, although lower sensitivity in outpatients was ex-pected in this setting. In rural Tanzania, people whohave dementia, and who are able to attend outpatientclinics, are likely to be in the early stages of disease.They may therefore be expected to perform relativelywell on brief cognitive screening, with the presence ofdementia only becoming apparent on more detailed as-sessment. It is not clear why the screen performed betterin Nigerian outpatients than in Tanzanian outpatients.The fact that UCH in Ibadan is a tertiary referral hos-pital may have played a part, with a broader spread ofpatients including those with more severe problems,

who may be easier to assess cognitively. Further valid-ation work in other settings in Nigeria is merited.The IDEA cognitive screen performed well in com-

parison with other major cognitive impairment screen-ing instruments developed for use in populations withlow levels of formal education [14,15]. Touré et al. [14]developed the ‘Test of Senegal’ and obtained an AUROCcurve of 0.967 on comparison with the DSM-IV-R cri-teria when blind assessment of 58 cases and 58 controlswas carried out. However, the test has 39 questions intotal and is therefore too lengthy for use in busy non-specialist hospital settings.The performance of the IDEA cognitive screen also

compares well to tests of cognitive performance vali-dated in HICs [16,17]. The six-item screener comprisesthree orientation questions and a three-word delayedrecall test. It was developed for use in emergency depart-ments and has a sensitivity of 63%, a specificity of 81%and an AUROC curve of 0.77 [18]. The mini-cog com-prises a clock-drawing test and a three-word delayedrecall test; it has a sensitivity of 75% and specificity of 85%[19]. The general practitioner assessment of cognition(GPCOG) combines the clock-drawing test with itemsassessing recall and orientation; its sensitivity was 85%and specificity 86% [20]. A review of other brief screeninginstruments was carried out in 2007 [21].UK and US good practice guidelines recommend cog-

nitive assessment of older adults in higher-prevalencesettings including primary care, and routinely in hospitalinpatient and outpatient populations [22]. The existingevidence base strongly suggests that identification ofcognitive impairment can improve outcomes and reducemorbidity and mortality through prevention of delirium[23]. Cognitive screening should form a core part ofassessment for older hospitalised adults. Surprisingly,even in HICs few of the recommended cognitive screen-ing tools have been validated in general hospital settings

Page 8: Validation of the Identification and Intervention for

Table 3 Validation of the Identification and Interventionfor Dementia in Elderly Africans (IDEA) cognitive screenin 2013: Univariate logistic regression models withdichotomised cognitive screen as the dependent(outcome) variable

Odds ratio (95% CI)

Nigerian outpatients

Major cognitive impairment present* -

Female 2.82 (0.92 to 8.69)

No formal education 3.11 (0.67 to 14.58)

Age

65-69 years 1

70-74 years 1.03 (0.27 to 3.98)

75-79 years 1.88 (0.47 to 7.51)

80-84 years 0.68 (0.07 to 6.26)

85 years and over 1.88 (0.18 to 19.68)

Tanzanian outpatients

Major cognitive impairment present 22.93 (4.55 to 115.67)

Female 2.25 (0.58 to 8.72)

No formal education 2.80 (0.72 to 10.97)

Age

65-69 years 1

70-74 years 6.75 (0.61 to 75.27)

75-79 years 4.00 (0.32 to 50.23)

80-84 years 6.00 (0.46 to 78.56)

85 years and over 7.71 (0.68 to 87.25)

Tanzanian inpatients

Major cognitive impairment present 70.00 (17.28 to 283.59)

Female 1.00 (0.44 to 2.27)

No formal education 1.99 (0.86 to 4.64)

Age

65-69 years 1

70-74 years 4.57 (0.83 to 25.21)

75-79 years 4.31 (0.76 to 24.38)

80-84 years 7.00 (1.17 to 41.76)

85 years and over 14.00 (2.54 to 77.21)

* An odds ratio cannot be calculated due to zero values. Only four subjects,from the 121 in the cohort, were misclassified, all identified as positive onscreen, but negative on clinical assessment.

Table 4 Validation of the Identification and Interventionfor Dementia in Elderly Africans (IDEA) cognitive screenin 2013: Multivariable logistic regression model withdichotomised cognitive screen as the dependent(outcome) variable

Odds ratio (95% CI) Significance (p)

Major cognitive impairmentpresent

108.82 (36.31 to 326.14) <0.001

Female 3.32 (1.20 to 9.19) 0.021

No formal education 1.07 (0.40 to 2.88) 0.895

Age

65-69 years 1

70-74 years 3.26 (0.83 to 12.74) 0.090

75-79 years 3.52 (0.87 to 14.31) 0.079

80-84 years 5.39 (1.05 to 27.68) 0.044

85 years and over 6.80 (1.58 to 29.21) 0.010

Hosmer and Lemeshow goodness of fit test, χ2 (7) = 5.60, p = 0.587.Nagelkerke R2 = 0.66.

Paddick et al. BMC Geriatrics (2015) 15:53 Page 8 of 9

[24]. Validation of appropriate cognitive screening methodsfor hospitalised older adults is therefore needed globally,not only in SSA.

LimitationsThe main limitation of our study is the relatively smallnumber of people in each cohort who had major cogni-tive impairment. However, the overall number of peoplewith major cognitive impairment (n = 58) was relatively

large and results were similar across all settings, allowingdata to be combined for multivariable analysis. Any at-tempt to increase the number of major cognitive impair-ment cases by assessing only people previously known tohave dementia and a group of controls would have re-duced the generalisability of our results and may haveresulted in substantial bias.In this hospital-based study, we did not attempt to

distinguish patients with delirium from those with de-mentia. It is not expected that a short screen will be ableto distinguish such conditions. The value of carrying outscreening is to alert the clinician to cognitive impair-ment meaning that delirium can be promptly recognisedand treated, and possible dementia considered in hos-pital discharge planning. Without an informant historyand follow-up it is difficult to be certain that dementia ispresent in hospital patients. Despite multiple attempts, itwas not always possible to obtain a history from a closerelative. Those patients who had a carer tended to beyounger and more independent, and were probably lesslikely to have cognitive impairment. Occasionally carerswere distant relatives who were less able to give a de-tailed history. In the Tanzanian sample, patients whowere seriously unwell, and were felt to need admissionto the tertiary referral hospital, were transferred, and thisis again likely to have led to an underestimate of cases ofdelirium. In the outpatient settings, resource limitationsmeant that it was not possible to see all screened pa-tients. However, almost half of those who screened nega-tively were randomly selected for clinical assessment andany bias is likely to be small. Finally, few people had abirth certificate or had had their birth registered and soa validated method of age estimation was used. Themethod has been shown to have excellent concordance

Page 9: Validation of the Identification and Intervention for

Paddick et al. BMC Geriatrics (2015) 15:53 Page 9 of 9

during validation work in other populations in SSA andwe feel that, at a cohort level, any bias will be small.

ConclusionsThe IDEA cognitive screen was administered by non-specialist healthcare workers and performed well in hos-pital settings in Nigeria and Tanzania. Further testing inother regions of SSA, and in primary care, is an importantnext step.

Competing interestsThe authors declare that they have no competing interests.

Authors’ contributionsS-MP conducted the literature search, collected data in Tanzania, analysedthe data, interpreted the results and wrote the first draft. WKG designed thestudy, conducted the literature search, analysed the data, interpreted theresults and wrote the first draft. LO collected data in Nigeria. BI collecteddata in Nigeria. OO collected data in Nigeria. AK collected data in Tanzania.GM collected data in Tanzania. JK collected data in Tanzania. SM collecteddata in Tanzania. CLD designed the study, analysed the data and interpretedthe results. RWW designed the study and interpreted the results. DMdesigned the study. CC collected data in Tanzania. AO designed the study,collected data in Nigeria and interpreted the results. All authors read,critically reviewed and approved the final manuscript.

AcknowledgementsWe wish to acknowledge the help of all health care workers, officials, carers,and family members who assisted in this study. This study was funded by aGrant from Grand Challenges Canada (Grant number: 0086–04). The sponsorsof this study had no role in designing the study; in the collection, analysis,and interpretation of data; in the writing of the report; or in the decision tosubmit the paper for publication.

Author details1Northumbria Healthcare NHS Foundation Trust, North Tyneside GeneralHospital, North Shields, UK. 2Institute of Neuroscience, Newcastle University,Newcastle upon Tyne, UK. 3University College Hospital, University of Ibadan,Ibadan, Nigeria. 4Mawenzi Regional Hospital, Moshi, Tanzania. 5Hai DistrictHospital, Boman’gombe, Kilimanjaro, Tanzania. 6Kilimanjaro Christian MedicalUniversity College, Moshi, Tanzania. 7Institute for Ageing, NewcastleUniversity, Newcastle upon Tyne, UK. 8Institute of Health and Society,Newcastle University, Newcastle upon Tyne, UK. 9The Medical School,Newcastle University, Newcastle upon Tyne, UK.

Received: 23 October 2014 Accepted: 25 March 2015

References1. Prince M, Guerchet M, Prina M. Policy brief for heads of government: the

global impact of dementia 2013–2050. London: Alzheimer DiseaseInternational; 2013.

2. Dotchin CL, Akinyemi RO, Gray WK, Walker RW. Geriatric medicine: servicesand training in Africa. Age Ageing. 2013;42:124–8.

3. Bower JH, Zenebe G. Neurologic services in the nations of Africa.Neurology. 2005;64:412–5.

4. Saxena S, Thornicroft G, Knapp M, Whiteford H. Resources for mental health:scarcity, inequity, and inefficiency. Lancet. 2007;370:878–89.

5. Eaton J, McCay L, Semrau M, Chatterjee S, Baingana F, Araya R, et al. Scaleup of services for mental health in low-income and middle-incomecountries. Lancet. 2011;378:1592–603.

6. Gray WK, Paddick SM, Kisoli A, Dotchin CL, Longdon AR, Chaote P, et al.Development and Validation of the Identification and Intervention forDementia in Elderly Africans (IDEA) Study Dementia Screening Instrument.J Geriatr Psychiatry Neurol. 2014;27:110–8.

7. Paraiso MN, Houinato D, Guerchet M, Agueh V, Nubukpo P, Preux PM, et al.Validation of the use of historical events to estimate the age of subjectsaged 65 years and over in Cotonou (Benin). Neuroepidemiology.2010;35:12–6.

8. Hall KS, Hendrie HC, Brittain HM, Norton JA, Rodgers DD, Prince CS, et al.The development of a dementia screening interview in 2 distinctlanguages. Int J Methods Psychiatr Res. 1993;3:1–28.

9. Welsh KA, Butters N, Mohs RC, Beekly D, Edland S, Fillenbaum G, et al. TheConsortium to Establish a Registry for Alzheimer’s Disease (CERAD). Part V. Anormative study of the neuropsychological battery. Neurology.1994;44:609–14.

10. Baiyewu O, Unverzagt FW, Lane KA, Gureje O, Ogunniyi A, Musick B, et al.The stick design test: a new measure of visuoconstructional ability. J IntNeuropsychol Soc. 2005;11:598–605.

11. American Psychiatric Association. Diagnostic and statistical manual ofmental disorders. 4th ed. Washington DC: American Medical Association;1994.

12. Paddick S-M, Longdon A, Gray WK, Dotchin C, Kisoli A, Chaote P, et al. Theassociation between educational level and dementia in rural Tanzania.Dementia & Neuropsychologia. 2014;8:117–25.

13. Paddick SM, Longdon AR, Kisoli A, Dotchin C, Gray WK, Dewhurst F, et al.Dementia prevalence estimates in sub-Saharan Africa: comparison of twodiagnostic criteria. Glob Health Action. 2013;6:19646.

14. Touré K, Coumé M, Ndiaye NND, Thiam MH, Zunzunegui MV, Bacher Y, et al.The test of senegal: a valid and reliable screening tool to assess fordementia in a senegalese elderly population. Afr J Neurol Sci. 2008;27:4–13.

15. Prince M, Acosta D, Ferri CP, Guerra M, Huang Y, Jacob KS, et al. A briefdementia screener suitable for use by non-specialists in resource poorsettings–the cross-cultural derivation and validation of the brief CommunityScreening Instrument for Dementia. Int J Geriatr Psychiatry. 2011;26:899–907.

16. Mitchell AJ. A meta-analysis of the accuracy of the mini-mental stateexamination in the detection of dementia and mild cognitive impairment.J Psychiatr Res. 2009;43:411–31.

17. Larner AJ, Mitchell AJ. A meta-analysis of the accuracy of the Addenbrooke’sCognitive Examination (ACE) and the Addenbrooke’s CognitiveExamination-Revised (ACE-R) in the detection of dementia. Int Psychogeriatr.2014;26:555–63.

18. Wilber ST, Carpenter CR, Hustey FM. The Six-Item Screener to detect cognitiveimpairment in older emergency department patients. Acad Emerg Med.2008;15:613–6.

19. Wilber ST, Lofgren SD, Mager TG, Blanda M, Gerson LW. An evaluation oftwo screening tools for cognitive impairment in older emergencydepartment patients. Acad Emerg Med. 2005;12:612–6.

20. Brodaty H, Pond D, Kemp NM, Luscombe G, Harding L, Berman K, et al. TheGPCOG: a new screening test for dementia designed for general practice.J Am Geriatr Soc. 2002;50:530–4.

21. Woodford HJ, George J. Cognitive assessment in the elderly: a review ofclinical methods. QJM. 2007;100:469–84.

22. National Institute for Health and Care Excellence (NICE). Delirium: Diagnosis,prevention and management. London: NICE; 2010.

23. Fong TG, Tulebaev SR, Inouye SK. Delirium in elderly adults: diagnosis,prevention and treatment. Nat Rev Neurol. 2009;5:210–20.

24. Jackson TA, Naqvi SH, Sheehan B. Screening for dementia in generalhospital inpatients: a systematic review and meta-analysis of availableinstruments. Age Ageing. 2013;42:689–95.

Submit your next manuscript to BioMed Centraland take full advantage of:

• Convenient online submission

• Thorough peer review

• No space constraints or color figure charges

• Immediate publication on acceptance

• Inclusion in PubMed, CAS, Scopus and Google Scholar

• Research which is freely available for redistribution

Submit your manuscript at www.biomedcentral.com/submit