bmj paediatrics open is committed to open peer review. as...

BMJ Paediatrics Open is committed to open peer review. As part of this commitment we make the peer review history of every article we publish publicly available. When an article is published we post the peer reviewers’ comments and the authors’ responses online. We also post the versions of the paper that were used during peer review. These are the versions that the peer review comments apply to. The versions of the paper that follow are the versions that were submitted during the peer review process. They are not the versions of record or the final published versions. They should not be cited or distributed as the published version of this manuscript. BMJ Paediatrics Open is an open access journal and the full, final, typeset and author-corrected version of record of the manuscript is available on our site with no access controls, subscription charges or pay-per-view fees (http://bmjpaedsopen.bmj.com). If you have any questions on BMJ Paediatrics Open’s open peer review process please email

[email protected]

on 11 May 2018 by guest. P

rotected by copyright.http://bm

jpaedsopen.bmj.com

/bm

jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O

ctober 2017. Dow

nloaded from

http://bmjpaedsopen.bmj.com/

mailto:[email protected]


Confidential: For Review O

nly

Validation of a classification system for treatment-related

mortality in children with cancer

Journal: BMJ Paediatrics Open

Manuscript ID bmjpo-2017-000082

Article Type: Original article

Date Submitted by the Author: 25-May-2017

Complete List of Authors: Hassan, Hadeel; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Rompola, Melpomeni; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology

Kinsey, Sally; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Glaser, Adam; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Phillips, Bob; University of York, Centre for Reviews and Dissemination; Bob Phillips

Keywords: Oncology, Haematology, Palliative Care

https://mc.manuscriptcentral.com/bmjpo

BMJ Paediatrics Open on 11 M

ay 2018 by guest. Protected by copyright.

http://bmjpaedsopen.bm

j.com/

bmjpo: first published as 10.1136/bm

jpo-2017-000082 on 30 October 2017. D

ownloaded from



nlyTitle:

Validation of a classification system for treatment-related mortality in children with cancer

Authors:

H. Hassan 1, 2, M. Rompola

1, 2, A.W. Glaser

1, 2, S. E. Kinsey

1, 2, R. S. Phillips

1, 3

Affiliations:

1 Department of Paediatric Haematology and Oncology, Leeds Teaching Hospital NHS Trust,

Leeds, UK

2 Leeds Institute of Cancer and Pathology,

University of Leeds, Leeds, UK

3 Centre for Reviews and Dissemination, University of York, York, UK

Full address for correspondence:

Dr Hadeel Hassan

Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, LS2 9JT, UK

Telephone: +44 (0)113 343 2596

Email address: [email protected]

Page 1 of 26


BMJ Paediatrics Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyAbstract

Background: Death not directly due to cancer has been termed “treatment-related mortality”

(TRM). Appreciating the differences between TRM and disease-related death is critical in

directing strategies to improve supportive care, interventions delivered or disease progression.

Recently, a global collaboration developed and validated a consensus-based classification tool

and attribution system.

Objectives: To evaluate the newly developed consensus-based definition of TRM and cause-

of-death attribution system outside the centre it was initially validated (Toronto, Canada).

Setting: The paediatric haematology and oncology department at Leeds Teaching Hospital in

Leeds, UK.

Participants: Two consultants and two clinical research associates (CRA).

Methods: Thirty medical records of the most recent deaths in children with cancer, two and

four weeks prior to death were anonymised and presented to the participants. Reviewers

independently classified deaths as “treatment related mortality” or “not treatment related”

according to the algorithm developed. When TRM occurred, reviewers applied the cause-of-

death attribution system. Inter-relater reliability was assessed using the Kappa statistic (k).

Main outcome: Inter-relater reliability between CRA and consultants.

Results: Reliability of the classification was deemed almost perfect between CRA and

consultants (k=0.86, 95% confidence interval 0.72-0.97). Ten deaths were classified as TRM;

of which infection was the most frequent cause identified. Reviewers disagreed on the primary

cause of death (e.g. respiratory vs infection) when applying the cause-of-death attribution

system in 6 cases. The study identified how the algorithm may not detect TRM in patients

receiving non-curative therapy.

Page 2 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyConclusions: The classification and cause of death attribution system could be implemented

in different health care settings. Adaptation of the classification tool in patients receiving non-

curative interventions and the cause of death attribution system should be considered.

What is known about the subject?

• Treatment-related mortality (TRM) is poorly defined in paediatric oncology and

haematology.

• A global collaboration developed and validated a consensus-based classification tool

and attribution system (in Toronto, Canada).

What this study adds?

• This study has further demonstrated criterion validity of the newly developed

classification tool in Leeds, UK, outside the centre it was initially tested.

• This study identified that the classification tool may not detect TRM in patients

receiving non-curative therapy, and proposes an alternative algorithm for these

patients.

• This study identifies the need for further research exploring the use of the attribution

system.

Page 3 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyBackground

Survival of children diagnosed with malignancies has improved with recent estimates

suggesting more than 80% of those diagnosed in high income countries will survive [1]. This

still means 20% of children will die - around 80,000 deaths worldwide in 2012 [1]. The cause

of death may be due directly to the disease, or complications or toxicities of interventions

delivered. The increasing success of cancer-directed interventions has contributed to a higher

relative proportion of deaths due to associated toxicities [2]. Death not directly due to the

cancer has been broadly termed “treatment-related mortality” (TRM). This includes death

from infection, bleeding, and organ dysfunction [3]. Incidence varies according to the

underlying disease, stage of treatment and socio-economic status [4]. Identifying the nature of

the mortality is critical in helping researchers and clinicians improve survival in particular

diagnostic groups by focusing on strategies to improve supportive care when death is

predominantly through TRM.

Despite the importance of TRM, varying definitions have been used. Systematic reviews [5]

[6] identified significant heterogeneity in TRM definitions used in randomised trials. This

inspired a global collaboration to develop a consensus-based classification tool for ascribing

death as TRM, and further specific attribution of cause of death [7], primarily to be used by

clinical research assistants (CRA). Reliability for the TRM classification was “almost

perfect” between medical and CRA reviewers that had been involved in developing the

system (kappa=0.92, 95% confidence interval 0.78-1.00) [7].

An ideal classification system for treatment-related mortality should be applicable across

different countries, treatment protocols, and health care settings. To assess this, it is important

to attempt to further validate the classification and attribution system in a different setting,

with professionals uninvolved in the development of the system. This paper reports such a

Page 4 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyvalidation at a large regional paediatric oncology centre in a different continent to the one in

which the system was originally developed.

Objectives

This study aimed to evaluate the newly developed consensus-based definition of TRM and

cause-of-death attribution system at a regional paediatric oncology centre in Leeds, England.

Methods

This study was approved by the University of Leeds School of Medicine Ethics committee

(Ref: MREC15-118) and did not require NHS ethics approval. Eligible patient records were

those of patients treated for a malignancy or who underwent a haematopoietic stem cell

transplant (HSCT) for a non-malignant diagnosis at Leeds Children’s Hospital (Leeds, UK)

while aged 18 years or younger at diagnosis. Five cases were excluded as patients had died

following relapse after the age of 18, or the medical records were not located. All included

patients died between 2014 and 2016. Data from the clinical records were anonymised, and

presented in a random order. Thirty patient records were included. Each set of records was

presented twice, once with information from 2 weeks prior to death, once with the

information extending back to 4 weeks prior to death, leading to a total of 60 assessments

being made. 4 participants were identified to review the case notes; the two CRAs were a data

analyst (AF) and research nurse (JT), and the two senior clinicians were a consultant

paediatric oncologist and consultant paediatric haematologist (AG and SK).

The study was undertaken on a single afternoon. After agreeing and signing a consent form,

participants received a 10 minute educational presentation explaining how to use the system,

and how the study would be undertaken. The reviewers then independently classified each

death according to the algorithm (fig. 1). For cases assessed as TRM, the reviewers were

asked to apply the cause-of-death attribution system (table 1). Following the individual

Page 5 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlycompletion of the assessments, a moderated group discussion was undertaken with notes

recorded by two facilitators and used to supplement the themes of the discussion.

Inter-relater reliability was assessed using the Kappa statistic (k). Criterion validity was

assessed by assuming classification by the Consultants as the gold standard. Group consensus

classification between and within the CRAs and Consultant group was evaluated using the

Cohen’s kappa statistic, and across all individuals using the Fleiss’ kappa statistic. The

strength of agreement was defined as slight (0.00-020), fair (0.21-0.40), moderate (0.41-0.60)

substantial (0.61-0.80) and almost perfect (0.81-1.00) [8]. Based on the previously published

study [7], we decided to include 30 cases for analysis. If the null hypothesis was 0.1, power

would be 0.75. A further 30 cases would be reviewed if validity was inadequate (defined a

priori as k<0.6). Calculation of the k statistic was completed using the R studio irr package

and a bootstrap with 200 iterations was used to calculate 95% confidence intervals [9].

Comparison of cause of death was qualitative, to provide further insight reviewers

participated in a discussion of the use of the algorithm and cause of death attribution system.

Page 6 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyFigure 1: Classification of TRM in children with cancer, taken from (7).

Page 7 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyTable 1 Summary of cause of death attribution system. Full system is supplied in a

supplementary S1 file.

Probable cause of death Possible cause of death

Infection Infection with associated

microbiologically

documented organism.

Infection without associated

microbiologically documented organism.

Haemorrhage Symptomatic

haemorrhage.

Symptomatic haemorrhage not shown by

imaging or pathology.

Thrombosis Symptomatic thrombosis

or embolism shown by


Symptomatic thrombosis or embolism not

shown by imaging or pathology.

Cardiac Symptomatic cardiac-

dysfunction defined by

ECG, cardiac imaging or

pathology.

Symptomatic arrhythmia excluding sinus

tachycardia or bradycardia not shown by

electrocardiography (ECG).

Immunomediated Acute anaphylaxis,

deteriorating or acute-

severe graft versus host

disease,

haemophagocytic

lymphohistocytosis, or

cytokine-release

syndrome.

Stable graft-versus-host disease.

Metabolic Symptomatic tumour

lysis syndrome.

CNS Symptomatic CNS

dysfunction shown by


Symptomatic CNS dysfunction not shown

by imaging or pathology.

Respiratory Symptomatic respiratory

distress with ventilator

support.

Symptomatic respiratory distress without

ventilator support.

Gastrointestinal

system

Symptomatic abdominal

disease resulting in or

perforation shown by


Symptomatic abdominal disease resulting

in or perforation not shown by imaging or

pathology.

Renal system Acute kidney injury

requiring intervention.

External causes E.g. suicide.

Page 8 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyResults

Demographics

Age of identified patients ranged from less than 1 to 17 years and 57% were male. 67% were

diagnosed with solid tumours, and 33% were diagnosed with malignant haematological

conditions. Collectively, sarcomas were the most frequent solid tumour diagnosed (40% of

total), followed by CNS tumours (30%). 25% of these patients had presented with metastatic

disease at diagnosis. All patients with malignant haematological conditions were diagnosed

with leukaemias. 40% of these patients had either received a transplant or presented with

relapsed disease. A summary is enclosed in a supplementary file.

Classification of treatment-related mortality

Ten deaths (33%) were classified as TRM. 10% of patients diagnosed with solid tumours,

and 80% of patients diagnosed with malignant haematological conditions were classified as

TRM. Reliability of classification was almost perfect between CRAs and consultants, with a

k statistic of 0.86 (95% confidence interval 0.72-0.97, with disagreement on 3 deaths). There

was also almost perfect agreement between CRAs (k=0.96, 95% confidence interval 0.87-

1.00, disagreement on one record) and between consultants (k=0.85, 95% confidence interval

0.67-0.97, with disagreement on two deaths) (See Table 3).

When the 2- and 4-week data were examined, there was a single difference between the

assessments of each of the four assessors (see Table 3).

Cause of death attribution system

Table 2 summarises the diagnoses of the ten patients whose deaths were classified as TRM,

alongside the cause of death as attribution by each assessor.

In brief, there was unanimous agreement on cause of death of 3 cases (J, V, AA), the other 7

cases has inconsistencies in either the nature of cause of death (e.g. infection vs. immune

Page 9 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlymediated), or strength of evidence (i.e. probably vs. possible). In one complex scenario (AD),

the CRAs did not feel able to record a cause of death.

Post-review discussion

The reviewers all agreed that the algorithm was straight forward to use, and that it would be

beneficial to have a standardised tool and attribution system to use in trials nationally and

globally.

Three difficulties with the system were identified: firstly, how to address the patients who

may die from TRM whilst receiving palliative care and particularly how the algorithm could

be used as part of palliative care trial. One case was a patient taking palliative etoposide

following a diagnosis of relapsed ALL, developed a febrile illness, and died. Secondly, how

should deaths in children after the completion of treatment be classified? A child (case V)

who had completed treatment for standard risk AML died of overwhelming pneumococcal

septicaemia 6 months after end of treatment while in complete remission. Another further

case the assessors had difficulty with (case M) followed presentation acutely with signs of

raised intra-cranial pressure and large mass noted on CT. This patient died on the operating

table whilst receiving surgical intervention.

Attribution of cause of death was felt to be more difficult than ascribing a death as TRM or

not. The case the reviewers felt was most difficult to assign was patient who died of multi-

organ failure following a HSCT (AD); both CRAs independently decided to not attribute a

cause of death.

Page 10 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from


Confidential: For Review Only

Table 2 Summary of cause of death attribution by reviewers for deaths classified as TRM. Bold font=probable causes of death.

Case Diagnosis CRA 1 CRA 2 Consultant 1 Consultant 2

J AML prior to HSCT

Immunomediated Immunomediated Immunomediated Immunomediated

M ATRT Acute symptomatic

intracranial haemorrhage

Acute symptomatic

intracranial

haemorrhage

NR Acute symptomatic intracranial

haemorrhage

V AML Infection Infection Infection Infection

X B-cell ALL Acute symptomatic


Acute symptomatic

intracranial haemorrhage Infection Infection

Y T-cell ALL Respiratory Infection Acute symptomatic


Infection

Z B-cell ALL Respiratory Infection Respiratory Infection

AA B-cell ALL Infection Infection Infection Infection

AB Ependymoma NR Infection NR Infection

AC T-cell ALL Worsening symptomatic

graft versus host disease.

Worsening symptomatic




Acute symptomatic pulmonary

haemorrhage

AD B-cell ALL post

HSCT

Unclear Difficult case Worsening symptomatic


Acute kidney Injury

Page 11 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on 11 May 2018 by guest. Protected by copyright. http://bmjpaedsopen.bmj.com/ bmjpo: first published as 10.1136/bmjpo-2017-000082 on 30 October 2017. Downloaded from



nlyTable 3 Summary of kappa statistic and 95% confidence intervals of independent

reviewers, consultants, CRAs and between CRA and consultants for all total case

reviews, 4 weeks and 2 weeks prior to death using the cause of death attribution system.

Inter-rater

comparison

Total k (95%

confidence interval)

4 weeks k (95%


2 weeks k (95%


Independent

reviewers

0.92 (0.83-0.98) 0.91(0.76-1.00) 0.92 (0.79-1.00)

Consultants 0.85 (0.67-0.97) 0.85 (0.59-1.00) 0.84 (0.59-1.00)

CRA 0.96 (0.87-1.00) 0.85 (0.59-1.00) 0.85 (0.60-1.00)

CRA vs consultants 0.86 (0.72-0.97) 0.87 (0.66-1.00) 0.86 (0.67-1.00)

Page 12 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyDiscussion

This study is, to the best of our knowledge, the first revalidation of the standardised definition

of treatment-related mortality and cause of death attribution system for paediatric cancer

patients [7]. It demonstrates the system is reliable and established criterion validity in an

alternative centre and health care system with different treatment protocols. It can be used

after very limited training, with “almost perfect” agreement between assessors irrespective of

discipline (Fleiss kappa 0.92, 95% CI 0.83-0.98). The study confirms the observations of the

development group and shows that information from two weeks prior to the death of a patient

is sufficient to consistently attribute death to TRM or disease.

Challenges and areas for future development

Our study highlights specific challenges with the system as it currently exists, both with the

classification of TRM and the attribution of a specific cause of death. Fundamentally, this

approach defines deaths as either ‘treatment related’ or ‘cancer related’. This gives rise to a

semantic challenge; “treatment-related mortality” implies that deaths that come under this

term occur directly because of the therapies delivered. However, the classification system

classifies deaths which occur prior to commencement of anti-cancer therapy, which are not

directly attributable to the cancer (for example, tumour lysis syndrome in high-count

leukaemia) as cases of TRM. This clash of language and “common sense” may confuse users

of the classification tool, for example, case M in which a patient presented acutely with signs

of raised intra-cranial pressure and died on the operating table.

A deeper challenge to this system addressed the philosophical distinction of assigning all

deaths into one of two categories; cancer or treatment related. There is a convincing argument

that a third category of death should be attributable, “other non-cancer death”, for those who

Page 13 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlydie of an event or illness external to their malignancy. This problem is particularly evident if

the current system is to be used after the completion of treatment. For example, a patient dies

as a passenger in an air traffic accident, 4 years after treatment for a localised Wilms tumour.

In the current system, this death is classified as TRM, even though the death is unrelated to

the child’s cancer diagnosis.

Conversely, it is also, important to note how a diagnosis of cancer may be included in ‘non-

cancer deaths’. For example a patient may commit suicide some years after the completion of

treatment because of the psychological effects of their diagnosis or treatment. Any addition to

the system would need to be sensitive to these potential issues.

We have identified the need to further refine the approach to categorising cause of death in

cancer patients receiving care without intent to cure. This is particularly important if the

system is applied in ‘routine’ settings, assessing deaths in the palliative setting rather than in

the original setting of use within a curative trial. Increasingly individuals destined not to be

cured are living for lengthier periods due to participation in clinical trials/studies. This group

of patients currently have all deaths classified as “not treatment-related mortality” as

clinicians would have specified progressive disease, or that cancer therapy has no curative

intent. This algorithm may fail to identify a significant group of patients who may die of

causes amenable to better supportive care whilst receiving palliative care. For example, a

patient can die of overwhelming sepsis whilst receiving palliative etoposide for refractory

neuroblastoma. This could be addressed by modifying the algorithm, for this type of use.

Another proposal includes using a separate classification tool for patients on palliative care

trials (fig. 2). The counter-argument to this suggested change is the risk of adding complexity

to a simple, effective, tool which can be used by people of different skills and health care

settings globally. It will also have similar issues with interpretation as with initial algorithm.

Page 14 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyReviewers failed to agree on cause of death in 6 episodes. Differences in cause of death

allocated could be attributed to the reviewers’ previous experience, clinical expertise and

interpretation of the clinical records, particularly in light of potential previous direct clinical

involvement with the cases under review. Currently, the tool is intended for use by any CRA,

however, it may require users to have a certain level of experience or clinical expertise, and

agreement may be reduced with CRA who are new to the role.

Understanding and interpretation of the system as proposed for attribution of a specific

mechanistic cause of death could potentially be improved by dedicating more time during the

presentations and using clinical scenarios. Alternatively, the cause of death attribution system

could be further refined. Currently reviewers, attribute death directly to one, most likely,

cause. As multiple organ systems may be involved, exploring the option of listing more than

one cause could be considered. Another possibility includes refining the current grouping, for

example rather than having ‘respiratory’ and ‘infection’ as 2 separate entities, ‘respiratory

infection’ could be listed, making it easier for the reviewer to identify the primary cause.

Page 15 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyFigure 2: Proposed classification of TRM in children receiving non-curative therapy

(NCT) only.

Page 16 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyConclusions

We have been able to confirm the reproducibility and criterion validity of the TRM

classification system. We believe this supports the hypothesis that the classification system

can be implemented easily and effectively in different health care settings, thereby improving

consistency and accuracy of outcome reporting in clinical trials. The TRM classification

system will be of immense value in the evaluation of deaths in the palliative setting. We

propose the addition of a separate classification tool in patients on palliative trials.

Page 17 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyConflict of interest statement

The authors declare no conflict of interest.

Acknowledgments

The authors would like to thank the reviewers for participating in this study. This work was

supported by Candlelighters charity who funded Dr Hadeel Hassan’s PhD, for which this

research was conducted. H.H wrote the manuscript and R.S.P edited the manuscript. M.R,

S.E.K and A.G contributed to the review process.

Page 18 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyReferences

1. CRUK. Children's cancers mortality in Europe and worldwide. 2014 [cited 2016

September]; Available from: http://www.cancerresearchuk.org/health-

professional/cancer-statistics/childrens-cancers/mortality#heading-Three.

2. Kremer, L.C., et al., A worldwide collaboration to harmonize guidelines for the long-

term follow-up of childhood and young adult cancer survivors: a report from the

International Late Effects of Childhood Cancer Guideline Harmonization Group.

Pediatr Blood Cancer, 2013. 60(4): p. 543-9.

3. Blanco, E., et al., Non-relapse mortality in pediatric acute lymphoblastic leukemia: a

systematic review and meta-analysis. Leuk Lymphoma, 2012. 53(5): p. 878-85.

4. White, Y., V.P. Castle, and A. Haig, Pediatric oncology in developing countries:

challenges and solutions. J Pediatr, 2013. 162(6): p. 1090-1, 1091.e1.

5. Ethier, M.C., et al., Lack of clarity in the definition of treatment-related mortality:

pediatric acute leukemia and adult acute promyelocytic leukemia as examples. Blood,

2011. 118(19): p. 5080-3.

6. Tran, T.H., et al., Lack of treatment-related mortality definitions in clinical trials of

children, adolescents and young adults with lymphomas, solid tumors and brain

tumors: a systematic review. BMC Cancer, 2014. 14: p. 612.

7. Alexander, S., et al., Classification of treatment-related mortality in children with

cancer: a systematic assessment. Lancet Oncol, 2015. 16(16): p. e604-10.

8. McHugh, M.L., Interrater reliability: the kappa statistic. Biochemia Medica, 2012.

22(3): p. 276-282.

9. Team, R., RStudio: Integrated Development for R. RStudio, Inc. 2015.

Figure legends

Figure 1: Classification of TRM in children with cancer, taken from (7).

Figure 2: Proposed classification of TRM in children receiving non-curative therapy (NCT)

only.

Supplementary files legend list

Supplemental table S1: Cause-of-death attribution system.

Supplemental table S2: Diagnosis of included patients results of classification by reviewers.

Page 19 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly

190x338mm (96 x 96 DPI)

Page 20 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from


Confidential: For R

eview O

nly

Page 21 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlySupplemental file S1


Infection A.1 Clinically or

radiographically

documented infection with

associated microbiologically

documented organism

1. Clinically or

radiographically

documented infection

without associated

microbiologically


Haemorrhage B.1 Acute symptomatic


shown by imaging or

pathology.

B.2 Acute symptomatic

pulmonary haemorrhage

shown by imaging or

pathology.


bleeding resulting in

hypotension, urgent

transfusion or fluid bolus.

2. Acute symptomatic

pulmonary haemorrhage not

shown by imaging or

pathology

Thrombosis C.1 Acute symptomatic

intracranial thrombosis or

embolism shown by


C.2 Acute symptomatic

pulmonary thrombosis or

embolism shown by



hepatic thrombosis or

embolism shown by



pulmonary thrombosis

or embolism not shown

by imaging or

pathology.

Cardiac D.1 Acute symptomatic

arrhythmia excluding

sinus tachycardia or

bradycardia shown by

ECG.

D.2 Acute symptomatic

cardiac- dysfunction defined

by ECG, cardiac imaging or

pathology.


arrhythmia excluding sinus

tachycardia or bradycardia

not shown by ECG.

Immunomediated

E.1 Acute allergic

reaction, anaphylaxis with

symptomatic

bronchospasm, oedema,

angioedema, or

hypotension.

E.2 Worsening

symptomatic graft

5. Stable graft-versus-host

disease.

Page 22 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyversus host disease.

E.3 Acute symptomatic

haemophagocytic

lymphohistocytosis,

macrophage activation

syndrome or cytokine-

release syndrome.

Metabolic F. Clinically diagnosed

tumour lysis syndrome

with cardiac arrhythmia,

seizure, or creatinine

concentrations greater

than 3 times the ULN.

CNS G.1 Acute symptomatic

CNS necrosis shown by

imaging or pathology

G.2 Acute symptomatic

encephalopathy shown by

imaging or

electroencephalography


hydrocephalus or raised

intracranial pressure shown

by imaging, pathology, or

measurement of intracranial

pressure

G.4 Seizure lasting at least

30 minutes within 48 hours

of death

6.1 Acute symptomatic CNS

necrosis not shown by


6.2 Acute symptomatic

encephalopathy not shown

by imaging or




intracranial pressure not

shown by imaging,

pathology, or measurement

of intracranial pressure

6.4 Seizure between 5 and


of death

Respiratory H. Acute symptomatic

respiratory distress with

ventilator support


respiratory distress without

ventilator support

Gastrointestinal system I.1 Acute symptomatic

bowel disease resulting in

necrosis, obstruction or



I.2 Acute, clinically

diagnosed hepatic

dysfunction associated with

conjugated bilirubin

concentrations greater than

10 x ULN, ammonium


2.5 x ULN, or international

normalised greater than 2.5

times the ULN


diagnosed pancreatitis with

haemorrhage, peritonitis,




perforation not shown by


8.2 Acute, clinically

diagnosed hepatic




1.5 and less than 10 x ULN,

ammonium concentrations

greater than 1.5 and less

than 2.5 x ULN, or

international normalised

greater than 1.5 but less than

2.5 times the ULN

Page 23 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlynecrosis or haemodynamic

instability (evidenced by

hypotension, urgent

transfusion, fluid bolus, or

vasopressers

Renal system J. Acute kidney injury with

dialysis or renal replacement

therapy (planned or

received)

External causes K.1 Unintentional injury

(e.g accident)

K.2 Suicide

K.3 Homicide

ULN: Upper Limit of Normal ECG: electrocardiogram CNS: central nervous system

Page 24 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from




Case Diagnosis CRA 1 CRA 2 Consultant

1

Consultant

2

A Germ cell CNS tumour Not TRM Not TRM Not TRM Not TRM

B Diffuse intrinsic

pontine glioma (DIPG)

Not TRM Not TRM Not TRM Not TRM

C Thalamic astrocytoma Not TRM Not TRM Not TRM Not TRM

D Relapsed B-cell ALL Not TRM Not TRM Not TRM Not TRM

E Spinal cord

glioblastoma

multiforme


F Metastatic Ewing’s

sarcoma


G Metastatic Ewing’s

sarcoma


H Metastatic

rhabdomyosarcoma


I Philadelphia positive

ALL


J AML prior to HSCT TRM TRM TRM TRM

K Metastatic

rhabdomyosarcoma


L Soft tissue

myoepithelial

carcinoma


M ATRT TRM TRM Not TRM TRM

N Metastatic Ewing’s

sarcoma


O High risk

neuroblastoma


P Osteosarcoma Not TRM Not TRM Not TRM Not TRM

Q DIPG Not TRM Not TRM Not TRM Not TRM

R Osteosarcoma Not TRM Not TRM Not TRM Not TRM

Page 25 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyCase Diagnosis CRA 1 CRA 2 Consultant

1

Consultant

2

S High risk

neuroblastoma


T Malignant melanoma of

the CNS


U Metastatic Ewing’s

sarcoma


V AML TRM TRM TRM TRM

W Epithelial sarcoma Not TRM Not TRM Not TRM Not TRM

X ALL TRM TRM TRM TRM

Y T-cell ALL TRM TRM TRM TRM

Z B-cell ALL TRM TRM TRM TRM

AA B-cell ALL TRM TRM TRM TRM

AB Ependymoma TRM TRM Not TRM TRM

AC T-cell ALL TRM TRM TRM TRM

AD B-cell ALL post HSCT TRM TRM TRM TRM

Page 26 of 26



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly




Manuscript ID bmjpo-2017-000082.R1


Date Submitted by the Author: 20-Sep-2017


Kinsey, Sally; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Glaser, Adam; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Phillips, Bob; University of York, Centre for Reviews and Dissemination; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology






j.com/


jpo-2017-000082 on 30 October 2017. D

ownloaded from



nlyTitle:


Authors:

H. Hassan 1, 2

, M. Rompola 1, 2

, A.W. Glaser 1, 2

, S. E. Kinsey 1, 2

, R. S. Phillips 1, 3

Affiliations:


Leeds, UK





Dr Hadeel Hassan


Telephone: +44 (0)113 343 2596


Page 1 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyAbstract






Objectives: To evaluate the reliability of the newly developed consensus-based definition of

TRM and explore the use of the cause-of-death attribution system outside the centre it was

initially validated (Toronto, Canada).


Leeds, UK.






death attribution system to identify the primary cause of death. Inter-relater reliability was

assessed using the Kappa statistic (k).


Results: Reliability of the classification was deemed “very good” between CRA and



cause of death (e.g. respiratory versus infection) when applying the cause-of-death attribution

Page 2 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlysystem in 6 cases and probable and possible causes in 4 cases. The study identified how the

algorithm may not detect TRM in patients receiving non-curative therapy.

Conclusions: The classification and cause of death attribution system could be implemented





haematology.




• The classification tool is not designed to detect TRM in patients receiving non-curative

therapy.

• The classification and cause of death attribution system can be used in different health

care settings.

Page 3 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyBackground

















clinical research assistants (CRA). Reliability for the TRM classification was “very good”

between medical and CRA reviewers that had been involved in developing the system

(kappa=0.92, 95% confidence interval 0.78-1.00) [7].




with professionals uninvolved in the development of the system. This paper reports such

Page 4 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from




which the system was originally developed.

Objectives

This study aimed to evaluate the reliability of the newly developed consensus-based definition

of TRM and explore the use of the cause-of-death attribution system at a regional paediatric

oncology centre in Leeds, England.

Methods







patients died between 2014 and 2016. Thirty patient records were included. Copies of the

clinical records, with information from both 2 weeks prior to death, and with the information

extending back to 4 weeks prior to death were anonymised. This resulted in 60 sets of

anonymised case-notes (30 patients, each with 2 time periods) which were presented in a

different random order for each assessor. 4 participants were identified to review the case

notes; the two CRAs were a data analyst (AF) and research nurse (JT), and the two senior

clinicians were a consultant paediatric oncologist and consultant paediatric haematologist

(AG and SK).

The study was undertaken on a single afternoon. After agreeing and signing a consent form,



Page 5 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlydeath according to the algorithm (fig. 1). For cases assessed as TRM, the reviewers were

asked to apply the cause-of-death attribution system (supplemental file 1) to identify a

primary cause of death. Following the individual completion of the assessments, a moderated

group discussion was undertaken with notes recorded by two facilitators and used to

supplement the themes of the discussion.





strength of agreement was defined as slight (0.00-020), fair (0.21-0.40), moderate (0.41-

0.60), good (0.61-0.80) and very good (0.81-1.00) [8]. A numerical code was used to

combine agreement/disagreement between the individual consultants (TRM was recorded as

“0” and non TRM outcomes were recorded as “1” in Excel). When calculating inter-relater

reliability between the CRAs and consultants if individual disagreement was recorded then

the outcome was recorded as “2”. Based on the previously published study [7], we decided

to include 30 cases for analysis. A sample size of 27 deaths determined whether k was good

(i.e., ≥0·61), with a power of 0·80, and two-sided α of 0·05 and assuming that treatment-

related mortality accounted for 20% of deaths (6). A further 30 cases would be reviewed if

validity was inadequate (defined a priori as k<0.6). Calculation of the k statistic was

completed using the R studio irr package, and a bootstrap with 200 iterations was used to

calculate 95% confidence intervals [9]. Comparison of cause of death was qualitative, and to

provide further insight reviewers participated in a discussion of the use of the algorithm and

cause of death attribution system.

Page 6 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyFigure 1: Classification of TRM in children with cancer, taken from (7)

Page 7 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly Results

Demographics

Age of identified patients ranged from less than 1 to 17 years and 57% (17) were male. 67%

(20) were diagnosed with solid tumours, and 33% (10) were diagnosed with malignant

haematological conditions. Collectively, sarcomas were the most frequent solid tumour

diagnosed (12 cases, 40% of total), followed by CNS tumours (9 cases, 30% of total). 27% (8

cases) of these patients had presented with metastatic disease at diagnosis. All patients with

malignant haematological conditions were diagnosed with leukaemia. 40% (12 cases) of

patients had either received a transplant or presented with relapsed disease. A summary is

enclosed in a supplementary file 2.


Ten deaths (33%) were identified as TRM by at least one reviewer. Fifteen percent (3/20) of

patients diagnosed with solid tumours, and 80% (8/10) of patients diagnosed with malignant

haematological conditions who died were classified as TRM. Reliability of classification was

very good between CRAs and consultants, with a k statistic of 0.86 (95% confidence interval

0.72-0.97, with disagreement on 3 deaths). There was also very good agreement between

CRAs (k=0.96, 95% confidence interval 0.87-1.00, disagreement on one record) and between

consultants (k=0.85, 95% confidence interval 0.67-0.97, with disagreement on two deaths)

(See Table 1).

When the 2 and 4 week data were examined, there no difference between the assessments of

each of the four assessors (see Table 1).




Page 8 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyIn brief, there was unanimous agreement on cause of death of 3 cases (J, V, AA), the other 7


mediated), or strength of evidence (i.e. probable vs. possible). In one complex scenario (AD),





globally.


may die from TRM whilst receiving non-curative therapy and particularly how the algorithm

could be used as part of palliative care trial. In one particular case a patient taking palliative

etoposide following a diagnosis of relapsed ALL developed a febrile illness, and died.

Secondly, how should deaths in children after the completion of treatment be classified? A

child (case V) who had completed treatment for standard risk AML died of overwhelming

pneumococcal septicaemia 6 months after end of treatment whist in complete remission.

Another further case the assessors had difficulty with (case M) followed presentation acutely

with signs of raised intra-cranial pressure and large mass noted on CT. This patient died on

the operating table whilst receiving surgical intervention.




cause of death.

Page 9 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from






*calculated using the Fleiss kappa statistic (between 4 reviewers).

**calculated using the Cohen’s kappa statistic (between 2 reviewers or 2 groups).

Inter-rater

comparison

Total k (95%


4 weeks k (95%


2 weeks k (95%


Independent

reviewers *

0.92 (0.83-0.98) 0.91(0.76-1.00) 0.92 (0.79-1.00)

Consultants ** 0.85 (0.67-0.97) 0.85 (0.59-1.00) 0.84 (0.59-1.00)

CRA ** 0.96 (0.87-1.00) 0.85 (0.59-1.00) 0.85 (0.60-1.00)

CRA vs

consultants**

0.86 (0.72-0.97) 0.87 (0.66-1.00) 0.86 (0.67-1.00)

Page 10 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from





J AML prior to HSCT




Acute symptomatic

intracranial

haemorrhage


haemorrhage




Acute symptomatic




Infection











haemorrhage

AD B-cell ALL post

HSCT



Acute kidney Injury

Page 11 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 12 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




nlyDiscussion

Strengths of study



patients [7]. It demonstrates that the system is reliable and established its validity in an


after very limited training, with “very good” agreement between assessors irrespective of




Limitations of study

Although consultants’ opinions are considered gold-standard, in this study we identified how

even experienced clinicians may disagree on use of the algorithm. Consultants disagreed on

the classification of death in two cases- this may have occurred due to the individual

consultant’s clinical experience, or previous contact with the patients. Even though the cases

were anonymised and randomised the physicians may have recognised the patient due to their

potential clinical involvement in direct patient care. The differences identified highlight how

the TRM classification tool is unlikely to ever have perfect agreement between reviewers

irrespective of experience, and clinical, and scientific knowledge.

In this study reviewers attributed death to one primary probable, or possible, cause. Whilst

developing the study protocol, we decided to limit the number of causes of death for

simplicity. However, reviewers found it challenging to identify only one cause of death, and

distinguish between probable and possible causes.

Page 13 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlySince the development of this study, a standard operating procedure TRM web-based tool has

been published (https://www.sungresearch.com/trm-training-manual/) and includes working

examples. Use of this tool when delivering the training package should help clarify how to

use the cause-of-death attribution system and minimise misunderstanding. Currently, the web-

based tool is available in English, having the tool available in other languages could

potentially reduce confusion and improve harmonisation across clinical trials.

Challenges and areas for future development














die of an event or illness external to their malignancy. This problem is particularly evident if



Page 14 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyIn the current system, this death is classified as TRM, even though the death is unrelated to


Conversely, it is also, important to note how a diagnosis of cancer may be included in ‘non-















Another proposal includes using a separate classification tool for patients on non-curative

therapy trials (fig. 2) although this should be further developed in conjunction with palliative

care physicians and researchers. The counter-argument to this suggested change is the risk of

adding complexity to a simple, effective, tool which can be used by people of different skills

and health care settings globally. It will also have similar issues with interpretation as with

initial algorithm.

Page 15 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyReviewers failed to agree on a primary cause of death in 6 episodes and probable and possible

causes in 4 cases. Differences in cause of death allocated could be attributed to the reviewers’

previous experience, clinical expertise and interpretation of the clinical records, particularly in

light of potential previous direct clinical involvement with the cases under review. Currently,

the tool is intended for use by any CRA, however, it may require users to have a certain level

of experience or clinical expertise, and agreement may be reduced with CRA who are new to

the role.



presentations and using the newly developed web-based training tool. Alternatively, the cause

of death attribution system could be refined, and multiple causalities permitted.

Page 16 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from




(NCT) only.

Page 17 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyConclusions






propose the addition of a separate classification tool in patients on palliative trials.

Page 18 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from





Acknowledgments





Page 19 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyReferences














2011. 118(19): p. 5080-3.







22(3): p. 276-282.


Figure legends



only.




Page 20 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly

81x45mm (300 x 300 DPI)

Page 21 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly

45x25mm (300 x 300 DPI)

Page 22 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from






radiographically



documented organism

1. Clinically or

radiographically


without associated

microbiologically




shown by imaging or

pathology.



shown by imaging or

pathology.



hypotension, urgent




shown by imaging or

pathology



embolism shown by




embolism shown by




embolism shown by





by imaging or

pathology.





ECG.




pathology.




not shown by ECG.

Immunomediated

E.1 Acute allergic


symptomatic


angioedema, or

hypotension.


disease.

Page 23 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyE.2 Worsening

symptomatic graft

versus host disease.


haemophagocytic

lymphohistocytosis,



release syndrome.












imaging or







pressure



of death






by imaging or





shown by imaging,





of death



ventilator support



ventilator support







diagnosed hepatic




10 x ULN, ammonium




times the ULN







diagnosed hepatic







than 2.5 x ULN, or


Page 24 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyI.3 Acute, clinically



necrosis or haemodynamic


hypotension, urgent


vasopressers


2.5 times the ULN



therapy (planned or

received)


(e.g accident)

K.2 Suicide

K.3 Homicide


Page 25 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from





1

Consultant

2


B Diffuse intrinsic





E Spinal cord

glioblastoma

multiforme



sarcoma



sarcoma


H Metastatic

rhabdomyosarcoma



ALL



K Metastatic

rhabdomyosarcoma


L Soft tissue

myoepithelial

carcinoma




sarcoma


O High risk

neuroblastoma





Page 26 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from




1

Consultant

2

S High risk

neuroblastoma


T Malignant melanoma

of the CNS



sarcoma











Page 27 of 27



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly




Manuscript ID bmjpo-2017-000082.R2


Date Submitted by the Author: 10-Oct-2017


Kinsey, Sally; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Glaser, Adam; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Phillips, Bob; University of York, Centre for Reviews and Dissemination; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology






j.com/


jpo-2017-000082 on 30 October 2017. D

ownloaded from



nlyTitle:


Authors:

H. Hassan 1, 2

, M. Rompola 1, 2

, A.W. Glaser 1, 2

, S. E. Kinsey 1, 2

, R. S. Phillips 1, 3

Affiliations:


Leeds, UK





Dr Hadeel Hassan


Telephone: +44 (0)113 343 2596


Page 1 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyAbstract






Objectives: To evaluate the reliability of the newly developed consensus-based definition of

TRM and explore the use of the cause-of-death attribution system outside the centre it was

initially validated (Toronto, Canada). In the initial study reviewers listed multiple causes of

death. In this study, reviewers identified a primary cause for simplicity.


Leeds, UK.






death attribution system to identify the primary cause of death. Inter-relater reliability was

assessed using the Kappa statistic (k).


Results: Reliability of the classification was deemed “very good” between CRA and



cause of death (e.g. respiratory versus infection) when applying the cause-of-death attribution

Page 2 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlysystem in 6 cases and probable and possible causes in 4 cases. The study identified how the

algorithm may not detect TRM in patients receiving non-curative therapy.

Conclusions: The classification and cause of death attribution system could be implemented





haematology.




• The classification tool is not designed to detect TRM in patients receiving non-curative

therapy.

• The classification and cause of death attribution system can be used in different health

care settings.

Page 3 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyBackground

















clinical research assistants (CRA). Reliability for the TRM classification was “very good”

between medical and CRA reviewers that had been involved in developing the system

(kappa=0.92, 95% confidence interval 0.78-1.00) [7].




with professionals uninvolved in the development of the system. This paper reports such

Page 4 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from




which the system was originally developed. In the original study reviewers attributed death to

multiple probable or possible causes. Whilst developing the study protocol, we decided to

limit the number of causes of death to one primary probable or possible cause for simplicity.

Objectives

This study aimed to evaluate the reliability of the newly developed consensus-based definition

of TRM and explore the use of the cause-of-death attribution system at a regional paediatric

oncology centre in Leeds, England. In the initial study reviewers listed multiple causes of

death. However, in this study reviewers identified a primary cause for simplicity.

Methods







patients died between 2014 and 2016. Thirty patient records were included. Copies of the

clinical records, with information from both 2 weeks prior to death, and with the information

extending back to 4 weeks prior to death were anonymised. This resulted in 60 sets of

anonymised case-notes (30 patients, each with 2 time periods) which were presented in a

different random order for each assessor. 4 participants were identified to review the case

notes; the two CRAs were a data analyst (AF) and research nurse (JT), and the two senior

clinicians were a consultant paediatric oncologist and consultant paediatric haematologist

(AG and SK).

Page 5 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyThe study was undertaken on a single afternoon. After agreeing and signing a consent form,



death according to the algorithm (fig. 1). For cases assessed as TRM, the reviewers were

asked to apply the cause-of-death attribution system (supplemental file 1) to identify a

primary cause of death. Following the individual completion of the assessments, a moderated

group discussion was undertaken with notes recorded by two facilitators and used to

supplement the themes of the discussion.





strength of agreement was defined as slight (0.00-020), fair (0.21-0.40), moderate (0.41-

0.60), good (0.61-0.80) and very good (0.81-1.00) [8]. A numerical code was used to

combine agreement/disagreement between the individual consultants (TRM was recorded as

“0” and non TRM outcomes were recorded as “1” in Excel). When calculating inter-relater

reliability between the CRAs and consultants if individual disagreement was recorded then

the outcome was recorded as “2”. Based on the previously published study [7], we decided

to include 30 cases for analysis. A sample size of 27 deaths determined whether k was good

(i.e., ≥0·61), with a power of 0·80, and two-sided α of 0·05 and assuming that treatment-

related mortality accounted for 20% of deaths (6). A further 30 cases would be reviewed if

validity was inadequate (defined a priori as k<0.6). Calculation of the k statistic was

completed using the R studio irr package, and a bootstrap with 200 iterations was used to

calculate 95% confidence intervals [9]. Comparison of cause of death was qualitative, and to

Page 6 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyprovide further insight reviewers participated in a discussion of the use of the algorithm and

cause of death attribution system.

Page 7 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyFigure 1: Classification of TRM in children with cancer, taken from (7)

Page 8 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly Results

Demographics

Age of identified patients ranged from less than 1 to 17 years and 57% (17) were male. 67%

(20) were diagnosed with solid tumours, and 33% (10) were diagnosed with malignant

haematological conditions. Collectively, sarcomas were the most frequent solid tumour

diagnosed (12 cases, 40% of total), followed by CNS tumours (9 cases, 30% of total). 27% (8

cases) of these patients had presented with metastatic disease at diagnosis. All patients with

malignant haematological conditions were diagnosed with leukaemia. 40% (12 cases) of

patients had either received a transplant or presented with relapsed disease. A summary is

enclosed in a supplementary file 2.


Ten deaths (33%) were identified as TRM by at least one reviewer. Fifteen percent (3/20) of

patients diagnosed with solid tumours, and 80% (8/10) of patients diagnosed with malignant

haematological conditions who died were classified as TRM. Reliability of classification was

very good between CRAs and consultants, with a k statistic of 0.86 (95% confidence interval

0.72-0.97, with disagreement on 3 deaths). There was also very good agreement between

CRAs (k=0.96, 95% confidence interval 0.87-1.00, disagreement on one record) and between

consultants (k=0.85, 95% confidence interval 0.67-0.97, with disagreement on two deaths)

(See Table 1).

When the 2 and 4 week data were examined, there no difference between the assessments of

each of the four assessors (see Table 1).




Page 9 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyIn brief, there was unanimous agreement on cause of death of 3 cases (J, V, AA), the other 7


mediated), or strength of evidence (i.e. probable vs. possible). In one complex scenario (AD),





globally.


may die from TRM whilst receiving non-curative therapy and particularly how the algorithm

could be used as part of palliative care trial. In one particular case a patient taking palliative

etoposide following a diagnosis of relapsed ALL developed a febrile illness, and died.

Secondly, how should deaths in children after the completion of treatment be classified? A

child (case V) who had completed treatment for standard risk AML died of overwhelming

pneumococcal septicaemia 6 months after end of treatment whist in complete remission.

Another further case the assessors had difficulty with (case M) followed presentation acutely

with signs of raised intra-cranial pressure and large mass noted on CT. This patient died on

the operating table whilst receiving surgical intervention.




cause of death.

Page 10 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from






*calculated using the Fleiss kappa statistic (between 4 reviewers).

**calculated using the Cohen’s kappa statistic (between 2 reviewers or 2 groups).

Inter-rater

comparison

Total k (95%


4 weeks k (95%


2 weeks k (95%


Independent

reviewers *

0.92 (0.83-0.98) 0.91(0.76-1.00) 0.92 (0.79-1.00)

Consultants ** 0.85 (0.67-0.97) 0.85 (0.59-1.00) 0.84 (0.59-1.00)

CRA ** 0.96 (0.87-1.00) 0.85 (0.59-1.00) 0.85 (0.60-1.00)

CRA vs

consultants**

0.86 (0.72-0.97) 0.87 (0.66-1.00) 0.86 (0.67-1.00)

Page 11 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from





J AML prior to HSCT




Acute symptomatic

intracranial

haemorrhage


haemorrhage




Acute symptomatic




Infection











haemorrhage

AD B-cell ALL post

HSCT



Acute kidney Injury

Page 12 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Page 13 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




nlyDiscussion



patients [7]. It demonstrates that the system is reliable and established its validity in an


after very limited training, with “very good” agreement between assessors irrespective of




Although consultants’ opinions are considered gold-standard, in this study we identified how

even experienced clinicians may disagree on use of the algorithm. Consultants disagreed on

the classification of death in two cases- this may have occurred due to the individual

consultant’s clinical experience, or previous contact with the patients. Even though the cases

were anonymised and randomised the physicians may have recognised the patient due to their

potential clinical involvement in direct patient care. The differences identified highlight how

the TRM classification tool is unlikely to ever have perfect agreement between reviewers

irrespective of experience, and clinical, and scientific knowledge.

In this study reviewers attributed death to one primary probable, or possible, cause. Whilst

developing the study protocol, we decided to limit the number of causes of death for

simplicity. However, reviewers found it challenging to identify only one cause of death, and

distinguish between probable and possible causes. In the original study reviewers attributed

death to multiple probable or possible causes. Despite this difference, reviewers only agreed

on the causes of death on 3 of 9 cases classified as TRM. This is similar to the results in this

study of which reviewers agreed on 3 of 10 cases.

Page 14 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlySince the development of this study, a standard operating procedure TRM web-based tool has

been published (https://www.sungresearch.com/trm-training-manual/) and includes working

examples. Use of this tool when delivering the training package should help clarify how to

use the cause-of-death attribution system and minimise misunderstanding. Currently, the web-

based tool is available in English, having the tool available in other languages could

potentially reduce confusion and improve harmonisation across clinical trials.














die of an event or illness external to their malignancy. This problem is particularly evident if



In the current system, this death is classified as TRM, even though the death is unrelated to


Page 15 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyConversely, it is also, important to note how a diagnosis of cancer may be included in ‘non-















Another proposal includes using a separate classification tool for patients on non-curative

therapy trials (fig. 2) although this should be further developed in conjunction with palliative

care physicians and researchers. The counter-argument to this suggested change is the risk of

adding complexity to a simple, effective, tool which can be used by people of different skills

and health care settings globally. It will also have similar issues with interpretation as with

initial algorithm.

Reviewers failed to agree on a primary cause of death in 6 episodes and probable and possible

causes in 4 cases. Differences in cause of death allocated could be attributed to the reviewers’

previous experience, clinical expertise and interpretation of the clinical records, particularly in

light of potential previous direct clinical involvement with the cases under review. Currently,

Page 16 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlythe tool is intended for use by any CRA, however, it may require users to have a certain level

of experience or clinical expertise, and agreement may be reduced with CRA who are new to

the role.



presentations and using the newly developed web-based training tool. Alternatively, the cause

of death attribution system could be refined, and multiple causalities permitted.

Page 17 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from




(NCT) only.

Page 18 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyConclusions






propose the addition of a separate classification tool in patients receiving palliative treatment.

Page 19 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from





Acknowledgments





Page 20 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyReferences














2011. 118(19): p. 5080-3.







22(3): p. 276-282.


Figure legends



only.




Page 21 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly

81x45mm (300 x 300 DPI)

Page 22 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly

45x25mm (300 x 300 DPI)

Page 23 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from




Table 1: Cause-of-death attribution system.



radiographically



documented organism

1. Clinically or

radiographically


without associated

microbiologically




shown by imaging or

pathology.



shown by imaging or

pathology.



hypotension, urgent




shown by imaging or

pathology



embolism shown by




embolism shown by




embolism shown by





by imaging or

pathology.





ECG.




pathology.




not shown by ECG.

Immunomediated

E.1 Acute allergic


symptomatic


angioedema, or

hypotension.


disease.

Page 24 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyE.2 Worsening

symptomatic graft

versus host disease.


haemophagocytic

lymphohistocytosis,



release syndrome.












imaging or







pressure



of death






by imaging or





shown by imaging,





of death



ventilator support



ventilator support







diagnosed hepatic




10 x ULN, ammonium




times the ULN







diagnosed hepatic







than 2.5 x ULN, or


Page 25 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nlyI.3 Acute, clinically



necrosis or haemodynamic


hypotension, urgent


vasopressers


2.5 times the ULN



therapy (planned or

received)


(e.g accident)

K.2 Suicide

K.3 Homicide


Page 26 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from



nly

Page 27 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from





1

Consultant

2


B Diffuse intrinsic





E Spinal cord

glioblastoma

multiforme



sarcoma



sarcoma


H Metastatic

rhabdomyosarcoma



ALL



K Metastatic

rhabdomyosarcoma


L Soft tissue

myoepithelial

carcinoma




sarcoma


O High risk

neuroblastoma





Page 28 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from




1

Consultant

2

S High risk

neuroblastoma


T Malignant melanoma

of the CNS



sarcoma











Page 29 of 29



123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jpaedsopen.bmj.com

/bm


ctober 2017. Dow

nloaded from


bmj paediatrics open is committed to open peer review. as...

Documents