TITLE PAGE

Title

Characteristics of randomized trials focusing on stroke due to intracerebral hemorrhage:

systematic review

Authors

Arina Tamborska BSc,1 Michael T.C. Poon MSc,2 Rustam Al-Shahi Salman PhD.2

1 College of Medicine and Veterinary Medicine, University of Edinburgh, Chancellor's

Building, 49 Little France Crescent, Edinburgh, EH16 4SB, UK

2 Centre for Clinical Brain Sciences, University of Edinburgh, Chancellor's Building, 49 Little

France Crescent, Edinburgh, EH16 4SB, UK

Author for correspondence

Prof. Rustam Al-Shahi Salman, Centre for Clinical Brain Sciences, University of Edinburgh,

Chancellor's Building, 49 Little France Crescent. Edinburgh. EH16 4SB. UK. Phone: +44

(0)1314659602. Fax: +44 (0)1315372944. Email: Rustam.Al-Shahi@ed.ac.uk

Cover title

Characteristics of RCTs focusing on ICH

Tables and figures

Tables 4; Figures 2.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

1

Indexing terms

Randomized Controlled Trial; Meta-analysis; Intracerebral Hemorrhage; Bias; Publication

Bias; Sample Size.

Subject terms

Cerebrovascular Disease/Stroke; Intracranial Hemorrhage; Meta Analysis.

Word Count

4664

26

27

28

29

30

31

32

33

34

2

ABSTRACT

Background and purpose: The absence of treatments for intracerebral hemorrhage (ICH)

with significant consistent benefit in randomized controlled trials (RCTs) could be due to

lack of treatment efficacy or the design of RCTs.

Methods: We searched the Cochrane Stroke Group Trials Register in December 2015 for

completed and published RCTs reporting clinical outcome in adults with ICH. We collected

data on publication year and language, study characteristics, and effect size. We regarded

RCTs to be at lower risk of bias if they performed 2 of describing randomization, using

blinding, or specifying the primary outcome. We registered this systematic review

(PROSPERO CRD42016051103).

Results: We found 136 eligible RCTs: 57% were phase II, 76% were single-center, 98%

studied acute treatments, 49% involved drug interventions, 24% were placebo-controlled, the

primary outcome was death or disability in 30%, and median sample size was 77 (inter-

quartile range 47-160). 46% explained randomization, 24% blinded treatment allocation, and

24% specified the primary outcome such that 38 (28%) were at lower risk of bias. RCTs at

lower risk of bias were more likely to use multicenter recruitment (adjusted odds ratio [OR]

6.95, 95% CI 2.2-21.5) and be published in English (adjusted OR 12.9, 95% CI 2.7-62.5).

RCTs with larger sample sizes were independently more likely to be phase III/IV (p<0.01)

and use multicenter recruitment (p<0.01). RCTs at lower risk of bias had smaller pooled

treatment effects on death/disability (p=0.02).

Conclusions: ICH RCTs have often been at high risk of bias, and these RCTs have been

characterized by small sample sizes and larger effect sizes.

1

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

3

INTRODUCTION

Half of the adults affected by stroke due to intracerebral hemorrhage (ICH) are dead within

one year1 and two-thirds of survivors are dependent on others.2 Sadly no interventions for

acute ICH have shown significant consistent benefit in randomized controlled trials (RCTs),3,

4 which could be due to the lack of treatment efficacy or due to RCT design.

Large sample sizes are needed to detect small, but meaningful treatment effects and to

conduct relevant subgroup analyses. However, the rarity and severity of ICH limit

recruitment. Increasing numbers of eligibility criteria reduce recruitment to ICH RCTs,5, 6 but

other determinants of ICH RCT sample size are unknown.

In general, RCTs lacking adequate sequence generation,7, 8 double-blinding,7-9 or specification

of the primary outcome10 can systematically overestimate (bias) effect sizes.7 These

characteristics have not been uncommon among RCTs in acute stroke.11-13 However, the risk

of bias and association with effect size in ICH RCTs is unknown.

Therefore, we systematically searched for ICH RCTs to summarize their characteristics and

investigate independent associations with their sample sizes and effect sizes.

METHODS

This systematic review was prospectively registered in PROSPERO (CRD42016051103) and

is reported according to the PRISMA Statement. No ethical approval was required. We

2

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

4

declare that all supporting data are available within the article and its online supplementary

files.

Eligibility criteria

We included all completed RCTs of interventions in adults who suffered from spontaneous

ICH reporting clinical outcomes (e.g. mortality, disability, or impairment). We included

RCTs of all stroke types, from which data on ICH alone were extractable. We did not apply

time or language restrictions. We excluded studies that were 'quasi-randomized' or not

randomized, and RCTs that did not report clinical outcomes.

Information sources

On 15th December 2015, the Cochrane Stroke Group Managing Editor searched the Cochrane

Stroke Group’s Trial Register for all RCTs coded with the term 'hemorrhagic stroke’ (this

search can be replicated via an online search of the register at http://www.askdoris.org). We

also reviewed authors' personal collections to identify additional RCTs.

Study selection

One reviewer (AT) screened all titles and abstracts retrieved from the search for eligibility.

Another reviewer (MTCP) assessed eligibility of titles and abstracts only available in

Chinese. Potentially eligible RCTs were read in full to determine eligibility, and uncertainties

were resolved by discussion with another reviewer (RA-SS). When a single RCT resulted in

multiple publications, we selected the primary publication for data collection. On 5th July

3

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

5

2016 we checked Medline to see if RCTs, which were ongoing at the time of the initial

search, had been completed and published.

Data collection & items

We developed and piloted a data collection spreadsheet using a subset of eligible RCTs. Two

reviewers used the spreadsheet to extract data from English and Chinese RCTs (AT and

MTCP respectively). A third reviewer (RA-SS) resolved uncertainties or disagreements by

discussion. We collected data on RCT characteristics including: year of publication,

language of publication, phase of trial (II and III/IV, as there were no randomized phase I

trials), sample size, recruitment duration, recruitment region (single center, one-country

multicenter, and multi-country multicenter), type of study (acute treatment, rehabilitation, or

prevention), type of intervention (drug(s), surgery, new system of care, rehabilitation,

traditional Chinese medicine, traditional Chinese procedure, and others), type of control

comparator (placebo, standard of care, active comparator), and time to the last follow-up.

Risk of bias assessment

We collected information on whether RCTs explained the method of randomization, used

blinding (none, single, or double), or specified their primary outcome measure and time-

point. We classified RCTs as being at higher risk of bias if they fulfilled 2 of these criteria:

method of randomization not explained, blinding not used, or primary outcome not specified.

Summary measures

4

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

6

In order to investigate associations with effect sizes, we calculated odds ratios (ORs) for

unfavorable outcome in all included two-arm RCTs. We excluded RCTs if they had more

than two arms or if they did not report sufficient data for the OR calculation. Because some

RCTs used multiple outcome measures without a specified primary outcome, we pre-

specified a hierarchical system for selecting an outcome to analyze, in the following

descending order of preference: death or disability (e.g. modified Rankin Score [mRS],

Glasgow Outcome Scale [GOS-E]); death; disability (e.g. modified Barthel Index); or

impairment (e.g. National Institutes of Health Stroke Scale, Glasgow Coma Scale, or another

neurological deficit scale). If multiple outcomes at one level in the hierarchy were available,

preference was given to the outcome collected after the longest follow-up. In RCTs that

primarily aimed to prevent adverse events (e.g. seizures or deep vein thrombosis), we used

the number of participants experiencing adverse events to calculate the OR. If the outcome of

interest was ordinal, we dichotomized the data for OR calculation. If the outcome of interest

was continuous, we converted the standardized mean difference and its corresponding

standard deviation to OR using an online calculator

(http://www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-Home.php).

Statistical analysis

We compared the characteristics of RCTs using chi-squared tests (or Fisher’s exact test if

covariates had cell counts <5) for nominal independent variables and non-parametric Mann-

Whitney tests for continuous independent variables. We investigated associations of RCTs’

characteristics with risk of bias using multiple logistic regression, and with the sample size

using multiple linear regression. We log-transformed sample size in the linear regression

because of a skewed distribution. We included all RCTs with complete data for each analysis,

5

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

7

so the total number of RCTs varies in each analysis. We pre-specified the inclusion of

recruitment duration in the multivariable analysis of associations with sample size. We used

DerSimonian and Laird random effects models to pool ORs in higher vs. lower risk of bias

RCTs and Wald tests to compare the pooled estimates. We assessed publication bias using a

funnel plot, and plot asymmetry using the Egger test. We performed statistical analyses using

StatsDirect statistical software V2.7.2 and STATA 13.0 (StataCorp). We set a statistical

significance threshold of p<0.05.

RESULTS

Search results

We screened 1,920 records, identified 159 potentially eligible RCTs, and included 136 RCTs

in analyses of associations with risk of bias, sample size, and language of publication (Online

Supplementary Figure http://stroke.ahajournals.org). Of the 136 included RCTs, eight had

multiple treatment arms and three had insufficient data for OR calculation, so 125 RCTs were

available for analyses of associations with effect size.

General characteristics

Of the 136 RCTs published 1961-2016 (Table 1), 80% were published in 2000 or later and

54% were published in English. Most RCTs were phase II (57%) and recruited at a single

center (76%) over a median of two years for a median sample size of 77 participants. 98% of

RCTs evaluated acute treatments for ICH (49% of which were drugs) and most (60%)

compared them to standard care, with a primary outcome measured at a median 90 days that

6

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

8

was death or disability in only 30% of RCTs.

Risk of bias

46% of RCTs described the method of randomization, 24% used some form of blinding, and

24% specified their primary outcome. 38 (28%) of RCTs had at least two of these

characteristics and were classified as being at lower risk of bias. RCTs at lower risk of bias

were significantly more likely to be published in English, published recently, phase III/IV,

multicenter, larger in sample size, placebo-controlled, and use death or disability as the

primary outcome (Table 1). RCTs published in English shared similar associations (Online

Supplementary Table I) because few RCTs at lower risk of bias were published in languages

other than English. All of the 14 multicenter, multi-country RCTs14-27 were at lower risk of

bias.

Characteristics associated with lower risk of bias

We included 136 RCTs in multiple logistic regression analyses, in which English language of

publication and multicenter recruitment were the only characteristics independently

associated with RCTs at lower risk of bias (Table 2).

Characteristics associated with larger sample sizes

ICH RCT sample sizes have increased over time (Figure 1). Only three RCTs have recruited

≥1000 participants16, 22, 25 and seven RCTs have recruited 500-999 participants.14, 18-20, 23, 24, 28 All of

these large RCTs were multicenter and all of them – with one exception28 – were international

7

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

9

collaborations published in English. We included 92 RCTs in multiple linear regression

analyses (after excluding 44 that did not report recruitment duration), in which phase III/IV

and multicenter design were the only characteristics independently associated with a larger

sample size (Table 3). These RCT attributes have become more common over time,

accounting for the apparent secular trend in sample size (Figure 1). Multiple linear regression

models stratified by the phase of the trial confirmed the association of multicenter

recruitment with a larger sample size in phase III/IV but not phase II trials (Online

Supplementary Tables II and III).

Association between RCT quality and effect size

125 RCTs reported clinical outcomes that could be converted to ORs permitting investigation

of association between risk of bias and pooled effect size, stratified by type of outcome or

intervention (Table 4). The 36 (29%) RCTs at lower risk of bias had significantly smaller

pooled effect sizes than RCTs at the high risk of bias, in the two most commonly used

outcome categories (death or disability, and neurological impairment) and regardless of the

type of intervention (Table 4). The results were similar when stratified by the phase of the

trial, with RCTs at a lower risk of bias reporting smaller pooled effect sizes across a range of

outcomes and intervention types (Online Supplementary Tables IV and V).

Small study effects

We assessed small study effects in the 63 RCTs of drugs in which we could estimate the OR,

as this was the largest group of RCTs with the same type of intervention and a similar

number of RCTs in English or another language (Online Supplementary Table I

8

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

10

http://stroke.ahajournals.org). We found evidence of funnel plot asymmetry (Egger test

p<0.01), seemingly driven by RCTs published in languages other than English (Figure 2).

DISCUSSION

In this systematic review of 136 ICH RCTs published over a 55-year period, we found that

only just over one quarter of RCTs were at lower risk of bias. RCTs at lower risk of bias were

more likely to use multicenter recruitment and to be published in English. The median sample

size of ICH RCTs has been only 77; sample size seems to have increased over time, but the

only independent associations with larger sample sizes were phase III/IV RCTs with

multicenter recruitment, and these RCTs have increased in frequency over time. RCTs at

lower risk of bias reported smaller treatment effects. Small phase II RCTs at low risk of bias

with promising treatment effects can be useful; readers may find them in online

supplementary table VI, and may wish to consider confirming the effects of these

interventions in multicenter phase III RCTs.

The strengths of this review are that it included all relevant RCTs in a comprehensive trial

register, without time or language restrictions, and we were able to extract information on a

broad range of RCT characteristics. However, our study has some limitations. Restriction of

our search strategy to ‘hemorrhagic stroke’ RCTs in the Cochrane Stroke Group Trials

Register focused our search on RCTs that were dedicated to ICH, thereby omitting RCTs of

all types of stroke (which may have also included ICH). We did not undertake a full risk of

bias assessment, but focused on three domains that correspond to trial quality29 and have been

used by other similar reviews.7, 11, 12, 30

9

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

11

In previous studies of RCTs of ischemic stroke or acute stroke in general, 74% used

blinding12 and 57% defined the primary outcome.13 Indirect comparison with our findings

suggests that the risk of bias of ICH RCTs has been higher than for stroke in general. The

median ICH RCT sample size is similar to median sample sizes reported for ischemic stroke

RCTs (n=73)12 and acute stroke trials in general (n=80).11 While almost half of the ischemic

stroke trials in the 20th century used multicenter recruitment,12 paradoxically this was used by

only a quarter of RCTs focused on the less frequent ICH subtype of stroke. Others found that

multicenter acute stroke RCTs were of higher quality11 and we have confirmed this for ICH.

Our finding that multicenter RCTs are more likely to achieve larger sample sizes underscores

the many advantages of multicenter multinational RCTs recognized by others.31

We found that 4/62 (6%) RCTs published in languages other than English were at lower risk

of bias, which is slightly better than a systematic review of 9,061 acute stroke RCTs

published in Mainland China, where only 6% used adequate randomization, 2.2% used

blinding and 0.2% defined the primary outcome.30 Limited endorsement of CONSORT

Statement in Chinese neurological journals30 may account for higher risk of bias in Chinese-

language ICH RCTs.

In conclusion, we have found that many ICH RCTs have been small or at risk of bias. Future

research could explore methods to encourage investigators to reduce bias and encourage

collaborators to increase recruitment in ICH RCTs. Small RCTs may miss modest treatment

effects, whilst RCTs at higher risk of bias may overestimate treatment effects. Reducing the

risk of bias and increasing sample size using multicenter recruitment will help future ICH

RCTs; support to do this from funders and regulators would maximize the likelihood of RCTs

finally detecting unequivocal true effect of treatments for ICH.32

10

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

12

ACKNOWLEDGEMENTS

We are very grateful to Hazel Fraser, the Managing Editor of the Cochrane Stroke Group,

who carried out the database search and helped us obtain full manuscripts of selected articles.

SOURCES OF FUNDING

None.

DISCLOSURES

RA-SS received a special project grant from the British Heart Foundation paid to the

University of Edinburgh for the REstart or STop Antithrombotics Randomised Trial.

11

285

286

287

288

289

290

291

292

293

294

295

296

13

REFERENCES

1. van Asch CJ, Luitse MJ, Rinkel GJ, van der Tweel I, Algra A, Klijn CJ. Incidence, case fatality, and functional outcome of intracerebral haemorrhage over time, according to age, sex, and ethnic origin: a systematic review and meta-analysis. The Lancet Neurology. 2010;9(2):167-76.2. Poon MTC, Fonville AF, Salman RA-S. Long-term prognosis after intracerebral haemorrhage: systematic review and meta-analysis. J Neurol Neurosurg Psychiatry. 2013:jnnp-2013-306476.3. Steiner T, Al‐Shahi Salman R, Beer R, Christensen H, Cordonnier C, Csiba L, et al. European Stroke Organisation (ESO) guidelines for the management of spontaneous intracerebral hemorrhage. International journal of stroke. 2014;9(7):840-55.4. Hemphill JC, Greenberg SM, Anderson CS, Becker K, Bendok BR, Cushman M, et al. Guidelines for the management of spontaneous intracerebral hemorrhage. Stroke. 2015;46(7):2032-60.5. Fonville AF, Samarasekera N, Hutchison A, Perry D, Roos YB, Salman RA-S. Eligibility for Randomized Trials of Treatments Specifically for Intracerebral Hemorrhage. Stroke. 2013:STROKEAHA. 113.001493.6. Hansen BM, Ullman N, Norrving B, Hanley DF, Lindgren A. Applicability of clinical trials in an unselected cohort of patients with intracerebral hemorrhage. Stroke. 2016;47(10):2634-7.7. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of internal medicine. 2001;135(11):982-9.8. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of internal medicine. 2012;157(6):429-38.9. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Jama. 1995;273(5):408-12.10. Chan A-W, Krleža-Jerić K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. Canadian Medical Association Journal. 2004;171(7):735-40.11. Bath FJ, Owen VE, Bath PM. Quality of full and final publications reporting acute stroke trials. Stroke. 1998;29(10):2203-10.12. Kidwell CS, Liebeskind DS, Starkman S, Saver JL. Trends in acute ischemic stroke trials through the 20th century. Stroke. 2001;32(6):1349-59.13. Duncan PW, Jorgensen HS, Wade DT. Outcome measures in acute stroke trials. Stroke. 2000;31(6):1429-38.14. Chapman N, Huxley R, Anderson C, Bousser M, Chalmers J, Colman S, et al. Effects of a Perindopril-Based Blood Pressure–Lowering Regimen on the Risk of Recurrent Stroke According to Stroke Subtype and Medical History The PROGRESS Trial. Stroke. 2004;35(1):116-21.15. Mayer SA, Brun NC, Broderick J, Davis S, Diringer MN, Skolnick BE, et al. Safety and feasibility of recombinant Factor VIIa for acute intracerebral hemorrhage. Stroke. 2005;36(1):74-9.16. Mendelow AD, Gregson BA, Fernandes HM, Murray GD, Teasdale GM, Hope DT, et al. Early surgery versus initial conservative treatment in patients with spontaneous supratentorial intracerebral haematomas in the International Surgical Trial in Intracerebral Haemorrhage (STICH): a randomised trial. Lancet. 2005;365(9457):387-97.

12

297

298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345

14

17. Mayer SA, Brun NC, Begtrup K, Broderick J, Davis S, Diringer MN, et al. Recombinant activated Factor VII for acute intracerebral hemorrhage. New England Journal of Medicine. 2005;352(8):777-85.18. Haley EC, Thompson JLP, Levin B, Davis S, Lees KR, Pittman JG, et al. Gavestinel does not improve outcome after acute intracerebral hemorrhage. An analysis from the GAIN International and GAIN Americas studies. Stroke. 2005;36:1006-10.19. Lyden PD, Shuaib A, Lees KR, Davalos A, Davis SM, Diener HC, et al. Safety and tolerability of NXY-059 for acute intracerebral hemorrhage. The CHANT trial. Stroke. 2007;38(8):2262-9.20. Mayer SA, Brun NC, Begtrup K, Broderick J, Davis S, Diringer MN, et al. Efficacy and safety of recombinant activated Factor VII for acute intracerebral hemorrhage. New England Journal of Medicine. 2008;358(20):2127-37.21. Anderson CS, Huang Y, Wang JG, Arima H, Neal B, Peng B, et al. Intensive blood pressure reduction in acute cerebral haemorrhage trial (INTERACT): a randomised pilot trial. Lancet Neurology. 2008;7(5):391-9.22. Anderson CS, Heeley E, Huang Y, Wang J, Stapf C, Delcourt C, et al. Rapid blood-pressure lowering in patients with acute intracerebral hemorrhage. New England Journal of Medicine. 2013;368(25):2355-65.23. Mendelow AD, Gregson BA, Rowan EN, Murray GD, Gholkar A, Mitchell PM, et al. Early surgery versus initial conservative treatment in patients with spontaneous supratentorial lobar intracerebral haematomas (STICH II): a randomised trial. Lancet. 2013;382(9890):397-408.24. Krishnan K, Woodhouse L, Scutt P, Berge E, Sprigg N, Dineen RA, et al. Glyceryl trinitrate for acute intracerebral hemorrhage: results from the efficacy of nitric oxide in stroke (ENOS) trial. International Journal of Stroke. 2015;10(Suppl 2):84.25. Qureshi AI, Palesch YY, Barsan WG, Hanley DF, Hsu CY, Martin RL, et al. Intensive Blood-Pressure Lowering in Patients with Acute Cerebral Hemorrhage. New England Journal of Medicine. 2016.26. Baharoglu MI, Cordonnier C, Salman RA-S, de Gans K, Koopman MM, Brand A, et al. Platelet transfusion versus standard care after acute stroke due to spontaneous cerebral haemorrhage associated with antiplatelet therapy (PATCH): a randomised, open-label, phase 3 trial. The Lancet. 2016.27. Steiner T, Poli S, Griebe M, Hüsing J, Hajda J, Freiberger A, et al. Fresh frozen plasma versus prothrombin complex concentrate in patients with intracranial haemorrhage related to vitamin K antagonists (INCH): a randomised trial. The Lancet Neurology. 2016;15(6):566-73.28. Cheng XC, Wu JS, Zhou XP, et al. The randomized multicentric prospective controlled trial in the standardized treatment of hypertensive intracerebral hematomas: the comparison of surgical therapeutic outcomes with conservative therapy. Chinese Journal of Clinical Neurosciences. 2001;9(4):365-8.29. Moher D, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? The Lancet. 1998;352(9128):609-13.30. He S, Wu S, Zeng Q, Zhang S, Lin S, Zhang C, et al. Assessment of methodological quality and outcome measures of acute stroke randomized controlled trials in China in recent 15 years. Journal of evidence-based medicine. 2012;5(3):174-82.31. Djurisic S, Rath A, Gaber S, Garattini S, Bertele V, Ngwabyt S-N, et al. Barriers to the conduct of randomised clinical trials within all disease areas. Trials. 2017;18(1):360.32. Berge E, Salman RA-S, van der Worp HB, Stapf C, Sandercock P, Sprigg N, et al. Increasing value and reducing waste in stroke research. The Lancet Neurology.

13

346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395

15

2017;16(5):399-408.

14

396397

16

FIGURE LEGENDS

Figure 1. Scatter plot of sample size (on a logarithmic scale) versus year of publication for

136 RCTs, stratified by language of publication and distribution of recruiting sites

Figure 2. Funnel plot of 63 RCTs of drug interventions, stratified by language of publication

15

398

399

400

401

17

TABLE 1. Characteristics of 136 included RCTs and comparison of trials at higher

versus lower risk of biasAll 136 RCTs (%)

Higher risk of biasRCTs (n=98)

Lower risk of biasRCTs (n=38)

Statistical significance of higher vs. lower risk of bias RCTs

Randomization explained <0.01Yes 62 (45.6) 27 (27.6) 35 (92.1)No 74 (54.4) 71 (72.5) 3 (7.9)

Blinding <0.01Blinded (single or double) 33 (24.3) 89 (90.8) 14 (36.8)Un-blinded (open) 103 (75.7) 9 (9.2) 24 (63.2)

Primary outcome specified <0.01Yes 33 (24.3) 4 (4.1) 29 (76.3)No 103 (75.7) 94 (95.9) 9 (23.8)

Language of publication <0.01English 74 (54.4) 40 (40.8) 34 (89.5)Other* 62 (45.6) 58 (59.2) 4 (10.5)

Phase of trial 0.03Phase II 77 (56.6) 61 (62.2) 16 (42.1)Phase III or IV 59 (43.4) 37 (37.8) 22 (57.9)

Year of publication <0.01Before 1994 17 (12.5) 16 (94.1) 1 (5.9)

1995-2004 48 (32.3) 45 (93.8) 3 (6.3)2005-2016 71 (52.2) 41 (57.8) 30 (42.3)

Distribution of recruiting sites <0.01Single center 103 (75.7) 90 (87.4) 13 (12.6)Multicenter 33 (24.3) 12 (36.4) 21 (63.6)

Recruitment duration (months)Median (IQR) 24.5 (17-37) 24 (15-36) 25 (20-39) 0.21Trials specified recruitment 92 (67.6) 63 (64.3) 29 (76.3)Unspecified 44 (32.4) 35 (35.7) 9 (23.7)

Follow-up duration (days)Median (IQR) 90 (21-90) 28 (20-90) 90 (90-90) <0.01Trials specified follow-up 127 (93.4) 89 (90.8) 100 (100)Unspecified 9 (6.6) 9 (8.2) 0 (0)

Sample sizeMedian (IQR) 77 (47-160) 72 (47-125) 134.5 (42-404) 0.04

Type of intervention 0.19Acute treatment 133 (97.8) 97 (99.0) 36 (94.7)Rehabilitation 2 (1.5) 1 (1.0) 1 (2.6)Secondary prevention 1 (0.7) 0 (0) 1 (2.6)

Intervention tested 0.22Drugs 67 (49.3) 44 (44.9) 23 (60.5)Surgery 24 (17.6) 20 (20.4) 4 (10.5)New system of care 11 (8.1) 4 (4.1) 7 (18.4)Rehabilitation 3 (2.2) 2 (2.0) 1 (2.6)TCM drug 16 (11.8) 14 (14.3) 2 (5.3)TCM procedure 7 (5.1) 7 (7.1) 0 (0)Others 8 (5.9) 7 (7.14) 1 (2.6)

Comparator <0.01Placebo 33 (24.3) 12 (12.2) 21 (55.3)Standard care 81 (59.6) 66 (6.4) 15 (39.5)Active comparator 22 (16.2) 20 (20.4) 2 (5.3)

Outcome category† <0.01Death or disability 37 (29.6) 18 (20.2) 19 (52.8)Death 29 (23.2) 21 (23.6) 8 (22.2)Disability 11 (8.8) 10 (11.2) 1 (2.78)Neurological impairment 34 (27.2) 30 (33.7) 4 (11.1)Others 14 (11.2) 10 (11.2) 4 (11.1)

*61 trials were published in Chinese and 1 in Russian. † Based on 125 trials that reported an outcome of interest.

16

402

403

18

TABLE 2. Multiple logistic regression analysis of 136 included RCTs, investigating

multivariable, adjusted associations with trials at lower risk of bias

RCTs Number of RCTs at lower risk of

bias (%)

Multiple logistic regression

Odds ratio 95% CI

Language of publication

English 74 32 (43.2) 12.9 2.7-62.5

Other 62 2 (3.2) 1.00 -

Year of publication

Before 1994 17 1 (5.9%) 1.00 -

1995-2004 48 3 (6.3%) 3.87 0.3-47.7

2005-2016 71 30 (42.3) 17.6 1.9-160

Phase of trial

Phase II 77 16 (20.8) 1.00 -

Phase III/IV 59 18 (30.5) 0.65 0.2-1.9

Distribution of recruiting sites

Single center 103 13 (12.6) 1.00 -

Multicenter 33 21 (63.6) 6.95 2.2-21.5

Multiple logistic regression model adjusted for language of publication, year of publication, phase of trial, and the distribution of recruiting sites.

1

404

405

406407

19

TABLE 3. Simple and multiple linear regression models of the associations between the characteristics of 136 RCTs and log (sample size)

RCTs Simple linear regression Multiple linear regression*Coefficient 95% CI Coefficient 95% CI

Phase of trialPhase II 77 ref - ref -Phase III/IV 59 0.91 0.61-1.20 0.85 0.50-1.19

Distribution of recruiting sitesSingle 103 ref - ref -Multicenter 33 1.12 0.79-1.46 0.71 0.26-1.15

RandomizationNot explained 74 ref - ref -Explained 62 0.52 0.20-0.84 0.16 -0.21-0.52

BlindingNot double-blinded 112 ref - ref -Double-blinded 24 -0.09 -0.53-0.35 -0.10 -0.53-0.33

Primary outcome definedNo 103 ref - ref -Yes 33 0.64 0.26-1.01 -0.07 -0.57-0.42

Log (duration of recruitment)Each unit increase 92 0.34 0.02-0.65 0.12 -0.16-0.40

Year of publicationEach year 136 0.03 0.01-0.04 0.004 -0.02-0.03

* Multiple linear regression based on 92 trials that reported the duration of recruitment.

2

408

409

20

TABLE 4. Univariate associations between the risk of bias of 125 RCTs and pooled

estimates of intervention effects

RCTs Pooled OR 95% CI Inconsistency (I2)

Type of outcome Death or disability

All RCTs 37 0.75 0.52-0.98 95.8%Higher risk of bias 18 0.45 0.23-0.67 75.0%Lower risk of bias 19 0.88 0.78-0.98 34.4%

DeathAll RCTs 29 0.41 0.30-0.52 0%Higher risk of bias 21 0.48 0.29-0.68 10.7%Lower risk of bias 8 0.39 0.23-0.54 0%

Disability All RCTs 11 0.09 0.03-0.15 72.4%Higher risk of bias 10 0.08 0.02-0.15 74.0%Lower risk of bias 1 - - -

Neurological impairmentAll RCTs 34 0.12 0.09-0.16 72.3%Higher risk of bias 30 0.09 0.05-0.12 64.3%Lower risk of bias 4 0.36 0.10-0.62 75.0%

Other outcomesAll RCTs 14 0.26 0.11-0.40 22.6%Higher risk of bias 10 0.15 0.02-0.29 0%Lower risk of bias 4 0.42 0.22-0.62 0%

Type of interventionDrug

All RCTs 63 0.32 0.26-0.38 89.6%Higher risk of bias 42 0.10 0.06-0.13 63.6%Lower risk of bias 21 0.65 0.43-0.88 86.6%

SurgicalAll RCTs 22 0.31 0.22-0.39 87.0%Higher risk of bias 18 0.14 0.07-0.21 78.1%Lower risk of bias 4 0.83 0.64-1.02 4.0%

OtherAll RCTs 40 0.38 0.28-0.47 87.6%Higher risk of bias 29 0.16 0.10-0.22 51.2%Lower risk of bias 11 0.69 0.50-0.87 60.9%

3

410

411

412

21

FIGURE 1

Sample size is plotted on a logarithmic scale because of its skewed distribution. Legend refers to language of

publication and distribution of recruiting sites. The trend line represents the regression line of sample size on

year of publication.

4

413

414

415

416

417

22

FIGURE 2

5

418

419

23