emqn and eqa in molecular genetic testing

Quality in the World ofMolecular Genetics

David Barton, EMQNNational Centre for Medical GeneticsDublin, Ireland

Quality Assurance

Quality Assurance Test

ValidationTest

Validation

AccreditationAccreditation

EQAEQA

Education and

Training

Education and

Training

Best Practice

Guidelines

Best Practice

Guidelines

Reference MaterialsReference Materials

Consistently high standard of laboratory output

Quality Assured Genetic Testing

IQAIQA

Molecular Genetics Testing

• Mostly yes / no answers• TAT 3 to 40 days• Scientists give the clinical interpretation of results• Permanent medical record

Current Molecular Genetics Testing

Mendelian disorders

Molecular pathology

Chromosomalimbalance

• Cancer• Neurological disorders• Developmental delay• Dysmorphology• Etc.

KRAS, GIST Non-invasive prenatal diagnosis

Future directions

Free fetal DNA analysis

Pharmacogenetics

Next generation sequencingmulti-gene disorders

Quality Assurance


ValidationTest

Validation


EQAEQA

Education and

Training

Education and

TrainingBest Practice GuidelinesBest Practice Guidelines



IQAIQA

Best practice

• Sequencing• Single nucleotide polymorphisms• Unclassified variants – pathogenic or not?

Best practice

• Sequencing– Uni – or bi-directional?– Genotyping

• Confirmation or exclusion of known sequence variant

– Mutation scanning• Full characterisation of a region of

DNA

http://www.cmgs.org/BPGs/Best_Practice_Guidelines.htm

Best practice

• Sequencing– Quality parameters not

standardized – Negative results

http://www.cmgs.org/BPGs/Best_Practice_Guidelines.htm

Best practice

SNP check

Case:– HNPCC family– Father affected – known mutation– Test result was negative for the son

– Need to check if there are any known SNPs under primer binding sites - SNPcheck

EQA genotyping results

Genotyping errors per case Genotyping errors per allele

No. ofcases

completed

No. oferrors

Errorrate

(%)

No. ofalleles

analysed

No. oferrors

ErrorRate (%)

Case 1 45 0 0.0 90 0 0.0

Case 2 47 10 21.2 94 10 10.6

Case 3 46 2 4.3 92 2 2.2

Total 138 12 8.7 276 12 4.35

Diagnostic error rate

Re-design primers

• Move primer binding site

• Use wobble primers

Best practice

• Unclassified variants – Mutation or normal variant?

• Guidelines published• Bioinformatics tools

– Training/experience/caution required– Very time-consuming

• Novel approaches– Alamut– Bayesian Classification (Tavtigian, BRCA)

Best practice

• Unclassified variants – DMuDB – Diagnostic Mutation Database

Now Open to All!

Quality Assurance

Quality Assurance Test ValidationTest Validation


EQAEQA

Education and

Training

Education and

Training

Best Practice

Guidelines

Best Practice

Guidelines




IQAIQA

Test validation

• What is a new test?

• Most tests used are ‘homebrews’

• Lack of reference material– Panel for FRAX 2008– Panel for PWS/AS– Panel for BCR-ABL Q-PCR

Quality Assurance


ValidationTest

Validation


Education and

Training

Education and

Training

Best Practice

Guidelines

Best Practice

Guidelines




IQAIQA

EQAEQA

• Manchester based international EQA organiser• Network started in 1997 (EU funding) • A management group of 9 scientists from 7 countries

• 52 scheme organisers and assessors from 11 countries• 586 registered members• 1400 participations

ManagementGroup

ExecutiveAdministrator

ProjectManager

QualityManager

SchemeOrganisers

SchemeAssessors

EMQN – network at a glance

EQA schemes: Offered

NLCNX26

GERVHL

GERCAH

NORPKUPKUPKUPKU

DKDMDM

MSCAN

MODY

SMA

WIL#

SCA

DNA-SEQ

HNPCC

HFE

FRAX

PW/AS

RB

AZF+

CF *CMT

BRCA

FRDA

DMD

HD

07

Men2

MSCAN

MonoDiab

SMA

WIL#

SCA

DNA-SEQ

HNPCC

HFE

FRAX

PW/AS

RB

AZF+

CF *CMT

BRCA

FRDA

DMD

HD

08

GER

UK

UKMODY

SMA

WIL#

SCA

DNA-SEQ

HNPCC

HFE

FRAX

PW/AS

RB

AZF+

CF *CMT

BRCA

FRDA

DMD

HD

06

TUR

UK

UKDNA-SEQDNA-SEQ

SCA

HNPCC

HFE

FRAX

PW/AS

RB

AZF+

CF *CMT

BRCA

FRDA

DMD

HD

04/05

FRAHNPCC

POR

UKHFE

FRAFRAX

FRAPW/AS

GERRB

GERAZF+AZF+

BELCF *CF *GERCMTCMT

GERBRCABRCA

IRLFRDAFRDAFRDA

UK / NLDMDDMDDMD

NLHDHDHD

Provider01/02/0399/0097/98

*European Thematic Network for Cystic Fibrosis

+ European Academy of Andrology

^ European PorphyriaInitiative

# EuroWilson Group

Scheme participation

286234283732SCA

+1673573200SMA

+20200000VHL

023232200WIL

913612712211495SEQ

831

0

11

47

11

14

12

0

51

60

59

33

79

41

0

0

0

42

70

97

2006

981

41

14

49

16

17

13

0

63

61

57

34

75

46

31

0

0

50

77

96

2007

+7675558HFE

-53600MSCAN

+33824744PWAS

+446 (+45%)1427688623Total:

0141010RB

-11590POR

-215911PKU

+72000MonoDiab

+444400Men2

+18814942HNPCC

+30874539HD

+18523326FRDA

+271025554FRAX

+12584233DMD

+427300DM

+303000CNX26

+383800CAH

+12733737CMT

+341015851BRCA

+291258891AZF

Difference200820052004Scheme

International schemes

Types of schemes

• Single gene disorder schemes• Technique specific schemes

– Qualitative– Quantitative

EQA

GeneticTesting

Sample technical examination

Report content -helpful for genetic

counseling ?

Interpretation and reporting

Sample reception

Disease specific EQA schemes

• Participants receive 3 samples once a year with mock clinical referral

• Laboratories carry out analysis and report findings• Assessment for genotyping, interpretation and

reporting• Participants receive their marks and a final report

Scheme assessment

Swiss Society for Medical Genetics http://www.ssgm.ch/sections/Documents/Statements/publications.htm

• Genotyping – correct?

• Reporting– Accuracy and consistency– Clear ‘take home’ message– Methods referenced– Authorised / audit trail

• Interpretation– Can be scored– Language can be accommodated– Reporting policies differ

Claudia CAPABLANCA (dob 21/08/1976) is the youngest of 6 sisters. Two of her older sisters and her grand-mother all died before menopause from breast or ovarian cancers. Her mother had died inan accident at age 49. Mrs. Capablanca’s gynaecologist recently suspected an ovarian cancer and referred her to an oncology centre. There, the diagnosis was confirmed and treatment initiated. Because of the high genetic risk for her nieces, Mrs. Capablanca decided to have a genetic screening done.

Please analyse exon 18 of the BRCA1 gene. Report back to the oncology centre in your standard reporting format.

EMQN BRCA scheme case 2007

EQA evaluation of results

Mutation affects 5’-splice site of exon 18. Listed once in BIC as ‘clinically relevant’. Most likely compatible with a hereditary predisposition to breast/ovarian cancer.Patient remains at high risk for secondary tumours and should join a monitoring programme. Predictive testing of relatives is possible after genetic counselling.Deductions: missing/incomplete biological interpretation; lack of suggestion of monitoring programme (0.4 marks); lack of suggestion of predictive testing and/or counselling (0.2 marks).

BIC-nomenclature: c.5194-2A>C; IVS17-2A>CComments

2.0Personal data of patient (name & DOB or name & lab no)Brief recapitulation of the patient’s personal & family historyClear presentation of the results Minor points (not leading to deduction of marks):Arrival and reporting dates,Signature of the report by two authorised persons,Indication of the reference sequence used.Deductions: (0.2 marks each)

Interpretation and reporting

2.0Genotype correct, deduction of marks only if genotype description is misleading.Genotyping

MarksCriteriaName

U14680.1: c.5075-2A>C 21/08/1976FEMALEClaudia CAPABLANCA

ResultsDate of BirthSexName

Technique specific EQA

• EQUAL – qual– EQA for DNA extraction and PCR

• EQUAL – quant– EQA for Real Time PCR

• EMQN schemes– Sequencing– Mutation scanning– CNVs/array CGH

EQUAL - qual

• EQA for DNA extraction and PCR– Participants received blood, primers and

DNA– Labs asked to extract DNA and set up PCR

reactions– Submit DNA concentration, quality,

quantity data as well as aliquots of DNA and PCR products

Orlando et al (2007) Clinical Chemistry 53:7, 1349-1357

EQUAL – qual results

• Assessment:– Look at data returned by participants– Reevaluation of returned material by

reference laboratory– Assessment of photometric measurements

of DNA quantity / quality– Performance of blood extraction procedure

EQUAL - qual

• 25 % of laboratories (42/165) gave out of limits readings for pre-extracted DNA

EQUAL - qualPCR from DNA extracted by the participants

PCR from DNA extracted by the organiser

• High variability of PCR efficiency• Differences in sizes• Contamination issues

EQUAL-quant

• EQA testing Real Time PCR– Used ABL proto oncogene– Participants given:

• Primers and fluorescent probes• Plasmid standards (10, 102, 103, 104, 105 copies/5μl)

• 3 unknown test samples (cloned cDNAs)– Laboratories asked to:

• Construct a calibration curve• Estimate cDNA copy numbers

Ramsden et al (2006) Clinical Chemistry 52:8, 1584-1591

EQUAL-quant• 93 labs returned results and 74 met performance criteria

Generic Technical EQA schemes

• DNA Sequencing– 6 years : 2002 - 2008

DNA Sequencing EQA scheme

• Materials – Unpurified PCR products

• CFTR, BRCA, OCRL-1 genes• Normal control, Normal, Heterozygote,

Homozygote & a deletion Heterozygote– Sequencing primers provided

• Validation:– 2 independent laboratories

Assessment of Genotyping & Interpretation

SAMPLE GENOTYPE SCORE HGVS INTERPRETATION ACCEPTABLE VARIATIONS REF SEQ

SEQ07_01 Mutation presentc.[733G>A] + [=] 2.00

c.[733G>A] + [=] c.[733G>A]c.733G>A NM_000276.3

p.[Gly245Arg] + [=]

p.[Gly245Arg]p.Gly245Argp.[G245R]p. G245R

NM_000276.3

SEQ07_02 Mutation presentc.[729dupT] 2.00

c.[729dupT] None NM_000276.3

p.[Val244CysfsX13]

p.[V244CfsX13]+p.[V244CfsX13]p.[Val244CysfsX13]p.[V244CfsX13]p.[Val244fs]p.[V244fs]p.Val244fsp.V244fs

NM_000276.3

SEQ07_03 Mutation absentc.[=] + [=] 2.00

c.[=] + [=]

c.[=] c.=Wild typeWT

NM_000276.3

p.[=] + [=] p.[=]p.= NM_000276.3

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

277

059

146

180

151

162

219

404

190

222

279

250

330

004

108

024

161

015

049

034

365

194

044

204

262

245

361

218

366

030

317

320

073

008

229

038

109

132

037

257

101

153

406

200

116

478

253

359

213

168

398

343

195

216

333

278

228

090

418

374

149

172

005

254

139

138

337

412

241

076

428

290

441

175

094

422

210

438

425

012

039

(Q)

332

270

(Q)

018

082

225

(Q)

183

(Q)

054

(Q)

431

(Q)

193

(Q)

160

(Q)

056

437

041

002

280

318

(Q)

244

014

075

(Q)

083

Perc

enta

ge

Lab ID

PHRED %>=20Base-calling confidence

Quality of raw data

• 5 different parameters– PHRED scores (20,30,40), Quality Read Length

(QRL), Quality Read Overlap (QRO)

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

180

049

320

039

(Q)

151

222

015

024

219

190

250

404

146

277

034

225

(Q)

194

108

262

330

054

(Q)

149

438

279

162

270

(Q)

183

(Q)

044

139

161

245

361

193

(Q)

153

365

138

333

030

317

218

431

(Q)

253

005

425

204

038

004

037

116

229

160

(Q)

008

195

290

200

366

109

073

343

337

257

478

101

132

241

441

076

254

359

082

168

216

418

428

213

374

172

090

412

012

278

228

398

059

280

422

210

175

406

094

018

318

(Q)

056

002

244

332

437

041

006

069

Perc

enta

ge

Lab ID

% possible QRODiagnostic data quality (QRO)

Benchmarking data quality

70.00%

75.00%

80.00%

85.00%

90.00%

95.00%

100.00%

2005 2006 2007

Mea

n %

QRO

Year

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

005

024

041

108

151

250

253

262

320

330

374

056

015

049

044

162

244

318

161

213

190

039

138

204

270

245

241

034

365

116

225

172

361

366

257

337

030

193

004

076

101

160

075

194

222

229

069

228

317

332

037

183

280

290

441

008

109

333

094

153

278

279

359

478

425

038

090

195

132

254

269

404

168

175

240

412

139

022

012

437

219

018

438

275

398

146

149

HIGH PERFORMANCE RANKING LOW

HIGH TECHN

ICALQU

ALITY (% Q

RO) LO

W

Benchmarking DNA sequence qualityPatton et al., Clinical Chemistry. 2006;52:728-736

Learning from EQA?2007 scheme scores

01.91.9434SMA

31.811.9455HD

31.81.9360HNPCC

21.821.9660HFE

11.781.9634FRDA

31.881.9473FRAX

71.831.8745DMD

31.911.9550CMT

31.91.9248BRCA

No of errors leading to misdiagnosis

Av. interpretation score

Av. genotyping score

No of reportsScheme

45(5.7%)

1.78(max 2.00)

1.91(max 2.00)

788Total

Error rate = number of genotyping errors over all returns

Common types of errors

• Wrong name / date of birth• Sample mix up

– 10 incidences in 2008• Incorrect genotype• Incorrect interpretation• Incorrect nomenclature

Summary

• Best practice issues– What quality needed for reporting– SNP– Unclassified variants

• EQA– Single disease specific schemes

• Issues specific for disorder (nomenclature etc)• Reporting and interpretation also assessed

– Technique specific EQA– Errors are still being made

Acknowledgments

• Rob Elles, Simon Patton, Simon Ramsden• EMQN management team• EMQN scheme organisers and assessors• Participating laboratories

• EuroGentest