the use of next-generation sequencing for the quality

A c A d em i c S u b j e c t S / m e d 0 0 8 6 0

1920 • jid 2020:222 (1 december) • Charlton et al

The Journal of Infectious Diseases

The Use of Next-Generation Sequencing for the Quality Control of Live-Attenuated Polio VaccinesBethany Charlton,1 Jason Hockley,2 Majid Laassri,3 Thomas Wilton,1 Laura Crawt,1 Mark Preston,4 NGS Study Group, Peter Rigsby,2 Konstantin Chumakov,3 and Javier Martin1

1Division of Virology, National Institute for Biological Standards and Control, Potters Bar, United Kingdom, 2Division of Biostatistics, National Institute for Biological Standards and Control, Potters Bar, United Kingdom, 3US Food and Drug Administration, Silver Spring, Maryland, USA, and 4Division of Bioinformatics, National Institute for Biological Standards and Control, Potters Bar, United Kingdom

Background. Next-generation sequencing (NGS) analysis was compared to the current MAPREC (mutational analysis by poly-merase chain reaction and restriction enzyme cleavage) assay for quality control of live-attenuated oral polio vaccine (OPV).

Methods. MAPREC measures reversion of the main OPV attenuating mutations such as uracil (U) to cytosine (C) at nucleotide 472 in the 5′ noncoding region of type 3 OPV. Eleven type 3 OPV samples were analyzed by 8 laboratories using their in-house NGS method.

Results. Intraassay, intralaboratory, and interlaboratory variability of NGS 472-C estimates across samples and laboratories were very low, leading to excellent agreement between laboratories. A high degree of correlation between %472-C results by MAPREC and NGS was observed in all laboratories (Pearson correlation coefficient r = 0.996). NGS estimates of sequences at nucleotide 2493 with known polymorphism among type 3 OPV lots also produced low assay variability and excellent between-laboratory agreement.

Conclusions. The high consistency of NGS data demonstrates that NGS analysis can be used as high-resolution test alternative to MAPREC, producing whole-genome profiles to evaluate OPV production consistency, possibly eliminating the need for tests in animals. This would be very beneficial for the quality assessment of next-generation polio vaccines and, eventually, for other live-attenuated viral vaccines.

Keywords. oral polio vaccine; neurovirulence; next-generation sequencing; vaccine safety; vaccine quality control; MAPREC.

Oral polio vaccine (OPV) has been the preferred vaccine throughout the World Health Organization (WHO) Global Poliovirus Eradication Initiative. However, Sabin poliovirus strains in OPV are genetically unstable and have been shown to lose their attenuating mutations and revert to a neurovirulent phenotype during passage in vivo and in vitro [1, 2]. It is there-fore essential that maintenance of the attenuation phenotype of OPV strains is carefully monitored during vaccine production. Tests have been available for this purpose since OPV was first released for use in humans. Increased neurovirulence of OPV is measurable by the monkey neurovirulence test (MNVT) and the neurovirulence test using transgenic mice expressing the human poliovirus receptor (TgmNVT) [3, 4].

In addition, molecular quantitative assays measuring the pro-portion of revertants in OPV are used to monitor the safety and genetic consistency of vaccine batches. In type 3 OPV, excessive

reversion at nucleotide 472 in the 5′ noncoding region from U (attenuated) to C (wild type) is associated with an increase in neurovirulence and makes the vaccine unacceptable for im-munization [5]. The level of 472-U to 472-C reversion in vac-cine lots can be quantified using the mutational analysis by polymerase chain reaction and restriction enzyme cleavage (MAPREC) assay [6]. The percentage content of 472-C directly correlates with neurovirulence in the MNVT [7]. However, the MAPREC assay is technically challenging and requires specific fluorescently labelled primers or radioisotopes as well as the use of restriction enzymes. Performing the MAPREC assay is time consuming and laborious and calls for the acquisition and spe-cific training of staff. These factors make MAPREC expensive and difficult to upkeep. Consequently, few laboratories can con-duct the assay.

In the advent of modern high-throughput, massively parallel deep (next-generation) sequencing (NGS), it begs the ques-tion whether these techniques might be suitable for measuring the content of mutants during vaccine manufacture of live-attenuated viral vaccines such as OPV. There is early evidence that NGS can accurately measure the 472-C content of type 3 OPV lots and that it could be an alternative to the MAPREC test [8, 9]. In addition, NGS has the potential to monitor the genetic stability and production consistency of OPV beyond nucle-otide 472 to identify other mutations potentially determining

Received 20 March 2020; editorial decision 22 May 2020; accepted 28 May 2020; published online June 3, 2020.

Correspondence: Javier Martin, PhD, National Institute for Biological Standards and Control, Blanche Lane, Potters Bar, Hertfordshire EN6 3QG, UK ([email protected]).

The Journal of Infectious Diseases® 2020;222:1920–7© Crown copyright 2020.This article contains public sector information licensed under the Open Government Licence v3.0 (http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/).DOI: 10.1093/infdis/jiaa299

Dow

nloaded from https://academ

ic.oup.com/jid/article/222/11/1920/5850979 by guest on 07 M

ay 2021

mailto:[email protected]?subject=

http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/

NGS for Quality Control of Polio Vaccine • jid 2020:222 (1 december) • 1921

changes in phenotype. For example, type 3 OPVs vary in their nucleotide sequence at nucleotide 2493, showing different rela-tive proportions of C and U that results in a Thr-Ile polymor-phism at amino acid VP1-6. Mutation from C to U at nucleotide 2493 can accumulate during vaccine production but does not affect clinical safety, although it appears to slightly increase neurovirulence in the TgmNVT.

The National Institute for Biological Standards and Control (NIBSC), UK and the Centre for Biologics Evaluation and Research, Food and Drug Administration, United States conducted an international collaborative study involving several national control laboratories (NCLs) and vaccine manufacturers to assess the utility of NGS for moni-toring molecular consistency of OPV. This study evaluated NGS as an alternative to MAPREC for measuring the 472-C content of type 3 OPV seeds and vaccine lots. The ability of NGS to measure the 2493-U content in such vaccines was also assessed.

METHODS

Study Samples

Poliovirus type 3 OPV bulk suspensions (n = 11) were chosen from historical samples held at NIBSC according to their known tests results obtained in MAPREC, MNVT, and TgmNVT. Each study sample is referred to by a code letter from A to K assigned at random to each sample. Samples 3B and 3I represent du-plicate samples. Full details of study samples are given in the Supplementary Material and Supplementary Table 1.

Participants

Eight laboratories participated in the study, 4 from manu-facturers and 4 from NCLs (Supplementary Table 2). Each participating laboratory is referred to by a code number as-signed at random and not necessarily representing the order of listing.

MAPREC Assay

The MAPREC assay was conducted using established methods [6]. Details are given in the Supplementary Material.

NGS Assays

Details of NGS methods used by the different laboratories are described in the Supplementary Material and summarized in Supplementary Table 3. They include details on RNA extrac-tion, polymerase chain reaction (PCR) and NGS strategies, as well as discussion on cost considerations between MAPREC and NGS. One of the limiting factors of NGS when it was first suggested as a replacement for MAPREC was the high overall cost of NGS approaches. That cost has come down significantly since then and NGS platforms have expanded, which may now make it much more feasible. Laboratories conducted 5 inde-pendent NGS determinations for each sample (starting from

viral RNA). All but 1 laboratory amplified the viral RNA by PCR and all laboratories used Illumina sequencing systems.

Estimation of Background NGS Error Frequency

A plasmid containing the whole-genome sequence of Sabin 3 polio-virus under the control of T7 RNA polymerase promotor was used for these experiments. This plasmid was sequenced directly on the Illumina MiSeq along with PV whole-genome PCR products gener-ated from it, from in vitro transcribed RNA, and from viral RNA ex-tracted from cDNA-derived Sabin 3 poliovirus. Details of methods used are given in the Supplementary Material.

Bioinformatics Analysis

Most participants performed bioinformatics analyses in-house using prepackaged software applications (Supplementary Table 3). Four of the participants used the CLC Genomic Workbench application (Qiagen) and 1 laboratory used Geneious R7.1.9 (Biomatters, https://www.geneious.com) (Biomatters). Three laboratories performed bioinformatics analyses using na-tive command-line–based tools. NIBSC reanalyzed all raw FASTQ sequence files using native command line programs (Supplementary Material).

Statistical Analysis

Pearson correlation coefficients were calculated to measure the linear correlation between NGS and MAPREC results for each laboratory and overall using mean 472-C NGS and MAPREC es-timates. Deming regression, a linear regression analysis that ac-counts for the measurement error on both axes (ie, in both NGS and MAPREC results), was used to model a linear relationship between NGS (NIBSC analysis) and MAPREC results for each individual laboratory and overall using the study mean NGS re-sults. Variance ratios were calculated using the intralaboratory variance components, which were pooled across laboratories for the overall study comparison. The model residuals and co-efficients of the slope, and y-axis intercept were calculated from the Deming regression model. Equivalence between 2 methods can be concluded by demonstrating the slope and intercept are equal to 1 and 0, respectively.

RESULTS

Data Received

In-house processed NGS 472-C data were received from all 8 participating laboratories, who also provided raw FASTQ sequencing files for all 5 determinations of all samples. Details of data received from the different participating laboratories are given in the Supplementary Material.

Determination of 472-C Content by MAPREC

MAPREC results of all study samples expressed as % of 472-C are shown in Table 1. The 472-C estimates were in line with pre-vious results for these samples obtained at NIBSC and in WHO collaborative studies [10].

Dow



ay 2021

http://academic.oup.com/jid/article-lookup/doi/10.1093/infdis/jiaa299#supplementary-data









https://www.geneious.com




Measurement of NGS Background Error Frequency

A Sabin 3 poliovirus recombinant plasmid and whole-genome poliovirus PCR products from this plasmid and from in vitro transcribed Sabin 3 RNA were sequenced with Illumina MiSeq to determine the background error mutation at nucleotide 472-C generated at each of the different steps required for re-verse transcription PCR (RT-PCR) amplification. In addition, a cDNA-derived virus from this plasmid that had undergone a single cell passage and therefore was expected to contain very few molecules with the 472-C mutation was included in the study as a low-mutation virus control (sample 3K). As shown in Figure 1, values of 472-C content for all these 4 DNA prod-ucts were very low (<0.1%). In agreement with this, MAPREC

results and NGS values for sample 3K in all laboratories showed mean estimates 472-C of 0.09% and 0.07% (NIBSC calcula-tions), respectively (Table 1 and Table 2). It was concluded that NGS had adequate sensitivity for estimating 472-C content in vaccines in the range required for making pass/fail decisions comparable to that of the MAPREC test.

Determination of 472-C Content by NGS

NGS estimates of 472-C content in study samples obtained in each laboratory are shown in Figure 2 and Supplementary Table 4. Mean values for each study sample are shown in Table 2. No differences in mean NGS estimates or levels of variability were detected when comparing in-house and NIBSC calcula-tions. Sequence coverage varied widely between laboratories and samples (Supplementary Figure 1) but did not have an ap-preciable impact on mean or standard deviation (SD) of NGS values across laboratories and samples (Table 2). The bioinfor-matics analysis applied at NIBSC resulted in lower sequence coverage, likely due to different quality threshold settings for the inclusion/exclusion of sequencing reads during trimming. The percentage of total filtered reads mapping to Sabin 3 po-liovirus sequences also varied between laboratories with la-boratories 6 and 8 showing the lowest proportions (78.7% and 68.7%, respectively, versus between 90.3% and 97.6% in the rest of laboratories). This is expected as laboratory 6 sequenced total RNA and laboratory 8 used random primers to generate PCR products for sequencing as compared to the rest of laboratories using poliovirus-specific primers.

Intralaboratory, Interlaboratory, and Intraassay Variability of NGS 472-C

Determinations

The intralaboratory (between-assay, within-laboratory) var-iability reported as the SD of 472-C estimates from 5 deter-minations across study samples for each laboratory is shown in Table 2. Pooled SDs from each sample and all samples to-gether across each laboratory are summarized in Table 2 and Supplementary Table 5, respectively. The results revealed very

Table 1. MAPREC (%472-C) Results Summary

Sample Mean SD Minimum Maximum Outcome

3K 0.09 0.09 0.00 0.22 Pass

3G 0.35 0.07 0.24 0.41 Pass

3A 0.50 0.11 0.38 0.63 Pass

3D 0.67 0.07 0.60 0.78 Pass

3E 0.75 0.12 0.60 0.89 Pass

3H 0.91 0.23 0.65 1.20 Pass

3I 1.14 0.14 1.06 1.39 Fail

3B 1.18 0.15 0.98 1.33 Fail

3F 1.56 0.22 1.33 1.82 Fail

3J 1.64 0.12 1.55 1.84 Fail

3C 3.21 0.21 2.94 3.42 Fail

Abbreviation: MAPREC, mutational analysis by polymerase chain reaction and restriction enzyme cleavage.

Samples are listed in descending order according to their mean value.

0.0

0.1

0.2

0.3

DNA Control Sample

% 4

72-C

by

NG

S

Plasmid

PCR-amplfi

ed Plas

mid

RT-PCR fr

om R

NA

Recov

ered V

irus

Figure 1. Estimates of NGS background error at nucleotide 472. Levels of re-version at nucleotide 472 in different Sabin 3 poliovirus DNA products measured by NGS are shown as mean %472-C content of 5 replicates ± SD. Plasmid DNA, poliovirus whole-genome PCR product from plasmid, and poliovirus whole-genome RT-PCR products from in vitro synthesized RNA or cDNA-derived viral RNA were analyzed. Abbreviations: NGS, next-generation sequencing; RT-PCR, reverse tran-scription polymerase chain reaction.

Dow



ay 2021






low variability across samples and laboratories (SD < 0.2%, NIBSC calculations). MAPREC variability (with results from a single laboratory) was generally higher than the intralaboratory variability observed by NGS, but not for all samples.

Between or interlaboratory variability was estimated as the SD of 472-C mean values in all laboratories (Table 2). Again, variability was very low and comparable for each sample across laboratories; SD < 0.22% for all samples except sample C which had higher 472-C content and for which the extent of variability was in line with that of the other samples if measured relative to the mean (coefficient of variation, %CV). Between-laboratory variability of NGS 472-C estimates for the low mutant virus ref-erence (LMVR) and high mutant virus reference (HMVR) in this study were lower than that observed using MAPREC in a previous collaborative study (Supplementary Table 6).

The ratio of mean 472-C estimates for duplicate samples 3B and 3I was used to test within-assay variability and they showed good agreement with their expected value of 1 for all samples (92.5% of values were within 0.75 to 1.25).

Correlation between MAPREC and NGS 472-C Measurements

A high degree of correlation between 472-C results measured by MAPREC and NGS was observed with Pearson correlation coefficients ranging between r = 0.979 and r = 0.997 across in-dividual laboratories and r = 0.996 overall (Figure 3). However,

Deming regression analysis established that concordance in results was not observed, as shown by the slope value of 0.89 being significantly less than 1.00 and therefore indicating a sig-nificant proportional difference between NGS and MAPREC data (Supplementary Table 7). This conclusion is based on the overall results, shown in Figure 4; however, significance of the slope and/or intercept values was not consistent across the dif-ferent laboratories (Supplementary Table 7). The main outcome of this comparison is still that a high degree of correlation was observed between MAPREC and NGS results, which means the reported differences are proportional and predictable; using a similar analysis approach to that designed for MAPREC could therefore result in the same pass or fail decision for test vaccines using NGS. This observation was further supported by com-paring study samples ranked by mean 472-C values. As shown in Supplementary Table 8, despite the very small differences in 472-C content between some study samples, the ranking order of study samples established by NGS were as expected from the MAPREC results in all laboratories with only 1 exception where sample 3H analyzed by laboratory 2 was ranked higher than samples 3B and 3I (which are duplicates of the same vac-cine sample). This result was not entirely unexpected as samples 3H, 3B, and 3I had very similar 472-C content: 0.90, 0.99, and 1.02% by NGS (NIBSC calculations) versus 0.91, 1.18and 1.14% by MAPREC, respectively. As mentioned above, 3B and 3I are

Table 2. NGS (%472-C) Results Summary

Analysis Sample No. of Labs Mean Minimum Maximum

Variance Components (as SD)

Intralab Interlab Total

In-house 3K 7 0.08 0.03 0.18 0.01 0.06 0.06

In-house 3G 7 0.24 0.18 0.36 0.05 0.06 0.08

In-house 3A 8 0.28 0.18 0.41 0.04 0.08 0.09

In-house 3D 8 0.40 0.26 0.53 0.06 0.08 0.11

In-house 3E 8 0.63 0.46 0.86 0.16 0.11 0.20

In-house 3H 8 0.87 0.68 1.09 0.10 0.14 0.17

In-house 3I 8 0.98 0.70 1.38 0.09 0.21 0.22

In-house 3B 8 0.98 0.73 1.20 0.10 0.16 0.19

In-house 3F 8 1.30 0.99 1.66 0.17 0.22 0.28

In-house 3J 8 1.33 1.05 1.69 0.13 0.22 0.25

In-house 3C 8 2.80 2.40 3.58 0.23 0.40 0.47

NIBSC 3K 8 0.07 0.02 0.23 0.02 0.07 0.08

NIBSC 3G 7 0.26 0.18 0.41 0.07 0.08 0.10

NIBSC 3A 8 0.30 0.20 0.50 0.07 0.09 0.11

NIBSC 3D 8 0.45 0.34 0.67 0.10 0.11 0.15

NIBSC 3E 8 0.62 0.47 0.84 0.15 0.13 0.20

NIBSC 3H 8 0.90 0.73 1.07 0.14 0.13 0.19

NIBSC 3B 8 0.99 0.74 1.26 0.13 0.20 0.24

NIBSC 3I 8 1.02 0.76 1.26 0.11 0.17 0.21

NIBSC 3F 8 1.32 1.00 1.55 0.16 0.21 0.26

NIBSC 3J 8 1.33 1.08 1.62 0.16 0.19 0.25

NIBSC 3C 8 2.81 2.44 3.37 0.20 0.34 0.39

Abbreviations: NGS, next-generation sequencing; NIBSC, National Institute for Biological Standards and Control.


Dow



ay 2021






duplicate samples and as such ranked above or below each other in different laboratories as expected. Similarly, samples 3F and 3J showing almost identical 472-C values by both NGS (1.32 and 1.33%, respectively) and MAPREC (1.56 and 1.64%, respectively) ranked above or below each other in different la-boratories as expected.

Determination of 2493-U Content by NGS

NGS estimates of the 2493-U content in study samples obtained in each laboratory are shown in Table 3. As a wide range of 2493-U frequencies (between 3.28% and 99.92%) was found in study samples, we used %CV to estimate assay vari-ability. Consistent with the findings for nucleotide 472-C, the intralaboratory, interlaboratory, and intraassay variability was found to be very low, resulting in excellent agreement between laboratories in 2493-U estimates for all samples (CV < 3.6%). The ranking order of study samples by their NGS mean 2493-U estimates was identical in all laboratories with only 1 exception; sample 3E analyzed by laboratory 3 was ranked between du-plicate samples 3B and 3I (Supplementary Table 9). This result was not unexpected as 2493-U estimates for samples 3B, 3I, and

3E were very similar: 46.35%, 45.86%, and 43.43%, respectively. Again, 3B and 3I are duplicate samples ranked above or below each other in different laboratories as expected.

DISCUSSION

NGS 472-C estimates across all samples and laboratories tested in this collaborative study were highly consistent and compa-rable. This remarkable agreement between laboratories was observed regardless of the different methods used to generate RNA or DNA products for sequencing, sequencing protocols, or bioinformatics processing/analyses of NGS raw sequences. This is an important finding as it means that there will be no need to develop a fully standardized NGS protocol, allowing la-boratories to use their in-house methods and quickly adopt an NGS test for OPV.

A high degree of correlation between %472-C estimates by MAPREC and NGS was observed in all laboratories, with an overall Pearson correlation coefficient of r = 0.996. Deming re-gression analysis found that NGS and MAPREC results were not completely equivalent with a proportional difference across 472-C values. Finding differences in absolute measurements

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

3A 3B 3C 3D 3E 3F 3G 3H 3I 3J 3K

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

3A 3B 3C 3D 3E 3F 3G 3H 3I 3J 3K

Lab 1 Lab 2 Lab 3 Lab 4 Lab 5 Lab 6 Lab 7 Lab 8

Sample

Sample

% 4

72-C

by

NG

S (in

-hou

se)

% 4

72-C

by

NG

S (N

IBSC

)

A

B

Figure 2. NGS results per laboratory (%472-C). Mean NGS %472-C results by laboratory and sample using in-house (A) or NIBSC (B) calculations are shown. Results from different laboratories are indicated with different colors. Abbreviations: NGS, next-generation sequencing; NIBSC, National Institute for Biological Standards and Control.

Dow



ay 2021



obtained by MAPREC and NGS is not surprising as NGS and MAPREC are both complex and very different quantitative

techniques; the former quantifying nucleotide changes among thousands of sequence reads and the latter measuring gel band intensities of cleaved PCR products on a gel. This observation does not alter the overall conclusion that NGS can be used as an alternative to MAPREC as the observed differences are propor-tional and predictable. Similar differences in data concordance are known to occur between MAPREC results obtained at dif-ferent laboratories or by different operators. These observations are among the reasons why a detailed test format and analyt-ical process was established for MAPREC and why similar pro-cesses should be adopted for NGS assays.

As for MAPREC, the standardization and validation of NGS as a regulatory test will involve establishing criteria for the ac-ceptance or rejection of vaccines. This will involve considera-tion of adopting common statistical and analytical approaches and reference standards to apply acceptance criteria for 472 U to C reversion and for the future direction of using NGS for the whole-genome profiling of vaccines. Based on expe-rience with MAPREC and the results of this study, a process for establishing test validity and pass/fail decisions can be en-visaged. A test format similar to that used in MAPREC in-cluding reference samples used in this study (or similar) may be suitable. Pass and fail decisions for vaccines in future NGS tests should be based on appropriate validity criteria. These

MAPREC Mean

43A3B3C3D3E3F3G3H3I3J3K

3

2

1

0

0 1 2

NG

S M

ean

3 4

Figure 4. Deming regression results NGS (%472-C; NIBSC analysis) vs MAPREC. The line showing perfect concordance between the 2 data sets is shown in black. See Supplementary Table 7 for Deming regression data for slope and intercept cal-culations. Abbreviations: MAPREC, mutational analysis by polymerase chain reac-tion and restriction enzyme cleavage; NGS, next-generation sequencing; NIBSC, National Institute for Biological Standards and Control.

AnalysisPearson correlation coe�cient r by lab

1 2 3 4 5 6 7 8 Overall

In-House 0.997 0.979 0.994 0.994 0.990 0.995 0.995 0.992 0.996

NIBSC 0.996 0.984 0.995 0.993 0.996 0.996 0.991 0.993 0.996

3.50

3.00

2.50

2.00

NG

S re

sults

1.50

1.00

0.50

0.000.00 0.50 1.00 1.50 2.00

MAPREC results

2.50 3.00 3.50

Lab 1 Lab 2 Lab 3 Lab 4 Lab 5 Lab 6 Lab 7 Lab 8

Figure 3. Correlation between NGS (%472-C; NIBSC calculations) mean values and MAPREC results for each laboratory. Results from different laboratories are indicated with different colors. Pearson correlation coefficient r calculations are shown. Abbreviations: MAPREC, mutational analysis by polymerase chain reaction and restriction enzyme cleavage; NGS, next-generation sequencing; NIBSC, National Institute for Biological Standards and Control.

Dow



ay 2021



criteria should be easy to adopt from the existing MAPREC test. An important advantage of NGS over MAPREC is that current NGS technology allows multiplexing many samples in a single run, enabling multiple vaccines and all required ref-erences to be run concurrently, while only 2 different vaccines can be tested concurrently with MAPREC in each assay. Given the level of variability in NGS 472-C measurements observed in this study, using at least 5 independent NGS determinations per sample seems reasonable. However, each laboratory will have to determine the number of determinations to perform in a routine assay to achieve a suitable precision. Inclusion of a 0% 472-C (or close to 0%) control sample such as the cDNA-derived virus sample 3K used in this study is essential to monitor that background in 472-C levels due to PCR and sequencing errors does not exceed predetermined levels. This would reduce the chance of samples falsely failing the NGS test. As described in this study, NGS can accurately estimate 472-C reversion comparable to MAPREC. Some upper limit for acceptable variability should be set as for MAPREC, for ex-ample, measuring within-assay variability between duplicated samples to ensure precise measurements are taken during each test. This was shown to be feasible for NGS as judged by re-sults in this study, which included 2 duplicate samples 3B and 3I. Another check on test validity can be provided by com-paring 472-C results for the LMVR and HMVR. Consistency in LMVR/HMVR ratios between the 5 independent NGS de-terminations can be used as a criterion for acceptable assay accuracy. As an individual laboratory builds up experience of the NGS test, it would be possible to set up acceptable limits based on observed variability. Where insufficient previous ex-perience is available, a possible guide might be that any test with a standard deviation for the ratios between LMVR and HMVR of greater than 0.3 should be considered to have un-acceptably high variability. This figure is chosen based on the experience of the different laboratories in this study.

The next step in this collaborative effort will be to vali-date NGS as an alternative to MAPREC for OPV serotypes 1 and 2. In addition to measuring nucleotide variation by the MAPREC test, NGS can be used to analyze the sequence com-position at other nucleotide positions in the poliovirus genome. As for the 472-C NGS measurements discussed above, we have shown in this study that the intralaboratory, interlaboratory, and intraassay variability of NGS estimates of 2493-U for all samples were found to be very low, resulting in excellent agree-ment between laboratories. As a wide range of 2493-U muta-tion frequencies was observed among study samples, this result suggests that NGS analysis can be expanded to other relevant nucleotides beyond 472-C, eventually establishing whole-genome NGS profiles that can be used in batch release tests, possibly eliminating the need for animal testing completely. A whole-genome NGS test approach would be beneficial for the quality assessment of vaccine seeds and production bulks used for the manufacture of inactivated polio vaccine based on Sabin OPV strains. Furthermore, NGS as a test would be ideal for the molecular characterization of next-generation OPVs (new OPV or nOPV), which incorporate several genetically engineered mutations for which the MAPREC test would be meaningless. However, further validation would be required in parallel with animal studies to establish safe genetic profiles for vaccines, particularly for new vaccines for which relevant de-terminants for genetic/phenotypic reversion (if any) have not yet been fully assessed. This technology could be eventually adapted for other live-attenuated viral vaccines to assess pro-duction consistency and correlate whole-genome mutational profiles with vaccine safety.

Supplementary Data

Supplementary materials are available at The Journal of Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and

Table 3. NGS (%2493-U; NIBSC Analysis) Results Summary

Variance Components (as CV%)

Sample No. of Labs Mean Min Max Intralab Interlab Total

3G 5 3.28 3.03 3.56 14.63 0.00 14.63

3F 6 21.41 20.75 21.89 8.73 0.00 8.73

3H 6 25.14 24.48 25.98 2.03 2.07 2.90

3E 6 43.43 42.36 45.04 4.10 1.70 4.44

3I 6 45.86 41.67 47.15 6.00 3.64 7.02

3B 6 46.35 44.32 47.11 2.93 1.83 3.47

3A 6 71.28 70.35 72.34 2.30 0.00 2.30

3D 6 81.24 80.66 82.09 1.27 0.33 1.30

3J 6 93.04 92.51 93.58 0.42 0.33 0.54

3C 6 94.78 94.18 95.48 1.11 0.01 1.11

3K 6 99.92 99.85 100.00 0.03 0.05 0.06

Abbreviations: CV, coefficient of variation; NGS, next-generation sequencing; NIBSC, National Institute for Biological Standards and Control.


Dow



ay 2021


are the sole responsibility of the authors, so questions or com-ments should be addressed to the corresponding author.

Notes

Acknowledgments. We thank all study participants, particu-larly manufacturers that contributed with candidate and study materials.

NGS Study Group members. Jean-Pol Cassart, Ahmed Essaghir, Olivier Vandeputte, and Christophe Lambert (GSK, Belgium); Mathias Janssen and Lucas Preux (Sciensano, Belgium); Murielle Andre (ANSM, France); Eric Sarcey, Isabelle Perret, Fabrice Tindy, and Laurent Mallet (Sanofi Pasteur, France); Steffen Matthijn de Boer (Intravacc, Netherlands); Tomofumi Nakamura and Susumu Ochiai (Biken, Japan); Martin Fritzsche, Nadine Holmes, Manasi Majumdar, Edward Mee, and Begona Valdazo-Gonzalez (NIBSC, UK); Majid Laassri and Konstantin Chumakov (FDA, United States).

Potential conflicts of interest. All authors: No reported con-flicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References

1. Evans DM, Dunn G, Minor PD, et al. Increased neurovirulence associated with a single nucleotide change in a noncoding region of the Sabin type 3 poliovaccine ge-nome. Nature 1985; 314:548–50.

2. Chumakov KM, Dragunsky EM, Norwood LP, et al. Consistent selection of mutations in the 5′-untranslated region of oral poliovirus vaccine upon passaging in vitro. J Med Virol 1994; 42:79–85.

3. Chumakov KM, Powers LB, Noonan KE, Roninson IB, Levenbook IS. Correlation between amount of virus with altered nucleotide sequence and the monkey test for accept-ability of oral poliovirus vaccine. Proc Natl Acad Sci U S A 1991; 88:199–203.

4. Dragunsky E, Taffs R, Chernokhvostova Y, et al. A poliovirus-susceptible transgenic mouse model as a pos-sible replacement for the monkey neurovirulence test of oral poliovirus vaccine. Biologicals 1996; 24:77–86.

5. Westrop GD, Wareham KA, Evans DM, et al. Genetic basis of attenuation of the Sabin type 3 oral poliovirus vaccine. J Virol 1989; 63:1338–44.

6. Dunn G, Chumakov KM. Standard operating procedure: mutant analysis by PCR and restriction enzyme cleavage (MAPREC) for oral poliovirus (Sabin) vaccine types 1, 2 or 3. World Health Organization, 2011. https://www.who.int/biologicals/vaccines/MAPREC_SOP_Final_09112012.pdf. Accessed 4 June 2020.

7. Chumakov K, Norwood L, Parker M, et al. Assessment of the viral RNA sequence heterogeneity for control of OPV neurovirulence. Dev Biol Stand 1993; 78:79–89.

8. Neverov A, Chumakov K. Massively parallel sequencing for monitoring genetic consistency and quality control of live viral vaccines. Proc Natl Acad Sci U S A 2010; 107:20063–8.

9. Sarcey E, Serres A, Tindy F, et al. Quantifying low-frequency revertants in oral poliovirus vaccine using next generation sequencing. J Virol Methods 2017; 246:75–80.

10. Wood DJ, Heath AB, Kersten GF, Hazendonk T, Lantinga M, Beuvery EC. A new WHO international reference reagent for use in potency assays of inactivated poliomyelitis vac-cine. Biologicals 1997; 25:59–64.

Dow



ay 2021

https://www.who.int/biologicals/vaccines/MAPREC_SOP_Final_09112012.pdf

https://www.who.int/biologicals/vaccines/MAPREC_SOP_Final_09112012.pdf

the use of next-generation sequencing for the quality

Documents