computational immune monitoring reveals abnormal double ... · t cells were abnormal and...

Research Article

Computational Immune Monitoring RevealsAbnormal Double-Negative T Cells Presentacross Human Tumor TypesAllison R. Greenplate1,2, Daniel D. McClanahan2,3, Brian K. Oberholtzer3,Deon B. Doxie2,3, Caroline E. Roe2,3, Kirsten E. Diggins2,3, Nalin Leelatian2,3,Megan L. Rasmussen3, Mark C. Kelley4, Vivian Gama3,5, Peter J. Siska1,6,Jeffrey C. Rathmell1,2,7, P. Brent Ferrell2,4, Douglas B. Johnson2,4, andJonathan M. Irish1,2,3,7

Abstract

Advances in single-cell biology have enabled measurementsof >40 protein features on millions of immune cells withinclinical samples. However, the data analysis steps followingcell population identification are susceptible to bias, time-consuming, and challenging to compare across studies. Here,an ensemble of unsupervised tools was developed to evaluatefour essential types of immune cell information, incorporatechanges over time, and address diverse immune monitoringchallenges. The four complementary properties characterizedwere (i) systemic plasticity, (ii) change in population abun-dance, (iii) change in signature population features, and (iv)novelty of cellular phenotype. Three systems immune moni-

toring studies were selected to challenge this ensembleapproach. In serial biopsies of melanoma tumors undergoingtargeted therapy, the ensemble approach revealed enrichmentof double-negative (DN) T cells. Melanoma tumor-resident DNT cells were abnormal and phenotypically distinct from thosefound in nonmalignant lymphoid tissues, but similar to thosefound in glioblastoma and renal cell carcinoma. Overall,ensemble systems immune monitoring provided a robust,quantitative view of changes in both the system and cellsubsets, allowed for transparent review by human experts, andrevealed abnormal immune cells present across multiplehuman tumor types.

IntroductionThe immune system is a complex network of cells and tissue

types, and it is increasingly important to simultaneously track cellsubsets and understand the system as a whole. Longitudinalmonitoring of changes in the immune system has providedinsight into drug response and disease progression (1–3). Differ-ences in response to perturbation can stratify clinical outcome(4, 5) and indicate mechanism of action (6). Challenges to theimmune system, such as vaccination, infection, surgical interven-

tion, or the emergence of a malignancy, can elicit detectablechanges above the relatively stable basal state of each individual(7, 8). Cytomic approaches that can characterize all the cells in asystem, such ashigh-dimensionalflowandmass cytometry, are anarea of active development in immunology (9). A commonframework for data analysis will allow researchers using thesecytomic approaches to compare and contrast immune systemsfrom diverse research areas, including tumor immunology andtreatment response (4, 10, 11), blood cancer (12, 13), bonemarrow failure (14, 15), and human immune variation andautoimmunity (3, 7, 16).

Systematically monitoring the cells of the immune system andtheir features is especially powered when serial samples from anindividual can be used as comparison points. However, thisapproach generates vast amounts of data. Simultaneously track-ing entire systems and the parts that comprise them can becomeoverwhelming in the context of clinical research, where cells ofinterest can be rare (4, 10), and the entire system can changequickly in unanticipatedways (15). Thus, in cytomic, system-levelstudies, the data analysis strategy is as important as the experimentdesign (4, 17, 18), and it is vital to track known referencepopulations and place new observations in the context of priorknowledge (19, 20). Tools exist for visualizing and gating(21–24), supervised population and biomarker analysis(25, 26), and describing cell population identity (19) withinhigh-dimensional data sets. In contrast, a great need exists forautomated analysis for the steps immediately after populationidentification (gating). Tools for cellular clustering are especially

1Pathology, Microbiology and Immunology, Vanderbilt University MedicalCenter, Nashville, Tennessee. 2Vanderbilt-Ingram Cancer Center, VanderbiltUniversity Medical Center, Nashville, Tennessee. 3Department of Cell andDevelopmental Biology, Vanderbilt University School of Medicine, Nashville,Tennessee. 4Department of Medicine, Vanderbilt University Medical Center,Nashville, Tennessee. 5Vanderbilt Center for Stem Cell Biology, VanderbiltUniversity School of Medicine, Nashville, Tennessee. 6Department of InternalMedicine III, University Hospital Regensburg, Regensburg, Germany. 7VanderbiltCenter for Immunobiology, Vanderbilt University School of Medicine, Nashville,Tennessee.

Note: Supplementary data for this article are available at Cancer ImmunologyResearch Online (http://cancerimmunolres.aacrjournals.org/).

Corresponding Author: Jonathan M. Irish, Vanderbilt University, 2220 PierceAvenue, 740 Preston Research Building, Nashville, TN 37232-6840. Phone:615-875-0965; E-mail: [email protected]

doi: 10.1158/2326-6066.CIR-17-0692

�2018 American Association for Cancer Research.

CancerImmunologyResearch

Cancer Immunol Res; 7(1) January 201986

on November 13, 2020. © 2019 American Association for Cancer Research. cancerimmunolres.aacrjournals.org Downloaded from

Published OnlineFirst November 9, 2018; DOI: 10.1158/2326-6066.CIR-17-0692

http://crossmark.crossref.org/dialog/?doi=10.1158/2326-6066.CIR-17-0692&domain=pdf&date_stamp=2018-12-19

http://cancerimmunolres.aacrjournals.org/

numerous, specific for each data type, and have been extensivelyaddressed in prior work (27). This ensemble aims to quantifychanges in the whole system, prior to cellular clustering, and tocharacterize how cellular populations change after the appropri-ate clustering tool has been applied to the system.

The growth and success of high-dimensional single-cell tech-nologies relies on the ongoing development of data analysis toolsneeded to parse large amounts of data and place results intocontext (18–20). It remains relatively rare for researchers tochoose data analysis approaches that explicitly incorporate timeor other changes. Across longitudinal studies, four dynamic ele-ments stand out as key features of cellular systems: (i) plasticity orstability of the systemas awhole, (ii) changes in abundance of cellsubsets, (iii) emergenceof unexpected features on cell subsets, and(iv) emergence of novel or unexpected cell types. The central goalhere was to simultaneously use distinct tools focused on thesedifferent data types as an ensemble and to organize this copiousinformation into an automated "first pass" analysis that could beeasily interpretable by an immunologist and that would highlightcell subsets for in-depth review. The components selected for theensemble toolkit here included the EarthMover'sDistance (EMD)algorithm (28), t-distributed stochastic neighbor embedding(t-SNE; ref. 21), andMarker EnrichmentModeling (MEM; ref. 19).

To validate and challenge computational analysis tools, it isvaluable to explore immunology problems representing differinglevels of prior knowledge, amounts of change, and abundances oftarget cell types. Here, the first challenge (data set 1) compriseddata from melanoma patients undergoing anti–PD-1 therapywith pembrolizumab. Data set 1 was chosen for the relevance ofthis therapeutic strategy and because the data set includes anunusual case from a patient with a distinct, unexpected immunesystem trajectory (15). This example was chosen to include anoutlier case,which is unusual enough that trainingdata setswouldnot normally include an example of it. Data set 2 comprised of apreviously published data set of peripheral blood from acutemyeloid leukemia (AML) patients undergoing chemotherapy(12). Data set 2 was chosen as an example of dynamic cellularpopulations shifting dramatically over therapy. This data set wasalso included to represent the challenge of tracking and charac-terizing treatment-refractory leukemic blasts, which did not con-verge on a single phenotype and, instead, shifted into differentphenotypic compartments, none of which matched the pheno-type of healthy cells (12). Data set 3 included serial melanomatumor biopsies from patients treated with dabrafenib, a BRAFV600

inhibitor (BRAFi), and trametinib, a MEK inhibitor (MEKi). Achallenge of data set 3 was to applymultiple tools in a system thatwas relatively less well studied and included a diverse set ofindividuals and mass cytometry panels. By providing bothhigh-level and detailed views of cellular systems changing overtime in human patients, the ensemble approach revealed knowl-edge about immune system interactions in these three study typeswith contrasting changes and challenges.

Materials and MethodsStudy designData set 1. In the case of peripheral blood collected from mela-nomapatients receivingpembrolizumab, thepurpose of the studywas to identify biological characteristics of melanoma occurringprior to treatment and at different time points following therapyfor patients being treated with immune-based therapies. Each

patient gave consent to an institutional review board (IRB)–approved research protocol. To be included in this study, patientsmet the following inclusion criteria: (i) pathologically provendiagnosis of melanoma, (ii) 18 years of age or older, (iii) treatedwith immune-based therapies, and (iv) willing to have severalserial blood draws. Patients not receiving immune-based therapyor unwilling or unable to provide consent were not included.There was no blinding or randomization process in this study.Blood specimens were obtained from patients during the time ofscheduled phlebotomy for routine clinical laboratory analysis.Peripheral blood draws were done on the day of therapy start and21days (�10days), 84 days (�21days), and 180days (�21days)following initiation of therapy.

Data set 2. This cohort has been previously published andincludes peripheral blood from AML patients undergoing che-motherapy (12). Briefly, patients consented to a blood drawthrough an IRB-approved research protocol. Peripheral bloodwas drawn pretreatment at the time of diagnosis, every 2 to 3days within the first 2 weeks after the start of chemotherapy, at the2-week time point, and at the time of recovery, if applicable.

Data set 3. In the case of tumors sampled sequentially frommelanoma patients treated with targeted therapy, the objectivewas to identify biomarkers of response and resistance to B-RAF–andMEK-targeted therapy in melanoma. Patients with advanced,operable BRAF mutation–positive melanoma received GSK-2118436 (BRAF inhibitor) for 2 weeks, followed by the combi-nation of GSK-2118436 and GSK-1120212 (MEK inhibitor) for 2weeks, followed by surgical resection of the disease. Tumorbiopsies were obtained prior to start of therapy and 2 weeks aftercombined GSK-2118436 and GSK-1120212 (29). To be includedin this study, patients met the following inclusion criteria: (i)signed written, informed consent, (ii) between the ages of 18 and90, (iii) patients with locally or regionally advanced melanomabeing considered for resection of the lesion(s) for local–regionalcontrol and potential cure, (iv) BRAF V600 mutation–positive bysnapshotmolecular analysis, (v)measurable disease, (vi) all priortreatment-related toxicities CTCAE � grade 1 at the time ofenrollment, (vii) adequate baseline organ function, (viii) womenof childbearing potential with a negative serum pregnancy testwithin 14 days of the first dose of study or men with femalepartner of childbearing potential must have had either had a priorvasectomy or agree to use effective contraception, and (ix) able toswallow and retain oral medication. There was no blinding orrandomization process in this study.

Human tissue sample collection and preservationAll human samples were obtained in accordance with the

Declaration of Helsinki following protocols approved byVanderbilt University Medical Center IRB. The patient informa-tion for unpublished samples can be found in SupplementaryTable S1.Healthydonor tonsil, adenoid, andbloodwere collectedas "non-human subjects," without gender or age information.Upon single-cell isolation, all cells were cryopreserved in 88%fetal calf serumplus 12%DMSO.Cells fromhuman sampleswerecollected and isolated as follows.

Peripheral bloodPeripheral blood mononuclear cells (PBMC) were collected,

isolated, and cryopreserved from approximately 20 mL of freshly

Systems Analysis Reveals Abnormal DN T Cells across Cancers

www.aacrjournals.org Cancer Immunol Res; 7(1) January 2019 87




drawn blood as previously described (15). Briefly, peripheralbloodwas drawn into sodiumheparin anticoagulant, and PBMCswere isolated by centrifugation after layering on top of a Ficoll-Paque PLUS (GE Healthcare Bio-Sciences) gradient.

Solid tissueMelanoma tumors, glioblastoma tumors, and nonmalignant

human adenoid and tonsil tissue were resected from patientswith consent and in accordance with the Declaration of Helsinki.All solid tissue samples were dissociated into live, single-cellsuspensions, and samples were cryopreserved using a previouslydocumented protocol (30, 31). Solid tissue samples werefirst manually dissociated using a scalpel. The minced tissuewas then incubated in RPMI 1640 (Corning/Mediatech) plus10% FBS, collagenase II (1 mg/mL; Sigma-Aldrich), and DNase(0.25mg/mL; Sigma-Aldrich) for 1 hour in a 37�C incubator with5% CO2. Cells were then strained through a 70-mm and 40-mmfilter prior to cryopreservation.

Renal cell carcinoma samples were processed and stored asdescribed by Siska and colleagues (32). Briefly, malignant tissuewas removed from patients and processed by mechanical disso-ciation and enzymatic digestion. Single-cell suspensions wereachieved by passing dissociated tissue through a 70-mm strainer.Finally, red blood cells were lysed, and the single-cell suspensionwas cryopreserved.

Human-induced pluripotent stem cells (iPSC)Reprogramming of human neonatal foreskin fibroblast cells

(strain BJ; ATCC no. CRL2522) was induced by transduction withCytoTune Sendai virus (Life Technologies). All experiments wereperformed under the supervision of the Vanderbilt InstitutionalHuman Pluripotent Cell ResearchOversight Committee. Inducedpluripotent stem cells were grown in feeder-free conditions inplates coated with Matrigel (BD Biosciences) and maintained inmTESR1 media (Stem Cell Technologies) at 37�C with 5% CO2.iPSCs were generated in the lab of Vivian Gama, PhD and werepassaged 30 to 35 times prior to staining by mass cytometry. TheiPSCswerefirst characterizedby live stainingwithTra1-60or Tra1-81 antibodies that recognize undifferentiated iPSCs. Every 30passages, iPSCs were characterized by gene-expression analysisusing the TaqMan hPSC score card panel (Life Technologies) andkaryotyping. Cells were checked daily for differentiation andwerepassaged every 3 to 4 days using Gentle dissociation solution(Stem Cell Technologies). iPSCs were treated with 0.5% EDTAprior to staining with mass cytometry antibody panel describedbelow. iPSCs used in this study were mycoplasma negative.

Mass cytometryThawed samples were first incubated with cisplatin (25

mmol/L, Enzo Life Sciences). After incubation with cisplatin,cells were washed in PBS containing 1% BSA. Staining occurredin 50 mL PBS/1% BSA for 30 minutes at room temperature usingthe antibodies listed in Supplementary Table S2. Cells werethen washed twice with PBS/1% BSA and fixed with a finalconcentration of 1.6% paraformaldehyde (PFA, ElectronMicroscopy Sciences). Cells were washed again, using PBS, andthen resuspended in ice-cold methanol to permeabilize. Cellswere incubated at �20�C overnight before being washed twicein PBS and stained with iridium DNA intercalator (FluidigmSciences). Purified, carrier-free antibodies were purchased fromthe listed provider and labeled with the listed metal using the

protocol provided by Fluidigm. Stained samples were collectedat Vanderbilt University Flow Cytometry Shared Resource on aCyTOF 1.0 mass cytometer (Fluidigm Sciences). All events werenormalized prior to analysis using Fluidigm normalizationbeads.

CyTOF data preprocessingData (FCS files) were collected and stored in the online analysis

platform Cytobank. Data analysis was performed in Cytobankand statistical programming environment R (version 3.4.0) via RStudio.

EMDEMDwas calculatedbetween eachpair of populations using the

"transport" library for R (refs. 28, 33; https://cran.r-project.org/web/packages/transport/citation.html). The parent population(e.g., live CD45þ events) was gated in Cytobank, followed bythe creation of a viSNE map in Cytobank. A viSNE analysis withtwo output dimensions was performed, equally sampling 5,000events per file, with 1,000 iterations, perplexity equal to 30, andtheta equal to 0.5. The events with their viSNE axes were thendownloaded from Cytobank, and the EMD was calculatedbetween each pair of files using the "transport" library for R. The"wpp" object was used to represent each set of points in the twoviSNE axes, and the "wasserstein" functionwas called on each pairof point sets to produce a distancematrix. Each pointwas assignedunit weight.

Because calculating a matrix with the EMD between each set of5,000 events from the viSNE analysis is computationally expen-sive, four optimizations were performed. (i) Each file was furtherdown-sampled to 1,000 out of the original 5,000 events per file inthe viSNE analysis. Each event was still assigned unit weight, andeach point set, therefore, still had an equal totalmass of 1,000. (ii)The "shortsimplex" method was used for the "wasserstein" func-tion in the "transport" library, which accepted no other para-meters besides the pair of weighted point sets (34). (iii) Eachpopulation was automatically assigned a zero EMD comparedwith itself, and EMD scores already computed across the diagonalwere simply copied because EMD is a metric. (iv) The "parallel"library was used to parallelize the computation of each row of thematrix, in addition to the above, using the number of coresdetected from the "detectCores" function in the "parallel" library.

EMD values computed by "emdist" were compiled in a CSV fileand used to create a heat map, in R, for visualization. Statisticalcomparisons of EMD values between groups were done in Excelusing a Student t test. CSV file and heat map are each produced asan output.

Change in population equationThe frequency of immune populations was determined in

Cytobank and exported into CSV files prior to reorganization.For data sets 1 and 3, populations were identified by traditionalbiaxial gating. For data set 2, populations were identified byfirst running a viSNE on nucleic acid expressing events from allpatients at all time points and then running a SPADE on the t-SNEaxes. Fifteen nodes (15) were identified with 5% downsampling.The following equation was used to determine the change infrequency for all data sets where FREQt is equal to the frequencyof a population at a given time point and FREQpre is thefrequency of that same population prior to the start of therapy.The addition of 0.01 to both the numerator and the

Greenplate et al.

Cancer Immunol Res; 7(1) January 2019 Cancer Immunology Research88



https://cran.r-project.org/web/packages/transport/citation.html

https://cran.r-project.org/web/packages/transport/citation.html


denominator is to account for the appearance of new popula-tions over the course of therapy.

Change in frequency ¼ ln((FREQt þ 0.01)/(FREQpre þ 0.01))R was used to conduct a paired Student t test to compare

samples from the same patient at different time points of treat-ment. R script provided by Carr and colleagues was used to createboxplots in R (7). In the case of data set 1, a Bonferroni correctionwas used for multiple hypothesis testing.

MEMMEM creates a quantitative label of cell identity for given

populations (19), and the MEM equation is implemented in R.MEM labels were either created for the indicated populationsusing the bulk, nonpopulation as the reference, except, whereindicated, when iPSCs or hematopoietic stem cells were stainedand run on mass cytometry as a respective common reference(19).MedianMEMlabelswere created by taking themedianMEMscore of each marker for each population. Standard deviation isshown. DMEM scores are calculated by subtracting theMEM scoreof the pretherapy sample from the MEM score of the indicatedtime point.

Similarity of MEM labelsRoot-mean-square deviation (RMSD) and hierarchical

clustering were used to compare MEM labels, as previouslydescribed (19). The MEM vectors for each nonreferencepopulation were calculated over phenotype channels whichwere shared across all nonreference populations and thesingle reference population. Each MEM vector contained thepopulation's MEM score, calculated for each of the commonphenotype channels, in reference to the single referencepopulation. The MEM RMSD between pairs of nonreferencepopulations was then calculated using the Euclidean distancebetween these MEM vectors.

Heat maps representing population similarity were generatedfrom each distance matrix using the "heatmap.2" function of the"gplots" library for R. The distance matrix was normalized by themaximum nonnormalized distance d_max between any pair ofpopulations, then multiplied by 100, then subtracted from 100.The result was that zero entries in the original distance matrixwould receive a similarity score of 100, whereas the pair ofpopulations with greatest distance in the original distance matrixwould receive a similarity score of 0. Thus, two populations withthe exact same enrichment score would have 100% similarity(19). To compare populations, median RMSD scores were com-pared using a two-tailed, Student t test.

Code availabilityOriginal data sets were provided as FCS files in FlowRepository.

The software for calculating EMDanddisplaying it as a heatmap isavailable as Supplementary Software. The software for generatingMEM scores is available in the Supplementary Software byDigginsand colleagues (ref. 19; http://mem.vueinnovations.com/).

Data availabilityData set 1.Peripheral blood frommelanomapatients treatedwithanti–PD-1 and healthy peripheral blood controls is available asFCS files in Flow Repository. SNaPshot genotyping was done inthe clinic on tumors resected from each patient. Forty-eight

mutations in NRAS, BRAF, KIT, CTNNB1, and GNAQ weremonitored (35).

Data set 2. Peripheral blood from AML patients treated withchemotherapy was generated by CyTOF analysis as describedby Ferrell and colleagues (12) and is available as FCS files inFlow Repository (http://flowrepository.org/id/FR-FCM-ZZMC).Patient characteristics and treatment details are available in Ferrelland colleagues.

Data set 3. Serially biopsied melanoma tumors from patientstreated with BRAFi and MEKi were generated in separate masscytometry experiments. Patients MP-034, MP-029, MP-031, MP-032, MP-055, and MP-059 were stained with the mass cytometrypanel described in Supplementary Table S2. PatientsMP-019,MP-023, MP-054, MP-052, and MP-062 were stained with the paneldescribed by Doxie and colleagues (36). FCS files are available inFlow Repository. SNaPshot genotyping was done as describedabove.

Additional dataData for comparison across cancer types were generated by us in

separate mass cytometry studies characterizing untreated melano-ma tumors, glioblastoma tumors, and nonmalignant tonsil andadenoid. FCS files are available in Flow Repository. Renal cellcarcinoma tumors RC-29, RC-37, and RC-52 were published bySiska and colleagues (32), and Staphylococcal enterotoxin B(SEB)–stimulated PBMCs were published by Nicholas and collea-gues (37). Briefly, PBMCs were stimulated with SEB (final con-centration 1mg/mL in in200mL) at a cell concentration of 10� 106

cells/mL. Cells were incubated at 37�C for 16 hours before beingwashed twice with PBS and stained for mass cytometry.

Data sets used in the RMSD heat map are described by Digginsand colleagues (19).

ResultsApplication of EMD to characterize the peripheral immunesystem during therapy

We sought to investigate how the overall composition of circu-lating immune populations changes over the course of anti–PD-1therapy. To do this, we generated viSNE maps, which reduce thedimensionality of the data by plotting cells in two dimensions onthe basis of their high-dimensional similarity. Changes in thedistribution,or topography, of thesemapsare indicativeof changesin the abundance or profile of individual cell populations. Quan-tifying changes that exist between viSNE maps relies on eitherdeconstructing the system into populations or qualitative assess-mentof theoverall "shape"of themap.Approaches for quantifyingdifferences in multidimensional cytometry distributions includeQuadratic Form (38) and EMD (28). Here, we used EMD toquantify the similarity of two viSNEmaps, where differences werequantified as the amount of work required tomake the images likeanother. In this instance, "work" was defined as the number ofunits, or cells, times the distancemoved. To quantify differences inviSNEmaps, all samplesmustbe run togetheron the sameaxes. It isimperative that measures be taken to reduce batch effects, such asthe use of normalization beads here.

Blood was drawn from melanoma patients immediately priorto therapy, and 3 weeks, 12 weeks, and 6 months after the start ofanti–PD-1 therapy (data set 1). Mass cytometry was used tocharacterize PBMCs from each patient at each time point





http://mem.vueinnovations.com/

http://flowrepository.org/id/FR-FCM-ZZMC


(Supplementary Tables S1 and S2). To monitor and quantify thephenotypic plasticity of theperipheral immune systemas awhole,the EMD algorithm was used to quantify differences betweenviSNEmaps as described above (21, 28, 33). One viSNEmap wascreated for 8 patients analyzed at 4 clinical time points and 8healthy controls (Fig. 1A; Supplementary Table S1). EMD wasthen used to quantify the differences between each viSNE map,and the numerical results were displayed in a heat map (Fig. 1B).Low EMD scores indicated that the maps were similar, whereaslarger EMD scores indicated divergentmaps. To determinewheth-er the peripheral immune systems of each patient remained stableor had increased phenotypic plasticity over the course of anti–PD-1 therapy, intrapatient EMDvalues, those generatedby comparingviSNE maps within a single patient over therapy, were comparedwith interpatient EMD values, those generated by comparingviSNEmaps across patients. In 7 of 8melanomapatients receivinganti–PD-1 therapy, the intrapatient EMD scorewas lower than theinterpatient EMD score, indicating that each patient's peripheral

blood immune system was more similar to itself than to that ofany of any other patient, regardless of any ongoing therapyresponse (Fig. 1C). The exceptional patient, MB-009, did notconform to this pattern (median intrapatient EMD value �standard deviation: 4.22� 2.69, versusmedian interpatient EMDvalue: 3.99 � 2.48). This patient was diagnosed with myeloiddysplastic syndrome 8 months after starting anti–PD-1 therapyand was known to have an expansion of mature and blastingmyeloid cells and a decrease in all other major cell types in theperiphery (15). Thus, combining EMD and viSNE allowed for anautomated approach to quantify stability and plasticity of asystem over the course of therapy.

Ensemble analysis revealed decreases in PD-1þ T cells duringanti–PD-1 therapy

Although viSNE and EMD provided a quantitative, first glanceat system stability, it is possible that shifts in abundanceor phenotype of small, biologically relevant populations may

MB-004MB-005MB-006MB-007MB-008MB-009MB-010MB-012Healthy

Pre3 weeks12 weeks6 months

Pretherapy

3 weeks

12 weeks

6 months

tSNE1

tSN

E2

MB-010MB-009

EM

D v

alue

Patients

Time points

Live PBMC

MB-005

A B

C

viSNE containing live PBMC

0 4 8 12

EMD value

20-20 0

20

-20

0

02468

1012

MB-004 MB-005 MB-006 MB-007 MB-010 MB-012MB-009MB-008

***

****

**

**

**** **

**

** **

Self Mel H Self Mel H Self Mel H Self Mel H Self Mel H Self Mel H Self Mel H Self Mel H

Figure 1.

EMD quantifies phenotypic plasticity of the system over therapy and identifies an outlier patient. PBMCs from melanoma patients undergoing anti–PD-1therapy and healthy donors were characterized by mass cytometry. Equal numbers of live events from each sample were run together on a viSNE map.A, Representative live leukocyte viSNE plots are shown for 3 patients at all collection points during therapy. B, EMD was calculated, pairwise, for all samples.Heat indicates magnitude of EMD value. C, Median EMD was calculated for each patient from pairwise EMD between samples from that same patient (light gray),between that patient and all other pembrolizumab samples (white), and between that patient and all healthy donors (dark gray). N ¼ 6, 104, and 32, respectively(with the exception of MB-007, where N ¼ 1, 52, and 16, respectively). � , P < 0.001; �� , P < 0.0001. The whiskers of the boxplot extend to the most extremedata point that is no more than 1.5� the interquartile range from the box. See also Supplementary Tables S1 and S2.

Greenplate et al.





be overlooked when using a single summary statistic. Toavoid missing small but crucial cell subsets, cells should bebroken into smaller populations manually or computationallyand compared. The cells were next manually split into classicimmune populations using traditional, biaxial gating (Supple-mentary Fig. S1). Although biaxial gates were used for cellpopulation identification, it is possible to integrate other auto-mated clustering tools, such as Phenograph (23), VorteX (39),and FlowSOM (40), into the ensemble at this step. The changein frequency of these populations was calculated as a foldchange over pretherapy, allowing for visualization of allpatients and populations on one graph.

Twenty-eight cell populations were identified, and the changein frequency of those populations compared with the pretherapytime point was calculated (Fig. 2A and B; Supplementary Fig. S2).Two populations showed significant changes in frequency.

CD4þPD-1þ and CD8þPD-1þ T cells had a significant change infrequency [N ¼ 10; P < 0.0001 (pre- vs. 3 weeks) and P ¼ 0.0011(pre- vs. 12 weeks) for CD4þPD-1þ T cells, and P¼ 0.00057 (pre-vs. 12weeks) forCD8þPD-1þT cells] over the course of therapy, inwhich both decreased (Fig. 2A–C). PD-1 gates were drawn withnonmalignant tonsil as a reference and considerations discussedand explored by Nicholas and colleagues (37). This decrease inrelative frequency is most likely the result of receptor occupancyby pembrolizumab, as opposed to a biological decrease in thesecell populations (41, 42).

MEM identified signature features of PD-1þ T cells in tumor andblood

The next step in the ensemble systems immune analysis pipe-line was to automatically quantify enrichment of measured para-meters and determine how those enriched parameters changed

Cha

nge

in p

opul

atio

n ab

unda

nce

(fold

cha

nge)

A

Unchanged populationSignificantly modified population

pre therapy 3 weeks 12 weeks 6 months−2.0

0

−1.5

0.5

1.0

−0.5

−1.0

Fre

quen

cy (%

)

PD-1+ CD8+ T cells PD-1+ CD4+ T cells

0

4

1

2

3

5

6*

*

Pre 3w 12w 6m0

4

1

2

3

5

6

***

*

Pre 3w 12w 6m

B

Pre 3w 12w 6m0

20

5

10

15

25

30Plasmablasts Monocytes

Pre 3w 12w 6m0

40

102030

506070

Fre

quen

cy (%

)

Cha

nge

in p

opul

atio

n ab

unda

nce

(fold

cha

nge)

C

B cells

CD4 T cellsCD8 T cells

CD8 TemCD8 naïve

CD4 naïve

CD4 Tcm

CD16bright CD56dim NK

Pre 3w 12w 6m

CD16dim CD56bright

NK

Monocytes

C Monocytes

GC-like B cells

CD8 Tcm

NK cells

PD-1+ CD8

Activated CD4

Activated CD8Plasmablasts

CD16- CD56bright

NK

MDS blasts

CD4 Temra

Memory B cells

DN T cells

PD-1+ CD4

NC monocytes

CD4 Tem

TCRγδ

T cells

CD8 Temra

pre 3w 12w 6m

MB-004 MB-005MB-006 MB-007MB-008 MB-009MB-010 MB-012MB-013 MB-041

–3–2

0

21

3

–1

–3–2

0

21

3

–1

–3–2

0

21

3

–1

–3–2

0

21

3

–1

–3–2

0

21

3

–1

–3–2

0

21

3

–1

Pre 3w 12w 6mPre 3w 12w 6mPre 3w 12w 6m

Figure 2.

Frequency tracking of populations identifies a loss of detectable PD-1þ T cells. Established immune populations were manually gated, and populationfrequencieswere determinedover the course of anti–PD-1 therapy.A,Population frequencies at 3weeks, 12weeks, and 6months after start of anti–PD-1 therapywerenormalized to the pretherapy frequency. Each line represents a change in frequency for one population. Significantly changing populations, compared withpretherapy, are shown in red.B,Change in population frequency is shown for individual patients for eachpopulationmedian shown inA. Each timepoint after the startof therapy was compared with the pretherapy time point using a two-tailed, paired t test. With a Bonferroni correction, significantly different populationshad a P value of <0.0018. P values are indicated for populations with significant changes. C, Boxplots of population frequency are shown for each significantlychanging population (top) and for two populations that did not significantly change (bottom). P values were derived from an uncorrected, two-tailed, pairedt test. � , P < 0.05; �� , P < 0.01. See also Supplementary Figs. S1 and S2 and Supplementary Tables S1 and S2.






during treatment. MEM was used to identify signature features ofeach population at each time point following established meth-ods (19). MEM scores can range from þ10 (maximum enrich-ment) through 0 (no enrichment) to �10 (maximum lack) andhere are reported as the median � standard deviation in MEMvalue for the cell population. Enrichment quantifications for eachpopulation are a vector to be used for additional computationalanalysis. In this ensemble workflow, vectors describing the high-dimensional phenotype of populations are compared usingRMSD (19).

PD-1þ CD8þ and CD4þ T cells from pretherapy samples wereenriched for canonical identity makers CD3, and CD8 and CD4,respectively (Fig. 3A and B, top). To indicate changes in enrich-ment patterns, changes inMEM score (DMEM)were calculated bysubtracting the median MEM score for each parameter at thepretherapy time point from the indicated time point after the startof therapy. After 3 weeks and 6months of anti–PD-1 therapy, PD-1þCD8þ T cells lost enrichment, but not expression, for CD3(DMEMof –5;median� standard deviation, CD3MMI 40.9� 11

(3 weeks) and 48.3 � 14.1 (6 months; Fig. 3A, bottom; Fig. 3C).This was not the case for PD-1þCD4þ T cells (Fig. 3B, bottom).Given a loss of enrichment of the TCR CD3 subunit on theseperipheral blood PD-1þCD8þ T cells, it was informative todetermine their novelty by comparing them to other subsets,including PD-1þCD8þ T cells from tumors or healthy donors. Toassess this population, MEM labels were created for PD-1þCD4þ

and PD-1þCD8þ T cells gated from (i) blood from melanomapatients during anti–PD-1 therapy, (ii) human melanomatumors, (iii) blood from healthy donors, and (iv) tonsil oradenoid tissue from healthy donors. IPSCs, analyzed by masscytometry, were used as a control cell reference for MEM calcula-tions. Similarity in MEM labels was then compared using RMSD(Fig. 4AandB). B cells gated fromhealthydonor bloodand tonsilswere also included for contrast because of their distinctly differentenrichment profiles. As expected, B cells clustered separately fromthe T-cell populations. PD-1þCD4þ T cells from the blood ofhealthy donors and melanoma patients receiving anti–PD-1therapy clustered together, whereas PD-1þCD4þ T cells from

PD-1+ CD8 T cells

Median

Pretherapy 3 weeks post 12 weeks post 6 months post▲CD8+7±4 CD45RO+5±2 CD3+4±4 PD+1 CD27+1 ±1 CD43+1 ±1

CD44+1±1

▼CD4−10±2 CD45RA−4±2 CCR7−1±2

▲CD8+6±3 CD45RO+5±2 CD27+1±1 HLA-DR+1±1 PD+1

▼CD4−9±2 CD45RA−5±2 CD3−1±4 CCR7−1±2 CD44−1±2

▲CD45RO+6±3 CD8+5±2 CD3+3±5 CD27+1±1 PD+1 CD44+1 ±1

▼CD4−10±2 CD45RA−5±1 CCR7−1±2

▲CD8+7±4 CD45RO+5±2 CD27+1±2

CD43+1±1 PD+1

▼CD4−10±2 CD45RA−4±2 CD3−2±4 CCR7−1±2

∆ ∆ ∆

▲CD4+1

▼CD3−5 CD45RA−1 CD43−1 CD8−1 CD44−1

▲CD45RO+1

▼CD8−2 CD43−1 ▲▼CD3−5 ∆MEM

(change from “pre”)

A

PD-1+ CD4 T cells

Median

Pretherapy 3 weeks post 12 weeks post 6 months post

∆ ∆ ∆

∆MEM(change from “pre”)

▲CD4+9±2 CD45RO+5±1 CD3+4±2 CD28+1±1 CD27+1±1 CD43+1±1 CCR4+1±1 PD+1 CD44+1±1

▼ CD45RA−6±2 CD8−2±3

▲CD4+9±2 CD45RO+6±2 CD3+4±1 CD28+1±1 PD+1

▼CD45RA−6±2 CD8−2±3

▲CD4+9±2 CD45RO+6±1 CD3+4±2 CD28+1 CD27+1±1 CCR4+1±1 PD+1 CD44+1±1

▼CD45RA−5±2 CD43−1±1 CD8−1±4

▲CD4+9±2 CD45RO+6±1 CD3+4±1 CD28+1±1 CD27+1±1 CCR4+1 PD+1 CD44+1±1

▼CD45RA−5±2 CD43−1±1 CD8−1±4

▲▼CD43−1

▲CD45RO+1 CD45RA+1 CD8+1

▼CD43−1▲CD45RA+1 CD8+1

▼CCR7−1 CD43−1

B

0

2

4

6

asin

h (C

D3

MM

I)

0 3w 12w 6m CD8 B

PD-1+ CD8 T cellsfrom mel. patients

during therapy

Healthy donors

C

100101 102 103 104

100101

102

103

104

PD-1+ CD8 T cells (density), CD45+ live cells (contour)Pretherapy 3 weeks post 12 weeks post 6 months post

CD

8

CD3

Figure 3.

MEM identifies signature features of populations over the course of therapy. Tissue-specific MEM labels were created for each cell population, from eachpatient, at each time point. A, Median MEM labels are shown for PD-1þCD8þ T cells at each time point during therapy (top). DMEM labels were calculated bysubtracting the median pretherapy MEM scores from the median MEM scores at each time point. DMEM labels indicate the change in MEM value comparedwith pretherapy (bottom). B, Median MEM labels are shown for PD-1þCD4þ T cells at each time point during therapy (top). DMEM labels are shown for PD-1þCD4þ

T cells from each time point during therapy (bottom). MEM values are represented as the median MEM value � standard deviation. C, Biaxial plots of CD3 andCD8 are shown for PD-1þCD8þ T cells from representative melanoma patients (MB-004) undergoing anti–PD-1 therapy (left). Density: PD-1þCD8þ T cells; contour:live CD45þ cells. Transformed (arcsinh5) CD3 median metal intensity (MMI) is shown for PD-1þCD8þ T cells at each time point during therapy [pretherapy (0),n ¼ 10; 3 weeks (3w), n ¼ 10; 12 weeks (12w), n ¼ 7; 6 month (6m), n ¼ 8; healthy, n ¼ 8]. Healthy PBMC donor CD8þ T and B cells are shown for reference.See also Supplementary Tables S1 and S2.

Greenplate et al.





melanoma tumors and healthy donor tonsils formed a differentcluster (Fig. 4A). Similarly, PD-1þCD8þ T cells from melanomatumors clustered with those from healthy donor tonsils, whereasPD-1þCD8 T cells from healthy donor blood and melanomablood formed two, intermixed clusters. These results indicatedPD-1þT cells found in thebloodwere distinct from those found inthe tumor or healthy tonsil.

Median MEM labels, which display enrichment scores for eachmeasured feature, are shown for each tissue's PD-1þ T-cell popu-lations (Fig. 4B). PD-1þCD8þT cells from thebloodofmelanomapatients were enriched for CD43 protein expression and traffick-ing markers, such as CCR4 and CXCR3, and specifically lackedactivation markers CD38 and CD69 compared with PD-1þCD8þ

T cells in melanoma tumors (Fig. 4C, left). PD-1þCD4þ T cellswere enriched for CD4 (þ3) compared with their melanomatumor counterparts. No difference in enrichment between PD-1þCD4þ T cells in the peripheral blood ofmelanomapatients andhealthy donors (Fig. 4C, right). PD-1þ T cells from the blood ofmelanoma patients and healthy donors were, therefore, pheno-

typically similar. Thus, MEM, as part of this ensemble, automatedquantitative comparisons of high-dimensional data from differ-ent tumor and donor types.

Increased CD4þ T-cell frequency following chemotherapy inpatients with AML

The ensemble systems immune monitoring pipeline wasapplied to a previously published data set of peripheral bloodfrom AML patients undergoing chemotherapy (data set 2; ref. 12)in order to describe anddissect system-wide changes. Five patientswith AML were consented for peripheral blood draws over thecourse of chemotherapy, which were then characterized bymass cytometry. All PBMCs, including blasts and nonblasts,were identified using the gating scheme published by Ferrell andcolleagues (12). EMD on t-SNE revealed lack of intrapatientstability in 3 of 5 patients, indicated by no significant differencebetween intra- and interpatient EMD values (Fig. 5A; Supplemen-tary Fig. S3). The remaining 2 patients showed intrapatientstability, most likely due to the presence of leukemic blasts

▲CD4+3 CD43+2 CXCR3+2 CCR5+1 CD44+1

▼CD69−2 CD45RO−2 CD38−1 ICOS−1

▲CD43+3 CD57+2 CCR4+2

CXCR3+2 CD27+1 CD44+1

▼CD38−2 CD69−2 CD45RO−2

PD

-1+

CD

4 T

cells

PD

-1+

CD

8 T

cells

B c

ells

A

Blood Tissue/tumorMelanoma Healthy85 90 95 100

Normalized RMSD of MEM label (%)

C PD-1+ CD8 T cells ∆MEM Melanoma tumor Healthy blood

∆ ∆

Markers enrichedin melanoma

PBMC

PD-1+ CD4 T cells ∆MEM Melanoma tumor Healthy blood

∆ ∆

▲CCR4+2 CD45RO+1

▼CD57−2 No difference

Median MEM labels iPSCs as referenceB

PD-1+ CD8 PD-1+ CD4▲CD45+9±1 CD8+5±1 CD43+5±1

CD3+4±1 CD45RO+4±1CD44+2±1

CXCR3+1±1 CCR5+1±1 CD27+1±1

PD-1+1±0 CCR4+1±2

▼ CD56−2±0 PD-L1−2±0 CD57−2±2

▲CD45+9±0 CD45RO+5±0 CD8+5±1 CD3+3±0 PD-1+2±0 CD43+2±0 CD69+2±1 CD38+2±1 CCR5+1±0

CD44+1±0

▼CD57−3±0 CD56−2±0 PD-L1−2±0

CXCR3−1±1 CCR4−1±2

▲CD45+9±1 CD8+5±1 CD43+5±0

CD3+4±0 CD45RO+3±2 CD44+2±1

PD-1+1±0 CCR5+1±1

▼CD56−2±0 PD-L1−2±0 CCR4−2±2

▲CD45+8±0 CD8+6±0 CD3+4±0

CD45RO+4±1 CD43+3±0 CD44+3±0

PD-1+2±0 CD69+2±0 CD27+1±0

▼CD57−3±2 CD56−2±0 PD-L1−2±0

CCR4−2±2 CXCR3−1±1

▲CD45+9±1 CD4+5±0 CD45RO+5±0

CD43+5±1 CD3+4±1CD44+2±1

CCR4+1±0 CD28+1±0 PD-1+1±0

▼CD57−3±0 CD56−2±0 PD-L1−2±0

▲CD45+10±0 CD4+5±0 CD45RO+5±0

CD43+5±0 CD3+4±0 CCR4+1±0

CD69+2±1 CD44+2±1 ICOS+1±0

CD38+1±1 CD28+1±1 PD-1+1±0

▼CD57−3±0 CD56−2±0 PD-L1−2±0

CXCR3−1±0

▲CD45+9±1 CD4+5±0 CD45RO+5±0

CD43+5±0 CD3+4±0 CD44+2±1

PD-1+1±0 CCR5+1±0 CCR4+1±1

CXCR3+1±1 CD27+1±1 CD28+1±1

▼CD57−3±1 CD56−2±0 PD-L1−2±0

▲CD45+9±0 CD45RO+5±1 CD4+4±0

CD3+3±0 CD43+3±0 CD69+3±0

PD-1+2±1 CCR4+1±0 CD38+1±0

CD44+1±0 ICOS+1±0 CD28+1±1

▼CD56−2±0 PD-L1−2±0 CXCR3−1±0

CCR5−1±1 CD57−1±1

Melanoma blood

Melanoma tumor

Healthy blood

Nonmalignant adenoid/tonsil

PD-1+ CD8 PD-1+ CD4

PD-1+ CD8 PD-1+ CD4

PD-1+ CD8 PD-1+ CD4

Figure 4.

MEM reveals that PD-1þ T cells from blood differ from those in the tumor. A, MEM labels were compared for each of the 112 populations (PD-1þ CD4þ andCD8þ T cells and B cells) from three human tissues. Populations were defined using traditional biaxial gates as in Supplementary Fig. S1. Tissue type and source areindicated in the bottom left. B,Median MEM labels are shown for PD-1þ CD4þ and CD8þ T cells from each tissue type. MEM values are shown� standard deviation.C, DMEM scores show the difference in median MEM scores between PD-1þCD8þ T cells (left) or PD-1þCD4þ T cells (right) in the peripheral blood and thosefound in the tumor or blood of healthy donors. See also Supplementary Tables S1 and S2.






throughout treatment. Unlike in data set 1, where cellular popu-lationswere identified by traditional biaxial gates, 15 populationswere defined automatically using the SPADE algorithm to clusteron t-SNE axes (Fig. 5B), as previously described (17, 24). Thesepopulations were then fed into the remaining 3 steps of theensemble analysis pipeline.

Of the 15 populations identified, 7 showed significant changesat some point during chemotherapy (Fig. 5C; SupplementaryFig. S4). MEM was used to label the automatically characterizedsignature features on each population (Fig. 5D; SupplementaryFig. S4). Two populations of blasts were defined by HLA-DRenrichment and were observed to decrease over the course of

Cha

nge

in p

opul

atio

n ab

unda

nce

(fold

cha

nge)

-4

6

-2

0

2

Unchanged populationSignificantly changing population

4

-60 2/3 5/6 8 10 14 Rec

0

20

40

60

80

100

0

20

40

60

80

0

10

20

30

40

50

0

10

20

30

40

0 2/3 5/6 8 10 14 Rec 0 2/3 5/6 8 10 14 Rec

Fre

quen

cy (%

)

▲CD4+4 HLADR+4 CD11b+3 CD64+3 CD33+3 CD38+3 CD184+3 CD123+2 CD11c+2

CD56+2 ▼CD7−4 CD61−4 CD34−3 CD62L−3 CD45−2

Population 1(AML Blasts)

▲HLADR+3 CD38+2 ▼CD4−7 CD7−7 CD45−4 CD11b−2 CD64−2 CD185−2


Population 13(CD4+ T cells)

Population 12(CD4+ T cells)

▲CD4+10 CD7+4 CD45+4 CD64+2 ▼HLADR−7 CD183−3 CD38−3 CD61−2 CD123−2

▲CD4+8 CD7+4 CD45+2 ▼HLADR−3 CD123−2

0 2/3 5/6 8 10 14 Rec 0 2/3 5/6 8 10 14 Rec

0 15 30

EMD Value

AML001

AML002

AML003

AML004

AML007

AML001 AML002 AML003 AML004 AML007

-30-30

30

30

0

0

-15 15

-15

15

-30-30

30

30

0

0

-15 15

-15

15

All AML patients and time points

Density

tNS

E2

tNSE1

1

Populations

1011

12 13

14 152

3

45

67

89

Intact cells, PBMCA

B

C

D

35 100


Exp

ert l

abel

: C

D4

T ce

llsE

xper

t lab

el: B

cel

ls

AML population 13(CD4+ T cells)

Healthy humanCD4+ T cells

Healthy humanB cells

E

Time over therapy (days)

Time over therapy (days) Time over therapy (days)

Time over therapy (days) Time over therapy (days)

▲CD15+4 CD7+3 CD38+3 CD56+3 ▼CD4−3


0

5

10

15

20

0 d5/6 d10 Recd2/3 d8 d14

0

5

10

15

20

25

30

0 2/3 5/6 8 10 14 Rec

Population 6(CD4- T cells)

▲CD7+4 CD62L+2 CD45+2 ▼CD4−6 CD38−4 HLADR−4 CD11b−2 CD64−2 CD123−2



Figure 5.

Ensemble immune analysis and automated gating identifies loss of peripheral blasts and increase in nonmalignant immune cells in patients with AMLundergoing chemotherapy. PBMCs from patients with AML undergoing chemotherapy were characterized by mass cytometry. Equal numbers of live eventsfrom each sample were run together on a viSNE map. A, EMD was calculated, pairwise, for all samples. Heat indicates magnitude of EMD value. B, Populations wereidentified (right) by SPADE of cell density on t-SNE axes (left). C, Frequency of populations identified in B was normalized to the pretherapy frequency andcompared using a paired t test. Populations with time points that significantly change from pretherapy are shown in red. D, Boxplots are shown for eachsignificantly changing population. Each population is labeled with an MEM label and an expert given name derived from the MEM label. E, MEM labels frompopulation 13 were compared with the 80 populations (CD4þ T cells and B cells) using RMSD (36). See also Supplementary Figs. S3 and S4.

Greenplate et al.





chemotherapy. In contrast, CD4– T cells and two subsets of CD4þ

T cells were observed to expand, relative to other populations,following chemotherapy. Population 13 was enriched for CD4(þ5), CD7 (þ4), and CD45 (þ2), while specifically lackingexpression of HLA-DR (�3) and CD123 (�2). Combined withthe expression of CD3 (Supplementary Fig. S5), population 13 islikely a population of T cells. Next, the similarity of population 13to CD4þ T cells from healthy donors was assessed. To do this, acommon reference population of hematopoietic stem cells wasused to create MEM labels for population 13 and for previouslypublished healthy donorCD4þ T cells and B cells (19). RMSDwasused to compare MEM labels and determine whether population13 shared a phenotype similar to that of previously characterizedCD4þ T cells. Population 13 clustered with CD4þ T cells fromhealthy donors, suggesting that they are phenotypically similar(Fig. 5E). Taken together, these results represent an automatedway to dissect immune responses to therapy and suggest thatchemotherapy resulted in a relative increase of a nonmalignantpopulation of CD4þ T cells.

Loss of activated T cells and expansion of CD4–CD8– T cells inmelanoma tumors with MEKi and BRAFi

Samples from a cohort of melanoma patients with BRAFV600E

mutations treated with targeted therapies dabrafenib and trame-tinib (n ¼ 11; data set 3) were characterized by mass cytometry,and the data were analyzed by the ensemble systems immunemonitoring pipeline. Combining EMD and viSNE to quantifystability of the immune compartment showed that each patientremainedmore similar to itself than toothermelanoma tumors orhealthy tonsils over the course of therapy. However, EMD run ont-SNE axes created from analysis of T cells revealed that the T-cellcompartment did not have lower intrapatient EMD values com-pared with interpatient EMD values (Fig. 6A). Thus, significantimmune plasticity followed therapy. Immune populations weredefined using traditional biaxial gates, and the change in frequen-cy for those populations was calculated. After 4 weeks of treat-ment, a statistically significant change in frequency of 5 immunepopulations was observed. One population of interest includeddouble-negative (DN) T cells lacking expression of CD4 or CD8(DN T cells) that comprised 7.23% � 17.18% in pretherapytumors, 26.27% � 16.36% in tumors 4 weeks after therapy, and3.57% � 1.52% of a healthy lymph node (median � standarddeviation; Fig. 6B; Supplementary Figs. S6–S7). Pretherapy MEMlabels showed that DN T cells were enriched for CD3 (þ2), CD45(þ2), CD45RO (þ1), CD4 (þ1), and CD28 (þ1) but specificallylacked CD8 (–3) and CD45RA (–2) when compared with all cellsfoundwithin themelanoma tumors.DMEMscores indicated that,over the course of therapy, DN T cells became more enriched forCD45RO(þ1) andCD44 (þ1) but lost enrichment ofCD69 (–1),CD43 (–1), CD27 (–1), and HLA-DR (–1; Fig. 6C).

Given that T cells can downregulate expression of CD4 or CD8if activated (43), DN T cells from melanoma tumors were com-pared with acutely activated peripheral blood T cells stimulatedthrough the TCR by Staphylococcal enterotoxin B (SEB) (37).Additionally, we sought to understand whether similar T cellswere found in other cancers, such as glioblastoma (GBM) andrenal cell carcinoma (RCC; Fig. 6D). SEB-stimulated T cellsformed their own cluster, apart from all other DN T cells, andshared 87.2% (�2.88%, n¼ 78 comparisons) similarity with DNT cells from melanoma tumors. In contrast, DN T cells frommelanoma, GBM, and RCC clustered together, with a similarity

score of 93.6% (�1.06%, n ¼ 78 comparisons). Of the T cellsactivated by SEB, close to half were CD69þ (41.26% � 7.41%),andmost retained CD4or CD8 coreceptor expression (Fig. 6E). Incontrast, DN T cells from posttreatment melanoma tumors con-tained fewer CD69þ cells (22.45% � 8.76%; Fig. 6F). Thus, DNT cells from melanoma, GBM, and RCC consisted of a pheno-typically distinct population of DN T cells not observed in restingnondiseased blood and tonsils or acutely stimulated blood. Takentogether, these data suggest that an unusual population of T cellsemerged in tumors from melanoma patients treated with BRAFiand MEKi and that this DN T-cell population occurs in multipletypes of tumors.

DiscussionThe article describes an analysis suite designed to be a common

starting point for immunologists tracking cells over time, providesthree reference data sets for testing tools designed to discover andcharacterize cell subsets, and reveals unexpected cells in thecontext of cancer therapies. This ensemble data analysis strategywas designed specifically for systems immune monitoring inlongitudinal, clinical studies, but it could be applied for anysystem with changes. We envision adapting it to study experi-mental perturbations to map signaling networks and drugresponses (44). Although we expect the algorithms in the ensem-blewill change and improve over time, the four cellular propertiesidentified should be considered essential features for immunemonitoring with any single-cell platform. Cancer immune mon-itoring strategies must expect the unexpected and be prepared fornovel phenotypes (9, 45, 46). Thefirst property, systemsplasticity,automatically quantifies the state of a system compared withbaseline. In the cohort described here, the peripheral blood ofmelanomapatients remained stable over the course of therapy. Bymonitoring systems plasticity with EMD and t-SNE, a patient whoexperienced bone marrow failure (15) was identified withoutgating or clustering. In this case, quantifying systems plasticityallowed for the unbiased identification of a patient experiencinglarge, biologically relevant changes during therapy. Caveats existfor using EMD to quantify differences between viSNE maps. Allsamplesmust be embedded on the same viSNEmap in order to beappropriately compared, and precautions, such as barcoding andbead normalization, must be taken to ensure that differences inviSNEmaps are attributed to biological differences and not batcheffects. In addition to EMD, the quadratic form (38) could be usedto quantify differences in t-SNE axes. Looking forward, we envi-sion automating a process inwhich t-SNE and aplasticity test suchas EMD are first run on the system as a whole, as described here,and then run iteratively on increasingly refined populations inorder to pinpoint the populations undergoing the greatest changeor stability.

The remaining components of the ensemble work together todescribe changes or stability revealed in the first step, systemsplasticity. To accomplish this, the system is divided into subpo-pulations, and the frequency, signature features, and novelty ofeach subpopulation are quantified. Identification of cells intogroups can be accomplished in whichever way is most appropri-ate, or in multiple ways, prior to ensemble analysis (22–24, 27).For example, FlowSOM (40) could be used to identify cellularpopulations, and those populations could be fed into the remain-ing parts of the ensemble. Because this ensemble toolkit does notrely on known populations, it remains independent from






methods of automated population identification (17, 27) and canbe used to compare and communicate analysis results from teamsrelying on computational approaches, immunologists, and bio-informatics experts. The ensemble approach was especially adeptat capturing shifts over time and was able to identify a shift inrelative abundance of a subset of T lymphocytes, their phenotype,and intrapatient recovery of these phenotypes, all of which couldhave substantial implications for maintenance of remissionand clinical outcomes (47, 48). A human immune monitoringstrategy using this ensemble toolkit might provide new insight

into the immune system's interaction with leukemia remissionand relapse.

This ensemble approach was robust across multiple, contrastingstudies in quantifying changes and stability in immune systemcells. This approach also provided a detailed analysis of the abun-dance and tractable quantitative phenotype of populations thatcomprised each system.We found a population of CD4–CD8–DNT cells observed to have a common phenotype in threehuman tumor types, melanoma, RCC, and glioblastoma thathad a phenotype distinct from that of resting CD4–CD8– DN and

0

10

20

30

40

50

0

10

20

30

40

50

Melv

Self

Melv

Mel

MelvH

HvH

EM

D v

alue

Melv

Self

Melv

Mel

MelvH

HvH

EM

D v

alue

Live, CD45+ T cells

****** ***

****** ***

***

**

*** ▲CD3+2 CD45+2 CD45RO+1 CD4+1

CD28+1

▼CD19−6 CD20−5 CD8−3 CD45RA−2

CD27−1 HLADR−1

DN T cells

∆

▲CD45RO+1 CD44+1

▼CD69−1 CD43−1 CD27−1 HLADR−1▲CD3+2 CD45+2 CD4+2 CD45RO+1

CD28+1 CD44+1

▼CD19−6 CD20−5 CD8−3 CD45RA−2

CD27−2 HLADR−2 CD69−1 CD43−1

Pretherapy 4 wks post MEKi/BRAFi

80 90 100


Blood Tissue/tumorGBMMelanoma Healthy RCC

* SEB stimulated

A C

D

B

MP031 MP032 MP034 MP055 MP059 MP029

30039 30043 30044 VU001 VU00230026

CD

45R

O

CD

4

CD

45R

O

CD

4

CD8

CD8CD69

CD69

E

F

T cells, SEB-stimulated PBMC

DN T cells, melanoma tumors (4 wks after therapy start)

Activated T cells, SEB-stimulated PBMC

T cells, melanoma tumors (4 wks after therapy start)

101 102 103 104

101

102

103

104

101 102 103 104

101

102

103

104

101 102 103 104

101

102

103

104

101 102 103 104

101

104

103

102

74.212.06 19.01

68.594.76 24.16

68.923.02 23.7

53.228.91 34.97

57.727.21 32.29

44.321.9330.84

22.0239.35 29.6

13.6541.87 30.40

29.2911.03 47.17

44.3313.02 37.17

59.8910.15 21.29

7.9751.3 32.6

56.11 41.15 37.36 41.37 51.2639.82

26.61 4.50 25.58 28.22 18.5119.32

30039 30043 30044 VU001 VU00230026

MP031 MP032 MP034 MP055 MP059 MP029

*****

**

Median MEM labels iPCs as referenceRCC tumor

Melanoma tumor

GBM tumor

Nonmalignant LN

Activated T cells PBMC

▲CD45+7±3 CD3+3±1 CD45RO+2±3 CD44+2±3

▼CD57−6±2 CCR4−3±1 CD56−2±1 PDL1−2±1

▲CD45+7±1 CD45RO+2±0 CD3+2±0 CD44+2±1

▼CD57−5±0 CD56−3±0 CCR7−1±0 PD1−1±0

CD45RA−1±1

▲CD45+8±1 CD45RO+3±1 CD3+2±0 CD69+2±1

CD44+2±1

▼CD57−5±2 CD56−3±1 PDL1−2±0 CCR5−1±1

▲CD45+8±1 CD3+4±0 CD44+2±1 CD45RA+1±2

▼CD57−5±1 CCR4−3±0 CD56−3±0 PDL1−2±0

CCR7−1±0 CXCR3−2±0

Healthy donor PBMC▲CD45+10±1 CD3+4±1 CD44+2±1 CD45RA+1±2

▼ CD57−5±2 CCR4−3±0 CD56−3±0 PDL1−2±0

CCR7−1±0 CD28−1±1 CXCR3−1±1 CCR51±1

▲CD45+10±1 CD45RO+6±2 CD69+5±1 CD4+4±2

CD3+3±1 CCR7+3±2 HLADR+2±1 CD8+2±1

CD27+2±2 CD25+2±3

▼CD57−5±1 CD16−1±0

DN T cells (n = 9)

0

-2.5

2.5

-1

1

0 4 wk

*

CD8 naïve (n = 9)

0

-2

10.5

-0.5-1

-1.5

0 4 wk

*

Activated CD8 (n = 6)

0

-1

10.5

-0.5

-1.5

1.5

0 4 wk

*

Activated CD4 (n = 6)

0

-1

1

0.5

-0.5

0

**

Activated DN (n = 6)

0

-2.5

2.5

-1

1

0 4 wk

*

APCs (n = 8)

0

-1

10.5

-0.5

-1.5

1.5

0 4 wk

*

Time points after start of BRAFi

Cha

nge

in p

opul

atio

n fre

quen

cy (l

og2)

MP-055 MP-059MP-031

MP-034MP-019 MP-029 MP-032

MP-054MP-023

4 wk

Figure 6.

Ensemble immune analysis identifies expansion of CD8 and CD4 double-negative T cells in tumors from patients treated with BRAF and MEK inhibitors.Single-cell suspensions of melanoma tumor biopsies from before and after treatment with BRAF and MEK inhibitors were characterized by mass cytometry.A, Equal numbers of live events (left) or T cells (right) from each sample were run together on a viSNE map. EMD was calculated, pairwise, for all samples.Boxplots showmedian, pairwise EMDvalues for listed comparisons. Unpaired Student t test. � ,P <0.05; �� ,P <0.01; �� ,P<0.001.B,Changes in population frequencyare shown for individual patients for each significantly changing population. Each time point after the start of therapy was compared with the pretherapytime point using a two-tailed, paired t test. � ,P<0.05; ��,P <0.01.C,MedianMEM labels are shown forDNpretherapy and4weeks after therapy. ADMEMscore showsthe difference in median MEM scores DN T cells before and after the start of therapy. D, MEM labels from tumor-resident DN T cells were compared withSEB-stimulated T cells fromperipheral blood using RMSD. MedianMEM scores for each tissue are shown on the right. E,Biaxial plots of all T cells fromSEB-stimulatedPBMCs (left) and activated T cells (CD69þ, plots on right). F, Biaxial plots of DN T cells from tumors of patients treated for 4 weeks with BRAFi and MEKi (left)and all T cells from (right). See also Supplementary Figs. S6 and S7; Supplementary Tables S1 and S2.

Greenplate et al.





SEB-activated T cells from healthy individuals. This result is con-sistent with reports characterizing the phenotypic similarity ofT cells infiltrating melanoma and colon cancer mouse tumormodels (49) and provides evidence that common changes toimmune cell mechanisms are shared across human tumor typesand play a role in response to targeted therapy.

The ensemble toolkit detected known biological occur-rences, as well as identified potential mechanisms for furtherstudy. The ensemble toolkit detected an overall loss of PD-1þ

T cells in the peripheral blood over the course of therapy. Thishas previously been described and attributed to receptoroccupancy by the drug itself (41, 50). Previous work revealedan expanded population of Ki67þ T cells in the peripheralblood during immunotherapy (10, 51). Although our com-parable study did not measure Ki67 in the blood and, thus,could not be compared directly, we did observe a trendtoward increased activated CD4þ and CD8þ T cells, definedby expression of HLA-DR, at 3 weeks after the start of therapy,and CD4þ effector memory T (TEM) cells at all time points afterthe start of therapy. In addition to capturing known clinicalevents, the ensemble tool kit identified a novel populationof DN T cells present in unexpectedly high frequencies inmelanoma tumors before and after treatment with BRAFi andMEKi compared with healthy lymphoid tissue. Expanded DNT-cell populations have previously been reported in metastaticlymph nodes of melanoma patients (52). However, theirdeep phenotype or frequency in response to inhibitor therapyhas not yet been described. Previous work has described theexpansion of a regulatory CD3þ T-cell population lackingboth CD4 and CD8 after TCR and cytokine stimulation(53–55). MEK inhibitors can support antitumor T-cell func-tion by blunting TCR-induced apoptosis (56). Therefore, itis possible that high frequencies of DN T cells after treatmentwith MEKi and BRAFi indicate an accumulation of T cellsderived from tumor-resident T cells, although additionalmechanistic studies are required.

Overall, the melanoma tumor data set (data set 3) highlightsthe complexity of tumor-associated T cells in human malig-nancies and provides further evidence that phenotypicallydiverse populations of tumor-resident T cells can be foundacross multiple, distinct tumor types (49). The data presentedhere will join a common immunology reference set (19, 57, 58)that can be mined further to characterize and understandchanges in immune and cancer cell populations in diversedisease settings and build a reference of cellular identity.

Disclosure of Potential Conflicts of InterestC.E. Roe reports receiving commercial research funding from Incyte Corpo-

ration, Pharmacyclics LLC, and Janssen Pharmaceutica. M.C. Kelley reportsprovidingMedical Professional Liability Reviews unrelated to the content of thisresearch manuscript. D.B. Johnson reports receiving commercial research fund-ing from Bristol-Myers Squibb and Incyte and is a consultant/advisory boardmember for Array, Bristol-Myers Squibb, Genoptix, Merck, and Novartis. J.M.Irish is co-founder of Cytobank Inc.; reports receiving commercial researchfunding from Janssen, Incyte, and Pharmacyclics; and has ownership interest inCytobank Inc. No potential conflicts of interest were disclosed by the otherauthors.

Authors' ContributionsConception and design: A.R. Greenplate, D.B. Johnson, J.M. IrishDevelopment of methodology: A.R. Greenplate, D.D. McClanahan,B.K. Oberholtzer, D.B. Doxie, J.M. IrishAcquisition of data (provided animals, acquired and managed patients,provided facilities, etc.): A.R. Greenplate, C.E. Roe, N. Leelatian, M.C. Kelley,P.J. Siska, J.C. Rathmell, P.B. Ferrell, D.B. JohnsonAnalysis and interpretation of data (e.g., statistical analysis, biostatistics,computational analysis): A.R. Greenplate, D.D. McClanahan, K.E. Diggins,N. Leelatian, P.B. Ferrell, J.M. IrishWriting, review, and/or revision of the manuscript: A.R. Greenplate,D.D. McClanahan, B.K. Oberholtzer, D.B. Doxie, C.E. Roe, N. Leelatian,M.C. Kelley, V. Gama, P.B. Ferrell, D.B. Johnson, J.M. IrishAdministrative, technical, or material support (i.e., reporting or organizingdata, constructing databases): A.R. Greenplate, D.B. Doxie, C.E. Roe,M.L. Rasmussen, V. GamaStudy supervision: J.C. Rathmell, J.M. IrishOther (generation and characterization of induced pluripotent stem cellsfrom human skin fibroblasts to be used as controls for analysis):M.L. Rasmussen

AcknowledgmentsThis study was supported by NIH/NCI R00 CA143231 (J.M. Irish), F31

CA199993 (A.R. Greenplate), U01 CA196405 (D.D. McClanahan and J.M.Irish), R25 GM062459 (D.B. Doxie), T32 CA009592 (D.B. Doxie), K23HL138291 (P.B. Ferrell), and K23 CA204726 (D.B. Johnson); the Vander-bilt-Ingram Cancer Center (VICC, P30 CA68485); a Vanderbilt UniversityDiscovery Grant (J.M. Irish and N. Leelatian); a VICC Provocative Questionaward (M.C. Kelley and J.M. Irish); and VICC Ambassadors (J.M. Irish andA.R. Greenplate).

The authors thank Holly Crandall and Kate Carson for their help withconsenting patients and collecting patient samples.

The costs of publication of this articlewere defrayed inpart by the payment ofpage charges. This article must therefore be hereby marked advertisement inaccordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Received December 1, 2017; revised July 17, 2018; accepted November 5,2018; published first November 9, 2018.

References1. Greenplate AR, Johnson DB, Ferrell PB Jr, Irish JM. Systems immune

monitoring in cancer therapy. Eur J Cancer 2016;61:77–84.2. Brodin P, Davis MM. Human immune system variation. Nat Rev Immunol

2017;17:21–9.3. Brodin P, Jojic V,GaoT, Bhattacharya S, AngelCJ, FurmanD, et al. Variation

in the human immune system is largely driven by non-heritable influences.Cell 2015;160:37–47.

4. Gaudilliere B, Fragiadakis GK, Bruggner RV, Nicolau M, Finck R, Tingle M,et al. Clinical recovery from surgery correlates with single-cell immunesignatures. Sci Transl Med 2014;6:255ra131.

5. Kaczorowski KJ, Shekhar K, Nkulikiyimfura D, Dekker CL, Maecker H,Davis MM, et al. Continuous immunotypes describe human immunevariation and predict diverse responses. PNAS 2017.

6. KotechaN, FloresNJ, Irish JM, Simonds EF, SakaiDS, Archambeault S, et al.Single-cell profiling identifies aberrant STAT5 activation in myeloid malig-

nancies with specific clinical and biologic correlates. Cancer Cell 2008;14:335–43.

7. Carr EJ, Dooley J, Garcia-Perez JE, Lagou V, Lee JC, Wouters C, et al. Thecellular composition of the human immune system is shaped by ageand cohabitation. Nat Immunol 2016;17:461–8.

8. Tsang JS, Schwartzberg PL, Kotliarov Y, Biancotto A, Xie Z, Germain RN,et al. Global analyses of human immune variation reveal baseline pre-dictors of postvaccination responses. Cell 2014;157:499–513.

9. Irish JM. Beyond the ageof cellular discovery.Nat Immunol 2014;15:1095–7.10. Spitzer MH, Carmi Y, Reticker-Flynn NE, Kwek SS, Madhireddy D, Martins

MM, et al. Systemic immunity is required for effective cancer immuno-therapy. Cell 2017;168:487–502 e15.

11. Huang AC, Postow MA, Orlowski RJ, Mick R, Bengsch B, Manne S, et al.T-cell invigoration to tumour burden ratio associated with anti-PD-1response. Nature 2017;545:60–65.






12. Ferrell PB Jr, Diggins KE, Polikowsky HG, Mohan SR, Seegmiller AC, IrishJM. High-dimensional analysis of acute myeloid leukemia reveals pheno-typic changes in persistent cells during induction therapy. PLoS One2016;11:e0153207.

13. Irish JM, Hovland R, Krutzik PO, Perez OD, Bruserud O, Gjertsen BT, et al.Single cell profiling of potentiated phospho-protein networks in cancercells. Cell 2004;118:217–28.

14. Kordasti S, Costantini B, Seidl T, Perez Abellan P, Martinez Llordella M,McLornan D, et al. Deep-phenotyping of Tregs identifies an immunesignature for idiopathic aplastic anemia and predicts response to treat-ment. Blood 2016.

15. Greenplate AR, JohnsonDB, RousselM, SavonaMR, Sosman JA, Puzanov I,et al. Myelodysplastic syndrome revealed by systems immunology in amelanoma patient undergoing anti-PD-1 therapy. Cancer Immunol Res2016;4:474–80.

16. RoedererM,QuayeL,ManginoM,BeddallMH,MahnkeY,ChattopadhyayP,et al. The genetic architecture of thehuman immunesystem: abioresource forautoimmunity and disease pathogenesis. Cell 2015;161:387–403.

17. Diggins KE, Ferrell PB Jr, Irish JM. Methods for discovery and character-ization of cell subsets in high dimensional mass cytometry data. Methods2015;82:55–63.

18. Saeys Y,Gassen SV, Lambrecht BN.Computationalflowcytometry: helpingto make sense of high-dimensional immunology data. Nat Rev Immunol2016;16:449–62.

19. Diggins KE, Greenplate AR, Leelatian N, Wogsland CE, Irish JM. Charac-terizing cell subsets using marker enrichment modeling. Nat Methods2017;14:275–8.

20. Spitzer MH, Gherardini PF, Fragiadakis GK, Bhattacharya N, YuanRT, Hotson AN, et al. IMMUNOLOGY. An interactive referenceframework for modeling a dynamic immune system. Science 2015;349:1259425.

21. Amir el AD, Davis KL, Tadmor MD, Simonds EF, Levine JH, Bendall SC,et al. viSNE enables visualization of high dimensional single-cell dataand reveals phenotypic heterogeneity of leukemia. Nat Biotechnol2013;31:545–52.

22. Qiu P, Simonds EF, Bendall SC, Gibbs KD Jr, Bruggner RV, LindermanMD,et al. Extracting a cellular hierarchy from high-dimensional cytometry datawith SPADE. Nat Biotechnol 2011;29:886–91.

23. Levine JH, Simonds EF, Bendall SC, Davis KL, Amir el AD, Tadmor MD,et al. Data-driven phenotypic dissection of AML reveals progenitor-likecells that correlate with prognosis. Cell 2015;162:184–97.

24. Shekhar K, Brodin P, Davis MM, Chakraborty AK. Automatic classificationof cellular expression by nonlinear stochastic embedding (ACCENSE).PNAS 2014;111:202–7.

25. Arvaniti E, Claassen M. Sensitive detection of rare disease-associated cellsubsets via representation learning. Nat Commun 2017;8:14825.

26. Bruggner RV, Bodenmiller B, Dill DL, Tibshirani RJ, Nolan GP. Automatedidentification of stratifying signatures in cellular subpopulations. PNAS2014;111:E2770–7.

27. Weber LM, Robinson MD. Comparison of clustering methods for high-dimensional single-cell flow and mass cytometry data. Cytometry Part A2016;89:1084–96.

28. Orlova DY, Zimmerman N, Meehan S, Meehan C, Waters J, Ghosn EE,et al. Earth Mover's Distance (EMD): a true metric for comparingbiomarker expression levels in cell populations. PLoS One 2016;11:e0151859.

29. JohnsonAS, CrandallH,Dahlman K, KelleyMC. Preliminary results fromaprospective trial of preoperative combined BRAF and MEK-targeted ther-apy in advanced BRAF mutation-positive melanoma. J Am Coll Surg2015;220:581–93 e1.

30. LeelatianN, Doxie DB, Greenplate AR,Mobley BC, Lehman JM, Sinnaeve J,et al. Single cell analysis of human tissues and solid tumors with masscytometry. Cytometry B Clin Cytom 2017;92:68–78.

31. Leelatian N, Doxie DB, Greenplate AR, Sinnaeve J, Ihrie RA, Irish JM.Preparing viable single cells from human tissue and tumors for cytomicanalysis. Curr Protoc Mol Biol 2017;118:25C 1 1-C 1 3.

32. Siska PJ, Beckermann KE, Mason FM, Andrejeva G, Greenplate AR, SendorAB, et al. Mitochondrial dysregulation and glycolytic insufficiency func-tionally impair CD8 T cells infiltrating human renal cell carcinoma.JCI Insight 2017;2.

33. Japp AS, Hoffmann K, Schlickeiser S, Glauben R, Nikolaou C, Maecker HT,et al. Wild immunology assessed by multidimensional mass cytometry.Cytometry Part A 2017;91:85–95.

34. Gottschlich C, Schuhmacher D. The Shortlist Method for fast computationof the Earth Mover's Distance and finding optimal solutions to transpor-tation problems. PLoS One 2014;9:e110214.

35. Lovly CM, Dahlman KB, Fohn LE, Su Z, Dias-Santagata D, Hicks DJ,et al. Routine multiplex mutational profiling of melanomas enablesenrollment in genotype-driven therapeutic trials. PLoS One 2012;7:e35309.

36. Doxie DB, Greenplate AR, Gandelman JS, Diggins KE, Roe CE, Dahl-man KB, et al. BRAF and MEK inhibitor therapy eliminates Nestin-expressing melanoma cells in human tumors. Pigment Cell MelanomaRes 2018.

37. Nicholas KJ, Greenplate AR, Flaherty DK, Matlock BK, Juan JS, Smith RM,et al. Multiparameter analysis of stimulated human peripheral bloodmononuclear cells: A comparison of mass and fluorescence cytometry.Cytometry Part A 2016;89:271–80.

38. Bernas T, Asem EK, Robinson JP, Rajwa B. Quadratic form: a robust metricfor quantitative comparison of flow cytometric histograms. Cytometry PartA 2008;73:715–26.

39. Samusik N, Good Z, Spitzer MH, Davis KL, Nolan GP. Automatedmapping of phenotype space with single-cell data. Nat Methods2016;13:493–6.

40. Van Gassen S, Callebaut B, Van Helden MJ, Lambrecht BN, Demeester P,Dhaene T, et al. FlowSOM: Using self-organizing maps for visualizationand interpretation of cytometry data. Cytometry Part A 2015;87:636–45.

41. Das R, Verma R, Sznol M, Boddupalli CS, Gettinger SN, Kluger H, et al.Combination therapy with anti-CTLA-4 and anti-PD-1 leads to distinctimmunologic changes in vivo. J Immunol 2015;194:950–9.

42. TopalianSL,Hodi FS, Brahmer JR,Gettinger SN, SmithDC,McDermottDF,et al. Safety, activity, and immune correlates of anti–PD-1 antibody incancer. N Engl J Med 2012;2012:2443–54.

43. Viola A, SalioM, Tuosto L, Linkert S, AcutoO, Lanzavecchia A.Quantitativecontribution of CD4 and CD8 to T cell antigen receptor serial triggering.J Exp Med 1997;186:1775–9.

44. Bodenmiller B, Zunder ER, Finck R, Chen TJ, Savig ES, BruggnerRV, et al. Multiplexed mass cytometry profiling of cellular statesperturbed by small-molecule regulators. Nat Biotechnol 2012;30:858–67.

45. Marvel D, Gabrilovich DI. Myeloid-derived suppressor cells in the tumormicroenvironment: expect the unexpected. J Clin Invest 2015;125:3356–64.

46. DeMets DL, Ellenberg SS. Data monitoring committees—expect theunexpected. N Engl J Med 2016;375:1365–71.

47. Behl D, Porrata LF, Markovic SN, Letendre L, Pruthi R, Hook C, et al.Absolute lymphocyte count recovery after induction chemotherapy pre-dicts superior survival in acute myelogenous leukemia. Leukemia 2006;20:29.

48. Mackall CL, Fleisher TA, BrownMR, Andrich MP, Chen CC, Feuerstein IM,et al. Age, Thymopoiesis, and CD4þ T-lymphocyte regeneration afterintensive chemotherapy. N Engl J Med 1995;332:143–9.

49. Wei SC, Levine JH, Cogdill AP, Zhao Y, Anang NAS, Andrews MC, et al.Distinct cellular mechanisms underlie anti-CTLA-4 and Anti-PD-1 check-point blockade. Cell 2017;170:1120–33e17.

50. TopalianSL,Hodi FS, Brahmer JR,Gettinger SN, SmithDC,McDermottDF,et al. Safety, activity, and immune correlates of anti-PD-1 antibody incancer. N Engl J Med 2012;366:2443–54.

51. Huang AC, Postow MA, Orlowski RJ, Mick R, Bengsch B, Manne S, et al. T-cell invigoration to tumour burden ratio associated with anti-PD-1response. Nature 2017;545:60–5.

52. Vallacchi V, Vergani E, Camisaschi C, Deho P, Cabras AD, Sensi M, et al.Transcriptional profiling of melanoma sentinel nodes identify patientswith poor outcome and reveal an association of CD30(þ) T lymphocyteswith progression. Cancer Res 2014;74:130–40.

53. Fischer K, Voelkl S, Heymann J, Przybylski GK, Mondal K, Laumer M,et al. Isolation and characterization of human antigen-specific TCRalpha betaþ CD4(-)CD8- double-negative regulatory T cells. Blood2005;105:2828–35.


Greenplate et al.




54. Priatel JJ, Utting O, Teh HS. TCR/self-antigen interactions drive double-negative T cell peripheral expansion and differentiation into suppressorcells. J Immunol 2001;167:6188–94.

55. Zhang D, Yang W, Degauque N, Tian Y, Mikita A, Zheng XX. Newdifferentiation pathway for double-negative regulatory T cells thatregulates the magnitude of immune responses. Blood 2007;109:4071–9.

56. Ebert PJ, Cheung J, Yang Y, McNamara E, Hong R, Moskalenko M, et al.MAP kinase inhibition promotes T cell and anti-tumor activity in

combination with PD-L1 checkpoint blockade. Immunity 2016;44:609–21.

57. Chevrier S, Levine JH, Zanotelli VRT, Silina K, Schulz D, Bacac M, et al.An immune atlas of clear cell renal cell carcinoma. Cell 2017;169:736–49 e18.

58. Wong MT, Ong DE, Lim FS, Teng KW, McGovern N, Narayanan S, et al.A high-dimensional atlas of human T cell diversity reveals tissue-specific trafficking and cytokine Signatures. Immunity 2016;45:442–56.






2019;7:86-99. Published OnlineFirst November 9, 2018.Cancer Immunol Res Allison R. Greenplate, Daniel D. McClanahan, Brian K. Oberholtzer, et al. Double-Negative T Cells Present across Human Tumor TypesComputational Immune Monitoring Reveals Abnormal

Updated version

10.1158/2326-6066.CIR-17-0692doi:

Access the most recent version of this article at:

Material

Supplementary

http://cancerimmunolres.aacrjournals.org/content/suppl/2018/11/09/2326-6066.CIR-17-0692.DC1

Access the most recent supplemental material at:

Cited articles

http://cancerimmunolres.aacrjournals.org/content/7/1/86.full#ref-list-1

This article cites 54 articles, 11 of which you can access for free at:

Citing articles

http://cancerimmunolres.aacrjournals.org/content/7/1/86.full#related-urls

This article has been cited by 9 HighWire-hosted articles. Access the articles at:

E-mail alerts related to this article or journal.Sign up to receive free email-alerts

Subscriptions

Reprints and

[email protected]

To order reprints of this article or to subscribe to the journal, contact the AACR Publications Department

Permissions

Rightslink site. Click on "Request Permissions" which will take you to the Copyright Clearance Center's (CCC)

.http://cancerimmunolres.aacrjournals.org/content/7/1/86To request permission to re-use all or part of this article, use this link



http://cancerimmunolres.aacrjournals.org/lookup/doi/10.1158/2326-6066.CIR-17-0692

http://cancerimmunolres.aacrjournals.org/content/suppl/2018/11/09/2326-6066.CIR-17-0692.DC1

http://cancerimmunolres.aacrjournals.org/content/7/1/86.full#ref-list-1

http://cancerimmunolres.aacrjournals.org/content/7/1/86.full#related-urls

http://cancerimmunolres.aacrjournals.org/cgi/alerts

mailto:[email protected]

http://cancerimmunolres.aacrjournals.org/content/7/1/86


computational immune monitoring reveals abnormal double ... · t cells were abnormal and...

Documents