lessons learned from measuring cell response by quantitative automated microscopy fda workshop,...
TRANSCRIPT
Lessons Learned from Measuring Cell Response by Quantitative Automated Microscopy
Lessons Learned from Measuring Cell Response by Quantitative Automated Microscopy
FDA Workshop, Potency Measurements for Cellular and Gene Therapy Products, Feb 2006FDA Workshop, Potency Measurements for Cellular and Gene Therapy Products, Feb 2006
John T. Elliott, Alex Tona, Kurt Langenbach and Anne PlantNIST, Biochemical Science Division, Gaithersburg, MD 20899
John T. Elliott, Alex Tona, Kurt Langenbach and Anne PlantNIST, Biochemical Science Division, Gaithersburg, MD 20899
NIST MissionNIST Mission
• Founded in 1901, NIST is a non-regulatory federal agency within the U.S. Department of Commerce
• NIST's mission: To develop and promote measurement, standards, and technology to enhance productivity, facilitate trade, and improve the quality of life.
Using Cells as Measurement DevicesUsing Cells as Measurement Devices
Mammalian Cell
Extracellular Matrix
NutrientsGrowth Factors
Cell-Cell InteractionsScaffold Materials
Inputs Signals
Other Factors
TopographyMechanical Forces
CellStatus
Inflammation
ProliferationDifferentiation
Remodeling
Apoptosis
Biomarkers
Protein “X”
Tenascin gene
Cell morphology
Protein “Y”
Cell Cycle Progression
Cell “Meter”Cell “Meter”
Inputs Signals
Output Signals
Assay Standards/Reference Materials
Measurement
Standards/ Reference Materials
Known materials/conditions
Controls
Unknown materials/test conditions
Test
Cell Response
Quantitative CellMeasurement/
InstrumentationStatistics
Schematic of a Cell-based AssaySchematic of a Cell-based Assay
•Calibration standards•Data extraction standards•SOP •Choice of statistical test
•Quality specifications•SOP
•Highly controlled environment• ± control reference materials•SOP=Standard operating procedures
•Valid biomarker for cell function•SOP for cell handling •Assay validation
•SOP
Requires Validation Steps
Precision, Robustness and Accuracy in Cell-based MeasurementsPrecision, Robustness and Accuracy in Cell-based Measurements
Precision – reproducibility in replicates Metric- mean ± SD, CV
Robustness – long term reproducibility Metric- variance of a quality factor (i.e. Z-factor, S/N)
Accuracy – obtaining correct answer from run Metric- comparison to a certified reference material (i.e. length or fluorescence intensity standards).
Expect a distribution of cell responsesExpect a distribution of cell responses
Cell Shape Gene Activation (TN1-GFP)
Single cell clone of NIH3T3-TN1-GFP-fibroblast on TCPS
•Measuring the distribution of responses provides a more accurate representation of the cell population
Computer w/image processing software
CCD camera
Quad Pass Beam splitterObjective
Excitation Filter wheel
X-Y translation stage
Focus motor
Emission filter wheel
Excitation lamp
Computer w/image processing software
CCD camera
Quad Pass Beam splitterObjective
Excitation Filter wheel
X-Y translation stage
Focus motor
Emission filter wheel
Excitation lamp
Automated Fluorescence MicroscopyAutomated Fluorescence MicroscopyMulti-fluorophore imagingMulti-fluorophore imaging
Cell Shape
Nucleus 3rd marker
•Image data is information rich; multiparameter information•Requires image analysis to extract data
Automated microscopy allows: -Unbiased data collection from a cell population. -Can be less labor intensive than flow cytometry
Automated microscopy allows: -Unbiased data collection from a cell population. -Can be less labor intensive than flow cytometry
Reference Materials for Calibrating InstrumentsReference Materials for Calibrating Instruments
•Validates instrumentation is operating properly (i.e. dynamic range, lamp intensity, and linearity of response.)
Fluorescent glass reference materials (+/- control) under development.
Calibrating fluorescence microscopes for quantitative cell-based measurements, J. Elliott et al. Under preparation.
•Length standards•Optical Property Standards•Chemical Standards
•SRM Fluorescein solution•SRM Flow Cytometry beads•SRM Fluorometer (in prep)•SRM Fluorescent wavelength
NIST Standard Reference Materials (SRM)
Distributions and “Mean” value of Cell-based MeasurementsDistributions and “Mean” value of Cell-based Measurements
Cell Area (m2)
0
0.1
0.2
0.3
0.4
0 100000 200000 300000 400000 500000 600000
0
0.05
0.1
0.15
0.2
0.25
0 500 1000 1500 2000 2500 3000
Relative GFP Fluorescence
Rel
ativ
e C
ell
Nu
mb
erR
ela
tive
Ce
ll N
um
ber
Gaussian-like Response Distribution
Non-Gaussian Response Distribution
Cell Morphology Measurement
GFP Fluorescence Measurement
Specification for reproducibility of replicate means
(precision)
=647±44 m2
=73297±8300Average
mean intensity
(n=4)
Average mean
area (n=4)
(CV=0.07)
(CV=0.11)
mean
mean
Dependence of Accuracy and Precision on Cell Number.Dependence of Accuracy and Precision on Cell Number.
•Precision and accuracy of the average mean GFP intensity measurement are influenced by number of cells sampled and distribution shape.
0
0.1
0.2
0.3
0.4
0 100000 200000 300000 400000 500000 600000
Relative GFP Fluorescence
Rel
ativ
e C
ell
Nu
mb
er
mean
Non-Gaussian Response Distribution
0
0.1
0.2
0.3
0.4
0.5
0 500 1000 1500 2000
0
20000
40000
60000
80000
100000
0 500 1000 1500 2000
Mean inte
nsi
ty f
rom
replic
ate
sC
V o
f re
plic
ate
means
Cell Number Sampled
Cell Number Sampled
Accuracy
Precision
Must use this number of cells in measurement
GFP Fluorescence Measurement
Setting up a Minimal AssaySetting up a Minimal Assay
Processing conditions
Control Samples
+
+
+
+
-
-
-
-
replic
ate
s
p1 p2 p3 p4
p1 p2 p3 p4
p1 p2 p3 p4
p1 p2 p3 p4
•Control samples allow validation of biological measurement•Replicates allow uncertainty metrics to be determined
Z-factor as a Metric for Assay QualityZ-factor as a Metric for Assay Quality
•Average means and standard deviations are obtained from positive and negative control replicates.•Z-factors can be used to establish an assay robustness specification.
C e l l r e s p o n s e
+ C t r l- C t r l
D y n a m i c r a n g e o f a s s a y
9 5 % C o n fi d e n c e I n t e r v a l s ( ± 3
M e a n - M e a n -
(3-+ 3+)
|m-- m+|1-Z=
Reference: Zhang, et al. (1999) J. Biomol. Screen. 4, 67.
Using Z-factor to Evaluate an AssayUsing Z-factor to Evaluate an Assay
+Ctrl-Ctrl
Cell response
+Ctrl-Ctrl
Cell response
+Ctrl-Ctrl
Cell response
•Dynamic range is larger than confidence interval
•Sensitivity~1•Specificity~1
Z>0.5
•Dynamic range is similar to confidence intervals
•Sensitivity~0.95•Specificity~0.95
Z=0.5
•Dynamic range is smaller than confidence interval
•Sensitivity<0.95•Specificity<0.95
Z<0.5
(Requires threshold assumptions)
Dynamic range
Dynamic range
Dynamic range
Z-factor for a Morphology/Biomaterial AssayZ-factor for a Morphology/Biomaterial Assay
•Multiple Assays-> Robustness Specification: Z=0.50±0.05
•We use a cell morphology assay to ensure quality control of a manufactured cell culture surface.
0
0.05
0.1
0.15
0.2
0.25
0.3
0 2000 4000 6000 8000 10000
-Ctrl+Ctrl
Mean-
Mean+
Histogram Distributions
Cell Size (m2)
Fra
ctio
n o
f ce
lls
(31+ 32)
|m1- m2|1-Z=
Average Mean+=1686±148 (n=6)Average Mean-=5282±404 (n=5)
Z=0.53 (~200 cells/well)
-Ctrl+Ctrl
Cell areaTC polystyreneCollagen films
From Replicate Controls:
Dynamic range
Selectivity~0.95Specificity~0.95
0
0.2
0.4
0.6
0.8
1
1.2
0 1000 2000 3000 4000 5000 6000 7000 8000
Test
Control
KS Test and the D-StatisticKS Test and the D-Statistic
•The KS test is a non-parametric test for statistically comparing The KS test is a non-parametric test for statistically comparing distributions of data.distributions of data.•The The D-statisticD-statistic is the maximum absolute vertical distance is the maximum absolute vertical distance between two cumulative distributions.between two cumulative distributions.•It is sensitive to changes in distribution position and shape.It is sensitive to changes in distribution position and shape.•It varies from 0 to 1.It varies from 0 to 1.
Prepare Cumulative Distribution
D-statisticD=max(abs(c1-c2))
Cell Response
Sum
num
ber
of c
ells
Cell Response
Rel
ativ
e #
of c
ells
0
0.05
0.1
0.15
0.2
0.25
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Test
Control
Advantage of using a D-statistic over mean value differencesAdvantage of using a D-statistic over mean value differences
mean2
mean1
mean2
mean1
Yes(Z~0.5)
Measurable Difference?
Mean Value D-statistic
Cumulative DistributionsResponse Distributions
mean2mean1
+Ctrl-Ctrl
D
D
D
Cell Response Cell Response
Fra
ctio
n of
cel
ls
Sum
of
cells
0
1
No(Z=0)
No(Z=0)
Yes(Z~0.5)
Yes(Z>0)
Yes(Z>0)
Reference: Vogt A., et al. (2005) J. Biol. Chem. 280(19) 19078.
Statistical EvaluationStatistical Evaluation
Statistics can help decide if the observed difference between two measurements is likely to be caused by random chance.
•Statistics requires measurements with uncertainty values
• This means having replicate experiments (n>3 recommended)
•Statistical evaluations are most helpful in deciding if small differences are significant.
SummarySummary• Cells exhibit a distribution of responses
• A valid measurement of the distribution of cellular responses requires sampling an adequate number of cells.
• Internal positive and negative controls during assay measurement can be used to evaluate assay quality and robustness.
• Alternative methods to measure differences in cell response can take advantage distribution shape information.
• Statistical analysis requires measurements with uncertainty values. It is most useful for determining the significance of small measurement differences.
0
0.05
0.1
0.15
0.2
0.25
0.3
0 5000 10000 15000
0
0.05
0.1
0.15
0.2
0.25
0.3
0 5000 10000 15000
Fibrillar FilmsFibrillar Films Non-fibrillar FilmsNon-fibrillar Films
Rel
ativ
e nu
mbe
r of
cel
ls
Prepared 8.12.02
Rel
ativ
e nu
mbe
r of
cel
ls
Cell Area (m2)
Prepared 8.12.02
Cell Area (m2)
5 Replicate Films 1 year later5 Replicate Films 1 year later
5 Replicate Films 1 year later5 Replicate Films 1 year later
Reproducibility of Morphology Results
The response distribution is highly reproducible.
Using a D-statistic to Measure Changes in Cell Measurements.Using a D-statistic to Measure Changes in Cell Measurements.
Native Fibrillar Collagen Thin FilmsNative Fibrillar Collagen Thin Films
Side ViewSide View
Average 23±2 nm
Max. ~400 nm
Large fibrils (~200 nm dia, >20 m long)
Monomer/Small fibrils (~5 nm dia, <500 nm long)
~100 nm
50 m1 1 mm
AFM
Zm
ax =300nm
Zm
ax =100nm1 1 mm
AFM
Zm
ax =300nm
Zm
ax =100nm5 5 mm
AFM
Zm
ax =
30
0nm
Optical Microscopy
Automated Quantitative MicroscopyAutomated Quantitative Microscopy
Computer w/image processing software
CCD camera
Beam splitterObjectiveExcitation Filter wheel
X-Y translation stage
Focus motor
Emission filter wheel
Excitation lamp
Computer w/image processing software
CCD camera
Multi Pass Beam splitterObjective
Excitation Filter wheel
X-Y translation stage
Focus motor
Emission filter wheel
Excitation lamp
Multi-fluorophore imagingMulti-fluorophore imaging
Cell Shape
Nucleus 3rd marker
Advantages: -Unbiased data collection -Sample large number of cells -Multi-fluorophore imaging -Live cell imaging -Evaluate cells in real culture conditions
Advantages: -Unbiased data collection -Sample large number of cells -Multi-fluorophore imaging -Live cell imaging -Evaluate cells in real culture conditions