THE PARADOX OF CHOICEEvaluating & Selecting the Right Cytometer for your Lab

Ryan DugganUniversity of Chicago

Adapted from: http://ucflow.blogspot.com/2013/07/10-tips-for-purchasing-your-next.html

Key concepts• Western Culture: Choice = Freedom = Happiness

Paradox of Choice• Too many options produce paralysis.• We’re less satisfied with our decision because of FOMO. • This conflict becomes even harder because choices are

so good.

• This is essentially where we are today with cytometers.

• Some choice leads to competition, too much choice leads to paralysis.

The Paradox of ChoiceBy Barry Schwartz

Tyranny of the Default• People are lazy efficient.• When presented a default option that is good enough,

they will be less inclined to explore other options, even when other options are better.

• IE on Windows• Today, we purchase cytometers for which we have

familiarity. The “BD” lab or “Coulter” lab.

Instrument Choices• In ~1999, we purchased the 3-laser BD LSR. • At the time we had the choice between BD, Beckman-

Coulter, and Partec• Also the Guava instrument became available in 1998, but

it was too limited.• Today, we can choose from a number of additional

platforms, not just within those same 3 companies, but also new companies who joined the world of cytometry.

10-step process

1. Define Needs2. Query the Userbase3. Refine Needs4. Survey the Market5. Navigate Marketing

6. Create the Matrix7. Hands-on8. Social Networks9. Negotiate10. Year-end deals

Define Needs• Create a generic specification of the needed equipment based on YOUR perception.

• What applications do you see being run immediately, near future?

Query the Userbase• Survey the users to find

out what they need from THEIR perspective.

• Analyze recent historical data

• Talk to key Investigators• Research current

industry trends• Ignore much of what the

users say they need.

Refine the Needs• Find the middle ground• Realistic specification with “room to grow”

Survey the Market• Research what’s available

• Base entry for consideration on min. spec.

• Use organizational tools like Evernote™ or ELN to track contenders.

Navigate the Marketing• Learn to read marketing materials• Differentiate between Technical spec (what the instrument

has) and Performance Spec (what the hardware can do).• Marketing materials are fine for listing Tech. Spec. but

NOT Performance Specification.• Samsung found itself accused of artificially (and secretly) boosting

benchmark scores on its flagship phone to ensure it would outperform the competition.” - The Verge (http://goo.gl/5vuCs9)

Create the Matrix• List of specifications vs. available hardware• Use marketing material tech. spec.• Include budget as a spec.

  # of Lasers (Total Avail.)

Total Detectors

Field Upgradable

Multi-well Sampler Event Rate Total

Events/FileOptical

Upgrades Avail.

Instrument A 1-3 (3) 3-8 No Yes 2,000 100,000 No

Instrument B 2-4 (7) 4-12 Yes Yes 10,000 5,000,000 Yes

Instrument C 2-6 (12) 4-18 Yes Yes 20,000 10,000,000 Yes

Instrument D 2-4 (4) 4-10 No No 20,000 2,000,000 No

Instrument E 2-5 (8) 4-14 Yes Yes 20,000 10,000,000 Yes

Instrument F 1-3 (4) 3-6 No No 2,000 100,000 No

Hands-on• NO CANNED DEMOS• NO EXAMPLE DATA SETS• Real data, collected by you, analyzed by you.• Create your own performance metric if none exists.• Make THIS, a high level priority that will greatly influence

your decision

Get Social• Read reviews• Ask peers• Request OEM response for any negative feedback

• Avoid Gen. 0 evangelists.

Negotiate Purchase• Negotiate with multiple OEM’s regardless of intent to

purchase• Make sure OEMs are aware of their competition• Competitive bid may be required anyway• Get everything in writing• Include Training, extended warranty, shipping, installation,

free upgrades, etc…

Be Mindful of Year-end deals• If you can time it right, year-end numbers can be a

powerful bargaining chip.

HANDS ON – IN DEPTH

Hands on evaluation opportunities• Travel to company to test

• Difficult to prep samples – ship• Usage based on “ideal” situation• Includes canned bead demos which have variable utility• Limited time with instrument

• Travel to lab who currently has the instrument• Still some difficulty in running samples• Get a feel for real-life use of the instrument• Limited time with instrument

• Get in-house instrument demo (hopefully for more than 1 day)• Prep samples of various kinds to test different aspects

What to test• Performance

• Real event rate• Resolution• Linearity• Carryover• Standard FL Panel

(Data not shown)

• Ease of use• Startup• QC• Software• Shutdown• Administration

Performance – Event Rate

Performance - Resolution

4-peak ABC beads

Unstained lymphocytes

Dimly stained capture beads

Resolution limit theoretical peak

10th %-ile ABC

Med

ian

AB

C

Median ABC values

10th %-ile ABC values

90th %-ile of AutoFluorescence = 10th %-ile of resolution limit

# of Antibodies Bound

Freq

uenc

y

qNORM

Performance - Linearity

Linearity

Performance - Carryover

Ease of Use – Startup/Shutdown• There should be a startup/shutdown routine (automated

preferably).

FACSCanto

NovoCyte

Attune NxT

0 5 10 15 20 25 30 35

StartupQCDots on Plots

Ease of Use – QA/QC• Either completely managed or not at all.• BD FACSDiVa CS&T

• Fully managed, pass/fail, historical information at-a-glance.• Requires specific, single-source bead set• Can be time-consuming depending on adjustments needed

• MoFlo• DIY QC/QC• No built-in platform for tracking or pass/fail confirmation• Can be fast depending on operator experience

Ease of Use - Administration• Generally overlooked even on modern cytometers• User on-boarding• User tracking/permissions• BD FACSDiVa

• Available via Import/Export• Runs on Windows allowing for custom methods (labstats, iLab)

• Miltenyi MACSQuant• Runs in terminal mode so no custom user tracking options• Limited in usability for large groups

Hands-on Summary• You need to spend some time on the instrument• Derive experiments that test the things that are most

important to you.• Resolution, Event Rate, Carryover, Linearity• Small Particle Detection• Sample volume flow rate• Fluorescence spillover

• Usability features can be measured too• Cold start -> Dots on Plots• Presence/Absence of fully managed QA/QC• Presence/Absence of Administrative functions

Conclusion• Recognize when you’re being paralyzed by choice,• Expand your evolutions beyond your default.• We’re in a period of great innovation and choice• Pare down list of contenders according to tech. spec.• Gain hands-on experience with the short list• Quantitatively describe performance that matters to you.• Usability is not a luxury anymore, it’s a necessity.

Thank you• Instrument manufacturers who let me test their stuff

• Acea Biosciences• Beckman Coulter• Becton Dickinson• BioRad• EMD-Millipore• Handyem• Miltenyi Biotec• Propel Labs• Stratedigm• Thermo Fisher

UCFlow.blogspot.com/UC