Guidelines and Gatekeepers: Reproducibility in Flow Cytometry

Guidelines and built-in safeguards ensure the reproducibility of flow cytometry data.

Imagine the ability to screen through hundreds of cells, measuring a snapshot of their dynamic molecular activity in real-time and gathering vital but often elusive cell population information. Flow cytometry makes this possible. Unlike other scientific techniques that characterize pooled sets of cells, such as Western blots or PCR, flow cytometry enables researchers to study individual cells, and if desired, even sort and capture cells.

Captured cells can then be interrogated further in a number of downstream single-cell assays, including single-cell RNA sequencing. Flow cytometry and cell sorting allow researchers to capture subtle and previously unexplored cellular changes, whether in phenotype, function, metabolic profile, or gene expression, that may be masked when using other analysis methods. With the growing developments in flow cytometry and increasing concerns about reproducibility in scientific data as a whole, flow cytometry researchers have made strides to ensure the reliability of flow cytometry data.

This is kind of my space: trying to get reliability in flow cytometry

I have seen those reports [of irreproducible data], and [flow cytometry] can be exquisitely reliable when done correctly.

The Guides

In flow cytometry experiments, individually labeled cells flow in a steady current of liquid. As the cell passes in front of a laser, it causes the light to scatter in a way that can be measured and correlated to cell size. Simultaneously, the laser excites fluorophores associated with labeled antigens on cells. The excited fluorophores emit distinct wavelengths of light that can be used to discern differently labeled surface antigens or intracellular proteins. Today, flow cytometers can recognize 20 or more different fluorophores and detect multiple cell parameters at once.

Immunology was one of the first fields to embrace flow cytometry and it remains intricately entwined with the technique today. In the beginning, many immunologists did not understand how to fully navigate this powerful research tool.

To address these questions, Cossarizza and his colleagues at ISAC came together in 2017 to compose guidelines for how to use flow cytometry correctly in the field of immunology. ¹ These guidelines, written by a number of authors from both academia and industry, were widely accepted among researchers. In 2019, Cossarizza and ISAC submitted a new, updated set of guidelines for immunologists.² These guidelines are applicable across cell types and research fields. “In the very beginning, it was for people doing immunology. But at the end, if you have to study lymphocytes, or if you have to study monocytes, or if you have to do leukemia, lymphoma, whatever, the technology is the same. You're only changing the patient, the subject, the number of cells, but the technology is absolutely the same,” said Cossarizza.

On the clinical side, The International Clinical Cytometry Society (ICCS) also adheres to a set of flow cytometry guidelines. ³ “Scientists are members of these societies and they have the expertise in how you manage a project,” said Rudd Hulspas, technical director of process development at Dana-Farber Cancer Institute and founder of Cellular Technologies Bioconsulting, LLC.

Litwin, from Caprion Biosciences, chairs another large interdisciplinary group with representatives from academia, the pharmaceutical industry, clinical laboratories, the manufacturing industry, and the government, which is creating a new universal set of flow cytometry guidelines. The new guidelines would be a Clinical and Laboratory Standard Institute (CLSI) document, which would make these guidelines more widely recognized and authoritative.⁴

The new guidelines are approachable, with tiers of data validation to ensure data reproducibility. Litwin and her group recommend that every flow cytometry study have some level of validation proportional to the objectives of the study, with clinical studies requiring more validation steps. “If you're a researcher, and you want to generate reproducible results, then you can get away with doing less, but you should characterize at a minimum, what's the variability in the population,” advised Litwin.

Gatekeepers

Repeating experiments helps scientists to better understand meaningful trends within their data and identify conditions that affect their experimental results. For example, sample stability can influence data precision and reproducibility. If researchers are not immediately performing flow cytometry, but rather mining their way through their samples, the viability of their cells might change from day-to-day in storage and influence data outcomes.

This is particularly important for rare cell populations.“If you have an event, a subpopulation that's rare, or it's going to become rare with therapy or the disease state, then you should look at your lower limit and ask, ‘where do we become imprecise?’,” said Litwin. “Then decide how you're going to handle those results.”

Flow cytometry assesses cells randomly."Imagine that you have one cell positive in 1000. If you analyze, 2000 cells, you can find 1, 2, 3, 4, or 0. This means that you have to acquire a higher number of cells and you have to calculate the number of cells that you want and have to acquire according to some statistical low,” said Cossarizza. Researchers determine this statistical low or the minimum number of cells needed for an experiment by performing statistical tests. In particular, researchers using flow cytometry on rare populations perform a statistical Poisson distribution analysis to inform how many cells they need to collect to be confident in their data.

In flow cytometry, researchers use fluorochromes to identify characteristic antibodies bound to cells. Choosing reliable and bright fluorochromes enhances flow cytometry reliability. “If you want to detect rare cells, you need a very bright fluorochrome. If you want cells that are very common, I will say CD4 cells for example, you can use a fluorochrome which is not as brilliant as the others,” said Cossarizza. He noted that there are important guidelines for choosing fluorochromes based on sample cell type. To avoid misidentifying cells, researchers perform preliminary tests on panels of antibodies to identify the fluorochrome or combination of fluorochromes that consistently provides the best results for their cell population and experimental conditions.

To further ensure reliability and inform gating strategies, researchers perform control experiments to test their antibodies and fluorophores. They also routinely calibrate their flow cytometry instruments.

“The level of instruments is absolutely high. In this moment in my life, I have instruments from different companies, and I'm very happy with all of them. Every time that we analyze one sample, the same sample with different machines, the data are the same. This is what has to be,” said the University of Moderna’s Cossarizza.

When researchers know what type of flow cytometer they need, they can plan their experiments accordingly to optimize their data integrity. Some instruments analyze larger cells (up to 200 µm) or detect 50 molecules of fluorescence per cell. Other instruments recognize smaller cells (less than 80 nm) and detect 200 molecules per cells.“It depends on what you have to do. But I will say that for standard measures, if you don't want to run a Ferrari, or if you don't need to run a Ferrari on the highway, you can really run samples on all the machines today,” said Cossarizza.

Using different flow cytometers is not likely to cause errors in downstream flow cytometry data analysis. In 1984, flow cytometry experts set data standards to ensure that the output from any vendor’s instrument could be used with any analysis software ⁵. “They have a data standard file, whereas with other technologies, I don't think you can do that,” said Litwin.

To ensure the reproducibility of their flow cytometry data, researchers can perform both supervised and unsupervised analysis. If the data is reproducible, both techniques yield the same results. Researchers may also ask an objective colleague to reanalyze their data.

Safeguards

Cytometry was one of the first journals to set standards specifically for flow cytometry data, but many other journals now require investigators to post their gating strategies and raw data. These journals then send this material to be reanalyzed by reviewers. Some flow cytometry journals also require investigators to deposit their raw data into repositories. . “If you put the files in a repository, everybody can control the data,” said Cossarizza.

The field of flow cytometry is still evolving. Flow cytometers now recognize increasingly more fluorophores and detect multiple cell parameters at once. While the field of flow cytometry continues to advance, experts maintain that the fundamental principles of flow cytometry remain the same. “What we put in [the guidelines] is foundational for anything that you do. So, if you're doing 20 colors, you should still be looking at what we suggested for 8 or 18 colors and following those practices and then looking at how you can develop your best assay,” said Litwin.

With careful experimental design and precisely calibrated instruments, flow cytometry is a remarkably reliable source of powerful scientific data.

What we put in [the guidelines] is foundational for anything that you do. So, if you're doing 20 colors, you should still be looking at what we suggested for 8 or 18 colors and following those practices and then looking at how you can develop your best assay

We can be exquisitely sensitive; we can be exquisitely specific and precise. When we look like at all CD4 cells, there's no difference. But when we segregate out the subsets, we see a change in this population

And that's meaningful because we know based on the phenotype of the cell that they have a different biology. There is a response. That informs us really about the biology.