robotic microscopy and information technology to … · robotic microscopy and information...

Robotic Microscopy and information technology to increase accuracy, sensitivity and availability of blood cell analyses

V. Medovyi and A. Pyatnitskiy

Medical Computer Systems (MECOS) Company, Ugreshskaya 2 , Moscow, 115088, Russia Until recently, time-consuming and subjective visual microscopy was the only implemented way to analyze morphologically complex objects. Yet now, modern specialized systems of automated (robotic) microscopy and multi-purpose information technologies can modernize blood smear analysis methods. The advantages and disadvantages of manual microscopy, specialized robotic systems, and automated hematological analyzers are considered. The most effective is the usage of the robotic microscopy with visual control of the results by a doctor to analyze complex forms and the hematological analyzers to analyze simple forms. Some properties of robotic systems that make possible adaptation to the changeable characteristics of the biological material are considered. The usage of the digital models of specimens produced by robotic systems for establishing the representative model of the analysis technique, for ist full-scale test, and for telemedicine consultations of blood smears is discussed.The realization of the analysis technique on the base of universal microscope-scanner and specialized software for virtual slide analysis is also to be considered.

Keywords robotic microscopy; blood smear analysis; virtual slide; laboratory telemedicine; image cytometry; virtual microscopy, image analysis; pattern recognition; artificial intelligence

Development of the medical techniques of microscopic analysis of biomaterials goes along the following major lines: 1) New technologies of image formation in a microscope; 2) New technologies of a specimen preparation; 3) Automation of analysis technique – replacement of eyes and hands of a doctor during a specimen screening; 4) Informatization of the analysis technique - storage, transmission, protection, access, visualization of images and results. We shall consider the 3rd and 4th directions, which had recently shown the significant results for medical practice.

1. Relevance of the blood smear analysis automation

Textbooks use fuzzy qualitative terms to describe cell morphology because of their volatility and diversity. Until recently, time-consuming and subjective visual microscopy was the only implemented way to analyze morphologically complex objects. But visual microscopy cannot guarantee the necessary accuracy: - High complexity and poor ergonomics prohibit collecting representative samples of cells, while small samples reduce reliability of diagnosis; - Analysis quality control is difficult and limited; - Staff training is complicated because of low concentration and diversity of atypical cells; - Constant shortage of qualified personnel reduces analysis accuracy; - Important investigation area exists, inaccessible to any automatic analyzers, either manual microscopy: detection and identification of atypical cells and atypical morphology in cases of their low concentration in a specimen. Automation of morphological analysis of cells could implement quality standard in one of the most important diagnostically and the most backward methodologically areas of laboratory analysis. The task of automation of blood smear microscopic analysis has the half-century history. There were numerous attempts by many companies to develop corresponding device, but only in the last decade effective devices for routine medical application have been worked out. Such device is the complex robotic microscopy system (RMS) which consists of light microscope, electromechanical organs handling specimens, videocamera, pump for immersion oil, controls, computer, software and support accessories.

2. Characteristics of contemporary robotic microscopy systems

Efficient RMS’s have emerged thanks to the overall progress in accessories: devices for precise movement of a specimen under microscopy, enhancement of videocamera resolution and speed, computer power. RMS accessories are basically universal and many companies currently produce equipment of necessary level. The equipment determines performance of RMS. Despite the important role of equipment, the main tasks of blood smear analysis automation are solved by RMS software. The software algorithms determine the ability of the RMS to properly collect the sample and identify cell types. The software is highly specialized and developed for the particular analysis technique. Currently a few companies supply medical level RMS software for blood smear analyses. There may be more than 30 morphological types of cells in blood smears. In the vast majority of cases, almost the whole sample is represented by 5 “normal” types, the rest (“young” and “atypical” forms) are rare. The detection and

Current Microscopy Contributions to Advances in Science and Technology (A. Méndez-Vilas, Ed.)

© 2012 FORMATEX 775

identification of these rare forms is important in the diagnosis of infectious, cancer and other diseases. The RMS displays the collected sample of cells on a computer screen, a doctor can quickly see and fix errors of RMS automatic sorting (fig.1).

Fig.1 A doctor can quickly see and fix errors of RMS automatic sorting by simply pressing a key. There are some characteristics of different models of RMS produced by Cellavision and MECOS companies [1-4] in the table 1.

Table 1 Characteristics of different models of robotic microscopy system for analysis of blood smears.

Model Glasses load Output smears/hour

Leukocyte presorting

Cellavision DM1200

Up to 12 Up to 40 Diff 18

Cellavision DM96

Up to 96, automated

Up to 40 Diff 18

МЕCOS-C2/Hemo1

1, manual Up to 60 Diff 12

МЕCOS-C2/Hemo8

Up to 8, manual Up to 50 Diff 12

МЕCOS-C2/Hemo200

Up to 200, automated

Up to 50 Diff 12

It should be noted that the accuracy of automatic sorting of young and atypical forms of both Cellavision and MECOS models is low, which makes the distinction between "Diff18" and "Diff12"insignificant.

3. Robotic microscopy systems, automatic hematology analyzers and manual microscopy

Modern RMS’s remain at the level of systems that only support visual analysis. The presence of a doctor as the key party of cell morphology analysis is crucial presently, and will be so for the foreseeable future. RMS’s are radically different from automatic hematology analyzers for this property. The analyzers shall have a significant advantage in the amount of cells collected for the analysis (More than 10000 against up to 1000 in RMS). However, modern automatic hematology analyzers cannot identify cells of complex morphology, they only flag the fact of their appearance. In some diseases such reports may be false, or unusual cells may be missed, and a doctor cannot examine whether cells have been sorted correctly. A doctor is to set the minimum level of the analyzer detection of young and atypical forms. However, due to the constraints of manual microscopy the area of low concentrations has so far remained obscure. The routine manual microscopy due to its high labour intensity is almost always based on the sample of 100 leukocytes. Such a sample is insufficient to accurately assess leukocyte , leaving a significant probability that atypical



forms will be missed. The proportion of young and atypical forms is small in most cases, so the low accuracy of their RMS automatic sorting has only a minor influence on a physician’s productivity. Solution of the very difficult task of precise sorting of all types of cells could turn the RMS on the level of automatic analyzer. However, complete exclusion of a doctor from the process is unlikely to be appropriate because of the huge variety of pathological characteristics related not only to the morphological types of cells. Generally, automatic hematology analyzers and blood smear microscopy are used together, complementing and testing each other. Technological progress of automatic analyzers, which can now sort up to 6 types of leukocytes, and the expected widespread use of RMSs, shift visual morphological analysis into the area of complex and rare forms. Therefore, addition of a modern RMS to the process of blood smear analysis results in: 1) exact and rapid collection of samples (up to 500-1000 leukocytes with the probability of missing under 0.01, up to 1000 erythrocytes, up to 500-1000 platelets), 2) exact sorting of the majority of cells (more than 95%), 3) visualization of cell images for doctor’s control. Modern RMS is better than a doctor at: - loading blood smears; - specimen screening of different types and volumes; -gathering of cell sample; - cell sample viewing; - cell morphometry and densitimetry; - specimen quality control; - database forming. A doctor makes better than RMS: - evaluation of atypical morphology; - atypical and young cells sorting. Working together, an RMS and a doctor can achieve improvement in analysis exactness and productivity: RMS gathers cell samples and presorts cells; a doctor verifies and supplements RMS analysis results on computer screen. In general, an RMS generates the following advantages over manual analysis techniques: - Comfortable workplace without manual microscopy; - Labor productivity increase; - Advanced large volume screening for atypical cells detection and cell population morphology measurements; - increased sensitivity to young and atypical forms especially at their low concentration; - Full quality control; - Virtual slides production; - Integrated process of automation, quality control and staff teaching; - Up to 200 glasses automated load; -Up to 60 specimen per hour output; -Telemedicine tools.

4. Robotic microscopy systems and methods of densitometry and morphometry

To be useful in every day practice of a medical laboratory, an RMS must perform automatic review of a lot of blood smears with different quality of biological material, sample preparation conditions, distribution of a blood smear on a glass, influence of pathology on cell morphology, while fulfilling all the recommendations of the manual methods of analysis. Take a look at some details around this multifaceted task. If a sample of objects has stationary distributions for the values of the objects parameters, there are many standard methods for creating rules for object sorting: learning neural networks, Bayes decision rules, Support Vector Machine, and others. But in case of a blood smear the values of the cells’ parameters are strongly influenced by unsteady uncontrollable factors, mentioned above. That is why recognition methods based on stationary distribution can’t correctly sort the images of blood cells. A doctor can recognize the types of such diverse images of cells visually, abstracting from the influence of external factors on cell populations. Apparently, a doctor’s eyes use the information about the sustainable characteristics of the specimen’s cell populations. RMS MECOS-C2 uses such an assumption in its sorting algorithm. The algorithm determines the characteristics of the cell population of the given smear using the so-called automatic clustering method. Consider a simple example of this approach, when two types of leukocytes, eosinophyles and neutrophyles, are sorted on the basis of one measured value - optical density of cytoplasm in the blue channel (Fig. 2-4).



Fig.2.Eosinophyles (left) and neutrophyles (right) have variable characteristics.

Fig.3. Separation of eosinophyles and neutrophylesbased on optical density of cytoplasm in different smears is different, so the threshold value (shown as horizontal blue line) should be found each time de novo.

Fig.4. Automatic clustering of the optical density histogram of the given specimen allows to find the threshold of sorting specifically for this specimen.

More complex sets of signs are used to sort other cell types. The RMS MECOS-C2 uses morphometry&densitometry to deal with internal tasks of the cell type recognition. The device is an example of the successful application of morphodensitometry in practice, the same cannot be said about many of numerous studies on medical diagnostic application of morphodensitometry. Perhaps the advanced technology of RMSs would make more progress in addressing those challenges. For example, it can be assumed that using cell optical density to assess chromosome ploidy should consider the characteristics of a particular specimen. RMS



technology can be used to solve other problems of diagnostic morphodensitometry such as collection of representative training and examination samples of specimens and cells inside specimen. For statistical reliability of formed diagnostic rules, both types of samples shall have a capacity of not less than several hundreds. A sample of this size is often impossible to obtain without automation and informatization of microscopic studies as implemented in the RMS.

5. Digital models of specimens, representative tests, and telemedicine

Sorted sample of cell images may be regarded as concentrated digital model of the natural specimen (DMS). Such a DMS of blood smear has volume usually under 10 Mb and is suitable for remote visual telemedicine consultations, including through the Internet. RMS can also make full digital copies of a natural specimens - virtual slides (VS). Due to their significant volume, VS are used mainly for research purposes, quality control and staff training. RMS MECOS-C2 uses export of DMSs as cell galleries as well as special small-size VSs of screening trajectory to the MECOS Company service center as a component of RMS operation and modernization (Fig.5).

Fig.5. Export of digital models of a smear for quality control. Systematically supplemented by many laboratories, the DMS Database is used as the model of the blood smear analysis technique environment to optimize analytic functions of RMS. Accumulated sample of DMS provides with test representativeness unattainable under conventional full-scale tests (Fig.6). DMS Database can be used for parallel developing and examination of all stages of automated analysis, supporting cooperation between users and developers.

Fig.6. Accumulated sample of specimen digital models provides with test representativeness. A similar technology of creating a collective archive of "Reference DMS" can be used by customers for their own research purposes.



The more sophisticated analysis tasks will become, the more value takes on the simulation of a specimen population. The above technology of DMS making and usage supports solving of such tasks. The Internet resource MECOS-Virt along with other similar resources provides services for receiving, storage, remote access, telemedicine consultations of DMSs. RMS robotic functions allow to implement in-depth techniques and improved working conditions in a laboratory, laboratory networking RMS-DMS functions allow efficient use of intellectual resources of a hospital, the Internet RMS-DMS-MECOS-Virt functions offer opportunities to improve quality of analyzes by leveraging the highly qualified staff across the region and country as a whole (Fig.7).

Fig.7. Local and remote access to a database of digital models of specimens.

6. The results of the overall informatization of microscopic research

The main results of the last decade are: - Universal microscopes-scanners (UMS) for production of digital copies of specimens – virtual slides (VS) accessible for mass application were created; - Routine usage of virtual slides for remote telemedicine consultation has started. Table 2 presents the characteristics of VS proceedings by some contemporary universal microscopes-scanners and RMS.

Table 2.Characteristics of VS production by some models of UMS and RMS of different manufacturers.

manufacturer Model Type of system

Max objective

options Glass load up to

Karl Zeiss MIRAX SCAN UMS 40х Fluorescence 300 Olympus Scan^R UMS 100x All kinds of light-

microscopy 1 various formats

Leica SL801 UMS 40х 3D, Fluorescence 384 Aperio ScanScope UMS 100x 3D, Fluorescence 400 Hamamutsu NanoZummer UMS 40х 3D, Fluorescence 210 3DHISTECH Pannoramic UMS 43х - 250 Cellavision DM96 RMS 100x oil 96 Fraunhofer Hemacam RMS 100x oil 8 Clemex HemaCyto RMS 100x oil 8 Tissuegnostics HemoFAXS RMS 100x oil 8 MECOS MECOS-C2 RMS 100x 3D, oil 200

The speed of VS production in some models of UMS is higher than in the RMS models. It can be expected that in the coming years microscopes-scanners will become routine devices in ordinary laboratories. MECOS-C2 software is adapted for computer analysis of VS, manufactured locally or transferred over communication lines. This allows the usage of specialized software and universal microscopes-scanners instead of RMS. A similar concept applies to upgrade



some other labour-intensive visual techniques of analysis, see, e.g., [5]. It is also possible to implement the cloud version of specialized software for blood smear analysis. Usage of the cloud version lets have in laboratories only microscope-scanners, without buying the expensive specialized software.

References

[1] Briggs C., Longair I., Slavik M., Thwaite K., Mills R., Thavaraja V., Foster A., Romanin D., Mashin S. Can automated blood film analysis replace the manual differential? An evaluation of the CellaVision DM96 automated image analysis system. Int. Jnl. Lab. Hem. 2009, 31, 48–60.

[2] www.cellavision.com [3] Plyasunova S.A., Balugyan R.Sh., Khmelnitsky K.Ye., Medovyi V.S., Parpara A.A., Pyatnitsky A.M., Sokolynsky B.Z.,

Demyanov V.L., Nikolaenko D.S. Computer-aided procedures for microscopic analyses of blood smears: medical tests of MECOS-C2 complex. Клиническая лабораторная диагностика, 2006, №10, 22-24, 33-39.

[4] http://mecos.ru [5] Grabe N, Lahrmann B, Pommerencke T, von Knebel Doeberitz M, Reuschenbach M, Wentzensen N.

A virtual microscopy system to scan, evaluate and archive biomarker enhanced cervical cytology slides. Cell Oncol. 2010; 32(1-2):109-19.



robotic microscopy and information technology to … · robotic microscopy and information...

Documents