Automated tissue image analysis

Automated Tissue Imaging Systems (ATIS) are computer-controlled automatic test equipment (ATE) systems classified as medical device and used as pathology laboratory tools (tissue-based cancer diagnostics) to characterize a stained tissue sample embedded on a bar-coded glass slide. After careful tissue sample preparation on to a slide, characterization of the sample is based on detection of cellular traits (color, shape, and numeration) observed in high-powered microscopic visual images (so-called bright-field microscopy).

A typical ATIS application includes measuring aggregate cellular activity or expression in a breast cancer tumor (carcinoma) biopsy to determine stage of disease for the appropriate course of treatment prescription. The technologies utilized in all the seven basic functions of Automated Tissue Imaging Systems are state-of-the-art and includes: Specimen Preparation; Image Acquisition; Image Analysis; Assessment Reporting; Data Storage Management; Network Communication, and System Self-Diagnostics.

Peripheral system equipment includes, but is not limited to, automated staining, hybridizing, pressurizing, and heating of the tissue sample. The integrated system software is complex and highly regulated, as is the entire system, by the U.S. Food and Drug Administration (FDA), Center for Devices and Radiological Health (CDRH), under Code of Federal Regulations (CFR) Title 21. As such, stringent compliance to validation and verification (V&V) methods with respect to safety, efficacy, and documentation is paramount.

Automated Tissue Imaging Systems (ATIS) redefine typical Histology Laboratories (HL), in a similar way that ATE systems redefined manufacturing testing, by applying Lean Manufacturing (LM) & Six Sigma (SS) tools to the Histology Laboratory production floor. Production in this context fundamentally means turn-around time and diagnosis quality. Although there are exceptions, typical HL today do not utilize modern industrial manufacturing lean methods (concurrent engineering methods) that provide world class manufacturing environments in other industries, they (HL) lack LM/SS methodology, automation, and connectivity. ATIS provides the automation component of a modern, safe, effective, and efficient Histology Laboratory system.

Applications

Breast cancer diagnosis

Large number of ATIS applications are for testing gene amplification or protein over expression to assess HER-2 ErbB-2 status and determine eligibility for appropriate adjuvant breast cancer treatments. There is good reason why this particular application is of large interest, mainly, that ATIS were conceived for the purpose of significantly reducing the variability involved in the manual and complex process of characterizing breast cancer tumors.

ATIS significantly reduce the procedural uncertainty involved in characterizing tumors, and hence, determining whether or not a specific type of treatment is appropriate when compared to the same procedure done manually by various histotechnologists. Adjuvant treatments may include chemotherapy, hormone therapy, radiation therapy, pharmacotherapy (Trastuzumab, antibody therapy), or biological therapy.

The reduction in uncertainty is attributable to the inherent consistency of machine action. The word “significantly” above is not used in the statistical sense, but in the fact that even a miniscule reduction of process variability can lead to few cancer cases whereby specific therapeutic treatments are not bypassed (so-called equivocal cases), and are then found to be beneficial. Test procedures include immunohistochemistry, in situ hybridization, and polymerase chain reaction having test target receptors ErbB-2, NR3A1, NR3A2, NR3C3, Ki-67, p53, and ErbB-1.

other applications

Lung cancer diagnosis
Liver cancer diagnosis
Colon cancer diagnosis
Prostate cancer diagnosis
Kidney cancer diagnosis
Lymph cancer diagnosis
Membrane research - quantify intensity of membranous immunohistochemistry stains.
Nuclear research - quantify intensity of nuclear immunohistochemistry stains.
Cytoplasmic research - analyze cellular cytoplasm staining.
Micro-Vessel Density research - analyze micro-vessels and density.
Rare Event in Tissue research - detection of rare positive cells in stained tissue sections.
Rare Event in Cytospin research - detection of rare positive cells in stained cytospin preparations.
DNA Ploidy research - analysis of DNA ploidy status.

Technologies & Methods

Classified by FDA as medical devices, ATIS fall into the general instrumentation category of automatic test equipment and are subject to the same basic principles of design, development, validation/verification, documentation, and support as do non-medical automatic test equipment (ATE) systems. These systems generally undergo computer-aided design and simulation, prototype development, validation & verification of design, various operation and specification documentation throughout the process, and manufacturing & customer support not unlike a new product development process.

What differentiates ATIS from more traditional-type ATE systems, for example the Agilent (HP) 3070 with multi-site test capability for testing electronics hardware and firmware, is the value placed in the unit under test (UUT) as an inanimate object versus urgency of an animate-related UUT. Of course, in the latter case, regulatory controls and validation/verification play a significantly stronger role in development.

ATIS have seven basic functions (sample preparation, image acquisition, image analysis, results reporting, data storage, network communication, and self-system diagnostics) and realization of these functions requires complex system integration of hardware and software from several state-of-the-art technologies in engineering disciplines that include chemical, computer science, electrical, electronics, hydraulics, mechanical, optical, pneumatics, and quality.

1. Preparation

The preparation of the quantity-limited tumor specimen is absolutely critical in successful characterization and generally, for example in situ hybridization say fluorescent or FISH (the so-called gold standard for HER-2 status), encompasses two basic processes over a period of a few days.

First process involves a complex multi-step procedure that includes cutting the breast cancer tumor or biopsy to appropriate size (typically 4mm), 24-hour fixation in buffered formalin, ethanol-xylene dehydration, paraffin embedding (heated), and mounting thin (typically 4um) slices onto at least two barcoded slides (control & test).

The second process involves staining which includes preparation of reagents, equilibration of reagents, removal of paraffin from slides, rehydration, pre-treatment, enzymes digestion, denaturation & hybridization, wash series, and mounting. Any variation or inconsistency introduced in these procedures from process to process or case to case will result in unpredictable irregularities and uncertainties in the outcome of the test or analysis findings. It is precisely these inconsistencies in analysis results that led to and motivated the development of automated tissue imaging systems.

It is estimated (based on Mod Pathol. 2005 Aug;18(8):1015-21) that 8% of HER-2 status findings have major scoring discrepancies (i.e. - non-amplification versus amplification), meaning that 8% of the cases are either deprived or robbed of proper treatment, or treated with an inappropriate therapy which could include chemo, hormone, radiation, Trastuzumab/Herceptin, or surgery.

2. Acquisition

Image acquisition involves digitizing photographic two-dimensional microscopic views of the stained specimen on a glass slide. The photographs are taken by a set of three charge-coupled devices (CCD, an array of photoelectric light sensors) color cameras integrated in a microscope field of view then digitized into ultra-high resolution terapixel (1012) digital computer images. A color filter array (Bayer filter) in the 3-CCD camera is made to arrange a red filter, a green filter, and a blue filter over a grid of sensors which output an electric signal in response to the light impinged upon it. Because each sensor input is color-filtered, each sensor will essentially measure the analog intensity of incident light at a particular frequency (red, green, or blue). The combination functions as a pseudo-pixel.

All these sensors require calibration so that each measurement is with respect to the same reference source. CCD calibration techniques can be single-point or multi-point. Single-point is simple but can not correct for non-linear effects across the spectrum. Multi-point is more complex, using B-spline fitting techniques. The analog signal or intensity measurement from the sensor undergoes signal conditioning and sampling at high frequency for conversion from an analog scale to a digital scale (for computer use as 0-1 bits) and to eventual image file formation (in Bitmap format or a zoo of other formats such as TIFF, JPEG, etc) for computer processing and storage. This is the digitizing process.

Modern high performance imaging applications require FPGA-based coprocessors system to supplement traditional DSP performance. For example, a 7x7 2-dimensional preprocessing filter kernel applied to broadcast HDTV 1080p video at 1920 by 1080 resolution at 30 frames per second and 24 bits per pixel requires more than 9 Giga MACS per second. This is more than the fastest commercial DSP can provide.

For video compression systems, FPGA coprocessing architectures are better solutions compared to platforms based on multiple DSP’s. Implementations of high-definition broadcast quality encoding utilizing video codecs MPEG2, MPEG4, and H.264 with a single FPGA and DSP solution are better.

The FPGA can implement algorithms that require the most cycles on the DSP, including the motion estimation block, entropy coding, and the de-blocking filter. The DSP can focus on algorithms that are more control oriented and better mapped to a C-code implementation. Newer entropy coding techniques, such as Context-Adaptive Variable Length Coding and Context Adaptive Binary Arithmetic Coder, are best realized as hardware accelerated blocks on the FPGA.

At 24 bits per pixel, 224 (over 16 million) different combination mixtures of red, green, and blue attributes leading to very large file sizes unsuitable for file transfer over a network but very suitable for discriminating minute color differences in the specimen image. It is this discrimination or resolution capacity, unmatched by humans, that makes digital image analysis a powerful pathology tool in the quest toward greater consistency. The digitizing process allows quantification of a finite number of well defined measurements, as opposed to, analog based vision having infinite degrees of subjectively defined measurements among histotechnologists.

3. Analysis

Image analysis involves complex computer algorithms which identify and characterize cellular color, shape, and quantity of the tissue sample using image pattern recognition technology based on vector quantization. Once the sample image has been acquired and resident in the computer’s random access memory as a large array of 0’s and 1’s, a programmer knowledgeable in cellular architecture can develop deterministic algorithms applied to the entire memory space to detect cell patterns from previously defined cellular structures and formations known to be significant.

The aggregate algorithm outcome is a set of measurements that is far superior to any human sensitivity to intensity or luminance and color hue, while at the same time improving test consistency from eyeball to eyeball.

For example, the process of signal enumeration in FISH (as described above in Preparation), is a scoring scheme that entails counting red and green signals (colored dots inside well bounded non-necrosis nuclei) representing hybridized ErbB-2 probe with normal and overly amplified ErbB-2, as well as, the control probe hybridizing to the middle of Chromosome 17, respectively. Signal enumeration (counting) is a subjective process among histotechnologist who may debate as to what constitutes an ambiguous nuclear border. No such ambiguity exist in an algorithm, whether clinically right or wrong, the algorithm will follow a deterministic set of rules to yield consistent outcomes, in this case, count the number of red and green pseudo-pixels within the border.

4. Reporting

Communicating test results involves presenting data (in text and graphic forms) to the system user in a format that is not only friendly to oncologists or qualified system users for verification and further analysis, but should also be as realistic as the actual specimen and not introduce additional artifacts. Specimen artifacts already exist including edge, retraction, thermal, crush, and decal artifacts. The media for data presentation is largely high quality computer monitors but can also include printers.

An appropriate type of high quality display for ultra-high resolution image acquisitions are High Definition (HD) monitors or televisions commonly used in operating rooms that employ 3-CCD telescopic camera head on endoscopes to display surgical areas of interest inside the body. These high definition monitors or televisions generally use progressive (non-interlaced) scanning techniques at refresh rates between 24 to 60 Hz.

A typical wide-screen HD device has an aspect-ratio of 16:9 which has been found to offer a more natural, panoramic view since human horizontal field of view is wider than the vertical field and therefore causes less surgeon fatigue during long procedures. A hi-def device can also provide improved depth perception and improved recognition of landmarks or rare-event sightings that may go undetected by computer algorithms. A 1080p (progressive) HD signal requires no interlacing conversion for a 1920x1200 HD monitor, which means that the vertical lines are all painted sequentially in one frame and hence minimizes possible noise between adjacent lines during the second scan of an interlaced process for fast moving images.

Furthermore, as hospitals and laboratories plan to adopt HD technology for their endoscopic surgery programs, compatibility with both existing and future technologies will be a critical factor in assessing the overall cost of ownership. Planning must include whether an HD system under consideration will be compatible with existing components, or will require the purchase of entirely new HD components. It must also include whether the system will readily accommodate future generations of HD technologies and components as they are developed. This includes touchtable display technology.

Computer printers, as relatively low image resolution devices, are used mostly to present final test reports (pathology reports) that could include text and graphics to match department, laboratory, or hospital formats. A typical pathology report may include: laboratory name, address, and logo; patient demographics; a graphical display of the antibody markers (including the control marker); a graphical display of bar charts representing numerical results; tabular representation of assays results with reference ranges; a comment section; and a system user signature.

An additional reporting feature of modern Automated Tissue Imaging Systems is the ability to provide real-time Statistical Process Control (SPC) metrics of the applicable Histology Laboratory processes undertaken by the automated system. For example, ATIS reporting includes real-time laboratory work flow control charts which monitor the on-going procedure in reference to all other previous procedures metrics and alerts when the current procedure is “out-of-control” with respect to process variability or any other desired metric. SPC, a Lean Manufacturing and Six Sigma tool, is but one of the tools available in sophisticated automated systems in a manufacturing or production environment.

5. Storage

Storage of the acquired data (graphical digital slide files and text data) involves saving system information in a data storage device system having well-defined schema and hierarchy for reference traceability, fast retrieval, and overall management. A fibre channel storage area network SAN is an appropriate data storage device system.

When ATIS is implemented in a laboratory with an existing database, or Laboratory Information System (LIS), the two storage systems may not be compatible in terms of content or format and will require integration. ATIS have several ways of integrating to LIS or LIMS depending on the size and activities of the laboratory sites.

Open architecture using industry-standard Web Services (Web Application Programming Interface, API) to communicate between clients and servers using Extensible Markup Language (XML) messaging having Simple Object Access Protocol (SOAP) standard provides best flexibility. For example, a laboratory already using barcodes for slide identification could benefit from established industry API standards for barcodes and simple implementation by having internal IT personnel setup appropriate internal and ATIS servers (SQL based for interacting with a database table manager such as MS SQL server). SOAP would be used to exchange XML messages over the network using http/https. ATIS could take requests from users via TCP/IP sockets to privately interact with the MS SQL server. Another common example providing a standard integration solution is a laboratory that using the international industry standard Heath Level Seven (HL7), an ASCII based protocol for participants of the health management community communicate.

6. Communication

Sharing saved system information among the medical community involves linking various computers located throughout a facility on a local area network using fibre channel storage area network SAN with fibre channel switches and host bus adaptors (HBA), or computers elsewhere throughout the world using an encrypted tunneling technique called Virtual Private Network (VPN) on public network (internet), both which would facilitate real-time group analysis. This is state-of-the-art connectivity not unlike telepresence.

As mentioned in the Acquisition section above, the bitmap format is not suitable for file transfer over a network. The standard format used for image files captured by CCD cameras for scientific image processing is the Tagged Image File Format (TIFF) or BigTIFF (files greater than 40GB) format and is a more suitable format than BMP (used by MS Windows operating system). Fibre Channel HBAs are available for all major open systems, computer architectures, and buses, including peripheral component interconnect (PCI). Each HBA has a unique World Wide Name (WWN), which is similar to an Ethernet MAC address in that it uses an OUI assigned by IEEE standard. There are two types of WWNs on a HBA; a node WWN, which is shared by all ports on a host bus adaptor, and a port WWN, which is unique to each port. HBA speeds vary 2-8 Gigabits/second.

7. Self-Diagnostics

Built-In Self Test (BIST) is a Design For Test concept implemented in complex, sophisticated systems to provide health-checks of its components and general operation in real-time, including power-up self testing, continuous background monitoring, and post processing troubleshooting. System diagnostics includes but is not limited to testing electromechanical components such as linear actuator, mixers, pressure/vacuum status, heaters, and waste overflow. An intelligence library database structured as a failure reporting analysis and corrective action system (FRACAS), and a failure mode and effects analysis (FMEA) catalog provides guidance to system operator with root cause troubleshooting and repair upon BIST failures.

The idea is to provide the system operator with on-the-spot guidance to problems like: interface between instrument software and the laboratory’s Information system; or software upgrades that change the performance of the assay (e.g. assay cut-off); or virus that infects the device operating software.

System Control

Integrated system software for Automated Tissue Imaging Systems has to be reflective of the procedural work flow in Histology laboratories, hence, concurrent engineering integration is highly complex. For example, the following are system modules (which are also stand-alone system applications) found in a typical ATIS with peripheral options.

ATIS System Software Built-In-Self Diagnosis and Support Software iShare Connectivity Professional Software Rare Event in Tissue Application Software Rare Event in Cytospin Application Software Nuclear Application Software Micro-Vessel Density Application Software Membrane Application Software IOD Ploidy Application Software DNA Ploidy Application Software Cytoplasmic Application Software PR Application Software ER Application Software HER2Test Application Software Tissue Micro-Array Application Software Workstation Pre-Treatment Link, Module for Tissue Specimens Autostainer Plus Autostainer Plus Link Autostainer Link 48

This degree of integration complexity requires rigorous validation/verification compliance to one or more regulatory agencies. As was mentioned in the section on Technology & Methods, ATIS as an ATE generally undergo computer-aided design and simulation, prototype development, validation & verification of design, various operation and specification documentation throughout the process, and manufacturing & customer support. However, because of the critical importance placed in system software by regulatory agencies and the inherent integration complexity of ATIS, software design V&V is discussed in the Safety and Efficacy section below.

Regulatory

Areas of concerns in the regulation of Automated Tissue Imaging Systems includes, but is not limited to: safety; efficacy; and documentation.

1. Safety and Efficacy

A search for software-related Adverse Event Reports in the FDA MAUDE Database will quickly reveal over eight thousand software-related medical device problems which range from Patient Outcome none to Patient Outcome Disability to Patient Outcome Death. Although in the case of an Automated Tissue Imaging System the patient never comes in direct contact with instrumentation, an improper diagnosis of the patient’s tumor as a result of faulty hardware, software, or both can either deprive appropriate cancer therapy or induce unnecessary harmful treatment, in either case the result can be death.

Some of the recognized standards bodies in the industry includes: DHHS – Department of Health and Human Services; ANSI – American National Standards Institute; NCVHS – National Committee on Vital and Health Statistics (United States); CHI – Consolidated Health Informatics Initiative (United States); DICOM – Digital Imaging and Communications in Medicine; HL7 – Health Level 7; X12 – Accredited Standards Committee; IOM – Institute of Medicine (United States); and CISB – United Kingdom Clinical Information Standards Board.

An on-going industry problem has been world-wide standardization of Histology Laboratory procedures. Specifically, process inconsistencies introduced by histotechnologist in sample preparations from case-to-case and lab-to-lab. The College of American Pathologist (CAP) can only impose standard operating procedures within each lab but not between them. A CAP/ISO working advocacy group is necessary to bring about enforceable world standards. Standardization is an ever growing necessity for efficacy improvements and progress in general.

FDA 21 CFR Part 820 Quality System Regulation; FDA 21 CFR Part 864; ISO 13485:2003 Medical Devices - Quality Management Systems; ISO 9001:2000 Quality Management Systems; SOR/98-282 Canadian Medical Device Regulations; and European Union 98/79/EC (IVD Directive) all have direct influence on Automated Tissue Imaging Systems and their peripherals. For network connectivity: ISO/IEC80001, Joint IEC TC62a, and ISO TC215.

1.a. Software V&V

Specific to software and firmware V&V is FDA 21 CFR Part 820.30 Design Controls; May 11, 2005 Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices; and January 11, 2002 General Principles of Software Validation. Others include IEC-61508 ; IEC-60601; IEC-62304 ; AAMI SW68:2001 Medical device software - Software life cycle processes; and ISO-14971 Medical devices - Risk management - Part 1: Application of risk analysis.

Design controls (21CFR820.30) is a general requirement for controlling the hardware and/or software design process. The code requires a design & development plan; inputs (specs); outputs (features); personnel review meetings; verification; validation; transition to manufacturing; design changes; and documentation (design history file).

Software V&V relates to efficacy. In a nutshell, software validation answers and documents the question: is it the right software? while software verification answers and documents the question: is the software right? Software is evaluated and reviewed against the software specifications during the ongoing development of the device design. When a final prototype is available, the software and hardware are validated to make certain manufacturer specifications for the device and process are met. Before testing the software in actual use, the detailed code should be visually reviewed versus flow charts and specifications. All cases, especially decision points and error/limit handling, should be reviewed and the results documented.

In all cases, algorithms should be checked for accuracy. Recalls have occurred because algorithms were incorrectly copied from a source and, in other cases, because the source algorithm was incorrect. During the development phase, complex algorithms may need to be checked by using a test subroutine program written in a highorder language, if the operational program is written in a lowlevel language. The validation program is planned and executed such that all relevant elements of the software and hardware are exercised and evaluated. The testing of software usually involves the use of an emulator and should include testing of the software in the finished device.

The testing includes normal operation of the complete device; and this phase of the validation program may be completed first to make certain that the device meets the fundamental performance, safety and labeling specifications. The combined system of hardware and software should be challenged with abnormal inputs and conditions. These inputs and conditions includes: operator errors; induced failure of sensors and cables or other interconnects; induced failure of output equipment; exposure to static electricity; power loss and restart; simultaneous inputs or interrupts; deliberate application of none, low, high, positive, negative, and extremely high input values. Design validation shall include software validation and risk analysis (RA is not included in 21CFR820.30, however, ISO-14971 is a good reference).

General heuristic software V&V testing might include one, a combination, or all of the following test techniques: functional testing; specification-based testing; domain testing; risk-based testing; scenario testing; regression testing; stress testing; all-pairs testing; combination testing; user testing; state-model based testing; high volume automated testing. Risk-based testing is helpful meeting FDA risk analysis requirement while specification-based testing might assist in compliance of 21CFR820.30(f).

1.b. Test Cases

A test case exercises one particular situation or conditions of the system being tested, it describes what to test and how. IEEE Std 610 defines a test case as a set of test inputs, execution conditions, and expected results developed for a particular program path or to verify compliance with a specific requirement. The test plan and test cases should be developed based on a risk assessment. A risk assessment is done early in the validation process to determine the degree of validation necessary based on the identified risks, and then develop the test plan and test cases.

Each test case in the plan includes the input, expected output, actual output, acceptance criteria, whether the test passed or failed, the name or initials of the person performing the test, and the date the test was performed. Test cases also includes normal results (results within the “normal range”), abnormal results (unacceptable results or those outside the “normal” range) and boundary results or values. Boundary (Domain) testing is done at the spec boundary, at the limit, just below the limit, and just over the limit.

Validation records are retained. The records include documented evidence of all test cases, test input data and test results. Test results includes screen shots. For traceability purposes and to facilitate quality assurance review and follow-up, supporting documentation, such as screen shots, are identified to link them to the specific test case. Retained test cases that previously passed can be used later for regression testing.

2. Documentation

FDA in 1997 issued 21 CFR Part 11 which applies to all electronic records that are created, modified, maintained, archived, retrieved, or transmitted in companies or departments that work under FDA regulation. Medical devices must comply with these regulations. Specifically, system software with capability to manage digital images, electronic data, and documents, as found in typical Automated Tissue Imaging Systems.

Hence, for regulatory purpose, ATIS software which includes integrated image acquisition and analysis as well as client/server-based data management, will also need to provide the following compliance features: Full computer-generated audit trails; validated time stamps; version control of all files; advanced user access management; electronic long-term archive option; document life cycle management; and electronic signatures (optional).

21 CFR Part 11 restricts system access. Images, data sheets, diagrams, and text files saved in the system must be tracked via audit-trail and version control. Data can not be deleted, and anytime an image is opened for viewing, the system should make the user must indicate whether it is being opened as Read-Only or Not-Read-Only (new version then created).

References

[1]
[2] FDA MAUDE
[3] FDA CFR Title 21
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]