Digital Pathology and Tissue Image Analysis

2.1

Scanning Modalities

Whole-slide images can be created using bright-field (including polarized light), fluorescent, and multispectral imaging, and the scanner type should be paired with the appropriate slide staining technique ( ; ; ). Commercially available scanners that accommodate several modalities are available. However, instruments with increased scanning modalities often trade-off increased imaging capability for other potentially desirable functionality such as a higher scanning speed and/or larger number of slides that can be loaded and scanned in a single batch. Therefore, it is important to consider the typical ranges of study types to be evaluated and the potential workflow in the laboratory when selecting a scanner.

Bright-field whole-slide scanning is similar to standard bright-field microscopy and represents the most common and cost-effective WSI approach. While most scanners are capable of creating bright-field images, some models are optimized for high scan throughput (by allowing for large batch sizes of thousands of slides or continuous slide feeding) and have enhanced features that enable minimal oversight of the instrument by laboratory personnel during the scan run. These enhanced features include automated tissue recognition and focusing, prioritization of specific slides in the batch, and skipping of slides that fail quality control (QC) without interrupting the overall scanning process. Such aspects permit round-the-clock scanning, which makes WSI cost effective in high-throughput workflows. Similarly, fluorescent whole-slide scanning emulates fluorescent microscopy and is used to digitize fluorescently labeled slides (i.e., fluorescent immunohistochemistry [IHC], fluorescent in situ hybridization [ISH]). Multispectral imaging captures information in separate spectral channels across the spectrum of light and can be used in both bright-field and fluorescent scanning ( ; ; ; ). This form of imaging is particularly well suited in fluorescent applications to overcome issues due to tissue autofluorescence because the spectrum of each individual marker is separated (unmixed) ( ; ). It also provides an opportunity to use bright-field multiplexing IHC, especially when there is colocalization of two or more IHC biomarkers. Fluorescent or multispectral scanners typically require more user interaction and provide low-throughput scanning, and therefore are more suited to laboratory workflows focused on in-depth characterization of a smaller number of specimens as is done in a special techniques laboratory or to support discovery research.

The two most common approaches for creating whole-slide scans are image capture by tile or by line scanning ( Figure 12.1 ) ( ). After capture, the tiles or lines are digitally stitched together by software algorithms to create the virtual image of the entire scanned slide ( ; ).

2.2

Scanning in Focus

Scanners apply focus to slides along the so-called z-axis, which is the tissue expanse (i.e., thickness) perpendicular to the x–y plane in which the tissue section and coverslip are arranged. Slides may contain tissue with slightly varying thickness or uneven cover-slipping, making it more challenging for the scanner to capture the entire section in focus in a single plane of the z-axis. Whole-slide scanners employ a variety of approaches to maintain focus across the slide, from focusing on every tile, focusing on a subset of tiles, or to distribute multiple focus points across the slide independent of tile positions ( Figure 12.1 ) ( ; ). Increasing the number of focus points and strategically positioning them across the slide typically decreases the number of tiles with “out-of-focus” regions, but also increases scanning time due to repeated automatic focus adjustments. In an attempt to balance image quality with slide throughput, some scanners allow for the operator to specify the number of focus points. Line scanning instruments typically approach focusing by use of focus maps (a method also used in some tile scanners). Focus maps are generated by placing a network of focus points (automatically or manually generated) across the tissue section, which often results in faster scan times. Similar to tile scanning, the trade-off consists of having a higher potential error rate (i.e., out-of-focus areas) ( ; ). Most recently, scanning technology has been developed that incorporates continuous automatic refocusing ( ; ).

2.3

Scanning Capacity and Time

Slide capacity is an important consideration when selecting a whole-slide scanner. Different models vary from holding just a single slide to automatically loading slide after slide from a preloaded unit holding up to thousands of slides arranged in one or more trays ( ; ; ). Technological advances have reduced the cost of low-capacity (4- to 8-slide) instruments to a few thousand dollars. Some scanners can accommodate bigger specimens, such as whole tissue mounts and nonstandard glass slides holding large organ sections (e.g., bone, brain, eye) from nonrodents ( ).

Scan time is another key element that impacts the choice of slide scanner. As discussed above, the mechanical capabilities of scanners are critical determinants of the amount of time and effort required to create a whole-slide image. Scan time is also affected by the desired magnification, where the amount of time required for a scan increases with the desired magnification, size of the tissue section, number of imaging modalities used, and the number of planes collected along the z-axis ( ; ). In addition, data transmission rates from the sensor to the image file being generated also can impact overall scanning speed.

2.4

Scanning Magnification

Scanners are typically capable of creating scans at various magnifications specified by the user, and the specific magnification choices may be workflow dependent. Most scanners are limited to one or two objective magnifications (20× and 40×), but some instruments include a wider range of objectives (1.25× or 2.5×, 5×, 10×, 20×, 40×, 63× [dry or oil], or 100× [oil]). Additionally, some scanners can perform both dry objective and oil immersion scanning. While pathologists often find 20× scans sufficient for routine hematoxylin and eosin (H&E) and IHC tissue section review in a research setting, many institutions with established fully digital workflows and appropriate information technology (IT) infrastructure are routinely scanning at 40×. Common reasoning for this routine use of 40× is that a 40× objective is the highest one usually available on a traditional light microscope used to screen tissue sections in anatomic toxicologic pathology, and therefore whole-slide images created using 40× have approximately equivalent resolution to the maximum resolution typically available by traditional microscopy. In addition, standard 40× scanning avoids the workflow disruptions that would result when rescanning selected slides at 40×. The trade-off for establishing this equivalence is that 40× whole-slide images may take approximately 4 times longer to scan than 20× images, and the resulting files are proportionally larger ( ; ; ). For slides from Good Laboratory Practice (GLP)–compliant studies, regulatory guidance will need to be considered for the minimum scanning magnification when seeking to use whole-slide images to perform primary histopathologic evaluation or pathology peer reviews ( ).

Higher scanning magnifications are required for certain applications. Digitization of ISH should always be performed with at least 40× magnification to resolve individual ISH dots that may be less than about 0.5 μm apart ( ). Even then, some dots will remain slightly out of focus. Specialized scanners that can accommodate higher magnifications are now available (60×/63× or 100×, under oil) and are used regularly for WSI of blood smears and other cytological preparations ( ; ; ).

2.5

Scanning Resolution

Resolution determines the minimum distance at which two distinct objects can be identified as separate objects. For a purely optical system, such as a light microscope, the resolution is determined by the numerical aperture of the objective. In a slide scanner, which combines optical and digital components, resolution is determined both by the numerical aperture of the objective and the number of pixels per unit area of the camera sensor (termed “pixel density”) ( ). If this sensor density creates a lower resolution than the objective's numerical aperture, not all information from the optical projection is gathered.

Vendors usually express the resolution of their scanners in units of μm per pixel. Color depth is expressed as the number of bits per pixel allocated to a channel, which in turn specifies the total number of distinct colors possible in the image. Typically, a 40× bright-field whole-slide scan has a resolution of about 0.25 μm per pixel and 24-bit color depth. This means that a 1 mm ² area of a scanned section is equivalent to 384 million bits of information.

Lastly, the resolution of the monitor used to view the image can impact the effective resolution of the digital system. In addition, because in a digital pathology system the light path is not projected directly onto the observer's retina, perceived resolution of the viewed image is also limited by the observer's visual acuity, ambient light conditions, and distance from the monitor.

2.6

Color Preservation

One of the challenges of digital pathology is the preservation of color along the workflow ( ). The color captured during scanning may not match the original slide, and the colors displayed on a monitor may not accurately reflect the colors captured during scanning.

Aside from general color variability introduced by preanalytical variables (e.g., fixation and histologic processing), the act of scanning itself may introduce color variation. Two different scanners may be used to digitize the same slide, but the precise colors rendered by each scanner are likely to be slightly different, especially if they have not been calibrated in the same manner. Similarly, displaying a single scanned image via different viewing software may result in visible color differences. The impact of monitors on variability in color displays is discussed in depth later in this chapter.

To control for color variability, it is recommended that a test slide should be scanned with each scanned batch of slides, or at least daily. The test slide should have multiple tissues (e.g., a tissue microarray [TMA] slide of the major organs) with different structures having a range of affinities for the dyes applied in staining to demonstrate the full array of expected colors in a routine study. Color variations can then be compared to the test slide. If the same test slide is run with the scan batches, any color artifacts should be obvious to the reader.

To mitigate the impact of color variability after the slide has been scanned, various color normalization algorithms can be applied to the whole-slide image ( ). These transform the colors of a series of images to align to a single standard or reference slide. While many of the tools have demonstrated a reduction in color variability among images by a significant margin, it is important to detect and assess any distortions in the image that may potentially be introduced during the color normalization process, such as incorrect color representation. Notably, regulatory guidance regarding color normalization tools is lacking, and therefore, it is not clear if these means are suited for usage in a regulated environment.

Color preservation can be achieved by color calibrating the entire digital workflow. For example, the International Color Consortium (ICC) has released an open, vendor-neutral color management protocol (with the most recent version available on the ICC website; www.color.org ). Briefly, this protocol involves scanning a reference slide that contains known color attributes and then comparing those attributes to the color values produced by the scanner. Similarly, the colors displayed on a monitor can be compared to known color attributes of the reference slide as initially scanned ( ; ). In addition, tools are commercially available to perform color calibration of computer monitors. The landscape for color calibration is rapidly evolving.

Color preservation is a demanding aim, which is further complicated by the fact that there is variability in staining within a single laboratory, and among different laboratories ( ). Especially when scans are intended for image analysis, setting thresholds for certain stain colors may be an important part of the preanalytical work, and the quality of the final data can be greatly influenced by staining and color consistency. Likewise, manual review of whole-slide scans can be hampered by suboptimal color preservation along the digitization and display process.

2.7

Image Compression

Since WSI creates large file sizes, usually in proprietary formats, image compression is commonly used to facilitate file transfer and storage ( ; ; ). Many commercially available scanners compress images in JPEG (Joint Photographic Expert Group), JPEG 2000, or LZW (Lempel–Ziv–Welch) codecs (i.e., where a codec is a device or program that compresses data to enable faster transmission and decompresses received data), which can reduce file size. While smaller files save resources, such “lossy” compression algorithms (e.g., JPEG) should be used with caution as the loss of information that accompanies the compression is unrecoverable. For this reason, compressed files may not be suitable for pathology peer review purposes. Some scanners allow the user to determine the level of compression (on a quality factor scale of 0 to 1). The selection of this value balances image quality and file size. It is important to avoid application of successive rounds of JPEG compression to the same image file, as each round will result in further reduction of image quality, potentially introducing visible image artifacts such as an excessively grainy appearance, blockiness (particularly in homogeneous patches such as white space), and color distortions ( Figure 12.2 ). The standard JPEG and JPEG2000 standards are compatible with Digital Imaging and Communications in Medicine (DICOM) standards for image storage ( ).

It has been shown that modest image compression in the range of 1:7 has no appreciable effect on the ability of skilled observers to perform qualitative and semiquantitative assessments of tissue sections and cells ( ; ). However, fully quantitative assessments such as tissue image analysis and densitometric assessments may be sensitive to a loss of information caused even by visually imperceptible image compression ( ; ). In addition to image compression, whole-slide scanners often reduce file sizes by excluding scanned regions of the slide that do not contain tissue.

2.8

Pyramid Representation

Although a variety of methods exist to reduce the file size, as described above, a single whole-slide scan file commonly exceeds the working capacity of the RAM (random access memory) in a typical office computer. Therefore, the whole-slide image in its entirety cannot be displayed at maximum resolution on commercially available monitors. The limitations of typically available computational power can therefore be managed by creating whole-slide images as “image pyramids” consisting of a base layer collected at high magnification, such as 40×, and multiple “down-sampled” layers stored, for example, at 10×, 2.5×, 1.25×, etc ( Figure 12.3 ). The image pyramid approach makes use of available resources by displaying the down-sampled layer when the observer is viewing the image at low magnification and displaying a smaller field of view at high resolution when the observer wishes to view a portion of the specimen at high magnification. The more layers in the pyramid, the larger the file's size, but the more efficient the bandwidth usage during viewing. Some systems allow for the user to determine the number of image levels within the pyramid. Typically, there are three to four levels, balancing bandwidth requirements with overall file size.

Other information (“metadata”) related to the image typically are embedded into the overall file. Examples of common metadata include image properties, scanner acquisition details, organization of the file, color profiles, subject-specific data, an image of the slide label, and a low-resolution snapshot of the entire slide. These data typically add a small fraction to the overall file size.

2.9