NIH-BISTI Center for Grid-Enabled Medical Image Analysis (GEMIAC)

• Home
  • Overview
  • Collaborations
• Active Projects
  • Hi-Res Cardiac
  • Tumor Imaging
  • Very Large Datasets
  • Cardiac Cycle Modeling
• Knowledge Resources
  • Education
  • Graduate Seminars
  • Medical Imaging Workshop
  • Publications
• Facilities/Personnel
  • Facilities Core
  • Personnel

High-Performance Integrated Environment for Manipulation, Feature Analysis, and Visualization of Large Image Datasets

• Principal Investigator: Joel Saltz, M.D., Ph.D

Main Goals

1. To design and implement the core software infrastructure, as Grid services, that will enable efficient data access, data sharing, and data manipulation in collaborative multi-institution environment.
2. To develop methods to quantitatively categorize textures, to functionally classify materials (e.g. bone, tissue), to characterize changes in shapes and structures, and to browse large repositories of data.

Development of Software Infrastructure

The software infrastructure will make it possible to access to accumulated results from multiple imaging modalities and examine relationships between features observed in one or more datasets at multiple institutions. We have designed a framework that consists of a suite of coupled services. The design of the system is driven by a number of functionality and performance considerations. First, a researcher or an institution should be able to create a database of biomedical images on disk-based storage systems and make it available for remote access. Second, the database should be able to scale to very large datasets and to large number of clients by taking advantage of heterogeneous collections of clusters and multiprocessor machines. Third, multiple databases should be able to be joined in a federation so that clients can interact with and submit requests across multiple image datasets.

The framework is made up of two tiers of services. The objective of the first tier services is to make it easier for a researcher to set up an ad hoc data warehouse to manage and manipulate large datasets for biomedical images on disk-based storage systems. The second tier services extend the functionality implemented the first tier services to distributed environments so that multiple image data servers can be federated.

We implemented a prototype of the framework using the middleware tools, DataCutter and Mobius, developed by our group.

Mobius is a middleware framework designed for efficient metadata and data management in dynamic, distributed environments. It provides a set of generic services and protocols to support distributed creation, versioning, management of database schemas, on-demand creation of databases, federation of existing databases, and querying of data in a distributed environment. Its services employ XML schemas to represent metadata definitions and XML documents to represent and exchange metadata instances.

DataCutter is a component-based framework designed to support coarse grain data flow applications. Through combined task and data parallelism, the DataCutter runtime system enables efficient execution of networks of processing components, called filters, in a distributed environment. Application filters can be placed on different hosts to minimize communication over slow links and to reduce computation overhead by executing the filters on less loaded hosts. Moreover, multiple concurrent instances of a filter group, which consists of a set of filters collectively carrying out application processing, or multiple copies of a single filter can be instantiated and executed.

The prototype supports

1. A distributed metadata management component that keeps track of metadata associated with workflows, input image datasets, intermediate results check-pointed during execution of a workflow, and output image datasets. This component allows registration, management, and versioning of workflows and image information as XML schemas. Schemas defined by other researchers can be included in a schema for the workflow and image information for a study. This component also manages the metadata associated with image datasets generated by an execution of a workflow.
2. A storage component that efficiently manages distributed collections of disks in the system for storage and staging of collections of input images, images generated by intermediate stages of the image analysis workflow, and output datasets. This service supports distributed storage of large images and index generation and lookup for efficient data retrieval. It enables an XML virtualization of relational databases, as defined by XML Schemas. It allows user-defined data types to automatically manifest custom databases at runtime, and data adhering to these data types to be stored in these databases.
3. A distributed execution component, which supports efficient execution of image analysis workflow as a network of image processing components in a distribute environment. The network of components can consist of operations that retrieve subsets of multi-resolution datasets, multiple stages of pipelined operations, and parameter studies. We have developed image analysis applications and toolkits using the prototype implementation.

Image Processing for the Grid (IP4G) toolkit

This toolkit employs the Insight Segmentation and Registration Toolkit (ITK) and Visualization Toolkit (VTK) layered on the distributed execution component of the framework. It allows rapid development and distributed execution of biomedical image analysis applications implemented using ITK and VTK functions.

A Distributed Execution Environment for Analysis of DCE-MR Image Datasets

We have implemented a Haralick-based texture analysis method for analysis of Dynamic Contrast Enhanced Magnetic Resonance Images (DCE-MRI). This application enables texture analyses of image datasets obtained from DCE-MR studies on distributed collections of computer clusters and data storage systems.

Virtual Microscopy System

Current virtual microscopy infrastructure at The Ohio State University consists of an Aperio ScanScope slide digitizer capable of acquiring full color images at 200x and 400x magnification. The digital slides are stored on an Aperio web-enabled image server. We are developing infrastructure to establish a distributed pathology PACS (Picture Archive and Communications System) which is supported by the Mobius distributed metadata management system. The distributed PACS, in conjunction with DataCutter, IP4G, VTK/ITK, and application specific algorithms, leverages the computational resources on clusters for active data processing as well as parallel storage. The distributed PACS allows for fast and multi-client access to images as well as on-demand and batched image analysis.

Parallel Volume Renderer

During this first year we also have focused on the functional utility to maintain interactive rates for the user working on a local, thin client, residing in their office or laboratory.. This work involves the manipulation and visualization of three-dimensional volume reconstructions and four-dimensional datasets. Functional utilities include interactive, interactions with volumetric data, i.e., selection, orientation, sectioning/removal, quantification (morphometrics, see Fig 1), markup, and remote collaboration. These functionalities provide a robust suite of functions that support interactive manipulation and visualization as well as selection for posing ad hoc queries to the database

Current activities include the acquisition of domain/study-specific extremely large, multi-modal and multi-scale datasets. In collaboration with colleagues, they are working to integrate these data into the OSC database, and place these into the hands of the user community. Upon completion of integration, designs for the actual "face" or look of the interface will be formalized.

Through many discussions, the user community has expressed concern of being overwhelmed with large images and datasets. Current and future work focuses on improving the ability to interactively select more tractable sub-regions within extremely large datasets via data mining and thereby optimizing the use of computational resources for interactive manipulation and queries. To address this issue, the texture-map-based direct volume rendering algorithm in the VolSuite framework has been enhanced with an early version of an interactive region-of-interest tool. This tool helps to maintain interactive performance as a user interactively selects a region of interest from a volume and effectively zooms in or out on the data (see Fig 2). This tool takes into account the capabilities of the local graphics hardware in order to select which resolutions of loaded datasets should be used. For example, a graphics card with 64MB of video memory will not render a full resolution 160MB dataset at interactive rates using a texture-map-based technique. The new tool automatically selects and renders a half-resolution version of the data that is only 20MB instead. The user can later "zoom in" to see the original resolution, but then are restricted to a roaming "window" that is 1/8 the size of the original data.

This tool will be integrated with other technologies such as database retrieval of regions of volume data at specific resolutions or 2 or 3D microscopy to provide a more seamless examination of data that would otherwise be too large to visualize on a single machine.

Although computational and display technologies are growing exponentially, the increases in data acquisition techniques and precision are far out-stripping these advances. We have subsequently investigated the scenario of a user on a local, thin client (e.g., office PC or laptop) being able to run complex, interactive session on large-scale data sets that are executed on remote computational resources. A new parallel rendering framework for regular volumetric data has been developed which utilizes a cluster outfitted with COTS graphics cards. This framework was written using MPI, C++ and OpenGL as the rendering backend, and includes a TCP sockets based communications protocol and daemon on a front-end machine to mediate remote access. With this framework, a remote user on a thin client connects to the daemon and negotiates the start of a new rendering "job", including how many nodes to use and the desired communications topology. The daemon subsequently spawns a new instance of the rendering "back-end" on the cluster. The renderer and remote client then establish a connection so the client can control all rendering parameters and receive the final rendered image. As the user interactively changes rendering parameters, the renderer constructs a final image on a single node of the cluster using a sort-last algorithm, before compressing it and streaming it to the remote client.

The back-end rendering algorithm can render a scene in orthographic or perspective, and can handle any data or visualization algorithm that can be adapted to the sort-last paradigm. The renderer can also dynamically control the amount of synchronization between the image contributions from each node, in order to optimize either frame rate or a final image free from artifacts. The parallel renderer implements a sort-last rendering algorithm requiring N+1 nodes. N nodes render their portion of the distributed scene, then send the "dirty" portion (sub-portion that has changed) of their framebuffer to the (N+1)st node (i.e. the "front-end" node) for compositing. The front-end selects the compositing method based on whether the incremental framebuffers represent "intermediate" images (e.g. during user interaction) or the "final" image (when the user ceases interaction).

Development of Advanced Image Analysis Algorithms

Our activities in this area mainly focused on analyses of digitized microscopy slides. These activities relate mainly through 3 projects:

1. Virtual Mouse Placenta with Dr. Gustavo Leone
2. mouse brain imaging via multiphoton confocal microscopy with the BIRN group

Virtual Mouse Placenta

Mouse placenta offers a convenient and controlled model for studying developmental biology as well as cancer genetics. Knockout mice typically show placental morphological differences from wild type. These differences can be visualized and analyzed computationally using digital microscopy images. The project consists of two primary tasks. The first task is to reconstruct in 3D a virtual placenta that can be used for visualization as well as image analysis. The second task is to segment the tissue layers and analyze the features to support classification of abnormal development and quantification. We are currently collaborating with Dr. Gustavo Leone at The Ohio State University.

To date, two mouse placentas have been sectioned serially. Selected sections at regular intervals are stained with hematoxylin and eosin for structural presentation, and the slides digitized on the Aperio ScanScope scanner. Approximately 400 H&E sections are being scanned for the two specimens at approximately 500MB per uncompressed images. The images are reconstructed into a volume post-acquisitionally by first detecting primary edges in the images and then registering the edges with successive slides. Visualization Toolkit, Insight Segmentation and Registration Toolkit, and DataCutter are used for distributed image processing. The remaining sections will be immunohistochemically stained, scanned, and registered, to create an interleaved multi-"modal" dataset. Image analysis begins with nuclear segmentation as the first step for generating cellularity measures. The algorithms used here leverage heavily the algorithms developed for neuroblastoma image analysis.

Two wild type placentas are currently being scanned, with knock out mice planned in the near future. Preliminary three-dimensional reconstruction results are shown below. The dataset has been volume rendered as well for visualization and validation of registration correctness.

Image analysis currently consists of nuclear segmentation of the scanned images. An example of the classification and segmentation results is shown below.

Preliminary results suggest that 3D registration is feasible for serial sections of placenta. Challenges remain, including the irregularity in the slide spacing, due to lost sections, as well as the inability to separate tissue morphology changes from registration inaccuracy at the finest scale. Additionally, registration of multiply stained, interleaved datasets present additional data interpolation and integration challenges. Image analysis for placenta images has demonstrated scientific relevance in cellularity and nuclear count measurements. Future tasks include tissue segmentation and cell type identification to show developmental differences between wild types and knock out mice.

We have also started some initial work with Dr. Carl Morrison from OSU Pathology to look at lung cancer nodule histopathology image analysis. Dr. Carl Morrison has evidence that a certain set of quantitative morphometric measures can be linked to oncogenetic measurements. Similar to neuroblastoma, the goal of this project is to quantitatively analyze images for specific features. Anant Madabushi from University of Pennsylvania has been looking at machine learning approaches for prostate MR image analysis. He has recently begun looking at microscopy images of prostate excision for the purpose of correlating and validating MR images. However, the analysis is computationally intensive. We plan to apply our grid-based computation expertise to his efforts in order to increase his algorithm performance. In addition, Dr. Madabushi's approaches serve as a good comparison for our image analysis decision tree methodology. We intend to apply his methods to our datasets in order to compare the capabilities of the two methods.

Simulation of EPR Imaging to Develop Better Hardware and Image Quality

A critical component in EPR imaging is the microwave resonator. The performance of the resonator is vital to obtaining good image quality and to maintaining low microwave heating in the subject to be imaged. In this project, a main task is to investigate the use of electromagnetic simulation tools to aid in the design of the resonator.

In this past year, an electromagnetic solver based on the finite difference time domain method has been developed to model resonators. These simulations can take a significant amount of computation time and memory, and in the future, it is likely that the use of multiple processors will become a necessity in order to generate the numerous simulation results necessary for the design of the resonator.

The first resonator considered was the L-Band transverse electric re-entrant resonator. This resonator is used extensively within the EPR group at OSU to perform imaging of rat and mouse hearts. The resonator is shown in Fig 6. To test the effectiveness of the electromagnetics code to predict the performance of the resonator, a homogeneous cylindrical phantom composed of TAM in a 0.45% NaCl solution was imaged in the actual resonator, and the simulation was run with a cylindrical sample having the same electrical properties as the actual phantom. The color levels are related to the EPR signal from the various sections of the phantom. The corresponding simulation result is shown in Fig 7 where the square of the magnetic field is plotted. For a homogeneous phantom, the distribution of the two plots should be the same. Generally the agreement is good. There are numerous differences between the resonator configuration in the actual measurement and the resonator that was actually simulated. Currently, we are investigating these differences to determine whether the correlation between the two images can be improved.