William Lorensen

GE Research

Niskayuna, NY

February 12, 2001

Insight Insight Segmentation Segmentation

and and Registration Registration

ToolkitToolkit

AgendaAgenda

IntroductionChronologyRecipe For SuccessObservations

What is it?What is it?

A common Application Programmers Interface (API).– A framework for software development– A toolkit for registration and segmentation– An Open Source resource for future research

A validation model for segmentation and registration.– A framework for validation development– Assistance for algorithm designers– A seed repository for validated segmentations

Chronology Chronology

October 1999, Bethesda– Project kick-off

January 2000, NiskayunaRequirements

March 2000, Big Apple– Validation Strategies

June 2000, Bethesda– Core classes

February 2001, Salt Lake– Code reviews

April 2001, Bethesda– Show and Tell

July 2001, Seattle– Refactoring

November 2001, Bethesda– Demos– Preparation for the beta

February 2002, UNC Chapel Hill– Developer exchange

Recipe for SuccessRecipe for Success

VisionStrong Core TeamOpennessCore ArchitectureLight Weight Software EngineeringCommunity SupportFunding

The NLM VisionThe NLM Vision

Create a dynamic, self-sustaining, public domain and extensible toolkit that will empower researchers throughout the

world to develop new segmentation and registration algorithms and create new applications that leverage the NLM’s

investment in the Visible Human Male and Female data sets

The TeamThe Team

GE CRD/Brigham and Womens– Architecture, algorithms, testing, validation

KitwareArchitecture, user community support

Insightful/UPenn– Statistical segmentation, mutual information

registration, deformable registration, level sets– Beta test management

Utah– Level sets, low level image processing

UNC/Pitt– Image processing, registration, high-dimensional

segmentation UPenn/Columbia

– Deformable surfaces, fuzzy connectedness, hybrid methods

OpennessOpenness

From the start, NLM recognized the value of Open Source software

There are NO restrictions on the software

The Team has embraced openness

Open Source ProductsOpen Source Products

The Core - RequirementsThe Core - Requirements

Shall handle large datasets– Visible Human data on a 512MB PC

Shall run on multiple platforms– Sun, SGI, Linux, Windows

Shall provide multiple language api’sShall support parallel processingShall have no visualization system

dependenciesShall support multi-dimensional imagesShall support n-component data

The Core - ArchitectureThe Core - Architecture

Established a core architecture and support classes in June, 2000

Team used the initial core to develop their algorithms

Architecture team adapted and modified the core

Architecture: Data FlowArchitecture: Data Flow

A sequence of process objects that operate on data objects to generate additional data objects

Data

Filter

Data Display/disk

Data

Data Filter

Mapper

Mapper

Source

ArchitectureArchitecture

Segmentation – Fuzzy connectedness– Supervised/Unsupervised Classification– Level Set Shape Detection– N-D Morphological Watershed– Voronoi based– Gradient Magnitude– Balloon Force Filter– Region Growing– Watershed

Dozens of basic filters

Registration ArchitectureRegistration Architecture

Multi-Resolution Registration Framework PDE Deformable Registration

InputReference

TargetReference

Mapper

Metric

Transform

Optimizer

Longitudinal MRI No Registration

Checkerboard

SourceOriginalimage

Difference

TargetOriginal image

Longitudinal MRI

Registration

Checkerboard

SourceOriginalimage

Difference

TargetOriginal image

Lung CT No Registration

Checkerboard

SourceOriginalImage

Difference

TargetOriginal Image

Lung CT Registration

Checkerboard

SourceOriginalImage

Difference

TargetOriginal Image

microPet/Volume CTmicroPet/Volume CT

microPet/Volume CTmicroPet/Volume CT

Process: Process: Extreme ProgrammingExtreme Programming

A Light Weight Software A Light Weight Software Engineering ProcessEngineering Process

Based on the new Extreme Programming process– High intensity design, test, implement cycle– Supported with web-enabled tools– Automated testing integrated with the

software development

Extreme ProgrammingExtreme Programming

Compression of standard analyze, design, implement, test cycle into a continuous process

Insight Development Insight Development CyclesCycles

Daily – dashboard Weekly – telephone conferencesBi-weekly – architecture reviewsQuarterly – developer meetings Yearly – work assignments

Extreme ProgrammingExtreme ProgrammingDaily Testing Is The KeyDaily Testing Is The Key

Testing anchors the development process (Dart)

Developers monitor the testing dashboard constantly

Problems are identified and fixed immediately

Developers receive e-mail if they“Break the Build”

Building a CommunityBuilding a Community

Initial community of consortium members Outreach to other groups

– Siggraph– Digital Human– DOE Genomes to Life– IEEE Visualization– NSF Shape Modeling Workshop– Supercomputer Conference

Public Beta, February 2002 Application development, both academic and

commercial

FundingFunding

Seed funding by NLM and its co-sponsors– Establish an architecture– Implement a representative set of algorithms– Produce frameworks to accommodate new algorithms– Define a development process– Establish a community support mechanism

Continued support will be needed– Outreach to other communities– Create applications– More algorithms to fill gaps– Infrastructure support

ObservationsObservations

Good mix of commercial and academicImportance of communicationThe daily rhythm of Extreme

ProgrammingThe Whole >>> Sum of the partsThis process can (and should) be

repeated

Insight Segmentation Insight Segmentation and Registration and Registration

ToolkitToolkit

William Lorensen

GE Research

Niskayuna, NY

February 12, 2001