PowerPoint Presentation

tipiXRapid Visualization of Large DatasetsAdrian V. Dalca, Ramesh Sridharan, Natalia Rost, Polina Golland1Hi everyoneI will talk about visualizing medical images in large datasets1

Were all used to looking at medical images, and were comfortable exploring one subject at a time.But in many cases what we really want is to see all of the images in a dataset.And with recent datasets, this means hundreds of scans.

2

3Alzheimers Disease, BrainMRI, PET2000+

Brain MappingMRI, fMRI, MRA, DTI, PET~800Heart, BrainMRI2500+, 6000

Stroke, BrainMRI4000+Autism, BrainMRI, fMRI 1000+

COPD LungCT10,000+And in fact, in the near future we may be looking at thousands of scans per collection

So how do we explore such large datasets?Today, Ill show you a new tool which enables us to visualize collections of images very quickly and intuitively. And Im showing you a preview here.

3OutlineCurrent methods for exploring datasetsOur method: tipiXUser studyConclusion

4Ill start by describing what people do now with large datasets, Ill then talk about our method to visualize such collectionIll discuss results from a user study, and Finally conclude.4OutlineCurrent methods for exploring datasetsOur method: tipiXUser studyConclusion

5So how do we currently look at our imaging data?5Current Methods: Visualize Subjects

6We have single subject viewers, like Slicer, ITK SNAP or Slice:DropThat are great at viewing one (or very few) subjects, very powerful platforms for very thorough analysis

6Current Methods: Visualize Subjects

7But what if we want to look at several subjects. I have to use the slider to go through slices, then the dropdown menu to switch subjectsIts a really cumbersome and slow way to go through hundreds of subjects7Current Methods: Grids

8

Grids are an alternative visualization method.They enables us to simultaneously visualize a few subjects, although you lose resolutionBut it gets quite difficult when you have even a few hundred subjects8StatisticsSummarize data and resultsAppropriate statistics are difficult to determineProvide concise, superficial overview of dataUsually wont detect patterns, imaging details

9As an alternative, we can give up on visualizing and concisely summarize our imaging data, or say imaging results, With statistics, measures and graphsUnfortunately, statistics that capture the desired information are hard to derive for each taskEven if we come up with a statistic, they only give a superficial overview idea of the data, they cant show us patterns and details of imagesTo really understand patterns in behaviour, we need to look across the entire dataset9OutlineCurrent methods for exploring datasetsOur method: tipiXUser studyConclusion

10In contrast, we harness the ability of humans to detect and isolate patterns visuallyOur method, which we call tipiX, enables us to quickly explore the dataset in high resolution.The best way to show you this is through a demo10tipiX Demonstration11tipiX demonstration DatasetOur idea is straighforward: we use the position of the pointer on a web-canvas to explore the space of the dataset.As the pointer moves up and down I see different slices from a subject, As the pointer moves left and right, I see different subjects of my datasetSo we can rapidly explore the space of subjects and slice depth simultaneously

11Exploring Dimensions

Subject IDDorsal-ventral

12Dataset DimensionsSpatial dimensions (2D slice)Volume DepthSubject IDLongitudinal dataTemporal data (fMRI)Different scan modalityClinical Factors (e.g. Age)Parameter sweepsTrying different algorithmsNow if we think of the 2D spatial dimensions of the axial slice, the volume depth and the subject ID.These are all, together the dimensions of our dataset.But datasets can have other dimensions, like age in longitudinal data or different modality types. And tipiX enables us to explore these different dimensions, and Ive set up different demonstrations to show you12tipiX DemoSlice vs subject numberLesionRegistrationDifficult registrationGroups vs ageTime evolution of vascular diseaseSlice vs parameter gridParameter sweepSlice vs subject number GridOriginal dataset exploration

13In this second case, the registration is very difficult: here, the green outlines should overlap with anatomy for successful registration, and weve sorted the subjects using a heuristic.We can see that the subjects on the left are decently registered. The subjects on the right are badly registered. Here, tipiX quickly lets you evaluate your registration in a hundred subjects.We will touch on this example again a bit later on

13White Matter Hyperintensity14

For our third demo, We have a 1000 patient cohort. WMH have been delineated in green, mask shown in yellow for every subject: The average mask was computed for several ages, and we can then explore this data. Here, as I move my mouse horizontally Ill vary age. So with my mouse on the left, I can see that young subjects have little WMH, and older subjects have more WMH. This is in one subgroup of the population. In another, there is barely any change. So these are dimensions of resulting images that we can explore with tipiX. Parameter Sweep: we can choose the optimal registration parameter very quicklyWe can combine tipiX with grids, where we have very few items to grid, such as the number of modalities. Here, we can explore a dataset while looking at 3 modalities at the same time.

14FeaturesOpen source, works directly on modern browser Processing is client-sideStreamlined collaboration: specific tipiX link encodes datasetAssumes collaborators have data accessMany features present and in development Display volume info, volume control slice:drop binding, website embeddingCome to the demo!15tipiX is developed on the modern browser, which has many implicationsno downloads are necessary yet all processing is done client-side, In many cases you can send a tipiX view to a collaborator through a simple linkWe have many other features both implemented and in development, please come by the demo to talk and see them in action!15OutlineCurrent methods for exploring datasetsOur method: tipiXUser studyConclusion

16So the demos showed just some of the possible things you can do with this interface. We also did a preliminary user study, to see how well others can use tipiXNine participants who had never seen tipiX before performing tasks similar to those in the demoedHere Im just going to talk about one task, which is modelled based on something we had to do in our own work.

16User Study17

ordered subjects

Threshold registration quality500 subjects1500In this task participants were shown 500 registered scans that were ordered by registration qualityUsers had to find a subjective threshold of where scans go from good registrations to bad registrations, The graph shows the time it took for each user to determine a subject index as a thresholdNow most users found a threshold within 2 minutes, and mostly agreed on the threshold! Tasks like these would take hours or be impossible with single subject viewers.17OutlineCurrent methods for exploring datasetsOur method: tipiXUser studyConclusion

18So our users were able to perform powerful tasks with a completely new interface. tipiX.18

ConclusionsVisualization of large datasets across many dimensions is importanttipiX presents an intuitive visualization platformEnables pattern detection, outlier detection, visualization of trendsWeb implementation enables easy collaboration, open-source developmentUser study demonstrates promising ability to explore datasets quickly

19Large datasets with hundreds of subjects need to be visualizedWe present a web-based tool that quickly and intuitively enables the exploration along several dimensionsWe showed very promising results from a user study for participants using our tool19