Overview ofAnalysis Result Manager

(ARM)Jeremy Magland

27 October, 2011brainomics group @ Penn

Motivations for ARMO Motivation #1:VisualizationI wanted to be able to SEE my fMRI data, intermediate files, and processing results, and to share these with my collaborators.

Motivations for ARMO Motivation #2:Data ManagementOrganize fMRI data on a central server, and access it from anywhere (cloud style)

Real-time Study REAL_23 Session 1 Raw fMRI series Anatomic localizer Session 2 REAL_36 Session 1 Session 2

Bone-Water Study Subject AP-1 Localizer Baseline, TR=300 Follow-up 1, TR=300

Bone-Water Study Subject AP-1 Localizer Baseline, TR=300 Follow-up 1, TR=300

Sample textAbout studyStuffTo fillsapace

Motivations for ARMO Motivation #3:Computing PowerUtilize the processing power of server computers (CPU power, RAM, automatic parallel processing)

Motivations for ARMO Motivation #4:Provenance TrackingTrack the processing steps required to reproduce each intermediate object and processing result.Advantages:* reproducibility* auto-non-redundant processing

O Browser-based applications are very convenient but often have limited functionality.

O Desktop applications can have rich functionality. However there are some cons:O software

downloads/updatesO OS incompatibilityO Security risksO Etc.

Challenge: Richly Interactive, Web-based Data Visualization

O RemoteWidgets:O Framework for

developing desktop applications in Qt4 with option to deploy on browser.

RemoteWidgets: Overview

O Write a GUI for the web using only one programming language: Qt4/C++

O Rapid development of highly interactive web applications.

O Direct access to server-side processes.

Seegull

ARM

Pickles

Math Puppy

RemoteWidgets:One App, Two Access Options

O Web Browser Interface (jQuery)

O Desktop Interface (thin client download)

Seegull in Web Browser

Seegull on Desktop

RemoteWidgets: Hello, World!

RemoteWidgets:Server-Side Event Handling

O Handling GUI events is as easy as connecting Qt signals and slots on the server… no javascript coding needed.

RemoteWidgets: IdeaO Interactive, desktop-style visualization

tools available from the convenience of a web browser.

Analysis Result Manager (ARM)

O Cloud-based approach to fMRI data management, visualization, and analysis.

O ARM is a RemoteWidgets application, and can therefore be accessed remotely via a web browser.

Tradeoff BetweenScripting and GUI for fMRI Processing

O Scripting is way more powerful for research purposes. And, if you have a good script interface, you could always throw a GUI on top of it.

O But… some problems with Matlab-style scripting:O Need to manually shuffle data around in

files/foldersO Not easy to remember exactly which

parameters were used >2 days agoO Run a 10-hour script, and discover the last

line had a bug.

ARM: Script-Generated WorkflowsWith Robust Tracking of Provenance Data

The script immediately sets up a workflow, creating all intermediate and result objects ahead of the processing, storing detailed information about how objects are related via processes.

The multi-threaded processing module then executes each processors as soon as all input objects have been defined.

Processors are Generic PluginsC++ Wrappers (for FSL, SPM, etc)

Views are also plugins… (custom views of custom datatypes)

Live ARM Demonstration

O Upload Scan DataO Online view, change view

settingsO Super-efficient browsing of

huge data files.O Browser access to same appO View example scriptO Running script does nothing

because processing has already completed

O Change design function and re-run script – show how processes are queued and executed in proper order (No redundant processing).

O View processing results and intermediate files.

O Show provenance (process history) of the output objects.

O Explain that changing scan id’s and even names of the processors causes no problems. Key assumption: objects, once created, never change.

O Clean up by deleting results. (In future, intermediate files will be auto-removed after a period if they are no longer linked to results)

O In development – share a processing result by a button click.

Live Seegull Demonstration

O It is often desirable to SHARE results with others. But you don’t always want to give them access to a huge system. Furthermore, since ARM is highly dynamic, you want to be sure that a result, once shared, is available for ever after (snapshot of a result)

O Hence, Seegull

Summary and FutureO Share interactive views of dataO Taking full advantage of cloud

technology to manage, process, visualize, compare fMRI data analyses

O Script-generated workflowsO Automatic provenance trackingO Convenient access via web browserO Import publicly available fMRI studies

into system (powerful meta-analyses)

Especially Thank You to…

O Anna Rose ChildressO Felix WehrliO Charlie Epstein

And…O Benny LamO Oscar BartraO Kimberly YoungO Dan WillardO Rob Fabianski

O John DetreO Jesse LupardusO Ning ZhangO Cheng LiO Ze Wang

O Chris TjoaO Brian Avants