National Center for Supercomputing ApplicationsUniversity of Illinois at Urbana–Champaign

NCSA Brown Dog

PaaS for SaaS for PaaS

Rob Kooper

Senior Research Programmer

Kenton McHenry, Ph.D.

Research Scientist

Kenton McHenry$10,519,7162013-2018

Bill Michener$21,194,5482009-2014

Golam Choudhury$10,085,1202009-2014

Reagan Moore$8,300,9922011-2016

Steven Ruggles$7,993,2662011-2016

Margaret Hedstrom$8,000,0002011-2016

Alex Szalay$7,603,7232013-2018

Long Term Access to Large Scientific Data Sets: The SkyServer and Beyond

Michael Levine$4,902,6012013-2018

The Data ExacellXiaohui Carol Song

$3,409,0292013-2018

Integrating Geospatial Capabilities into HUBzero

NSF ACI Data Program

• Data Access Proxy (DAP)• A highly extensible and distributed service for carrying out

file format conversions• Move towards an internet/world that is agnostic to file

formats• Aid in accessing a files contents independent of how it

is represented on disk

• Data Tilling Service (DTS)• An extensible and distributed service for the extraction of

new data or metadata from a file’s contents• Provide means to query and/or relate collections of

data without metadata

• Data Conversion: A transformation on digital data that largely preserves the entirety of the data.  Largely eversible.

• Data Extraction: A transformation on digital data which creates new, often higher level, data from the contents of the given data (e.g. tags, signatures). Not reversible.

DAPNative Byte Encoding

File Formats, Data Bases, Websites,

Documents

Data StructuresArrays, Strings, Images, Videos,

Audio, 3D Models, …

Derived Data/

MetadataTags,

Signatures

ApplicationsSearch, Relate, View, Process

Data CollectionURL, File System, …

DTS

Usable Data

DAPNative Byte Encoding

Various Image Formats

Data StructuresImage

Derived Data/

MetadataText,

Number Values

ApplicationsClimate Modeling

Data CollectionHandwritten

Settlement Vegetation Data

DTSDAPNative Byte Encoding

R data

Data StructuresTabular

Derived Data/

MetadataMeteorological

attributes

ApplicationsClimate Modeling

Data CollectionWeather Data

DTSDAPNative Byte Encoding

Various Image Formats

Data StructuresImage

Derived Data/

MetadataLand

Cover/Usage/…

ApplicationsClimate Modeling

Data CollectionMODIS Satellite Data

DTSDAPNative Byte Encoding

Various Image Formats

Data StructuresImage

Derived Data/

MetadataMeasure of Aesthetic Appeal

ApplicationsGreen

Infrastructure Design

Data CollectionArchitecture/Design

Images

DTSDAPNative Byte Encoding

Various 3D Formats

Data Structures3D Model

Derived Data/

MetadataSynthetic Images

ApplicationsGreen

Infrastructure Design

Data CollectionArchitecture/

Landscape Models

DTSDAPNative Byte Encoding

Various Image Formats

Data StructuresImage

Derived Data/

Metadata3D Model

ApplicationsGreen

Infrastructure Design

Data CollectionPhotographs

DTSDAPNative Byte EncodingVarious Video

Formats

Data StructuresVideo

Derived Data/

MetadataPeople

Locations/ Interactions

ApplicationsLarge Dynamic Group Behavior

Data CollectionGroupscope

DTS

https://dap.ncsa.illinois.edu/conversion/

https://dts.ncsa.illinois.edu/extraction/

http://browndog.ncsa.illinois.edu

Medici

https://opensource.ncsa.illinois.edu/stash/projects/MMDB/

Polyglot

https://opensource.ncsa.illinois.edu/stash/scm/pol/polyglot.git

Versus

https://opensource.ncsa.illinois.edu/stash/projects/VS/

Daffodil

https://opensource.ncsa.illinois.edu/stash/scm/dfdl/daffodil.git