wolfram data summit: new frontiers in astronomy

New Frontiers in Astronomy Dr Alberto ContiSpace Telescope Science Institute

April 2006

• Visible – Hubble Space Telescope• Gamma rays - Compton Gamma Ray Obs.• X-rays - Chandra X-ray Observatory• Infrared - Spitzer Space Telescope

Optimize the science from community-led astrophysics missions and projects. Develop, nurture, and share innovations in space astronomy science operations.

Collaborate on the next generation of space astrophysics programs.

Optical & UVData Archive

Astronomy Project TimelineA Partial List of Key Astrophysics Facilities

Start date and Probable Duration

HST

Spitzer

Chandra

FUSE

GALEX

GLAST

Kepler

WMAP

JWST

SWIFT

Beyond Einstein

XMM

SOFIA

INTEGRAL

Ares V Flights

Herschel - Planck

WISE

NVO Operations

ALMA

TMT

LSST

PANSTARRS

NVO Development

SDSS

VLT & Gemini Observatories

SIM? TPF?

2000 2005 2010 2015 2020

Astronomy Project TimelineSTScI Project and Mission Activity

HST

Spitzer

Chandra

FUSE

GALEX

GLAST

Kepler

WMAP

JWST

SWIFT

Beyond Einstein

XMM

SOFIA

INTEGRAL

Ares V Flights

Herschel - Planck

WISE

NVO Operations

ALMA

TMT

LSST

PANSTARRS

NVO Development

SIM? TPF?

SDSS

VLT & Gemini Observatories

Start date and Probable Duration2000 2005 2010 2015 2020

LSST

Challenges for the Future

Space is big!For one picture you need a 2 Trillion pixels camera!

Challenges for the Future

Monochrome : 4 Terabytes or

5% of the Library of Congress

Challenges for the Future

Color: 100 Terabytes or

the 21% more than the entire Library of Congress

Challenges for the Future

Time: 10 Petabytesor

120 times the entireLibrary of Congress

Challenges for the Future

New analysis & visualization tools are required

Astronomy is changing

Detectors follow Moore’s Law

Total data doubles every year

Growth over 25 years is a factor of 30 in glass, 3000 in pixels

ComputerScience Biology Economics

Medicine Government Astronomy

Massive amounts of information

ComputerScience Biology Economics

Medicine Government Astronomy

Massive amounts of information

e-Science

New Science Paradigm for Astronomy

Time

Dat

a Vo

lum

e

New Paradigm Issues

• Moving data around is hard

• Extracting knowledge is hard

• Complex, difficult to use

• Hard for user to publish their own data

• Many distributed services are unreliable

ASTRONOMY IS SPECIAL!

No commercial value

Ideal testbed for complex algorithms

Interesting problems

Plenty of data

ADAPT OR PERISH

Google Earth, Microsoft Virtual Earth have revolutionized the way we look at our planet

We need a new synergistic approach to the challenge of bringing the universe to our desktops

New Science Paradigm:First Iteration

Mission A

Mission B

Mission C

Observatory X

Observatory Y

Few Data Standards, Some Protocols

New Science Paradigm:Second Iteration

Data Standards, Protocols Simple Mining Tools

Mission A

Mission B

Mission C

Observatory X

Observatory Y

Metadata

NASA Data Centers

Intʼl Data Centers

IndividualUsers

Kitchen Sink

New Science Paradigm:Science 2.0

MAST @ STScI

Global Challenges

• Reduce obstacles to Capturing, Organizing, Summarizing, Analyzing, Visualizing, and Curating

• Consider data and algorithms as “the product”

• Adopt semantic technologies to enable automated metadata tagging, clustering and mining

• Transition to the new astronomy

• Sociological issues

• Infrastructure not available for intensive data mining

• Solutions for handling large datasets are lacking

• Cloud hosting solutions still expensive

• Unclear if commercial solutions can fit science needs

Technological Challenges

• We must partner with other academic disciplines: Computer Science, Statistics, ...

• We must leverage partnerships with industry interested in enabling Science 2.0

• We must learn to be humble and ask for help

• We must remember that we have the greatest datasets in the world (universe really!)