Introduction to DANSE Brent Fultz

Prof. Materials Science and Applied PhysicsCalifornia Institute of Technology

• Distributed Data Analysis Architecture for Neutron Scattering Experiments• Technical Issues, Scientific Organization• From a Concept to a National Project

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

$ 1,411,000,000 hardware10,000,000 software

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

ARCS

Analysis Software for Neutron Scattering • Enables science

• Helps new users get up to speed

• Better science with better software optimize beamtime usage experimental procedures tuned on-the-fly sample knowledge can constrain data interpretations

• Some experiments are impossible with today’s software

• New opportunities to connect to materials theory

Logo for ARCS SpectrometerHeadstock

Neck

Frets

Pickups

Volume/ToneSolidBody

Tailpiece(Whammy Optional)

Logo for DANSE??

Software with Power and Beauty

$?

Software with Coherence

DANSE architecture is GeneralbutAll developers must know itHence this Workshop

Data Analysis as a Web Service• Data analysis is a service controlled by the user • User’s laptop issues commands and receives results • Computation is arranged by the web server

Present Example from inelastic scattering: Born - von Kármán Lattice Dynamics

Simplicity: Complexity:

1. Undergrowth of indices for tensor quantities:

2. Crystal structure:

€

mκ ˙ ̇ u κl = − Vκ ,l

∑ κl,κ ' l 'uκ ' l '

€

uκl

€

V κl ,κ ' l '

lattice, l, basis, allowed elements depend on symmetry

The Bigger Concept Underneath

Components Pre-compiled Python objects called and re-arranged

by the Python Interpreter

Data Streams Standard communication protocol between

components (standard streams can connect components located anywhere)

DANSE Project: Develop Tools for Programmers

Component Templates Standard Data Streams

Levels of User Expertise• For using existing scripts, entry barrier nearly zero.• For altering existing Python scripts, entry barrier is very

low. • For writing new Python code, entry barrier is modest.

Performance may be comparable to IDL or Matlab. • Transition to high performance compiled code

ARCS: Writing Python bindings for C++DANSE: Component templates for C++, FORTRAN,

Java? Focus of this Workshop

• Importance of remote control of neutron instruments• Remote collaboratory capability• Remote data analysis capability

• No strong support for remote control -- too risky

(sample temperature and saving data okay)• Strong support for remote data analysis capability• Overwhelming support for standardization of data analysis

(across different instruments and different facilities)

NSSA Neutron Software Poll

Making a Software Project Plan

• Define and justify scientific needs

• Define and justify the software architecture

• Define the scope of the project

• Accurately estimate the effort

Proposal -- what, why, who, where, when, how?

A DANSE Project Would Have 2 Parts

• Build Tools (hardware and software) for Developers

• Extend Software Beyond ARCSDiffractionEngineering diffractionReflectometrySANSInelastic scattering (ARCS+chemistry)

Money

• Software tools development

• Hardware for multiple users

• Science-based subprojects

Four-Year Budget (salaries mostly)

10 M$ 4-5 subprojects

10 M$ Central Resources 7 FTE + hardware

Software Development

ARCS Software Project

Recent Developments• DANSE is “the SNS standard” architecture

• NIST reflectometry sub-proposal

• TeraGrid Extension to ORNL (DANSE is core)

• SNS software developers at HFIR(?)

• SNS to write specifications for data analysis software• censored

• NSF intermediate-scale instrumentation program (December 2003)

• NSF ITR (Spring 2004)

Reasons for Optimism about DANSE

• Upsurge in interest from DOE and beyond

• Alternative architectures are not full solutions

• Technically possible within the scale discussed

• No better group to do it

DANSE Software Workshop

Wednesday Sept. 3, 2003 Introductions Lunch Python and OOP Dinner Evening tutorials

Thursday Sept. 4, 2003 The Python/C API, NeXus Lunch Subgroup Talks/Funding Evening tutorials

Friday Sept. 5, 2003 Architectural Framework Lunch Programmers' Tools Dinner Evening tutorials

Saturday Sept. 6, 2003 Software design for DANSE Lunch "Edward Weston: A Legacy"

Sunday

Monday Sept. 8, 2003 Python Extensions in C++ Lunch FORTRAN, Operating systems. DANSE Project integration. Dinner

Welcome toCalifornia

End of Presentation

Michael is Next