Mantid Stakeholder Review

Nick Draper

01/11/2007

Agenda

• Why do we need Mantid?• What will Mantid do?• Progress so far• Plan for the future• Questions

Why do we need Mantid?

• Previously different tools were used on the different beamlines.– PC Collette, Ariel, SXD2000, OpenGenie, LibISIS.– Confusing for users.– Replicated effort across the tools.– Difficult to support.

• Mantid– To standardise data reduction tools between instruments.– To improve the support and documentation for the tools.– To handle the expansion of data volume.– While having the flexibility to be extended by scientists as

required.

What will Mantid do?

• Mantid– Manipulation and Analysis Toolkit for ISIS Data

• Aims– To provide a framework for Data Analysis that is not

instrument or technique/dependent.– To support multiple target platforms (Windows, Linux).– The framework must be easily extensible by Instruments

Scientists/Users.– The framework must be freely redistributable to visiting

scientists.– The framework should provide low-level functionalities

for Scripting, Visualization, Data transformation, Implementing Algorithms, Virtual Instrument Geometry.

What will Mantid do?

• Scope– Data reduction and analysis, not Instrument Control. – Creation of an extensible framework.– Creation of a command line and scripting interface.– Creation of a Visualization tool.– Creation of a repeatable automated test environment.– Provision of user and support documentation.– Providing user support for usage and extension of the

framework.– Creation of specific beamline interfaces as required.

(Later)

• Milestones– Architecture design completed October 2007 – Prototype framework released: March 2008 – Functional framework + 2-3 complete applications

released March 2009

Timeline

Meet the team

• Project Manager– Nick Draper, Tessella

• Developers– Russell Taylor, Tessella– Laurent Chapon– Matt Clarke– Dickon Champion– Freddie Akeroyd– Anders Markvarsden– Stuart Ansell

• Sponsors– Paolo Radaelli– Toby Perring

Progress so far

• Requirements Gathering and Analysis– Dunstan Thomas – Large meetings– Myself – one to one discussions

• Architecture Prototypes– GAUDI– LOQ – PC Collette

• Architectural Design• Development of the core framework functionality

Top Requirements

• Easily extensible.– Support all current and future analysis.– Support current and future file formats.– Provide a simple but powerful objects and services to

support user created algorithm code.

• No user license costs.• Supportable.• Portable.

– Operating System (Windows, Linux).– Computing Power (Laptop, Server).

Architecture Prototypes - GAUDI

• A data analysis framework for event based data.• Created at CERN for LHCb experiment.• Built to be generic enough to be used by other

institutions.• Good points

– Careful design based on good ideas• Separating algorithms from data.• Encapsualting “user code” in specific places.• Ensuring the framework is extensible.

Architecture Prototypes - GAUDI

• Bad Points– Control of the analysis was complex and rigid.– File input and output was complex and confusing.– No memory management was included.

• All down to the design concept– Event Based data

• 1,000s of files in a single run• Small individual data files• Repeated analysis for each file, and generally the same

across runs.

Architecture Prototypes - LOQ

• Collette software – currently undergoing reimplementation on PC/Linux.

• Good points– Neutron specific framework.– Aimed at handling similar file sizes to what we need.

• Bad Points– A bit too specific for the LOQ instrumentation.– Being developed by a single developer.– Not complete.

Architecture Prototypes - Conclusion

• Options1. Convert Gaudi

• To handle Histogram data and a more flexible control system.

2. Extend LOQ• To better support the various instruments across ISIS.

3. Take the best ideas and approaches from both.• Build a bespoke framework to support Neutron data and

practices.

• We chose option 3.

Architectural Design - Overview

Mantid Framework

Command line & Scripting interface

Visualization tool

RAW data files

NEXUS data files

Future Data analysis GUI

Instrument log files

APIStandard

Algorithms

User Defined Algorithms

Development Process

• Iterative development – Why?– Adaptability - the ability to rapidly respond to changes

in strategy, priorities, and plans – Value - continuous delivery of more useful functionality – Visibility - stakeholder collaboration and validation

throughout the development life-cycle – Risk - the reduction in overall project risk as a result of

#1-3 above

Development Process

• Iterative development.– Top level aim defined.– Tasks identified and allocated to fulfill the aim. – 4 week of development and testing.– Automated tests built alongside all functionality.– Review, and plan the next iteration.

Development Process

• Iterations so far– 1

• Aim: Create enough of the framework structure to support the same test as performed in the Pilots.– Load a raw file -> integrate the counts for all time bins ->

output• Result: Success but with a few compromises.

– 2• Aim:

– Add sufficient geometry support to allow conversion of tof bins to wavelength.

– To add a python interface to the framework– To include the ability to add new algorithms through user

created libraries, loaded at run time.• This iteration completes at the end of this week.

Software led by Scientists

• This is project is intended to support the needs of all of the beamlines.

• We need input from each group of instruments– To ensure what we build fits what you need.– To ensure good communication between the Mantid

project and the scientists.– To allow each group to have equal input into the project.

• To do this we will set up a scientific steering committee.

Scientific Steering Committee

• Who?– The Mantid project manager and one representative

from each group of instruments.

• What will they do?– Raise new enhancement requests and report issues.– Prioritise the list of tasks to be done over the next

iterations.– Review the progress of each iteration.

Scientific Steering Committee

• How often?– Once every 4 weeks just before an iteration starts.

• When will this start?– We plan to start at the end of January.

• Where do I sign up?– Voluteers (with the support of their group) should

contact me (Nick Draper).

Further Information

• Project Web Page– www.mantidproject.org

• Project Introduction Document– http://svn.mantidproject.org/mantid/trunk/Documents/Requirements/Project%20I

ntroduction%20Document.doc

• User Requirements Document– http://svn.mantidproject.org/mantid/trunk/Documents/Requirements/URD.doc

• Architectural Design Document– http://svn.mantidproject.org/mantid/trunk/Documents/Design/Architecture%20De

sign%20Document.doc