grid technology for controlled fusion: conception …

GRID TECHNOLOGY FOR CONTROLLED FUSION:

CONCEPTION OF THE UNIFIED CYBERSPACE AND ITER DATA MANAGEMENT

N. Marusov, I. Semenov Project Center ITER (ITER Russian Domestic Agency

[email protected])

Challenges of ITER data usage

• Data accumulation & distribution ~5 PB/year, ~100 PB for a lifetime

• Huge amount of data cannot be analyzed by humans

• Automated conversion of raw data to knowledge

• Scientific collaboration all over the world

• Diversity of end-user needs and tools

• Future integration of ITER scientific achievements with other TOKAMAKs

3 June 2015, Nice, France

1st IAEA TM on Fusion Data Processing, Validation and Analysis

2

Fusion Technology

Knowledge Management

System

ITER machine

Fusion Community

Results Goals

Knowledge

Data

Fusion community formulates the goals and

gets the results

Fusion technology is viewed as the

controlled object

Impact to technology is done trough the

Knowledge Management System

ITER machine work as sensors to assess the quality of technology

Information circulating in control loops represents experimental data and

knowledge

ITER research as control process

3 June 2015, Nice, France 1st IAEA TM on Fusion Data Processing,

Validation and Analysis 3

Fast data processing (< 10 min)

Slow data processing

(10 min - hours)

Domain knowledge allocation



Unified cyberspace (UC) concept

• Knowledge Management System (KMS) is one of key components of ITER computing model

• Distributed nature of ITER collaboration • Community-oriented research workflow • Liaison standards for interoperation in heterogeneous

computing environments • Machine learning-enabled environment



Distributed community-driven KMS semantically linking diverse computing resources, modeling codes and data storages on the base of existing standards

Most of such problems are already solved within existing projects

• Worldwide LHC Computing Grid (CERN + 42 countries)

• National Fusion Collaboratory Project (USA)

• European Grid Infrastructure

• European Middleware Initiative



based on well-known concepts such as e-Science, Science 2.0, Cyberinfrastructure and

Semantic Web

General view of UC infrastructure



Wiki-sites & Social

Networks

User’s Applications

Basic Grid Services

High-level Services

Storage Element

Computing Element

Computing Resources

End User

Personal Environment

VO Web Portal

Virtual Laboratory

Robotic Agents

Applications

SaaS Application

Information Services

Experimental

Facilities

Workflows

Network

Storage

Other

Data Providers

Basic Grid middleware

• Security / Virtual organizations management service

• User Interface

• Computing element

• Storage element

• Information services

• Workload management



Data accumulation & distribution - LHC

• Worldwide LHC Computing Grid

• Tier 0 (CERN) –Data recording

–Initial data reconstruction

–Data distribution

• Tier 1 (11 + KISTI, Korea in progress) –Permanent storage

–Re-processing

–Analysis

–10 Gbit/s links

• Tier 2 (~150 centres) –Simulation

–End-user analysis

• Tier 3 (End-users) – Customized processing

3 June 2015, Nice, France

1st IAEA TM on Fusion Data Processing, Validation and Analysis

9

Overall

~160 sites, 39 countries

300,000 cores

200 PB of storage

2 million jobs/day

Data accumulation & distribution - ITER



ITER Site

Domestic Agency

Domestic Agency

Domestic Agency

National Institute

National Institute

Tier 0

Tier 1

Tier 2

Full replication

End User Tier 3

Processed Sample or partial replica

Processed Sample

Unified Data providers



End User Storage

Facility-independent Metadata scheme

HTTP

MDSPlus Sampling processor

Server-side data manipulation

Native data format

Information Service

Unified Data Provider

Metadata update

On demand data selection

Cache

Fusion research poses a lot of loosely-tied tasks

• Parametric optimization of fusion facilities and algorithms

• Data mining & scaling laws

• Empirical dynamic models generation

• 3D-animation of dynamic processes

• Monte-Carlo & Ray tracing



Job flow in the WLCG/EGEE Grid



Grid-enabled applications as Services



End User

Workflow composer

SaaS Application 1

SaaS Application 2

SaaS Application 3

Workflow sharing

Grid interface

Grid interface

Grid interface

Grid

Direct Grid API usage

Direct Service API usage

Cooperation within ITER & ITER as a partner of cooperation



ITER

JET

KSTAR

DIII-D

T-15M

Step 1: Prove feasibility of Worldwide Fusion Laboratory on the base of ITER and existing solutions. Step 2: Integrate storages of other TOKAMAKs with the unified cyberspace on the base of proven technology. Step 3: New super-collaboration endeavors for DEMO – optimized fusion power plant.

Fusion community cyberspace

DEMO

Thank you for your attention!



Discussion

grid technology for controlled fusion: conception …

Documents