Convergence between HPC and Big Data:

The Day After TomorrowBilel Hadri – KAUST Supercomputing Lab

Maxime Martinasso Katie Antypas Dan Reed Rio Yokota

Panelists� Maxime Martinasso CTO deputy at CSCS (Swiss National Supercomputing

Centre) where he leads the interactive computing program.

� Katie Antypas) is the Division Deputy and Data Department Head at the National Energy Research Scientific Computing (NERSC)

� Daniel A. Reed is the Senior Vice President for Academic Affairs (Provost) at the University of Utah.

� Rio Yokota is an Associate Professor at the Global Scientific Information and Computing Center at Tokyo Institute of Technology.

Survey

� Who is working directly/indirectly both on HPC and Big Data ?

� Is the community serious about the convergence between HPC and Big Data?”

� SC17 panel “How serious are we about the convergence between HPC and Big Data?” where all panelists agreed about a love based marriage between HPC and Big Data.

� Various efforts and techniques to bridge both communities ranging from virtualization, containerization and cloud to help researchers in their simulation workflow

� Already great success stories: Gordon Bell Finalists 2018

� Still an open problem: Warrant on “Who is @HPC_Guru?” #UnmakHPC_Guru

Relationship between HPC and Big Data Analytic (BDA)

Agenda� Panel presentations ( 10 min each)

� Maxime Martinasso on for the convergence of HPC ecosystem.

� Katie Antypas on solutions developed at NERSC for Big Data users and the lessons learned for future procurements to meet new workflow.

� Daniel A. Reed on the recommendation from Big Data and Extreme-scale Computing (BDEC)

� Rio Yokota on moving forward to further reduce the gap between HPC and Big Data in term of applications at large scale.

� Audience � Ask questions/comments ( use SC18 website for written questions)

Open Questions� What are the policy challenges for production runs with HPC and BDA workflow in

regards to the storage and workload manager?

� Are the current and projected developments of HPC systems and software aligned with the needs of scientific community?

� Can users influence these developments or do they need to adapt to the new hardware?

� What are the challenges and the opportunities that the community will face out of this synergistic relationship?

� How to develop sustainable competencies around selecting, implementing, and managing new technologies to support diverse workload?