hep data grid in korea

HEP Data Grid in Korea

Kihyeon ChoCenter for High Energy PhysicsKyungpook National University

and

Hyoungwoo ParkSupercomputing Center, KISTI

(On behalf of the HEP Data Grid Working Group in Korea)

The 3rd International Workshop on HEP Data Grid August 26-28, 2004

KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS

ContentsContents

Goal of Korea HEP Data Grid Regional Data Center for CMS

Network Storage Computing

Application for HEP Data Grid CDF Belle AMS

Conclusions


Tier 1

Tier2 Center

Online System

eventreconstruction

French Regional Center

German Regional Center

InstituteInstituteInstituteInstitute ~0.25TIPS

Workstations

~100 MBytes/sec

~0.6-2.5 Gbps

100 - 1000

Mbits/sec

Physics data cache

~2.5 Gbits/sec

Tier2 CenterTier2 CenterTier2 Center

~0.6-2.5 Gbps

Tier 0 +1

Tier 3

Tier 4

Tier2 Center Tier 2

CERN/CMS data goes to 6-8 Tier 1 regional centers, and from each of these to 6-10 Tier 2 centers.

Physicists work on analysis “channels” at 135 institutes. Each institute has ~10 physicists working on one or more channels.

2000 physicists in 31 countries are involved in this 20-year experiment in which CERN/DOE are major players.

CMS detector: 15m X 15m X 22m

12,500 tons, $700M.

human=2m

analysis

event simulation

Italian Center FermiLab, USA Regional Center

Courtesy Harvey

Newman, CalTech and

CERN

CHEP, Korea Regional Center

Goal of Korea HEP Data GridGoal of Korea HEP Data Grid


Korea

To make Tier1 Regional Data Center for LHC-CMS Experiment.

Other Experiments (CDF, Belle, AMS, PHENIX) may use it.


Korea HEP Data Grid ActivitiesKorea HEP Data Grid Activities

History 2001.10, Organized working group for HEP Data Grid 2002. 1, Access Grid workshop 2002.11, 1st international workshop on HEP Data Grid 2003. 8, 2nd international workshop on HEP Data Grid 2004. 8, 3rd international workshop on HEP Data Grid

Plan 2005. 5, Networking, HEP Grid and Digital divide workshop

Working Group Activities HEP Data Grid working Group Advance Network Forum HEP working group APII/TEIN physics working Group


Europe CERN (CMS)

US FNAL (CDF)

US BNL (PHENIX)

Japan KEK (Belle)

Korea CHEP

Space Station (AMS)

Korea CHEP at CERN

Experiments for HEP Data Grid Project in Korea

Regional Data Center


This year’s ActivitiesThis year’s Activities1. Regional Data Center for CMS

Network => See Kihwan Kwon’s talk Storage – SRB connectivity with KISTI and

KBSI Computing - Grid3+, LCG

2. Application for HEP Data Grid CDF Belle =>See Youngjoon Kwon’s talk AMS => See Guinyun Kim’s talk


Regional Data Center for CMSRegional Data Center for CMS


SRB (Storage Resource Broker)SRB (Storage Resource Broker)

SRB(Storage Resource Broker) To provide homogeneous interface at heterogeneous

data resource. To access dataset easily and provide copy and store To handle metadata using MCAT (Metadata Catalog)

Connection KISTI, KBSI and CHEP To connect KEK and CHEP


SRBSRB connection between connection between KISTI and KBSIKISTI and KBSI

SRB for HPSS at KISTI

SRB for cluster at KNU

Can connect KBSI server using MCAT

To install ZoneSRB (SRB3.1) at cluster38


2003: Demo at SC2003. 28 Sites(1 Korea, 27 US, ~2000CPUs) VO: USCMS

2004: Number of CPUs : 3 CPUs -> 84CPUs Dedicated CPU:3 Dual batch system of CMS(28CPU), DCAF(53CPU)

Middleware is up-to-date: VDT upgrade completed on July 15 (VDT1.11->1.14)

Grid3+Grid3+


Grid3+ MonitoringGrid3+ Monitoring


[root@cluster28 root]# condor_q

-- Submitter: cluster28.knu.ac.kr : <155.230.20.58:38142> : cluster28.knu.ac.kr ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD 36835.0 uscms02 6/9 03:48 1+10:06:33 R 0 3.7 data 38914.0 uscms02 6/24 03:38 0+17:09:52 R 0 3.7 data 42606.0 btev 7/16 13:47 0+16:10:51 R 0 5.3 data -t 313 12024 44323.0 uscms02 7/22 08:53 0+01:44:15 R 0 3.6 data 44395.0 ivdgl 7/22 09:15 0+01:35:48 R 0 5.3 data -t 159 11053 44405.0 lsc01 7/22 09:17 0+01:28:55 R 0 5.3 data -t 305 11041 44407.0 lsc01 7/22 09:17 0+01:18:13 R 0 5.3 data -t 307 11043 44424.0 ivdgl 7/22 09:18 0+01:31:20 R 0 5.3 data -t 265 12066 44427.0 ivdgl 7/22 09:19 0+01:28:20 R 0 5.3 data -t 268 12069 44429.0 usatlas1 7/22 09:20 0+01:38:06 R 0 5.3 data -t 313 12049 44430.0 usatlas1 7/22 09:20 0+01:34:08 R 0 5.3 data -t 314 12050 44432.0 usatlas1 7/22 09:20 0+01:39:57 R 0 5.3 data -t 291 12052 44433.0 usatlas1 7/22 09:20 0+01:35:28 R 0 5.3 data -t 290 12051 44434.0 usatlas1 7/22 09:20 0+01:31:07 R 0 5.3 data -t 292 12053 44435.0 usatlas1 7/22 09:20 0+01:26:20 R 0 5.3 data -t 293 12054 44436.0 usatlas1 7/22 09:20 0+01:33:07 R 0 5.3 data -t 294 12055 44437.0 usatlas1 7/22 09:20 0+01:28:27 R 0 5.3 data -t 295 12056 44756.0 uscms02 7/22 18:16 0+00:08:59 R 0 0.0 data 44757.0 uscms02 7/22 18:22 0+00:07:20 R 0 0.0 data 44758.0 uscms02 7/22 18:23 0+00:06:58 R 0 0.0 data

20 jobs; 0 idle, 20 running, 0 held

Grid3+ at KNU siteGrid3+ at KNU site


LCG (LHC Computing Grid)LCG (LHC Computing Grid)

To join LCG

Installing LCG2 test bed

Making Korean HEP CA


LCG2 StatusLCG2 Status


LCG InteroperabilityLCG Interoperability

Korea CMS is trying to allow transparent use of KNU resources through LCG and Grid3+.

Interoperating grid resources can be approached in a number of ways.


iVDGL

Data Challenge (Fermilab-CHEP)

CERN LHC

Grid3+

CMS Grid

Testbed

CERN Tier0

LCG2Batch Q

Korea CMS GridKorea CMS Grid

18 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

WN Node

LCG CEGrid3 CEPBS

WN Node

WN Node

WN Node

WN Node

WN Node

LSFCondor-G

WN Node

WN Node

WN Node

WN Node

WN Node

CEBatch Q

Korea CMS TestbedKorea CMS Testbed

Copy from USCMS Testbed


A Photo of ClustersA Photo of Clusters


Grid Testbed at CHEPGrid Testbed at CHEP

Connecting among KNU, KT, KISTI and SKKU

To use all work nodes as LCG work nodes

LCG Testbed


Production FarmRef. Mark Neubauer

CDF GridCDF Grid


Requirement set by goal 2005 200 simultaneous users should

analyze 107 events in a day. Need ~700 TB of disk and ~5THz

of CPU by end of FY’05 DCAF (DeCentralized Analysis Farm)

SAM (Sequential Access through Meta data) To handle real data at Fermilab for

DCAF around world Gridification of DCAF via SAM Grid

KorCAF (DeCenteralized Analysis Farm in Korea)

Why DCAF and CDF Grid?Why DCAF and CDF Grid?

CAF(Central Analysis Farm)

DCAF(DeCentralized Analysis Farm)

CDF SAMGrid

CDF Grid


INFN users only

KNU(KorCAF)

Fermilab (CAF)

SAMGrid

users

Korea, Taiwan and Italy already is working for a User MC Farm

INFN(Italy)

Tawian

Scheme of DCAFScheme of DCAF


User PerspectiveUser Perspective

Uses SAM

Only FermilabOutside Lab

Uses SAM

Grid

Oct. 2004, JIM will deployed


CAF Gui

User PerspectiveUser Perspective

DCAF Gui

Grid

Toronto ItalyKorea Taiwan FermiCAF UK


Disk July 04

FNAL

INFN

Taiwan

Korea

UCSD

Rutgers

Canada

CAF + DCAF July 2004

FNAL

INFN

Taiwan

Korea

UCSD

Rutgers

Canada

Disk - Dec 04

GridKa

FNAL

INFN

Taiwan

Korea

UCSD

Rutgers

Canada

Japan

MIT

UK

Dec 2004 DCAF+JIM

GridKa

FNAL

INFN

Taiwan

Korea

UCSD

Rutgers

Canada

Japan

MIT

UK

July

Dec

Disk CPU

04

04

FNAL FNAL

FNALFNAL

CDF Computing PlanCDF Computing Plan


Summary for CDF GridSummary for CDF Grid

CDF can capture more resources using the Grid to achieve its physics mission.

DCAF and SAM is working for CDF and will reduce operational loads.

We add new features and rely on software supported for or by the LHC.


Korea Belle GridKorea Belle Grid

Globus connection among SKKU, KISTI and KNU

To construct SRB with KEK and KNU

To connect between KNU-KEK and Australia


* KNU-KEK Belle Computing Farm

KNU & SKKU KEK Belle

Direct QQ Simulation

File control and transfer

Geant Simulation

Analysis

Data Storages

GEANT Simulation

Analysis

Data Storages

Globus Environment

SRB

KISTI site

Plan

Compared between Grid Testbed and KEK testbed Compared between Grid Testbed and KEK testbed using Busing B00->J/->J/ψψKKss ( (Sunmin Kim’s M.S.’s thesis)Sunmin Kim’s M.S.’s thesis)

Korea Belle with KEKKorea Belle with KEK


* GSIM event Display in GT environment

1. Access to GT environment

2. Interactive executable under GT env.

1

2

Event Display at Grid Tool Kit Event Display at Grid Tool Kit


Summary for Korea BelleSummary for Korea Belle

Korea Belle Grid system is operating at KNU, SKKU and KISTI testbed

SC2003 demo – KorBelle To install SRB between KISTI-KNU-KEK Korea-KEK-Australia

On Oct 2003, PRAGMA – Data Grid Working Group has discussed basic understanding on how KorBelle and Australian Belle Grid work together.

To work with KEK about Korea-Japan Data Grid and Australia based on network and SRB

More information on

Prof. Youngjoon Kwon’s talk


AMS GridAMS Grid

Using Grid environment between KNU and KT

Testing file transfer using bbftp between CHEP-CERN and CHEP - Zurich

OpenSSI (Single System Image)

=> Science Data Center

Tape Library(~200 TB)

Disk Storage(20TB)

DB Server

Gigabit Ethernet

Linux ClustersCluster Servers

Hub

Internet Server

AMS RC

CHEP

Analysis Facility

Data Storage

Display Facility

Ewha

AMS RS

무궁화위성


Grid Application at AMSGrid Application at AMS


{chep18.knu.ac.kr:ams machine}2004JUN1-1/ams02mcscripts.tar.gzgen_jobs.pl ams02mcscripts.tar.gzknu.***1.{PART#}.*.jobknu.***2.{PART#}.*.jobknu.***3.{PART#}.*.jobknu.***4.{PART#}.*.jobrun.***.1.sh -> for submitting jobs to Queuing Serverrun.***.2.shrun.***.3.shrun.***.4.shlist_jou.*** -> control file will be used by bbftplist_log.***list_root.***after running,the gen_jobs.pl will check files and update list_jou,list_root,list_log .

{chep18.knu.ac.kr:ams machine}uploading files to CERN via bbftp :bbftp -i list_root.*** -u 'jwshin' pcamsf2.cern.chbbftp -i list_jou.*** -u 'jwshin' pcamsf2.cern.chbbftp -i list_log.*** -u 'jwshin' pcamsf2.cern.ch{speed : 1~9 MB/s}

CERN DB

gen_jobs.pl (perl script)

File Transfer at AMSFile Transfer at AMS

More information on

Prof. Guinyun Kim’s talk


ConclusionsConclusions

To participate SC2004 Bandwidth Challenge with Caltech

Installing LCG2 and to join LCG

To apply SRB between KISTI, KBSI, KNU, KEK

The application of CDF, Belle, AMS Grid -> SC2004

hep data grid in korea

Documents

hep data gridaugust

hep data grid2002

hep data grid2003

hep data grid2004

hep data gridplan2005

cerncms data

hep grid

tier1 regional data