belle-ii data handling working group status junghyun kim, daejung kong kihyeon cho* (high energy...
TRANSCRIPT
![Page 1: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/1.jpg)
Belle-II Data Handling working group status
Junghyun Kim, Daejung KongKihyeon Cho*
(High Energy Physics Team, KISTI)
1
Korea Belle II meetingSNU, Seoul, Korea
2009. 6. 25
* Corresponding Author
![Page 2: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/2.jpg)
Contents
• KISTI Supercomputing center• Activities of Belle II DH working group• Problems of Belle Data Handling • Status of Belle II Data Handling• Plan
2
![Page 3: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/3.jpg)
3
응용지원실
융합자원실
응용지워팀
선도연구팀
슈퍼컴 자원팀 연구망 자원팀
사이버인프라 응용연구자
KISTI KISTI 슈퍼컴퓨팅 본부슈퍼컴퓨팅 본부KISTI KISTI 슈퍼컴퓨팅 본부슈퍼컴퓨팅 본부SC-SPSC-SP
SMB 팀
과학기술정보보호팀
차세대연구환경개발실
기반기술개발실
HEP연구팀
응용환경개발팀
응용서비스팀 서비스기반기술팀
e-Cell 팀
NSDC 구축팀
과학기술융합망팀
GEOSS 팀
일반사용자 전략분야
사용자
서비스기술사용자
협업가시화팀
e-Science 그리드기술개발팀
Ref. 이지수본부장 (2009.3.12)
![Page 4: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/4.jpg)
4
Who is who?
• 고에너지물리연구팀 (Physicists)– 조기현 ( 팀장 )– 김정현 , 공대정 , 남수현
• e-Science 그리드 기술개발팀 (Developers)– 황순욱 ( 팀장 )– 유상수 , 김유상 , 김남규 , 안선일 등
• NSDC 구축팀 (Computing Technicians)– 장행진 ( 팀장 )– 김법균 , 유진승 , 윤희준 , 권석면 , Christopher 등
• Other teams– 김현우 , 이상동 , 김정한 , 홍태용 등
![Page 5: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/5.jpg)
Belle II DH Working Group
• Group Leader: K. Cho• Participants: J.H. Kim, D.J.Kong, Thomas
Kuhr, Takanori Hara, Tom Fifield, Nishida Shohei, Martin Sevier, Henryk Palka
• KISTI AMGA group: Sunil Ahn, Soonwook Hwang
5
![Page 6: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/6.jpg)
6
![Page 7: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/7.jpg)
7
![Page 8: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/8.jpg)
Group Activities• Data Handling Working Group Meeting
– May 27, 2009 16:00-17:00– Contents
• Define meta data• Extraction of meta data• Interaction between AMGA and Tom’s activities• Discussion – Attributes of meta data
– Existing Belle database– SAM metadata structure (CDF/D0)– Different levels of metadata
8
![Page 9: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/9.jpg)
Group Activities• Belle II Computing Meeting Report:
– Feb. 17, 09 - Introduction of AMGA (Dr. Hwang)– April 17, 09 - Plan (Dr. J.H. Kim)– May 20, 09 - DH Scenario (Dr. J.H. Kim)– June 16, 09 - The architecture in AMGA (Dr. J.H.Kim)
9
![Page 10: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/10.jpg)
10
![Page 11: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/11.jpg)
Problems of Belle Data Handling
• DH based on PHP, HTTP and postgreSQL
• No Grid • Durability• Scalability
SAM, SRB (No Grid) AMGA (Grid) Working with KISTI AMGA group
11
![Page 12: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/12.jpg)
12
![Page 13: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/13.jpg)
Belle II Data Handling
• Scenario• Attribute of Meta Data• Architecture
13
![Page 14: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/14.jpg)
14
![Page 15: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/15.jpg)
15
![Page 16: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/16.jpg)
Extraction of Attributes
Done
Doing
16
![Page 17: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/17.jpg)
Total used data Exp#7, Run 6-2865, uds MC
Extraction of information
17
?Stream
![Page 18: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/18.jpg)
-rw-r--r-- 1 cho hep 125 May 25 12:57 mcprod-on_resonance-evtgen-uds-00-all-e000007r002865-b20030807_1600.list-rw-r--r-- 1 cho hep 310980 May 25 12:57 mcprod-on_resonance-evtgen-uds-00-all-e000007r002865-b20030807_1600.log-rw-r--r-- 1 cho hep 373 May 25 12:57 mcprod-on_resonance-evtgen-uds-00-all-e000007r002865-b20030807_1600.lum
File from MDST
1) ***. list => Logical file location2) ***.log => Properties of events3) ***.lum => File (Run) information
The information
18
![Page 19: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/19.jpg)
[cho@hep on_resonance]$ more mcprod-on_resonance-evtgen-uds-00-all-e000007r002865-b20030807_1600.listbfss08:/bdata/mcprod/dat/e000007/evtgen/uds/00/all/0807/on_resonance/28/evtgen-uds-00-all-e000007r002865-b20030807_1600.mdst
[cho@hep on_resonance]$ more mcprod-on_resonance-evtgen-uds-00-all-e000007r002865-b20030807_1600.log0: Parameters ; 7 ; 2865 ; 1 ; 3 ; 3 ; 0 ; 10: Parameters ; 7 ; 2865 ; 5 ; 4 ; 4 ; 1 ; 00: Parameters ; 7 ; 2865 ; 6 ; 5 ; 4 ; 3 ; 01: Parameters ; 7 ; 2865 ; 2 ; 5 ; 4 ; 2 ; 10: Parameters ; 7 ; 2865 ; 8 ; 3 ; 4 ; 1 ; 00: Parameters ; 7 ; 2865 ; 9 ; 0 ; 2 ; 0 ; 01: Parameters ; 7 ; 2865 ; 7 ; 6 ; 5 ; 3 ; 11: Parameters ; 7 ; 2865 ; 12 ; 2 ; 2 ; 1 ; 40: Parameters ; 7 ; 2865 ; 10 ; 6 ; 6 ; 1 ; 02: Parameters ; 7 ; 2865 ; 4 ; 4 ; 4 ; 1 ; 41: Parameters ; 7 ; 2865 ; 11 ; 1 ; 1 ; 0 ; 00: Parameters ; 7 ; 2865 ; 13 ; 5 ; 5 ; 4 ; 03: Parameters ; 7 ; 2865 ; 3 ; 5 ; 5 ; 3 ; 11: Parameters ; 7 ; 2865 ; 15 ; 6 ; 5 ; 2 ; 02: Parameters ; 7 ; 2865 ; 14 ; 4 ; 3 ; 1 ; 00: Parameters ; 7 ; 2865 ; 16 ; 8 ; 8 ; 4 ; 11: Parameters ; 7 ; 2865 ; 19 ; 4 ; 4 ; 2 ; 13: Parameters ; 7 ; 2865 ; 17 ; 3 ; 7 ; 3 ; 00: Parameters ; 7 ; 2865 ; 20 ; 3 ; 4 ; 0 ; 02: Parameters ; 7 ; 2865 ; 18 ; 5 ; 4 ; 5 ; 2………..=> Dummy; Dummy; Exp.#; Run#; Evt.#; Total #of positive charged track; Total # of negative charged track; Total # of Ks; Total #of KL
=> Logical file location
1) ***.list
2) ***.log
19
![Page 20: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/20.jpg)
[cho@hep on_resonance]$ more mcprod-on_resonance-evtgen-uds-00-all-e000007r002865-b20030807_1600.lum Exp Run #evts Lum #evts/lum 4S Continuum 3S above 4S other 7 2865 6583 3.15 2089.84 3.15 0 0 0 0Total On4S lum. = 3.15 Continuum Lum. = 0 3S Lum. = 0 Above4S Lum. = 0 Other Lum. = 0
Exp.#, Run#, Total # of events, Total Lum./file, #evts/lum, Luminosity type
=> File information
3) ***.lum
20
=> Any suggestions about attributes?
![Page 21: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/21.jpg)
Architecture of AMGA
21
![Page 22: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/22.jpg)
22
![Page 23: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/23.jpg)
23
![Page 24: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/24.jpg)
24
![Page 25: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/25.jpg)
25
![Page 26: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/26.jpg)
Plan
• MC & Real data test with AMGA– Using Belle flat form– Writing and Reading
• Long Term – Prototype Oct. 2009
• Birger visit KISTI for two weeks (7/8/09)– Hand down AMGA to KISTI AMGA group– Questions on AMGA
26
![Page 27: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/27.jpg)
Questions on AMGA
• Scalability– 8 million events * 50 times
• Durability– How many users?
• Performance– How fast?
27
![Page 28: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/28.jpg)
Backup
28
![Page 29: Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,](https://reader035.vdocuments.site/reader035/viewer/2022062217/5697bf831a28abf838c8680d/html5/thumbnails/29.jpg)
29