dpdk ci & open lab · 2020-03-21 · dpdk ci & open lab dpdk userspace –dublin - 2018...
TRANSCRIPT
![Page 1: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/1.jpg)
x
DPDK CI & Open Lab
DPDK Userspace – Dublin - 2018
Lijuan Tu,Intel
Jeremy Plsek,UNH-IOL
![Page 2: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/2.jpg)
Agenda
Continuous Integration Status
Proposal for CI improvement
CI in Open Lab
Open Lab CI dashboard
![Page 3: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/3.jpg)
3
DPDK CI introduction
dpdk.org
CI control center
(Jenkins)
Server
Server
re
po
rt
p
ull
Build
Function test
Performance test
Reposistories)(Patchwork
![Page 4: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/4.jpg)
4
DPDK CI current status
Before merge After merge
build ✔(Intel) ✔(Intel)
Unit test ❌ ✔(Intel)
Function test ❌ ✔(Intel)
Performance test ✔(Open lab)❌
DPDK CI has two parts:
Before merge: Process of new patch set which haven’t merge into any tree
After merge: Process of main-tree (master)
✔ Implemented
❌ Gap
![Page 5: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/5.jpg)
5
Proposal for CI improvement
Gaps
Before merge• Need identify patch set to go to which tree• Need cover unit test and function test• Need more performance test
After merge• Need cover performance test
Proposal• Two options on tree-selected issue.
• Step by step cover the test gaps
• Before merge > after merge
![Page 6: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/6.jpg)
6
Workflow - Before merge
Centos/gcc/x86_64startPull main-tree
(master)
Downloadpatch set
Apply patchset
Ubuntu/clang/x86_32
OS*/Compiler*/arch* Performance test
Function test
Unit test
End
...... ......
Report to dpdk.org
Whenever error or final pass
Stage 1 Stage 2 Stage 3
![Page 7: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/7.jpg)
7
Workflow - Before merge
Centos/gcc/x86_64startPull main-tree
(master)
Downloadpatch set
Apply patchset
Ubuntu/clang/x86_32
OS*/Compiler*/arch* Performance test
Function test
Unit test
End
...... ......
Report to dpdk.org
Whenever error or final pass
Stage 1 Stage 2 Stage 3
![Page 8: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/8.jpg)
8
Options1. Go to next-master
2. Go to exact sub-tree
next-eventdev
next-net
next-crypto
next-virtio
dpdk-master
Patch set
Open: Which tree patch set should go ?
![Page 9: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/9.jpg)
9
Option 1: Go to next-master
next-master: Local tree which formed by merging all sub-trees into master
patch set master
next-master
next: build/testapply
apply
Report to dpdk.org
pass
conflict
pass
conflict
![Page 10: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/10.jpg)
10
Option 2: Go to exact sub-tree
• Information in MAINTAINERS file
• Script to analyze the git history and gives a percentage of probability.
sub treepatch set
![Page 11: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/11.jpg)
11
Option comparison
Go to next-master (Preferred) exact sub-tree
Pros✔ 1. No dependency
2. No extra script
3. Already have weekly merge: sub-trees to master
4. Early to find issues
1. Timely reports
Cons❌ 1. Will be delayed if merge conflict
2. Maintainers need to manually fix conflict
1. Extra script effort
2. Still uncertainty
![Page 12: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/12.jpg)
12
Workflow – After merge
Centos/gcc/x86_64start
Download main-tree (master)
Ubuntu/clang/x86_32
OS*/compiler*/arch* Performance test
Function test
Unit test
End
...... ......
Report to dpdk.org
Whenever error or final pass
Stage 1 Stage 2 Stage 3
![Page 13: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/13.jpg)
13
DPDK CI in Open Lab
Future plan (before merge > after merge)1. CI before merge implementation deploy (option 1)
2. More basic & critical performance test
3. Expend to function test & unit test
Before merge After merge
build ✔(Intel ) ✔(Intel)
Unit test Open lab ✔(Intel )
Function test Open lab ✔(Intel )
Performance test ✔(Open lab) Open lab
![Page 14: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/14.jpg)
14
Open Lab CI dashboard
• What is it?
• Who is it hosted by?
• What are the plans for the future?
• How to get involved?
![Page 16: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/16.jpg)
16
Detailed View
• View relative performance
numbers
• View hardware configuration
information
• Download artifacts for logged
in members
![Page 17: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/17.jpg)
17
Who is it hosted by?
• University of New Hampshire InterOperability Laboratory (UNH IOL)
• Provides a neutral environment for members
• Established in 1988 with over 100 currently participating companies
• Current Open Lab Participants: Intel, Mellanox, NXP
![Page 18: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/18.jpg)
18
What are the plans for the future?
• Running different test cases
• User management
• Graphing Integration
• Continuous Integration
![Page 19: DPDK CI & Open Lab · 2020-03-21 · DPDK CI & Open Lab DPDK Userspace –Dublin - 2018 Lijuan Tu,Intel Jeremy Plsek,UNH-IOL. Agenda Continuous Integration Status Proposal for](https://reader034.vdocuments.site/reader034/viewer/2022052320/5f07914f7e708231d41da0e3/html5/thumbnails/19.jpg)
19
Getting Involved
Dashboard: https://lab.dpdk.org/
Issue reporting: https://bugs.dpdk.org/
CI Mailing List: http://mails.dpdk.org/listinfo/ci
Submitting Equipment: https://www.iol.unh.edu/testing/hpc/dpdk
Contact: [email protected]