Tue-4-Mar, 10:05am, F.Boudra, M.Wasilewski

LCA14-203: Intro to CI@Linaro 1 of 3

Hour 1● QA Services will introduce you to existing Linaro Test Plans● Builds and Baselines will provide an overview to our build system, how

to create, manage and close out a build and how to submit the build artifacts to LAVA for a test on lab hardware.

Hour 2● LAVA will describe the process of adding new test capabilities (New

Platforms and/or New external test equipment)● QA Services will overview how they use LAVA for tests

Hour 3● LAVA will show you new features to improve test result filtering and

Visualization in LAVA● QA Services will show you how Linaro rolls all the Build results, Test

Results (Manual / LAVA) into our test reports

Intro to CI @ Linaro

• Engineering builds• Android• OpenEmbedded• Ubuntu• LSK

• Working groups• LEG• LNG

• Requesting new Test Plan• Contact QA team (Milosz Wasilewski)• File REQ card• Creating new test plans will be covered on Thursday session

Existing Test Plans

• Some test plans are not public (like LSK)• Sharing to selected people isn’t easy as it implies google

account• Turning ‘paper’ test plan into JSON test definition is

done manually by editing Jenkins configs

Existing Test Plans - problems

• Jenkins as the build system• android-build.linaro.org for Android builds• ci.linaro.org for everything else

• individual components• acpica, libvirt QEMU, UEFI, etc...

• Ubuntu and OpenEmbedded based engineering builds• hardware packs, rootfs, pre-built images

• Kernel CI• mainline, ARM SoC (for-next), linux-next, linux-stable• LSK, Linux Linaro, LEG, LNG

• LAVA as the automated testing system• validation.linaro.org, Linaro’s production server

• running LAVA jobs on Cambridge lab hardware, restricted to Linaro• community.validation.linaro.org, Linaro’s Community server

• running LAVA jobs on community hardware, open to everybody

Anatomy of CI loop

• Select a view• Click “New job” button

• the new job is created on the selected view

How to create a build job

• Define a job name• avoid names with spaces, it’s used in the URL

• Select the build job type• free-style project• Multijob project• Multi-configuration project• Copy an existing job

How to create a build job

• Configure the general options• discard old builds

• days to keep build• max number of builds to keep

• enabled project based security• users/groups from login.linaro.org and “anonymous” account• permissions: configure, read, extended read, build, worskpace

• this build is parametrized• file parameter (properties), password parameter

• restrict where this project can be run• label expression• various builders (ephemeral vs persistent, regular user vs sudoer)

• Amazon EC2 instances• VPS instances• native builders

How to create a build job

• Configure the advanced project options• use custom workspace (useful for multi-configuration project)

• directory for sub-builds “.”• Configure source code management

• git plugin• repository URL• advanced -> refspec• branch specifier• additional behavior

• advanced clone behaviours -> shallow clone• check out to a sub-directory• clean after checkout• wipe out repository and force clone

How to create a build job

• Configure build triggers• trigger builds remotely (e.g. git hook)• build periodically (cron syntax)• gerrit event• URL trigger (useful in particular use cases)

• polling, should be run on persistent slave!• Configure configuration matrix (only for multi-

configuration project)• add axis

• dynamic axis (e.g. defconfig list)• slaves• user-defined (e.g. custom string for gcc version, rootfs, etc...)

• run each configuration sequentially• execute touchstones build first

How to create a build job

• Configure build environment• abort the build if it’s stuck• add timestamps to the console output• copy files into the job’s workspace (e.g. Linaro GCC toolchain)• set build name (e.g. GIT_REVISION, require to use git plugin)• tie parent to a build node (only for multi-configuration)

• Configure build - 1/3• “execute shell”

• run shell script, written inline or pulled from SCM• can use regular git cli instead of Jenkins git plugin• some environment variables are defined by Jenkins (axis too)• can use Linaro CI build tools to build kernels• can use Linaro jenkins setup scripts to build OpenEmbedded images• generate configuration(s) to submit LAVA job (post-build-lava CI helper) • generate BUILD-INFO.txt (required for publishing on snapshots)

• define ACLs

How to create a build job

• Configure build - 2/3• “send files or execute command over ssh” to publish build artifacts on

snapshots.linaro.org• it’s a two step process, transfer files then move files

• SSH server: snapshots.linaro.org■ define “source files”■ define “remote directory”: ${JOB_NAME}

/${BUILD_NUMBER}/path/to/artifact• add server -> SSH server: snapshots.linaro.org file-move

■ exec command: “reshuffle-files --job-type prebuilt --job-name ${JOB_NAME} --build-num ${BUILD_NUMBER}”

• advanced -> select “always SSH from master”

How to create a build job

• Configure build - 3/3• trigger/call builds on other projects

• ubuntu-builder, CI helper used to cross build debian packages• ubuntu-builder-native, CI helper used to build naively debian packages• post-build-repo, CI helper used to upload packages to Linaro repository• post-build-lava, CI helper used to submit a LAVA job (JSON job

definition)• support fully custom job definition (CUSTOM_JSON_URL)• support pre-built image (IMAGE_URL)• support custom linaro_deploy_image parameters

(DEPLOY_IMAGE_PARAMETERS_URL)• support custom metadata (METADATA_URL)• support custom boot image parameters

(BOOT_IMAGE_PARAMETERS_URL)• partial support for multi-node jobs

How to create a build job

• Configure post-build actions• scan for compiler warnings (GCC, LLVM, doxygen, etc...)• groovy postbuild

• powerful groovy scripting, access to Jenkins API (e.g. used to add a direct link to LAVA job id, or rename a build job dynamically)

• Jenkins text finder (use regexp to find pattern)• e-mail notification

How to create a build job

• Improve the publishing API• Get latest rootfs/hwpack using API• Support Jenkins CLI• Jenkins job descriptions in human readable format,

maintained under SCM (Jenkins job builder)

Improvements

Any Questions?

Q&A

More about Linaro Connect: http://connect.linaro.orgMore about Linaro: http://www.linaro.org/about/

More about Linaro engineering: http://www.linaro.org/engineering/Linaro members: www.linaro.org/members