install concourse ci with bosh

InstallConcourse  CIwith  BOSH

Toshiaki  Maki  (@making)Sr.  Solutions  Architect  @Pivotal2016-‐‑‒05-‐‑‒19Concourse  CI  Tokyo  Meetup

Who  am  I  ?•Toshiaki  Maki  (@making)•https://blog.ik.am•Sr.  Solutions  Architect•Spring  Framework  enthusiast

SpringFramework徹底⼊入⾨門(Coming  Soon?)

PerfectJava  EE(Coming  Soon?)

How  to  Install

•Vagrant  Image•Executable  Binaries•BOSH  Release

https://concourse.ci/installing.html

Easy

Hard Production  Ready

Local  Dev

Vagrant

$  vagrant  init concourse/lite$  vagrant  up

Vagrant

BOSH•Powerful  Provisioning  Tool  to  deploy  distributed  system•Monitor  /  Health  Check• Rolling  Updates•Networking• Storage  Management• VM  Provisioning• IaaS Agnostic• ...

Do  we  really  need  BOSH?"If  you're  not  yet  familiar  with  BOSH,  learning  it  will  be  a  bit  of  an  investment,  but  itshould  pay  off  in  spades"

http://concourse.ci/clusters-‐‑‒with-‐‑‒bosh.html

BOSH  components

https://bosh.io/docs/bosh-‐‑‒components.html

How  to  install  with  BOSH1. Set  up  IaaS2. Install  BOSH  Director  using  bosh-‐‑‒init3. Upload  Stemcell4. Upload  Releases5. Create  manifest  file6. Deploy  the  system  using  manifest  file

•Deploy  Concourse  to  AWS•Deploy  Concourse  to  Azure

Manifest  samples

https://github.com/making/concouse-‐‑‒ci-‐‑‒manifest-‐‑‒samples

Deploy  Concourse  to  AWS

Prepare  VPC

VPC  (10.0.0.0/16)public(10.0.0.0/24) private(10.0.1.0/24)NATEIP

Install  BOSH  Director

Create  Keypair

Security  Group

Manifest  for  bosh-‐‑‒init• https://bosh.io/docs/init-‐‑‒aws.html#create-‐‑‒manifest

Copy  &  Pasteto  bosh.yml

Replace  placeholdersName ValueELASTIC-‐‑‒IP <Your  New  EIP  for  Director> there are  3  places  to  

replaceSUBNET-‐‑‒ID subnet-‐‑‒xxxxxxxx public subnetAVAILABILITY-‐‑‒ZONE ap-‐‑‒northeast-‐‑‒1aACCESS-‐‑‒KEY-‐‑‒ID xxxxxxxxSECRET-‐‑‒ACCESS-‐‑‒KEY xxxxxxxxREGION ap-‐‑‒northeast-‐‑‒1

DeployDownload  from  https://bosh.io/docs/install-‐‑‒bosh-‐‑‒init.html

$ bosh-init deploy bosh.yml

After  20min

VPC  (10.0.0.0/16)public(10.0.0.0/24) private(10.0.1.0/24)NATBOSHDirector

EIP

EIP

$ sudo gem install bosh_cli --no-ri --no-rdoc

$ bosh target 52.196.159.101Target set to `my-bosh'Your username: adminEnter password: Logged in as `admin'

IaaS Specific  Configuration

az: ...vm_types: ...disk_types: ...networks: ...compilation: ...

aws.yml

IaaS Specific  Configuration

azs:- name: z1 cloud_properties:

{availability_zone: ap-northeast-1a}

aws.yml

IaaS Specific  Configurationvm_types:- name: web

cloud_properties: instance_type: t2.micro ephemeral_disk: {size: 3000, type: gp2}security_groups: [web, bosh]

- name: databasecloud_properties: ...

aws.yml

IaaS Specific  Configuration

disk_type:- name: database

disk_size: 50_000 cloud_properties: {type: gp2}

aws.yml

Set  Iaas Config

$ bosh update cloud-config aws.ymlActing as user 'admin' on 'my-bosh'Successfully updated cloud config

Upload  Stemcell

$ bosh upload stemcellhttps://bosh.io/d/stemcells/bosh-aws-xen-hvm-ubuntu-trusty-go_agent?v=3232.3

Upload  Releases  (garden,  concourse)

$ bosh upload release https://bosh.io/d/github.com/cloudfoundry-incubator/garden-linux-release?v=0.337.0$ bosh upload release https://bosh.io/d/github.com/concourse/concourse?v=1.2.0

name: concoursedirector_uuid: ...releases: ...stemcells: ...instance_groups: ...update: ...

Create  manifest  for  Concourseconcourse.yml

Create  manifest  for  Concourseinstance_groups: ...- name: web

...- name: db

...- name: worker

...

concourse.yml

Deploy  Concourse  CI

$ bosh deployment concourse.yml$ bosh -n deploy

VPC  (10.0.0.0/16)public(10.0.0.0/24) private(10.0.1.0/24)NATBOSHDirector

EIP

EIP

WebEIP Worker

DB

Scale  out

- name: workerinstances: 3vm_type: worker

concourse.yml

$ bosh -n deploy

VPC  (10.0.0.0/16)public(10.0.0.0/24) private(10.0.1.0/24)NATBOSHDirector

EIP

EIP

WebEIP

WorkerWorkerWorker

DB

Health  Check

Health  Check

Version  up

$ bosh upload release https://bosh.io/d/github.com/concourse/concourse?v=1.3.0$ bosh deploy

Deploy  Concourse  to  Azure

Use  templatehttps://azure.microsoft.com/en-‐‑‒us/documentation/templates/bosh-‐‑‒setup/

After  1  hour  ...

Already  prepared👍

$ ssh bosh@40.115.251.81

bosh@bosh:~$ bosh targetCurrent target is https://10.0.0.4:25555 (bosh)

IaaS Specific  Configuration

az: ...vm_types: ...disk_types: ...networks: ...compilation: ...

azure.yml

IaaS Specific  Configurationvm_types:- name: web

cloud_properties: instance_type: Standard_D1

- name: databasecloud_properties: ...

azure.yml

Set  Iaas Config

bosh@bosh:~$ bosh update cloud-config azure.ymlActing as user 'admin' on 'bosh'Successfully updated cloud config

Upload  Stemcell

$ bosh upload stemcellhttps://bosh.io/d/stemcells/bosh-azure-hyperv-ubuntu-trusty-go_agent?v=3232.4

Upload  Releases  (garden,  concourse)

$ bosh upload release https://bosh.io/d/github.com/cloudfoundry-incubator/garden-linux-release?v=0.337.0$ bosh upload release https://bosh.io/d/github.com/concourse/concourse?v=1.2.0

Deploy  Concourse  CI

$ bosh deployment concourse.yml$ bosh -n deploy

Manifest  file  is  as  same  as  AWS's  

AWS AzureIaaS Setup Create

VPC, NAT,  Keypair,  SG,  EIPUse  "BOSH  Setup"  Template(other  than  SG)Director • Create  Manifestfile (bosh.yml)

• bosh-‐‑‒initCloud Config • Create  IaaS config (aws.yml)

• bosh  update  cloud-‐‑‒cofig• Create  IaaS config(azure.yml)

• bosh  update  cloud-‐‑‒cofigManifest  for  Concourse

Create manifest  file  (IaaS Independent)

Stemcell Upload  for  AWS  Xen Upload for  Azure  Hyper-‐‑‒VRelease Upload

• garden-‐‑‒linux• concourse

Deploy bosh  deploy

Installation  Article  (Japanese)

• AWS https://blog.ik.am/entries/383• Azure  https://blog.ik.am/entries/384

Cloud  Foundry  Tokyo  Meetup #2http://www.meetup.com/ja-‐‑‒JP/Cloud-‐‑‒Foundry-‐‑‒Tokyo-‐‑‒Meetup/events/230285366/

Gwenn(@The_̲Shinji62)will  talk  about  BOSH  2.0