Advanced load balancing with Varnish Plus

Dag Haavi FinstadVarnish Software

daghf@varnish-software.com

Agenda● Load balancing (LB) with Varnish● (to the backend)● Pinterest case: seamless gradual code deploy.● Format: part presentation, part VCL

examples.

Jargon● “Front”: client facing part of Varnish● Backends: origin servers.● VCL: Varnish Configuration Language.● a director: a Varnish load balancer instance.

Not so known trick: admin health

● Since 3.0 Varnish you can disable backends administratively.

● varnishadm backend.set_health backend1 (healthy|sick|probe)

● Overrides probes.● Glob-ing match, save you some typing. (“b*”,

“backend?”, “foo*_app”)

Entry level: Round-robin● Most common approach: Two backend

servers.● Varnish will literally walk a circular list for

each backend request.● You can take one down for maintenance

without anyone noticing.

Stickiness● If your application needs statefulness, Varnish

can send the client to the same backend every time.

● By client identifier, or● by any other means readable in VCL. (cookie,

GET argument, something..)

Other not very known additions● Fallback director. Pick the first healthy

backend.● Directors are stackable from 4.0. Round-

robin within fallback? We can do that.● New in 4.1: idle backend connections time

out and will be closed (backend_idle_timeout)

Reintroducing: Saint mode● Mark backends as sick for specific objects

● Went away for 4.0○ now back as a VMOD in 4.1

https://github.com/varnish/libvmod-saintmode

Pinterest case● Core idea: let existing user sessions continue

on the backend app-version they loaded, give n% of new users the latest deploy.

● Main goal: avoid full page loads/refreshes.● Varnish VCL can do this for you.

Pinterest case (II)

● “current” and “stable” appversions running.● client signals its JS bundle version.● If stable or current, set that director and go.● If older (or none), draw a random sample and

assign the client accordingly.

Pinterest case (III)● “Somewhat” Pinterest-specific.

● Main point: VCL configurability allows for a whole lot of magic...

Questions?

Linksrr.vcl:

https://gist.github.com/daghf/bf35bf15584f560220cc

sticky-ip.vcl:https://gist.github.com/daghf/ba4d96641755fcc8e3bb

sticky-cookie.vcl:https://gist.github.com/daghf/4f0df791486c764ffffa

pinterest-deploy.vcl:https://gist.github.com/daghf/c72ce762f4faf70dd9e1

pinterest engineering blog: https://engineering.pinterest.com/blog/stealthy-shipping-atomic-deploys )