Varnish - The 9 circles of hell

04 Dec 2013 The 9 circles of hell

Varnish

By Luis Ferro

Who am i

Luis Ferro

Lead Developer in Rewards and Loyalty Programs (Metropolis International)

When not coding, you can find me shooting arrows or cooking, mostly...

http://www.linkedin.com/profile/view?id=73429631

Varnish before Hell Entrance

Varnish is an HTTP accelerator designed for content-heavy dynamic web sites. Varnish is focused exclusively on HTTP. - wikipedia

https://www.varnish-cache.org

Installing (debian/ubuntu)sudo apt-get updatesudo apt-get install varnish

1st Circle - basics

Simple caching

Varnish, NGinX and PHP should be working

Default configurations apart from the ports which should be:Port 80 Varnish, with backend to 8080Port 8080 NGinX, with backend to 9000Port 9000 PHP

1st Backend in default.vcl

backend default { .host = "127.0.0.1"; .port = "8080"; .connect_timeout = 30s; .first_byte_timeout = 30s; .between_bytes_timeout = 30s; }Check configuration with:varnishd -C -f default.vcl

1st - Checking it in - NGinX

NGinX Testab -n 10000 -c 10 http://127.0.0.1:8080/index.php

Requests per second: 253.01 [#/sec] (mean)Time per request: 39.524 [ms] (mean)Time per request: 3.952 [ms] (mean, across all concurrent requests)Transfer rate: 29931.75 [Kbytes/sec] received

1st - Checking it in - Varnish

Varnish Testab -n 10000 -c 10 http://127.0.0.1:80/index.php

Requests per second: 2624.59 [#/sec] (mean)Time per request: 3.810 [ms] (mean)Time per request: 0.381 [ms] (mean, across all concurrent requests)Transfer rate: 311418.96 [Kbytes/sec] received

10x FASTERWith almost default config

2nd Circle The path to cache

Add hit / miss information to all requests sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; set resp.http.X-Cache-Hits = obj.hits; } else { set resp.http.X-Cache = "MISS"; } return (deliver); }

2nd - Check

3rd Circle Force march

Force Refreshsub vcl_recv { [...] if (req.http.Cache-Control ~ "no-cache") { ban_url(req.url); }[] }

You may want to protect / restrict who can call this ban

3rd - Check

4rd Circle Expired content

Add php headers to force content to expire after a certain quantity of secondsfunction headers_for_page_cache($cache_length=600){ $cache_expire_date = gmdate("D, d M Y H:i:s", time() + $cache_length); header("Expires: $cache_expire_date"); header("Pragma: cache"); header("Cache-Control: max-age=$cache_length, s-maxage=$cache_length"); header("User-Cache-Control: max-age=$cache_length");}

4th Circle - Check

5th Circle All in the bag

Ensure cache by delete cookies on unwanted items (images, css, js)sub vcl_recv { [] if (req.url ~ "\.(png|gif|jpg)$") { remove req.http.Cookie; }[...]}

Varnish doesn't cache anything with a cookie!

5th - Check

6th Circle Watch your steps

Install GeoIp

Get GeoipUsingInlineC from:https://www.varnish-cache.org/trac/wiki/GeoipUsingInlineC

Compile and install it (findable by your OS)

6th - Geoip.vcl

C{ #include #include #include

static const char* (*get_country_code)(char* ip) = NULL;

__attribute__((constructor)) void load_module() { const char* symbol_name = "get_country_code"; const char* plugin_name = "/usr/lib/geoip_plugin.so"; void* handle = NULL;

handle = dlopen( plugin_name, RTLD_NOW ); if (handle != NULL) { get_country_code = dlsym( handle, symbol_name ); if (get_country_code == NULL) fprintf( stderr, "\nError: Could not load GeoIP plugin:\n%s\n\n", dlerror() ); else printf( "GeoIP plugin loaded successfully.\n"); } else fprintf( stderr, "\nError: Could not load GeoIP plugin:\n%s\n\n", dlerror() ); } }C

6th - Add it to default.vcl

sub vcl_recv { [] C{ VRT_SetHdr(sp, HDR_REQ, "\017X-Country-Code:", (*get_country_code)( VRT_IP_string(sp, VRT_r_client_ip(sp)) ), vrt_magic_string_end); }C[] }

6th - Check

7th Circle Know your enemies

Device Detection

Download the devicedetection.vcl from: http://github.com/varnish/varnish-devicedetect/

Add it to default.vclsub vcl_recv { [] call detectdevice;[] }

7th - Check

8th Circle Tongue in cheek

Accepted language

Download and create the accept-language.vcl from: http://github.com/cosimo/varnish-accept-language

Include the generated file and call it in default.vclsub vcl_recv { [] C{ vcl_rewrite_accept_language(sp); }C[] }

8th - Check

9th Circle Divide and conquer

To activate add in your default.vcl sub vcl_fetch { [...] if( beresp.http.esi-enabled == "1") { set beresp.do_esi = true; /* Do ESI processing */ set beresp.ttl = 120s; /* Sets the TTL on the HTML above */ unset beresp.http.esi-enabled; } [...]

9th - Check

Remember, ESI has issues with non-xml like files. So, the file where you place the