varnish in action confoo11
DESCRIPTION
The slides for my Varnish talk at Confoo 2011 in Montreal CanadaTRANSCRIPT
![Page 1: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/1.jpg)
Varnish in ac*on
Thijs FerynEvangelist+32 (0)9 218 79 [email protected]
ConfooFriday March 11th 2011Montrea, Canada
![Page 2: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/2.jpg)
About me
I’m an evangelist at Combell
![Page 3: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/3.jpg)
About me
I’m a board member at PHPBenelux
![Page 4: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/4.jpg)
![Page 5: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/5.jpg)
<?php$country = new Country();$country->setName('Belgium');$country->setHasGovernment(false);
![Page 6: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/6.jpg)
I live in the wonderful city of Bruges
MPBecker -‐ Bruges by Night hVp://www.flickr.com/photos/galverson2/3715965933
![Page 7: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/7.jpg)
Give me feedback: hVp://joind.in/2882
Read my blog: hVp://blog.feryn.eu
Follow me on TwiVer: @ThijsFeryn
![Page 8: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/8.jpg)
![Page 9: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/9.jpg)
Varnish
![Page 10: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/10.jpg)
Varnish
Reverse proxy?
Caching proxy?
Loadbalancer?
![Page 11: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/11.jpg)
Varnish
HTTP accelerator !
![Page 12: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/12.jpg)
Caching
![Page 13: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/13.jpg)
Caching
Storing computed data for faster serving of future requests
![Page 14: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/14.jpg)
Basically
Pucng stuff in boxes for later use
![Page 15: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/15.jpg)
Why?
![Page 16: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/16.jpg)
Protect your server
![Page 17: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/17.jpg)
In the beginning there was ...
![Page 18: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/18.jpg)
Browser cache
![Page 19: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/19.jpg)
Browser cache: expira*on
Expires: Sun, 25 Jun 2006 14:57:12
Cache-‐Control: max-‐age=86400, must-‐revalidate
![Page 20: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/20.jpg)
Browser cache: valida*on
Last-‐Modified: Sun, 25 Jun 2006 14:57:12 GMT
ETag: “686897696a7c876b7e”
![Page 21: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/21.jpg)
Browser cache: valida*on
Last-‐Modified: Sun, 25 Jun 2006 14:57:12 GMT
ETag: “686897696a7c876b7e”
Compared with the “If-‐None-‐Match” header. Throws “HTTP/1.0 304 Not Modified” accordingly
![Page 22: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/22.jpg)
Browser cache: in PHP
![Page 23: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/23.jpg)
Browser cache: in PHP
<?php$seconds = 10;$expires = gmdate('D, j M Y H:i:s T', time() + $seconds);header("Expires: $expires");header("Cache-Control: max-age=$seconds, must-revalidate");?><ul> <li>Current date/time: <?= gmdate('D, j M Y H:i:s T')?></li> <li>Expires: <?= $expires ?></li> <li>Cache-control: <?= $seconds?> seconds</li></ul>
![Page 24: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/24.jpg)
Browser cache: in Apache
![Page 25: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/25.jpg)
Browser cache: in Apache
ExpiresAc*ve onExpiresDefault "access plus 1 month"ExpiresDefault "now plus 4 weeks"ExpiresDefault "modifica*on plus 30 days"ExpiresByType text/html "access plus 1 month 15 days 2 hours"
![Page 26: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/26.jpg)
Browser cache is your friend !
![Page 27: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/27.jpg)
Problems with browser cache
![Page 28: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/28.jpg)
Problems with browser cache
Mul*ple standards
Can be ignored by browser
Force refresh
Cache per client
![Page 29: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/29.jpg)
Solu*ons: use a caching/reverse proxy!
Use a caching/reverse proxy
Varnish does that
It’ll make your site fly!
![Page 30: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/30.jpg)
Info
•Website: h$p://www.varnish-‐cache.org•Wiki: h$p://www.varnish-‐cache.org/trac/wiki•Examples: h$p://www.varnish-‐cache.org/trac/wiki/VCLExamples•DocumentaHon site: h$p://www.varnish-‐cache.org/docs/2.1
![Page 31: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/31.jpg)
Setup Varnish
![Page 32: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/32.jpg)
Setup Varnish
curl http://repo.varnish-cache.org/debian/GPG-key.txt | apt-key add -
apt-get update
echo "deb http://repo.varnish-cache.org/debian/ $(lsb_release -s -c) varnish-2.1" >> /etc/apt/sources.list
apt-get install varnish
![Page 33: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/33.jpg)
Setup Varnish
DAEMON_OPTS="-a :80 \-T localhost:6082 \-f /etc/varnish/default.vcl \-s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"
In “/etc/default/varnish”
START=yes
![Page 34: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/34.jpg)
Setup the backend
![Page 35: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/35.jpg)
Setup the backend
Listen 8080
In “/etc/apache2/ports.conf”
![Page 36: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/36.jpg)
Setup the backend
backend default { .host = "127.0.0.1"; .port = "8080";}
In “/etc/varnish/default.vcl”
![Page 37: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/37.jpg)
Out of the box Varnish will only ...
![Page 38: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/38.jpg)
... obey your caching headers
![Page 39: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/39.jpg)
... and cache sta*c files
![Page 40: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/40.jpg)
Time for ac*on !
![Page 41: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/41.jpg)
Request vcl_recv
In cache?
vcl_hash Cacheable?
vcl_hit() vcl_miss()
vcl_deliver()
vcl_fetch()
No
Yes
NoYes
Response
![Page 42: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/42.jpg)
Hits & misses
![Page 43: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/43.jpg)
Logging & sta*s*cs
![Page 44: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/44.jpg)
Varnishstat
Real *me stats of running varnishd instance
hVp://www.varnish-‐cache.org/docs/2.1/reference/varnishstat.html
![Page 45: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/45.jpg)
0+00:40:27 Hitrate ratio: 10 100 121Hitrate avg: 0.5365 0.5287 0.5277
9957 0.00 4.10 Client connections accepted 58932 67.92 24.28 Client requests received 30483 1.00 12.56 Cache hits 8592 0.00 3.54 Cache hits for pass 18991 66.92 7.82 Cache misses 28449 66.92 11.72 Backend connections success 22562 66.92 9.30 Backend connections reuses 28357 66.92 11.68 Backend connections recycles 488 . . N struct srcaddr 7 . . N active struct srcaddr 81 . . N struct sess_mem 21 . . N struct sess 4398 . . N struct object 4401 . . N struct objecthead 9220 . . N struct smf 355 . . N small free smf 6 . . N large free smf 2 . . N struct vbe_conn 10 . . N struct bereq 17 . . N worker threads 28 0.00 0.01 N worker threads created 164 0.00 0.07 N overflowed work requests 1 . . N backends 14608 . . N expired objects 24113 . . N LRU moved objects 53778 66.92 22.16 Objects sent with write 9957 0.00 4.10 Total Sessions 58935 67.92 24.28 Total Requests 9458 0.00 3.90 Total pass 28445 66.92 11.72 Total fetch 22496844 23506.62 9269.40 Total header bytes 606897837 263427.01 250060.91 Total body bytes 1384 0.00 0.57 Session Closed 82 0.00 0.03 Session Pipeline 58 0.00 0.02 Session Read Ahead 57535 67.92 23.71 Session herd 3539767 5090.85 1458.49 SHM records 181567 221.73 74.81 SHM writes 1399 0.00 0.58 SHM flushes due to overflow 213 1.00 0.09 SHM MTX contention 1 0.00 0.00 SHM cycles through buffer 55123 133.84 22.71 allocator requests 8859 . . outstanding allocations 103968768 . . bytes allocated 969773056 . . bytes free 2 0.00 0.00 SMS allocator requests 930 . . SMS bytes allocated 930 . . SMS bytes freed 28448 66.92 11.72 Backend requests made 1 0.00 0.00 N vcl total 1 0.00 0.00 N vcl available
![Page 46: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/46.jpg)
Varnishlog
Display requests of running varnishd instance
hVp://www.varnish-‐cache.org/docs/2.1/reference/varnishlog.html
![Page 47: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/47.jpg)
Notable tags
Varnishlog
• RxHeader• RxLostHeader• RxProtocol• RxRequest• RxResponse• RxStatus• RxURL
• TxHeader• TxLostHeader• TxProtocol• TxRequest• TxResponse• TxStatus• TxURL
• ReqEnd• ReqStart• Hit• Error• VCL_call• VCL_return• VCL_acl
![Page 48: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/48.jpg)
Varnishlog
varnishlog -‐c -‐o RxRequest POST
All POST requests
![Page 49: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/49.jpg)
Varnishlog
varnishlog -‐c -‐i RxHeader -‐I \^User-‐Agent
All user agents
![Page 50: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/50.jpg)
Varnishlog
varnishlog -‐i TxURL
All URL’s that “missed”
![Page 51: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/51.jpg)
Varnishlog
varnishlog -‐o VCL_call hit | grep RxURL
All URL’s that “hit”
![Page 52: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/52.jpg)
Varnishlog
varnishlog -‐w /tmp/yourfile.log
Write log to file
varnishlog -‐r /tmp/yourfile.log
Read log from file
![Page 53: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/53.jpg)
12 SessionOpen c 172.16.26.1 50396 :8080 12 ReqStart c 172.16.26.1 50396 668213522 12 RxRequest c GET 12 RxURL c /test.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: varnish.dev:8080 12 RxHeader c User-‐Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; nl; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 12 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 RxHeader c Accept-‐Language: nl,en-‐us;q=0.7,en;q=0.3 12 RxHeader c Accept-‐Encoding: gzip,deflate 12 RxHeader c Accept-‐Charset: ISO-‐8859-‐1,utf-‐8;q=0.7,*;q=0.7 12 RxHeader c Keep-‐Alive: 115 12 RxHeader c Connection: keep-‐alive 12 RxHeader c Cookie: PHPSESSID=2n2pkit81qdgk6k4trf1crft16 12 VCL_call c recv 12 VCL_return c pass 12 VCL_call c hash 12 VCL_return c hash 12 VCL_call c pass 12 VCL_return c pass 14 BackendClose -‐ default 14 BackendOpen b default 127.0.0.1 34267 127.0.0.1 80 12 Backend c 14 default default
![Page 54: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/54.jpg)
14 TxRequest b GET 14 TxURL b /test.php 14 TxProtocol b HTTP/1.1 14 TxHeader b Host: varnish.dev:8080 14 TxHeader b User-‐Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; nl; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 14 TxHeader b Accept-‐Language: nl,en-‐us;q=0.7,en;q=0.3 14 TxHeader b Accept-‐Encoding: gzip,deflate 14 TxHeader b Accept-‐Charset: ISO-‐8859-‐1,utf-‐8;q=0.7,*;q=0.7 14 TxHeader b Cookie: PHPSESSID=2n2pkit81qdgk6k4trf1crft16 14 TxHeader b X-‐Forwarded-‐For: 172.16.26.1 14 TxHeader b X-‐Varnish: 668213522 14 RxProtocol b HTTP/1.1 14 RxStatus b 200 14 RxResponse b OK 14 RxHeader b Date: Tue, 19 Oct 2010 12:54:49 GMT 14 RxHeader b Server: Apache/2.2.16 (Ubuntu) 14 RxHeader b X-‐Powered-‐By: PHP/5.3.3-‐1ubuntu9 14 RxHeader b Vary: Accept-‐Encoding 14 RxHeader b Content-‐Encoding: gzip 14 RxHeader b Content-‐Length: 36 14 RxHeader b Content-‐Type: text/html
![Page 55: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/55.jpg)
12 TTL c 668213522 RFC 120 1287492889 0 0 0 0 12 VCL_call c fetch 12 TTL c 668213522 VCL 10 1287492889 12 VCL_return c pass 12 ObjProtocol c HTTP/1.1 12 ObjStatus c 200 12 ObjResponse c OK 12 ObjHeader c Date: Tue, 19 Oct 2010 12:54:49 GMT 12 ObjHeader c Server: Apache/2.2.16 (Ubuntu) 12 ObjHeader c X-‐Powered-‐By: PHP/5.3.3-‐1ubuntu9 12 ObjHeader c Vary: Accept-‐Encoding 12 ObjHeader c Content-‐Encoding: gzip 12 ObjHeader c Content-‐Type: text/html 14 Length b 36 14 BackendReuse b default 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 200 12 TxResponse c OK 12 TxHeader c Server: Apache/2.2.16 (Ubuntu) 12 TxHeader c X-‐Powered-‐By: PHP/5.3.3-‐1ubuntu9 12 TxHeader c Vary: Accept-‐Encoding 12 TxHeader c Content-‐Encoding: gzip 12 TxHeader c Content-‐Type: text/html 12 TxHeader c Content-‐Length: 36 12 TxHeader c Date: Tue, 19 Oct 2010 12:54:49 GMT 12 TxHeader c X-‐Varnish: 668213522 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: keep-‐alive 12 TxHeader c X-‐Cache: MISS 12 Length c 36 12 ReqEnd c 668213522 1287492889.383963823 1287492889.389129400 0.000204563 0.005021572 0.000144005
![Page 56: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/56.jpg)
Varnishtop
Read memory logs and presents them as a con*nuously updated list of commonly occuring log entries
hVp://www.varnish-‐cache.org/docs/2.1/reference/varnishtop.html
![Page 57: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/57.jpg)
Varnishtop
varnishtop -‐i RxHeader -‐I \^User-‐Agent
Get the top user agents
![Page 58: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/58.jpg)
Varnishtop
varnishtop -‐i RxRequest
Get the top request methods
![Page 59: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/59.jpg)
Varnishtop
varnishtop -‐i TxURL
Get the top misses
![Page 60: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/60.jpg)
list length 90 dev
76.42 VCL_return pass 50.95 RxProtocol HTTP/1.1 50.95 TxProtocol HTTP/1.1 27.54 Length 110 25.48 Debug herding 25.47 RxRequest GET 25.47 RxHeader Host: varnish.dev:8080 25.47 RxHeader User-‐Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; nl; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 25.47 RxHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 25.47 RxHeader Accept-‐Language: nl,en-‐us;q=0.7,en;q=0.3 25.47 RxHeader Accept-‐Encoding: gzip,deflate 25.47 RxHeader Accept-‐Charset: ISO-‐8859-‐1,utf-‐8;q=0.7,*;q=0.7 25.47 RxHeader Keep-‐Alive: 115 25.47 RxHeader Connection: keep-‐alive 25.47 RxHeader Cookie: PHPSESSID=2n2pkit81qdgk6k4trf1crft16 25.47 VCL_call recv 25.47 VCL_call hash 25.47 VCL_return hash 25.47 VCL_call pass 25.47 Backend 14 default default 25.47 TxRequest GET 25.47 TxHeader Host: varnish.dev:8080 25.47 TxHeader User-‐Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; nl; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 25.47 TxHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 25.47 TxHeader Accept-‐Language: nl,en-‐us;q=0.7,en;q=0.3 25.47 TxHeader Accept-‐Encoding: gzip,deflate 25.47 TxHeader Accept-‐Charset: ISO-‐8859-‐1,utf-‐8;q=0.7,*;q=0.7 25.47 TxHeader Cookie: PHPSESSID=2n2pkit81qdgk6k4trf1crft16 25.47 TxHeader X-‐Forwarded-‐For: 172.16.26.1 25.47 RxStatus 200 25.47 RxResponse OK 25.47 RxHeader Server: Apache/2.2.16 (Ubuntu) 25.47 RxHeader X-‐Powered-‐By: PHP/5.3.3-‐1ubuntu9 25.47 RxHeader Vary: Accept-‐Encoding 25.47 RxHeader Content-‐Encoding: gzip 25.47 RxHeader Content-‐Type: text/html 25.47 VCL_call fetch 25.47 ObjProtocol HTTP/1.1 25.47 ObjStatus 200 25.47 ObjResponse OK 25.47 ObjHeader Server: Apache/2.2.16 (Ubuntu) 25.47 ObjHeader X-‐Powered-‐By: PHP/5.3.3-‐1ubuntu9
![Page 61: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/61.jpg)
Varnishncsa
Read memory logs and presents them in the Apache / NCSA “combined” log format
hVp://www.varnish-‐cache.org/docs/2.1/reference/varnishncsa.html
![Page 62: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/62.jpg)
Varnishncsa
varnishncsa -‐D -‐a -‐w /var/log/some.log
172.16.26.1 -‐ -‐ [19/Oct/2010:16:08:02 +0200] "GET hVp://varnish.dev:8080/ HTTP/1.1" 200 97 "-‐" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; nl; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10"172.16.26.1 -‐ -‐ [19/Oct/2010:16:08:10 +0200] "GET hVp://varnish.dev:8080/test.php HTTP/1.1" 200 36 "-‐" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; nl; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10"
![Page 63: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/63.jpg)
Varnishreplay
The varnishreplay u*lity parses varnish logs and aVempts to reproduce the traffic
hVp://www.varnish-‐cache.org/docs/2.1/reference/varnishreplay.html
![Page 64: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/64.jpg)
Varnishreplay
varnishlog -‐w /tmp/yourfile.log
varnishreplay -‐r /tmp/yourfile.log
![Page 65: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/65.jpg)
Management via telnet
![Page 66: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/66.jpg)
Management via telnet
root@dev:/# telnet localhost 6082Trying ::1...Connected to localhost.Escape character is '^]'.200 154 -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐Varnish HTTP accelerator CLI.-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐Type 'help' for command list.Type 'quit' to close CLI session.
![Page 67: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/67.jpg)
Notable ac*ons
Management via telnet
• vcl.load <configname> <filename>• vcl.use <configname>• vcl.discard <configname>• vcl.list• vcl.show <configname>
Avoid interrup*on
![Page 68: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/68.jpg)
More ac*on?
![Page 69: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/69.jpg)
Request vcl_recv
In cache?
vcl_hash Cacheable?
vcl_hit() vcl_miss()
vcl_deliver()
vcl_fetch()
No
Yes
NoYes
Response
![Page 70: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/70.jpg)
sub vcl_recv { if ( (req.request == "GET" || req.request == "HEAD") && req.url ~ "\.(png|gif|jpe?g|swf|css|js|html?|sstm)$" ) { return(lookup); } else { return(pass); }}
Cache sta*c files
![Page 71: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/71.jpg)
sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; }}
Provide HIT/MISS informa*on
![Page 72: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/72.jpg)
sub vcl_fetch { set beresp.ttl = 10s;}
Manipulate TTL
![Page 73: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/73.jpg)
sub vcl_recv { if (req.request == "GET" || req.request == "HEAD") { return (lookup); }}sub vcl_hash { if (req.http.Cookie) { set req.hash += req.http.Cookie; }}
Cache ... even with cookies
![Page 74: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/74.jpg)
sub vcl_recv { unset req.http.cookie;}sub vcl_fetch { unset beresp.http.set-cookie;}
Remove cookies
![Page 75: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/75.jpg)
sub vcl_recv {if (req.http.Cookie) { set req.http.Cookie = regsuball(req.http.Cookie,"(̂ |; ) *__utm.=[̂ ;]+;? *","\1"); if (req.http.Cookie == "") { remove req.http.Cookie; }}
Remove Google Analy*cs cookies
![Page 76: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/76.jpg)
backend default { .host = "default.server.com"; .port = "80";}backend other { .host = "other.server.com"; .port = "80";}sub vcl_recv { if(req.http.host ~ "^www\."){ set req.backend= default; } else { set req.backend = other; }}
Mul*ple backends: hostname filtering
![Page 77: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/77.jpg)
Default backend Other backend
www.example.com example.com
![Page 78: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/78.jpg)
sub vcl_recv { if (req.backend.healthy) { set req.grace = 1s; } else { set req.grace = 10s; }}
Grace mode
Keep reading from cache, even when expired
![Page 79: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/79.jpg)
backend default { .host = "default.server.com"; .port = "80"; .probe = { .url = "/"; .timeout = 50 ms; .interval = 1s; .window = 2; .threshold = 2; }}
Mul*ple backends: health checks
![Page 80: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/80.jpg)
backend other { .host = "other.server.com"; .port = "80"; .probe = { .url = "/"; .timeout = 50 ms; .interval = 1s; .window = 2; .threshold = 2; }}
Mul*ple backends: health checks
![Page 81: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/81.jpg)
sub vcl_recv { if (req.restarts == 0) { set req.backend = default; } else { set req.backend = other; } return(pass);}sub vcl_error { if (obj.status == 503 && req.restarts < 4) { restart; }}
Mul*ple backends: health checks
![Page 82: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/82.jpg)
Default backend Other backend
![Page 83: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/83.jpg)
Default backend Other backend
![Page 84: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/84.jpg)
director back round-robin { {.backend = default;} {.backend = other;}}sub vcl_recv { set req.backend = back; return(pass);}
Loadbalancing via directors
![Page 85: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/85.jpg)
Default backend Other backend
50% 50%
![Page 86: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/86.jpg)
director back random { {.backend = default; .weight=1;} {.backend = other; .weight=2;}}sub vcl_recv { set req.backend = back; return(pass);}
Loadbalancing via directors
![Page 87: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/87.jpg)
Default backend Other backend
33% 66%
![Page 88: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/88.jpg)
acl purge_acl { "localhost"; "1.2.3.4"; "some.host.ext";}sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge_acl) { error 405 "Not allowed."; } purge_url(req.url); error 200 "Purged."; } return(lookup);}
Purging
curl -‐X PURGE hVp://yoursite.com/yourpage
![Page 89: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/89.jpg)
Edge Side Includes: the final piece of the puzzle
![Page 90: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/90.jpg)
header.php
menu.php main.php
footer.php
![Page 91: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/91.jpg)
<html><body> <table> <tr> <td colspan="2" > <?php include('header.php') ?>
</td> </tr> <tr> <td><?php include('menu.php') ?></td> <td><?php include('main.php') ?></td> </tr> <tr> <td colspan="2" > <?php include('footer.php') ?> </td> </tr> </table> </body></html>
![Page 92: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/92.jpg)
header.php
menu.php main.php
footer.php
TTL 5s
No cachingTTL 10s
TTL 2s
![Page 93: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/93.jpg)
<html><body> <table> <tr> <td colspan="2" > <esi:include src="/header.php" />
</td> </tr> <tr> <td><esi:include src="/menu.php" /></td> <td><esi:include src="/main.php" /></td> </tr> <tr> <td colspan="2" > <esi:include src="/footer.php" /> </td> </tr> </table> </body>
![Page 94: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/94.jpg)
<?phpfunction esi($file){ if(array_key_exists('HTTP_X_VARNISH',$_SERVER)){ header("esi-enabled: 1"); $url = 'http://'.$_SERVER['HTTP_HOST'].substr($_SERVER['SCRIPT_NAME'],0,1+strrpos($_SERVER['SCRIPT_NAME'],'/')).$file; echo "<esi:include src=\"$url\" />".PHP_EOL; } else { include($file); }}
![Page 95: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/95.jpg)
<?phpclass My_View_Helper_Esi extends Zend_View_Helper_Abstract{ protected $_requestHeader; protected $_responseHeader; protected $_frontController; public function esi($controller = 'index', $action='index', $requestHeader = 'X_VARNISH', $responseHeader = 'esi-enabled: 1') { $this->_requestHeader = $requestHeader; $this->_responseHeader = $responseHeader; $this->_frontController = Zend_Controller_Front::getInstance(); if($this->_getRequestHeader()){ $this->_getResponseHeader(); return $this->_buildEsi($controller,$action); } else { return $this->_dispatch($controller, $action); } }...
![Page 96: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/96.jpg)
...protected function _getRequestHeader(){ if($this->_requestHeader === null || $this->_requestHeader === false){ return false; } $request = $this->_frontController->getRequest(); if($request->getHeader($this->_requestHeader) === false){ return false; } else { return true; }}protected function _getResponseHeader(){ if($this->_responseHeader !== null && $this->_responseHeader !== false){ header($this->_responseHeader); }}...
![Page 97: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/97.jpg)
... protected function _dispatch($controller,$action) { $request = new Zend_Controller_Request_Simple($action, $controller); $response = new Zend_Controller_Response_Http(); $dispatcher = $this->_frontController->getDispatcher(); $dispatcher->dispatch($request, $response); return $response->getBody(); } protected function _buildEsi($controller,$action) { $router = $this->_frontController->getRouter(); $url = $router->assemble(array('controller'=>$controller,'action'=>$action)); return "<esi:include src=\"{$url}\"/>"; }}
![Page 98: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/98.jpg)
<html><body> <table> <tr> <td colspan="2" > <?php echo $this->esi('index',‘header’) ?> </td> </tr> <tr> <td><?php echo $this->esi('index',‘menu’) ?></td> <td><?php echo $this->esi('index',‘main’) ?></td> </tr> <tr> <td colspan="2" > <?php echo $this->esi('index',‘footer’) ?> </td> </tr> </table>
![Page 99: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/99.jpg)
sub vcl_recv { if(req.url == "/header.php") { return (pass); } if(req.url ~ "^/(footer|main|menu)\.php$") { return (lookup); }}sub vcl_fetch { if (beresp.http.esi-enabled == "1") { esi; unset beresp.http.esi-enabled; } if(req.url == "/footer.php") { set beresp.ttl = 5s; } if(req.url == "/menu.php") { set beresp.ttl = 10s; } if(req.url == "/main.php") { set beresp.ttl = 2s; }}
![Page 100: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/100.jpg)
![Page 101: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/101.jpg)
![Page 102: Varnish in action confoo11](https://reader037.vdocuments.site/reader037/viewer/2022102814/54b7aa844a7959d3328b45e4/html5/thumbnails/102.jpg)
Q&A