„enterprise class load balancer“ - sebastian feldmann
TRANSCRIPT
Set up an
„Enterprise Class Load Balancer“with free software
About me
Sebastian Feldmann
Teamlead CHECK24.de
PHP since < 2000
@movetodevnull
https://phpbu.de sebastianfeldmann
Why?
?
?
?
??
?
?
??
?
??
High Availability&
Performance
High Availability
Redundancy, redundancy, redundancy…
Performance
Scaling your application
Load BalancerInternet Web Servers
Setup
Load BalancerInternet
Software
EncryptionCaching
Load Balancing
NGINXserver { listen 192.0.2.70:80; listen 192.0.2.70:443 ssl; server_name www.example.com;
ssl_certificate ssl/com.example.www.crt; ssl_certificate_key ssl/com.example.www.key;
location / { proxy_pass http://localhost:90; proxy_set_header Host $host; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Protocol $scheme; proxy_pass_header Set-Cookie; }}
NGINX
location / { proxy_pass http://localhost:90; proxy_set_header Host $host; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Protocol $scheme; proxy_pass_header Set-Cookie;}
HAProxy
frontend fe.com.domain.www bind localhost:90 option httplog option http-server-close maxconn 20000 capture request header Host len 20 capture request header X-Forwarded-For len 15
default_backend be.com.domain.default
backend be.com.domain.default balance roundrobin server server1 192.168.0.11 server server2 192.168.0.12
Congratulations
Tuning
• Improve the setup
• Tuning the Operating System
• Tuning the Software
• Monitoring
Load BalancerInternet Web Servers
Setup
DNS
www.domain.com. 600 IN A 192.0.2.70 www.domain.com. 600 IN A 192.0.2.71
Operating System
Network Redundancy
• SixPack configuration
• NIC bonding
Network Redundancy auto bond0 iface bond0 inet static address 192.0.2.70 netmask 255.255.255.0 network 192.0.2.0 broadcast 192.0.2.255 gateway 192.0.2.1 dns-nameservers 192.0.2.1 bond-slaves eth0 eth1 bond-mode 1 bond-miimon 100 bond-updelay 200 bond-downdelay 200
Network Tuning
time
band
wid
th
Network Tuning # recommended default congestion control is not reno net.ipv4.tcp_congestion_control=htcp # allow testing with buffers up to 64MB net.core.rmem_max = 67108864 net.core.wmem_max = 67108864
# increase Linux autotuning TCP buffer limit to 32MB net.ipv4.tcp_rmem = 4096 87380 33554432 net.ipv4.tcp_wmem = 4096 65536 33554432
# increase the length of the processor input queue # set at least as high as haproxy maxconn net.core.netdev_max_backlog = 30000
/etc/sysctl.conf
NGINX
Certificate
• Use SHA256 for the signature algorithm (not SHA1)
• Use a well known/trusted CA
• Make sure your cert and chain are in the correct order
intermediate
Chain of trust
example.com
intermediate
root
Protocol Support
• Don’t use SSLv2 or SSLv3
• Use TLS
• For a perfect score use TLS 1.2 only
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
Key Exchange
• Default Diffie-Hellman Key (weak)
• Create new key
• Use the generated key
ssl_dhparam ssl/dhparam.pem;
openssl dhparam -out /etc/nginx/ssl/dhparam.pem 2048
Cipher Strength
• Forward Secrecy
• Set preferred ciphers
ssl_prefer_server_ciphers on;ssl_ciphers "EECDH+ECDSA+AESGCM:EECDH+...."
SSL Stapling
• OCSP modification
• Part of the TLS handshake
• Revocation information
• Chain of trust including root certificate
ssl_stapling on;ssl_stapling_verify on;ssl_trusted_certificate ssl/com.example.www.trusted.crt;
Protocol Tuning
time
band
with
SPDY & HTTP/2
time
band
with
Protocol Tuning
• Use HTTP/2 or SPDY
listen 192.0.2.70:443 ssl spdy;
listen 192.0.2.70:443 ssl http2;
NGINX 1.5.10 - 1.9.4
NGINX 1.9.5 - 1.10.*
HTTP/2 Support
caniuse.com
1 - Partial support in IE11 refers to being limited to Windows 10.2 - Only supports HTTP/2 over TLS (https)3 - Partial support in Safari refers to being limited to OSX 10.11+
SPDY or HTTP/2
Caching
proxy_cache_path /var/lib/nginx/cache levels=1:2 keys_zone=MYCACHE:10m inactive=10m max_size=1g;
server { location ~ ^/assets/ { proxy_cache MYCACHE; proxy_cache_key "$scheme$host$request_uri"; proxy_cache_valid 200 10m; proxy_no_cache $http_pragma $http_cache_control; expires 30d; ...
Stale is better than nothing
server { location ~ ^/html/ { ...
proxy_cache_use_stale error timeout http_500 http_502 http_503; ...
HAProxy
Checking Backends
backend be.com.domain.default ... option httpchk GET /health/check.html http-check disable-on-404
server server1 192.168.0.11 check server server2 192.168.0.12 check
HAProxy Monitoring
• Connect via socket
• Display html page
• Get data CSV
HAProxy: Socket
global stats socket /var/run/haproxy.sock mode 600 level admin stats timeout 2m
HATop
HAProxy: Stats Page
listen admin_stats 192.0.2.70:9090 stats uri /stats stats realm HAProxy\ Stats stats auth admin:superSecretPassword stats show-desc stats show-legends stats show-node stats refresh 15s
acl authorized_hosts src 192.168.0.33
tcp-request content accept if authorized_hosts tcp-request content reject
HAProxy: Stats Page
HAProxy: Stats CSV
stats uri /stats/csv;csv
• Machine readable version
• Add ;csv
Recap
• Redundant Load Balancer
• Improved Network speed & redundancy
• Improved HTTPS performance & Caching
• HAProxy backend checking & monitoring
Thank you
Sebastian Feldmann - sebastian-feldmann.info - @movetodevnull
https://joind.in/talk/a6aca