nginx internals

Nginx Internals

[email protected]

Agenda

• What

• Why

• Architecture

• Infrastructure

• Http Module

• Conf & Location &Static

• Upstream & fastcgi

• Extension

What’s Nginx

• October 4, 2004

• 6.8% of the top 1 million

• HTTP Server

• Reverse proxy server

• Mail proxy server

• http://nginx.org/en/

Why Nginx

• Features : Nginx = transmit + apache

• Stability : Nginx > Lighttpd

• Performance : Nginx > Lighttpd > apache

• Security : Acess & Limit Zone & Limit Requests

• Community : sina/qq/tudou/taobao

• Third party modules

• more

Code Organization

• 117590

• Coremain/lib/data struct

• Eventepool/time/mutex

• Http http server/upstream

• Os & mail

Architecture

• Multi-process

• Master+worker

• Event driven

• Non-blocking

• Single-threaded

• Highly modular

• Cache management process

Infrastructure

• String

• Array/List/Queue

• Hash Table

• Red black tree

• Memory pool

• Buff

• Radix Tree / Slab / Spinlock / shmtx

• http://code.google.com/p/nginxsrp/wiki/NginxCodeReview

Memory Management

• ngx_alloc ， ngx_calloc ， ngx_memalign

• Memory pool

–Avoid memory fragmentation and memory leaks

–Allocation once, deallocation once

–lifetime ： cycle 、 connection 、 Request

–Optimizing large memory allocation

–ngx_palloc ngx_pnalloc ngx_pcalloc ngx_pmemalign

Memory pool

Master process

Worker process

Master & woker

• Socketpair

• Accept Lock, avoid thundering herd

• Multi Accept

• Worker Processes Load-balance (7/8)

• Connection List per process

• Timeout handling. Red black tree

Config & command

• Command-style configuration files, if/echo

• Configuration block, ‘{}’

• Support Include command

• Support inheritance(Http main/Server/Location)

• Multi-parameter

• Variables

• Conf demo

Config parse

• Typeparse_fileparse_blockparse_param

• Command handler

Modular

Http Module

• http command, http {}

• ngx_http_block

• ngx_http_module_t

Http request processing

ngx_http_process_request_line

ngx_http_init_connection

ngx_http_init_request

ngx_http_process_request_headers

NGX_HTTP_POST_READ_PHASE

NGX_HTTP_SERVER_REWRITE_PHASE

NGX_HTTP_FIND_CONFIG_PHASE

NGX_HTTP_REWRITE_PHASE

NGX_HTTP_POST_REWRITE_PHASE

NGX_HTTP_PREACCESS_PHASE

NGX_HTTP_ACCESS_PHASE

NGX_HTTP_POST_ACCESS_PHASE

NGX_HTTP_TRY_FILES_PHASE

NGX_HTTP_CONTENT_PHASE

NGX_HTTP_LOG_PHASE

ngx_http_log_module

ngx_http_static_module

ngx_http_access_module

ngx_http_index_module

ngx_http_rewrite_module

ngx_http_finalize_request

1 、 phase checker2 、 phase handler, array3 、 sub request

http phase

• NGX_HTTP_POST_READ_PHASE (realip)

• NGX_HTTP_SERVER_REWRITE_PHASE (Rewrite)

• NGX_HTTP_FIND_CONFIG_PHASE Can not be extended

• NGX_HTTP_REWRITE_PHASE (Rewite)

• NGX_HTTP_POST_REWRITE_PHASE Can not be extended

• NGX_HTTP_PREACCESS_PHASE (limit_req 、 limit_zone 、 degradation 、 realip)

• NGX_HTTP_ACCESS_PHASE (ACCESS 、 auth_basic)

• NGX_HTTP_POST_ACCESS_PHASE Can not be extended

• NGX_HTTP_TRY_FILES_PHASE Can not be extended

• NGX_HTTP_CONTENT_PHASE (Autoindex 、 static 、 random_index 、 gzip_static)

• NGX_HTTP_LOG_PHASE

content phase handler vs content_handler

• checker: ngx_http_core_content_phase

• Fastcgi & autoindex

Nginx Filter

• Header Filter

• Content Filter

• ngx_http_send_header

• ngx_http_output_filter

• Example : gzip 、 more_set_header

Location features

• Exactly match

• Regular(sensitive/ insensitive)

• Location inheritance

• How to find the right location?

Location internals

• Ngx_http_block loc_conf

• ngx_http_init_locationsLocation inheritance check

• ngx_http_init_static_location_trees Ternary Search Tree(better than trie)

• Find location(ngx_http_core_find_location) first static location tree then regex locations

Static module

• Sendfile http://linux.die.net/man/2/sendfile

• NGX_HTTP_CONTENT_PHASEngx_http_static_handler

• ngx_http_static_handler1 、 method check. GET/HEAD/POST2 、 Cache processing3 、 ngx_http_discard_request_body4 、 output

• Only Cache fd

• Red black tree

Nginx Upstream

• Protocol : Http/Fastcgi/Memcache/SCGI

• Load-balance: hash/random/round_robin

• Short connection

• Long connection only for memcached

Nginx Upstream ： fastcgi

Extension Develop

• Core Module Extension, such as nginx-php

• Http Module ExtensionHandler ： echoFilter ： gzip, usertrackupstream : fastcgiLoad-balancers ： rand

• Emiller's Guide To Nginx Module Development

Extension Develop—where?

• Just before the server reads the config file

• For every configuration directive for the location and server for which it appears;

• When Nginx initializes the main configuration

• When Nginx initializes the server (i.e., host/port) configuration

• When Nginx merges the server configuration with the main configuration

• When Nginx initializes the location configuration

• When Nginx merges the location configuration with its parent server configuration

• When Nginx's master process starts

• When a new worker process starts

• When a worker process exits

• When the master exits

• Handling a request

• Filtering response headers

• Filtering the response body

• Picking a backend server

• Initiating a request to a backend server

• Re-initiating a request to a backend server

• Processing the response from a backend server

• Finishing an interaction with a backend server


• Http Phase handler

• Content_handler

• Header Filter

• Content Filter

Extension Develop—step

• “Config” filengx_addon_name=ngx_http_baidu_usertrack_module

HTTP_AUX_FILTER_MODULES="$HTTP_AUX_FILTER_MODULES ngx_http_baidu_usertrack_module"

NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/src/ngx_http_baidu_usertrack_module.c $ngx_addon_dir/src/baidu_des.c"

NGX_ADDON_DEPS="$NGX_ADDON_DEPS $ngx_addon_dir/src/baidu_des.h"

ORE_LIBS="$CORE_LIBS -lcrypto"

• Develop and build ./configure -add-module=path/to/your/new/module/directoryMakeMake install

Extension Develop—demo

• Demo

Q&A

Q&AThanks

nginx internals

Technology