web应用中listen函数的backlog默认会给我们内核参数的net.core.somaxconn限制到128,而nginx定义的NGX_LISTEN_BACKLOG默认为511,所以有必要调整这个值。如: net.core.netdev_max_backlog = 262144 我们线上服务器net.core.somaxconn都是默认的128,这个参数会影响到所有AF_INET类型socket的listen队列 Man 2 listen可以知道: int listen(int s, int backlog); The backlog parameter defines the maximum length the queue of pending connections may grow to. If a connection request arrives with the queue full the client may receive an error with an indication of ECONNREFUSED or, if the underlying pro- tocol supports retransmission, the request may be ignored so that retries succeed. BUGS If the socket is of type AF_INET, and the backlog argument is greater than the constant SOMAXCONN (128 in Linux 2.0 & 2.2), it is silently truncated to SOMAXCONN. 也就是说,web应用中listen函数的backlog会给我们内核参数的net.core.somaxconn 限制到128,在高突发的请求中可能会导致链接超时或者触发重传
相信其他应用比如squid也会有类似问题,突发的大并发connect请求会由于内核listen队列的限制导致链接超时或者重传,从而影响用户体验 以下是实验测试情况,使用2台机器分别以1000个并发,benchmark方式,请求服务器 ( 相当于2000个并发请求同时请求服务器 ) 情景1,默认配置, net.core.somaxconn=128,服务器为nginx 测试客户端A: Transactions: 2072870 hits Availability: 99.99 % Elapsed time: 179.59 secs Data transferred: 6096.59 MB Response time: 0.08 secs Transaction rate: 11542.24 trans/sec Throughput: 33.95 MB/sec Concurrency: 927.34 Successful transactions: 2072871 Failed transactions: 300 Longest transaction: 45.30 Shortest transaction: 0.00 错误率大概是1.5%
Transactions: 1859454 hits Availability: 99.99 % Elapsed time: 179.11 secs Data transferred: 5468.90 MB Response time: 0.09 secs Transaction rate: 10381.63 trans/sec Throughput: 30.53 MB/sec Concurrency: 904.45 Successful transactions: 1859454 Failed transactions: 276 Longest transaction: 49.60 Shortest transaction: 0.00 错误率大概也是1.5% 错误提示大都为: socket: connection timed out warning: socket: -1803417280 select timed out: Connection timed out
测试客户端A: ** SIEGE 2.69 ** Preparing 1000 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 1789818 hits Availability: 100.00 % Elapsed time: 180.00 secs Data transferred: 5264.09 MB Response time: 0.10 secs Transaction rate: 9943.43 trans/sec Throughput: 29.24 MB/sec Concurrency: 997.06 Successful transactions: 1789818 Failed transactions: 0 Longest transaction: 0.87 Shortest transaction: 0.00 错误率是0
** SIEGE 2.69 ** Preparing 1000 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 1768585 hits Availability: 100.00 % Elapsed time: 179.31 secs Data transferred: 5201.65 MB Response time: 0.10 secs Transaction rate: 9863.28 trans/sec Throughput: 29.01 MB/sec Concurrency: 998.30 Successful transactions: 1768588 Failed transactions: 0 Longest transaction: 3.10 Shortest transaction: 0.03 错误率是0 nginx的配置中增加监听队列的数量,当然前提是不能超过net.core.somaxconn的值。如: server { listen 80 backlog=8192; server_name http://www.; } |
|