This article is a supplement to the previous article:
Optimization reference for Nginx in high concurrency scenarios
Optimization of kernel parameters in the /etc/sysctl.conf file
1.net.ipv4.tcp_max_tw_buckets
The number of timewaits, the default is 180000. So if you want to lower the timewait, you need to reduce the tcp_max_tw_buckets value.
|
1 |
net.ipv4.tcp_max_tw_buckets = 6000 |
2.net.ipv4.ip_local_port_range
Port range that allows the system to open
|
1 |
net.ipv4.ip_local_port_range = 1024 65000 |
3.net.ipv4.tcp_tw_recycle
Enable TIME-WAIT state sockets fast reclaim function; used to quickly reduce the number of TCP connections in the TIME-WAIT state. 1 means enabled; 0 means closed. However, it is important to note that this option is generally not recommended, because under the Network (Network Address Translation) network, a large number of TCP connection establishment errors will occur, causing website access failure.
|
1 |
net.ipv4.tcp_tw_recycle = 0 |
PS.
In fact, the opening of the net.ipv4.tcp_tw_recycle function requires net.ipv4.tcp_timestamps (the system defaults to enable this function).
When tcp_tw_recycle is turned on (tcp_timestamps is turned on at the same time, the effect of quickly reclaiming the socket is reached), it is a disaster for the client behind the NAT device! This will cause the Client Connection Server behind the NAT device to be unstable (some Clients can connect to the server, and some Clients cannot connect to the server).
In other words, the tcp_tw_recycle function is designed for the internal network (the network environment is controllable by itself – there is no NAT), and it should not be used in the public network environment.
In general, the socket in the TIME_WAIT state is reclaimed because “the remote cannot be actively connected” because there is no port available, and it should not be reclaimed (not necessary).
That is: the demand is the demand of the client, the Server will have the problem of “the port is not enough”?
Unless it is a front-end machine, it requires a lot of connection back-end services, which acts as a client.
The correct way to solve this problem is always:
|
1 2 3 4 |
net.ipv4.ip_local_port_range = 9000 6553 #The default value range is smaller net.ipv4.tcp_max_tw_buckets = 10000 #Default value is small, but can be adjusted appropriately net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_fin_timeout = 10 |
4.net.ipv4.tcp_tw_reuse
Turns on re-use, allowing TIME-WAIT state sockets to be reused for new TCP connections. This feature is enabled to be safe, generally do not change!
|
1 |
net.ipv4.tcp_tw_reuse = 1 |
5.net.ipv4.tcp_syncookies
Enable SYN Cookies. When a SYN wait queue overflow occurs, cookies are enabled for processing.
|
1 |
net.ipv4.tcp_syncookies = 1 |
6.net.core.somaxconn
The backlog of the listen function in the web application will limit the net.core.somaxconn of the kernel parameter to 128 by default, and the NGX_LISTEN_BACKLOG defined by nginx defaults to 511, so it is necessary to adjust this value.
|
1 |
net.core.somaxconn = 262144 |
7.net.core.netdev_max_backlog
The maximum number of packets that are allowed to be sent to the queue when each network interface receives packets at a faster rate than the kernel processes them.
|
1 |
net.core.netdev_max_backlog = 262144 |
8.net.ipv4.tcp_max_orphans
The maximum number of TCP sockets in the system is not associated with any user file handle. If this number is exceeded, the orphan connection will be reset immediately and a warning message will be printed. This restriction is only to prevent a simple DoS attack, not to rely too much on it or artificially reduce this value, but should increase this value (if memory is added).
|
1 |
net.ipv4.tcp_max_orphans = 262144 |
9.net.ipv4.tcp_max_syn_backlog
The maximum number of connection requests logged that have not yet received client acknowledgment information. For systems with 128M memory, the default is 1024, and for small memory systems is 128.
|
1 |
net.ipv4.tcp_max_syn_backlog = 262144 |
10.net.ipv4.tcp_timestamps
The time stamp prevents the winding of the serial number. A 1Gbps link will definitely encounter a serial number that was previously used. The timestamp allows the kernel to accept this “abnormal” packet.
|
1 |
net.ipv4.tcp_timestamps = 1 |
There are a lot of servers in order to improve performance, open the net.ipv4.tcp_tw_recycle option, in the NAT network environment, it is easy to cause some connection failures in the website access
Close the net.ipv4.tcp_tw_recycle option instead of net.ipv4.tcp_timestamps;
Because net.ipv4.tcp_tw_recycle does not work under the condition that net.ipv4.tcp_timestamps is turned off; net.ipv4.tcp_timestamps can be started and functioned independently.
11.net.ipv4.tcp_synack_retries
In order to open the peer connection, the kernel needs to send a SYN with an ACK that responds to the previous SYN. This is the second handshake in the so-called three-way handshake. This setting determines the number of SYN+ACK packets sent before the kernel abandons the connection.
|
1 |
net.ipv4.tcp_synack_retries = 1 |
12.net.ipv4.tcp_syn_retries
The number of SYN packets sent before the kernel abandoned the connection.
|
1 |
net.ipv4.tcp_syn_retries = 1 |
13.net.ipv4.tcp_fin_timeou
If the socket is requested to be closed by the local end, this parameter determines when it remains in the FIN-WAIT-2 state. The peer can make mistakes and never close the connection, or even crash unexpectedly. The default is 60 seconds. 2.2 The usual value of the kernel is 180 seconds, you can press this setting, but keep in mind that even if your machine is a light-loaded WEB server, there is a risk of memory overflow due to a large number of dead sockets, FIN- WAIT-2 is less dangerous than FIN-WAIT-1 because it can only eat up to 1.5K of memory, but they have a longer lifetime
|
1 |
net.ipv4.tcp_fin_timeout = 30 |
14.net.ipv4.tcp_keepalive_time
The frequency at which TCP sends keepalive messages when keepalive is enabled. The default is 2 hours.
|
1 |
net.ipv4.tcp_keepalive_time = 30 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
[root@test-nginxer ~]# cat /etc/sysctl.conf net.ipv4.ip_forward = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 //These four lines content, generally a solution when a large number of TIME_WAIT is found kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.shmmax = 68719476736 kernel.shmall = 4294967296 net.ipv4.tcp_max_tw_buckets = 6000 net.ipv4.tcp_sack = 1 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_rmem = 4096 87380 4194304 net.ipv4.tcp_wmem = 4096 16384 4194304 net.core.wmem_default = 8388608 net.core.rmem_default = 8388608 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.core.netdev_max_backlog = 262144 net.core.somaxconn = 262144 net.ipv4.tcp_max_orphans = 3276800 net.ipv4.tcp_max_syn_backlog = 262144 net.ipv4.tcp_timestamps = 1 //When net.ipv4.tcp_tw_recycle is set to 1, this option is best added net.ipv4.tcp_synack_retries = 1 net.ipv4.tcp_syn_retries = 1 net.ipv4.tcp_tw_recycle = 1 //Turning on this feature can reduce the TIME-WAIT state, but opening in NAT network mode may cause tcp connection errors, be cautious. net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_mem = 94500000 915000000 927000000 net.ipv4.tcp_fin_timeout = 30 net.ipv4.tcp_keepalive_time = 30 net.ipv4.ip_local_port_range = 1024 65000 net.ipv4.ip_conntrack_max = 6553500 |
PS.
Net.ipv4.tcp_tw_recycle = 1 When this function is turned on, it can really reduce the TIME-WAIT state, but opening this parameter will cause a lot of TCP connection establishment errors, which will cause website access failure. In case of failure, only set net.ipv4.tcp_tw_recycle to 0 to solve the problem.
This article was first published by V on 2018-10-12 and can be reprinted with permission, but please be sure to indicate the original link address of the article :http://www.nginxer.com/records/optimization-of-linux-kernel-parameters-when-using-nginx/