|
和同事的联调时碰到一个诡异的问题，server端的业务逻辑层面没有任何异常，但返回结果给client端时，并没有发送完所有的数据，就直接发送了RST给client，导致client读取结果失败。注: client与server直接是短连接，server在write所有的数据后会直接close。
通过tcpdump截包对比之前旧版本正常的client端通信，发送server与旧版本的client通信时，三次握手 => client发送数据 => server发送响应数据 => 四次挥手。但与新版本client通信时，则直接是 三次握手 => client发送数据 => server发送响应数据(未完成) => server发送RST。
借助强大的Google大神，以及万能Stackoverflow。原因如下：当close断开连接时，如果缓冲区中又未被读取的数据，则tcp不会发送正常的FIN包，而发送RST给对端。
简单测试一下，代码如下，client和server的主要代码如下：
- /*
- * client
- */
- void send_plain_buf(int fd, size_t buf_size)
- {
- char buf[buf_size];
- memset(buf, 0, buf_size);
- int ret = write(fd, buf, buf_size);
- if(ret < 0) {
- fprintf(stderr, "send failed, err[%m]\n");
- return;
- }
- ret = read(fd, buf, 1);
- if(ret == 0) {
- fprintf(stderr, "client get FIN\n");
- } else {
- fprintf(stderr, "read ret[%d], err[%m]", ret);
- }
- return;
- }
-
- /*
- *server
- */
-
- void recv_plain_buf(int fd, size_t buf_size)
- {
- char buf[buf_size];
- memset(buf, 0, buf_size);
- int ret = read(fd, buf, buf_size);
- if(ret < 0) {
- fprintf(stderr, "server recv failed, err[%m]\n");
- return;
- }
- return;
- }
测试一：client发送1个byte，server读取1个byte，tcpdump截包
- 21:35:43.829700 IP tc-im-nrd306.tc.baidu.com.22579 > tc-im-nrd301.tc.baidu.com.8654: S 3630522968:3630522968(0) win 5840 <mss 1460,sackOK,timestamp 1827311072 0,nop,wscale 7>
- 21:35:43.829706 IP tc-im-nrd301.tc.baidu.com.8654 > tc-im-nrd306.tc.baidu.com.22579: S 2472593503:2472593503(0) ack 3630522969 win 5792 <mss 1460,sackOK,timestamp 2429650378 1827311072,nop,wscale 7>
- 21:35:43.829846 IP tc-im-nrd306.tc.baidu.com.22579 > tc-im-nrd301.tc.baidu.com.8654: . ack 1 win 46 <nop,nop,timestamp 1827311073 2429650378>
- 21:35:43.829873 IP tc-im-nrd306.tc.baidu.com.22579 > tc-im-nrd301.tc.baidu.com.8654: P 1:2(1) ack 1 win 46 <nop,nop,timestamp 1827311073 2429650378>
- 21:35:43.829876 IP tc-im-nrd301.tc.baidu.com.8654 > tc-im-nrd306.tc.baidu.com.22579: . ack 2 win 46 <nop,nop,timestamp 2429650378 1827311073>
- 21:35:43.829888 IP tc-im-nrd301.tc.baidu.com.8654 > tc-im-nrd306.tc.baidu.com.22579: F 1:1(0) ack 2 win 46 <nop,nop,timestamp 2429650378 1827311073>
- 21:35:43.830035 IP tc-im-nrd306.tc.baidu.com.22579 > tc-im-nrd301.tc.baidu.com.8654: . ack 2 win 46 <nop,nop,timestamp 1827311073 2429650378>
- 21:35:43.830080 IP tc-im-nrd306.tc.baidu.com.22579 > tc-im-nrd301.tc.baidu.com.8654: F 2:2(0) ack 2 win 46 <nop,nop,timestamp 1827311073 2429650378>
- 21:35:43.830083 IP tc-im-nrd301.tc.baidu.com.8654 > tc-im-nrd306.tc.baidu.com.22579: . ack 3 win 46 <nop,nop,timestamp 2429650378 1827311073>
测试二：client发送2个byte，server读取1个byte，tcpdump截包
- 21:36:31.137496 IP tc-im-nrd306.tc.baidu.com.22580 > tc-im-nrd301.tc.baidu.com.8654: S 3686863514:3686863514(0) win 5840 <mss 1460,sackOK,timestamp 1827358388 0,nop,wscale 7>
- 21:36:31.137501 IP tc-im-nrd301.tc.baidu.com.8654 > tc-im-nrd306.tc.baidu.com.22580: S 2526723815:2526723815(0) ack 3686863515 win 5792 <mss 1460,sackOK,timestamp 2429697693 1827358388,nop,wscale 7>
- 21:36:31.137666 IP tc-im-nrd306.tc.baidu.com.22580 > tc-im-nrd301.tc.baidu.com.8654: . ack 1 win 46 <nop,nop,timestamp 1827358388 2429697693>
- 21:36:31.137671 IP tc-im-nrd306.tc.baidu.com.22580 > tc-im-nrd301.tc.baidu.com.8654: P 1:3(2) ack 1 win 46 <nop,nop,timestamp 1827358388 2429697693>
- 21:36:31.137676 IP tc-im-nrd301.tc.baidu.com.8654 > tc-im-nrd306.tc.baidu.com.22580: . ack 3 win 46 <nop,nop,timestamp 2429697693 1827358388>
- 21:36:31.137705 IP tc-im-nrd301.tc.baidu.com.8654 > tc-im-nrd306.tc.baidu.com.22580: R 1:1(0) ack 3 win 46 <nop,nop,timestamp 2429697693 1827358388>
在net/ipv4/tcp.c:1900行附近的代码如下：
- /* As outlined in RFC 2525, section 2.17, we send a RST here because
- * data was lost. To witness the awful effects of the old behavior of
- * always doing a FIN, run an older 2.1.x kernel or 2.0.x, start a bulk
- * GET in an FTP client, suspend the process, wait for the client to
- * advertise a zero window, then kill -9 the FTP client, wheee...
- * Note: timeout is always zero in such a case.
- */
- if (data_was_unread) {
- /* Unread data was tossed, zap the connection. */
- NET_INC_STATS_USER(sock_net(sk), LINUX_MIB_TCPABORTONCLOSE);
- tcp_set_state(sk, TCP_CLOSE);
- tcp_send_active_reset(sk, sk->sk_allocation);
- ..
标签： TCP
|
