一、问题背景两台Linux服务器compute2和nj-rack01-06的指定网卡ens4np0和enp101s0np1通过Dell交换机二层互联，并且配置同网段的IP地址，在两个服务器上分别指定网口ens4np0和enp101s0np1互ping，结果在两台服务器上都始终无法ping通，接下来将通过抓包来进行问题分析定位。 二、组网架构两台服务器的ens4np0和enp101s0np1网卡采取简单二层连接，组网拓扑图如下所示： 
三、网络配置1、compute2网络配置信息[root@compute2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether b0:26:28:ff:c7:5a brd ff:ff:ff:ff:ff:ff
altname enp1s0f0
inet 10.8.13.12/24 brd 10.8.13.255 scope global dynamic eno3
valid_lft 6131sec preferred_lft 6131sec
inet6 fe80::b226:28ff:feff:c75a/64 scope link
valid_lft forever preferred_lft forever
3: eno4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether b0:26:28:ff:c7:5b brd ff:ff:ff:ff:ff:ff
altname enp1s0f1
4: eno1np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether b0:26:28:ff:c7:5c brd ff:ff:ff:ff:ff:ff
altname enp25s0f0np0
5: eno2np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether b0:26:28:ff:c7:5d brd ff:ff:ff:ff:ff:ff
altname enp25s0f1np1
14: bond2: <NO-CARRIER,BROADCAST,MULTICAST,MASTER,UP> mtu 1600 qdisc noqueue state DOWN group default qlen 1000
link/ether 82:94:2d:68:e1:26 brd ff:ff:ff:ff:ff:ff
25: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:ee:5f:4f:68 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
26: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether e2:ba:b8:a9:7b:23 brd ff:ff:ff:ff:ff:ff
27: mirror0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether fe:a4:0a:5b:9f:bd brd ff:ff:ff:ff:ff:ff
inet6 fe80::fca4:aff:fe5b:9fbd/64 scope link
valid_lft forever preferred_lft forever
28: ovn0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 00:00:00:84:fd:41 brd ff:ff:ff:ff:ff:ff
inet 100.64.0.2/16 brd 100.64.255.255 scope global ovn0
valid_lft forever preferred_lft forever
inet6 fe80::200:ff:fe84:fd41/64 scope link
valid_lft forever preferred_lft forever
29: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
link/ether 92:de:04:77:6a:b5 brd ff:ff:ff:ff:ff:ff
inet6 fe80::90de:4ff:fe77:6ab5/64 scope link
valid_lft forever preferred_lft forever
30: br-int: <BROADCAST,MULTICAST> mtu 1400 qdisc noop state DOWN group default qlen 1000
link/ether 86:b4:b3:6f:b5:3a brd ff:ff:ff:ff:ff:ff
39: ens1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 32:14:eb:bd:2a:15 brd ff:ff:ff:ff:ff:ff
altname enp59s0
40: ens1np0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 00:15:4d:16:66:d5 brd ff:ff:ff:ff:ff:ff
altname enp59s0np0
41: ens1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:15:4d:16:66:d6 brd ff:ff:ff:ff:ff:ff
altname enp59s0np1
42: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 92:cd:7a:0b:c7:a0 brd ff:ff:ff:ff:ff:ff
43: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 06:fc:64:f0:c6:77 brd ff:ff:ff:ff:ff:ff
altname enp175s0
inet6 fe80::4fc:64ff:fef0:c677/64 scope link
valid_lft forever preferred_lft forever
44: ens4np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:15:4d:16:20:6a brd ff:ff:ff:ff:ff:ff
altname enp175s0np0
inet 10.18.0.12/24 brd 10.18.0.255 scope global ens4np0
valid_lft forever preferred_lft forever
inet6 fe80::215:4dff:fe16:206a/64 scope link
valid_lft forever preferred_lft forever
45: ens4np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:15:4d:16:20:6b brd ff:ff:ff:ff:ff:ff
altname enp175s0np1
46: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 32:a3:d8:8e:61:a8 brd ff:ff:ff:ff:ff:ff
inet6 fe80::30a3:d8ff:fe8e:61a8/64 scope link
valid_lft forever preferred_lft forever
48: e12921368e14_h@if47: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master ovs-system state UP group default qlen 1000
link/ether 86:0b:13:97:a8:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::840b:13ff:fe97:a834/64 scope link
valid_lft forever preferred_lft forever
50: 8f49fd46cbd7_h@if49: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master ovs-system state UP group default qlen 1000
link/ether 26:17:92:7f:c3:62 brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::2417:92ff:fe7f:c362/64 scope link
valid_lft forever preferred_lft forever
51: ens4v0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether ca:7b:a7:9b:4d:c7 brd ff:ff:ff:ff:ff:ff
altname enp175s0v0
inet6 fe80::c87b:a7ff:fe9b:4dc7/64 scope link
valid_lft forever preferred_lft forever
52: ens4v1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether ea:d9:39:76:3c:72 brd ff:ff:ff:ff:ff:ff
altname enp175s0v1
inet6 fe80::e8d9:39ff:fe76:3c72/64 scope link
valid_lft forever preferred_lft forever
53: ens4v2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether d6:4a:e2:d0:39:c6 brd ff:ff:ff:ff:ff:ff
altname enp175s0v2
inet6 fe80::d44a:e2ff:fed0:39c6/64 scope link
valid_lft forever preferred_lft forever
54: ens4v3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether f6:50:c1:82:86:3e brd ff:ff:ff:ff:ff:ff
altname enp175s0v3
inet6 fe80::f450:c1ff:fe82:863e/64 scope link
valid_lft forever preferred_lft forever
55: nfp_v0.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether ae:ac:45:bf:8f:d7 brd ff:ff:ff:ff:ff:ff
inet6 fe80::acac:45ff:febf:8fd7/64 scope link
valid_lft forever preferred_lft forever
56: nfp_v0.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 7a:07:8e:22:2c:04 brd ff:ff:ff:ff:ff:ff
inet6 fe80::7807:8eff:fe22:2c04/64 scope link
valid_lft forever preferred_lft forever
57: nfp_v0.2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 32:42:eb:73:bb:6f brd ff:ff:ff:ff:ff:ff
inet6 fe80::3042:ebff:fe73:bb6f/64 scope link
valid_lft forever preferred_lft forever
58: nfp_v0.3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8e:b0:54:a3:7a:86 brd ff:ff:ff:ff:ff:ff
inet6 fe80::8cb0:54ff:fea3:7a86/64 scope link
valid_lft forever preferred_lft forever
由ip a的结果可知，compute2服务器上ens4np0网口的状态和ip地址信息均正常。 2、nj-rack01-06服务器网络配置信息：[root@nj-rack01-06 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 20:04:0f:f2:4a:e8 brd ff:ff:ff:ff:ff:ff
inet 10.8.8.16/24 brd 10.8.8.255 scope global dynamic noprefixroute em1
valid_lft 7000sec preferred_lft 7000sec
inet6 fe80::5395:4226:33be:a549/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: em2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 20:04:0f:f2:4a:e9 brd ff:ff:ff:ff:ff:ff
4: em3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 20:04:0f:f2:4a:ea brd ff:ff:ff:ff:ff:ff
5: em4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 20:04:0f:f2:4a:eb brd ff:ff:ff:ff:ff:ff
6: enp101s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 62:a5:06:71:9a:f1 brd ff:ff:ff:ff:ff:ff
inet6 fe80::60a5:6ff:fe71:9af1/64 scope link
valid_lft forever preferred_lft forever
7: enp101s0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:15:4d:16:66:33 brd ff:ff:ff:ff:ff:ff
8: enp101s0np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:15:4d:16:66:34 brd ff:ff:ff:ff:ff:ff
inet 10.18.0.16/24 brd 10.18.0.255 scope global enp101s0np1
valid_lft forever preferred_lft forever
inet6 fe80::215:4dff:fe16:6634/64 scope link
valid_lft forever preferred_lft forever
9: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 5e:79:ac:37:ba:9c brd ff:ff:ff:ff:ff:ff
inet6 fe80::5c79:acff:fe37:ba9c/64 scope link
valid_lft forever preferred_lft forever
10: p1p1_0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 7e:4a:20:8a:7f:6c brd ff:ff:ff:ff:ff:ff
11: p1p1_1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether fe:6c:db:ae:d4:58 brd ff:ff:ff:ff:ff:ff
12: p1p1_2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 5a:0a:6a:3e:57:b2 brd ff:ff:ff:ff:ff:ff
13: p1p1_3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 7e:18:0a:65:82:85 brd ff:ff:ff:ff:ff:ff
14: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 7e:1c:cf:42:16:cb brd ff:ff:ff:ff:ff:ff
15: eth6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 92:f2:db:e7:a8:e1 brd ff:ff:ff:ff:ff:ff
16: eth7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether de:be:1c:ea:e6:b1 brd ff:ff:ff:ff:ff:ff
17: eth8: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ce:86:a0:0e:dd:99 brd ff:ff:ff:ff:ff:ff
18: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:39:be:34:40 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
19: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 52:54:00:20:20:25 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
20: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN group default qlen 1000
link/ether 52:54:00:20:20:25 brd ff:ff:ff:ff:ff:ff
21: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master virbr0 state UNKNOWN group default qlen 1000
link/ether fe:54:00:fa:dc:00 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc54:ff:fefa:dc00/64 scope link
valid_lft forever preferred_lft forever
22: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 82:06:fa:8d:b1:2a brd ff:ff:ff:ff:ff:ff
23: mirror0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether c2:f5:3b:c9:9f:27 brd ff:ff:ff:ff:ff:ff
inet6 fe80::c0f5:3bff:fec9:9f27/64 scope link
valid_lft forever preferred_lft forever
24: ovn0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 00:00:00:c8:76:7b brd ff:ff:ff:ff:ff:ff
inet 100.64.0.3/16 brd 100.64.255.255 scope global ovn0
valid_lft forever preferred_lft forever
inet6 fe80::200:ff:fec8:767b/64 scope link
valid_lft forever preferred_lft forever
25: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
link/ether 8a:44:7d:eb:aa:c0 brd ff:ff:ff:ff:ff:ff
inet6 fe80::8844:7dff:feeb:aac0/64 scope link
valid_lft forever preferred_lft forever
26: br-int: <BROADCAST,MULTICAST> mtu 1400 qdisc noop state DOWN group default qlen 1000
link/ether c6:4a:8e:92:8d:42 brd ff:ff:ff:ff:ff:ff
36: cf62c26a92f3_h@if35: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue master ovs-system state UP group default qlen 1000
link/ether a6:5e:d5:bf:22:65 brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet6 fe80::a45e:d5ff:febf:2265/64 scope link
valid_lft forever preferred_lft forever
由ip a的结果可知，nj-rack01-06服务器上enp101s0np1网口的状态和ip地址信息均正常。 四、问题分析1、故障现象nj-rack01-06服务器在nj-rack01-06服务器上通过enp101s0np1网口ping相连的对端网口的地址，情况如下： [root@nj-rack01-06 ~]# ping -I enp101s0np1 10.18.0.12
PING 10.18.0.12 (10.18.0.12) from 10.18.0.16 enp101s0np1: 56(84) bytes of data.
^C
--- 10.18.0.12 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2076ms
compute2服务器在compute2服务器上通过ens4np0网口ping相连的对端网口的地址，情况如下： [root@compute2 ~]# ping -I ens4np0 10.18.0.16
PING 10.18.0.16 (10.18.0.16) from 10.18.0.12 ens4np0: 56(84) bytes of data.
^C
--- 10.18.0.16 ping statistics ---
161 packets transmitted, 0 received, 100% packet loss, time 163856ms
pipe 4
根据上述结果可知，在两台服务器上的指定网卡ping都无法ping通对端；接下来以compute2服务器上的现象来进行分析，首先在compute2服务器上进行抓包查看包是否已发送出去，如下： [root@compute2 ~]# tcpdump -i ens4np0 -nevvv
dropped privs to tcpdump
tcpdump: listening on ens4np0, link-type EN10MB (Ethernet), capture size 262144 bytes
16:29:05.644390 00:15:4d:16:20:6a > Broadcast, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.18.0.16 tell 10.18.0.12, length 28
16:29:06.704807 00:15:4d:16:20:6a > Broadcast, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.18.0.16 tell 10.18.0.12, length 28
16:29:07.728773 00:15:4d:16:20:6a > Broadcast, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.18.0.16 tell 10.18.0.12, length 28
16:29:09.729959 00:15:4d:16:20:6a > Broadcast, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.18.0.16 tell 10.18.0.12, length 28
^C
根据抓包报文可知，compute2服务器的网络协议栈已经将arp请求报文发出，接下来查看对端服务器是否收到arp请求报文，如下： [root@nj-rack01-06 ~]# tcpdump -i enp101s0np1 -e -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp101s0np1, link-type EN10MB (Ethernet), capture size 262144 bytes
15:30:00.449732 0c:29:ef:bf:b1:a3 > 01:80:c2:00:00:0e, ethertype LLDP (0x88cc), length 414: LLDP, length 400: nj-dell-switch.nji.corigine.com
15:30:30.451960 0c:29:ef:bf:b1:a3 > 01:80:c2:00:00:0e, ethertype LLDP (0x88cc), length 396: LLDP, length 382: nj-dell-switch.nji.corigine.com
15:31:00.454307 0c:29:ef:bf:b1:a3 > 01:80:c2:00:00:0e, ethertype LLDP (0x88cc), length 414: LLDP, length 400: nj-dell-switch.nji.corigine.com
^C
根据上述抓包结果可知，nj-rack01-06服务器上未收到对端服务器的arp请求报文。因此推测可能是交换机出现了问题，因此，接下来需要查看交换机上端口是否正常，是否能正常转发arp报文。 nj-dell-switch# show interface status
--------------------------------------------------------------------------------------------------
Port Description Status Speed Duplex Mode Vlan Tagged-Vlans
--------------------------------------------------------------------------------------------------
Eth 1/1/1 down 0 full T 93 103,105
Eth 1/1/2 down 0 full T 93 103,105
Eth 1/1/3 up 25G full A 66 -
Eth 1/1/4 down 0 full A 66 -
Eth 1/1/5:1 down 0 full A 5 -
Eth 1/1/6:1 down 0 full A 5 -
Eth 1/1/7:1 down 0 full A 5 -
Eth 1/1/8:1 down 0 full A 5 -
Eth 1/1/9:1 down 0 full A 100 -
Eth 1/1/10:1 up 10G full A 66 -
Eth 1/1/11:1 down 0 full A 100 -
Eth 1/1/12:1 down 0 full A 66 -
Eth 1/1/13 down 0 full A 5 -
Eth 1/1/14 down 0 full A 5 -
Eth 1/1/15 down 0 full A 5 -
--------------------------------------------------------------------------------------------------
由上可知，交换机连接的服务器的两个接口状态均是up的，并且同处于一个vlan 66中.接下来再查看一下mac地址转发表是否正常，如下： nj-dell-switch# show mac address-table
VlanId Mac Address Type Interface
66 22:15:4d:16:66:34 dynamic ethernet1/1/3
nj-dell-switch#
2、故障分析通过mac地址转发表可知，交换机只学习到了和ethernet1/1接口连接的网卡的mac地址，而未学习到与ethernet1/1/10:1接口连接的网卡的mac地址。至于原因，不得而知，此时就需要搞明白为什么交换机没有学习到与ethernet1/1/10:1接口连接的网卡的mac地址。因此决定在交换机上进行抓包，从而确定交换机是否收到网络报文。 通常来讲，交换机上抓包，网络上的博文一般都会告诉你通过镜像抓包的方式，但是简单看了一下镜像抓包的博文，看起来就很麻烦，而且我所使用的Dell交换机在网络上资料极少，基本搜不到详细的镜像抓包的指导方法。此时，便抱着试试的心态，求助其他大佬，大佬很谦虚，告诉我他已经很久没有搞过交换机了，但是依然很热心的帮我看了一下，一手操作，简直秀翻我，接下来一起欣赏他的骚操作。 nj-dell-switch# show interface ethernet 1/1/10:1
Ethernet 1/1/10:1 is up, line protocol is up
Hardware is Eth, address is 0c:29:ef:bf:b1:aa
Current address is 0c:29:ef:bf:b1:aa
Pluggable media present, SFP28 type is SFP28 25GBASE-SR
Wavelength is 850
Receive power reading is -1.988708 dBm
Interface index is 23
Internet address is not set
Mode of IPv4 Address Assignment: not set
Interface IPv6 oper status: Disabled
MTU 1532 bytes, IP MTU 1500 bytes
LineSpeed 10G, Auto-Negotiation off
Flowcontrol rx off tx off
ARP type: ARPA, ARP Timeout: 60
Last clearing of 'show interface' counters: 7 weeks 2 days 08:41:05
Queuing strategy: fifo
Input statistics:
178525367 packets, 11432189964 octets
178454818 64-byte pkts, 14780 over 64-byte pkts, 51913 over 127-byte pkts
3855 over 255-byte pkts, 0 over 511-byte pkts, 1 over 1023-byte pkts
66 Multicasts, 11799 Broadcasts, 70959 Unicasts
0 runts, 4 giants, 178442543 throttles
0 CRC, 0 overrun, 4 discarded
Output statistics:
202637 packets, 102558633 octets
75004 64-byte pkts, 16167 over 64-byte pkts, 29153 over 127-byte pkts
22923 over 255-byte pkts, 7004 over 511-byte pkts, 52386 over 1023-byte pkts
17650 Multicasts, 57789 Broadcasts, 127198 Unicasts
0 throttles, 0 discarded, 0 Collisions, wred drops
Rate Info(interval seconds):
Input 0 Mbits/sec, 454 packets/sec, 0% of line rate
Output 0 Mbits/sec, 1 packets/sec, 0% of line rate
Time since last interface status change: 00:22:59
nj-dell-switch# show interface ethernet 1/1/10:1
Ethernet 1/1/10:1 is up, line protocol is up
Hardware is Eth, address is 0c:29:ef:bf:b1:aa
Current address is 0c:29:ef:bf:b1:aa
Pluggable media present, SFP28 type is SFP28 25GBASE-SR
Wavelength is 850
Receive power reading is -1.991455 dBm
Interface index is 23
Internet address is not set
Mode of IPv4 Address Assignment: not set
Interface IPv6 oper status: Disabled
MTU 1532 bytes, IP MTU 1500 bytes
LineSpeed 10G, Auto-Negotiation off
Flowcontrol rx off tx off
ARP type: ARPA, ARP Timeout: 60
Last clearing of 'show interface' counters: 7 weeks 2 days 08:41:54
Queuing strategy: fifo
Input statistics:
178548095 packets, 11433644556 octets
178477546 64-byte pkts, 14780 over 64-byte pkts, 51913 over 127-byte pkts
3855 over 255-byte pkts, 0 over 511-byte pkts, 1 over 1023-byte pkts
66 Multicasts, 11799 Broadcasts, 70959 Unicasts
0 runts, 4 giants, 178465271 throttles
0 CRC, 0 overrun, 4 discarded
Output statistics:
202688 packets, 102562941 octets
75052 64-byte pkts, 16167 over 64-byte pkts, 29153 over 127-byte pkts
22926 over 255-byte pkts, 7004 over 511-byte pkts, 52386 over 1023-byte pkts
17653 Multicasts, 57837 Broadcasts, 127198 Unicasts
0 throttles, 0 discarded, 0 Collisions, wred drops
Rate Info(interval seconds):
Input 0 Mbits/sec, 378 packets/sec, 0% of line rate
Output 0 Mbits/sec, 0 packets/sec, 0% of line rate
Time since last interface status change: 00:23:48
nj-dell-switch# show interface ethernet 1/1/10:1
Ethernet 1/1/10:1 is up, line protocol is up
Hardware is Eth, address is 0c:29:ef:bf:b1:aa
Current address is 0c:29:ef:bf:b1:aa
Pluggable media present, SFP28 type is SFP28 25GBASE-SR
Wavelength is 850
Receive power reading is -1.991455 dBm
Interface index is 23
Internet address is not set
Mode of IPv4 Address Assignment: not set
Interface IPv6 oper status: Disabled
MTU 1532 bytes, IP MTU 1500 bytes
LineSpeed 10G, Auto-Negotiation off
Flowcontrol rx off tx off
ARP type: ARPA, ARP Timeout: 60
Last clearing of 'show interface' counters: 7 weeks 2 days 08:43:17
Queuing strategy: fifo
Input statistics:
178584469 packets, 11435972492 octets
178513920 64-byte pkts, 14780 over 64-byte pkts, 51913 over 127-byte pkts
3855 over 255-byte pkts, 0 over 511-byte pkts, 1 over 1023-byte pkts
66 Multicasts, 11799 Broadcasts, 70959 Unicasts
0 runts, 4 giants, 178501645 throttles
0 CRC, 0 overrun, 4 discarded
Output statistics:
202770 packets, 102569233 octets
75131 64-byte pkts, 16167 over 64-byte pkts, 29153 over 127-byte pkts
22929 over 255-byte pkts, 7004 over 511-byte pkts, 52386 over 1023-byte pkts
17656 Multicasts, 57916 Broadcasts, 127198 Unicasts
0 throttles, 0 discarded, 0 Collisions, wred drops
Rate Info(interval seconds):
Input 0 Mbits/sec, 454 packets/sec, 0% of line rate
Output 0 Mbits/sec, 1 packets/sec, 0% of line rate
Time since last interface status change: 00:25:11
因为ethernet1/1/10:1接口未学习到mac地址，因此多次查看该接口的状态，发现入向throttles统计的值一直在不停的增加，该统计值的意思是“节流或抑制”；交换机流量过载会触发该节流操作，其数值表示端口上的接收器被禁用的次数，持续的查找设备的MAC地址的过载流量导致其发生，短时间内无法接收任何新的数据包，因此决定抓包确认一下。 大佬告诉我交换机有一种Linux模式，可以切换到该模式下，直接使用tcpdump进行端口抓包。听到这我顿时眼前一亮，竟然还有这样的骚(黑)操(科)作(技)！只见他直接就在交换机上切换到Linux模式，开始一系列操作： #在交换机上使用ifconfig命令查看两个接口所属vlan66所对应的网桥，原来交换机上是通过网桥的方式实现的vlan
nj-dell-switch# system 'ifconfig'
bo1: flags=5123<UP,BROADCAST,MASTER,MULTICAST> mtu 1568
inet6 fe80::e29:efff:febf:b1ba prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:ba txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 5 overruns 0 carrier 0 collisions 0
bo2: flags=5123<UP,BROADCAST,MASTER,MULTICAST> mtu 1500
inet 6.6.6.10 netmask 255.255.255.0 broadcast 6.6.6.255
inet6 fe80::e29:efff:febf:b1bb prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:bb txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 3 overruns 0 carrier 0 collisions 0
bo3: flags=5123<UP,BROADCAST,MASTER,MULTICAST> mtu 1568
inet6 fe80::e29:efff:febf:b1bc prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:bc txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 4 overruns 0 carrier 0 collisions 0
bo4: flags=5123<UP,BROADCAST,MASTER,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1bd prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:bd txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0
bo5: flags=5123<UP,BROADCAST,MASTER,MULTICAST> mtu 1500
inet 5.5.5.10 netmask 255.255.255.0 broadcast 5.5.5.255
inet6 fe80::e29:efff:febf:b1be prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:be txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0
bo1.555: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 0c:29:ef:bf:b1:ba txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
bo3.888: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 0c:29:ef:bf:b1:bc txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br1: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br2: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br3: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br4: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br5: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 590 bytes 38612 (37.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1205 bytes 110594 (108.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br66: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 154750 bytes 7761460 (7.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 5146 bytes 459568 (448.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br93: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 7 bytes 452 (452.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 24 bytes 2728 (2.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br100: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 113 bytes 7136 (6.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 78 bytes 7116 (6.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br103: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 9184
inet 12.8.8.200 netmask 255.255.255.0 broadcast 12.8.8.255
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 18 bytes 2148 (2.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br105: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 19 bytes 2278 (2.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br111: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br112: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet 12.12.12.10 netmask 255.255.255.0 broadcast 12.12.12.255
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br123: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br135: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br222: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br246: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br555: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br888: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br3055: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-001-0: flags=4099<UP,BROADCAST,MULTICAST> mtu 9184
inet6 fe80::e29:efff:febf:b1a1 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-001-0.103: flags=4099<UP,BROADCAST,MULTICAST> mtu 9184
ether 0c:29:ef:bf:b1:a1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-001-0.105: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 0c:29:ef:bf:b1:a1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-002-0: flags=4099<UP,BROADCAST,MULTICAST> mtu 9184
inet6 fe80::e29:efff:febf:b1a2 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a2 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-002-0.103: flags=4099<UP,BROADCAST,MULTICAST> mtu 9184
ether 0c:29:ef:bf:b1:a2 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-002-0.105: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 0c:29:ef:bf:b1:a2 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-003-0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1a3 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a3 txqueuelen 1000 (Ethernet)
RX packets 100517 bytes 6448390 (6.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 41414 bytes 16157928 (15.4 MiB)
TX errors 0 dropped 82 overruns 0 carrier 0 collisions 0
e101-004-0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1a4 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a4 txqueuelen 1000 (Ethernet)
RX packets 51163 bytes 4662460 (4.4 MiB)
RX errors 0 dropped 8756 overruns 0 frame 0
TX packets 19779 bytes 7863844 (7.4 MiB)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0
e101-005-1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::c0e3:b6ff:fe13:ef15 prefixlen 64 scopeid 0x20<link>
inet6 fe80::e29:efff:febf:b1a5 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a5 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-006-1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::48b1:c4ff:fe53:f38b prefixlen 64 scopeid 0x20<link>
inet6 fe80::e29:efff:febf:b1a6 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a6 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-007-1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1a7 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a7 txqueuelen 1000 (Ethernet)
RX packets 783 bytes 112048 (109.4 KiB)
RX errors 0 dropped 686 overruns 0 frame 0
TX packets 568 bytes 204976 (200.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-008-1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1a8 prefixlen 64 scopeid 0x20<link>
inet6 fe80::d8af:76ff:feaf:18aa prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a8 txqueuelen 1000 (Ethernet)
RX packets 21218 bytes 3182748 (3.0 MiB)
RX errors 0 dropped 20757 overruns 0 frame 0
TX packets 11030 bytes 4292132 (4.0 MiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0
e101-009-1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1a9 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a9 txqueuelen 1000 (Ethernet)
RX packets 319 bytes 44574 (43.5 KiB)
RX errors 0 dropped 274 overruns 0 frame 0
TX packets 223 bytes 71226 (69.5 KiB)
TX errors 0 dropped 10 overruns 0 carrier 0 collisions 0
e101-010-1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1aa prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:aa txqueuelen 1000 (Ethernet)
RX packets 12276 bytes 785716 (767.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16846 bytes 6830524 (6.5 MiB)
TX errors 0 dropped 18 overruns 0 carrier 0 collisions 0
e101-011-1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1ab prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:ab txqueuelen 1000 (Ethernet)
RX packets 5616 bytes 827516 (808.1 KiB)
RX errors 0 dropped 5330 overruns 0 frame 0
TX packets 2896 bytes 1128536 (1.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-012-1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1ac prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:ac txqueuelen 1000 (Ethernet)
RX packets 2973 bytes 196128 (191.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 13306 bytes 5271746 (5.0 MiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0
e101-013-0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1ad prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:ad txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-014-0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1ae prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:ae txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
e101-015-0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1af prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:af txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1468
inet 10.8.8.49 netmask 255.255.255.0 broadcast 10.8.8.255
inet6 fe80::e29:efff:febf:b1a0 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:a0 txqueuelen 1000 (Ethernet)
RX packets 2131062 bytes 191899479 (183.0 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1980745 bytes 171880577 (163.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xdfe00000-dfe7ffff
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
inet6 fe80::200:ff:fe00:0 prefixlen 64 scopeid 0x20<link>
loop txqueuelen 1 (Local Loopback)
RX packets 1108153681 bytes 103323200462 (96.2 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1108153681 bytes 103323200462 (96.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vn123: flags=4355<UP,BROADCAST,PROMISC,MULTICAST> mtu 1500
inet6 fe80::e29:efff:febf:b1d3 prefixlen 64 scopeid 0x20<link>
ether 0c:29:ef:bf:b1:d3 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
#直接使用brctl网桥指令查看网桥上的接口
nj-dell-switch# system 'brctl show br66'
bridge name bridge id STP enabled interfaces
br66 8000.0c29efbfb1d1 no e101-003-0
e101-004-0
e101-010-1
e101-012-1
#使用brctl help指令查看brctl的其他指令如何使用
nj-dell-switch# system 'brctl help'
never heard of command [help]
Usage: brctl [commands]
commands:
addbr <bridge> add bridge
delbr <bridge> delete bridge
addif <bridge> <device> add interface to bridge
delif <bridge> <device> delete interface from bridge
hairpin <bridge> <port> {on|off} turn hairpin on/off
setageing <bridge> <time> set ageing time
setbridgeprio <bridge> <prio> set bridge priority
setfd <bridge> <time> set bridge forward delay
sethello <bridge> <time> set hello time
setmaxage <bridge> <time> set max message age
setpathcost <bridge> <port> <cost> set path cost
setportprio <bridge> <port> <prio> set port priority
show [ <bridge> ] show a list of bridges
showmacs <bridge> show a list of mac addrs
showstp <bridge> show bridge stp info
stp <bridge> {on|off} turn stp on/off
%Error: while executing(system).Ret val[1].
#查看br66网桥上的mac地址，发现未学习到compute2服务器上ens4np0网口的mac地址
nj-dell-switch# system 'brctl showmacs br66'
port no mac addr is local? ageing timer
1 0c:29:ef:bf:b1:a3 yes 0.00
1 0c:29:ef:bf:b1:a3 yes 0.00
3 0c:29:ef:bf:b1:a4 yes 0.00
3 0c:29:ef:bf:b1:a4 yes 0.00
4 0c:29:ef:bf:b1:aa yes 0.00
4 0c:29:ef:bf:b1:aa yes 0.00
2 0c:29:ef:bf:b1:ac yes 0.00
2 0c:29:ef:bf:b1:ac yes 0.00
1 22:15:4d:16:66:34 no 0.77
#通过sudo tcpdump直接在br66网桥上抓包，必须加sudo，否则会报操作不允许的错误
nj-dell-switch# system 'tcpdump -i br66'
tcpdump: br66: You don't have permission to capture on that device
(socket: Operation not permitted)
%Error: while executing(system).Ret val[1].
nj-dell-switch# system 'sudo tcpdump -i br66'
[sudo] password for admin:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br66, link-type EN10MB (Ethernet), capture size 262144 bytes
16:26:42.997761 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:26:44.021806 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:26:45.045766 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:26:46.070754 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:26:47.093782 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:26:48.117703 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:26:49.141717 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:26:50.165738 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
^C
#查看有几个接口处于vlan 66下
nj-dell-switch# show vlan
Codes: * - Default VLAN, M - Management VLAN, R - Remote Port Mirroring VLANs,
@ – Attached to Virtual Network
Q: A - Access (Untagged), T - Tagged
NUM Status Description Q Ports
1 Inactive
2 Inactive
3 Inactive
4 Inactive
* 5 Inactive A Eth1/1/5:1,1/1/6:1,1/1/7:1,1/1/8:1,1/1/13-1/1/15
A Po1,3
66 Active A Eth1/1/3-1/1/4,1/1/10:1,1/1/12:1
93 Inactive A Eth1/1/1-1/1/2
100 Inactive A Eth1/1/9:1,1/1/11:1
103 Inactive T Eth1/1/1-1/1/2
105 Inactive T Eth1/1/1-1/1/2
111 Inactive
112 Inactive
123 Inactive
135 Inactive
222 Inactive
246 Inactive
555 Inactive T Po1
888 Inactive T Po3
nj-dell-switch# system 'brctl showstp br66'
br66
bridge id 8000.0c29efbfb1d1
designated root 8000.0c29efbfb1d1
root port 0 path cost 0
max age 20.00 bridge max age 20.00
hello time 2.00 bridge hello time 2.00
forward delay 15.00 bridge forward delay 15.00
ageing time 1800.00
hello timer 0.00 tcn timer 0.00
topology change timer 0.00 gc timer 393.92
flags
e101-003-0 (1)
port id 8001 state forwarding
designated root 8000.0c29efbfb1d1 path cost 100
designated bridge 8000.0c29efbfb1d1 message age timer 0.00
designated port 8001 forward delay timer 0.00
designated cost 0 hold timer 0.00
flags
e101-004-0 (3)
port id 8003 state disabled
designated root 8000.0c29efbfb1d1 path cost 100
designated bridge 8000.0c29efbfb1d1 message age timer 0.00
designated port 8003 forward delay timer 0.00
designated cost 0 hold timer 0.00
flags
e101-010-1 (4)
port id 8004 state forwarding
designated root 8000.0c29efbfb1d1 path cost 100
designated bridge 8000.0c29efbfb1d1 message age timer 0.00
designated port 8004 forward delay timer 0.00
designated cost 0 hold timer 0.00
flags
e101-012-1 (2)
port id 8002 state disabled
designated root 8000.0c29efbfb1d1 path cost 100
designated bridge 8000.0c29efbfb1d1 message age timer 0.00
designated port 8002 forward delay timer 0.00
designated cost 0 hold timer 0.00
flags
STP（Spanning Tree Protocol）是一个用于局域网中消除环路的协议，它的标准是IEEE 802.1D。STP通过将部分冗余链路强制为阻塞状态，其他链路处于转发状态，将环形网络结构修剪成无环路的树形网络结构，可实现消除环路。当处于转发状态的链路不可用时，STP重新配置网络，并激活合适的备用链路状态，恢复恢复网络连通性。 在一个复杂的网络环境中，由于冗余备份的需要，网络设计者都倾向于在设备之间部署多条物理链路，其中一条作主用链路，其他链路作备份，偶然或必然中都会导致环路产生。环路会产生广播风暴，最终导致整个网络资源被耗尽，网络瘫痪不可用。环路还会引起MAC地址表震荡导致MAC地址表项被破坏。 STP技术可以有效的解决环路问题，将彼此交互信息发现网络中的环路，将部分冗余链路强制为阻塞状态，其他链路处于转发状态。最终将环形网络结构修剪成无环路的树形网络结构，从而防止报文在环形网络中不断增生和无限循环，避免设备由于重复接收相同的报文造成处理能力下降。
由show interface status指令结果可知,Eth1/1/3接口连接nj-rack01-06服务器上enp101s0np1网口，由brctl show br66指令可知，br66桥上的e101-003-0接口即对应交换机的Eth1/1/3接口,因此在e101-003-0接口抓包，即就是在Eth1/1/3接口抓包。如下： nj-dell-switch# system 'sudo tcpdump -i e101-003-0'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on e101-003-0, link-type EN10MB (Ethernet), capture size 262144 bytes
16:28:47.900070 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:28:48.948548 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:28:49.972517 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:28:50.996668 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:28:52.020531 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
16:28:53.044518 ARP, Request who-has 10.18.0.12 tell 10.18.0.16, length 50
^C
由上述结果，可知，交换机上能够收到nj-rack01-06服务器上enp101s0np1网口发来的arp请求包，所以交换机上的mac转发表有enp101s0np1网口的mac地址；让我们再在e101-010-1 接口抓包，也就是在Eth1/1/10:1 接口抓包，如下： nj-dell-switch# system 'sudo tcpdump -i e101-010-1'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on e101-010-1, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
%Error: while executing(system).Ret val[0].
nj-dell-switch#
由上述结果可知，交换机该接口没有网络报文，结合前面throttles的数值的不断增加，可知该接口因持续的收到查找设备MAC地址的报文，导致该接口输入队列持续过载，从而触发节流操作，无法再接收新的报文。因此在交换机该接口无法再抓到任何报文，compute2服务器上ens4np0网口无法抓到对端发来的arp请求报文也印证了这一点。 为什么会导致该接口输入队列持续过载？因为持续的收到查找设备MAC地址的报文。为什么会持续的收到找设备MAC地址的报文？因此推测可能是连接该交换机端口的设备异常，导致无法响应查找MAC地址的报文。因此，开始检查和该交换机端口连接的网络设备是否异常，如下： [root@compute2 lxy]# ethtool -S ens4np0
NIC statistics:
mac.rx_octets: 2441345
mac.rx_frame_too_long_errors: 0
mac.rx_range_length_errors: 0
mac.rx_vlan_received_ok: 0
mac.rx_errors: 0
mac.rx_broadcast_pkts: 16200
mac.rx_drop_events: 0
mac.rx_alignment_errors: 0
mac.rx_pause_mac_ctrl_frames: 0
mac.rx_frames_received_ok: 19943
mac.rx_frame_check_sequence_err: 0
mac.rx_unicast_pkts: 55
mac.rx_multicast_pkts: 3688
mac.rx_pkts: 19943
mac.rx_undersize_pkts: 0
mac.rx_pkts_64_octets: 16216
mac.rx_pkts_65_to_127_octets: 427
mac.rx_pkts_128_to_255_octets: 29
mac.rx_pkts_256_to_511_octets: 3270
mac.rx_pkts_512_to_1023_octets: 0
mac.rx_pkts_1024_to_1518_octets: 1
mac.rx_pkts_1519_to_max_octets: 0
mac.rx_jabbers: 0
mac.rx_fragments: 0
mac.rx_oversize_pkts: 0
mac.rx_pause_frames_class0: 0
mac.rx_pause_frames_class1: 0
mac.rx_pause_frames_class2: 0
mac.rx_pause_frames_class3: 0
mac.rx_pause_frames_class4: 0
mac.rx_pause_frames_class5: 0
mac.rx_pause_frames_class6: 0
mac.rx_pause_frames_class7: 0
mac.rx_mac_ctrl_frames_received: 0
mac.rx_mac_head_drop: 19252
mac.tx_queue_drop: 0
mac.tx_octets: 2188771300
mac.tx_vlan_transmitted_ok: 0
mac.tx_errors: 1
mac.tx_broadcast_pkts: 7
mac.tx_pause_mac_ctrl_frames: 34199164
mac.tx_frames_transmitted_ok: 34199209
mac.tx_unicast_pkts: 37
mac.tx_multicast_pkts: 1
mac.tx_pkts_64_octets: 34199172
mac.tx_pkts_65_to_127_octets: 2
mac.tx_pkts_128_to_255_octets: 23
mac.tx_pkts_256_to_511_octets: 12
mac.tx_pkts_512_to_1023_octets: 0
mac.tx_pkts_1024_to_1518_octets: 0
mac.tx_pkts_1519_to_max_octets: 1
mac.tx_pause_frames_class0: 0
mac.tx_pause_frames_class1: 0
mac.tx_pause_frames_class2: 0
mac.tx_pause_frames_class3: 0
mac.tx_pause_frames_class4: 0
mac.tx_pause_frames_class5: 0
mac.tx_pause_frames_class6: 0
mac.tx_pause_frames_class7: 0
通过ethtool工具查看该网络设备的统计信息，rx_mac_head_drop统计值明显异常，可知网卡无法处理mac报文头从而丢掉。再查看dmesg信息，如下： [root@compute2 ~]# dmesg | tail
[44688.426342] nfp 0000:af:00.0: nfp: Reading media link modes not supported. Please update flash
[44688.630940] nfp 0000:af:00.0 ens4: TX watchdog timeout on ring: 4
[44694.774418] nfp 0000:af:00.0 ens4: TX watchdog timeout on ring: 4
[44700.405916] nfp 0000:af:00.0 ens4: TX watchdog timeout on ring: 4
[44705.525421] nfp 0000:af:00.0 ens4: TX watchdog timeout on ring: 4
[44710.644983] nfp 0000:af:00.0 ens4: TX watchdog timeout on ring: 4
[44716.789408] nfp 0000:af:00.0 ens4: TX watchdog timeout on ring: 4
[44722.419914] nfp 0000:af:00.0 ens4: TX watchdog timeout on ring: 4
[44727.539463] nfp 0000:af:00.0 ens4: TX watchdog timeout on ring: 4
[44732.659012] nfp 0000:af:00.0 ens4: TX watchdog timeout on ring: 4
到此，终于知道问题所在了，原来是网卡的固件和BSP异常，使用23.04版本更新固件和BSP版本；加载成功之后，重启主机，再次配置网络信息后，网络终于通了，如下： [root@compute2 ~]# ping -I ens4np0 10.18.0.16
PING 10.18.0.16 (10.18.0.16) from 10.18.0.12 ens4np0: 56(84) bytes of data.
64 bytes from 10.18.0.16: icmp_seq=1 ttl=64 time=0.293 ms
64 bytes from 10.18.0.16: icmp_seq=2 ttl=64 time=0.312 ms
^C
问题终于解决了，简直太不容易了，这个问题困扰了小编好久，没想到被大佬三下五除二就解决了，大佬不愧是大佬，就是强。 查看交换机上的MAC转发表，如下： nj-dell-switch# show mac address-table
VlanId Mac Address Type Interface
66 00:15:4d:16:20:6a dynamic ethernet1/1/10:1
66 00:15:4d:16:66:34 dynamic ethernet1/1/3
交换机也能正常学习到mac了。 五、总结本文通过一个简单的二层流量不通的小例子，讲解流量不通的排障方法，包括服务器网卡抓包、交换机接口抓包等操作。其中，交换机通过tcpdump工具抓包不得不说是一个黑科技，第一次知道交换机还可以这样玩，小编一直以为交换机抓包只能通过端口镜像操作，有点刷新了小编对交换机的认知。我相信其他厂家的交换机应该也有这种神操作，方法未必完全一样，但是通过tcpdump抓包的神迹应该都具备，大家感兴趣的可以去试一下。
|
