我们这一节来看Port
一般来说一个Port就是一个Interface，当然也有一个Port对应多个Interface的情况，成为Bond
VLAN Configuration
Port的一个重要的方面就是VLAN Configuration，有两种模式：
trunk port，这个port不配置tag，配置trunks，如果trunks为空，则所有的VLAN都trunk，也就意味着对于所有的VLAN的包，本身带什么VLAN ID，就是携带者什么VLAN ID，如果没有设置VLAN，就属于VLAN 0，全部允许通过。如果trunks不为空，则仅仅带着这些VLAN ID的包通过。
access port，这个port配置tag，从这个port进来的包会被打上这个tag，如果从其他的trunk port中进来的本身就带有VLAN ID的包，如果VLAN ID等于tag，则会从这个port发出，从其他的access port上来的包，如果tag相同，也会被forward到这个port。从access port发出的包不带VLAN ID。如果一个本身带VLAN ID的包到达access port，即便VLAN ID等于tag，也会被抛弃。
我们创建下面的拓扑来做一个实验：
sudo ovs-vsctl add-port helloworld first_br
sudo ovs-vsctl add-port helloworld second_br
sudo ovs-vsctl add-port helloworld third_br
sudo ovs-vsctl set Port vnet8 tag=101
sudo ovs-vsctl set Port vnet9 tag=102
sudo ovs-vsctl set Port vnet10 tag=103
sudo ovs-vsctl set Port first_br tag=103
sudo ovs-vsctl clear Port second_br tag
sudo ovs-vsctl set Port third_br trunks=101,102
sudo ovs-vsctl set bridge helloworld flood-vlans=101,102,103
sudo ovs-vsctl -- --id=@mirror get Mirror mirrorvnet9 -- remove Bridge helloworld mirrors @mirror
$ sudo ovs-vsctl show c24322e6-8453-402a-afaf-64757ef231e9 Bridge helloworld Port "vnet8" tag: 101 Interface "vnet8" Port first_br tag: 103 Interface first_br Port second_br Interface second_br Port "vnet10" tag: 103 Interface "vnet10" Port helloworld Interface helloworld type: internal Port "vnet9" tag: 102 Interface "vnet9" Port third_br trunks: [101, 102] Interface third_br ovs_version: "2.0.1"
我们首先从10.10.10.3来ping 10.10.10.4，应该first_if和second_if能够收到包。
first_if收到包了，从first_br出来的包头是没有VLAN ID的
$ sudo tcpdump -n -e -i first_if arp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on first_if, link-type EN10MB (Ethernet), capture size 65535 bytes 23:22:45.707470 ca:d4:fd:47:a6:ce > 52:54:00:9b:d5:be, ethertype ARP (0x0806), length 42: Request who-has 10.10.10.3 tell 10.10.10.4, length 28 23:22:45.708170 52:54:00:9b:d5:be > ca:d4:fd:47:a6:ce, ethertype ARP (0x0806), length 42: Reply 10.10.10.3 is-at 52:54:00:9b:d5:be, length 28
second_if也收到包了，由于second_br是trunk port，因而出来的包头是有VLAN ID的，103
$ sudo tcpdump -n -e -i second_if arp tcpdump: WARNING: second_br: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on second_br, link-type EN10MB (Ethernet), capture size 65535 bytes 23:22:45.707727 ca:d4:fd:47:a6:ce > 52:54:00:9b:d5:be, ethertype 802.1Q (0x8100), length 46: vlan 103, p 0, ethertype ARP, Request who-has 10.10.10.3 tell 10.10.10.4, length 28 23:22:45.708176 52:54:00:9b:d5:be > ca:d4:fd:47:a6:ce, ethertype 802.1Q (0x8100), length 46: vlan 103, p 0, ethertype ARP, Reply 10.10.10.3 is-at 52:54:00:9b:d5:be, length 28
third_if收不到包
$ sudo tcpdump -n -e -i third_if arp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on third_if, link-type EN10MB (Ethernet), capture size 65535 bytes
然后我们从10.10.10.1在ping 10.10.10.6，则second_if和third_if可以收到包
first_if收不到包
second_br能够收到包，而且包头里面是VLAN ID = 101
$ sudo tcpdump -n -e -i second_br arp tcpdump: WARNING: second_br: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on second_br, link-type EN10MB (Ethernet), capture size 65535 bytes 23:27:22.645546 52:54:00:9b:d5:bc > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 10.10.10.6 tell 10.10.10.1, length 28 23:27:23.644828 52:54:00:9b:d5:bc > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 10.10.10.6 tell 10.10.10.1, length 28 23:27:24.644837 52:54:00:9b:d5:bc > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 10.10.10.6 tell 10.10.10.1, length 28 23:27:25.651223 52:54:00:9b:d5:bc > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 10.10.10.6 tell 10.10.10.1, length 28 23:27:26.648770 52:54:00:9b:d5:bc > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has
third_if也能收到包，而且包头里面是VLAN ID =101
$ sudo tcpdump -n -e -i third_if arp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on third_if, link-type EN10MB (Ethernet), capture size 65535 bytes 23:27:22.645561 52:54:00:9b:d5:bc > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 10.10.10.6 tell 10.10.10.1, length 28 23:27:23.644844 52:54:00:9b:d5:bc > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 10.10.10.6 tell 10.10.10.1, length 28 23:27:24.644853 52:54:00:9b:d5:bc > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 10.10.10.6 tell 10.10.10.1, length 28 23:27:25.651238 52:54:00:9b:d5:bc > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 101, p 0, ethertype ARP, Request who-has 10.10.10.6 tell 10.10.10.1, length 28
最后我们从10.10.10.2来ping 10.10.10.5，则second_if和third_if可以收到包
first_if收不到包
second_br能够收到包，而且包头里面是VLAN ID = 102
$ sudo tcpdump -n -e -i second_br arp tcpdump: WARNING: second_br: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on second_br, link-type EN10MB (Ethernet), capture size 65535 bytes 23:31:17.641237 52:54:00:9b:d5:bd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 102, p 0, ethertype ARP, Request who-has 10.10.10.5 tell 10.10.10.2, length 28 23:31:18.638875 52:54:00:9b:d5:bd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 102, p 0, ethertype ARP, Request who-has 10.10.10.5 tell 10.10.10.2, length 28 23:31:19.638881 52:54:00:9b:d5:bd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 102, p 0, ethertype ARP, Request who-has 10.10.10.5 tell 10.10.10.2, length 28 23:31:20.657189 52:54:00:9b:d5:bd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 102, p 0, ethertype ARP, Request who-has 10.10.10.5 tell 10.10.10.2, length 28 23:31:21.654874 52:54:00:9b:d5:bd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 102, p 0, ethertype ARP, Request who-has 10.10.10.5 tell 10.10.10.2, length 28
third_if也能收到包，而且包头里面是VLAN ID =103
$ sudo tcpdump -n -e -i third_if arp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on third_if, link-type EN10MB (Ethernet), capture size 65535 bytes 23:31:22.654872 52:54:00:9b:d5:bd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 102, p 0, ethertype ARP, Request who-has 10.10.10.5 tell 10.10.10.2, length 28 23:31:23.672558 52:54:00:9b:d5:bd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 102, p 0, ethertype ARP, Request who-has 10.10.10.5 tell 10.10.10.2, length 28 23:31:24.670893 52:54:00:9b:d5:bd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 102, p 0, ethertype ARP, Request who-has 10.10.10.5 tell 10.10.10.2, length 28 23:31:25.670796 52:54:00:9b:d5:bd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 102, p 0, ethertype ARP, Request who-has 10.10.10.5 tell 10.10.10.2, length 28
Bonding Configuration
bond_mode:
balance-slb: Balances flows among slaves based on source MAC address and output VLAN
active-backup:Assigns all flows to one slave, failing over to a backup slave when the active slave is disabled.
如果bridge的另一端支持LACP协议，则还有两种状态：
balance−tcp：Balances flows among slaves based on L2, L3, and L4 protocol information such as destination MAC address, IP address, and TCP port.
stable：Attempts to always assign a given flow to the same slave consistently.
other_config : bond-detect-mode:
Defaults to carrier which uses each interface’s carrier to detect failures.
When set to miimon, will check for failures by polling each interface’s MII. 仅仅check接口是否正常
LACP Configuration:
LACP：Link Aggregation Control Protocol
Link Aggregation
Link aggregation allows you to bond multiple parallel links into a single virtual link (from the STP perspective).
With parallel links being replaced by a single link, STP detects no loops and all the physical links can be fully utilized.
Traditional LA : port channel, Etherchannel, link bonding or multi-link trunking
Traditional Link Aggregation
A port channel bundles up to eight individual interfaces into a group to provide increased bandwidth and redundancy.
Port channeling also load balances traffic across these physical interfaces.
You create a port channel by bundling compatible interfaces.
You can configure and run either static port channels or ports channels running the Link Aggregation Control Protocol (LACP).
LACP (Link Aggregation Control Protocol)
individual links can be combined into LACP port channels and channel groups
Static LACP : creation of channel groups and addition of ports are manually configured. LACP is to determine the ports are selected or standby
Dynamic LACP : all above are negotiated via LACPDU between both sides
Multi-Chassis Link Aggregation
In Multichassis EtherChannel (MCEC), the DHD is dual-homed to two upstream PoAs(points of attachment).
The DHD is incapable of running any loop prevention control protocol such as Multiple Spanning Tree (MST).
One method is to place the DHD's uplinks in a LAG, commonly referred to as EtherChannel. (LACP enabled).
LACP is a link-level control protocol that allows the dynamic negotiation and establishment of LAGs.
Multichassis LACP: An extension of the LACP implementation to PoAs is required to convey to a DHD that it is connected to a single virtual LACP peer and not to two disjointed devices.
我们创建下面的拓扑结构：
$ sudo ovs-vsctl add-bond hello bond0 first_br second_br $ sudo ovs-vsctl add-bond world bond1 first_if second_if $ sudo ovs-vsctl show c24322e6-8453-402a-afaf-64757ef231e9 Bridge world Port world Interface world type: internal Port "bond1" Interface second_if Interface first_if Port "vnet10" Interface "vnet10" Port "vnet11" Interface "vnet11" Bridge hello Port "vnet8" Interface "vnet8" Port "vnet9" Interface "vnet9" Port hello Interface hello type: internal Port "bond0" Interface first_br Interface second_br ovs_version: "2.0.1"
$ sudo ovs-vsctl set Port bond0 lacp=active $ sudo ovs-vsctl set Port bond1 lacp=active
$ sudo ovs-appctl bond/show ---- bond0 ---- bond_mode: active-backup bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms lacp_status: negotiated
slave first_br: enabled active slave may_enable: true
slave second_br: enabled may_enable: true
---- bond1 ---- bond_mode: active-backup bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms lacp_status: negotiated
slave first_if: enabled active slave may_enable: true
slave second_if: enabled may_enable: true
$ sudo ovs-appctl lacp/show ---- bond0 ---- status: active negotiated sys_id: 4a:f5:ee:f2:40:40 sys_priority: 65534 aggregation key: 3 lacp_time: slow
slave: first_br: current attached port_id: 3 port_priority: 65535 may_enable: true
actor sys_id: 4a:f5:ee:f2:40:40 actor sys_priority: 65534 actor port_id: 3 actor port_priority: 65535 actor key: 3 actor state: activity aggregation synchronized collecting distributing
partner sys_id: 8a:7b:f0:8b:94:40 partner sys_priority: 65534 partner port_id: 4 partner port_priority: 65535 partner key: 4 partner state: activity aggregation synchronized collecting distributing
slave: second_br: current attached port_id: 4 port_priority: 65535 may_enable: true
actor sys_id: 4a:f5:ee:f2:40:40 actor sys_priority: 65534 actor port_id: 4 actor port_priority: 65535 actor key: 3 actor state: activity aggregation synchronized collecting distributing
partner sys_id: 8a:7b:f0:8b:94:40 partner sys_priority: 65534 partner port_id: 5 partner port_priority: 65535 partner key: 4 partner state: activity aggregation synchronized collecting distributing ---- bond1 ---- status: active negotiated sys_id: 8a:7b:f0:8b:94:40 sys_priority: 65534 aggregation key: 4 lacp_time: slow
slave: first_if: current attached port_id: 4 port_priority: 65535 may_enable: true
actor sys_id: 8a:7b:f0:8b:94:40 actor sys_priority: 65534 actor port_id: 4 actor port_priority: 65535 actor key: 4 actor state: activity aggregation synchronized collecting distributing
partner sys_id: 4a:f5:ee:f2:40:40 partner sys_priority: 65534 partner port_id: 3 partner port_priority: 65535 partner key: 3 partner state: activity aggregation synchronized collecting distributing
slave: second_if: current attached port_id: 5 port_priority: 65535 may_enable: true
actor sys_id: 8a:7b:f0:8b:94:40 actor sys_priority: 65534 actor port_id: 5 actor port_priority: 65535 actor key: 4 actor state: activity aggregation synchronized collecting distributing
partner sys_id: 4a:f5:ee:f2:40:40 partner sys_priority: 65534 partner port_id: 4 partner port_priority: 65535 partner key: 3 partner state: activity aggregation synchronized collecting distributing
默认情况下bond_mode是active-backup模式，一开始active的是first_br和first_if
这个时候我们从10.10.10.1 ping 10.10.10.3，以及10.10.10.2 ping 10.10.10.4，都是从first_if通过
$ sudo tcpdump -n -e -i first_if tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on first_if, link-type EN10MB (Ethernet), capture size 65535 bytes 18:58:34.402793 f2:ac:70:72:49:61 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 18:58:35.403821 ca:d4:fd:47:a6:ce > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 18:59:04.402847 f2:ac:70:72:49:61 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 18:59:05.403779 ca:d4:fd:47:a6:ce > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 18:59:09.950809 52:54:00:9b:d5:bc > 52:54:00:9b:d5:be, ethertype IPv4 (0x0800), length 98: 10.10.10.1 > 10.10.10.3: ICMP echo request, id 1166, seq 1, length 64 18:59:09.951228 52:54:00:9b:d5:be > 52:54:00:9b:d5:bc, ethertype IPv4 (0x0800), length 98: 10.10.10.3 > 10.10.10.1: ICMP echo reply, id 1166, seq 1, length 64 18:59:33.294940 52:54:00:9b:d5:bd > 52:54:00:9b:d5:bf, ethertype IPv4 (0x0800), length 98: 10.10.10.2 > 10.10.10.4: ICMP echo request, id 1148, seq 1, length 64 18:59:33.295820 52:54:00:9b:d5:bf > 52:54:00:9b:d5:bd, ethertype IPv4 (0x0800), length 98: 10.10.10.4 > 10.10.10.2: ICMP echo reply, id 1148, seq 1, length 64
从上面我们也可以看到LACP包
如果把first_if设成down，则包的走向会变
sudo ip link set first_if down
我们发现second_if开始有流量，京first_if变成down，10.10.10.1和10.10.10.2似乎没有收到影响
$ sudo tcpdump -n -e -i second_if tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on second_if, link-type EN10MB (Ethernet), capture size 65535 bytes 19:14:34.415216 66:63:74:a9:0e:f2 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 19:14:35.411181 76:fa:16:61:d7:0e > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 19:16:01.162066 52:54:00:9b:d5:bc > 52:54:00:9b:d5:be, ethertype IPv4 (0x0800), length 98: 10.10.10.1 > 10.10.10.3: ICMP echo request, id 1167, seq 33, length 64 19:16:01.162298 52:54:00:9b:d5:be > 52:54:00:9b:d5:bc, ethertype IPv4 (0x0800), length 98: 10.10.10.3 > 10.10.10.1: ICMP echo reply, id 1167, seq 33, length 64 19:16:01.781051 52:54:00:9b:d5:bd > 52:54:00:9b:d5:bf, ethertype IPv4 (0x0800), length 98: 10.10.10.2 > 10.10.10.4: ICMP echo request, id 1149, seq 29, length 64 19:16:01.781565 52:54:00:9b:d5:bf > 52:54:00:9b:d5:bd, ethertype IPv4 (0x0800), length 98: 10.10.10.4 > 10.10.10.2: ICMP echo reply, id 1149, seq 29, length 64
second_br和second_if变成active
$ sudo ovs-appctl bond/show ---- bond0 ---- bond_mode: active-backup bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms lacp_status: negotiated
slave first_br: disabled may_enable: false
slave second_br: enabled active slave may_enable: true
---- bond1 ---- bond_mode: active-backup bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms lacp_status: negotiated
slave first_if: disabled may_enable: false
slave second_if: enabled active slave may_enable: true
重启first_if，但是second_br和second_if仍然是active
$ sudo ip link set first_if up
$ sudo ovs-appctl bond/show ---- bond0 ---- bond_mode: active-backup bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms lacp_status: negotiated
slave first_br: enabled may_enable: true
slave second_br: enabled active slave may_enable: true
---- bond1 ---- bond_mode: active-backup bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms lacp_status: negotiated
slave first_if: enabled may_enable: true
slave second_if: enabled active slave may_enable: true
如果我们把bond_mode设为balance
$ sudo ovs-vsctl set Port bond0 bond_mode=balance-slb $ sudo ovs-vsctl set Port bond1 bond_mode=balance-slb
$ sudo ovs-appctl bond/show ---- bond0 ---- bond_mode: balance-slb bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms next rebalance: 3412 ms lacp_status: negotiated
slave first_br: enabled active slave may_enable: true
slave second_br: enabled may_enable: true
---- bond1 ---- bond_mode: balance-slb bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms next rebalance: 7415 ms lacp_status: negotiated
slave first_if: enabled active slave may_enable: true
slave second_if: enabled may_enable: true
这个时候10.10.10.1 ping 10.10.10.3走first_br
$ sudo tcpdump -n -e -i first_if tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on first_if, link-type EN10MB (Ethernet), capture size 65535 bytes 19:57:04.093878 ca:d4:fd:47:a6:ce > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 19:57:04.093992 f2:ac:70:72:49:61 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 19:57:52.559762 52:54:00:9b:d5:bc > 52:54:00:9b:d5:be, ethertype IPv4 (0x0800), length 98: 10.10.10.1 > 10.10.10.3: ICMP echo request, id 1173, seq 22, length 64 19:57:52.560002 52:54:00:9b:d5:be > 52:54:00:9b:d5:bc, ethertype IPv4 (0x0800), length 98: 10.10.10.3 > 10.10.10.1: ICMP echo reply, id 1173, seq 22, length 64 19:57:53.560999 52:54:00:9b:d5:bc > 52:54:00:9b:d5:be, ethertype IPv4 (0x0800), length 98: 10.10.10.1 > 10.10.10.3: ICMP echo request, id 1173, seq 23, length 64 19:57:53.561278 52:54:00:9b:d5:be > 52:54:00:9b:d5:bc, ethertype IPv4 (0x0800), length 98: 10.10.10.3 > 10.10.10.1: ICMP echo reply, id 1173, seq 23, length 64
而10.10.10.2 ping 10.10.10.4走second_br
$ sudo tcpdump -n -e -i second_if tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on second_if, link-type EN10MB (Ethernet), capture size 65535 bytes 19:57:03.719596 76:fa:16:61:d7:0e > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 19:57:03.723928 66:63:74:a9:0e:f2 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110 19:57:49.695881 52:54:00:9b:d5:bf > 52:54:00:9b:d5:bd, ethertype IPv4 (0x0800), length 98: 10.10.10.4 > 10.10.10.2: ICMP echo reply, id 1154, seq 13, length 64 19:57:50.697434 52:54:00:9b:d5:bd > 52:54:00:9b:d5:bf, ethertype IPv4 (0x0800), length 98: 10.10.10.2 > 10.10.10.4: ICMP echo request, id 1154, seq 14, length 64 19:57:50.697581 52:54:00:9b:d5:bf > 52:54:00:9b:d5:bd, ethertype IPv4 (0x0800), length 98: 10.10.10.4 > 10.10.10.2: ICMP echo reply, id 1154, seq 14, length 64 19:57:51.699024 52:54:00:9b:d5:bd > 52:54:00:9b:d5:bf, ethertype IPv4 (0x0800), length 98: 10.10.10.2 > 10.10.10.4: ICMP echo request, id 1154, seq 15, length 64
发表评论