Issue with vlan pids / vpid

Infrastructure & Networking Cumulus Linux Routing, Switching & Virtualization
1

so we have a config, where we have openstack deployed, as following, where the hosts on bond0.1648 10.16.48.X/24 cannot ping their own gateway or each other. however all other vlans work as expected. i am trying to debug/fix the issue, could anyone with openstack knowledge / a cumulus please enlighten me if you see the mistake?

from what i can tell, not being a cumulus expert, can a

bridge-pvid 1648
bridge-vids 1680 1696 1648 1672(

contain the same untagged/tagged vlan ?

I also noted that
vlan1648 shows address-virtual 44:38:39:FF:00:02 10.16.48.1

where all others show

address-virtual 44:38:39:FF:00:03 10.16.72.1
address-virtual 44:38:39:FF:00:03 10.16.96.1
address-virtual 44:38:39:FF:00:07 10.16.64.1

shouldnt they all be the same MAC ??

best case scenerio from my perspective is
move all pvid to 1

and all address-virtual to mac 44:38:39:FF:00:03

Ive included both the openstack config, and below the cumulus config, any help or insight is welcome and appreciated.

bonds:
    bond0:
        interfaces:
        - enp33s0
        - enp49s0
        macaddress: b8:59:9f:12:21:fc
        mtu: 9000
        parameters:
            down-delay: 0
            lacp-rate: fast
            mii-monitor-interval: 100
            mode: 802.3ad
            transmit-hash-policy: layer3+4
            up-delay: 0
bridges:
    br-mgmt:
        addresses:
        - 10.16.48.101/24
        gateway4: 10.16.48.1
        interfaces:
        - bond0.1648
        macaddress: b8:59:9f:12:21:fc
        mtu: 9000
        nameservers:
            addresses:
            - 10.16.48.10
            - 1.1.1.1
            search:
            - maas
        parameters:
            forward-delay: 15
            stp: false
    br-storage:
        addresses:
        - 10.16.72.101/24
        interfaces:
        - bond0.1672
        macaddress: b8:59:9f:12:21:fc
        mtu: 9000
        nameservers:
            addresses:
            - 10.16.48.10
            - 1.1.1.1
            search:
            - maas
        parameters:
            forward-delay: 15
            stp: false
    br-vlan:
        interfaces:
        - bond0
        macaddress: b8:59:9f:12:21:fc
        mtu: 9000
        parameters:
            forward-delay: 15
            stp: false
    br-vxlan:
        addresses:
        - 10.16.80.101/24
        interfaces:
        - bond0.1680
        macaddress: b8:59:9f:12:21:fc
        mtu: 9000
        nameservers:
            addresses:
            - 10.16.48.10
            - 1.1.1.1
            search:
            - maas
        parameters:
            forward-delay: 15
            stp: false

and cumulus hosts example

interface swp12
mtu 9000

interface openstack-infra
bond-slaves swp12
bridge-pvid 1648
bridge-vids 1680 1696 1648 1672
clag-id 12
mstpctl-bpduguard yes
mstpctl-portadminedge yes
bond-lacp-bypass-allow 1
mtu 9000

2

Hello Scott

Your approach to change the PVID to 1 on those ports will most likelly address your connectivity issues, let me elaborate why:

  • The command “bridge-pvid 1648” under the bond interface sets the cumulus switchport to process DOT1Q untagged a RX/TX packets and assign them to vlan 1648

  • The command "bridge-vids 1680 1696 1648 1672 " under the bond interface sets the cumulus switchport to process DOT1Q and untagged traffic for vlans 1680, 1696, 1648 and 1672. It’s the list of vlans that should be allowed on the port

  • If you change pvid under the port(s) to 1, it will assign untagged packets to vlan 1, and if the other “bridge-vid” command remains unchanged, it will accept and process taggeg packets for vlan1648. You server configuration makes me suspect you are tagging traffic for interface br-mgmt which has bond0.1648 assigned

As for the virtual mac addresses, there is no need to make them all “44:38:39:FF:00:03”. Changing the PVID should suffice. MAC OUI 44:38:39 is reserved by cumulus for Peerlink/Clag and the intend is to avoid having a host duplicate the MAC address

You can find the vlan documentation and confiugraiton guide here:
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-42/Layer-2/Ethernet-Bridging-VLANs/VLAN-aware-Bridge-Mode/

3

Great thanks for the info, and the fast reply, ill give it a shot