Cumulus switchd and /32 routes in software


It seems that adding /32 routes in Cumulus / switchd causes routing to happen in software.

We can add a /24 route, everything is fast and seems to be in hardware. Then we test with a /32 route to the same IP and things slow down to 4Mbps.

Where can I read about when /32 routes trigger software routing? I believe the 4Mbps we are seeing is related to L3 COPP limits.

Testing with SN2010, Cumulus 5.3.

If I do the same on Linux / SwitchDev (not switchd) it seems to work at hardware speeds.

We should get a case opened to track this. This should not be the case.

Outside of bandwidth testing, did you confirm that the packets are actually software switched by doing a tcpdump on the switch to see if the CPU sees the packets?

Also, are you using any other overlays? (ie. VXLAN, EVPN, etc) Or is this just a simple L3 static route?

I can see the packets with tcpdump, so fairly sure routing is happening in software.

Also if I change the value of copp.l3_local.rate in /etc/cumulus/control-plane/policers.conf it the download speed changes, so I think it’s triggering a COPP rule.

Doing very basic L3 routing, still in a ‘lab’ style setup. Have tried with vlan aware and traditional bridges. As soon as we add a /32 route things break.

Can you share some configuration on what you’re doing? As @rdarbha said though, this would definitely be a support case.

Hi, the config is trivial, SN2010, cumulus 5.3

Set up /24 network, PC can route via switch and get full hardware speeds.

Add a /32 route to the PC (same IP), speed drops to 4Mbps and things happen in software.

I think we’ll just use Debian + SwitchDev, that’s working as expected.

Sure! Everything is fine. :-)