How do we assign MACVLAN networks to RDMA devices

[TonyDim]

Using the operator we create NicClusterPolicy where we have multiple RDMA devices as:

rdmaSharedDevicePlugin:
image: k8s-rdma-shared-dev-plugin
repository: nvcr.io/nvstaging/mellanox
version: network-operator-v25.7.0-rc.1

The config below directly propagates to k8s-rdma-shared-device-plugin configuration.

Replace ‘devices’ with your (RDMA capable) netdevice name.

config: |
{
“configList”: [
{
“resourceName”: “rdma_shared_device_a”,
“rdmaHcaMax”: 63,
“selectors”: {
“vendors”: [“15b3”],
“deviceIDs”: [“101b”]
}

If we want to prepare secondary interfaces with MACVLAN for RDMA ports, how do we do it?

This is an example from the github of MACVLAN, but we assgin the port directly.

Is it possible to assign the resource_name?

apiVersion: mellanox.com/v1alpha1

kind: MacvlanNetwork

metadata:

name: example-macvlannetwork

spec:

networkNamespace: “default”

master: “ens2f0”

mode: “bridge”

mtu: 1500

ipam: |

{

  "type": "whereabouts",

  "datastore": "kubernetes",

  "kubernetes": {

    "kubeconfig": "/etc/cni/net.d/whereabouts.d/whereabouts.kubeconfig"

  },

  "range": "192.168.2.225/24",

  "exclude": \[

   "192.168.2.229/30",

   "192.168.2.236/32"

  \],

  "log_file" : "/var/log/whereabouts.log",

  "log_level" : "info",

  "gateway": "192.168.2.1"

}

[juliaj]

No. It is not possible to assign the resourceName directly in the MacvlanNetwork CRD. The RDMA device plugin (rdma_shared_device_a) and the MACVLAN network (MacvlanNetwork) operate at different layers:

The RDMA device plugin advertises RDMA-capable devices as K8s resources.
The MACVLAN CRD attaches L2/L3 secondary interfaces to Pods using spec.master (the Linux NIC name) with IPAM configuration.

To use both together, you may consider reference the NIC name in the MacvlanNetwork CRD and the RDMA resource name in the Pod specification:

In the MacvlanNetwork CRD, set master: (for example, ens2f0).

In the Pod spec, request the RDMA resource (rdma_shared_device_a).

This way, the Pod will receive both a MACVLAN secondary interface (from the NIC defined in master) and access to the RDMA device (from the device plugin).

Example:

apiVersion: v1
kind: Pod
metadata:
name: rdma-pod
annotations:
k8s.v1.cni.cncf.io/networks: example-macvlannetwork
spec:
containers:

 -name: test-container
  image: rdma-app:latest
  resources:
    limits:
      rdma/rdma_shared_device_a: 1

Deploy MacVLAN Network with RDMA Shared Device - NVIDIA Docs