Does BlueField-3 DPA support CAS in device code?

Infrastructure & Networking BlueField
1

Hello,
I am working on DPA/FlexIO device-side code on BlueField-3, and I would like to confirm whether CAS / compare-and-exchange is supported on BF3 DPA.
My environment:

  • Hardware: NVIDIA BlueField-3 SoC, MT43244
  • NIC: BlueField-3 integrated ConnectX-7 network controller
  • Firmware version: 32.47.1026
  • DOCA/FlexIO version: libflexio 25.10.3034
  • dpacc version: 2.0.0.73
  • DPA target: -mcpu=nv-dpa-bf3

I tested several C11 standard atomic operations in the DPA device code, and I found that the CAS/compare-swap operation could not be compiled.
My test code:

#include <stdint.h>
#include <stdatomic.h>

__dpa_global__ void dpa_cas_repro(uint64_t out_daddr)
{
    uint64_t *out = (uint64_t *)(uintptr_t)out_daddr;
    atomic_uint c11_value;
    uint32_t c11_expected;
    uint32_t c11_ok;
    uint32_t builtin_value;
    uint32_t builtin_expected;
    uint32_t builtin_ok;

    atomic_init(&c11_value, 0u);
    c11_expected = 0u;
    c11_ok = atomic_compare_exchange_strong_explicit(&c11_value,
                                                     &c11_expected,
                                                     123u,
                                                     memory_order_seq_cst,
                                                     memory_order_seq_cst);

    builtin_value = 0u;
    builtin_expected = 0u;
    builtin_ok = __atomic_compare_exchange_n(&builtin_value,
                                             &builtin_expected,
                                             456u,
                                             0,
                                             __ATOMIC_SEQ_CST,
                                             __ATOMIC_SEQ_CST);

    out[0] = c11_ok;
    out[1] = atomic_load_explicit(&c11_value, memory_order_seq_cst);
    out[2] = builtin_ok;
    out[3] = builtin_value;
}

Build command:

/opt/mellanox/doca/tools/dpacc \
  /home/pengyu/dpa_cas_repro/dpa_cas_repro.c \
  -o /home/pengyu/dpa_cas_repro/dpa_cas_repro.a \
  -hostcc=gcc \
  --devicecc-options="-DE_MODE_LE,-Wall,-Wextra,-Wpedantic,-ffreestanding,-mcmodel=medany,-O2" \
  -mcpu=nv-dpa-bf3 \
  --app-name=dpa_cas_repro_app

Compiler error:

/home/pengyu/dpa_cas_repro/dpa_cas_repro.c:16:11: error: builtin not supported for current target CPU
   16 |  c11_ok = __c11_atomic_compare_exchange_strong(&c11_value,
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...
/home/pengyu/dpa_cas_repro/dpa_cas_repro.c:24:15: error: builtin not supported for current target CPU
   24 |  builtin_ok = __atomic_compare_exchange_n(&builtin_value,
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...
2 errors generated.

Is CAS / compare_exchange unsupported on BlueField-3 DPA device code? If not, is there any DPA-specific intrinsic or DOCA API that provides CAS-like semantics on BF3?

Thanks.

2

1/ Yes — this is expected. The local C11 atomics and the __atomic_compare_exchange* / __sync_*compare_and_swap builtins that lower to LR/SC (load-reserved/store-conditional) sequences are not supported for the BF3/CX7 DPA target, and dpacc intentionally errors them out. So the compile failure you see is by design, not a setup problem on your side.

2/ For CAS-like semantics on BF3, use the hardware/DOCA atomic paths instead of the C builtins:

+ DOCA RDMA Atomic Compare and Swap — an 8-byte atomic read-modify-write on a (remote) memory address, where the value is swapped only if it matches the compared value. Enable it via doca_rdma_task_atomic_cmp_swp_set_conf and check availability with doca_rdma_cap_task_atomic_cmp_swp_is_supported. Note the destination must be 8-byte aligned and only the first 8 bytes are used.

+ FlexIO atomic WQE segment — issue the atomic operation through the NIC via a work-queue element (e.g. flexio_dev_wqe_atomic_seg / the atomic SWQE segment) rather than relying on a CPU LR/SC instruction in device code.

See the DOCA RDMA guide (Atomic Compare and Swap Task) and the FlexIO SDK API reference for the exact prototypes and capability queries:

If you need help mapping this to your specific code path, please open a support case and we can take it from there. If this is part of a larger design (e.g. choosing a synchronization model for your DPA application), your NVIDIA account team can engage the right resources to work through it.