On BF3 sku 900-9D3B6-00CV-AA0, lspci show crypto disabled, but this sku should have crypto enabled? (confirmed BF3 SKU does have crypto)

Infrastructure & Networking BlueField
1

the lspci on this BF3 shows crypto disabled, even though the website https://docs.nvidia.com/networking/display/BlueField3DPU shows this SKU should have crypto. If so, how to turn on crypto on this board. The mlxfwmanager does show crypto enabled. So it is looking like BF3 does have crypto enabled, even thou 00:00.0 RSHIM over PCIE says it does not. Got confirmation that this SKU does have crypto.

ubuntu@localhost:~$ lspci | grep Blue
00:00.0 PCI bridge: Mellanox Technologies MT43244 BlueField-3 SoC Crypto disabled (rev 01)
01:00.0 PCI bridge: Mellanox Technologies MT43244 Family [BlueField-3 SoC PCIe Bridge] (rev 01)
02:00.0 PCI bridge: Mellanox Technologies MT43244 Family [BlueField-3 SoC PCIe Bridge] (rev 01)
02:03.0 PCI bridge: Mellanox Technologies MT43244 Family [BlueField-3 SoC PCIe Bridge] (rev 01)
03:00.0 Ethernet controller: Mellanox Technologies MT43244 BlueField-3 integrated ConnectX-7 network controller (rev 01)
03:00.1 Ethernet controller: Mellanox Technologies MT43244 BlueField-3 integrated ConnectX-7 network controller (rev 01)

ubuntu@localhost:~$ sudo bfcfg -d
icm: LARGE_ICM_SIZE=0x200
mfg: MFG_OOB_MAC=…
mfg: MFG_OPN=900-9D3B6-00CV-AA0
mfg: MFG_SKU=900-9D3B6-00CV-AA0
mfg: MFG_MODL=D3B6
mfg: MFG_SN=…
mfg: MFG_UUID=…
mfg: MFG_REV=A8
sys: ENABLE_SMMU=1
sys: DISABLE_SPMI=0
sys: ENABLE_2ND_EMMC=0
sys: BOOT_PROTECT=0
sys: ENABLE_SPCR=0
sys: DISABLE_PCIE=0
sys: ENABLE_OPTEE=0
sys: ENABLE_I2C0=0
sys: DISABLE_FORCE_PXE_RETRY=0
sys: ENABLE_BMC_FIELD_MODE=0
misc: NET_RSHIM_MAC=…
misc: PXE_DHCP_CLASS_ID=NVIDIA/BF/PXE
ubuntu@localhost:~$ sudo bfver
–/dev/mmcblk0boot0
BlueField ATF version: v2.2(release):4.2.0-80-g0ad5a0e
BlueField UEFI version: 4.2.0-56-ge28fcb7
BlueField BSP version: 4.2.0.12855

OS Release Version: DOCA_2.2.0_BSP_4.2.0_Ubuntu_22.04-2.23-07.prod

ubuntu@localhost:~$ sudo bfb-info --dump
Versions:
ATF: v2.2(release):4.2.0-80-g0ad5a0e
UEFI: 4.2.0-56-ge28fcb7
BSP: mlxbf-bootimages 4.2.0-12855
Kernel: 5.15.0-1021-bluefield
DOCA Base (OFED): 23.07-0.5.0.0
MFT: 4.25.0-62
DOCA: 2.2.0080-1

Firmware:
BF2 FW: 24.38.1002
BF3 FW: 32.38.1002

mlnx-dpdk: ‘MLNX_DPDK 22.11.2307.2.0’
mlx-regex 1.2-ubuntu1
virtio-net-controller 1.6.14-1
collectx-clxapi 1.13.2
libvma 9.8.31-1
libxlio 3.10.5-1.2307050
dpcp 1.1.40-1.2307050
dpkg-query: no packages found matching mlnx-snap

SNAP3:

  • mlnx-libsnap 1.5.4-7
  • spdk 23.01-11

DOCA:

  • doca-apps 2.2.0080-1
  • doca-apps-dev 2.2.0080-1
  • doca-grpc 2.2.0080-1
  • doca-grpc-dev 2.2.0080-1
  • doca-libs 2.2.0080-1
  • doca-prime-runtime 2.2.0080-1
  • doca-prime-sdk 2.2.0080-1
  • doca-prime-tools 2.2.0080-1
  • doca-runtime 2.2.0080-1.23.07.0.5.0.0.bf.4.2.0.12855.2.23
  • doca-runtime-kernel 2.2.0080-1.23.07.0.5.0.0.bf.4.2.0.12855.2.23
  • doca-runtime-user 2.2.0080-1.23.07.0.5.0.0.bf.4.2.0.12855.2.23
  • doca-samples 2.2.0080-1
  • doca-sdk 2.2.0080-1.23.07.0.5.0.0.bf.4.2.0.12855.2.23
  • doca-sdk-kernel 2.2.0080-1.23.07.0.5.0.0.bf.4.2.0.12855.2.23
  • doca-sdk-user 2.2.0080-1.23.07.0.5.0.0.bf.4.2.0.12855.2.23
  • doca-services 2.2.0080-1
  • doca-tools 2.2.0080-1.23.07.0.5.0.0.bf.4.2.0.12855.2.23
  • dpa-compiler 1.5.0
  • dpacc 1.5.0
  • flexio 23.7.1694
  • libdoca-libs-dev 2.2.0080-1
  • librxpcompiler-dev 23.07.1
  • rxp-compiler 23.07.1

OFED:
dpcp 1.1.40-1.2307050
ibacm 2307mlnx47-1.2307050
ibutils2 2.1.1-0.1.MLNX20230719.gbbfde94d.2307050
ibverbs-providers:arm64 2307mlnx47-1.2307050
ibverbs-utils 2307mlnx47-1.2307050
infiniband-diags 2307mlnx47-1.2307050
libibmad-dev:arm64 2307mlnx47-1.2307050
libibmad5:arm64 2307mlnx47-1.2307050
libibnetdisc5:arm64 2307mlnx47-1.2307050
libibumad-dev:arm64 2307mlnx47-1.2307050
libibumad3:arm64 2307mlnx47-1.2307050
libibverbs-dev:arm64 2307mlnx47-1.2307050
libibverbs1:arm64 2307mlnx47-1.2307050
libopensm 5.16.0.MLNX20230719.c143fc96-0.1.2307050
libopensm-devel 5.16.0.MLNX20230719.c143fc96-0.1.2307050
libopenvswitch:arm64 2.17.8-1
librdmacm-dev:arm64 2307mlnx47-1.2307050
librdmacm1:arm64 2307mlnx47-1.2307050
libvma 9.8.31-1
libvma-dev 9.8.31-1
libvma-utils 9.8.31-1
libxlio 3.10.5-1.2307050
libxlio-dev 3.10.5-1.2307050
libxlio-utils 3.10.5-1.2307050
mlnx-dpdk 22.11.0-2307.2.0.23070500.2307.2.0
mlnx-dpdk-dev:arm64 22.11.0-2307.2.0.23070500.2307.2.0
mlnx-ethtool 6.3-1.2307050
mlnx-iproute2 6.3.0-1.2307050
mlnx-ofed-kernel-utils 23.07-OFED.23.07.0.5.0.1.bf.kver.5.15.0-1021-bluefield
mlnx-tools 23.07-0.2307050
mstflint 4.16.1-2.2307050
opensm 5.16.0.MLNX20230719.c143fc96-0.1.2307050
openvswitch-common 2.17.8-1
openvswitch-ipsec 2.17.8-1
openvswitch-switch 2.17.8-1
perftest 23.07.0-0.25.g149fbd6.2307050
python3-openvswitch 2.17.8-1
python3-pyverbs:arm64 2307mlnx47-1.2307050
rdma-core 2307mlnx47-1.2307050
rdmacm-utils 2307mlnx47-1.2307050
srptools 2307mlnx47-1.2307050
ucx 1.15.0-1.2307050

root@localhost:/home/ubuntu# mlxfwmanager
Querying Mellanox devices firmware …

Device #1:

Device Type: BlueField3
Part Number: 900-9D3B6-00CV-A_Ax
Description: NVIDIA BlueField-3 B3220 P-Series FHHL DPU; 200GbE (default mode) / NDR200 IB; Dual-port QSFP112; PCIe Gen5.0 x16 with x16 PCIe extension option; 16 Arm cores; 32GB on-board DDR; integrated BMC; Crypto Enabled
PSID: MT_0000000884
PCI Device Name: /dev/mst/mt41692_pciconf0
Base MAC: 946daec76ec4
Versions: Current Available
FW 32.38.1002 N/A
PXE 3.7.0201 N/A
UEFI 14.31.0020 N/A
UEFI Virtio blk 22.4.0010 N/A
UEFI Virtio net 21.4.0010 N/A

Status: No matching image found

2

Hello @thomas.li2,

Thank you for posting your query on our community. We would like to request you to open a support ticket with us by emailing " Networking-support@nvidia.com " for further investigation of the issue. Please also provide us with a snapshot from your server which can be collected using the tool and instruction found at - GitHub - Mellanox/linux-sysinfo-snapshot: Linux Sysinfo Snapshot 2

Please note that an active support contract would be required for the same. If you do not have a current support contract, please reach out to our Contracts team at networking-contracts@nvidia.com.

Thanks,
Bhargavi

3

Thanks for responding, here is the sysinfo for the prototype setup I am testing with. It has one bf2 8G, one bf3 card.

Here are the data point I am trying to get in an evaluation we are trying.

  1. Transmit and receive throughput data with pktgen running on the arm processor of BF3.
  2. Transmit and receive throughput data with pktgen from host to bf3 with ovr offload (tc flow)

I ran the experiment on BF2 8G and was able to get the throughput data. Will need to do the same for BF3.

Asking my colleagues to see if they have current support contract that I can use as well. The contract my colleagues have are for their specific project. Will try to email the networking-contracts@nvidia.com.
linux-sysinfo-snapshot-3.7.6.tar.gz (56.1 KB)

4

Thanks for confirmation from correspondence, this sku was confirmed to have crypto enabled even though the lspci showed a different info. This issue is resolved.