Broken HPC-Benchmarks container (v25.02/v25.04)

Accelerated Computing NGC GPU Cloud Container: HPC
1

Hi,

I would like to report that the two lastest versions of the HPC-Benchmarks containers are shipped with missing nvshmem and a bugged version numactl.

[Missing NVSHMEM]

$ singularity shell --nv hpc-benchmarks_25.04.sif
Singularity> ldd /workspace/hpl-linux-x86_64/xhpl
        linux-vdso.so.1 (0x00007ffde2c3d000)
        nvshmem_bootstrap_pmi2.so.3 => not found
        libcublas.so.12 => /usr/local/cuda/targets/x86_64-linux/lib/libcublas.so.12 (0x00007f3fbe800000)
        libcublasLt.so.12 => /usr/local/cuda/targets/x86_64-linux/lib/libcublasLt.so.12 (0x00007f3f8aa00000)
        libnvidia-ml.so.1 => /.singularity.d/libs/libnvidia-ml.so.1 (0x00007f3f89800000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3fc5af7000)
        libcuda.so.1 => /usr/local/cuda/compat/lib.real/libcuda.so.1 (0x00007f3f85369000)
        libnccl.so.2 => /lib/x86_64-linux-gnu/libnccl.so.2 (0x00007f3f6e568000)
        libnvshmem_host.so.3 => not found
        libmpi.so.40 => /opt/hpcx/ompi/lib/libmpi.so.40 (0x00007f3fc59d4000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3fbe717000)
        libnvToolsExt.so.1 => /usr/local/cuda/targets/x86_64-linux/lib/libnvToolsExt.so.1 (0x00007f3f6e200000)
        libiomp5.so => not found
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3fc59cd000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3fc59c8000)
        libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f3fc5972000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3f6dfee000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3fc5b04000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3fc5942000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3f6dd70000)
        libopen-rte.so.40 => /opt/hpcx/ompi/lib/libopen-rte.so.40 (0x00007f3f6e4aa000)
        libopen-pal.so.40 => /opt/hpcx/ompi/lib/libopen-pal.so.40 (0x00007f3f6dc55000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f3fc5924000)

Instead of shipping nvshmem, the container is shipped with amazon fabric plugin:

In v24.9

Singularity> ls /opt/
hpcx  nvidia  nvshmem

But in v25.2/v25.4

Singularity> ls /opt/
amazon  hpcx  nvidia

[Bugged numactl]

v25.2/v25.4 is based on ubuntu 24.04.1

Singularity> numactl --version
2.0.18

hpl relies on numactl for affinity setting.
But 2.0.18 version has a bug with –membind option.

Please upgrade to 2.0.19

Thanks.