I had system where GDS used to work , now running gdscheck after updating kernel it is showing NVME not supported
6.17.0-19-generic
nvidia-fs loaded - 1.17.0.44
Driver Version: 590.48.01
CUDA Version: 13.1
NVIDIA H200 NVL X2
e3:00.0 3D controller: NVIDIA Corporation Device 233b (rev a1)
e4:00.0 3D controller: NVIDIA Corporation Device 233b (rev a1)
nvidia driver loaded with NVreg_RegistryDwords="RMForceStaticBar1=1;ForceP2P=0;RmForceDisableIomapWC=1;"
cat /proc/driver/nvidia/params | grep -i static
RegistryDwords: "RMForceStaticBar1=1;ForceP2P=0;RmForceDisableIomapWC=1;"
p2p dma support in kernel
cat /proc/kallsyms | grep -i p2pdma_pgmap_ops
0000000000000000 d p2pdma_pgmap_ops
-TNRT2:~/alok/gds-nvidia-fs/src$ sudo lspci -s e3:00.0 -vvvvv | grep "BAR 1"
BAR 1: current size: 8GB, supported: 64MB 128MB 256MB 512MB 1GB 2GB 4GB 8GB 16GB 32GB 64GB 128GB 256GB 256TB 512TB 1PB 2PB 4PB 8PB 16PB 32PB 64PB 128PB 256PB 512PB 1EB 2EB 4EB 8EB
TNRT2:~/alok/gds-nvidia-fs/src$ /usr/local/cuda/gds/tools/gdscheck -p
GDS release version: 1.11.1.6
nvidia_fs version: 2.28 libcufile version: 2.12
Platform: x86_64
============
ENVIRONMENT:
============
=====================
DRIVER CONFIGURATION:
=====================
NVMe : Unsupported
NVMeOF : Unsupported
SCSI : Unsupported
ScaleFlux CSD : Unsupported
NVMesh : Unsupported
DDN EXAScaler : Unsupported
IBM Spectrum Scale : Unsupported
NFS : Unsupported
BeeGFS : Unsupported
WekaFS : Unsupported
Userspace RDMA : Unsupported
--Mellanox PeerDirect : Disabled
--rdma library : Not Loaded (libcufile_rdma.so)
--rdma devices : Not configured
--rdma_device_status : Up: 0 Down: 0
=====================
CUFILE CONFIGURATION:
=====================
properties.use_compat_mode : false
properties.force_compat_mode : false
properties.gds_rdma_write_support : true
properties.use_poll_mode : false
properties.poll_mode_max_size_kb : 4
properties.max_batch_io_size : 128
properties.max_batch_io_timeout_msecs : 5
properties.max_direct_io_size_kb : 16384
properties.max_device_cache_size_kb : 131072
properties.max_device_pinned_mem_size_kb : 33554432
properties.posix_pool_slab_size_kb : 4 1024 16384
properties.posix_pool_slab_count : 128 64 64
properties.rdma_peer_affinity_policy : RoundRobin
properties.rdma_dynamic_routing : 0
fs.generic.posix_unaligned_writes : false
fs.lustre.posix_gds_min_kb: 0
fs.beegfs.posix_gds_min_kb: 0
fs.weka.rdma_write_support: false
fs.gpfs.gds_write_support: false
profile.nvtx : false
profile.cufile_stats : 0
miscellaneous.api_check_aggressive : false
execution.max_io_threads : 4
execution.max_io_queue_depth : 128
execution.parallel_io : true
execution.min_io_threshold_size_kb : 8192
execution.max_request_parallelism : 4
properties.force_odirect_mode : true
properties.prefer_iouring : false
=========
GPU INFO:
=========
GPU index 0 NVIDIA H200 NVL: Model Not Supported
GPU index 1 NVIDIA H200 NVL: Model Not Supported
==============
PLATFORM INFO:
==============
Found ACS enabled for switch 0000:e2:00.0
Found ACS enabled for switch 0000:e0:01.1
Found ACS enabled for switch 0000:e2:01.0
Found ACS enabled for switch 0000:e0:01.1
IOMMU: disabled
Nvidia Driver Info Status: Supported(Nvidia Open Driver Installed)
Cuda Driver Version Installed: 13010
Platform: AS -5126GS-TNRT2, Arch: x86_64(Linux 6.17.0-19-generic)
Platform verification succeeded