Segmentation Fault with Singularity Container on Multi-GPU System

I am running a training using Singularity containers on a multi-GPU setup with 4 A6000 GPUs installed. When I set CUDA_VISIBLE_DEVICES to use only one GPU according to Create camera sensor fail on buffer , I encounter a segmentation fault (core dumped) error in gym.create_env. What could be causing this issue?

Thank you for your assistance.