I found the explanation to my problem in this thread: Host libraries for nvidia-container-runtime - #2 by dusty_nv
JetPack 5.x, only l4t.csv gets used (because CUDA/cuDNN/TensorRT/ect are installed inside the containers on JetPack 5 for portability). Does that answer your question?
This means you need to use l4t-tensorrt instead of l4t-base, or install the cuda and tensorrt deps inside manually.