NaN output specifically while running inference on Jetson

Hi,

I’m currently running inference on this repo edgeai-yolox/README_6d_pose.md at main · TexasInstruments/edgeai-yolox · GitHub using the given demo script with the given pretrained weight. Inference runs well on my local PC, but the output of the backbone right at the stem is giving out NaNs only columns. Upon further inspections, I’m getting NaNs only columns at the 28th, 60th, 92nd … indices, which I suspect is a kernel-related problem, since CUDA utilizes warps of 32 threads and I’m getting NaNs at every 28th column of a warp.

Here’s the launch.json of the demo’s config, inferring on the ycb-v dataset:
” {

        "name": "y6d_demo",

“type”: “debugpy”,

“request”: “launch”,

“module”: “yolox.tools.demo”,

“cwd”: “/workspace/yolox6d/”,

“args”: [

“image”,

“–path”,

“/workspace/data/test/000048/rgb/000001.png”,

“-n”,

“yolox-s-object-pose”,

“–save_result”,

“-c”,

“/workspace/weights/y6d_s_pytorch/best_ckpt.pth”

        \],

“console”: “integratedTerminal”,

“python”: “/bin/python3”

    }”

Hi,

Have you checked with the repo owner about this issue?
Thanks.

Hi, since the repo has been put on-read for some time I have not been able to contact them.