YOLOX-S works with deepstream6, yolox-nano and tiny return segmentation fault

[AndreiAlexTa]

So I try to run deepsort with yolox and a nice nvdsanalytics GUI for counting cars,

 - NVIDIA Jetson AGX Xavier [16GB]
   * Jetpack 4.6 [L4T 32.6.1]
   * NV Power Mode: MODE_30W_ALL - Type: 3
   * jetson_stats.service: active
 - Libraries:
   * CUDA: 10.2.300
   * cuDNN: 8.2.1.32
   * TensorRT: 8.0.1.6
   * Visionworks: 1.6.0.501
   * OpenCV: 4.5.1 compiled CUDA: YES
   * VPI: ii libnvvpi1 1.1.15 arm64 NVIDIA Vision Programming Interface library
   * Vulkan: 1.2.70

Using deepstream 6.0 python SDK.
I have a problem. When I convert yolox-s from pytorch to TensorRT it works fine, but when I try to do the same with yolox-nano or tiny, it does not work and I get a segmentation fault

cesiro@cesiro-desktop:/opt/nvidia/deepstream/deepstream-6.0/sources/deepstream_python_apps/apps/deepstream-nvdsanalytics$ sudo python3 deepstream_nvdsanalytics.py file:///opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.mp4
Creating Pipeline 
 
Creating streamux 
 
Creating source_bin  0  
 
Creating source bin
source-bin-00
Creating Pgie 
 
Creating nvtracker 
 
Creating nvdsanalytics 
 
Creating tiler 
 
Creating nvvidconv 
 
Creating nvosd 
 
Creating transform 
 
Creating EGLSink 

Adding elements to Pipeline 

Linking elements in the Pipeline 

Now playing...
1 :  file:///opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_720p.mp4
Starting pipeline 


Using winsys: x11 
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is ON
[NvMultiObjectTracker] Loading TRT Engine for tracker ReID...
[NvMultiObjectTracker] Loading Complete!
[NvMultiObjectTracker] Initialized
0:00:06.484977658 13732      0x691b120 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1900> [UID = 1]: deserialized trt engine from :/home/cesiro/YOLOX/YOLOX_outputs/yolox_nano/model_trt.engine
INFO: [Implicit Engine Info]: layers num: 2
0   INPUT  kFLOAT input_0         3x416x416       
1   OUTPUT kFLOAT output_0        3549x85         

0:00:06.485452503 13732      0x691b120 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2004> [UID = 1]: Use deserialized engine model: /home/cesiro/YOLOX/YOLOX_outputs/yolox_nano/model_trt.engine
0:00:06.496518077 13732      0x691b120 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:YOLOX-deepstream/config_infer_primary.txt sucessfully
Decodebin child added: source 

Decodebin child added: decodebin0 

Decodebin child added: qtdemux0 

Decodebin child added: multiqueue0 

Decodebin child added: h264parse0 

Decodebin child added: capsfilter0 

Decodebin child added: aacparse0 

Decodebin child added: faad0 

Decodebin child added: nvv4l2decoder0 

Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f7dc5c3a8 (GstCapsFeatures at 0x7ef8002ee0)>
In cb_newpad

gstname= audio/x-raw
Segmentation fault

To reproduce:
Use this as a baseline for yolox-deepstream: GitHub - nanmi/YOLOX-deepstream: deploy yolox algorithm use deepstream

Then convert original yolox models(download weights from original repo) from their repo to tensorRT with torch2trt using the readme process from : GitHub - Megvii-BaseDetection/YOLOX: YOLOX is a high-performance anchor-free YOLO, exceeding yolov3~v5 with MegEngine, ONNX, TensorRT, ncnn, and OpenVINO supported. Documentation: https://yolox.readthedocs.io/

What really bothers me is that yolox-s runs at 1 fps and i cannot run smaller models. The primary detector from the python example apps, runs at 30 FPS.

[mchi]

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi @AndreiAlexTa ,
Could you try to use “gdb” to capture the backtrace of the segment error?

Or, could you share us a package with which I can simply decompress and run to reprodce this issue?