Face Detection post-processing not working in DeepStream 6.2 on Jetson Orin Nano

[cpauyac]

Platform: Jetson Orin Nano Developer Kit
JetPack Version: 5.1.2 (L4T 35.4.1)
DeepStream Version: 6.2
Model: Face detection model (resnet18_facedetectir
Engine: .engine file generated using trtexec (FP16)

Issue:
When running the DeepStream app, the model loads and inference runs, but no bounding boxes are displayed in the output. The pipeline works (no GStreamer or nvinfer crash), but it’s like the post-processing parser is not decoding the outputs correctly.

Questions:

1. Is a custom bounding box parser required for this model?
2. What’s the correct format for the output_bbox/BiasAdd post-processing?
3. Can I use an existing parser from TAO models or must I write one?

Thanks in advance for your help!
config_infer_primary.txt (501 Bytes)
deepstream_app_config.txt (466 Bytes)

[francis.guindon]

Hi @cpauyac,

I’m initially curious to not see any [osd] configuration in your app config file. While this doesn’t address your questions about post processing, have you ruled out that you are not missing any config to draw the bounding boxes?

Regards,
Francis Guindon

Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com

[fanzh]

Is this still an DeepStream issue to support? Thanks!
nvinfer low-level implementaton is opnsource. The default parser function has been specifically written for the sample resnet10 model. deepstream-test1 does not need a custom bbox parser function. the outputs of the test1 model are

1   OUTPUT kFLOAT output_bbox/BiasAdd 16x34x60
2   OUTPUT kFLOAT output_cov/Sigmoid 4x34x60

If “DetectPostprocessor::parseBoundingBox” can process the outputs of your model, pleaase write a custom bbox parser function. please refer to this sample cfg and code.

[cpauyac]

Platform Info:

• Jetson Orin Nano Developer Kit
• JetPack 5.1.2 (L4T R35.4.1)
• DeepStream SDK 6.3 (native install, not container)
• TAO Converter 5.0.0 with TensorRT 8.5.2

Model Info:

• TAO pre-trained model: resnet18_facedetectir.etlt
• Converted to TensorRT .engine using tao-converter on Jetson
• Input dimensions: 3x240x384
• Output layers:
  • output_bbox/BiasAdd: 4x15x24
  • output_cov/Sigmoid: 1x15x24

Custom BBox Parser:

• Implemented based on layer layout from resnet18_facedetectir
• Built and compiled on Jetson using DeepStream headers
• Placed at /usr/lib/aarch64-linux-gnu/libnvdsinfer_custombboxparser_tao.so
• Added to config_infer_primary.txt:
  custom_lib=/usr/lib/aarch64-linux-gnu/libnvdsinfer_custombboxparser_tao.so
  parse_bbox_func=NvDsInferGetParseFunc

include
include
include “nvdsinfer_custom_impl.h”

extern “C” bool
parse_custom_bbox_facedetectir(NvDsInferLayerInfo const layersInfo,*

•                            const uint32_t numLayers,*
  

•                            NvDsInferNetworkInfo const& networkInfo,*
  

•                            NvDsInferParseDetectionParams const& detectionParams,*
  

•                            std::vector<NvDsInferObjectDetectionInfo>& objectList)*
  

{

• int bboxLayerIndex = -1;*

• int confLayerIndex = -1;*

• for (uint32_t i = 0; i < numLayers; ++i) {*

•    if (std::strcmp(layersInfo[i].layerName, "output_bbox/BiasAdd") == 0) {*
  

•        bboxLayerIndex = i;*
  

•    } else if (std::strcmp(layersInfo[i].layerName, "output_cov/Sigmoid") == 0) {*
  

•        confLayerIndex = i;*
  

•    }*
  

• }*

• if (bboxLayerIndex == -1 || confLayerIndex == -1) {*

•    return false;*
  

• }*

• const float* bboxData = static_cast<const float*>(layersInfo[bboxLayerIndex].buffer);*

• const float* confData = static_cast<const float*>(layersInfo[confLayerIndex].buffer);*

• const int gridW = 24;*

• const int gridH = 15;*

• for (int h = 0; h < gridH; ++h) {*

•    for (int w = 0; w < gridW; ++w) {*
  

•        int idx = h * gridW + w;*
  

•        float confidence = confData[idx];*
  

•        if (confidence < detectionParams.perClassPreclusterThreshold[0])*
  

•            continue;*
  

•        NvDsInferObjectDetectionInfo obj;*
  

•        obj.classId = 0;*
  

•        obj.detectionConfidence = confidence;*
  

•        float bx = bboxData[idx * 4 + 0];*
  

•        float by = bboxData[idx * 4 + 1];*
  

•        float bw = bboxData[idx * 4 + 2];*
  

•        float bh = bboxData[idx * 4 + 3];*
  

•        obj.left = bx - bw / 2.0f;*
  

•        obj.top = by - bh / 2.0f;*
  

•        obj.width = bw;*
  

•        obj.height = bh;*
  

•        objectList.push_back(obj);*
  

•    }*
  

• }*

• return true;*
  }

extern “C” NvDsInferParseCustomFunc NvDsInferGetParseFunc()
{

• return (NvDsInferParseCustomFunc)parse_custom_bbox_facedetectir;*
  }

Issue Summary:

• The engine deserializes successfully and the parser loads without error.
• I can see inference is running briefly, but the app crashes within seconds.
• Running with [sink0] type=2 (Fakesink) gives:
  Segmentation fault (core dumped)

ascc@ubuntu:~/facedetectir$ deepstream-app -c deepstream_app_config.txt
0:00:00.141392107 82652 0xaaaaf220f870 WARN nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1174> [UID = 1]: Warning, OpenCV has been deprecated. Using NMS for clustering instead of cv::groupRectangles with topK = 20 and NMS Threshold = 0.5
0:00:03.706303894 82652 0xaaaaf220f870 INFO nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1988> [UID = 1]: deserialized trt engine from :/home/ascc/facedetectir/resnet18_facedetectir_int8.engine
INFO: [Implicit Engine Info]: layers num: 3
*0 INPUT kFLOAT input_1 3x240x384 *
*1 OUTPUT kFLOAT output_bbox/BiasAdd 4x15x24 *
*2 OUTPUT kFLOAT output_cov/Sigmoid 1x15x24 *

0:00:03.913054313 82652 0xaaaaf220f870 INFO nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2091> [UID = 1]: Use deserialized engine model: /home/ascc/facedetectir/resnet18_facedetectir_int8.engine
0:00:03.945849531 82652 0xaaaaf220f870 INFO nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/home/ascc/facedetectir/primary.txt sucessfully

Runtime commands:

• h: Print this help*

• q: Quit*

• p: Pause*

• r: Resume*

*** INFO: <bus_callback:239>: Pipeline ready*

*Opening in BLOCKING MODE *
*NvMMLiteOpen : Block : BlockType = 261 *
NvMMLiteBlockCreate : Block : BlockType = 261 *
*** INFO: <bus_callback:225>: Pipeline running

Segmentation fault (core dumped)

• Running with [sink0] type=4 (UDP) gives:
  create_udpsink_bin failed (sink misconfiguration)

ascc@ubuntu:~/facedetectir$ deepstream-app -c deepstream_app_config.txt
*** ERROR: <create_udpsink_bin:722>: create_udpsink_bin failed*
*** ERROR: <create_sink_bin:831>: create_sink_bin failed*
*** ERROR: <create_processing_instance:956>: create_processing_instance failed*
*** ERROR: <create_pipeline:1576>: create_pipeline failed*
*** ERROR: main:697: Failed to create pipeline*
Quitting
App run failed

Questions:

1. Is my custom bounding box parser correctly handling the output format of resnet18_facedetectir?
2. Are there known stability issues or alignment constraints for this TAO model in DeepStream 6.3?
3. Could the segmentation fault be caused by memory layout, grid indexing, or invalid coordinates in the parser?
4. What additional debugging tools (e.g., GDB, DeepStream debug flags) would you recommend to trace the crash?

deepstream_app_config.txt (784 Bytes)
config_infer_primary.txt (543 Bytes)

[fanzh]

1. it seems the model layers are similar to resnet10. please comment out ‘custom-lib-path’ and ‘parse-bbox-func-name’ to use the default bbox parsing function, then try agian.
2. from “create_udpsink_bin failed”, some elements failed to create. you can run the application with “export GST_DEBUG=6” to get more detailed information. or you can use ‘type=2’(3dsink) first. Please refer to this FAQ for accuray issue.
3. about the crash, you can use ‘gdb’ to debug.

[cpauyac]

Thank you for the suggestions. I’ve followed each of them carefully, but I’m still unable to see any bounding boxes during inference. Here’s what I’ve done so far:

1. Model Parser:
   I commented out both custom-lib-path and parse-bbox-func-name in the [primary-gie] section to use the default bbox parser, as you suggested. The model (ResNet18-based FaceDetectIR) loads successfully and the engine deserializes without errors.
2. Sink Configuration:
   I’m currently using type=2 for [sink0] to test 3dsink (since udpsink caused a crash previously). The pipeline runs and displays the video, but no bounding boxes are drawn.
3. Debugging Attempts:
   I’ve tested with:

• GST_DEBUG="*:2,*nvds*:6"
• Verified my .engine file loads correctly
• Observed warnings such as:
  • pad not activated yet
  • qtdemux: unknown QuickTime node type
  • Failed to probe pixel aspect ratio
• The source .mp4 plays fine in gst-play-1.0 and ffplay
• Re-encoded the input video with ffmpeg to avoid container issues

4. System Info:

• Jetson Orin Nano
• JetPack 5.1.2
• DeepStream 6.3
• Model: resnet18_facedetectir_int8.engine from NGC, generated via tao-converter

At this point, the model loads, the video renders, but no bounding boxes are visible, and no inference outputs appear in the logs.

---

Question:

Given that I’ve tried the default parser and ensured the source video is valid, what else can I check to troubleshoot why inference outputs are not being drawn? Should I consider switching to the older resnet10 model as a sanity test, or is there a known issue with using FaceDetectIR in DeepStream 6.3?

Also, would using a raw deepstream-test1-app with this engine help isolate the issue?

Any further insights are appreciated!

[fanzh]

please refer to this topic for facedetectir nvinfer cfg. As written in the topic, facedetectir is too old. please refer to this ready-made facenet sample.

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.