Semantic segmetation nvdspreprocessing+invinfer

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 8.2.3.0
• NVIDIA GPU Driver Version (valid for GPU only) 470.141.03
• Issue Type( questions, new requirements, bugs) Question

Hi,
I want to run the sample deepstream-segmentation-test while injecting a preprocessing plugin into the pipeline.

I first converted the pipeline to CLI command and the segmentation itself run seemingly without issues, I used this CLI command to launch the pipeline: gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvvideoconvert ! nvinfer config-file-path=/../../dstest_segmentation_config_semantic.txt ! nvsegvisual width=1920 height=1080 ! nvmultistreamtiler rows=1 columns=1 ! nvvideoconvert ! x264enc ! mp4mux ! filesink location=seg_output.mp4

However, I get no segmentation results when I inject a preprocessing element with config file adapted from /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt using the CLI command gst-launch-1.0 filesrc location = /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvvideoconvert ! nvdspreprocess config-file= /../../config_preprocess.txt ! nvinfer config-file-path=/../../dstest_segmentation_config_semantic.txt ! nvsegvisual width=1920 height=1080 ! nvmultistreamtiler rows=1 columns=1 ! nvvideoconvert ! x264enc ! mp4mux ! filesink location=seg_pre_output.mp4

I made sure that all nvinfer-side config-file parameters that are disabled in the case of using preprocessing element are compensated in the preprocessing config file.

What I want to know is if there are some missing parameters that may cause faulty input to the network that I didn’t mention here.

Thanks!

3
  1. from you command, please set input-tensor-meta=1, please refer these sample in Gst-nvdspreprocess (Alpha) — DeepStream 6.1.1 Release documentation
  2. please refer to sample deepstream-preprocess-test.
4

Hi,

Thanks for your reply, I actually have input-tensor-meta=1 in the config file of nvinfer, here are my config files for nvdspreprocess and nvinfer respectively. Semantic segmentation is running smoothly if input-tensor-meta=0 when input-tensor-meta=1 and nvdspreprocessing plugin is enabled I get no segmentation results.

I use the custom lib provided by nvdspreprocess.
Please let me know if you need any additional material to reproduce the issue at your side.

config_preprocess.txt (3.2 KB)
dstest_segmentation_config_semantic.txt (3.6 KB)

5
  1. in the attached file , input-tensor-from-meta is 0, do you mean preprocess +nvinfer can work if input-tensor-from-meta is 1?
  2. if yes, nvinfer is opensource , please add logs in nvinfer to debug, please refer to DeepStream SDK FAQ - #9 by mchi, dump_infer_input_to_file.patch.txt is used to dump the image fed to the input layer of the nework, for exmaple, if the dimension and format of your classification is 3:150:150 & RGB, this change dumps a 150x150 RGB viewable image.
6

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.