Nvdspreprocess: is it possible to access original source frame data?

Please provide complete information as applicable to your setup.

• RTX 4070 Laptop dGpu
• DeepStream 6.3
• TensorRT v8503
• NVIDIA GPU Driver Version 550.107.02
• Question Is it possible to access the original frame data from nvdspreprocess, if so, how to do it?
I am trying to create a custom library for nvdspreprocess to customize the TensorPreparation for secondary inference. My goal is to apply instance segmentation masks from primary inference, cut out the detected objects using the masks from the original source frame, so the object is on black background.

Is it even possible to access the original frame data from nvdspreprocess, if so, how to do it?
So far i could not find any way to access the frame, i guess it is replaced by the scaled and padded image for primary inference. After research i found only three ways to achieve my goal:

  • Write a custom deepstream element, which has two sinks, one for nvinfer results and one for original frame which is split after source element using a tee element, this aproach seems to be the most “Deepstream” way.
  • customize nvinfer open source code to only use the scaled down image internally and only pass the original source data forward to next element
  • Increase inference Size to be >= source size, this would increase inference time by a lot since source frame is 3x2462x2062, currently primary inference runs on 3x1280x1280

I think this is answered in Cascaded Instance Segmentation for defect analysis - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums