I am using- • Hardware Platform Jetson • DeepStream Version 6.2 • JetPack Version 5.1 I am working on a deepstream inference pipeline in python. I am able to run inference on RGB stream with detection model. My pipeline is: source → nvstreammux → nvdspreprocess → nvinfer → nvdsosd → sink. Nvdspreprocess plugin processes the frame for multiple ROIs in single frame and then nvinfer runs inference on it as batch. This is based on apps available in deepstream_python_apps. My requirement is to access the frame’s ROIs that are getting sent for inference in the python code, so as to visualize them. For example, in the pipeline below - gst-launch-1.0 v4l2src ! “video/x-raw, height=1080, width=1920” ! queue ! nvvidconv ! “video/x-raw(memory:NVMM), format=NV12” ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 ! nvvideoconvert ! nvdspreprocess config-file=/opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt input-tensor-meta=1 batch-size=2 ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! autovideoconvert ! autovideosink sync=false we can see that there are two ROIs in the frame and batch size 2 is used in nvinfer. If I convert this pipeline in python, can I access the ROIs? I do not want to visualize the detections or get full frame (like they have done in deepstream-imagedata-multistream), I want to check the images that are being sent to the model. Is there a way to do that?

Yes. As I attached before [image] yuweiw: The data here already separates the channels to R G B. You can use OpenCV to show the 3 channels separately.

[image] gauri26297: I want to check the images that are being sent to the model. If you want to get the image that are being sent to the model, you need to dump the image from our C/C++ open source code. You can refer to the queueInputBatch function in the sources\libs\nvdsinfer\nvdsinfer_context_impl.cpp dir.

I tried changes in queueBatchOutput refering to the following: Inference classifier results differ between ds6.0 and ds6.3 - #7 by fanzh DeepStream SDK FAQ - #8 by ersheng But the images were not getting saved. After some debugging I found that the function queueInputBatch is not called, but queueInputBatchPreprocessed is called instead. So I replicated the same changes from the patch to the later function. I am now getting the following error when I run it: Caught SIGSEGV #0 0x0000ffff8acc3098 in __GI___poll #1 0x0000ffff8adf0b38 in () at /lib/aarch64-linux-gnu/libglib-2.0.so.0 #2 0x0000ffff8adf0ef8 in g_main_loop_run () #3 0x0000ffff8af96e48 in gst_bus_poll () #4 0x0000aaaab3f84980 in event_loop #5 0x0000aaaab3f83868 in main (argc=, argv=) After further debug, I found that the issue is in line “float scale = m_Preprocessor->getScale();” of the patch. I do not have enough knowledge about nvinfer and c++ to solve this. Please help.

Yes. I have used that patch itself to make the changes before enqueue() function.

[image] gauri26297: getScale OK. For DS 7.0, there are no getScale() for the m_Preprocessor class. You may consider using the following parameters m_Scale. Also since you are using the nvdspreprocess, you can dump that from the nvdspreprocess plugin. You can refer to prepare_tensor in the sources\gst-plugins\gst-nvdspreprocess\nvdspreprocess_lib\nvdspreprocess_impl.cpp. Just open the DEBUG_LIB macro according to your needs.

Okay, but I am using: [image] gauri26297: DeepStream Version 6.2

I will check how I can do this from nvdspreprocess. Thank you!

I tried saving the .bin files in nvdspreprocess. I defined DEBUG_LIB as 1, no other change was done. The .bin files that are saved with this are 0 bytes in size. Is there anything else that needs to be done?

Yes. The default buffer should be the GPU buffer. If you want to dump that to a file, you need to change to CPU buffer. You can refer to the code below. #ifdef DEBUG_LIB static int batch_num2 = 0; int debug_size = 4*m_NetworkSize.channels*m_NetworkSize.width*m_NetworkSize.height; char *debug_buffer = (char *)malloc(debug_size); std::ofstream outfile2("impl_out_batch_" + std::to_string(batch_num2) + ".bin"); cudaMemcpy (debug_buffer, outPtr, debug_size, cudaMemcpyDeviceToHost); outfile2.write((char*) debug_buffer, debug_size); outfile2.close(); free(debug_buffer); batch_num2 ++; #endif

Thank you, this worked. Now the .bin files are not empty. But when I tried to read them in python using numpy.fromfile(“path_to_bin_file.bin”) I am getting an array of shape (353280,); whereas when I printed the debug_size, channels, width and height in nvdspreprocess code, I got size = 2826240 channels = 3 width = 640 height = 368 Now how do I reshape the numpy array so as to see the actual image? Edit : 353280 is actually 640 x 368 x 1.5 Edit2 : Also, I have observed that all values in the array are zero

Access ROI from nvdspreprocess in python

Accelerated Computing Intelligent Video Analytics DeepStream SDK

yuweiw June 3, 2024, 7:59am 5

Yes. If you use preprocess plugin, you refer to the queueInputBatchPreprocessed function.
About the patch, did you refer to the 2nd part of the Dump the Inference Input?

Topic		Replies	Views
Using nvdspreprocess for improved small object detection DeepStream SDK deepstream61	7	989	October 8, 2023
Accessing the preprocess ROIs information after inferencing DeepStream SDK gstreamer , python , deepstream61	5	1104	June 21, 2023
Issue while using nvdspreprocess plugin in deepstream 7.0, ROI displayed but not considered for infernece DeepStream SDK deepstream	1	226	July 16, 2024
Nvdspreprocess (ROI) no detection inside ROI DeepStream SDK deepstream	12	466	September 26, 2025
How to get the input of nvinfer in deepstream python app? DeepStream SDK opencv , python	11	1925	June 14, 2022
When to use NvDsPreprocess with multiple ROIs? DeepStream SDK	10	1283	October 3, 2023
No detections on deepstream (nvinfer) with nvdspreprocess plugin add DeepStream SDK deepstream	16	798	October 14, 2024
Extracting Processed Frames After Inference in Nvidia DeepStream Efficiently DeepStream SDK gstreamer , python , deepstream	9	631	February 12, 2025
Clarifications about the nvdspreprocess component DeepStream SDK	1	483	August 4, 2023
The inference works outside of the ROI (set by nvdspreprocess) DeepStream SDK	2	609	November 3, 2023

Access ROI from nvdspreprocess in python

Related topics