Nvvideoconvert src-crop & dest-crop question

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
I’m testing this on a Jetson TX2 w/ Jetpack 4.6.2 but I think this is a basic Deepstream nvvideoconvert question. No errors - I just can’t figure out the correct configuration.

• DeepStream Version Jetpack 4.6.2
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I’m working on a TX2 Jetpack 4.6.2 (fresh flash) but I think this is just a nvvideoconvert src-crop and dest-crop question

I am trying to preprocess my 1920x1080 video stream for a Detectnet model with input of 1248x384 (with no distortion). I want the maximum image from my video feed.

To get the maximum view from my camera, I want a 1920x591 (3.25 aspect ratio) crop and let’s assume I want the middle, so crop 244 off the top, 245 off the bottom and take the full width. But I need to get the 1920x591 (correct aspect ratio, wrong size) down to the right size, 1248x384.

nvvideoconvert appears to do what I want but it’s puzzling me. Here is a test source stream similar to my v4l2 image/jpeg stream

Here are some test streams but I can’t figure out the pattern:

No crops - full 1920x1080

gst-launch-1.0 -v videotestsrc pattern=21 ! video/x-raw,width=1920, height=1080, framerate=30/1 ! nvvideoconvert ! nvegltransform ! “video/x-raw(memory:EGLImage),format=RGBA” ! nveglglessink sync=false

src-crop

gst-launch-1.0 -v videotestsrc pattern=21 ! video/x-raw,width=1920, height=1080, framerate=30/1 ! nvvideoconvert src-crop=0:244:1920:591 ! “video/x-raw(memory:NVMM),format=I420” ! nvegltransform ! “video/x-raw(memory:EGLImage),format=RGBA” ! nveglglessink sync=false

results:

  • output size = 1920x1080
  • correct content but distorted to fit aspect ratio: 1.778

src-crop & an output size caps filter

gst-launch-1.0 -v videotestsrc pattern=21 ! video/x-raw,width=1920, height=1080, framerate=30/1 ! nvvideoconvert src-crop=0:244:1920:591 ! “video/x-raw(memory:NVMM),width=1248,height=384,format=I420” ! nvegltransform ! “video/x-raw(memory:EGLImage),format=RGBA” ! nveglglessink sync=false

results:

  • output size is correct
  • correct content - but it is squished towards center

src-crop & dest-crop - no output size caps filter

gst-launch-1.0 -v videotestsrc pattern=21 ! video/x-raw,width=1920, height=1080, framerate=30/1 ! nvvideoconvert src-crop=0:244:1920:591 dest-crop=0:0:1248:384 ! “video/x-raw(memory:NVMM),format=I420” ! nvegltransform ! “video/x-raw(memory:EGLImage),format=RGBA” ! nveglglessink sync=false

results:

  • output size 1920x1080
  • correct contents & correct aspect ratio

src-crop & dest-crop + output size caps filter

gst-launch-1.0 -v videotestsrc pattern=21 ! video/x-raw,width=1920, height=1080, framerate=30/1 ! nvvideoconvert src-crop=0:244:1920:591 dest-crop=0:0:1248:384 ! “video/x-raw(memory:NVMM),width=1248,height=384,format=I420” ! nvegltransform ! “video/x-raw(memory:EGLImage),format=RGBA” ! nveglglessink sync=false

results:

  • output size correct = 1248x384
  • content correct but distorted (squished - looks like 1.778 aspect ratio)

It’s obvious at this point I’m just guessing. I don’t understand src-crop and dest-crop

3

please find the explanation of src-crop and dest-crop in this link.

4

This pipeline works - but it relies on GStreamer, not Deepstream.

gst-launch-1.0 -v videotestsrc pattern=21 ! video/x-raw,width=1920, height=1080, framerate=30/1 ! videocrop top=244 bottom=245 ! videoscale add-borders=false ! nvvideoconvert ! "video/x-raw(memory:NVMM),width=1248,height=384,format=I420" ! nveglglessink sync=false

just my opinion but this is more straight forward to understand than the Deepstream plug-in. I think the main issue is I don’t know how to instruct the Deepstream nvvideoconvert plugin to scale with no border. Seems that add-border=true is the default and there is no way to get rid of that. This pipeline is simpler - no crops, just a size/aspect ratio change that I think highlights the problem:

gst-launch-1.0 -v videotestsrc pattern=21 ! video/x-raw,width=1920, height=1080, framerate=30/1 ! nvvideoconvert ! "video/x-raw(memory:NVMM),width=1248,height=384,format=I420" ! nveglglessink sync=false

note - for someone reading the details. My earlier pipeline was on a TX2 and the nvegltransform was needed. I switched over to AMD64 & Deepstream 6.2 and nvegltransform is not available nor is it needed.

While the first pipeline works, it is using too much of my CPU - the primary incentive to use Deepstream plugins.

5

I did not see any border in the second pipeline, could you provide a commandline to reproduce the issue? thanks!

6

Sorry, I meant that logically. I should have said the nvvideoconvert doesn’t have an add-border=true/false switch like the videoscale and it defaults rescaling the frame and maintaining the image aspect ratio by adding borders. Please tell me if I’m wrong in my observations.

For me this is an important use case because I’m trying to mimic the behaviour of the nvdspreprocess in my camera stream. nvdspreproces is still in alpha - it does what I need it to do but has other issues. I want to pull ROIs out of the stream with dimensions that match my detection model (e.g. detectnet is 1248x384) and I want to do that without distorting the image.

7

yes, there is no border parameter for nvvideoconvert.

nvpreprocess plugin is opensource. this plugin supports scaling with padding by “maintain-aspect-ratio” of “symmetric-padding” paramters. please find the explanation in this link.

8

conclusions

again the goal was taking a v4l2 src (mjpeg output), pulling out an ROI that will be compatible with Detectnet v2 (1248x384)

this pipeline works on Deepstream 6.2 (couple of adjustments required for TX2). This pipeline will display the a 1920x591 ROI reduced to the model’s 1248x384. CPU resource requirements looked more reasonable than previous pipelines.

gst-launch-1.0 -v v4l2src device=/dev/video2 ! image/jpeg, width=1920, height=1080, framerate=30/1, format=MJPG ! jpegdec ! video/x-raw,width=1920,height=1080,format=I420 ! videocrop top=244 bottom=245 ! videoscale ! video/x-raw, width=1248,height=384 ! videorate ! video/x-raw,width=1248,height=384,framerate=4/1 ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=I420" ! nveglglessink

this pipeline will stream via RTSP

./test-launch "v4l2src device=/dev/video2 ! image/jpeg, width=1920, height=1080, framerate=30/1, format=MJPG ! jpegdec ! video/x-raw,width=1920,height=1080,foramt=I420 ! videocrop top=244 bottom=245 ! videoscale ! video/x-raw, width=1248,height=384 ! videorate ! video/x-raw,width=1248,height=384,framerate=4/1 ! nvvideoconvert ! queue ! nvv4l2h264enc profile=4 ! video/x-h264,format=byte-stream ! rtph264pay name=pay0 pt=96"

you can close this ticket

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.