Crop/rotate RTSP input via NvBufSurface APIs does not work on every frame

I am trying to do the same thing as this link, but with RTSP input instead of CSI camera.

My setup:
Jetson Xavier NX
Jetpack 4.4 DP
Deepstream 5.0.0 DP

I’ve applied the suggested patch to sources\apps\apps-common\src\deepstream_source_bin.c:

} else{
NVGSTDS_LINK_ELEMENT (bin->cap_filter, bin->nvvidconv);
NVGSTDS_LINK_ELEMENT (bin->nvvidconv, bin->cap_filter1);

/// added here, line 888
// also tried to probe source pad of bin->cap_filter and bin->cap_filter1, same result
// probe source pad of bin->src_elem, does not rotate at all
GstPad *src_pad;
src_pad = gst_element_get_static_pad (bin->nvvidconv, "src");  
gst_pad_add_probe (src_pad, GST_PAD_PROBE_TYPE_BUFFER,
    nvargus_src_pad_buffer_probe, NULL, NULL);
///

NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter1, "src");

I’ve also made small changes such that it rotates 90 degrees, rather than 180 degrees as in the previous post.

static GstPadProbeReturn
nvargus_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
    gpointer u_data)
{
    GstBuffer *buf = (GstBuffer *) info->data;
    GstMapInfo outmap = GST_MAP_INFO_INIT;
    gst_buffer_map (buf, &outmap, GST_MAP_WRITE);
    NvBufSurface*  surface = (NvBufSurface *)outmap.data;

    NvBufSurfTransformRect src_rect, dst_rect;
    src_rect.top   = 0;
    src_rect.left  = 0;
    src_rect.width = (guint) surface->surfaceList[0].width;
    src_rect.height= (guint) surface->surfaceList[0].height;

    dst_rect.top   = 0;
    dst_rect.left  = 0;
    dst_rect.width = (guint) surface->surfaceList[0].width;
    dst_rect.height= (guint) surface->surfaceList[0].height;

    NvBufSurface *dst_surface = NULL;
    NvBufSurfaceCreateParams nvbufsurface_create_params;

    nvbufsurface_create_params.gpuId  = surface->gpuId;
    nvbufsurface_create_params.width  = (gint) surface->surfaceList[0].width;
    nvbufsurface_create_params.height = (gint) surface->surfaceList[0].height;
    nvbufsurface_create_params.size = 0;
    nvbufsurface_create_params.colorFormat = surface->surfaceList[0].colorFormat;
    nvbufsurface_create_params.layout = surface->surfaceList[0].layout;
    nvbufsurface_create_params.memType = surface->memType;

    NvBufSurfaceCreate(&dst_surface,1,&nvbufsurface_create_params);

    NvBufSurfTransformParams nvbufsurface_params;
    nvbufsurface_params.src_rect = &src_rect;
    nvbufsurface_params.dst_rect = &dst_rect;
    nvbufsurface_params.transform_flag =  0;
    nvbufsurface_params.transform_filter = NvBufSurfTransformInter_Default;

    NvBufSurfTransformConfigParams transform_config_params;
    NvBufSurfTransform_Error err;

    transform_config_params.compute_mode = NvBufSurfTransformCompute_Default;
    transform_config_params.gpu_id = surface->gpuId;
    transform_config_params.cuda_stream = NULL;
    err = NvBufSurfTransformSetSessionParams (&transform_config_params);
    // copy to dst_surface
    err = NvBufSurfTransform (surface, dst_surface, &nvbufsurface_params);
    // rototate 180 degree to original surface
    nvbufsurface_params.transform_flag =  NVBUFSURF_TRANSFORM_FLIP;
    nvbufsurface_params.transform_flip = NvBufSurfTransform_Rotate90;
    err = NvBufSurfTransform (dst_surface, surface, &nvbufsurface_params);
    NvBufSurfaceDestroy(dst_surface);

    gst_buffer_unmap (buf, &outmap);
    return GST_PAD_PROBE_OK;
}

The rotation does work, but it seems like the same frame is being rotated a few times and pushed downstream. So instead of having a 90 degree rotated stream, I have some frames rotated 90, 180, 270, 360 degrees, in that particular order. See the output video (1.0 MB)

This is my config file

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=0
rows=4
columns=2
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://192.168.2.216:8554/test
#num-sources=8
#drop-frame-interval=2
latency=200
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
#iframeinterval=10
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=testf.mp4
source-id=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=8
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0
## If set to TRUE, system timestamp will be attached as ntp timestamp
## If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached
## attach-sys-ts-as-ntp=1

Once again, any help is greatly appreciated.

Bump for visibility.

We are looking into this issue, and will response you ASAP. Before that, can you upgrade to DeepStreamSDK 5.0.1 version? There are some important bugs fixed with latest version.

I can reproduce the issue. We already have a internal bug for tracking this problem. We will response when there is any progress.

Hi,

Has this problem been solved?
I had the same problem.

DeepStream will provide rotation function with nvvideoconvert in future release.

1 Like