RTSP stream works on Xavier NX, but is distorted on Orin AGX

Hello everyone,

Jetson Xavier NX
The below code, “RTSP from camera” and “Output to RTSP” along with it’s “Supporting functions”, worked fine on Xavier NX. It was grabbing a RTSP feed from a camera and making it available on another RTSP stream.

Jetson Orin AGX
The same code runs on Orin AGX but the feed that comes through is heavily distorted as per the attached image.
I then tried running the below Python script (which is very similar in content) and the video stream comes through undistorted.

Is this a behavior that anyone recognises?
Thank you in advance.

RTSP from camera:

videoCaptureBackend = cv::CAP_GSTREAMER;
videoSource = "rtspsrc location=" + addressIn + " latency=0 ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! queue ! videoconvert ! video/x-raw, format=BGR ! queue ! appsink";
video_cap = cv::VideoCapture(videoSource, videoCaptureBackend);

Output to RTSP :

gst_init(nullptr, nullptr);
rtsp_loop = g_main_loop_new(nullptr, FALSE);
GstRTSPServer* rtsp_server = gst_rtsp_server_new();
gst_rtsp_server_set_service(rtsp_server, port_number.c_str());
GstRTSPMediaFactory* factory = gst_rtsp_media_factory_new();

std::string launch = "( appsrc name=rtsp_src format=GST_FORMAT_TIME max-latency=0 caps=video/x-raw,format=BGR,width=1920,height=1080 ! videoconvert ! video/x-raw,format=(string)RGBA ! nvvidconv ! nvv4l2h265enc ! video/x-h265 ! rtph265pay name=pay0 pt=96 )";
gst_rtsp_media_factory_set_launch(factory, launch.c_str());

g_signal_connect(factory, "media-configure", (GCallback)rtsp_media_configure, nullptr);
GstRTSPMountPoints* mounts = gst_rtsp_server_get_mount_points(rtsp_server);
gst_rtsp_mount_points_add_factory(mounts, address.substr(address_last_slash_pos).c_str(), factory);

Supporting functions:

void rtsp_need_data(GstElement* appsrc, guint unused, RTSPContext *ctx) {
	GstBuffer* buffer = gst_buffer_new_allocate(NULL, rtsp_data_size, NULL);
	gst_buffer_fill(buffer, 0, rtsp_manager.receive().data, rtsp_data_size);

	GST_BUFFER_PTS(buffer) = ctx->timestamp;
	GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 30);
	ctx->timestamp += GST_BUFFER_DURATION(buffer);

	GstFlowReturn ret; // Not used, but prevent annoying warnings
	g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);
void rtsp_media_configure(GstRTSPMediaFactory* factory, GstRTSPMedia* media, gpointer user_data) {
	GstElement *element = gst_rtsp_media_get_element(media);
	GstElement *appsrc = gst_bin_get_by_name_recurse_up(GST_BIN(element), "rtsp_src");

	g_object_set_data_full(G_OBJECT(media), "my-extra-data", nullptr, (GDestroyNotify)g_free);
	g_signal_connect(appsrc, "need-data", (GCallback)rtsp_need_data, g_new0(RTSPContext, 1));


Python example:

mainloop = GLib.MainLoop()
server = GstRtspServer.RTSPServer()
mounts = server.get_mount_points()
test_factory = GstRtspServer.RTSPMediaFactory()
test_factory.set_launch('rtspsrc location=' + address_in+ ' latency=0 ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc ! rtph264pay name=pay0')
mounts.add_factory("/test", vimbasrc_factory)

Please try the sample and see if it works:
Opencv with rtsp source and backgroundsubtractor in Xavier AGX - #3 by DaneLLL

May try the public source first to confirm it works. And then try your RTSP source.

Using the public big buck bunny RTSP feed that you mentioned in one of the posts:

  • Ran it on Orin AGX through the pipeline mentioned in “RTSP from camera” and “Output to RTSP”.
    It plays, but is heavily distorted.

  • Ran it on Orin AGX through the pipeline mentioned in “Python example”.
    It plays just fine there.

By the way, I can’t use any kind of rendering for troubleshooting this pipeline as it is running in a container with no access to a graphic interface.

I noticed that VLC was reported resolution as 1920x1080 but with a buffer dimension of 1920x1088.

I changed the settings in my app to output 1920x1088 and the distortion disappeared.

In the output pipeline above i specify 1920x1080. Any idea why it only works with 1920 x 1088 set?

We would like to confirm the issue. So you run this on Orin for video decoding:

videoSource = "rtspsrc location=" + addressIn + " latency=0 ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! queue ! videoconvert ! video/x-raw, format=BGR ! queue ! appsink";

The decoded frame data is in 1920x1088. Is this correct?

And please share the release version. Not sure if you use 5.0.2 or 5.1.

The Jetpack version is: 5.0.2.

In “RTSP from camera”:
The pipeline you posted is indeed the one in use.
The data handed from nvv4l2decoder to nvvidconv is 1920x1080.

In “Output to RTSP”:
A VLC player that opens the stream will say that the resolution is 1920x1080, but a buffer size of 1920x1088.

@aron.h is a genuine commentator on this topic and should not have his post restricted

Sorry about that, it seems the system flagged it - we unflagged it.

In addition to above, I’ve found that the distorted picture only shows when OpenCV overlay graphics are enabled on the 1080 RTSP output. If I either set the output to 1088 or alternatively disable the graphics overlay then the video is distortion free.

Please upgrade to latest Jetpack 5.1 and give it a try.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.