How to transcode H264 rtp packet to H265 rtp packet

Can I take a H264 stream, and output it via rtsp as H265? My goal is to use the more efficient H265 compression so I can have high quality video even at lower bitrates.

I send a stream from udpsrc to an rtspserver. If the stream is encoded as H265, I use this pipeline

udpsrc buffer-size=1310702 port=10000 ! application/x-rtp,media=video,encoding-name=H265,payload=96 ! queue ! rtph265depay ! rtph265pay name=pay0 pt=96

If the stream is encoded as H264, I use this pipeline

udpsrc buffer-size=1310702 port=10000 ! application/x-rtp,media=video,encoding-name=H264,payload=96 ! queue ! rtph264depay ! rtph264pay name=pay0 pt=96

I made this pipeline after referring to another post: h264 to h265 transcode

udpsrc buffer-size=1310702 port=10000 ! application/x-rtp,media=video,encoding-name=H264,payload=96 ! queue ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! nvv4l2h265enc ! rtph265pay name=pay0 pt=96

However when I try to connect to the rtspserver and stream, I get the error gst_mini_object_unref. Could someone please tell me where my pipeline is wrong?

Hi,
Please try to launch RTSP server through test-launch. There are steps in Jetson Nano FAQ
Please take a look and give it a try.

Hi Dane,
Thank you for the suggestions. I tried test-launch with this pipeline:

./test-launch “udpsrc buffer-size=1800000 port=10106 ! application/x-rtp,media=video,encoding-name=H264,payload=96,interlace-mode=progressive ! queue ! rtph264depay ! h264parse ! nvv4l2decoder disable-dpb=true enable-max-performance=1 ! nvvidconv ! nvv4l2h265enc ! video/x-h265,stream-format=byte-stream,interlace-mode=progressive ! rtph265pay name=pay0 pt=96”

With GST_DEBUG=3 it showed me that the h264parse element were dropping frames because of a 'broken/invalid nal ’ error. I also saw warnings and a message that the pipeline failed to preroll. :

Opening in BLOCKING MODE
0:00:06.071679584 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f980994f0 Failed to determine interlace mode
0:00:06.071773696 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f980994f0 Failed to determine interlace mode
0:00:06.071814400 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f980994f0 Failed to determine interlace mode
0:00:06.071966304 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2h265enc0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE
0:00:06.074335968 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:06.074381504 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f9808d5e0 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
0:00:06.116150240 16214 0x7f90031230 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 13147 will be dropped
0:00:06.153737472 16214 0x7f90031230 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 16353 will be dropped
0:00:06.182337280 16214 0x7f90031230 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 14529 will be dropped
0:00:06.214613504 16214 0x7f90031230 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 12681 will be dropped

(test-launch:16214): GStreamer-CRITICAL **: 09:35:08.547: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed
0:00:26.062516000 16214 0x557c3e9990 WARN rtspmedia rtsp-media.c:2991:wait_preroll: failed to preroll pipeline
0:00:26.062918176 16214 0x557c3e9990 WARN rtspmedia rtsp-media.c:3295:gst_rtsp_media_prepare: failed to preroll pipeline

I tried dropping the h264parse element, but the warnings about the decoder and encoder remain. as well as the warning of failed to preroll. As you can see from my pipeline, I tried to be explicit with the interlace_mode, but the warning remains. Any ideas?

Hi,
Please confirm the pipeline works in gst-launch-1.0:

$ gst-launch-1.0 udpsrc buffer-size=1800000 port=10106 ! application/x-rtp,media=video,encoding-name=H264,payload=96,interlace-mode=progressive ! queue ! rtph264depay ! h264parse ! nvv4l2decoder disable-dpb=true enable-max-performance=1 ! nvvidconv ! nvv4l2h265enc ! video/x-h265,stream-format=byte-stream,interlace-mode=progressive ! rtph265pay name=pay0 pt=96 ! fakesink

Hi Dane,

I tried the pipeline using gst-launch-1.0 and it returns the same warnings as the test-launch does. I guess this means the pipeline is wrong. Unfortunately I do not have enough gstreamer experience to identify where the error is. Is there anyone on your who would know where I went wrong?

gst-launch-1.0 udpsrc buffer-size=1800000 port=10106 ! application/x-rtp,media=video,encoding-name=H264,payload=96,interlace-mode=progressive ! queue ! rtph264depay ! h264parse ! nvv4l2decoder disable-dpb=true enable-max-performance=1 ! nvvidconv ! nvv4l2h265enc ! video/x-h265,stream-format=byte-stream,interlace-mode=progressive ! rtph265pay name=pay0 pt=96 ! fakesink
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
0:00:00.088091168 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55881a4880 Failed to determine interlace mode
0:00:00.088165280 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55881a4880 Failed to determine interlace mode
0:00:00.088205312 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55881a4880 Failed to determine interlace mode
0:00:00.088331744 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2h265enc0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE
0:00:00.090015648 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.090072576 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x5588195f20 Failed to determine interlace mode
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
0:00:00.128348416 10376 0x55881afb20 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 13150 will be dropped
0:00:00.157965984 10376 0x55881afb20 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 13311 will be dropped
0:00:00.200131360 10376 0x55881afb20 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 13115 will be dropped

(gst-launch-1.0:10376): GStreamer-CRITICAL **: 22:40:32.765: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed

Hi,
Please check if you can run this command and see video preview:

gst-launch-1.0 udpsrc buffer-size=1800000 port=10106 ! application/x-rtp,media=video,encoding-name=H264,payload=96,interlace-mode=progressive ! queue ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink

Seems like the stream is invalid. Would like to make sure hardware decoder can decoder it correctly. You should see preview if the stream is well decoded.

Hi Dane,

To remove my camera feed from the possible list of suspects, I am currently using this command line to send a videotestsrc to port 10106

gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080 ! nvvidconv ! nvv4l2h264enc insert-vui=1 ! rtph264pay config-interval=1 ! udpsink port=10106

I tried running the command you suggested, and I receive the same error as before.

gst-launch-1.0 udpsrc buffer-size=1800000 port=10106 ! application/x-rtp,media=video,encoding-name=H264,payload=96,interlace-mode=progressive ! queue ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink

I did notice that if I replace the nvv4l2decoder element with avdec_h264 element in the command you gave me, I am able to see the video preview. Is it possible that the nvv4l2decoder element has an issue? I am currently running jetpack 4.4, Ubuntu 18.04.4 LTS.

-Mercy

Hi,
For UDP we have validated the steps in this post:
Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL

Please give it a try and check if it works.

Hi Dane,
I’m not sure which steps from the post you wanted me to try? I did try these two commands, and both work.

l also had the change the idrinterval to be iframeinterval as idrinterval doesn’t exist on my version of nvv4l2h24enc

gst-launch-1.0 udpsrc port=5000 ! ‘application/x-rtp,encoding-name=H264,payload=96’ ! rtph264depay ! avdec_h264 ! xvimagesink sync=0

gst-launch-1.0 videotestsrc is-live=1 ! video/x-raw,width=1280,height=720 ! timeoverlay valignment=4 halignment=1 ! nvvidconv ! ‘video/x-raw(memory:NVMM),width=1280,height=720’ ! tee name=t ! nvv4l2h264enc insert-sps-pps=1 iframeinterval=15 ! h264parse ! rtph264pay ! udpsink host=10.19.106.10 port=5000 sync=0 t. ! queue ! nvegltransform ! nveglglessink sync=0

But neither of them use the nvv4l2decoder element which is causing me issues. In fact, the nvv4l2decoder is not mentioned in the post…
I will also add that the problem goes away if I replace the nvv4l2decoder element with the omxh264dec element. I believe there is a bug with nvv4l2decoder element. Could your team please try my test case?

send command:

gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080 ! nvvidconv ! nvv4l2h264enc insert-vui=1 ! rtph264pay config-interval=1 ! udpsink port=10106

consume command that doesn’t work:

gst-launch-1.0 udpsrc buffer-size=1800000 port=10106 ! application/x-rtp,media=video,encoding-name=H264,payload=96,interlace-mode=progressive ! queue ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink

consume command that does work:

gst-launch-1.0 udpsrc buffer-size=1800000 port=10106 ! application/x-rtp,media=video,encoding-name=H264,payload=96,interlace-mode=progressive ! queue ! rtph264depay ! h264parse ! omxh264dec ! nvoverlaysink

Hi,
So you have two Xavier. First Xavier runs:

$ gst-launch-1.0 videotestsrc is-live=1 ! video/x-raw,width=1280,height=720 ! timeoverlay valignment=4 halignment=1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1280,height=720' ! tee name=t ! nvv4l2h264enc insert-sps-pps=1 ! h264parse ! rtph264pay ! udpsink host=_IP_OF_SECOND_XAVIER_ port=5000 sync=0 t. ! queue ! nvegltransform ! nveglglessink sync=0

Second Xavier runs this command:

$ gst-launch-1.0 udpsrc port=5000 ! 'application/x-rtp,encoding-name=H264,payload=96' ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink

And you can observe corrupted scene in video playback. Is this correct? We don’t observe any issue on Jetpack 4.6. Looks like you use another version. Please share the releaser version for reference:

$ cat /etc/nv_tegra_release

Hi Dane,

Correction. I have 1 Xavier that runs both scripts. And I do not see video corruption with the 2nd script. The video stream never launches with the 2nd script.

pps@apps-desktop:~/staleVideo$ gst-launch-1.0 udpsrc port=5000 ! ‘application/x-rtp,encoding-name=H264,payload=96’ ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink
0:00:00.071969824 11340 0x5594e7d780 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/apps/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
0:00:00.086388736 11340 0x5594e7d780 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.086461568 11340 0x5594e7d780 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x5594e7f770 Failed to determine interlace mode
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
(gst-launch-1.0:11340): GStreamer-CRITICAL **: 22:42:39.307: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed

Here is the output from cat /etc/nv_tegra_release

apps@apps-desktop:~/staleVideo$ cat /etc/nv_tegra_release
R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020

If I run this script in place of the 2nd script, the video launches:

gst-launch-1.0 udpsrc buffer-size=1800000 port=10106 ! application/x-rtp,media=video,encoding-name=H264,payload=96,interlace-mode=progressive ! queue ! rtph264depay ! h264parse ! omxh264dec ! nvoverlaysink

Hi,
Please run both commands on one Xavier and check if it works:

$ gst-launch-1.0 videotestsrc is-live=1 ! video/x-raw,width=1280,height=720 ! timeoverlay valignment=4 halignment=1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1280,height=720' ! tee name=t ! nvv4l2h264enc insert-sps-pps=1 ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=5000 sync=0
$ gst-launch-1.0 udpsrc port=5000 ! 'application/x-rtp,encoding-name=H264,payload=96' ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink

Would like to eliminate effect of network. Probably the network is not stable.

Hi Dane,

I have tried running both commands on one Xavier, and the error remains the same. I am posting the pipeline and the resulting errors below. As I mentioned before, if I replace nvv4l2decoder with omxh264dec, the error will go away. Is it possible that it’s because of the jetpack version I’m using? Is it possible to fix this without having to change the jetpack version?

apps@apps-desktop:~$ gst-launch-1.0 udpsrc port=5000 ! ‘application/x-rtp,encoding-name=H264,payload=96’ ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink
0:00:00.076193088 11424 0x55aa564780 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/apps/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
0:00:00.092417376 11424 0x55aa564780 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.092502144 11424 0x55aa564780 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55aa566770 Failed to determine interlace mode
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
0:00:00.121785344 11424 0x55aa510000 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 15986 will be dropped
0:00:00.132474560 11424 0x55aa510000 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 15981 will be dropped
0:00:00.193282528 11424 0x55aa510000 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 15944 will be dropped
0:00:00.219550560 11424 0x55aa510000 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 15875 will be dropped
(gst-launch-1.0:11424): GStreamer-CRITICAL **: 10:01:12.168: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed

Hi,
If possible, we would suggest upgrade to latest release. One more thing to try is to clean cache:

$ rm .cache/gstreamer-1.0/registry.aarch64.bin

If it still does not help, would suggest upgrade.

Hi Dane,

I will try to upgrade my development unit later this week. But I have a question, is there a way to upgrade the jetpack on a system that is already out in the field without having to reflash it using the sdk manager?
To put it more plainly, how do I update the jetpack on a system that is already sold and in the hands of the customer? Is there a way to do it by just installing a pre-downloaded package?

-Mercy

Hi,
Please refer to this section:
https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3261/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/updating_jetson_and_host.html#

You can try image-based OTA.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.