Can I take a H264 stream, and output it via rtsp as H265? My goal is to use the more efficient H265 compression so I can have high quality video even at lower bitrates.
I send a stream from udpsrc to an rtspserver. If the stream is encoded as H265, I use this pipeline
However when I try to connect to the rtspserver and stream, I get the error gst_mini_object_unref. Could someone please tell me where my pipeline is wrong?
With GST_DEBUG=3 it showed me that the h264parse element were dropping frames because of a 'broken/invalid nal ’ error. I also saw warnings and a message that the pipeline failed to preroll. :
Opening in BLOCKING MODE
0:00:06.071679584 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f980994f0 Failed to determine interlace mode
0:00:06.071773696 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f980994f0 Failed to determine interlace mode
0:00:06.071814400 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f980994f0 Failed to determine interlace mode
0:00:06.071966304 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2h265enc0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE
0:00:06.074335968 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:06.074381504 16214 0x7f9800d630 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f9808d5e0 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
0:00:06.116150240 16214 0x7f90031230 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 13147 will be dropped
0:00:06.153737472 16214 0x7f90031230 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 16353 will be dropped
0:00:06.182337280 16214 0x7f90031230 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 14529 will be dropped
0:00:06.214613504 16214 0x7f90031230 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 12681 will be dropped
…
(test-launch:16214): GStreamer-CRITICAL **: 09:35:08.547: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed
0:00:26.062516000 16214 0x557c3e9990 WARN rtspmedia rtsp-media.c:2991:wait_preroll: failed to preroll pipeline
0:00:26.062918176 16214 0x557c3e9990 WARN rtspmedia rtsp-media.c:3295:gst_rtsp_media_prepare: failed to preroll pipeline
I tried dropping the h264parse element, but the warnings about the decoder and encoder remain. as well as the warning of failed to preroll. As you can see from my pipeline, I tried to be explicit with the interlace_mode, but the warning remains. Any ideas?
I tried the pipeline using gst-launch-1.0 and it returns the same warnings as the test-launch does. I guess this means the pipeline is wrong. Unfortunately I do not have enough gstreamer experience to identify where the error is. Is there anyone on your who would know where I went wrong?
gst-launch-1.0 udpsrc buffer-size=1800000 port=10106 ! application/x-rtp,media=video,encoding-name=H264,payload=96,interlace-mode=progressive ! queue ! rtph264depay ! h264parse ! nvv4l2decoder disable-dpb=true enable-max-performance=1 ! nvvidconv ! nvv4l2h265enc ! video/x-h265,stream-format=byte-stream,interlace-mode=progressive ! rtph265pay name=pay0 pt=96 ! fakesink
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
0:00:00.088091168 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55881a4880 Failed to determine interlace mode
0:00:00.088165280 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55881a4880 Failed to determine interlace mode
0:00:00.088205312 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55881a4880 Failed to determine interlace mode
0:00:00.088331744 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2h265enc0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE
0:00:00.090015648 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.090072576 10376 0x55881b4e00 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x5588195f20 Failed to determine interlace mode
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
0:00:00.128348416 10376 0x55881afb20 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 13150 will be dropped
0:00:00.157965984 10376 0x55881afb20 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 13311 will be dropped
0:00:00.200131360 10376 0x55881afb20 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 13115 will be dropped
…
(gst-launch-1.0:10376): GStreamer-CRITICAL **: 22:40:32.765: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed
Seems like the stream is invalid. Would like to make sure hardware decoder can decoder it correctly. You should see preview if the stream is well decoded.
I did notice that if I replace the nvv4l2decoder element with avdec_h264 element in the command you gave me, I am able to see the video preview. Is it possible that the nvv4l2decoder element has an issue? I am currently running jetpack 4.4, Ubuntu 18.04.4 LTS.
But neither of them use the nvv4l2decoder element which is causing me issues. In fact, the nvv4l2decoder is not mentioned in the post…
I will also add that the problem goes away if I replace the nvv4l2decoder element with the omxh264dec element. I believe there is a bug with nvv4l2decoder element. Could your team please try my test case?
And you can observe corrupted scene in video playback. Is this correct? We don’t observe any issue on Jetpack 4.6. Looks like you use another version. Please share the releaser version for reference:
Correction. I have 1 Xavier that runs both scripts. And I do not see video corruption with the 2nd script. The video stream never launches with the 2nd script.
pps@apps-desktop:~/staleVideo$ gst-launch-1.0 udpsrc port=5000 ! ‘application/x-rtp,encoding-name=H264,payload=96’ ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink
0:00:00.071969824 11340 0x5594e7d780 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/apps/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
0:00:00.086388736 11340 0x5594e7d780 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.086461568 11340 0x5594e7d780 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x5594e7f770 Failed to determine interlace mode
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
(gst-launch-1.0:11340): GStreamer-CRITICAL **: 22:42:39.307: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed
Here is the output from cat /etc/nv_tegra_release
apps@apps-desktop:~/staleVideo$ cat /etc/nv_tegra_release
R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020
If I run this script in place of the 2nd script, the video launches:
I have tried running both commands on one Xavier, and the error remains the same. I am posting the pipeline and the resulting errors below. As I mentioned before, if I replace nvv4l2decoder with omxh264dec, the error will go away. Is it possible that it’s because of the jetpack version I’m using? Is it possible to fix this without having to change the jetpack version?
apps@apps-desktop:~$ gst-launch-1.0 udpsrc port=5000 ! ‘application/x-rtp,encoding-name=H264,payload=96’ ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink
0:00:00.076193088 11424 0x55aa564780 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/apps/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
0:00:00.092417376 11424 0x55aa564780 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.092502144 11424 0x55aa564780 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55aa566770 Failed to determine interlace mode
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
0:00:00.121785344 11424 0x55aa510000 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 15986 will be dropped
0:00:00.132474560 11424 0x55aa510000 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 15981 will be dropped
0:00:00.193282528 11424 0x55aa510000 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 15944 will be dropped
0:00:00.219550560 11424 0x55aa510000 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 1 Slice, Size: 15875 will be dropped
(gst-launch-1.0:11424): GStreamer-CRITICAL **: 10:01:12.168: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed
I will try to upgrade my development unit later this week. But I have a question, is there a way to upgrade the jetpack on a system that is already out in the field without having to reflash it using the sdk manager?
To put it more plainly, how do I update the jetpack on a system that is already sold and in the hands of the customer? Is there a way to do it by just installing a pre-downloaded package?