We are having an issue to display JVC camera’s H.264 stream by gstreamer.
Stream seems fine at first, but about 4 hours later it starts dropping frames from queue.
We have used the gstreamer pipeline below to receive the stream:
gst-launch-1.0 rtspsrc location="<JVC camera url>" latency=500
! queue max-size-time=2000000000 max-size-buffers=0 max-size-bytes=0 leaky=2
! h264parse ! omxh264dec
! queue max-size-time=0 max-size-buffers=4 max-size-bytes=0 leaky=2
! nvvidconv top=0 left=0 right=1920 bottom=1080
! video/x-raw, width=1920, height=1080, format=YUY2
! xvimagesink sync=false max-lateness=5000000 qos=true async=false
About 4 hours later, in the
queue element between
h264parse, frames starts dropping because buffers crosses a threshold.
We have confirmed that this issue happens only when gstreamer is receiving H.264 stream from JVC cameras.
We have also tried these below, but not improved.
nvv4l2decoder instead of
- Increase the amount of queue buffer’s threshold.
On the other hand, streams looks fine on the camera web UI even if this problem is happening.
From these results, our suspicion is that gstreamer plugin
nvv4l2decoder do not work correctly only in this case.
Are these any known issues or compatibility problems?
And we also want to solve this problem. So Could you please assist?
- Jetson TX2 [Revision 32.7.3]
- Gstreamer 1.16.2
Thanks in advance.
We have deprecated omx plugin. Please use nvv4l2decoder and enabler the property:
enable-max-performance: Set to enable max performance
flags: readable, writable
Boolean. Default: false
And default gstreamer version is 1.14.5. You may try the default version and see it is more stable. The default system image has done and passed SQA tests before it is published.
Thanks for reply, we tried the option you mentioned by enabling
We’re not sure what this property does to be honest, but the problem didn’t get improved.
We got these logs only after pipeline starts dropping frames.
0:34:15.321717472 9933 0x7f600736d0 WARN h264parse gsth264parse.c:1351:gst_h264_parse_handle_frame:<h264parse> broken/invalid nal Type: 1 Slice,
Size: 4473 will be dropped
0:34:15.321993504 9933 0x7f600736d0 WARN h264parse gsth264parse.c:1351:gst_h264_parse_handle_frame:<h264parse> broken/invalid nal Type: 1 Slice,
Size: 5429 will be dropped
0:34:15.322148384 9933 0x7f600736d0 WARN h264parse gsth264parse.c:1351:gst_h264_parse_handle_frame:<h264parse> broken/invalid nal Type: 1 Slice,
Size: 5041 will be dropped
This is the warning of
h264parse , so we don’t think this parse element is also working well.
We also tested this pipeline on Orin AGX (Revision 35.3.1, gstreamer 1.16.3), and this issue did not happen.
We haven’t tested on the default gstreamer version yet.
Are there any other suggestions to solve this?
The issue looks to be in rtspsrc ! h264parse of 1.14.5. Looks like it does not handle the packets well and cannot send valid h264 stream to next nvv4l2decoder. One possible solution is to manually build 1.16.3 and upgrade to the version for a try. Please refer to guidance in developer guide
Most gstreamer plugins are open source in
Jetson Linux R32.7.4 | NVIDIA Developer
Driver Package (BSP) Sources
You should also need to rebuild the plugins.
Another possible solution it to check the fix in rtspsrc between 1.14.5 and 1.16.3, and port the fix to 1.14.5 and rebuild rtspsrc only.
Sorry for being late. We took a bit time to find TX2 reference board to test default gstreamer version 1.14.5.
This issue still happened even at gstreamer 1.14.5.
We will try it on TX2 (Revision 32.7.4) with gstreamer 1.16.3.
As we already said, the issue didn’t happened on Orin AGX (Revision 35.3.1, gstreamer 1.16.3).
So if this still exists on new version, we guess
nvv4l2decoder at R32.7.4 might have some issues or bugs.
In that case, can’t we fix this issue at R32 environment for now?
It doesn’t seem to be an issue in nvv4l2decoder. If valid and continuous bitstream can be fed to decoder , we would expect it to work well. We would suggest investigate why h264parse drops the bitstream intermittently.
We tested some cases after last reply.
We tried these two cases with fakesink.
fakesink which elements causes
queue | h264parse | fakesink ) → no stack frames on queue
queue | h264parse | nvv4l2decoder | fakesink) → frames stacked on queue
From this results, we agree with your comment.
The issue looks to be in rtspsrc ! h264parse of 1.14.5. Looks like it does not handle the packets well and cannot send valid h264 stream to next nvv4l2decoder .
And we also tried this down below.
One possible solution is to manually build 1.16.3 and upgrade to the version for a try.
We upgraded gstreamer 1.16.3 on TX2 (R. 32.7.4), but it did not solve the problem.
As I already said, orin AGX (Revision 35.3.1, gstreamer 1.16.3) didn’t cause any issues.
The results are different even though
nvv4l2decoder version is same between Orin AGX and TX2.
These two have only revision difference.
We tried all options that you suggested, but unfortunately things haven’t being well.
We are investigateing
h264parsebut we’ve been getting nothing right now.
Do you have any other options or suggestions?
If you think it is something wrong in nvv4l2decoder, please dump the stream:
gst-launch-1.0 rtspsrc ! h264parse ! video/x-h264,stream-format=byte-stream ! filesink location=dump.264
And try both software decoder and hardware decoder:
gst-launch-1.0 filesrc location=dump.h264 ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw ! xvimagesink sync=0
gst-launch-1.0 filesrc location=dump.h264 ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw ! xvimagesink sync=0
If software decoder can decode the stream without issue and hardware decoder malfunctions, please attach the dumped stream so that we can try and debug further.
Sorry for delay to respond.
First, there was no issue for dumped stream to decode by whichever software decoder or hardware decoder.
So we are thinking
nvv4l2decoder is not main reason of this issue.
Next, we also thought that timestamp might cause this issue, So we dumped the RTP packets and replayed them by
However replaying them, there was no issue on that.
The only difference between receiving normal stream and replaying stream is the src element,
rtspsrc is created by combining
rtpbin might be the main reason for this issue on whatever the gstreamer version is. (at least between gstreamer1.14.5 and 1.16.3.)
We couldn’t attach dumped RTP packets because of file size.
This issue happens after a while from receiving stream, so it is a bit hard to make attachable size file.
Is there any way to share the packet file?
rtpbincauses this issue, we have two questions,
- Is there anything to solve this? (It happens both of gstreamer versions 1.14.5 and 1.16.3)
- On Orin AGX(R 35.3.1, gst1.16.3), there is no issue. What is the difference?
The rtspsrc and h264parse plugins are open source, so you may add prints to the plugins to catch more information. To clarify why invalid nal Type is reported in the error condition.
Thanks for reply, we will continue on it.
Lastly, we have one question related to nvidia decoder element.
As we said before, this issue doesn’t happen on Orin AGX(R35.3.1, gst1.16.3).
But it does happen on TX2(R32.7.3, gst1.14.5 and 1.16.3).
Because of using same gstreamer version, we’re thinking only difference is
nvv4l2decoder made by nvidia which does HW decode.
Does that mean Orin AGX can decode stream data which include invalid nal type sent from
rtpbin ! h264parse, and TX2 cannot do that?
Are there any difference between them?
For decoding H264, TX2, Xavier, and Orin have identical hardware block.
Both Jetpack 4 and 5 can be run on Xavier. If you have Xavier or Xavier NX, it may worth a try for comparing Jetpack 4 and 5.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.