Hlssink an h264 stream

Accelerated Computing Intelligent Video Analytics General
1

I am trying to develop a camera connected to a Xavier system. My camera produces an h264 stream using nvidia’s cuda based encoder. I want to create an HLS stream from this so that my camera output can be accessed from a web browser. I think I am almost there but there is one hurdle. The hlssink (or hlssink2) modes only produce a single segment file: I cannot get it to split into multiple segments. Firstly, the following doesn’t work:

gst-launch-1.0 v4l2src device=/dev/video2 ! mpegtsmux ! hlssink target-duration=2 max-files=5

It says “could not create handle for stream”. Then I tried adding h264parse:

gst-launch-1.0 v4l2src device=/dev/video2 ! h264parse ! mpegtsmux ! hlssink target-duration=2 max-files=5

This works but always a single segment file is created and it keeps growing in size. Then I tried hlssink2 as follows:

gst-launch-1.0 v4l2src device=/dev/video2 ! h264parse ! hlssink2 target-duration=2 max-files=5

The result is practically the same as before: a single segment file is created. To debug this system, I converted a video to H264 format using nvidia jetson samples (03_video_cuda_enc). The video has about 1500 frames. Then I tried using filesrc instead of v4lsrc:

gst-launch-1.0 filesrc location=test.h264 ! h264parse ! mpegtsmux ! hlssink target-duration=2 max-files=5

The result is the same as before: I only get one segment file (segment00000.ts). In order to further debug the system, I used videotestsrc:

gst-launch-1.0 videotestsrc is-live=true ! x264enc ! h264parse ! mpegtsmux ! hlssink target-duration=2 max-files=5

This works perfectly: 2 second segments are created and the playlist.m3u8 file is also correctly updated. Then I wanted to try getting rid of h264parse:

gst-launch-1.0 videotestsrc is-live=true ! x264enc ! mpegtsmux ! hlssink target-duration=2 max-files=5

This also worked perfectly, which confused me a little because when I output h264 from my camera, I had to specify h264parse before hlssink. But after x264enc apparently it wasn’t needed. I thought my h264 and x264enc would produce compatible streams.

To summarize I have one main and one sub question. The main question is why my h264 files (or the camera stream) do not get split into multiple segments by hlssink? The subquestion is why do I need to specify h264parse for my stream but this is not necessary for the videotestsrc. Thanks for any help.

3

Some further info on the subject. When I open my .h264 file with a video player, I do not see a seek bar option (I cannot seek to a random location within the video). To my knowledge, this is related to missing I/IDR frames. However, I examined the type of each frame and I see that every 30th frame is an I-Frame (I had used setIFrameInterval(30)).

4

H264 parse is used to segment the h264 frame data. X264enc may have segment it internally.

Who developed this module for camera? There may be some issues with its design and development. I use the pipeline below, it works well:

 gst-launch-1.0 videotestsrc is-live=true ! nvvideoconvert ! nvv4l2h264enc  ! h264parse ! mpegtsmux ! hlssink target-duration=2 max-files=5
5

We are developing our own camera system. The data is coming through the MIPI connection from a Sony sensor. Actually, I think we found a solution. We were missing VUI information in the encoded frames. We used the setInsertVuiEnabled(true) function defined in “Jetson Linux API Reference: NvVideoEncoder Class Reference | NVIDIA Docs”. Combined with the following pipeline, we can now obtain multiple segments during HLS streaming:

gst-launch-1.0 v4l2src device=/dev/video2 ! h264parse ! hlssink2 target-duration=2 max-files=5

The only remaining problem is that hlssink2 seems to ignore the max-files=N parameter. Even if we change it, it always produces 5 output files. Is it a known issue?

6

Sorry for quick posting. I think we found a solution for the max-files problem as well. We found that the playlist-length parameter works instead of the max-files parameter. So if we do:

gst-launch-1.0 v4l2src device=/dev/video2 ! h264parse ! hlssink2 target-duration=1 playlist-length=10

we can get 10 fragments (that are continuously updated) with each 1 second long. I think the max-files parameter is really ignored. Hopefully, this thread turns out useful for other people as well. We were actually quite puzzled by the setInsertVuiEnabled(true) function as we couldn’t see an example of it in our online searches.

7

You can find some samples from the github, like:
https://github.com/fastvideo/gpu-camera-sample/blob/8bcf51a8b5378c578d5ceac2e7fe223f2ace6e9f/src/CameraSample/jetson_api/v4l2encoder.cpp#L111

8

Thank you. This sample looks quite useful indeed.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.