Video Transcode with G-Streamer

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

HI,
First I am by no means a developer. I have a Nano because I thought it was going to be a nice toy to use to transcode video with,

I originally started trying to use jetson-ffmpeg as I responably familiar with that, but there appears to be an issue causing intermittant blocking on video.

So I have just started to look at gstreamer. i’ve had a look at the docs, not in depth, and had a google around, and have got as far as a a command like this:-

gst-launch-1.0 filesrc location=2.mkv ! queue ! h264parse ! nvv412decoder ! nvv412h265enc bitrate=200000000 ! filesink location=2222.mkv

this gets me the closest i have got to anything happening but it stops with a :-

ERROR: from element /GstPipeline:pipeline0/GetH264Parse:h264Parse0: error parsing H.264 stream
Additional debug info:
gsth264parse.c(1193): gst_h264_parse_handle_frame (): /GstPipeline:pipeline0/GstH264Parse:h264Parse0:
No H.264 NAL Unit found

i googled that and that just confused me more.

I’m at a loss how to make this work, and would appreciate some help.

The majority of what i want to do is transcode a local h264 mkv to a H265 mkv, so one command will suffice to make my life happy.

Thanks in advance.

Toby

2

First thing you have to manage is mkv container demuxing/muxing.
Second thing is that there is a bug in nvv4l2decoder with H264 parsing in some L4T versions. The workaround is just removing h264parse before nvv4l2decoder.
So just try:

gst-launch-1.0 filesrc location=2.mkv ! matroskademux ! h264parse ! nvv4l2decoder ! nvv4l2h265enc bitrate=200000000 ! h265parse ! matroskamux ! filesink location=2222.mkv

#or
gst-launch-1.0 filesrc location=2.mkv ! matroskademux ! video/x-h264 ! nvv4l2decoder ! nvv4l2h265enc bitrate=200000000 ! h265parse ! matroskamux ! filesink location=2222.mkv
3

That’s awesome, thanks a bundle, the first one works (once I reigned in the bitrate!!).

Next thing to figure out is how to copy all the audio and subtitle streams.

Thanks again

Toby

4

You would find details about matroska muxer and demuxer with:

gst-inspect-1.0 matroskamux
gst-inspect-1.0 matroskademux

You would use multiple sub-pipelines, using queue for each output from muxer (as you would do after tee) and adding a queue for each input of muxer. This would save synchronization problems.
Assuming you have no transcoding for audio, you would use the following for transcoding first H264 stream into H265 while keeping the first audio track with same encoding:

 gst-launch-1.0 filesrc location=2.mkv ! matroskademux name=demux     demux.video_0 ! queue ! video/x-h264 ! h264parse ! nvv4l2decoder ! nvv4l2h265enc bitrate=20000000 ! h265parse ! queue ! mux.video_0       demux.audio_0 ! queue ! mux.audio_0     matroskamux name=mux ! filesink location=2222.mkv
6

Hi,
Please also refer to
Jetson Nano H264 to H265 transcode - #3 by DaneLLL

7

Hi,
Thanks both for the replies, i have what i need now.

Appreciate it, wish all forums had this level of response :)

8

HI ,
Once again thanks for the help so far, its been useful in my little home project. I do have need of some help, i am really struggling understanding much about gstreamer, its pipeline, pads,caps etc. if you could provide an example based on this example that will recode the video from x264 to x265 and copy two audio tracks, and copy two subtitle tracks that would help me a bunch.

this is my starting point, which transcodes the video and copies one audio track. i have tried to copy more than one audio track, but fail. And i have no idea where to start with subtitles.

gst-launch-1.0 filesrc location=2.mkv ! matroskademux name=demux demux.video_0 ! queue ! video/x-h264 ! h264parse ! nvv4l2decoder ! nvv4l2h265enc bitrate=20000000 ! h265parse ! queue ! mux.video_0 demux.audio_0 ! queue ! mux.audio_0 matroskamux name=mux ! filesink location=2222.mkv

9

Hi,
The case is a bit complicated and we suggest go to gstreamer forum to get further help. Once you get a working pipeline, please replace software decoder(such as avdec_h264)/encoder(x265enc) with hardware decoder(nvv4l2decoder)/encoder(nvv4l2h265enc) to enable hardware acceleration.

10

For gstreamer learning, you may check this for global architecture. You may also read this post for more jetson-related details and experiment yourself, it should be harmless if taking care not to fill your root filesystem recording for long or in raw mode.
Not sure, but I may give it a try if :

  • you tell more about your home project
  • you can provide a short sample (< 10 MB) with both audio and both subtitles.
11

Hi thanks for these resources.

I have actually got to the bottom of a lot of it. And can deal with what’s left left in other ways

But thanks to yourselves and the gstreamer mailing list I have a working pipeline that transcodes video and passes all audio through (apart from true hd for some reason).

I’ll post the pipeline once I have a few minutes to sort sort myself out.

Thanks again for the help with this.

Regards

Toby

Regards

Toby

1 Like
12

Hi,

So, with the help from yourselves and the people on the gstreamer-devel@lists.freedesktop.org mailing list I have a nearly working example

Assuming 6 audio tracks and 1 video track.

gst-launch-1.0 filesrc location=“/external/Staging/2.mkv” ! matroskademux name=demux
demux.video_0 ! queue ! video/x-h264 ! h264parse ! nvv4l2decoder ! nvv4l2h265enc bitrate=3500000 ! h265parse ! queue ! mux.video_0
demux.audio_0 ! queue ! mux.audio_0
demux.audio_1 ! queue ! mux.audio_1
demux.audio_2 ! queue ! mux.audio_2
demux.audio_3 ! queue ! mux.audio_3
demux.audio_4 ! queue ! mux.audio_4
demux.audio_5 ! queue ! mux.audio_5
matroskamux name=mux ! progressreport update-freq=1 ! filesink location=“/external/Staging/2new.mkv”

This will recode the video from H264 to H265 and then passthrough audio tracks using the track index. I do need to know how many tracks there are, and build the audio parts in a loop, but I can do this in a script quite happily.

There 2 niggles with this:-

  1. If there are TRUEHD audio tracks it throws its toys out of the pram, with a message saying it could not link to the PAD of the MUX. I’m presuming the matroskamux does not recognise TRUEHD.

  2. I have not been able to use the same method for subtitles.

If you are interested my home project is to re-encode my collection of ripped DVD’s, I ripped my collection over the years to H264, but want to reduce the storage, hence using H265. The Nvidia Jetson seemed like the perfect toy for this.

As it stands with the two niggles above, I can work around it by dumping the TRUEHD tracks to individual files using MKVExtract along with timestamps, and then remux them back in using ffmpeg, or MKVMerge, the same goes for subtitles. Its a shame as it adds a fair bit of time to the process, So if you did have some insight into the subtitles that would be fantastic . I tried doing subtitles like this:-

demux.subtitle_1 ! queue ! mux.subtitle_0 \

Any the help received so far is much appreciated.

Regards

Toby

13

Thanks for sharing your findings. I may not be able to help much more without a sample, but you may try to install mkvtoolnix-gui:

sudo apt install mkvtoolnix-gui
mkvtoolnix-gui <your_file>.mkv

for checking what’s inside your container, and you may be able to replace h264 video by h265, but I can’t help much for this.
Jobs may be queued, or you may try command line interface for scripts.

If you want to get a gstreamer transcoding pipeline, it may be less generic depending on how many formats you have, but you would post the output of this command that may tell what items/formats are in your mkv container:

gst-launch-1.0 -v filesrc location= <your_file>.mkv ! matroskademux name=demux      demux.subtitle_0 ! queue ! fakesink       demux.video_0 ! queue ! fakesink        demux.audio_0 ! queue ! fakesink
14

HI,

Ok so i ran you fakesink commmand and this is what I got

/GstPipeline:pipeline0/GstCapsFilter:capsfilter1: caps = video/x-h264
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-h264, level=(string)4.1, profile=(string)high, codec_data=(buffer)01640029ffe1003567640029ac1b1a501e0089f9701100000303e90000bb80e2600004692600007270e8c4b8c4c00008d24c0000e4e1d18970f8e1852c01004268ea8dce51d1d84388c54548223311cd9a2128e8ec21c462a2a4111988e6cd109444c521b93c57c9f93f93f27c9e6e4c9250890a4272793ebf27f5f93ebc9c9a91c0fdf8f800, stream-format=(string)avc, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)24000/1001
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-h264, level=(string)4.1, profile=(string)high, codec_data=(buffer)01640029ffe1003567640029ac1b1a501e0089f9701100000303e90000bb80e2600004692600007270e8c4b8c4c00008d24c0000e4e1d18970f8e1852c01004268ea8dce51d1d84388c54548223311cd9a2128e8ec21c462a2a4111988e6cd109444c521b93c57c9f93f93f27c9e6e4c9250890a4272793ebf27f5f93ebc9c9a91c0fdf8f800, stream-format=(string)avc, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)24000/1001
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-h264, level=(string)4.1, profile=(string)high, codec_data=(buffer)01640029ffe1003567640029ac1b1a501e0089f9701100000303e90000bb80e2600004692600007270e8c4b8c4c00008d24c0000e4e1d18970f8e1852c01004268ea8dce51d1d84388c54548223311cd9a2128e8ec21c462a2a4111988e6cd109444c521b93c57c9f93f93f27c9e6e4c9250890a4272793ebf27f5f93ebc9c9a91c0fdf8f800, stream-format=(string)avc, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)24000/1001
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-h264, level=(string)4.1, profile=(string)high, codec_data=(buffer)01640029ffe1003567640029ac1b1a501e0089f9701100000303e90000bb80e2600004692600007270e8c4b8c4c00008d24c0000e4e1d18970f8e1852c01004268ea8dce51d1d84388c54548223311cd9a2128e8ec21c462a2a4111988e6cd109444c521b93c57c9f93f93f27c9e6e4c9250890a4272793ebf27f5f93ebc9c9a91c0fdf8f800, stream-format=(string)avc, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)24000/1001
/GstPipeline:pipeline0/GstFakeSink:fakesink1.GstPad:sink: caps = video/x-h264, level=(string)4.1, profile=(string)high, codec_data=(buffer)01640029ffe1003567640029ac1b1a501e0089f9701100000303e90000bb80e2600004692600007270e8c4b8c4c00008d24c0000e4e1d18970f8e1852c01004268ea8dce51d1d84388c54548223311cd9a2128e8ec21c462a2a4111988e6cd109444c521b93c57c9f93f93f27c9e6e4c9250890a4272793ebf27f5f93ebc9c9a91c0fdf8f800, stream-format=(string)avc, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)24000/1001
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = audio/x-true-hd, channels=(int)8, rate=(int)48000
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = audio/x-true-hd, channels=(int)8, rate=(int)48000
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = audio/x-true-hd, channels=(int)8, rate=(int)48000
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = audio/x-true-hd, channels=(int)8, rate=(int)48000
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = text/x-raw, format=(string)pango-markup
/GstPipeline:pipeline0/GstFakeSink:fakesink2.GstPad:sink: caps = audio/x-true-hd, channels=(int)8, rate=(int)48000

the video i have no problem with, it works fine. All audio is good apart true-hd this won’t find a pad on the Matroskademux, like wise any subtitle tracks.

Where can up load a sample?

regards

Toby

15

I don’t think that gstreamer will fit your needs… Dolby TrueHD support in gstreamer may still be experimental and matroska container plugins may not be easy to use from gst-launch for subtitles (container does store utf8 as does muxer expects, but demuxer sends pango-markup format).

I have also checked with mkvtoolnix-gui that is possible to replace H264 video with a transcoded H265 video. If you’re already familiar with mkvextract and mkvmerge, you may be able to automate with scripts extracting the items, get H264 video bitrate (mediainfo may be easier than mkvinfo for this), transcode h264 video into h265 with appropriate bitrate for reducing size from H264 source, and merge items with h265 video instead of h264.

Good luck and have fun.

16

Hi,

I think thats pretty much what I had come to realise.

I would have been nice to have gstreamer do all the work, but I guess it is what it is.

I’ve already made a start on demuxing the truehd, and subtitle out of the file, getting gstreamer to do what it can with the video, and audio formats it can passthrough properly.

I’m having to revisit my old, vb skills to do it, as scripting became to messy and complicated, but have made some good progress.

Thanks to help, and investigations on this.

regards

Toby