GStreamer to convert from MPEG2 to MP4

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Hi,

I need help with getting GStreamer to convert Mpeg2 files into MP4. No matter what I do, it would just keep on giving me error.

Here is info from discover:
/mnt/videos$ gst-discoverer-1.0 cut1.ts
Analyzing file:///mnt/videos/cut1.ts
Done discovering file:///mnt/videos/cut1.ts

Topology:
container: MPEG-2 Transport Stream
audio: MPEG-1 Layer 2 (MP2)
video: MPEG-2 Video

Properties:
Duration: 4:00:00.189702111
Seekable: yes
Live: no
Tags:
audio codec: MPEG-1 Audio
has crc: false
channel mode: stereo
nominal bitrate: 384000
video codec: MPEG-2 Video

I have huge 100GB+ .ts files that I want to convert into something more manageable and I want to use GPU. Usually we use ffmpeg, but on Xavier NVIDIA suggests to use gstreamer and I am stuck. Could someone please help?

3

Hi,
You may refer to the posts:

4

Hi,

thank you for your reply. I saw it and tried it, and it just creates 0 sized mp4 files.

/mnt/videos$ sudo gst-launch-1.0 qtmux name=mx ! filesink location=test.mp4 filesrc location= test.ts ! tsdemux name=dmx dmx. ! queue ! h264parse ! mx. dmx. ! queue ! aacparse ! faad ! audioresample ! audioconvert ! voaacenc ! mx.
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
WARNING: from element /GstPipeline:pipeline0/GstTSDemux:dmx: Delayed linking failed.
Additional debug info:
./grammar.y(510): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstTSDemux:dmx:
failed delayed linking some pad of GstTSDemux named dmx to some pad of GstQueue named queue0
WARNING: from element /GstPipeline:pipeline0/GstTSDemux:dmx: Delayed linking failed.
Additional debug info:
./grammar.y(510): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstTSDemux:dmx:
failed delayed linking some pad of GstTSDemux named dmx to some pad of GstQueue named queue1
ERROR: from element /GstPipeline:pipeline0/GstTSDemux:dmx: Internal data stream error.
Additional debug info:
mpegtsbase.c(1620): mpegts_base_loop (): /GstPipeline:pipeline0/GstTSDemux:dmx:
streaming stopped, reason not-linked (-1)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …
Freeing pipeline …

This is like the most basic TS to MP4 conversion. What am I doing wrong?

This is a security system for our public school and I am an impromptu sysadmin here and our storage system is about to get filled unless I convert it. Would really appreciate some help!

5

Hi,
cut1.ts has mpeg2 stream instead of h264. You should need to change h264parse to
https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad/html/gst-plugins-bad-plugins-mpegvideoparse.html

And audio from aac to MPEG-1 Audio.

6

Dane,

thank you!

I made a little bit of progress, but it is not perfect.

I managed to get it to play! But then I tried to encode it into mp4 and whole hell broke lose again

sudo gst-launch-1.0 qtmux name=mx ! filesink location=test.mp4 filesrc location= test.ts ! tsdemux name=dmx dmx. ! queue ! mpegvideoparse ! mx. dmx. ! queue ! aacparse ! faad ! audioresample ! audioconvert ! voaacenc ! mx.

WARNING: erroneous pipeline: could not link mpegvparse0 to mx

I spent four hour reading last night GStreamer guide by NVIDIA, and based on my understanding, I am to process video and audio separately, so I need to create separate queues for it and then mix them back up.

Could someone please tell me where did I go astray?

7

Hi,
MPEG-2 is not supported in MP4, so you need to transcode it to H264(or H265) stream.

FYR. We can run the command and get valid MP4:
https://filesamples.com/samples/video/mts/sample_640x360.mts

$ gst-launch-1.0 filesrc location=sample_640x360.mts ! tsdemux ! mpegvideoparse ! avdec_mpeg2video ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=a.mp4
8

I was able to get this far:

sudo gst-launch-1.0 filesrc location=/mnt/videos/stream1/20201218_122145.ts ! tsdemux ! mpegvideoparse ! avdec_mpeg2video ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=/mnt/videos/stream1/a.mp4
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is PREROLLING …
Redistribute latency…
Redistribute latency…
ERROR: from element /GstPipeline:pipeline0/GstTSDemux:tsdemux0: Internal data stream error.
Additional debug info:
mpegtsbase.c(1620): mpegts_base_loop (): /GstPipeline:pipeline0/GstTSDemux:tsdemux0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …
Freeing pipeline …

This is what ffmpeg tells me:
Input #0, mpegts, from ‘20201218_122145.ts’:
Duration: 24:00:00.08, start: 1.422422, bitrate: 3341 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: mpeg2video (4:2:2) ([2][0][0][0] / 0x0002), yuv422p(tv, bt470bg/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 44100 Hz, stereo, s16p, 384 kb/s

I am slowly going mad here…

9

Your problem is that your mpeg2 video is using 4:2:2 profile. You would have to convert into a suitable format for nvvidconv:

gst-launch-1.0 -ev filesrc location=test_mpeg2_422.ts ! tsdemux ! mpegvideoparse ! avdec_mpeg2video ! video/x-raw, format=Y42B ! videoconvert ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test_mpeg2_422.trancoded_h264.mp4
10

You are awesome!!! I got it to run!!! But… it is only using CPUs…my GPU is taking a long and by now highly undeserved rest. I am looking through the developer guide, and it seems as the whole idea of using nvv encoder is that it will use GPU.

Right now, I get about 18 fps if I am lucky. Needless to say, it will be impossible to convert thousand hours of footage at this fps…

sudo gst-launch-1.0 -ev filesrc location=20201224_122157.ts ! tsdemux ! mpegvideoparse ! avdec_mpeg2video ! ‘video/x-raw, format=Y42B’ ! videoconvert ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=(string)I420’ ! queue! nvv4l2h264enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=false -v

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 8609, dropped: 0, current: 19.65, average: 20.10

I noticed that t is 96 fps before nvvconvert:

sudo gst-launch-1.0 -ev filesrc location=20201224_122157.ts ! tsdemux ! mpegvideoparse ! avdec_mpeg2video ! ‘video/x-raw, format=Y42B’ ! videoconvert ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=false -v

It is over 900 fps before avdec_mpeg2video.

I also thought that I need to specify NVV but as you can see from the tests above, it does not help. Somehow, it feels as Xavier is doing everything on CPU:
sudo gst-launch-1.0 -ev filesrc location=20201224_122157.ts ! tsdemux ! mpegvideoparse ! avdec_mpeg2video ! video/x-raw, format=Y42B ! videoconvert ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test_mpeg2_422.trancoded_h264.mp4

I used sudo tegrastats and jtop

This is what tegrastats give me. I see GR3D_FREQ as 0, which means GPU is still somewhere in Bahamas…

RAM 1958/31927MB (lfb 179x4MB) SWAP 78/15963MB (cached 5MB) CPU [41%@2264,41%@2264,78%@2264,74%@2264,95%@2264,50%@2264,78%@2264,60%@2264] EMC_FREQ 7%@2133 GR3D_FREQ 0%@1377 NVENC 115 NVENC1 115 APE 150 MTS fg 0% bg 3% AO@40.5C GPU@42.5C Tdiode@44.25C PMIC@100C AUX@43C CPU@45C thermal@43.25C Tboard@38C GPU 1080/1079 CPU 7869/7295 SOC 4629/4518 CV 0/0 VDDRQ 1080/1051 SYS5V 2688/2721

Could it be because I am using avdec_mpeg2video which does not support HW acceleration that it forces the whole thing to run on CPU only? I do not really care if the result is MP4 or MKV - I just need to compress these horrendous 200 GB per day per stream files into something more manageable. I was hoping to use GStreamer with GPU to do this, but it feels as the whole world is against me.

11

Hi,
Since Y42B format is not supported by hardware converter, you need software converter for doing conversion. The takes certain CPU usage.

12

Indeed GPU is not involved here. The mpeg2 decoding is CPU only, and H264 encoding is done with dedicated HW encoder referred as NVENC in tegrastats.

I’m a bit surprised by your numbers. On my XavierNX with L4T R32.4.4, I get these:

# Boost your NX
sudo nvpmodel -m2
sudo jetson_clocks

gst-launch-1.0 -ev filesrc location=test_mpeg2_422.ts ! tsdemux ! mpegvideoparse ! avdec_mpeg2video ! video/x-raw, format=Y42B ! videoconvert ! video/x-raw, format=I420 ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! nvv4l2h264enc ! h264parse ! fpsdisplaysink video-sink=fakesink text-overlay=0 sync=false
...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 37, dropped: 0, current: 73,98, average: 73,98
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 78, dropped: 0, current: 81,25, average: 77,63
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 118, dropped: 0, current: 78,71, average: 77,99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 156, dropped: 0, current: 74,56, average: 77,13
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 196, dropped: 0, current: 79,66, average: 77,63
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 237, dropped: 0, current: 81,22, average: 78,23
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 277, dropped: 0, current: 78,06, average: 78,21

gst-launch-1.0 -ev filesrc location=test_mpeg2_422.ts ! tsdemux ! mpegvideoparse ! avdec_mpeg2video ! video/x-raw, format=Y42B ! videoconvert ! video/x-raw, format=I420 ! omxh264enc ! h264parse ! fpsdisplaysink video-sink=fakesink text-overlay=0 sync=false
...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 41, dropped: 0, current: 81,18, average: 81,18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 85, dropped: 0, current: 87,69, average: 84,43
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 131, dropped: 0, current: 86,78, average: 85,24
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 175, dropped: 0, current: 86,84, average: 85,63
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 219, dropped: 0, current: 86,80, average: 85,86
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 264, dropped: 0, current: 87,35, average: 86,12
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 308, dropped: 0, current: 87,07, average: 86,25

Note I am running Linux on a NVME SSD (Samsung 970 EVO Plus).

13

Dane, thanks. I suspected as much!

  1. so since my videos are Y42B this means that I cannot use GPU for conversion? Can this be somehow bypassed?
  2. I would still expect GPU to do the encoding using NVENC…why is it not being used at all? HW accelerated means use of GPU, no?
14

Honey, yeah, I am perplexed. I tried with nvpmode 0 and 2 and 3 and numbers are the same. clock on and off also do little to change anything. As per jtop and tegrastats GPU is 0Hz through out. It seems as it is not even being loaded…yet conversion works, and log shows that NVENC is being invoked. What is going on?

15

Hi,
Since Y42B is not supported in nvvidconv plugin. Honey_Patouceul’s pipeline is the optimal solution for transcoding:

gst-launch-1.0 -ev filesrc location=test_mpeg2_422.ts ! tsdemux ! mpegvideoparse ! avdec_mpeg2video ! video/x-raw, format=Y42B ! videoconvert ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test_mpeg2_422.trancoded_h264.mp4

The nvv4l2h264enc plugin supports YUV420(I420 or NV12), so need to convert Y42B to the format through software converter videoconvert plugin.