No Video for HLS on iOS

Hello,

I am using gstreamer to create files for HLS playback using the tx1. What I have noticed is that the files playback in Chrome on Android. However, on iOS only audio is present. There is no video visible on iOS. I suspect that it is an encoding issue.

Can the omxh264enc plugin for gstreamer produce output that is compatible with iOS? If yes, what are the correct parameters to use?

For the record my pipeline is as follows.

gst-launch-1.0 -v -e mpegtsmux name=m ! hlssink filesrc location=bees_hevc.ts ! tsdemux name=dem dem. ! queue ! h265parse ! omxh265dec ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420’ ! omxh264enc profile=8 ! m. dem. ! queue ! avdec_aac ! audioconvert ! voaacenc ! m.

Hi RandDGraham,
Please try to set ‘profile=1’ or ‘profile=2’. If there is still no video visible on iOS, we need you to share the sample file which is playable on iOS so that we can compare/analyze the video bitstream.

Hi DaneLLL,

Changing the profile did not fix the issue. I still cannot see the video on iOS.

Would you like me to share the video files that have the problem? How can I share them/where should I upload them to?

I have not been able to play back any transcoded video on iOS yet.

Regards,
Rand

Hi RandDGraham,
We would like you to share the video files can run HLS on iOs well, but it looks like you don’t have any.

Could you share how to test HLS on iOS? If you can share the steps, we can test it also.

Hi DaneLLL,

In order to test hls on iOS you need a file to start with. This test case is transcoding h.265 to h.264.

You need to run a web server to serve the files (I server the files from the TX1 using python -m SimpleHTTPServer)

You need a wireless router or access point so that your web server and iOS device are on the same network.

The pipeline I showed earlier creates the HLS files in a directory and the files are served via python http server from the same directory.

HLS files consist of a playlist and the segmented TS video files. So all that is needed is to open Safari on iOS (Chrome on Android) and download the playlist file from the web server. The video should begin playing.

Here is an example url of the playlist file in my setup:

http://192.168.1.36:8000/playlist.m3u8

In this case the TX1 running the webserver has ip address 192.168.1.36 and the web server is listening on port 8000.

I hope that these instructions are sufficient.

Regards,
Rand

Hi Rand,
Thanks for the steps. We will try to reproduce the issue but it probably take some time.

BTW, are you able to do the quick test below?
1 Record a clip via iPhone(or iPad/iPod)
2 Re-mux it into ts
gst-launch-1.0 filesrc location=iOS_sample.MOV ! qtdemux ! h264parse ! mpegtsmux ! filesink location=remux.ts
3 Test HLS

With this the h264 stream is from iOS. Let’s see if it can run well(ideally it has to run well).

Hi DaneLLL,

I will continue my testing.

I think my issue may be related to this previous issue:

https://devtalk.nvidia.com/default/topic/977438/access-unit-delimiter-in-h264-and-h265-stream/

Does version 1.8.2 of gstreamer that is pre installed on the TX1 include the patch to insert AU delimiters?

Thanks,
Rand

Hi Rand,
No, this is an option disabled by default. If you would like to enable it. You have to build libgstomx.so per
https://devtalk.nvidia.com/default/topic/983587/jetson-tx1/gst-omx-plugin-build-error/post/5059343/#5059343

BTW, I am able to run ‘python -m SimpleHTTPServer’ on TX1, but I am not able to download
http://192.168.1.36:8000/playlist.m3u8

Please attach playlist.m3u8 here for our reference. And what is the path to put playlist.m3u8 and the ts files? Are these put under /home/ubuntu/?

Hi Rand,
We have confirmed the issue is due to lack of AU delimiter in the bitstream. Please build libgstomx.so with AU delimiter enabled and give it a try.

Attach the good playlist set we generated for reference.
NVIDIA_aud.zip (4.65 MB)

DaneLLL,

I would like to compile the libgstomx.so

However, the post you link to describes problems with the instructions. I tried to follow the readme and the instructions mentioned in the post.

However, I am running into problems.

  1. The readme says to create a link

  2. sudo ln -s /usr/lib/arm-linux-gnueabihf/libgstnvegl-1.0.so.0 /usr/lib/arm-linux-gnueabihf/libgstnvegl-1.0.so

My TX1 does not have a /usr/lib/arm-linux-gnueabihf directory. My TX1 doesn’t have any arm-linux directory under /usr/lib

Also the post mentions needing the 64 bit version of the libraries to build with. For example, the readme says this library is needed

libffi6_3.1~rc1+r3.0.13-12_armhf.deb

So for 64 bit arm I tried the following and the package was not found:

ubuntu@tegra-ubuntu:~/nvidia-src/sources$ sudo apt-get install libffi6_3.1~rc1+r3.0.13-12_arm64.deb
Reading package lists… Done
Building dependency tree
Reading state information… Done
E: Unable to locate package libffi6_3.1~rc1+r3.0.13-12_arm64.deb
E: Couldn’t find any package by glob 'libffi6_3.1~rc1+r3.0.13-12_arm64.deb

Can you provide updated instructions to build the library on a TX1 running 64 bit Ubuntu 16.04?

Note that “arm-linux-gnueabihf” is for 32-bit ARMv7, while a TX1 is 64-bit ARMv8-A. Substitute “aarch64-linux-gnu” (this is present on a JTX1 because aarch64-linux-gnu is 64-bit).

I have this package installed on the JTX1 (I don’t know if this was from the JetPack extra libraries or from one of the other repositories I enabled):

# dpkg --list | egrep -i libffi
ii  libffi6:arm64                                 3.2.1-4                                             arm64        Foreign Function Interface library runtime

Note that this is package “libffi6”…no versioning is used in the package name so far as 3.2.1-4 goes.

DaneLLL,

I saw that there was another readme in the gstomx1_src_quick_fix_r24_2_1.zip file. I tried following this readme. But there is still a problem with creating the link. I get the following error when I try to compile.

/usr/bin/ld: cannot find -lgstnvegl-1.0
collect2: error: ld returned 1 exit status
Makefile:632: recipe for target ‘libgstomx.la’ failed
make[2]: *** [libgstomx.la] Error 1
make[2]: Leaving directory ‘/home/ubuntu/gstomx1_src/gst-omx1/omx’
Makefile:526: recipe for target ‘all-recursive’ failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory ‘/home/ubuntu/gstomx1_src/gst-omx1’
Makefile:458: recipe for target ‘all’ failed
make: *** [all] Error

Can you tell me how to resolve this issue?

Thanks,
Rand

linuxdev,

Thanks for pointing this out. It looks like libffi6 was already installed. I have a list to install:

libffi-dev_3.0.11~rc1-5_armhf.deb
libglib2.0-0_2.40.0-2_armhf.deb
libglib2.0-dev_2.40.0-2_armhf.deb
libgstreamer1.0-0_1.2.3-1_armhf.deb
libgstreamer1.0-dev_1.2.3-1_armhf.deb
libgstreamer-plugins-base1.0-0_1.2.3-1_armhf.deb
libgstreamer-plugins-base1.0-dev_1.2.3-1_armhf.deb
libpcre3_8.31-2ubuntu2_armhf.deb
libpcre3-dev_8.31-2ubuntu2_armhf.deb
libxml2_2.9.1+dfsg1-3ubuntu4_armhf.deb
libxml2-dev_2.9.1+dfsg1-3ubuntu4_armhf.deb
zlib1g_1.2.8.dfsg-1ubuntu1_armhf.deb
zlib1g-dev_1.2.8.dfsg-1ubuntu1_armhf.deb
libgegl-dev_0.2.0-4ubuntu1_armhf.deb

To install them I am using commands like

sudo apt-get install libffi-dev

This should give me the latest 64 bit arm version because the TX1 is a 64 bit arm.

Yes, libffi-dev without the rest of the versioning or architecture stuff (32-bit armhf versus 64-bit aarch64/arm64) is the correct way to name packages.

You may need to enable repositories via “/etc/apt/sources.list” and “sudo apt update” for some packages to show up…I don’t know which repos make which package available. There are also alternate ways of adding repos other than sources.list (those other means more or less just edit this file or insert content from sources.list.d subdirectory).

So I have the packages installed but I am still having a problem with /usr/lib/arm-linux0gnueabihf

I am not sure what is supposed to create this directory.

I get the following error trying to create a ln as directed in the readme.

sudo ln -s /usr/lib/arm-linux-gnueabihf/libgstnvegl-1.0.so.0 /usr/lib/arm-linux-gnueabihf/libgstnvegl-1.0.so
ln: failed to create symbolic link ‘/usr/lib/arm-linux-gnueabihf/libgstnvegl-1.0.so’: No such file or directory

There is no gnueabihf for 64-bit. Substitute aarch64-linux-gnu in place since this is 64-bit.

linuxdev,

Thanks again.

Now I am getting a compilation error when I try to follow the instructions to enable the AUD, which is the reason I want to compile the library. (https://devtalk.nvidia.com/default/topic/977438/access-unit-delimiter-in-h264-and-h265-stream/)

I get the following error

gstomxh264enc.c:212:34: error: ‘true’ undeclared (first use in this function)
oEncodeProp.bInsertAUD = true;

I am going to use the c convention and go ahead and replace true with the number 1 because anything non zero in C evaluates to true. I will give it a try.

Regards,
Rand

I have compiled libgstomx.so according to instructions. I copied the file to

/usr/lib/aarch64-linux-gnu/gstreamer-1.0/

Now I am getting two errors with a pipeline that used to produce output segment files.

  1. When I include audio in the pipeline I see this:

WARNING: erroneous pipeline: no element “avdec_aac”

So something about following the build instructions has now made it so that gstreamer cannot find the avdec_aac element. This used to work.

  1. I try the pipeline without the audio track I get this error.

ERROR: from element /GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0: Internal data stream error.
Additional debug info:
gstomxvideodec.c(2852): gst_omx_video_dec_loop (): /GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0:
stream stopped, reason not-supported
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …

So something about following the build instructions has made the 265decoder say that the stream is not compatible. This used to work.

DaneLL, can you help resolve these issues?

Thanks,
Rand

Hi Rand,

It seems to be removed from gstreamer1.0-libav. We figured out another command:
https://devtalk.nvidia.com/default/topic/988771/jetson-tx1/tsdemux-failure-with-audio-and-video-/post/5061867/#5061867

We will resolve it and ensure it being build-able in next release.
Please use the libgstomx.so attached for r24.2.1. It adds the encode property ‘insert-aud’.
command example:

gst-launch-1.0 videotestsrc num-buffers=90 ! 'video/x-raw,format=I420' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! omxh264enc insert-aud=true ! 'video/x-h264,stream-format=byte-stream' ! filesink location=wAUD.h264

libgstomx.so.txt (304 KB)

DaneLLL,

I am now able to playback HLS video on iOS. Thanks.

I still don’t understand what happened to the avdec_aac element. Why does recompiling libgstomx.so eliminate this element? Was it one of the package update commands that pulled down a newer version of gstreamer that removed this element?

If that is the case it would mean the .deb package maintainers removed the element.

According to the release notes for 1.6 avdec is the prefferred over faad. (src: https://gstreamer.freedesktop.org/releases/1.6/) I don’t think this changed for 1.8 which is what is installed on the TX1.

Do you agree it was removed by the package maintainers?

Thanks,
Rand