Hi everybody, I’m experiencing a quite strange issue with gstreamer-1.0 and TX2. In the project I’m currently working on, I’ve got a bunch of jetsons employed in a distributed vision system. These boards have the official nvidia distro (L4T 28.2.1, ubuntu 16.04 with kernel 4.4.38). They have the tegra-customized (by nvidia) gstreamer plugins in /usr/lib/aarch64-linux-gnu/gstreamer-1.0 folder and I’ve installed all the other gstreamer packages (tools, libs and base-good-bad plugins) through apt-get. That is basically what nvidia-customized gst-install does, with the only difference that this script compiles those tegra-customized plugins from sources (there are several ifdefs USE_OMX_TARGET_TEGRA in their code) instead of decompressing them out from nvgstapps.tbz2.
My gstreamer pipeline is the following:
appsrc name=videosource is-live=true stream-type=GST_APP_STREAM_TYPE_STREAM format=GST_FORMAT_TIME ! video/x-raw, format=BGR, width=1920, height=1080, framerate=30/1 ! videoconvert ! videoscale ! video/x-raw, width=1280, height=720, framerate=15/1 ! omxh265enc ! queue max-size-buffers=0 max-size-time=0 flush-on-eos=true ! mux. alsasrc name=audiosource device=hw:2,0 ! audioconvert ! vorbisenc ! queue max-size-buffers=0 max-size-time=0 flush-on-eos=true ! mux. matroskamux name=mux ! filesink location=<file location> name=filesink
As you can see, the goal here is to collect frames from the camera (I’m using OpenCV for that), feed them to the pipeline through appsrc, convert and rescale them, encode them with h265 (using tegra acceleration), capture audio with camera microphone and finally mux everything into a .mkv file.
In my dev board everything is working fine. All gstreamer plugins are correctly shown by gst-inspect and there are no blacklisted ones. The application starts collecting frames and feeding them to the pipeline. I constantly log out all internal gstreamer messages, and they are all visualized correctly. The video is marked as finished by the application, gst_event_new_eos is sent and asyncronously received. Finally the video is correctly closed.
For deployment purposes, we created a custom distro with debootstrap. Then, we use apply-binaries to provide the iso with nvidia binaries. Also, we use the exact same kernel of L4T 28.2.1. Our default user is root. I’ve installed gstreamer through apt-get, recreating the same context as before. All gstreamer plugins are correctly shown by gst-inspect and there are no blacklisted ones. The weird behavior that happens here is that when I run the application on the other boards, a huge memory consumption is reported.
By looking at the kernel messages, it turns out that it fails to load the falcon nvenc firmware. The reported error is the following:
[ 277.499132] xhci-tegra 3530000.xhci: tegra_xhci_mbox_work mailbox command 6
[ 277.601864] falcon 154c0000.nvenc: Direct firmware load for tegra18x/nvhost_nvenc061.fw failed with error -2
[ 277.611905] falcon 154c0000.nvenc: Falling back to user helper
[ 277.619602] falcon 154c0000.nvenc: failed to get firmware
[ 277.625097] falcon 154c0000.nvenc: failed to get firmware
[ 277.630554] falcon 154c0000.nvenc: nvhost_flcn_init_sw: failed
As a consequence, it seems that it’s falling back using the same omxh265 encoder but without hw acceleration, which is much more memory consuming (for some reason).
If I run for some seconds the following test gstreamer pipeline:
videotestsrc ! omxh265enc ! queue ! mux. matroskamux name=mux ! fakesink
no error is reported in the kernel and the hw acceleration is used correctly. Now if I run my own application everything starts working correctly. My guess is that there is some firmware loading which needs some time to finish. Can you give me some hints regarding which is and how can I consistently wait for it before starting my pipeline?
Thank you in advance.