GStreamer NVV4L2H264enc fails at runtime with cross compiler

Hello all,

I’m building an application that uses Unreal Engine 5 running on the Xavier NX in Jetpack 5. The actual UE5 application and GStreamer pipeline works fine when using conventional gstreamer elements eg: x264enc.

But when I include Jetpack 5 specific components, specifically nvv4l2h264enc , a runtime error occur related to memory allocation. (See stack trace below)

The elements are being constructed via a parse launch string command, and I also include the nvidia specific .so files in the Unreal Build scripts.

Here’s a list of some debugging steps I took:

  1. As a test, I made at Jetpack 5 natively compiled gstreamer pipeline. It launches and works well. (See code below)

  2. I then created a static library output of the above mentioned program, and built an executable that was compiled natively on Jetpack 5 which consumed my statically(and dynamically) built GStreamer pipeline. This works fine.

  3. But when I transfer the either the static(.a) or dynamic(.s0) library , into Unreal and calling the function that invokes the pipeline. The same exact error occur at runtime (See stack trace error). Keep in mind the program is built in Unreal on a Windows 10 machine.

Here’s my question, does the native Jetpack5 compiler/linker toolchain have special flags/features that allow access to NVENC accelerated gstreamer components?

Does this mean that in order to take advantage of any Jetson specific hardware acceleration. All programs must be compiled with the native toolchain on the Xavier NX, even if the (hardware accelerated) libraries are dynamically linked via .so files and executed at runtime.

Are there any workarounds for this issue?

test.cpp of Gstreamer pipeline:

#include <gst/gst.h>

  

int main2(int arg, char *argv[]) {    

        GstElement *pipeline = nullptr;

    GstBus *bus = nullptr;

    GstMessage *msg = nullptr;

  

    // gstreamer initialization

    gst_init(&arg, &argv);

    // building pipeline

    pipeline = gst_parse_launch(

            "videotestsrc is-live=true ! videoconvert ! video/x-raw,format=NV12 ! nvvidconv ! nvv4l2h264enc ! h264parse ! avdec_h264 ! videoconvert ! autovideosink",

            nullptr);

  

    // start playing

    gst_element_set_state(pipeline, GST_STATE_PLAYING);

  

    //wait until error or EOS ( End Of Stream )

    bus = gst_element_get_bus(pipeline);

    msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE,

                                     static_cast<GstMessageType>(GST_MESSAGE_ERROR | GST_MESSAGE_EOS));

  

    // free memory

    if (msg != nullptr)

        gst_message_unref(msg);

    gst_object_unref(bus);

    gst_element_set_state(pipeline, GST_STATE_NULL);

    gst_object_unref(pipeline);

  

    return 0;

  

}

Stacktrace of error when trying to use nvv4l2h264enc:

Fatal error: [File:.\Runtime/Core/Private/HAL/MallocBinned2.cpp] [Line: 1430] 
FMallocBinned2 Attempt to realloc an unrecognized block 0xfffeec050000   canary == 0x0 != 0xe3
Signal 11 caught.
Malloc Size=262146 LargeMemoryPoolOffset=262162 
CommonUnixCrashHandler: Signal=11
[2023.04.20-20.54.05:976][688]LogCore: === Critical error: ===
Unhandled Exception: SIGSEGV: invalid attempt to access memory at address 0x0000000000000003

[2023.04.20-20.54.05:976][688]LogCore: Fatal error: [File:.\Runtime/Core/Private/HAL/MallocBinned2.cpp] [Line: 1430] 
FMallocBinned2 Attempt to realloc an unrecognized block 0xfffeec050000   canary == 0x0 != 0xe3
0x0000000005ac1e74 UnrealHelloWorld!void DispatchCheckVerify<void, FMallocBinned2::CanaryFail(FMallocBinned2::FFreeBlock const*) const::$_78, FLogCategoryLogMemory, char16_t [84], void*, int, EBlockCanary>(FMallocBinned2::CanaryFail(FMallocBinned2::FFreeBlock const*) const::$_78&&, FLogCategoryLogMemory const&, char16_t const (&) [84], void* const&, int const&, EBlockCanary const&)()
0x0000000005ac08b4 UnrealHelloWorld!FMallocBinned2::GetAllocationSizeExternal(void*, unsigned long&)()
0x0000000005b24178 UnrealHelloWorld!FMallocPoisonProxy::Free(void*)()
0x0000000005af13f0 UnrealHelloWorld!FMemory::Free(void*)()
0x0000ffffb187bab8 libc.so.6!__cxa_finalize(+0x177)
0x0000fffedf893690 libnvfnet.so!UnknownFunction(0x768f)
0x0000ffffb1e7ff18 ld-linux-aarch64.so.1!UnknownFunction(0x12f17)
0x0000ffffb194f07c libc.so.6!_dl_catch_exception(+0xd3)
0x0000ffffb1e80500 ld-linux-aarch64.so.1!UnknownFunction(0x134ff)
0x0000ffffb1e81020 ld-linux-aarch64.so.1!UnknownFunction(0x1401f)
0x0000ffffb194f01c libc.so.6!_dl_catch_exception(+0x73)
0x0000ffffb194f0e8 libc.so.6!_dl_catch_error(+0x3f)
0x0000ffffb1e09838 libdl.so.2!UnknownFunction(0x1837)
0x0000ffffb1e091d0 libdl.so.2!dlclose(+0x27)
0x0000fffeeb5fca14 libv4l2.so.0!UnknownFunction(0x8a13)
0x0000fffeeb5f7270 libv4l2.so.0!v4l2_fd_open(+0xb7)
0x0000fffeeb74e9e4 libgstnvvideo4linux2.so!UnknownFunction(0x2f9e3)
0x0000fffeeb73b444 libgstnvvideo4linux2.so!UnknownFunction(0x1c443)
0x0000fffeeb74c378 libgstnvvideo4linux2.so!UnknownFunction(0x2d377)
0x0000ffff60d3f310 libgstvideo-1.0.so.0!UnknownFunction(0x5630f)
0x0000ffffb1b55ddc libgstreamer-1.0.so.0!gst_element_change_state(+0x43)
0x0000ffffb1b5651c libgstreamer-1.0.so.0!UnknownFunction(0x6851b)
0x0000ffffb1b2c434 libgstreamer-1.0.so.0!UnknownFunction(0x3e433)
0x0000ffffb1b55ddc libgstreamer-1.0.so.0!gst_element_change_state(+0x43)
0x0000ffffb1b5651c libgstreamer-1.0.so.0!UnknownFunction(0x6851b)
0x0000ffffb1b2c434 libgstreamer-1.0.so.0!UnknownFunction(0x3e433)
0x0000ffffb1b55ddc libgstreamer-1.0.so.0!gst_element_change_state(+0x43)
0x0000ffffb1b5651c libgstreamer-1.0.so.0!UnknownFunction(0x6851b)
0x0000ffffb1a96400 libgstrtspserver-1.0.so.0!UnknownFunction(0x1b3ff)
0x0000ffffb1a96ef4 libgstrtspserver-1.0.so.0!UnknownFunction(0x1bef3)
0x0000ffffb1d1f804 libglib-2.0.so.0!g_main_context_dispatch(+0x133)
0x0000ffffb1d1fbb4 libglib-2.0.so.0!UnknownFunction(0x51bb3)
0x0000ffffb1d1fef0 libglib-2.0.so.0!g_main_loop_run(+0x7f)
0x0000ffffb1a936a8 libgstrtspserver-1.0.so.0!UnknownFunction(0x186a7)
0x0000ffffb1d49e18 libglib-2.0.so.0!UnknownFunction(0x7be17)
0x0000ffffb1d4947c libglib-2.0.so.0!UnknownFunction(0x7b47b)
0x0000ffffb1e23624 libpthread.so.0!UnknownFunction(0x7623)
0x0000ffffb191649c libc.so.6!UnknownFunction(0xd149b)

Gstreamer Debug Log of crash:

You may try:

videotestsrc ! videoconvert ! video/x-raw,format=NV12 ! nvvidconv ! nvv4l2h264enc ! h264parse ! avdec_h264 ! nvvidconv ! autovideosink

# Or better use nvv4l2decoder
videotestsrc ! video/x-raw,width=320,height=240,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! nvv4l2h264enc ! h264parse ! nvv4l2decoder ! nvvidconv ! queue ! autovideosink

Hmm, I see errors using these…not sure what is the issue. I may later update.
[EDIT: The issue seems related to using is-live flag and pipeline not going through prerolling, but no idea why so far.]

Interesting

For the avdec_h264 to work, I had installed some dependencies first. Otherwise, the element doesn’t exist. Reference to dependency install It should work with the is-live flag on app-src.

The command below worked for me after installing the deps in the link above

gst-launch-1.0 videotestsrc is-live=true ! videoconvert ! video/x-raw,format=NV12 ! nvvidconv ! nvv4l2h264enc ! h264parse ! avdec_h264 ! nvvidconv ! autovideosink

Using the nvv4l2decoder was unsuccessful, even after removing the is-live=flag. Here’s the exact command I ran that lead to failure:

gst-launch-1.0 videotestsrc ! video/x-raw,width=320,height=240,framerate=30/1 ! nvvidconv ! "video/x-raw(memory:NVMM),format=NV12" ! nvv4l2h264enc ! h264parse ! nvv4l2decoder ! nvvidconv ! queue ! autovideosink

EDIT: Failure meaning the autovideosink window never renders. It just gets stuck here.

Can you provide the exact gst-launch command used for the nvv4l2decoder?

Seems you would just need to do:

    // start playing
    gst_element_set_state(pipeline, GST_STATE_READY);
    gst_element_set_state(pipeline, GST_STATE_PAUSED);
    gst_element_set_state(pipeline, GST_STATE_PLAYING);

with your original pipeline.

No idea why using nvv4l2decoder may not be able to keep sync. Maybe @DaneLLL could comment on this.
Disabling sink sync works in my case :

videotestsrc is-live=true ! video/x-raw, width=320,height=240, framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! nvv4l2h264enc ! h264parse ! queue ! nvv4l2decoder ! nvvidconv ! autovideosink sync=0

Hmm. Disabling the sync doesn’t allow me to launch that pipeline.

But regardless, I accidently diverged from the root error that I was trying to solve.

When a C++ program is built via the native Jetpack5 toolchain. Specifically using GStreamer’s gst_parse_launch() api. The pipeline works as expected.

But when a C++ program (also using gst_parse_launch) is created via Unreal Engine 5’s Clang cross compile toolchain. It works when using normal Gstreamer elements eg x264enc.

But when I try to use a Jetpack 5 specific element in the Unreal program, such as nvv4l2h264enc . A runtime crash occurs. (The stack trace is in the first post).

Do you have any insight in what might cause this?

I was under the impression that since everything in GStreamer is a shared library, The Xavier NX would resolve the needs of the nvv4l2h264enc element at runtime.

EDIT: I also forgot to say thank you for helping me with this issue. It’s really appreciated!

Currently no… I haven’t been playing with UnrealEngine since a few years. If you have a way to reproduce your case I may try to reproduce, but not sure I can solve it. You may also try to preload libgomp as here Unable find avdec_h264 after intsalled gstreamer plug-ins into Jetson AGX Orin - #6 by Honey_Patouceul

I’ll try tomorrow with a Xavier NX running R35.2 (so far sharing testing on AGX Orin with R35.1).

1 Like

Hi,
Please try

$ DISPLAY=:0 gst-launch-1.0 videotestsrc num-buffers=166 is-live=true ! videoconvert ! video/x-raw,format=NV12 ! nvvidconv ! nvv4l2h264enc ! h264parse ! matroskamux ! queue ! matroskademux ! h264parse ! nvv4l2decoder ! nv3dsink

Looks like it needs to mux the stream so that the timestamp information is present for decoding.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

I finally figured out what’s wrong! Basically removing a shared library on the Xavier fixed the issue

$ sudo mv /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus.so /usr/lib/aarch64-linux-gnu/tegra/libv4l2_nvargus_back.so

Here’s the reference thread that I used to figure it out.

It has something to do with a bug in the libv4l2_nvargus.so being exposed when it’s called in a gstreamer program that’s running as a child process.

I was able to loosely figure this out by running ldd on /user/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideo4linux2.so and poking at the nvh264enc source code

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.