Multimedia Video and UI at the same time Please

[david.r.wyatt]

All I want to do is create a standalone User Interface app on the Nvidia jetson Orin nano/NX dev kit that can be ported to a compatible production carrier board in the Nvidia ecosystem using a mipi and/or usb camera that is supported by the carrier board’s Nvidia compatible BSP.

Oh yeah and I want to monitor screen clicks and/or the gpio pins to change the state of the stream and do other tasks. Maybe put up a splash. Seems so simple.

For JP 5.1.2 (which I must stay at to do GMSL on my carrier’s BSP) I am finding that Gtk3 doesn’t play well with live video, gtk4 doesn’t work on Ub 20.04, and glade is deprecated, qt5 cant be found in the cmake files using the Hello AI examples, which don’t always build properly (but I have fixed most). So it seems I have to build in the sandbox from the jetson-inference build directory.

Seems like gstreamer has some windowing capability but I am not seeing the event threads. Could be wrong. Doesn’t matter b/c it can’t be found outside the sandbox either. (Yes I did do the sudo install @dusty_nv ).

Can anyone please point me to a simple deepstream example that has a realtime video window with a few software and hardware buttons? Maybe a pulldown menu???

This should not be so hard please. Seems like a first use case. c++ or python I dont care.

The Nvidia argus camera source is crazy complicated for this simple use case.

Best - David

[junshengy]

Please refer this topic. I modified the deepstream-test1-app.c to work with GTK

Send video to GTK window and not deepstream-test1-app.c standard output window

This is the method of gtk±2.0, which is outdated. Please refer this documetion about GstOverlay. Here is a patch for deepstream-test1-app.c , I had tried, It’s can work normal. out.patch (4.6 KB) This is not a problem about DeepStream. Perhaps more information can be obtained from the gstreamer or gtk forums

You may need to add controls and menus yourself.

However, gstreamer already provides methods to work with GUIs,

https://gstreamer.freedesktop.org/documentation/video/gstvideooverlay.html?gi-language=c#gstvideooverlay-and-gtk

[david.r.wyatt]

thank you junshengy for pointing out the GstVideoOverlay and Gtk+ scenario. Its not clear to me where the gpio monitor would occur, if for instance we wanted to stop the stream and restart the stream from a state change on a gpio pin. But I will investigate.

Same is true for the patch when gtk_main () takes control.

Correct it is not a deepstream problem. I ask for deepstream b/c isnt that why we are here? I was able to get my deepstream_test1_app to build and render to a gtk widget on a file but the buttons never showed up. I will try yours thanks :-) and report back.

This all started when I tried and reproduced the problem in

(Deepstream fails in Gtk3 window with additional Widgets)

But the final solution was never reported. I see you were helping there too.

[junshengy]

Have you tried this gtk program? It works fine. The controls are displayed correctly,I use AGX Orin, DS-7.1 and Jetpack 6.1

Deepstream fails in Gtk3 window with additional Widgets

This seems to be a GTK problem, I don’t know much about it. Please try the solution below, I think it is caused by nv3dsink being incompatible with GTK. You can try to go to the GTK community to discuss the problem. /* * SPDX-FileCopyrightText: Copyright (c) 2018-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved. * SPDX-License-Identifier: LicenseRef-NvidiaProprietary * * NVIDIA CORPORATION, its affiliates and licensors retain all intellectual * property and proprietary rights in…

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

[david.r.wyatt]

The solution we ultimately used in JP5.1.2 is to use QT5 like in the hello ai world nvidia example camera-capture. You can add the deepstream yourself and more widgets.