Unable to stream video using Gstreamer (no element nvv4l2h264enc )

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
1

I’m trying to send my camera video stream to my windows pc. The windows pc will get the video stream using gstreamer and then convert it to an opencv frame to later use. I tried sending it using the command described here: Gstreamer TCPserversink 2-3 seconds latency - #4 by Bazziil however it just raises an error: WARNING: erroneous pipeline: no element "nvv4l2h264enc" .
I’m using Raspberry Pi HQ camera which is based on IMX477R. The camera works when doing: nvgstcapture-1.0 or gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! ximagesink

How do I install the encoder?

2

found a way to stream the video using ffmpeg and v4l2 using this command: ffmpeg -f v4l2 -framerate 30 -video_size 1920x1080 -input_format nv12 -i /dev/video0 -c:v h264_nvenc -preset ultrafast -tune zerolatency -b:v 4M -f mpegts udp://127.0.0.1:12940.

However I cant seem to get the video stream using ffplay udp://127.0.0.1:12940. ffplay starts but doesnt get the video stream.

3

nevermind, the ffmpeg command wasnt working but I didnt notice it since I was running it as a subprocess from python(with outputs hidden).

ffmpeg -f v4l2 -framerate 30 -video_size 1920x1080 -input_format nv12 -i /dev/video0 -c:v h264_nvenc -b:v 4M -f mpegts udp://127.0.0.1:12940

[video4linux2,v4l2 @ 0xaaaafaca8430] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0xaaaafaca8430] Time per frame unknown
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1920x1080, 1000k tbr, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
Finishing stream 0:0 without any data written to it.
[h264_nvenc @ 0xaaaafacab820] Cannot load libnvidia-encode.so.1
[h264_nvenc @ 0xaaaafacab820] The minimum required Nvidia driver for nvenc is (unknown) or newer
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Exiting normally, received signal 2.
4

I’m testing both sending and receiving data on my jetson right now.

doing: gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! 'video/x-raw(memory:NVMM), format=NV12' ! nvvidconv ! 'video/x-raw, format=I420' ! x264enc ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=12940 sync=false for sending

and doing: gst-launch-1.0 udpsrc port=12940 ! application/x-rtp, payload=96 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! queue ! autovideosink sync=false
for receiving works, but the delay is 5 - 6 seconds. I think the delay is caused by slow encoding.

I tried using other encoders but they all raise errors saying WARNING: erroneous pipeline: no element "nvv4l2h264enc", WARNING: erroneous pipeline: no element "nvv4l2h265enc", WARNING: erroneous pipeline: no element "omxh264enc" etc.

I tried installing all the necessary dependencies but I still cant use any other codec. What do I need to do? Here’s the command I used: sudo apt-get install gstreamer1.0-plugins-bad gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly

6

How can I send the output of gstream recording to ffmpeg

7

does anyone have any solutions to this, I can send video stream to 127.0.0.1:12949 but when I try to open cv2.VideoCapture() with “udp://127.0.0.1:12949” it just says could not open video stream.

Here’s output of cv2.getBuildInformation():

General configuration for OpenCV 4.6.0 =====================================
  Version control:               b0dc474

  Platform:
    Timestamp:                   2024-07-24T13:01:43Z
    Host:                        Linux 5.15.122-tegra aarch64
    CMake:                       3.30.1
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/gmake
    Configuration:               Release

  CPU/HW features:
    Baseline:                    NEON FP16

  C/C++:
    Built as dynamic libs?:      NO
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 11.4.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
    Linker flags (Debug):        -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          /usr/lib/aarch64-linux-gnu/liblapack.so /usr/lib/aarch64-linux-gnu/libcblas.so /usr/lib/aarch64-linux-gnu/libatlas.so /usr/lib/aarch64-linux-gnu/libjpeg.so /usr/lib/aarch64-linux-gnu/libpng.so /usr/lib/aarch64-linux-gnu/libtiff.so /usr/lib/aarch64-linux-gnu/libz.so dl m pthread rt
    3rdparty dependencies:       libprotobuf ade ittnotify libwebp libopenjp2 IlmImf quirc tegra_hal

  OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java python2 ts
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:                           NONE
    VTK support:                 NO

  Media I/O: 
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.37)
    TIFF:                        /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 / 4.3.0)
    JPEG 2000:                   build (ver 2.4.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                NO
    GStreamer:                   YES (1.20.3)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Lapack:                      YES (/usr/lib/aarch64-linux-gnu/liblapack.so /usr/lib/aarch64-linux-gnu/libcblas.so /usr/lib/aarch64-linux-gnu/libatlas.so)
    Eigen:                       YES (ver 3.4.0)
    Custom HAL:                  YES (carotene (ver 0.0.1))
    Protobuf:                    build (3.19.1)

  OpenCL:                        YES (no extra features)
    Include path:                /tmp/tmp.B8rPyKMriY/opencv-python-master/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.10.12)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.10.so (ver 3.10.12)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.21.5)
    install path:                python/cv2/python-3

  Python (for build):            /usr/bin/python2.7

  Java:                          
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /tmp/tmp.B8rPyKMriY/opencv-python-master/_skbuild/linux-aarch64-3.10/cmake-install
-----------------------------------------------------------------

(I built opencv locally so it has gstreamer support, it still doesnt work.
(It cannot open the video stream capture)

8

Hi,
Orin Nano does not have hardware encoder. So if you need hardware encoding, would suggest use Orin NX or Xavier NX.

9

Thanks, but could I ask, how could I use ffmpeg with my orin nano(for raspberyy pi high quality camera)? Just specifying /dev/video0 doesnt really work. What do i need to give for it to work?(or is it even possible to do this)

10

Hi,
We have the Argus stack to support Bayer sensors such as RPi camera v2 and RPi Hi Q camera. You would need to use gstreamer or jetson_multedia_api. It is not supported in ffmpeg.

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.