Problem with software encoder on JETSON ORIN

walid.brini · October 6, 2023, 11:19am

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) NVIDIA-Jetson-Orin-Nano-Developer-kit
• DeepStream Version 6.3
• JetPack Version (valid for Jetson only) JetPack 5.1.2

Hello I’m trying to run the deepstream-test5 app i’ve noticed that the Jetson Orin doesnt support HW encoding, even if i switch to software encoding I get 0 FPS.
This is the output :

sudo ./deepstream-test5-app -c configs/custom.txt 

 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***


Runtime commands:
        h: Print this help
        q: Quit

        p: Pause
        r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.

** INFO: <bus_callback:239>: Pipeline ready

** INFO: <bus_callback:225>: Pipeline running

GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 4 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Active sources : 1

**PERF:  FPS 0 (Avg)
Wed Oct  4 12:18:47 2023
**PERF:  30.14 (3.22)
Active sources : 0
Wed Oct  4 12:18:52 2023
**PERF:  0.00 (1.55)
Active sources : 0
Wed Oct  4 12:18:57 2023
**PERF:  0.00 (1.02)

My config is below.

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP 5=CSI
type=5
camera-width=1280
camera-height=720
camera-fps-n=30
camera-fps-d=1


[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=nvdrmvideosink
type=4
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=1
sync=0
bitrate=4000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
# set profile only for hw encoder, sw encoder selects profile based on sw-preset
profile=2
# set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400

Is there a way to solve this ?

deepstream_app.txt (3.3 KB)

fanzh · October 7, 2023, 2:20am

using your deepstream_app.txt, I can’t reproduce this issue. here is the output log
argus.txt (2.2 KB)
please do some tests.

please check if the source’s fps is stable, please refer to test command:
gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! nvvideoconvert ! fpsdisplaysink
noticing the fps dropped from 30 to 0, please refer to DeepStream SDK FAQ - #12 by bcao 4 to measure the latency of the pipeline components.

walid.brini · October 9, 2023, 9:39am

Thanks for your reply, i was able to run the app with the RTSP Sink but i’m still having low fps. Is it because of the software encoder or is there a way to solve it ?

Output with RTSP Sink diasabled :

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 3280 H = 2464 
   seconds to Run    = 0 
   Frame Rate = 21.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Active sources : 1

**PERF:  FPS 0 (Avg)
Mon Oct  9 10:36:01 2023
**PERF:  21.41 (21.22)
Active sources : 1
Mon Oct  9 10:36:06 2023
**PERF:  21.01 (21.10)
Active sources : 1
Mon Oct  9 10:36:11 2023
**PERF:  21.00 (21.06)
Active sources : 1
Mon Oct  9 10:36:16 2023
**PERF:  21.00 (21.05)
Active sources : 1
Mon Oct  9 10:36:21 2023
**PERF:  21.00 (21.04)
Active sources : 1
Mon Oct  9 10:36:26 2023
**PERF:  21.00 (21.03)

Output with RTSP Sink enabled :
But i noticed when i enable RTSP sink the fps drops and it is unstable:

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 3280 H = 2464 
   seconds to Run    = 0 
   Frame Rate = 21.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Active sources : 1

**PERF:  FPS 0 (Avg)
Mon Oct  9 10:37:19 2023
**PERF:  19.74 (19.39)
Active sources : 1
Mon Oct  9 10:37:24 2023
**PERF:  17.55 (18.44)
Active sources : 1
Mon Oct  9 10:37:29 2023
**PERF:  10.60 (15.65)
Active sources : 1
Mon Oct  9 10:37:34 2023
**PERF:  19.31 (16.62)
Active sources : 1
Mon Oct  9 10:37:39 2023
**PERF:  13.83 (16.08)
Active sources : 1
Mon Oct  9 10:37:44 2023
**PERF:  9.28 (14.95)
Active sources : 1
Mon Oct  9 10:37:49 2023
**PERF:  19.08 (15.53)
Active sources : 1
Mon Oct  9 10:37:54 2023
**PERF:  15.95 (15.44)
Active sources : 1
Mon Oct  9 10:37:59 2023
**PERF:  10.27 (14.94)```

fanzh · October 10, 2023, 7:03am

from the log, 720p resolution was not chosen, please refer to gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM), width=1280, height=720’ ! …

walid.brini · October 10, 2023, 7:21am

Even if i switch to the 720p resolution i still get the same fps drop :

 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***


Runtime commands:
        h: Print this help
        q: Quit

        p: Pause
        r: Resume

** INFO: <bus_callback:239>: Pipeline ready

** INFO: <bus_callback:225>: Pipeline running

GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 4 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Active sources : 1

**PERF:  FPS 0 (Avg)
Tue Oct 10 08:14:05 2023
**PERF:  18.26 (17.89)
Active sources : 1
Tue Oct 10 08:14:10 2023
**PERF:  6.29 (11.93)
Active sources : 1
Tue Oct 10 08:14:15 2023
**PERF:  9.18 (10.93)
Active sources : 1
Tue Oct 10 08:14:20 2023
**PERF:  9.18 (10.49)```

This is my current config :
deepstream_app.txt (3.7 KB)

fanzh · October 10, 2023, 7:36am

if testing performance, please use fakesink first. if fakesink 's fps is fine, it should be output sink 's problem.

walid.brini · October 10, 2023, 7:42am

This is the output with fakesink enabled, the problem should be related to RTSP output sink

GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 4 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Active sources : 1

**PERF:  FPS 0 (Avg)
Tue Oct 10 08:39:49 2023
**PERF:  30.42 (30.28)
Active sources : 1
Tue Oct 10 08:39:54 2023
**PERF:  30.01 (30.13)
Active sources : 1
Tue Oct 10 08:39:59 2023
**PERF:  30.00 (30.09)
Active sources : 1
Tue Oct 10 08:40:04 2023
**PERF:  30.01 (30.06)
Active sources : 1
Tue Oct 10 08:40:09 2023
**PERF:  30.00 (30.05)

fanzh · October 11, 2023, 7:30am

if using softencoder x264enc, here are some suggestions:

modify nvpmodel to improve performance. please refer to power.
you can modify x264enc’s speed-preset property to imperove encoding speed. the default value is medium.

walid.brini · October 11, 2023, 11:53am

I guess that it did improve after running these commands
sudo nvpmodel -m 0
sudo jetson_clocks

This is the output :

**PERF:  FPS 0 (Avg)
Wed Oct 11 12:51:34 2023
**PERF:  24.49 (24.48)
Active sources : 1
Wed Oct 11 12:51:39 2023
**PERF:  23.30 (23.71)
Active sources : 1
Wed Oct 11 12:51:44 2023
**PERF:  25.44 (24.36)
Active sources : 1
Wed Oct 11 12:51:49 2023
**PERF:  25.14 (24.47)
Active sources : 1
Wed Oct 11 12:51:54 2023
**PERF:  25.39 (24.70)
Active sources : 1
Wed Oct 11 12:51:59 2023
**PERF:  21.87 (24.21)
Active sources : 1
Wed Oct 11 12:52:04 2023
**PERF:  25.22 (24.41)
Active sources : 1
Wed Oct 11 12:52:09 2023
**PERF:  24.39 (24.36)

How can i modify the encoder speed ?

fanzh · October 12, 2023, 1:47am

you can set x264enc’s speed-preset property after creating the element.
the code path is create_udpsink_bin of opt\nvidia\deepstream\deepstream\sources\apps\apps-common\src\deepstream_sink_bin.c
then rebuild the code.

walid.brini · October 12, 2023, 7:30am

I have made this adjustment, and i’m still getting the same number of fps, i tried for the 3280x2464 resolution with the RTSP sink enabled i got the same fps drop:

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 3280 H = 2464 
   seconds to Run    = 0 
   Frame Rate = 21.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Active sources : 1

**PERF:  FPS 0 (Avg)
Thu Oct 12 08:22:55 2023
**PERF:  17.09 (15.63)

Active sources : 1
Thu Oct 12 08:23:00 2023
**PERF:  5.54 (10.34)
Active sources : 1
Thu Oct 12 08:23:05 2023
**PERF:  9.76 (10.22)
Active sources : 1
Thu Oct 12 08:23:10 2023
**PERF:  9.38 (9.95)
Active sources : 1
Thu Oct 12 08:23:15 2023
**PERF:  9.51 (9.88)
Active sources : 1
Thu Oct 12 08:23:20 2023
**PERF:  9.44 (9.80)
Active sources : 1
Thu Oct 12 08:23:25 2023
**PERF:  9.38 (9.74)

This the modification i made :

 if (config->enc_type == NV_DS_ENCODER_TYPE_SW) {
    //bitrate is in kbits/sec for software encoder x264enc and x265enc
    g_object_set (G_OBJECT (bin->encoder), "bitrate", config->bitrate / 1000,
        NULL);
    // added line of code 
    g_object_set(G_OBJECT(bin->encoder), "preset", "ultrafast", NULL);

  } else {
    g_object_set (G_OBJECT (bin->encoder), "bitrate", config->bitrate, NULL);
    g_object_set (G_OBJECT (bin->encoder), "profile", config->profile, NULL);
    g_object_set (G_OBJECT (bin->encoder), "iframeinterval",
        config->iframeinterval, NULL);
  }

Could this be related to the jetson setup or one of the drivers ? should i reinstall or use another jetpack/deepstream version ?

walid.brini · October 12, 2023, 7:37am

Another thing to mention, is when i enable mp4 output with SW encoder i get a stable FPS.

[sink1]
enable=1
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=1
sync=0
#iframeinterval=10
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=out.mp4
source-id=0

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 3280 H = 2464 
   seconds to Run    = 0 
   Frame Rate = 21.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Active sources : 1

**PERF:  FPS 0 (Avg)
Thu Oct 12 08:33:28 2023
**PERF:  21.54 (21.31)
Active sources : 1
Thu Oct 12 08:33:33 2023
**PERF:  21.00 (21.14)
Active sources : 1
Thu Oct 12 08:33:38 2023
**PERF:  21.00 (21.09)
Active sources : 1
Thu Oct 12 08:33:43 2023
**PERF:  21.00 (21.07)

Does this confirm that the problem isnt caused by the software encoder ?

fanzh · October 12, 2023, 8:00am

from the result, low fps should be caused by rtsp sink. you can compare the create_encode_file_bin and create_udpsink_bin, the work flows are different.

walid.brini · October 12, 2023, 8:57am

Does this mean that i have to make changes to the rtsp_sink implementation under deepstream_sink_bin.c file ? Or should i make changes to the RTSP sink group ?

fanzh · October 12, 2023, 9:32am

you can check which element in RTSP sink will cause low fps. AYK, rtph264pay， udpsink are Gstreamer opensource code, you can check if modifying some parameter can accelerate.

walid.brini · October 12, 2023, 9:36am

Is this a common problem with Jetson Orin, or is it related to the current JetPack and DeepStream version? Do you think that by reinstalling JetPack and DeepStream, I can solve this?

fanzh · October 15, 2023, 2:56pm

if using software, both filesink and rtspsink will use Gstreamer opensource plugin. reinstalling JetPack and Deepstream can’t improve fps.

walid.brini · October 16, 2023, 6:30am

It doesn’t make sense that the Jetson Orin has better performance than the Jetson Nano, yet the software encoder performs worse than the hardware encoder in the Nano. Is there another way to have the video output without a drastic decrease in the FPS?

In my case, I need to run DeepStream on four video sources and have the output using RTSP. Do you think that the board isn’t suitable for this kind of task?

fanzh · October 17, 2023, 3:20am

There is no update from you for a period, assuming this is not an issue any more. Hence we are closing this topic. If need further support, please open a new one. Thanks.
Using SW encoding and “nvpmodel -m 0”, I can’t reproduce this low fps issue on Jetson Xavier, please refer to the test results. I did not modify x264enc’s speed-preset.
cfg.txt (3.9 KB)
log.txt (3.1 KB)

system · November 10, 2023, 5:01am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Deepstream FPS drops to 0 When i enable Sink DeepStream SDK	5	736	October 7, 2023
DeepStream 6.2 - RTSP streaming is not working for NVIDIA Jetson AGX Orin kit DeepStream SDK	4	464	May 17, 2023
Change video settings on RTSP streaming Jetson Nano gstreamer	15	4632	March 9, 2022
Nvv4l2h264enc plugin not working on jetson orin nano DeepStream SDK	17	1735	November 10, 2023
DeepStream on Jetson AGX Xavier - Jetpack 4.3 installed with sdkmanager DeepStream SDK	18	1979	October 12, 2021
Jetson Nano CSI Raspberry Pi Camera V2 upside down video when run an example with deepstream-app Jetson Nano	14	5827	October 14, 2021
Jetson Orin Nano Devkit fails to output file / rtsp DeepStream SDK	11	419	December 8, 2023
RTSP stream not work "Stream format not found, dropping the frame" DeepStream SDK rtsp , gstreamer , nvbugs	24	1578	January 3, 2024
Gstreamer reports Raspberry Pi camera streaming at 120fps when in reality it is only 60fps Jetson Nano camera , gstreamer	53	3775	July 19, 2022
RTSP latency does not work with NVSTREAMMUX DeepStream SDK nvbugs	37	5144	January 23, 2022

Problem with software encoder on JETSON ORIN

Related topics