What is limiting the camera framerate (gstreamer)

Nxt-1 · November 30, 2021, 4:15pm

I am debugging framerate issues the team and I recently noticed in our application. Part of the issue is that the reported framerate does not match in different places and the actual number of frames does not correspond with either of them. Our application uses a high speed Alvium CSI-2 camera attached to a jetson nano and is accessed using python gstreamer bindings. Normally the camera runs at 153.917 fps @1408x1088.

Encoded video shows a framerate of 153.917 fps as reported by vlc in the media information → coded tab and using an online tool such as https://www.metadata2go.com/. 1) The actual number of frames does not correspond with that. 2) Opening the video in matlab or windows explorer properties shows 76.96 fps, ie. exactly half.

To make debugging easier and I went back to cli gstreamer using gst-launch-1.0. Using the pipeline below the fpsdisplay sink displays about 135fps. This number also corresponds with the number of frames that is acutally in the encoded output if a proper filesink is used. Full output attached for reference.
Note the reported 135fps is actually double (270), I am not sure if this a completely separate issue or not but I did start a thread for it here
gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)GRAY8' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2vp8enc ! webmmux ! fpsdisplaysink video-sink=fakesink --verbose

nvbuf_utils: Could not get EGL display connection
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)GRAY8, width=(int)1408, height=(int)1088, framerate=(fraction)153/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)1:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)GRAY8, width=(int)1408, height=(int)1088, framerate=(fraction)153/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)1:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1408, height=(int)1088, framerate=(fraction)153/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1408, height=(int)1088, framerate=(fraction)153/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/nvv4l2vp8enc:nvv4l2vp8enc0.GstPad:src: caps = video/x-vp8, profile=(string)0, width=(int)1408, height=(int)1088, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)153/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstMatroskamuxPad:video_0: caps = video/x-vp8, profile=(string)0, width=(int)1408, height=(int)1088, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)153/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
NvMMLiteOpen : Block : BlockType = 7 
Redistribute latency...
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 7 
/GstPipeline:pipeline0/nvv4l2vp8enc:nvv4l2vp8enc0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1408, height=(int)1088, framerate=(fraction)153/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1408, height=(int)1088, framerate=(fraction)153/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)I420
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw, format=(string)GRAY8, width=(int)1408, height=(int)1088, framerate=(fraction)153/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)1:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)GRAY8, width=(int)1408, height=(int)1088, framerate=(fraction)153/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)1:4:7:1, interlace-mode=(string)progressive
NVMEDIA_ENC: bBlitMode is set to TRUE 
/GstPipeline:pipeline0/GstWebMMux:webmmux0: streamable = true
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = video/webm
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/webm
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:src: caps = video/webm(memory:SystemMemory, meta:GstVideoOverlayComposition)
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/webm(memory:SystemMemory, meta:GstVideoOverlayComposition)
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:video_sink: caps = video/webm
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/webm
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
/GstPipeline:pipeline0/GstWebMMux:webmmux0.GstPad:src: caps = video/webm, streamheader=(buffer)< 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000562ad7b1830f42404d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31342e35005741994753747265616d6572204d6174726f736b61206d75786572004461880928b9a498da5c001654ae6b0100000000000046ae010000000000003dd7810183810173c588ac36674daf8901a123e3838363bb0b536e86566964656f00e0010000000000000bb0820580ba8204409a81028686565f56503800 >
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/webm, streamheader=(buffer)< 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000562ad7b1830f42404d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31342e35005741994753747265616d6572204d6174726f736b61206d75786572004461880928b9a498da5c001654ae6b0100000000000046ae010000000000003dd7810183810173c588ac36674daf8901a123e3838363bb0b536e86566964656f00e0010000000000000bb0820580ba8204409a81028686565f56503800 >
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:src: caps = video/webm(memory:SystemMemory, meta:GstVideoOverlayComposition), streamheader=(buffer)< 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000562ad7b1830f42404d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31342e35005741994753747265616d6572204d6174726f736b61206d75786572004461880928b9a498da5c001654ae6b0100000000000046ae010000000000003dd7810183810173c588ac36674daf8901a123e3838363bb0b536e86566964656f00e0010000000000000bb0820580ba8204409a81028686565f56503800 >
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/webm(memory:SystemMemory, meta:GstVideoOverlayComposition), streamheader=(buffer)< 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000562ad7b1830f42404d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31342e35005741994753747265616d6572204d6174726f736b61206d75786572004461880928b9a498da5c001654ae6b0100000000000046ae010000000000003dd7810183810173c588ac36674daf8901a123e3838363bb0b536e86566964656f00e0010000000000000bb0820580ba8204409a81028686565f56503800 >
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay.GstPad:video_sink: caps = video/webm, streamheader=(buffer)< 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000562ad7b1830f42404d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31342e35005741994753747265616d6572204d6174726f736b61206d75786572004461880928b9a498da5c001654ae6b0100000000000046ae010000000000003dd7810183810173c588ac36674daf8901a123e3838363bb0b536e86566964656f00e0010000000000000bb0820580ba8204409a81028686565f56503800 >
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/webm, streamheader=(buffer)< 1a45dfa301000000000000104282857765626d0042878102428581021853806701ffffffffffffff1549a96601000000000000562ad7b1830f42404d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31342e35005741994753747265616d6572204d6174726f736b61206d75786572004461880928b9a498da5c001654ae6b0100000000000046ae010000000000003dd7810183810173c588ac36674daf8901a123e3838363bb0b536e86566964656f00e0010000000000000bb0820580ba8204409a81028686565f56503800 >
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 144, dropped: 0, current: 284,18, average: 284,18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 144, dropped: 0, current: 284,18, average: 284,18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 272, dropped: 0, current: 255,90, average: 270,13
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 272, dropped: 0, current: 255,90, average: 270,13
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 408, dropped: 0, current: 269,04, average: 269,77
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 408, dropped: 0, current: 269,04, average: 269,77
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 544, dropped: 0, current: 271,60, average: 270,22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 544, dropped: 0, current: 271,60, average: 270,22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 680, dropped: 0, current: 270,87, average: 270,35
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 680, dropped: 0, current: 270,87, average: 270,35
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 816, dropped: 0, current: 270,37, average: 270,35
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 816, dropped: 0, current: 270,37, average: 270,35
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 952, dropped: 0, current: 271,39, average: 270,50
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 952, dropped: 0, current: 271,39, average: 270,50
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 1088, dropped: 0, current: 269,86, average: 270,42
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1088, dropped: 0, current: 269,86, average: 270,42
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 1224, dropped: 0, current: 270,52, average: 270,43
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1224, dropped: 0, current: 270,52, average: 270,43
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 1360, dropped: 0, current: 269,84, average: 270,37
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1360, dropped: 0, current: 269,84, average: 270,37
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 1496, dropped: 0, current: 270,10, average: 270,35
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1496, dropped: 0, current: 270,10, average: 270,35
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 1632, dropped: 0, current: 269,80, average: 270,30
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1632, dropped: 0, current: 269,80, average: 270,30
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 1768, dropped: 0, current: 270,61, average: 270,33
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1768, dropped: 0, current: 270,61, average: 270,33
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 1904, dropped: 0, current: 271,82, average: 270,43
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1904, dropped: 0, current: 271,82, average: 270,43
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 2040, dropped: 0, current: 269,50, average: 270,37
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 2040, dropped: 0, current: 269,50, average: 270,37
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 2176, dropped: 0, current: 270,17, average: 270,36
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 2176, dropped: 0, current: 270,17, average: 270,36
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 2312, dropped: 0, current: 269,69, average: 270,32
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 2312, dropped: 0, current: 269,69, average: 270,32
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 2448, dropped: 0, current: 270,65, average: 270,34
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 2448, dropped: 0, current: 270,65, average: 270,34
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 2584, dropped: 0, current: 271,81, average: 270,41
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 2584, dropped: 0, current: 271,81, average: 270,41

DaneLLL · December 1, 2021, 2:39am

Hi,
We would suggest break down the pipeline to confirm which plugin does not achieve target frame rate. Can check the source frame rate first:

gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)GRAY8' ! fpsdisplaysink text-overlay=0 video-sink=fakesink --verbose

Nxt-1 · December 1, 2021, 9:41am

This pipe results in a stable 153.03 fps which is as desired.

Edit: I can also confirm the following pipes to be working:
gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)GRAY8' ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=fakesink --verbose
gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)GRAY8' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! fpsdisplaysink text-overlay=0 video-sink=fakesink --verbose

But the introduction of the nvv4l2vp8enc seems to limit the framerate.
gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)GRAY8' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2vp8enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink --verbose

This suggest to me the hw encoder just cannot keep up so I check resource usage with jtop. Neither of the 4 cores are being maxed out at any point (<60%). The nvenc stays around 563MHz and gpu usage is 0%. Either way I tried lowering the bitrate to 8000 just to try something and that did not help eiter. The reported avarge climbs slowly, to stabalize around 132 fps.

gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)GRAY8' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2vp8enc bitrate=800 ! fpsdisplaysink text-overlay=0 video-sink=fakesink --verbose

Using gst-inspect I did notice that the encoder has a maxperf-enable option so I tried that as well.This seems to force the hwencoder to run at 716MHz and increases the framerate to 133fps but it did stabalize faster ¯_(ツ)_/¯.

gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)GRAY8' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2vp8enc bitrate=8000 maxperf-enable=true ! fpsdisplaysink text-overlay=0 video-sink=fakesink --verbose

EDIT2: I can also add that the other hw encoders give the same framerate give or take 2 fps. I tried the nvv4l2h264enc, nvv4l2h265enc, omxh265enc and omxvp8enc.

DaneLLL · December 2, 2021, 3:15am

Hi,
Please check this suggestion:
Gstreamer fpsdisplaysink reports double framerate - #6 by DaneLLL

Nxt-1 · December 2, 2021, 8:45am

Hi @DaneLLL forgive me but I don’t see how that suggestion is relevant here. All the pipes in my previous post are already without a muxer in them. So the rate is reported is not double, just lower than what the camera outputs.

DaneLLL · December 2, 2021, 10:57am

Hi,
Looks like the performance is capped by CPU capability of Jetson Nano. Please execute sudo tegrastats and check if CPU cores are at full loading.

DaneLLL · December 2, 2021, 12:51pm

Hi,
We try to simulate 1408x1088p154 frame data with videotestsrc but cannot achieve 154 fps:

$ gst-launch-1.0 videotestsrc is-live=1 num-buffers=1500 ! video/x-raw,format=GRAY8,framerate=154/1,width=1408,height=1088 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

If we simulate 640x480p154 frame data and upscale to 1408x1088 through nvvidconv plugin, it can achieve 154fps:

$ gst-launch-1.0 videotestsrc is-live=1 num-buffers=1500 ! video/x-raw,format=GRAY8,framerate=154/1,width=640,height=480 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420,width=1408,height=1088' ! nvv4l2vp8enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

So if we can feed 1408x1088 frame data in NVMMM buffer to hardware encoder, it can achieve target fps. The performance looks to be capped by copying 1408x1088 frame data from CPU buffer to NVMM buffer. This is possible whiling using v4l2src. If the source is Bayer sensor and goes through Argus, it can use nvarguscamerasrc to put frame data into NVMM buffer directly, and can eliminate the memory copy.

Nxt-1 · December 2, 2021, 1:24pm

Using the below pipe I get ~135fps as before. I included a couple of lines from sudo tegrastats.
gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)GRAY8' ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2vp8enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink --verbose

RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [18%@1479,8%@1479,61%@1479,11%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 563 VIC_FREQ 49%@140 APE 25 PLL@37.5C CPU@41.5C PMIC@100C GPU@39C AO@44.5C thermal@40.75C POM_5V_IN 3588/3820 POM_5V_GPU 159/159 POM_5V_CPU 717/940
RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [18%@1479,12%@1479,64%@1479,7%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 563 VIC_FREQ 45%@140 APE 25 PLL@37.5C CPU@41.5C PMIC@100C GPU@38.5C AO@44.5C thermal@40.5C POM_5V_IN 3588/3813 POM_5V_GPU 159/159 POM_5V_CPU 717/934
RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [11%@1479,19%@1479,63%@1479,5%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 563 VIC_FREQ 46%@140 APE 25 PLL@37C CPU@42C PMIC@100C GPU@39C AO@44.5C thermal@40.5C POM_5V_IN 3588/3807 POM_5V_GPU 159/159 POM_5V_CPU 757/929
RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [16%@1479,12%@1479,60%@1479,11%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 563 VIC_FREQ 66%@140 APE 25 PLL@37.5C CPU@42C PMIC@100C GPU@39C AO@44.5C thermal@40C POM_5V_IN 3588/3802 POM_5V_GPU 159/159 POM_5V_CPU 717/923
RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [11%@1479,21%@1479,59%@1479,12%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 408 VIC_FREQ 67%@140 APE 25 PLL@37.5C CPU@41.5C PMIC@100C GPU@39C AO@44.5C thermal@40.5C POM_5V_IN 3628/3797 POM_5V_GPU 159/159 POM_5V_CPU 757/919
RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [12%@1479,11%@1479,62%@1479,16%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 563 VIC_FREQ 68%@140 APE 25 PLL@37.5C CPU@41.5C PMIC@100C GPU@39C AO@45C thermal@40.5C POM_5V_IN 3588/3792 POM_5V_GPU 159/159 POM_5V_CPU 717/914
RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [10%@1479,19%@1479,62%@1479,10%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 563 VIC_FREQ 61%@140 APE 25 PLL@37.5C CPU@41.5C PMIC@100C GPU@39C AO@44.5C thermal@40.25C POM_5V_IN 3588/3787 POM_5V_GPU 159/159 POM_5V_CPU 717/909
RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [8%@1479,14%@1479,65%@1479,17%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 563 VIC_FREQ 48%@140 APE 25 PLL@37.5C CPU@42C PMIC@100C GPU@39C AO@44.5C thermal@40.5C POM_5V_IN 3588/3782 POM_5V_GPU 159/159 POM_5V_CPU 757/906
RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [18%@1479,18%@1479,56%@1479,10%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 563 VIC_FREQ 59%@140 APE 25 PLL@37.5C CPU@41.5C PMIC@100C GPU@38.5C AO@45C thermal@40.5C POM_5V_IN 3588/3778 POM_5V_GPU 159/159 POM_5V_CPU 757/902
RAM 1488/3963MB (lfb 326x4MB) SWAP 0/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [24%@1479,9%@1479,61%@1479,11%@1479] EMC_FREQ 17%@1600 GR3D_FREQ 0%@921 NVENC 563 VIC_FREQ 41%@140 APE 25 PLL@37.5C CPU@41.5C PMIC@100C GPU@38.5C AO@45C thermal@40.25C POM_5V_IN 3588/3773 POM_5V_GPU 159/159 POM_5V_CPU 717/898

With regards to your 2nd post, I must admit my knowledge seems to be lacking :) I don’t know wheter it is a Bayer sensor, nor am I familiar with Argus. Either way I am using a Alvium 1800 C-158m monochrome sensor. Since it is a native monochrome sensor I assume it is not using a bayer filter.

I tried adapting the pipe to the pipe below. I get an error that no camara’s are available, so I assumbe my camera is not argus compatible? Again I am not familar so I don’t know.
gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2h264enc ! h264parse ! fpsdisplaysink text-overlay=0 video-sink=fakesink -v

Nxt-1 · December 2, 2021, 1:33pm

Requesting the available formats from the camera give me (among others but I need this one for the highest framerate) this result. None of the other formats mention Bayer either.
v4l2-ctl -d /dev/video0 --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'GREY'
	Name        : 8-bit Greyscale
		Size: Discrete 1408x1088
			Interval: Discrete 0.007s (153.000 fps)

DaneLLL · December 3, 2021, 7:10am

Hi,
Your gstreamer command looks optimal and there may not be room for improvement. Is it possible to set up the camera and run in lower frame rate such as 120fps? Jetson Nano may not be able to support 1408x1088p154.

The camera should not be a Bayer sensor, so it cannot use nvarguscamerasrc. You would need to use v4l2src in gstreamer.

Nxt-1 · December 8, 2021, 11:51am

I seems you are right about the buffer throughput being the issue. Lower framerates indeed work without issue, as well as cropping the image (on the sensor itself).

Thank you for your help, at least I now know what the issue is.

system · December 22, 2021, 11:52am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.