# High CPU usage for video capturing

**URL:** <https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408>\
**Category:** Jetson Xavier NX\
**Tags:** gstreamer\
**Created:** [May 9, 2021, 10:19am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408 "2021-05-09T10:19:30Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 9, 2021, 10:19am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/1 "2021-05-09T10:19:30Z")

</div>

Hello,  
We are using Jetson NX Xavier, capturing video (v4l2) like this:  
gst-launch-1.0 v4l2src device=/dev/video0 ! ‘video/x-raw, format=UYVY, width=4192, height=3120, framerate=10/1’ ! nvvideoconvert ! ‘video/x-raw, format=NV12, width=4192, height=3120, framerate=10/1’ ! identity drop-allocation=1 ! v4l2sink device=/dev/video100 sync=false

The Stream looks fine besides it uses 100% CPU.

Is this expected?  
I have verified omx plugin is good.

Any clues?

Thanks

---

<div class="post-metadata">

**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\
**Post date:** [May 10, 2021, 2:08am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/3 "2021-05-10T02:08:38Z")

</div>

Hi,  
You may try to use _nvv4l2camerasrc_. Please check  
[Jetson Nano FAQ](https://forums.developer.nvidia.com/t/jetson-nano-faq/82953)  
[Q: I have a USB camera. How can I launch it on Jetson Nano?]

This may reduce partial CPU loading. The optimal solution is to pass NVMM buffers from source to sink, but it only accepts CPU buffers in _v4l2sink_. Certain CPU loading is expected in this usecase.

---

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 10, 2021, 4:59am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/4 "2021-05-10T04:59:42Z")

</div>

I tried this before (taken from your reference):  
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! ‘video/x-raw(memory:NVMM),format=UYVY,width=4192,height=3120,framerate=10/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),format=NV12’ ! nvoverlaysink -e  
It gives me this:  
Setting pipeline to PAUSED …  
Pipeline is live and does not need PREROLL …  
Setting pipeline to PLAYING …  
New clock: GstSystemClock  
ERROR: from element /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0: Internal data stream error.  
Additional debug info:  
gstbasesrc.c(3055): gst\_base\_src\_loop (): /GstPipeline:pipeline0/GstNvV4l2CameraSrc:nvv4l2camerasrc0:  
streaming stopped, reason error (-5)  
EOS on shutdown enabled – waiting for EOS after Error  
Waiting for EOS…

What I would like to do is:  
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! ‘video/x-raw,format=UYVY,width=4192,height=3120,framerate=10/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),format=NV12,width=4192,height=3120,framerate=10/1’ ! v4l2sink device=/dev/video100 sync=false  
WARNING: erroneous pipeline: could not link nvv4l2camerasrc0 to nvvideoconvert0, nvv4l2camerasrc0 can’t handle caps video/x-raw, format=(string)UYVY, width=(int)4192, height=(int)3120, framerate=(fraction)10/1

But it doesnt work either…  
Please note I changed the example you gave: nvvidconv → nvvideoconvert because nvvidconv is not known.

Thanks

---

<div class="post-metadata">

**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\
**Post date:** [May 10, 2021, 5:18am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/5 "2021-05-10T05:18:59Z")

</div>

Hi,  
For linking _nvv4l2camerasrc_ and _nvvideoconvert_, please set the property to _nvv4l2camerasrc_:

```auto
  bufapi-version : set to use new Buffer API
                        flags: readable, writable
                        Boolean. Default: false

```

---

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 10, 2021, 5:30am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/6 "2021-05-10T05:30:44Z")

</div>

Thanks, with this I am able to run the first command and use nvoverlaysink.

However, still not able to sink to a v4l2loopback device:  
gst-launch-1.0 nvv4l2camerasrc bufapi-version=TRUE device=/dev/video0 ! ‘video/x-raw(memory:NVMM),format=UYVY,width=4192,height=3120,framerate=10/1’ ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),format=NV12’ ! v4l2sink device=/dev/video100 sync=false  
WARNING: erroneous pipeline: could not link nvvideoconvert0 to v4l2sink0, v4l2sink0 can’t handle caps video/x-raw(memory:NVMM), format=(string)NV12

---

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 10, 2021, 8:17am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/7 "2021-05-10T08:17:59Z")

</div>

Actually I now got this to work by adding identity drop allocation before the sink.  
Not sure how this solved the issue, would appreciate if you can explain

---

<div class="post-metadata">

**Author:** ![acamacho.ruiz](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@acamacho.ruiz](https://forums.developer.nvidia.com/u/acamacho.ruiz)\
**Post date:** [May 10, 2021, 9:05am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/8 "2021-05-10T09:05:22Z")

</div>

Hi I have the same problem, but I’m not familiar with Gsstreamer, could you please share the command that worked? Maybe works for me too  
thanks

---

<div class="post-metadata">

**Author:** ![Honey\_Patouceul](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/honey_patouceul/32/389777_2.png) [@Honey\_Patouceul](https://forums.developer.nvidia.com/u/Honey_Patouceul)\
**Post date:** [May 10, 2021, 10:47am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/9 "2021-05-10T10:47:12Z")

</div>

> [@tirank](#):
>
> got this to work by adding identity drop allocation before the sink.  
> Not sure how this solved the issue, would appreciate if you can explain

The root cause is probably in v4l2loopback. See [this](https://github.com/umlaeute/v4l2loopback/issues/83#issuecomment-255106609).

For CPU usage, it is probably v4l2loopback as well causing high CPU load. Try with fakesink instead of v4l2sink for checking.

---

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 10, 2021, 2:50pm UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/10 "2021-05-10T14:50:31Z")

</div>

Hi,  
Now having issue with the data itself…  
The pipeline seems to work (at least says its working), but I get a scrambled image on screen (no matter which sink I will use).  
FOr example:  
gst-launch-1.0 nvv4l2camerasrc bufapi-version=TRUE device=/dev/video0 ! ‘video/x-raw(memory:NVMM),format=UYVY,width=4192,height=3120,framerate=10/1’ ! nvvideoconvert ! nvegltransform ! nveglglessink

Same thing if I do this (stream to v4l2loopback device and consume from it):  
gst-launch-1.0 nvv4l2camerasrc bufapi-version=TRUE device=/dev/video0 ! ‘video/x-raw(memory:NVMM),format=UYVY,width=4192,height=3120,framerate=10/1’ ! nvvideoconvert ! ‘video/x-raw,format=NV12’ ! identity drop-allocation=1 ! v4l2sink device=/dev/video100 sync=false

Camera is 13MP, needed resolution is 4192_3120.  
It works with lower resolution (1920_1080).

Please advice.

---

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 10, 2021, 5:13pm UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/11 "2021-05-10T17:13:08Z")

</div>

This works for me for 1920,1080 and lower resolution:  
gst-launch-1.0 nvv4l2camerasrc bufapi-version=TRUE device=/dev/video0 ! ‘video/x-raw(memory:NVMM),format=UYVY,width=1920,height=1080,framerate=30/1’ ! nvvideoconvert ! ‘video/x-raw,format=NV12’ ! identity drop-allocation=1 ! v4l2sink device=/dev/video100 sync=false

It doesn’t work for 4192,3120…

---

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 10, 2021, 9:00pm UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/12 "2021-05-10T21:00:06Z")

</div>

Please advice.  
I am able to capture video till 4096x2160 resolution  
Using 4192x3120 the video goes up scrambled, like lines…  
Same command being used for both resolutions except for resolution itself and fps:

This one works:  
gst-launch-1.0 nvv4l2camerasrc bufapi-version=TRUE device=/dev/video0 ! ‘video/x-raw(memory:NVMM),format=UYVY,width=4096,height=2160,framerate=15/1’ ! nvvideoconvert ! ‘video/x-raw,format=NV12’ ! identity drop-allocation=1 ! v4l2sink device=/dev/video100 sync=false

This one doesnt:  
gst-launch-1.0 nvv4l2camerasrc bufapi-version=TRUE device=/dev/video0 ! ‘video/x-raw(memory:NVMM),format=UYVY,width=4192,height=3120,framerate=10/1’ ! nvvideoconvert ! ‘video/x-raw,format=NV12’ ! identity drop-allocation=1 ! v4l2sink device=/dev/video100 sync=false

In both cases pipelines are not complaining during execution.

This sounds like memory or capability. I looked at the nvv4l2camerasrc source code but didnt see any limitation with regards to resolution.

There is no problem with the camera since it works with the v4l2 plugin but consumes too much CPU due to not being able to use the NVMM memory.

Thanks

---

<div class="post-metadata">

**Author:** ![Honey\_Patouceul](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/honey_patouceul/32/389777_2.png) [@Honey\_Patouceul](https://forums.developer.nvidia.com/u/Honey_Patouceul)\
**Post date:** [May 10, 2021, 9:41pm UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/13 "2021-05-10T21:41:52Z")

</div>

I think there may be a 4096 colums HW limit for VIC (and maybe other HW blocks).  
Someone from NVIDIA may further clarifiy these.

---

<div class="post-metadata">

**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\
**Post date:** [May 11, 2021, 2:14am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/14 "2021-05-11T02:14:23Z")

</div>

Hi,  
In _nvv4l2camerasrc_ plugin, the frames are captured and stored into NVMM buffer directly. NVMM buffer is hardware DMA buffer and for certain resolution, the pitch and width may not be identical and the captured frames look broken. So for 4192x3120, you may not be able to use _nvv4l2camersrc_ plugin and have to use _v4l2src_ plugin.

---

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 11, 2021, 4:40am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/15 "2021-05-11T04:40:47Z")

</div>

Thanks.  
Is there a list of resolutions that are supported by this plugin?

Going back to v4l2src is horrible. Using this plugin results in a very high CPU usage for the q first 10 minutes (100%) and video is quite bad during this time. After 10 minutes CPU usage goes down to 40% which is much higher than using the nvv4l2camerasrc and the video becomes ok.

Any other suggestions?  
This will be very disappointing if we are left like this especially for a production system…

Thanks

---

<div class="post-metadata">

**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\
**Post date:** [May 11, 2021, 7:54am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/16 "2021-05-11T07:54:38Z")

</div>

Hi,  
The data alignment is a hardware contraint, so cannot be changed. One possible solution is to check if your source can output in different pitch and width. You can call _NvBufferGetParams_ to get value of pitch, width, height. See if your source can output data to match the buffer.

---

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 11, 2021, 8:06am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/17 "2021-05-11T08:06:05Z")

</div>

I already checked this and it doesn’t…  
I mean it can go down to 8mp, 4096 width, but nothing in between.

---

<div class="post-metadata">

**Author:** ![tirank](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@tirank](https://forums.developer.nvidia.com/u/tirank)\
**Post date:** [May 11, 2021, 8:08am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/18 "2021-05-11T08:08:20Z")

</div>

Using v4l2src is there a way I can use dma/nvmm?  
Does this pipeline looks good:  
gst-launch-1.0 v4l2src device=/dev/video0 ! ‘video/x-raw, format=UYVY, width=4192, height=3120, framerate=10/1’ ! nvvideoconvert ! ‘video/x-raw, format=NV12, width=4192, height=3120, framerate=10/1’ ! identity drop-allocation=1 ! v4l2sink device=/dev/video100 sync=false

This one works but with high CPU usage.  
Maybe pipeline can be optimized, need your feedback on this…

---

<div class="post-metadata">

**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\
**Post date:** [May 11, 2021, 10:23am UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/19 "2021-05-11T10:23:29Z")

</div>

Hi,  
A user has shared a solution for GREY format:  
[Gstnvv4l2camerasrc with GRAY8 support - #17 by rbi](https://forums.developer.nvidia.com/t/gstnvv4l2camerasrc-with-gray8-support/175651/17)

Please check if you can apply it to your usecase. If not, the only method is to match the source data with hardware DMA buffer.

---

<div class="post-metadata">

**Author:** ![nadeemm](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nadeemm/32/367020_2.png) [@nadeemm](https://forums.developer.nvidia.com/u/nadeemm)\
**Post date:** [October 18, 2021, 6:41pm UTC](https://forums.developer.nvidia.com/t/high-cpu-usage-for-video-capturing/177408/21 "2021-10-18T18:41:22Z")

</div>

Closed on October 18, 2021, 6:41pm UTC


