Tx2-4g R32.3.1 gstreamer "pool" thread using excessive cpu usage

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

using nvsight and see that the top cpu usage in my app/shell script is “pool”

This script causes the problem I removed everything except the create video which is at the end of the script and it still has high cpu usage, it is running at around 140%, in my app I can cause a video to be created and the cpu usage can drop to below what nvargus_daemon is using, and nvsight shows the “pool” threads gone or not using any cpu time.

Might need a gstreamer expert?

gst-launch-1.0 -vvvv nvarguscamerasrc
! ‘video/x-raw(memory:NVMM),width=3864,height=2180’
! queue max-size-buffers=1 leaky=2 name=q1
! tee name=t
t. ! queue name=q3
! nvvidconv name=stalkerLoc flip-method=2 left=1692 right=2172 top=900 bottom=1280
! ‘video/x-raw(memory:NVMM),width=480,height=380’
! nvoverlaysink name=displaySink async=false sync=false overlay-x=0 overlay-y=361 overlay-w=480 overlay-h=380
t. ! queue name=q4
! valve drop=true
! nvvidconv
! clockoverlay time-format=“%D %H:%M:%S” halignment=right
! nvjpegenc ! multifilesink name=stalkerPicture async=false location=/mnt/sdcard/pic1.jpg
t. ! queue name=q5
! nvvidconv name=b ! ‘video/x-raw(memory:NVMM),width=(int)1920,height=(int)1080’
! nvivafilter cuda-process=true customer-lib-name=“libnvsample_cudaprocess.so”
! ‘video/x-raw(memory:NVMM),width=(int)1920,height=(int)1080, format=NV12’
! nvv4l2h264enc maxperf-enable=1 bitrate=8000000
! ‘video/x-h264, stream-format=(string)byte-stream’
! h264parse
! valve drop=true
! splitmuxsink name=stalkerVideoSink async-handling=true location=/tmp/tmpFile

3

Hi,
Please share sudo tegrastats when running the pipeline for reference. And release version( $ head -1 /etc/nv_tegra_release ). (It is r32.3.1).

4

RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [22%@1190,off,off,27%@1190,76%@1189,46%@1191] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.6C Tdiode@45.25C VDD_SYS_GPU 79/79 VDD_SYS_SOC 838/838 VDD_IN 3594/3594 VDD_SYS_CPU 479/479 VDD_SYS_DDR 841/841
RAM 1564/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [23%@1190,off,off,54%@1190,18%@1190,64%@1190] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@45.5C BCPU@47C thermal@46.9C Tdiode@45.25C VDD_SYS_GPU 79/79 VDD_SYS_SOC 838/838 VDD_IN 3554/3580 VDD_SYS_CPU 479/479 VDD_SYS_DDR 841/841
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [39%@1190,off,off,28%@1190,24%@1191,90%@1192] EMC_FREQ 10%@1331 GR3D_FREQ 11%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.6C Tdiode@45.25C VDD_SYS_GPU 119/89 VDD_SYS_SOC 838/838 VDD_IN 3594/3584 VDD_SYS_CPU 479/479 VDD_SYS_DDR 841/841
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [29%@1191,off,off,39%@1190,77%@1187,27%@1191] EMC_FREQ 10%@1331 GR3D_FREQ 1%@828 NVENC 1164 APE 150 PLL@47.5C MCPU@47.5C PMIC@100C Tboard@43C GPU@46C BCPU@47.5C thermal@46.6C Tdiode@45.5C VDD_SYS_GPU 119/95 VDD_SYS_SOC 838/838 VDD_IN 3614/3590 VDD_SYS_CPU 479/479 VDD_SYS_DDR 841/841
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [22%@1193,off,off,16%@1191,96%@1191,28%@1191] EMC_FREQ 10%@1331 GR3D_FREQ 12%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.9C Tdiode@45.5C VDD_SYS_GPU 79/92 VDD_SYS_SOC 838/838 VDD_IN 3574/3587 VDD_SYS_CPU 479/479 VDD_SYS_DDR 841/841
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [31%@1192,off,off,51%@1190,46%@1190,50%@1191] EMC_FREQ 10%@1331 GR3D_FREQ 10%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.6C Tdiode@45.5C VDD_SYS_GPU 79/90 VDD_SYS_SOC 838/838 VDD_IN 3574/3585 VDD_SYS_CPU 479/479 VDD_SYS_DDR 841/841
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [30%@1191,off,off,26%@1191,14%@1191,93%@1192] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.6C Tdiode@45.5C VDD_SYS_GPU 79/89 VDD_SYS_SOC 838/838 VDD_IN 3614/3589 VDD_SYS_CPU 479/479 VDD_SYS_DDR 841/841
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [26%@1190,off,off,22%@1192,20%@1192,99%@1190] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47.5C MCPU@47.5C PMIC@100C Tboard@43C GPU@46C BCPU@47.5C thermal@46.9C Tdiode@45.25C VDD_SYS_GPU 79/87 VDD_SYS_SOC 838/838 VDD_IN 3557/3585 VDD_SYS_CPU 439/474 VDD_SYS_DDR 841/841
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [34%@1189,off,off,29%@1193,62%@1190,50%@1194] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47.5C MCPU@47.5C PMIC@100C Tboard@43C GPU@46C BCPU@47.5C thermal@46.6C Tdiode@45.5C VDD_SYS_GPU 79/87 VDD_SYS_SOC 838/838 VDD_IN 3574/3584 VDD_SYS_CPU 479/475 VDD_SYS_DDR 841/841
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [25%@1190,off,off,33%@1189,66%@1189,44%@1189] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.6C Tdiode@45.25C VDD_SYS_GPU 79/86 VDD_SYS_SOC 838/838 VDD_IN 3577/3583 VDD_SYS_CPU 479/475 VDD_SYS_DDR 822/839
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [30%@1191,off,off,32%@1191,35%@1192,68%@1187] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.6C Tdiode@45.5C VDD_SYS_GPU 79/85 VDD_SYS_SOC 838/838 VDD_IN 3557/3581 VDD_SYS_CPU 439/472 VDD_SYS_DDR 841/839
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [27%@1191,off,off,39%@1190,25%@1191,87%@1190] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@45.5C BCPU@47C thermal@46.6C Tdiode@45.25C VDD_SYS_GPU 79/85 VDD_SYS_SOC 838/838 VDD_IN 3577/3581 VDD_SYS_CPU 479/472 VDD_SYS_DDR 841/839
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [30%@1191,off,off,57%@1190,47%@1189,27%@1192] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47.5C MCPU@47.5C PMIC@100C Tboard@43C GPU@46C BCPU@47.5C thermal@46.6C Tdiode@45.5C VDD_SYS_GPU 79/84 VDD_SYS_SOC 838/838 VDD_IN 3554/3579 VDD_SYS_CPU 439/470 VDD_SYS_DDR 822/838
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [37%@1190,off,off,44%@1192,40%@1191,57%@1191] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@45.5C BCPU@47C thermal@46.6C Tdiode@45.5C VDD_SYS_GPU 79/84 VDD_SYS_SOC 838/838 VDD_IN 3557/3577 VDD_SYS_CPU 479/471 VDD_SYS_DDR 822/837
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [25%@1190,off,off,24%@1192,24%@1192,99%@1191] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47.5C MCPU@47.5C PMIC@100C Tboard@43C GPU@46C BCPU@47.5C thermal@46.6C Tdiode@45.5C VDD_SYS_GPU 79/84 VDD_SYS_SOC 838/838 VDD_IN 3597/3578 VDD_SYS_CPU 479/471 VDD_SYS_DDR 841/837
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [22%@1189,off,off,17%@1189,18%@1191,100%@1190] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.4C Tdiode@45.5C VDD_SYS_GPU 79/83 VDD_SYS_SOC 838/838 VDD_IN 3557/3577 VDD_SYS_CPU 479/471 VDD_SYS_DDR 822/836
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [37%@1191,off,off,53%@1191,42%@1192,45%@1191] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.4C Tdiode@45.5C VDD_SYS_GPU 79/83 VDD_SYS_SOC 838/838 VDD_IN 3557/3576 VDD_SYS_CPU 439/470 VDD_SYS_DDR 822/835
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [35%@1187,off,off,29%@1192,24%@1190,79%@1189] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.6C Tdiode@45.5C VDD_SYS_GPU 79/83 VDD_SYS_SOC 838/838 VDD_IN 3574/3576 VDD_SYS_CPU 479/470 VDD_SYS_DDR 841/836
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [37%@1190,off,off,37%@1191,28%@1191,68%@1192] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47C MCPU@47C PMIC@100C Tboard@43C GPU@46C BCPU@47C thermal@46.9C Tdiode@45.5C VDD_SYS_GPU 79/83 VDD_SYS_SOC 838/838 VDD_IN 3557/3575 VDD_SYS_CPU 439/469 VDD_SYS_DDR 822/835
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [40%@1190,off,off,27%@1191,84%@1191,23%@1187] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 150 PLL@47.5C MCPU@47.5C PMIC@100C Tboard@43C GPU@46C BCPU@47.5C thermal@46.9C Tdiode@45.5C VDD_SYS_GPU 79/82 VDD_SYS_SOC 838/838 VDD_IN 3557/3574 VDD_SYS_CPU 439/467 VDD_SYS_DDR 822/834
RAM 1565/3835MB (lfb 15x4MB) SWAP 409/1918MB (cached 7MB) CPU [32%@1188,off,off,44%@1191,66%@1191,34%@1190] EMC_FREQ 10%@1331 GR3D_FREQ 0%@828 NVENC 1164 APE 1

Please share what you are looking for in tegrastats, I can use htop and see that there are multiple threads associated with the shell script. Nsight and my app clearly shows “pool” 4 threads each using around 25%,

apt-get update and upgrade has been done in the past few days.

Thanks,

5

Hi,
It looks like the loading is OK on TX2. No specific core is loaded to 100% always. The pipeline which takes much CPU usage is

t. ! queue name=q4
! valve drop=true
! nvvidconv
! clockoverlay time-format="%D %H:%M:%S" halignment=right
! nvjpegenc ! multifilesink name=stalkerPicture async=false location=/mnt/sdcard/pic1.jpg

It accesses CPU buffers in clockoverlay plugin, so it takes certain loading. May remove it and check the CPU usage for a try.

6

valve is set to drop so clockoverlay is not being in the actual pipe line until a photo is taken,

I removed clockoverlay and am still seeing 4 “pool” taking around 25% each.

Thanks
Terry

7

Any info, bump, is anybody looking at this?
Terry

8

Hi,
Do you observe bad performance in certain thread(nvoverlaysink, jpeg encoding, h264 encoding)? In sudo tegrastats, the loading looks fine to online CPUs.

The source is 4K and if you use r32.4.3/r32.4.4, you may run VIC engine at max clock to get full performance of nvvidconv. This should bring performance improvement.

9

the thread is about four threads found with nsight named pool using 25% of four cpus What is this “pool” thread and how do I make it use less cpu.

10

Hi,
In your pipeline, there is one source and three sinks. In the three sinks, one is video preview, one is JPEG encoding, and the other is h264 ecoding. You may eliminate either sink to clarify which one takes high CPU usage. The suspicious one is JPEG encoding since it has clockoverlay plugin and need to copy NVMM buffers to CPU buffers.

11

DaneLLL,

Have you tried nsight on the shell script and seen the “pool” threads, I need to find how to get rid of them.

Thanks,

12

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi,
We have checked and don’t think it triggers issue. Do you see any issue with the pool threads? The framerate does not achieve 3864x2160p30?