Orin nvgpu

[lpz_thread]

我们使用Linux R35.3.1版本的 的Orin 64G module，我们在测试相机时执行gst-launch-1.0 -v v4l2src device=“/dev/Video0” ! video/x-raw,framerate=30/1,width=1920,height=1080,format=UYVY ! xvimagesink ，偶尔会出现gpu问题，这个时候goome-shell cpu 100%

通过dmesg ，得到如下错误：
[ 108.109993] ga10b HDR cf1bf093 SHADOW0 81d31fca SHADOW1 19c3d90a
[ 108.118654] ga10b pbdma 3:
[ 108.125917] ga10b id: -1 - [channel] next_id: - -1 [channel] | status: invalid
[ 108.128983] ga10b PBDMA_PUT 00000015f47cd584 PBDMA_GET 000000ed9a9123dc
[ 108.136770] ga10b GP_PUT 00000000 GP_GET 03a21b7d FETCH 00000000 HEADER 219493fc
[ 108.143945] ga10b HDR 79b062df SHADOW0 b2b0a74b SHADOW1 90596fc1
[ 108.152605] ga10b pbdma 4:
[ 108.159866] ga10b id: -1 - [channel] next_id: - -1 [channel] | status: invalid
[ 108.162932] ga10b PBDMA_PUT 000000bfc35fe0f8 PBDMA_GET 00000075d707cab8
[ 108.170721] ga10b GP_PUT 00000000 GP_GET ca276c56 FETCH 00000000 HEADER 2102b4c0
[ 108.177894] ga10b HDR 2c632100 SHADOW0 207490ca SHADOW1 22576d27
[ 108.186554] ga10b pbdma 5:
[ 108.193814] ga10b id: -1 - [channel] next_id: - -1 [channel] | status: invalid
[ 108.196884] ga10b PBDMA_PUT 000000e1d1918fb4 PBDMA_GET 000000e1d1918fb4
[ 108.204671] ga10b GP_PUT 00000000 GP_GET 90f2a6ea FETCH 00000000 HEADER 4097ac24
[ 108.211844] ga10b HDR dadae679 SHADOW0 a8c7fff9 SHADOW1 44a44833
[ 108.220503] ga10b
[ 108.227769] ga10b ga10b eng 0:
[ 108.230305] ga10b id: 0 (tsg), next_id: -1 (channel), ctx status: valid
[ 108.233978] ga10b
[ 108.240980] ga10b ga10b eng 1:
[ 108.243597] ga10b id: -1 (channel), next_id: -1 (channel), ctx status: invalid
[ 108.247104] ga10b
[ 108.254982] ga10b ga10b eng 2:
[ 108.257603] ga10b id: -1 (channel), next_id: -1 (channel), ctx status: invalid
[ 108.261190] ga10b
[ 108.268981] ga10b ga10b eng 3:
[ 108.271511] ga10b id: -1 (channel), next_id: -1 (channel), ctx status: invalid
[ 108.275190] ga10b
[ 108.282893] ga10b ga10b eng 4:
[ 108.285526] ga10b id: -1 (channel), next_id: -1 (channel), ctx status: invalid
[ 108.289190] ga10b
[ 108.296805] ga10b ga10b eng 5:
[ 108.299423] ga10b id: -1 (channel), next_id: -1 (channel), ctx status: invalid
[ 108.302929] ga10b
[ 108.310628] ga10b
[ 108.313080] nvgpu: 17000000.ga10b report_pbdma_error:330 [ERR] pbdma_intr_0(0)= 0x04000000
[ 108.325332] nvgpu: 17000000.ga10b nvgpu_cic_mon_report_err_safety_services:55 [ERR] Error reporting is not supported in this platform
[ 108.337146] nvgpu: 17000000.ga10b nvgpu_set_err_notifier_locked:149 [ERR] error notifier set to 24 for ch 509
[ 115.360479] nvgpu: 17000000.ga10b ga10b_pbdma_handle_intr_0_legacy:437 [ERR] semaphore acquire timeout!
[ 115.360855] ga10b Channel Status - chip ga10b
[ 115.360857] ga10b ---------------------------
[ 115.360993] ga10b 509-ga10b, TSG: 2, pid 2120, refs: 5, deterministic: no, domain name: (default)
[ 115.361134] ga10b channel status: in use on_pbdma, pbdma_busy busy
[ 115.361393] ga10b RAMFC: TOP: 8000001ff9be065c PUT: 001ff9be0670 GET: 001ff9be065c FETCH: 000000000000 HEADER: 2140006c COUNT: 11110000 SEMAPHORE: addr 002000180000 payload 000000000000006f execute 00081003
[ 115.361597] ga10b
[ 115.362110] ga10b 510-ga10b, TSG: 1, pid 1762, refs: 2, deterministic: no, domain name: (default)
[ 115.362187] ga10b channel status: in use idle not busy
[ 115.362439] ga10b RAMFC: TOP: 8000002004033f78 PUT: 002004033f78 GET: 002004033f78 FETCH: 000000000000 HEADER: 2140006c COUNT: 00000000 SEMAPHORE: addr 002004320000 payload 0000000000000000 execute 00000001
[ 115.362594] ga10b
[ 115.365236] ga10b 511-ga10b, TSG: 0, pid 1762, refs: 2, deterministic: no, domain name: (default)
[ 115.365606] ga10b channel status: in use on_eng not busy
[ 115.371024] ga10b RAMFC: TOP: 800000200404b090 PUT: 00200404b090 GET: 00200404b090 FETCH: 000000000000 HEADER: 2140006c COUNT: 00000000 SEMAPHORE: addr 002004020000 payload 0000000000000000 execute 00100001
[ 115.376802] ga10b
[ 115.395442] ga10b PBDMA Status - chip ga10b
[ 115.398065] ga10b -------------------------

全部信息通过附件上传
Orin_gpu.md (136.0 KB)
请问下这是NVGPU的什么bug

[ShaneCCC]

Could you try fakesink?

st-launch-1.0 -v v4l2src device=“/dev/Video0” ! video/x-raw,framerate=30/1,width=1920,height=1080,format=UYVY ! fpsdisplaysink video-sink=fakesink

[lpz_thread]

我们使用v4l2编程打开多个相机的时候，测试了一两个月的过程中，同样偶尔会出现这个gpu报错的问题，这个时候通过top查看xorg 占用CPU100%，各相机的vi-ouput占用cpu100%

[lpz_thread]

当我们停止的时候，close相机出现卡死，然后断电重启，重启进入系统后，报图片这个异常

[ShaneCCC]

Does your v4l2 APP have involve display for previewing?

[lpz_thread]

我们v4l2读取相机数据，不同相机数据通过ros发布到不同的主题，使用rviz app进行显示，这个时候出现的问题

[ShaneCCC]

Maybe need to narrow down which module cause the problem first.

Thanks

[lpz_thread]

只有rviz 会订阅主题并显示，应该是显示的时候导致gpu出错了

[ShaneCCC]

Sorry don’t have experience for the RVIZ, you may want to post about RVIZ on the ROS forums.

[lpz_thread]

@ShaneCCC

Sorry don’t have experience for the RVIZ, you may want to post about RVIZ on the ROS forums

但是就像最前面第一楼提到的，使用的gst-launch-1.0 一样会出现同样的问题，只是很偶现

[ShaneCCC]

Please check if any VI/CSI error before the GPU error shows.

Thanks

[lpz_thread]

@ShaneCCC

Please check if any VI/CSI error before the GPU error shows

行，我们尝试用gst-launch-1.0 -v v4l2src device=“/dev/Video0” ! video/x-raw,framerate=30/1,width=1920,height=1080,format=UYVY ! fpsdisplaysink video-sink=fakesink来复现这个问题，并抓下完整的log，但是不确定什么时候能复现