Gst-launch-1.0 nvcompositor bug in tegra-l4t-r32.5.1?

Autonomous Machines Jetson & Embedded Systems Jetson Nano
#12

After several hours, the bug is occurred, in this case I have to reboot the OS to use the gst-launch again:

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://admin:Dexon1991@192.168.1.210/
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://labi:Dexon1991@192.168.1.210:554/Streaming/Channels/101?transportmode=unicast&profile=Profile_1
Progress: (open) Retrieving server options
Progress: (open) Retrieving server options
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 20, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 20, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 20, SyncPointValue = 0)
Progress: (open) Retrieving media info
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
Progress: (open) Opened Stream
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 279
Opening in BLOCKING MODE
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 279
NvMMLiteOpen : Block : BlockType = 279
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 279
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 24, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 24, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 24, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 24, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 24, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 24, SyncPointValue = 0)
Redistribute latency…
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 24, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 13, SyncPointValue = 0)
fence_set_name ioctl failed with 22
NvDdkVicExecute Failed
nvbuffer_composite Failed
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 20, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 20, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 20, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.463166094
Setting pipeline to PAUSED …
Setting pipeline to READY …
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 20, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 21, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 24, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
Setting pipeline to NULL …
Freeing pipeline …

#13

Hi,
Have you tried four 1080p60 sources? The supported case is single 4Kp60. Looks like you have four 4Kp60 sources.

#14

I’m using:
1x 1080p @ 24fps (dubai)
1x 2160p @ 30fps (new york)
2x 720p @ 20fps (ip cameras)

So I am not using 4k60 sources. Only the jetson’s output timing is 4K.

#15

Hi,
The log shows VIC engine hangs. Not sure if it helps but please disable DFS and try. The steps to disable DFS:
Nvvideoconvert issue, nvvideoconvert in DS4 is better than Ds5? - #3 by DaneLLL

#16

I run this command:

sudo echo on > /sys/devices/50000000.host1x/54340000.vic/power/control

and after I started the my script and the issue occurred after about 4-5 hours.

#17

Hi,
Please execute sudo tegrastats and check thermal condition. Probably the Nano module is overheating and hardware engine hangs. If it is the case, we suggest try this fan:
Jetson Nano Fan - #7 by dusty_nv

#18

Thanks, I added the tegrastats into my script.

Which temperature values are meaning the overheat?

After several working loop I got these values:
RAM 1796/3956MB (lfb 84x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [25%@102,5%@102,3%@102,5%@102] EMC_FREQ 15%@1600 GR3D_FREQ 0%@76 VIC_FREQ 0%@627 APE 25 PLL@33.5C CPU@36C PMIC@100C GPU@35.5C AO@39C thermal@35.75C POM_5V_IN 3015/3015 POM_5V_GPU 0/0 POM_5V_CPU 158/158

#19

Here is the last messages before hang, is this values high or not?

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://admin:Dexon1991@192.168.1.210/
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://labi:Dexon1991@192.168.1.210:554/Streaming/Channels/101?transportmode=unicast&profile=Profile_1
Progress: (open) Retrieving server options
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
Progress: (open) Opened Stream
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
Opening in BLOCKING MODE
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 279
Opening in BLOCKING MODE
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteOpen : Block : BlockType = 279
NvMMLiteBlockCreate : Block : BlockType = 279
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 279
Redistribute latency…
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 13, SyncPointValue = 0)
fence_set_name ioctl failed with 22
NvDdkVicExecute Failed
nvbuffer_composite Failed
Got EOS from element “pipeline0”.
Execution ended after 0:00:02.258893176
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
gst-launch-1.0: no process found

RAM 1818/3956MB (lfb 85x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [18%@102,9%@102,5%@102,3%@102] EMC_FREQ 8%@1600 GR3D_FREQ 0%@76 VIC_FREQ 0%@627 APE 25 PLL@33C CPU@36C PMIC@100C GPU@35C AO@39C thermal@35.5C POM_5V_IN 3015/3015 POM_5V_GPU 0/0 POM_5V_CPU 158/158
RAM 1818/3956MB (lfb 85x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [24%@102,11%@102,6%@204,6%@204] EMC_FREQ 7%@1600 GR3D_FREQ 0%@76 VIC_FREQ 0%@627 APE 25 PLL@33C CPU@36C PMIC@100C GPU@35C AO@39C thermal@35.5C POM_5V_IN 3015/3015 POM_5V_GPU 0/0 POM_5V_CPU 158/158

#20

Hi,
On r32.5.1/Jetson Nano, we ran the script but did not hit the issue:

#!/bin/bash

i=1
while [ "$i" != "65536" ]
do
echo "loop" $i

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=1080 sink_2::xpos=0 sink_2::ypos=1080 sink_2::width=1920 sink_2::height=1080 sink_3::xpos=1920 sink_3::ypos=1080 sink_3::width=1920 sink_3::height=1080 ! nvoverlaysink sync=true uridecodebin uri="file:///opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4" ! nvvidconv ! queue ! comp. uridecodebin uri="file:///opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4" ! nvvidconv ! queue ! comp. uridecodebin uri="file:///opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_1080p_h264.mp4" ! nvvidconv ! queue ! comp. uridecodebin uri="file:///home/nvidia/jellyfish-120-mbps-4k-uhd-h264.mkv" ! nvvidconv ! comp. &

sleep 12
killall -2 gst-launch-1.0
sleep 1

gst-launch-1.0 uridecodebin uri="file:///home/nvidia/jellyfish-120-mbps-4k-uhd-h264.mkv" name=bin ! queue ! nvoverlaysink overlay=2 overlay-x=1920 overlay-y=0 overlay-w=1920 overlay-h=2160  &
gst-launch-1.0 uridecodebin uri="file:///home/nvidia/jellyfish-120-mbps-4k-uhd-h264.mkv" name=bin ! queue ! nvoverlaysink overlay=1 overlay-x=0 overlay-y=0 overlay-w=1920 overlay-h=2160  &

sleep 12
killall -2 gst-launch-1.0
sleep 1
    i=$(($i+1))
done

It run for 2527 times(around 12 hours) and still keeps running. Are you able to try the script?

And your tegrastats show 0% loading in VIC engine. The nvvidconv and nvcompositor plugins utilize the engine, so you should see loading like VIC_FREQ 80%@486. A bit strange it is 0%.

GStreamer pipeline not cleans up if started as a service
#22

I run your very same script with your media files on two jetson. For me after ~10-11 hours the issue is occurred both of them!

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=1080 sink_2::xpos=0 sink_2::ypos=1080 sink_2::width=1920 sink_2::height=1080 sink_3::xpos=1920 sink_3::ypos=1080 sink_3::width=1920 sink_3::height=1080 ! nvoverlaysink sync=true uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/sample_720p.mp4” ! nvvidconv ! queue ! comp. uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/sample_720p.mp4” ! nvvidconv ! queue ! comp. uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/sample_1080p_h264.mp4” ! nvvidconv ! queue ! comp. uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/jellyfish-120-mbps-4k-uhd-h264.mkv” ! nvvidconv ! comp.
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
Redistribute latency…
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 13, SyncPointValue = 0)
fence_set_name ioctl failed with 22
NvDdkVicExecute Failed
nvbuffer_composite Failed
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.004703177
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Now I run your script again with tegrastats…after reboot.

Do you have any idea that I would not need to reboot method, something reset or clear procedure?

#23

Hi,
Do you hit the issue on Nano devkits or your custom board? Once it happens, are you able to utilize VIC engine by running another command like:

videotestsrc ! nvvidconv ! nvoverlaysink
#24

I use my custom board. I will try this command.

(Now I started a new jetson nano onto the eval board which is the number 3.)

This is the current logs:
loop 299
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
Redistribute latency…
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
RAM 1957/3956MB (lfb 190x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [38%-1479,44%-1479,40%-1479,31%-1479] EMC_FREQ 21%-1600 GR3D_FREQ 0%-76 NVDEC 473 VIC_FREQ 0%-268 APE 25 PLL-36C CPU-39C PMIC-100C GPU-38C AO-41.5C thermal-38.5C POM_5V_IN 4912/4912 POM_5V_GPU 0/0 POM_5V_CPU 1148/1148
RAM 1959/3956MB (lfb 190x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [37%-825,19%-825,23%-825,16%-825] EMC_FREQ 24%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-140 APE 25 PLL-36C CPU-39C PMIC-100C GPU-38C AO-41.5C thermal-38.5C POM_5V_IN 4833/4872 POM_5V_GPU 0/0 POM_5V_CPU 792/970
RAM 1959/3956MB (lfb 189x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [28%-825,17%-825,15%-825,15%-825] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 652 VIC_FREQ 66%-140 APE 25 PLL-35.5C CPU-39C PMIC-100C GPU-37.5C AO-41.5C thermal-38.5C POM_5V_IN 4444/4729 POM_5V_GPU 0/0 POM_5V_CPU 673/871
RAM 1959/3956MB (lfb 189x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%-518,18%-518,17%-518,17%-518] EMC_FREQ 28%-1600 GR3D_FREQ 0%-76 NVDEC 601 VIC_FREQ 20%-268 APE 25 PLL-35.5C CPU-39C PMIC-100C GPU-37.5C AO-41.5C thermal-38.25C POM_5V_IN 4245/4608 POM_5V_GPU 0/0 POM_5V_CPU 594/801
RAM 1959/3956MB (lfb 189x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [36%-1036,19%-1036,18%-1036,20%-1036] EMC_FREQ 29%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 3%-140 APE 25 PLL-36C CPU-39.5C PMIC-100C GPU-38C AO-41C thermal-38.5C POM_5V_IN 4912/4669 POM_5V_GPU 0/0 POM_5V_CPU 911/823
RAM 1959/3956MB (lfb 189x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%-825,19%-825,18%-825,18%-825] EMC_FREQ 30%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-332 APE 25 PLL-35.5C CPU-39C PMIC-100C GPU-37.5C AO-41.5C thermal-38.25C POM_5V_IN 4674/4670 POM_5V_GPU 0/0 POM_5V_CPU 911/838
RAM 1959/3956MB (lfb 189x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [33%-518,18%-518,16%-518,16%-518] EMC_FREQ 31%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 66%-627 APE 25 PLL-36C CPU-39C PMIC-100C GPU-38C AO-41.5C thermal-38.25C POM_5V_IN 4602/4660 POM_5V_GPU 0/0 POM_5V_CPU 594/803
RAM 1960/3956MB (lfb 189x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [36%-825,23%-825,19%-825,19%-825] EMC_FREQ 31%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 1%-537 APE 25 PLL-36C CPU-39C PMIC-100C GPU-38C AO-41.5C thermal-38.5C POM_5V_IN 4991/4701 POM_5V_GPU 0/0 POM_5V_CPU 871/811
RAM 1960/3956MB (lfb 189x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [31%-710,20%-710,20%-710,18%-710] EMC_FREQ 32%-1600 GR3D_FREQ 0%-76 NVDEC 601 VIC_FREQ 0%-627 APE 25 PLL-36C CPU-39C PMIC-100C GPU-38C AO-41.5C thermal-38.5C POM_5V_IN 4682/4699 POM_5V_GPU 0/0 POM_5V_CPU 594/787
RAM 1960/3956MB (lfb 189x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [33%-1479,19%-1479,16%-1479,17%-1479] EMC_FREQ 32%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 72%-627 APE 25 PLL-36C CPU-39.5C PMIC-100C GPU-38C AO-41.5C thermal-38.5C POM_5V_IN 4991/4728 POM_5V_GPU 0/0 POM_5V_CPU 911/799
RAM 1960/3956MB (lfb 189x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [29%-518,20%-518,15%-518,15%-518] EMC_FREQ 32%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 3%-627 APE 25 PLL-36C CPU-39C PMIC-100C GPU-38C AO-41.5C thermal-38C POM_5V_IN 4714/4727 POM_5V_GPU 0/0 POM_5V_CPU 554/777
handling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:11.229043486
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Setting pipeline to PAUSED …
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Pipeline is PREROLLING …
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
RAM 1886/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [44%-921,41%-921,38%-921,53%-921] EMC_FREQ 24%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-140 APE 25 PLL-36C CPU-39C PMIC-100C GPU-38C AO-41.5C thermal-38C POM_5V_IN 4364/4364 POM_5V_GPU 0/0 POM_5V_CPU 554/554
RAM 1883/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [29%-403,18%-403,17%-403,17%-403] EMC_FREQ 25%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-140 APE 25 PLL-35.5C CPU-39C PMIC-100C GPU-37.5C AO-41C thermal-38.25C POM_5V_IN 4166/4265 POM_5V_GPU 0/0 POM_5V_CPU 396/475
RAM 1885/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [34%-614,16%-614,16%-614,17%-614] EMC_FREQ 25%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-140 APE 25 PLL-36C CPU-39C PMIC-100C GPU-38C AO-41.5C thermal-38.5C POM_5V_IN 4325/4285 POM_5V_GPU 0/0 POM_5V_CPU 515/488
RAM 1885/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [33%-825,18%-825,17%-518,17%-518] EMC_FREQ 26%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-140 APE 25 PLL-36C CPU-39C PMIC-100C GPU-38C AO-41C thermal-38.5C POM_5V_IN 4325/4295 POM_5V_GPU 0/0 POM_5V_CPU 554/504
RAM 1885/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [36%-825,14%-825,15%-825,16%-825] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-140 APE 25 PLL-36C CPU-38.5C PMIC-100C GPU-38C AO-41C thermal-38.25C POM_5V_IN 4364/4308 POM_5V_GPU 0/0 POM_5V_CPU 515/506
RAM 1885/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [37%-614,16%-614,19%-614,13%-614] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 652 VIC_FREQ 0%-140 APE 25 PLL-35.5C CPU-39C PMIC-100C GPU-37.5C AO-41C thermal-38C POM_5V_IN 4007/4258 POM_5V_GPU 0/0 POM_5V_CPU 476/501
RAM 1885/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%-403,17%-403,18%-403,17%-403] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 652 VIC_FREQ 0%-140 APE 25 PLL-35.5C CPU-38.5C PMIC-100C GPU-37.5C AO-41C thermal-37.75C POM_5V_IN 4007/4222 POM_5V_GPU 0/0 POM_5V_CPU 396/486
RAM 1885/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [34%-1479,18%-1479,19%-1479,11%-1479] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-140 APE 25 PLL-35.5C CPU-39C PMIC-100C GPU-37.5C AO-41C thermal-38C POM_5V_IN 4047/4200 POM_5V_GPU 0/0 POM_5V_CPU 476/485
RAM 1885/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [34%-403,21%-403,17%-403,15%-403] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-140 APE 25 PLL-35.5C CPU-38.5C PMIC-100C GPU-37.5C AO-41C thermal-38C POM_5V_IN 4166/4196 POM_5V_GPU 0/0 POM_5V_CPU 396/475
RAM 1885/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [34%-307,14%-307,21%-307,15%-307] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 601 VIC_FREQ 0%-140 APE 25 PLL-35.5C CPU-38.5C PMIC-100C GPU-37.5C AO-40.5C thermal-38C POM_5V_IN 3968/4173 POM_5V_GPU 0/0 POM_5V_CPU 396/467
RAM 1885/3956MB (lfb 194x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [35%-518,17%-518,21%-518,18%-518] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 563 VIC_FREQ 0%-140 APE 25 PLL-35.5C CPU-39C PMIC-100C GPU-37.5C AO-41C thermal-38C POM_5V_IN 4126/4169 POM_5V_GPU 0/0 POM_5V_CPU 476/468
handling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:11.517075933
Setting pipeline to PAUSED …
handling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:11.477948068
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to READY …
Setting pipeline to NULL …
Setting pipeline to NULL …
Freeing pipeline …
Freeing pipeline …

loop 300

#25

Hi,
It still looks like the system goes into certain thermal condition and VIC engine hangs. Would suggest check the thermal solution.

And after hitting the issue, please let the system rest for a few minutes and check if you can run simple command to utilize VIC engine:

videotestsrc ! nvvidconv ! nvoverlaysink
#26

The issue was hit on the devkit jetson also after 2263 loop. (2 of 3 were hang) And I run your command for 10 minutes without success:

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=1080 sink_2::xpos=0 sink_2::ypos=1080 sink_2::width=1920 sink_2::height=1080 sink_3::xpos=1920 sink_3::ypos=1080 sink_3::width=1920 sink_3::height=1080 ! nvoverlaysink sync=true uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/sample_720p.mp4” ! nvvidconv ! queue ! comp. uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/sample_720p.mp4” ! nvvidconv ! queue ! comp. uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/sample_1080p_h264.mp4” ! nvvidconv ! queue ! comp. uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/jellyfish-120-mbps-4k-uhd-h264.mkv” ! nvvidconv ! comp.
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
Redistribute latency…
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 13, SyncPointValue = 0)
fence_set_name ioctl failed with 22
NvDdkVicExecute Failed
nvbuffer_composite Failed
New clock: GstSystemClock
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.030308281
Setting pipeline to PAUSED …
Setting pipeline to READY …
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 25, SyncPointValue = 0)
Setting pipeline to NULL …
Freeing pipeline …
nv@DEXONNV:~ nv@DEXONNV:~
nv@DEXONNV:~ gst-launch-1.0 videotestsrc ! nvvidconv ! nvoverlaysink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock ^Chandling interrupt. Interrupt: Stopping pipeline ... Execution ended after 0:00:13.272896974 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ... nv@DEXONNV:~
nv@DEXONNV:~ nv@DEXONNV:~ gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=1920 sink_1::ypos=0 sink_1::width=1920 sink_1::height=1080 sink_2::xpos=0 sink_2::ypos=1080 sink_2::width=1920 sink_2::height=1080 sink_3::xpos=1920 sink_3::ypos=1080 sink_3::width=1920 sink_3::height=1080 ! nvoverlaysink sync=true uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/sample_720p.mp4” ! nvvidconv ! queue ! comp. uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/sample_720p.mp4” ! nvvidconv ! queue ! comp. uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/sample_1080p_h264.mp4” ! nvvidconv ! queue ! comp. uridecodebin uri=“file:///tmp/network/192.168.1.2_DEXON//7_Production/8_Test/jellyfish-120-mbps-4k-uhd-h264.mkv” ! nvvidconv ! comp.
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
Opening in BLOCKING MODE
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
Redistribute latency…
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 13, SyncPointValue = 0)
fence_set_name ioctl failed with 22
NvDdkVicExecute Failed
nvbuffer_composite Failed
New clock: GstSystemClock
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.030134948
Setting pipeline to PAUSED …
Setting pipeline to READY …
NvRmChannelSubmit: NvError_IoctlFailed with error code 22
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 23, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 23, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 23, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 23, SyncPointValue = 0)
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 23, SyncPointValue = 0)
Setting pipeline to NULL …
Freeing pipeline …
nv@DEXONNV:~$

#27

I rerun the test script with correct colling:

jetson_fan_dexon
jetson_fan_dexon1000×315 98.8 KB

#28

This issue is not depend on the temperature I think.
The issue hit again on the jetson nano on eval board with correct colling:

  1. ápr. 16., péntek, 13:27:26 CEST
    loop 302
    Setting pipeline to PAUSED …
    Pipeline is PREROLLING …
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    NvMMLiteOpen : Block : BlockType = 261
    NvMMLiteOpen : Block : BlockType = 261
    NvMMLiteOpen : Block : BlockType = 261
    NvMMLiteOpen : Block : BlockType = 261
    NVMEDIA: Reading vendor.tegra.display-size : status: 6
    NVMEDIA: Reading vendor.tegra.display-size : status: 6
    NVMEDIA: Reading vendor.tegra.display-size : status: 6
    NVMEDIA: Reading vendor.tegra.display-size : status: 6
    NvMMLiteBlockCreate : Block : BlockType = 261
    NvMMLiteBlockCreate : Block : BlockType = 261
    NvMMLiteBlockCreate : Block : BlockType = 261
    NvMMLiteBlockCreate : Block : BlockType = 261
    Redistribute latency…
    Pipeline is PREROLLED …
    Setting pipeline to PLAYING …
    New clock: GstSystemClock
    RAM 1866/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [38%-1479,38%-1479,48%-1479,34%-1479] EMC_FREQ 21%-1600 GR3D_FREQ 0%-76 NVDEC 550 VIC_FREQ 0%-576 APE 25 PLL-25C CPU-30.5C PMIC-100C GPU-28C AO-32.5C thermal-29.25C POM_5V_IN 4440/4440 POM_5V_GPU 39/39 POM_5V_CPU 958/958
    RAM 1869/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [37%-518,18%-518,17%-518,20%-518] EMC_FREQ 24%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-627 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32.5C thermal-28.75C POM_5V_IN 4086/4263 POM_5V_GPU 40/39 POM_5V_CPU 480/719
    RAM 1868/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%-921,15%-921,18%-921,17%-921] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 71%-307 APE 25 PLL-25C CPU-30.5C PMIC-100C GPU-28C AO-32.5C thermal-29C POM_5V_IN 4512/4346 POM_5V_GPU 39/39 POM_5V_CPU 758/732
    RAM 1868/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [33%-825,16%-825,17%-825,18%-825] EMC_FREQ 28%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 22%-307 APE 25 PLL-25C CPU-30C PMIC-100C GPU-27.5C AO-32.5C thermal-29C POM_5V_IN 4784/4455 POM_5V_GPU 39/39 POM_5V_CPU 837/758
    RAM 1868/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [27%-614,19%-614,18%-614,18%-614] EMC_FREQ 30%-1600 GR3D_FREQ 0%-76 NVDEC 550 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-28C AO-32.5C thermal-28.75C POM_5V_IN 4166/4397 POM_5V_GPU 40/39 POM_5V_CPU 560/718
    RAM 1868/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [33%-403,20%-403,19%-403,18%-403] EMC_FREQ 30%-1600 GR3D_FREQ 0%-76 NVDEC 576 VIC_FREQ 0%-192 APE 25 PLL-25C CPU-30.5C PMIC-100C GPU-27.5C AO-32.5C thermal-29C POM_5V_IN 4280/4378 POM_5V_GPU 40/39 POM_5V_CPU 680/712
    RAM 1868/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [31%-614,15%-614,19%-614,15%-614] EMC_FREQ 31%-1600 GR3D_FREQ 0%-76 NVDEC 550 VIC_FREQ 17%-409 APE 25 PLL-25C CPU-30.5C PMIC-100C GPU-27.5C AO-32.5C thermal-28.75C POM_5V_IN 4280/4364 POM_5V_GPU 40/39 POM_5V_CPU 600/696
    RAM 1868/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [37%-825,19%-825,17%-825,18%-825] EMC_FREQ 32%-1600 GR3D_FREQ 0%-76 NVDEC 576 VIC_FREQ 0%-524 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32.5C thermal-28.75C POM_5V_IN 4360/4363 POM_5V_GPU 40/39 POM_5V_CPU 600/684
    RAM 1869/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%-614,16%-614,17%-614,17%-614] EMC_FREQ 32%-1600 GR3D_FREQ 0%-76 NVDEC 550 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30.5C PMIC-100C GPU-27.5C AO-32C thermal-29C POM_5V_IN 4280/4354 POM_5V_GPU 40/39 POM_5V_CPU 560/670
    RAM 1869/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%-614,16%-614,19%-614,17%-614] EMC_FREQ 32%-1600 GR3D_FREQ 0%-76 NVDEC 576 VIC_FREQ 30%-409 APE 25 PLL-24.5C CPU-30.5C PMIC-100C GPU-27.5C AO-32.5C thermal-28.75C POM_5V_IN 4480/4366 POM_5V_GPU 39/39 POM_5V_CPU 599/663
    RAM 1869/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [31%-614,18%-614,17%-614,18%-614] EMC_FREQ 32%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-627 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32.5C thermal-29C POM_5V_IN 4240/4355 POM_5V_GPU 40/39 POM_5V_CPU 560/653
    handling interrupt.
    Interrupt: Stopping pipeline …
    Execution ended after 0:00:11.218200464
    Setting pipeline to PAUSED …
    Setting pipeline to READY …
    Setting pipeline to NULL …
    Freeing pipeline …

Setting pipeline to PAUSED …
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Pipeline is PREROLLING …
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
Opening in BLOCKING MODE
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
RAM 1790/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [36%-518,50%-518,42%-518,40%-518] EMC_FREQ 24%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-28C AO-32.5C thermal-29.25C POM_5V_IN 4160/4160 POM_5V_GPU 40/40 POM_5V_CPU 560/560
RAM 1786/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%-518,16%-518,18%-710,16%-710] EMC_FREQ 25%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-28C AO-32.5C thermal-29.25C POM_5V_IN 4086/4123 POM_5V_GPU 40/40 POM_5V_CPU 560/560
RAM 1787/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [36%-307,15%-307,18%-307,13%-307] EMC_FREQ 25%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-28C AO-32C thermal-29C POM_5V_IN 3966/4070 POM_5V_GPU 40/40 POM_5V_CPU 440/520
RAM 1787/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [34%-403,19%-403,19%-403,13%-403] EMC_FREQ 26%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-25C CPU-30.5C PMIC-100C GPU-27.5C AO-32.5C thermal-29C POM_5V_IN 4006/4054 POM_5V_GPU 40/40 POM_5V_CPU 440/500
RAM 1785/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [39%-518,15%-518,18%-518,18%-518] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-25C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-29C POM_5V_IN 4166/4076 POM_5V_GPU 40/40 POM_5V_CPU 480/496
RAM 1786/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [35%-403,20%-403,20%-403,18%-403] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 576 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-28C AO-32C thermal-29C POM_5V_IN 3892/4046 POM_5V_GPU 40/40 POM_5V_CPU 441/486
RAM 1786/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [36%-614,15%-614,19%-614,15%-614] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32.5C thermal-28.75C POM_5V_IN 3892/4024 POM_5V_GPU 40/40 POM_5V_CPU 400/474
RAM 1786/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [37%-403,17%-403,19%-403,17%-403] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 652 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30.5C PMIC-100C GPU-27.5C AO-32C thermal-28.75C POM_5V_IN 4046/4026 POM_5V_GPU 40/40 POM_5V_CPU 400/465
RAM 1786/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [35%-403,17%-403,17%-403,16%-403] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 576 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-28.75C POM_5V_IN 3812/4002 POM_5V_GPU 40/40 POM_5V_CPU 401/458
RAM 1786/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [38%-921,19%-921,19%-921,19%-921] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-28.75C POM_5V_IN 3926/3995 POM_5V_GPU 40/40 POM_5V_CPU 480/460
RAM 1786/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [35%-403,20%-403,18%-403,16%-403] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 652 VIC_FREQ 0%-192 APE 25 PLL-25C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-28.75C POM_5V_IN 3926/3988 POM_5V_GPU 40/40 POM_5V_CPU 440/458
handling interrupt.
handling interrupt.
Interrupt: Stopping pipeline …
Interrupt: Stopping pipeline …
Execution ended after 0:00:11.549797860
Execution ended after 0:00:11.437325829
Setting pipeline to PAUSED …
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to READY …
Setting pipeline to NULL …
Setting pipeline to NULL …
Freeing pipeline …
Freeing pipeline …

  1. ápr. 16., péntek, 13:27:52 CEST
    loop 303
    Setting pipeline to PAUSED …
    Pipeline is PREROLLING …
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    Opening in BLOCKING MODE
    NvMMLiteOpen : Block : BlockType = 261
    NvMMLiteOpen : Block : BlockType = 261
    NvMMLiteOpen : Block : BlockType = 261
    Opening in BLOCKING MODE
    NvMMLiteOpen : Block : BlockType = 261
    NVMEDIA: Reading vendor.tegra.display-size : status: 6
    NVMEDIA: Reading vendor.tegra.display-size : status: 6
    NVMEDIA: Reading vendor.tegra.display-size : status: 6
    NVMEDIA: Reading vendor.tegra.display-size : status: 6
    NvMMLiteBlockCreate : Block : BlockType = 261
    NvMMLiteBlockCreate : Block : BlockType = 261
    NvMMLiteBlockCreate : Block : BlockType = 261
    NvMMLiteBlockCreate : Block : BlockType = 261
    Redistribute latency…
    Pipeline is PREROLLED …
    Setting pipeline to PLAYING …
    New clock: GstSystemClock
    NvRmChannelSubmit: NvError_IoctlFailed with error code 22
    NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 13, SyncPointValue = 0)
    fence_set_name ioctl failed with 22
    NvDdkVicExecute Failed
    nvbuffer_composite Failed
    Got EOS from element “pipeline0”.
    Execution ended after 0:00:00.207135885
    Setting pipeline to PAUSED …
    Setting pipeline to READY …
    RAM 1866/3956MB (lfb 195x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [43%-1479,40%-1479,35%-1479,44%-1479] EMC_FREQ 21%-1600 GR3D_FREQ 0%-76 NVDEC 550 VIC_FREQ 0%-192 APE 25 PLL-25C CPU-30C PMIC-100C GPU-27.5C AO-32.5C thermal-28.75C POM_5V_IN 4592/4592 POM_5V_GPU 39/39 POM_5V_CPU 1158/1158
    Setting pipeline to NULL …
    Freeing pipeline …
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [13%-307,5%-307,10%-307,5%-307] EMC_FREQ 17%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-24C CPU-30C PMIC-100C GPU-27C AO-32C thermal-28.75C POM_5V_IN 2786/3689 POM_5V_GPU 40/39 POM_5V_CPU 201/679
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [10%-102,7%-102,9%-102,5%-102] EMC_FREQ 14%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-24C CPU-29.5C PMIC-100C GPU-27.5C AO-32C thermal-28.5C POM_5V_IN 2746/3374 POM_5V_GPU 40/39 POM_5V_CPU 161/506
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [6%-204,7%-204,10%-204,4%-204] EMC_FREQ 12%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-23.5C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-28.5C POM_5V_IN 2746/3217 POM_5V_GPU 40/39 POM_5V_CPU 161/420
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [7%-204,5%-204,11%-204,5%-204] EMC_FREQ 10%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-24C CPU-29.5C PMIC-100C GPU-27C AO-32C thermal-28.5C POM_5V_IN 2746/3123 POM_5V_GPU 40/39 POM_5V_CPU 161/368
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [6%-102,2%-102,12%-102,5%-102] EMC_FREQ 9%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-23.5C CPU-29.5C PMIC-100C GPU-27C AO-32C thermal-28.5C POM_5V_IN 2746/3060 POM_5V_GPU 40/39 POM_5V_CPU 161/333
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [8%-102,6%-102,11%-102,3%-102] EMC_FREQ 9%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-24C CPU-29.5C PMIC-100C GPU-27.5C AO-32C thermal-28.5C POM_5V_IN 2746/3015 POM_5V_GPU 40/39 POM_5V_CPU 161/309
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [9%-204,3%-204,12%-204,5%-204] EMC_FREQ 8%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-24C CPU-29.5C PMIC-100C GPU-27.5C AO-32C thermal-28.5C POM_5V_IN 2746/2981 POM_5V_GPU 40/39 POM_5V_CPU 161/290
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [4%-102,5%-102,8%-102,8%-102] EMC_FREQ 8%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-24C CPU-29.5C PMIC-100C GPU-27C AO-32C thermal-28.25C POM_5V_IN 2746/2955 POM_5V_GPU 40/39 POM_5V_CPU 161/276
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [6%-204,5%-204,10%-204,5%-204] EMC_FREQ 8%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-23.5C CPU-29.5C PMIC-100C GPU-27.5C AO-32C thermal-28.5C POM_5V_IN 2746/2934 POM_5V_GPU 40/39 POM_5V_CPU 161/264
    RAM 1575/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [8%-204,4%-204,9%-204,8%-204] EMC_FREQ 8%-1600 GR3D_FREQ 0%-76 VIC_FREQ 0%-192 APE 25 PLL-24C CPU-29.5C PMIC-100C GPU-27.5C AO-32C thermal-28.5C POM_5V_IN 2746/2917 POM_5V_GPU 40/39 POM_5V_CPU 161/255
    gst-launch-1.0: no process found

Setting pipeline to PAUSED …
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
Pipeline is PREROLLING …
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [48%-1479,39%-1479,44%-1479,41%-1479] EMC_FREQ 12%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-28.5C POM_5V_IN 4086/4086 POM_5V_GPU 40/40 POM_5V_CPU 560/560
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%-518,17%-518,15%-518,10%-518] EMC_FREQ 17%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-29.5C PMIC-100C GPU-27.5C AO-32C thermal-28.5C POM_5V_IN 4006/4046 POM_5V_GPU 40/40 POM_5V_CPU 440/500
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [34%-518,15%-518,10%-518,15%-518] EMC_FREQ 20%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27C AO-32C thermal-28.75C POM_5V_IN 4046/4046 POM_5V_GPU 40/40 POM_5V_CPU 440/480
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [34%-403,15%-403,17%-403,18%-403] EMC_FREQ 22%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-29.5C PMIC-100C GPU-27.5C AO-31.5C thermal-28.75C POM_5V_IN 4086/4056 POM_5V_GPU 40/40 POM_5V_CPU 440/470
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [33%-403,17%-403,17%-403,19%-403] EMC_FREQ 24%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-29.5C PMIC-100C GPU-27.5C AO-32C thermal-28.75C POM_5V_IN 4006/4046 POM_5V_GPU 40/40 POM_5V_CPU 400/456
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [31%-403,13%-403,19%-403,20%-403] EMC_FREQ 25%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-28.25C POM_5V_IN 3892/4020 POM_5V_GPU 40/40 POM_5V_CPU 400/446
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [36%-518,14%-518,19%-518,16%-518] EMC_FREQ 26%-1600 GR3D_FREQ 0%-76 NVDEC 716 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-29.5C PMIC-100C GPU-27C AO-32C thermal-28.25C POM_5V_IN 3966/4012 POM_5V_GPU 40/40 POM_5V_CPU 440/445
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [33%-403,18%-403,21%-403,22%-403] EMC_FREQ 26%-1600 GR3D_FREQ 0%-76 NVDEC 614 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-28.25C POM_5V_IN 3852/3992 POM_5V_GPU 40/40 POM_5V_CPU 401/440
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [35%-403,14%-403,20%-403,19%-403] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 614 VIC_FREQ 0%-192 APE 25 PLL-24C CPU-29.5C PMIC-100C GPU-27.5C AO-32.5C thermal-28.25C POM_5V_IN 3852/3976 POM_5V_GPU 40/40 POM_5V_CPU 441/440
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [34%-403,18%-403,21%-403,17%-403] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 652 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-28.5C POM_5V_IN 3892/3968 POM_5V_GPU 40/40 POM_5V_CPU 400/436
RAM 1797/3956MB (lfb 196x4MB) SWAP 0/1978MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [37%-518,17%-518,19%-518,16%-518] EMC_FREQ 27%-1600 GR3D_FREQ 0%-76 NVDEC 614 VIC_FREQ 0%-192 APE 25 PLL-24.5C CPU-30C PMIC-100C GPU-27.5C AO-32C thermal-28.75C POM_5V_IN 3926/3964 POM_5V_GPU 40/40 POM_5V_CPU 440/436
handling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:11.585702027
Setting pipeline to PAUSED …
Setting pipeline to READY …
handling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:11.475095829
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Setting pipeline to NULL …
Freeing pipeline …
Freeing pipeline …

#30

Hi,
Now I can run 8pcs jetson nano in same time and I could see the hang issue all of them the average hang time about 10 hours.

Have you ever face such an issue in your environment?

If not, can we recognize the differences?

I use 10W mode, r35.5.1, Model: P3448 (180-13448-DAAA-B01) (900-13448-0020-000, S/N:1422720005556)

#31

Hi,
So far we have not hit the issue on Jetson Nano devkits. Do you observe it on Nano devkits or your custom boards?

#32

I use same type of modules in custom board and evaluation board also (eval board: 945-13450-0000-100, S/N: 1422919091668))

I saw the issue every board…

This module is P3448 (180-13448-DAAA-B01) (900-13448-0020-000, S/N:1422720005556) which is use only eMMC.

Can you use only sd-card module?

#33

We use 5V~3A AC adapter, could you tell me what do you use for power supply? Maybe it is another different.

//

This main basic feature is very urgent for us. We are ready to sell our new product, but it is now impossible with this issue.

If there is any possibility to fix this issue faster? If that would cost us additionally towards your support team we are ready to pay this additional cost in order to get it fixed.