Jetson nano nvv4l2h264enc stop working sometimes

Autonomous Machines Jetson & Embedded Systems Jetson Nano
22

Hi,
We try on r32.6.1/Jetson Nano B01 + Logitech Brio 4K USB camera and run this pipeline:

cap = cv2.VideoCapture("v4l2src device=/dev/video1 ! video/x-raw, width=1920
,height=1080,format=YUY2,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM),
format=I420 ! tee name=camNVMM ! queue ! nvv4l2h264enc preset-level=1 MeasureEnc
oderLatency=1 maxperf-enable=true insert-vui=true insert-sps-pps=1 bitrate=10000
000 control-rate=1 vbv-size=734000 ! tee name=h264_stream ! queue ! h264parse !
rtph264pay ! udpsink host=10.19.107.227 port=5000 camNVMM. ! queue ! nvvidconv !
 video/x-raw, format=GRAY8, width=640,height=480 ! appsink")

Do not observe the issue in one-hour run. Always can see NVENC in tegrastats:

RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [28%@1479,16%@1479,13%@1479,15%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 7%@921 NVENC 716 APE 25 PLL@29.5C CPU@33C PMIC@50C GPU@31.5C AO@38.5C thermal@32C POM_5V_IN 3604/3707 POM_5V_GPU 204/217 POM_5V_CPU 695/675
RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [29%@1479,14%@1479,14%@1479,12%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 8%@921 NVENC 716 APE 25 PLL@30C CPU@32.5C PMIC@50C GPU@31.5C AO@38C thermal@32C POM_5V_IN 3721/3707 POM_5V_GPU 245/218 POM_5V_CPU 695/675
RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [25%@1479,18%@1479,10%@1479,17%@1479] EMC_FREQ 12%@1600 GR3D_FREQ 5%@921 NVENC 716 APE 25 PLL@30C CPU@32.5C PMIC@50C GPU@31.5C AO@38C thermal@32.25C POM_5V_IN 3645/3707 POM_5V_GPU 204/218 POM_5V_CPU 695/675
RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [28%@1479,13%@1479,15%@1479,14%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 8%@921 NVENC 716 APE 25 PLL@29.5C CPU@33C PMIC@50C GPU@31.5C AO@38.5C thermal@32C POM_5V_IN 3762/3707 POM_5V_GPU 245/218 POM_5V_CPU 654/675
RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [28%@1479,16%@1479,16%@1479,13%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 4%@921 NVENC 716 APE 25 PLL@29.5C CPU@32.5C PMIC@50C GPU@32C AO@38.5C thermal@32C POM_5V_IN 3563/3707 POM_5V_GPU 204/218 POM_5V_CPU 695/675
RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [31%@1479,13%@1479,13%@1479,18%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 8%@921 NVENC 716 APE 25 PLL@29.5C CPU@32.5C PMIC@50C GPU@31.5C AO@38.5C thermal@32C POM_5V_IN 3844/3707 POM_5V_GPU 245/218 POM_5V_CPU 695/675
RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [28%@1479,16%@1479,13%@1479,14%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 3%@921 NVENC 716 APE 25 PLL@29.5C CPU@33C PMIC@50C GPU@31.5C AO@38C thermal@32.25C POM_5V_IN 3604/3707 POM_5V_GPU 204/218 POM_5V_CPU 654/675
RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [30%@1479,17%@1479,13%@1479,17%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 9%@921 NVENC 716 APE 25 PLL@29.5C CPU@33C PMIC@50C GPU@31.5C AO@38.5C thermal@32C POM_5V_IN 3762/3707 POM_5V_GPU 245/218 POM_5V_CPU 695/675
RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [28%@1479,16%@1479,11%@1479,18%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 3%@921 NVENC 716 APE 25 PLL@29.5C CPU@33C PMIC@50C GPU@32C AO@38C thermal@32C POM_5V_IN 3604/3707 POM_5V_GPU 204/218 POM_5V_CPU 613/675
RAM 1776/3956MB (lfb 103x4MB) SWAP 192/1978MB (cached 19MB) IRAM 0/252kB(lfb 252kB) CPU [28%@1479,19%@1479,10%@1479,20%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 8%@921 NVENC 716 APE 25 PLL@29.5C CPU@33C PMIC@50C GPU@31.5C AO@38.5C thermal@32.5C POM_5V_IN 3721/3707 POM_5V_GPU 245/218 POM_5V_CPU 695/675
23

The case occurs randomly. It may have no issue number of days. It also may have number of times within one hour. I don’t know how to reproduce it quickly at this moment. But the issue occurs more frequently recent. On 16/12/2021, I tested on three developer kits, all of them have the same issue.
Is there any log file helpful for your investigation, such as system log?
Thanks.

24

Hi,
We run overnight test and don’t hit the issue. NVENC is always present in tegrastats. We would like suggest try this string:

#define TEST_1    "videotestsrc \
    ! video/x-raw, format=UYVY, width=(int)1280, height=(int)720, framerate=(fraction)24/1 \
    ! nvvidconv ! video/x-raw(memory:NVMM), format=I420, width=(int)2592, height=(int)1944, \
    ! tee name=camNVMM ! queue \
    ! nvv4l2h264enc preset-level=1 MeasureEncoderLatency=1 maxperf-enable=true insert-vui=true insert-sps-pps=1 bitrate=10000000 control-rate=1 vbv-size=734000 \
    ! tee name=h264_stream ! queue ! h264parse ! rtph264pay ! udpsink clients=192.168.18.18:38298 \
    h264_stream. ! queue ! h264parse ! matroskamux ! filesink location=test_h264.mkv \
    camNVMM. ! queue ! nvvidconv ! video/x-raw, format=GRAY8, width=640,height=480 ! appsink"

Maybe the camera consumes much power and triggers instability. Please try videotestsrc.

25

Can’t run the suggested test string. This is the error message:

Debugging starts
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (711) open OpenCV | GStreamer warning: Error opening bin: could not parse caps "video/x-raw(memory:NVMM), format=I420, width=(int)2592, height=(int)1944,"
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
ERROR: Could not open camera

Debugging has finished
26

I found the issue of the test string did not work is this line

! nvvidconv ! video/x-raw(memory:NVMM), format=I420, width=(int)2592, height=(int)1944, \

After remove the last “,” the test code works now.
I am testing if the encoder stopped work in this case

27

I tested. It has the same issue as the camera.
I tried to log the tegratats output to a file based on https://docs.nvidia.com/drive/drive_os_5.1.6.1L/nvvib_docs/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/Utilities/util_tegrastats.html from a terminal, but the log file is different from the output of by clicking on Run tegrastats from top right corner of the window as the NVENC not logged to file.
How can I log the tegrastats output to a file with NVENC when the encoder is running?

Any suggestion?

Thanks

1 Like
28

Has your developer kid board installed an Intel m.2 key E WIFI module: 8265? If no, are you able to put one Intel WIFI module on and test again?

I connected the Jetson nano console serial port to my laptop, I have noticed that it has something wrong on PCIe port during the video streaming,
This is the error message from Jetson nano console:

[13566.467755] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=                                                hysical Layer, id=0008(Receiver ID)
[13566.479124] pcieport 0000:00:01.0:   device [10de:0fae] error status/mask=00                                                00001/00002000
[13566.487622] pcieport 0000:00:01.0:    [ 0] Receiver Error         (First)
[13588.159816] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=                                                hysical Layer, id=0008(Receiver ID)
[13588.170532] pcieport 0000:00:01.0:   device [10de:0fae] error status/mask=00                                                00001/00002000
[13588.178977] pcieport 0000:00:01.0:    [ 0] Receiver Error         (First)
[13644.437167] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=                                                hysical Layer, id=0008(Receiver ID)
[13644.447817] pcieport 0000:00:01.0:   device [10de:0fae] error status/mask=00                                                00001/00002000
[13644.456292] pcieport 0000:00:01.0:    [ 0] Receiver Error         (First)
[13752.403013] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=                                                hysical Layer, id=0008(Receiver ID)
[13752.413467] pcieport 0000:00:01.0:   device [10de:0fae] error status/mask=00                                                00001/00002000
[13752.421926] pcieport 0000:00:01.0:    [ 0] Receiver Error         (First)

Do you know is there any link between the hardware encoder nvv4l2h264 and the PCIe port?
If has, the encoder stopped work sometimes may be the Inter WIFI module compatible issue?

This is just for your reference.
Thanks

CX

29

I tested videotestsrc on another Jetson nano developer kid without Intel WIFI module, the NVENC missing has been detected as well. Please refer to the part of tegrastats log below:

RAM 1822/3963MB (lfb 80x4MB) SWAP 9/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%@1036,18%@1036,18%@1036,19%@1036] EMC_FREQ 11%@1600 GR3D_FREQ 19%@153 NVENC 716 APE 25 PLL@39C CPU@39C PMIC@50C GPU@39C AO@49C thermal@38.75C POM_5V_IN 3500/3604 POM_5V_GPU 81/91 POM_5V_CPU 814/820
RAM 1822/3963MB (lfb 80x4MB) SWAP 9/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [28%@825,26%@825,24%@825,15%@825] EMC_FREQ 11%@1600 GR3D_FREQ 23%@153 NVENC 716 APE 25 PLL@39C CPU@39C PMIC@50C GPU@39C AO@48.5C thermal@38.75C POM_5V_IN 3616/3604 POM_5V_GPU 162/91 POM_5V_CPU 853/820
RAM 1822/3963MB (lfb 80x4MB) SWAP 9/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [31%@710,24%@710,21%@710,21%@710] EMC_FREQ 11%@1600 GR3D_FREQ 0%@153 NVENC 716 APE 25 PLL@39C CPU@39C PMIC@50C GPU@39C AO@48.5C thermal@38.75C POM_5V_IN 3622/3604 POM_5V_GPU 81/91 POM_5V_CPU 853/820
RAM 1822/3963MB (lfb 80x4MB) SWAP 9/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [19%@825,9%@825,11%@825,18%@825] EMC_FREQ 9%@1600 GR3D_FREQ 0%@153 APE 25 PLL@38.5C CPU@39C PMIC@50C GPU@38.5C AO@48C thermal@39C POM_5V_IN 2383/3604 POM_5V_GPU 41/91 POM_5V_CPU 205/820
RAM 1822/3963MB (lfb 80x4MB) SWAP 9/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [40%@825,32%@825,35%@825,23%@825] EMC_FREQ 11%@1600 GR3D_FREQ 19%@153 NVENC 716 APE 25 PLL@39C CPU@39.5C PMIC@50C GPU@39C AO@48.5C thermal@39C POM_5V_IN 3342/3604 POM_5V_GPU 81/91 POM_5V_CPU 693/820
RAM 1822/3963MB (lfb 80x4MB) SWAP 9/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [45%@1479,32%@1479,21%@1479,19%@1479] EMC_FREQ 11%@1600 GR3D_FREQ 36%@153 NVENC 716 APE 25 PLL@39C CPU@39.5C PMIC@50C GPU@38.5C AO@48.5C thermal@39C POM_5V_IN 3541/3604 POM_5V_GPU 122/91 POM_5V_CPU 773/820
RAM 1822/3963MB (lfb 80x4MB) SWAP 9/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [24%@614,24%@614,22%@614,22%@614] EMC_FREQ 11%@1600 GR3D_FREQ 19%@153 NVENC 716 APE 25 PLL@39C CPU@39C PMIC@50C GPU@39C AO@48.5C thermal@39C POM_5V_IN 3342/3604 POM_5V_GPU 81/91 POM_5V_CPU 652/820
RAM 1822/3963MB (lfb 80x4MB) SWAP 9/1981MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [32%@921,20%@921,22%@921,22%@921] EMC_FREQ 11%@1600 GR3D_FREQ 0%@153 NVENC 716 APE 25 PLL@39C CPU@39C PMIC@50C GPU@38.5C AO@48.5C thermal@39C POM_5V_IN 3302/3604 POM_5V_GPU 122/91 POM_5V_CPU 693/820
30

Is there any suggestion on this case?
Thanks

31

Hi,
Somehow we are not able to run long run tests and observe the issue. Do you use Nano module with emmc? There are several Nano modules and the module with emmc is production module.

And may check if the adapter meets requirement:
Jetson Nano FAQ
Q: What power supply should I use with Jetson Nano Developer Kit?

32

Hi
I knew you are not able to run long run test that is why I expect you give me some suggestion if it is able to get some debug info or log from OS or run any debug test to get the data for you to analysis.
I am using Jetson nano developer kit module with rev B carry board. I don’t know if it has emmc or not.
Our product will use your product module with emmc, but the carry board for our product is not available at this moment. Can I put a Jetson nano product module on rev B carry board?
Thanks

CX

33

Hi,
The symptom is more like instability in hardware. Do you have other power supply for a try? Maybe in the condition current from power supply is not enough.

34

I have another power supply unit. Both of them output current at 4A.
I think the current is not the issue.
Do you know that is there any requirement of noise level for the DC power supplying to Jetson nano module? I am thinking if it is possible that the noise on main power passed to the DC power at some level and interference the Jetson nano working reliability?
I have noticed that if the issue occurred on some day, it more easily to reproduce the issue on that day. If the issue does not occur on some day, it almost has no issue on that day.
I do not test the noise level of the DC power supply unit.

36

If you suspect it is noise issue of your DC power supply, you should measure the power supply to check if it is stable first. Basically the key point is to make the 5V output of onboard switch is stable like listed in module datatsheet.

38

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.