We have identified a production issue with the Jetson Orin NX 16GB. We use the e-cam24 CUNX camera with a custom driver that e-con Systems has made available to us. We have already received 83 Jetson for our series production. However, the processors are exhibiting instability, with 21 of them malfunctioning. We have already ordered another 147 pieces that we need for production.
We unpacked the Jetson Orin NX with great care and in compliance with ESD regulations. We plugged each Jetson into the Xavier NX developer kit and flashed it. Then we checked whether we received a camera image. With 21 pieces we got a frozen or a green picture. Even with repeated flashing, we could not fix the problem.
The hardware and the test setup was exactly the same for every test. Is such a problem known? How can we solve this problem? Such a high error rate is not acceptable to us and makes the system unstable.
We strongly assume that this is a production error as we had no problems with the Xavier NX last year. We urgently need to find a solution for this.
We are currently undertaking an intensive testing process for each processor, which is highly time-consuming and not sustainable in the long run. Our testing procedure is as follows: We only approve the Jetson for mass production if it passes our rigorous tests. Initially, we power up the Jetson and run DeepStream (video stream) for 2 minutes. After shutting the jetson down, we repeat this process ten times. Following these initial tests, we then run DeepStream continuously for 3 hours. It’s not uncommon for the camera to crash during the initial power tests. I use the ‘dmesg | grep -i csi’ command to detect the root of the problem. In some instances, the issue only surfaces after 2 hours of extended testing.
[ 1.571226] SCSI subsystem initialized
[ 2.204085] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 240)
[ 6.736645] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: context isolation disabled due to no IOMMU
[ 6.745925] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: initialized
[ 6.752379] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e40000.host1x:nvcsi@15a00000- bound
[ 6.761600] tegra-camrtc-capture-vi tegra-capture-vi: subdev 13e40000.host1x:nvcsi@15a00000- bound
[ 39.186506] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[ 39.204882] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
We have been in communication with E-con systems for several months, initially suspecting an issue with their driver. After exhausting all possible solutions, it now appears to be a hardware issue. This error occurs both with GStreamer and OpenCV. We connected a USB camera for testing and did not find any problems with it. Are you aware of such a problem? What do you suggest we do with the faulty processors? This is a significant concern for us, as our company’s operations heavily rely on the stable and consistent performance of these processors.
e-CAM24_CUNX_JETSON_L4T35.3.1_25-APR-2023_R01_RC1_VERSION3.zip (189.1 KB)
ReadMe.txt (7.7 KB)
We are currently using jetpack 5.1.1