Orin AGX CUDA/Tensorrt C++ segfault on encrypted drive but works on unencrypted drive

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
, ,
1

We’ve been flashing our Jetson Orins with encryption and we’re using c++ and tensorrt to load an onnx file and for some reason it’s causing a segfault on the encrypted drive. I’ve tested the exact same code on an unencrypted drive, even on the same jetson and it works fine. Anyone experienced anything like this before?

Hardware

  • Jetson Orin AGX
  • Avermedia D315 carrier board
  • 2TB samsung 980 PRO hard drive

Tests Performed

  • Flash same jetson with encrypted drive and normal drive. Exactly the same software. Segfault when encrypted, works fine when unencrypted.
  • Flash a different jetson with a different hard drive. Same as above
  • Flash a jetson with a 256GB SSD with encryption, weirdly enough it works fine on this setup?!
2

*** Please note that this reply is generated by LLM automatically ***

The issue you’re experiencing with a segfault on an encrypted drive with your Jetson Orin AGX, Avermedia D315 carrier board, and 2TB Samsung 980 PRO hard drive is unusual, especially since the same code works fine on an unencrypted drive. Given the tests you’ve performed, here are some potential reasons and comparisons to consider:

  1. Encryption overhead: When using disk encryption, there’s an additional overhead due to the encryption and decryption processes. This might cause issues with certain applications, especially those that rely heavily on disk I/O operations. However, in your case, the code works fine on an unencrypted drive, so this might not be the primary cause.

  2. File system and storage: You’ve tested the code on a 2TB Samsung 980 PRO hard drive with encryption, which resulted in a segfault. However, when you tested it on a 256GB SSD with encryption, it worked fine. This suggests that the issue might be related to the storage device or file system. The SSD might be more resilient to the encryption overhead or have better error correction mechanisms.

  3. Device-specific issues: The Avermedia D315 carrier board might have some device-specific issues or quirks that are causing the problem. You’ve tested the code on a different Jetson with a different hard drive, and the issue persisted. However, it’s still possible that the carrier board is contributing to the problem.

  4. ONNX file loading: The segfault occurs when loading an ONNX file using C and TensorRT. This might indicate an issue with the way the file is being loaded or processed. You could try loading the ONNX file on the unencrypted drive and then copying it to the encrypted drive to see if the issue persists.

  5. TensorRT and C library issues: There might be an issue with the TensorRT and C libraries you’re using. You could try updating these libraries or using different versions to see if the problem resolves.

To further troubleshoot the issue, you could try:

  • Checking the system logs for any error messages related to the segfault
  • Running a memory debugger like Valgrind to detect any memory-related issues
  • Testing the code on a different file system or storage device
  • Disabling encryption and testing the code with other security features enabled
  • Reaching out to the Avermedia support team to see if they’ve encountered similar issues with their carrier board

In summary, the issue seems to be related to the combination of the Jetson Orin AGX, Avermedia D315 carrier board, and 2TB Samsung 980 PRO hard drive with encryption. Further testing and debugging are needed to determine the root cause of the problem.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

5

There is no update from you for a period, assuming this is not an issue anymore.
Hence, we are closing this topic. If need further support, please open a new one.
Thanks ~1216

Hi,

To know more about your issue, could you try to deploy your model with trtexec and share the output with us?
Thanks.