Applications dying with "ILLEGAL INSTRUCTION" error

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Hardware Platform: [DRIVE AGX Xavier™ Developer Kit]
Software Version: [DRIVE Software 10]
Host Machine Version: [native Ubuntu 18.04]
SDK Manager Version: [1.2.0.6733]

I have been encountering very frequent instances of applications dying with “Illegal Instruction” error.
This seems to be happening when the system is experiencing heavy disk I/O (not sure if this is related)
This is happening most frequently with our NvSIPL camera driver application but I am able to reproduce this even with simple unix commands like mv and stress

Screenshot from 2020-11-15 12-33-50
Screenshot from 2020-11-15 12-33-50972×92 24.2 KB

Wondering if anyone else has experienced anything similar? Is this some know bug in Drive OS 5.1.6.1?
Would moving to 5.1.15.1 help with this issue?

2

Dear @anurag08upx,
Can you share reproducible case which can be tested on our board. If possible, could you check flashing DRIVE SW 10.0 again to check if the issue persists?

4

Dear @anurag08upx,
Can you provide any update?

5

I believe the issue is related to kernel disk cache usage. Currently I am able to avoid such crashes by periodically flushing the disk cache with echo 3 | sudo tee /proc/sys/vm/drop_caches
But this doesn’t feel like a viable long term solution.

I will try to create/post a reproducible example using just dd and stress