While developing our camera application based on TX2, I have noticed several bugs. But as it goes, without replication nothing could be solved.
Here is one: When broken, ISP sends few last frames forever.
The issue is traced down to nvcamera-daemon not doing any recovery. It can be triggered with this script:
#!/bin/bash datetime=`date +%Y%m%d-%H%M%S` log_file=/home/nvidia/nvcamd-$datetime.log #stop nvcamera-daemon killall nvcamera-daemon export enableCamPclLogs=1 export enableCamScfLogs=1 /usr/sbin/nvcamera-daemon 2>&1 | tee "$log_file"
Run this in one shell, while run a gstreamer with nvoverlaysink on another. As lot of output is produced, press SCROLL-LOCK in order to examine the messages - but after about tens of seconds the live image on the overlay stops - likely the buffer for scroll-lock is full and the nvcamera-daemon is put to not-running state. Pressing SCROLL-LOCK to deassert the blocking, will leave the program running again, but it will repeat the last frames. This happens with a high likelyhood and I have observed only 1 in 10 cases when the stream continues from live camera source.
The normal use case is to not use verbose messages, but the repeated frames issue was observed by us prior we knew there is this userspace dameon handling the processing. The result is - a high application load which causes scheduling inbalance will render the camera into non-working state. Sometimes it was enough to move windows around the Ubuntu desktop UI and it crashed the camera operation.
A recovery method needs to be put into nvcamera-deamon, because doing realtime tasks from regular userspace is not possible on TX2.
This was tested on single 12MP/30fps camera from Leopard Imaging.