I’ve deployed three Jetson units, two devkits and one Connectech Rudi TX2. They are all using the same neural network confirmed by checking the hashes of the files. When I run the same image set on all three units I get different results from each unit. If I run the image set again on the same TX2 it will reliably produce the same results.
Is this normal? Sometimes the detections vary by more than 100%. For example, Connecteh Rudi will detect 1258 objects. With the same images and the same network on one of the devkits I get 692 objects detected.
There are minor software version differences. Can the version differences below really be causing this?
ConnectTech Rudi
nvidia@tegra-ubuntu:~$ apt list --installed | grep cuda
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
cuda-command-line-tools-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-core-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cublas-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cublas-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cudart-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cudart-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cufft-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cufft-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-curand-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-curand-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cusolver-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cusolver-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cusparse-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cusparse-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-documentation-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-driver-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-libraries-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-license-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-misc-headers-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-npp-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-npp-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvgraph-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvgraph-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvml-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvrtc-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvrtc-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-repo-l4t-9-0-local/now 9.0.252-1 arm64 [installed,local]
cuda-samples-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-toolkit-9-0/unknown,now 9.0.252-1 arm64 [installed]
libcudnn7/now 7.0.5.13-1+cuda9.0 arm64 [installed,upgradable to: 7.0.5.15-1+cud 9.0]
libcudnn7-dev/now 7.0.5.13-1+cuda9.0 arm64 [installed,upgradable to: 7.0.5.15-1 cuda9.0]
libcudnn7-doc/now 7.0.5.13-1+cuda9.0 arm64 [installed,local]
libgie-dev/unknown,now 4.0.4-1+cuda9.0 arm64 [installed]
libnvinfer-dev/unknown,now 4.0.4-1+cuda9.0 arm64 [installed,automatic]
libnvinfer-samples/unknown,now 4.0.4-1+cuda9.0 arm64 [installed,automatic]
libnvinfer4/unknown,now 4.0.4-1+cuda9.0 arm64 [installed,automatic]
nv-tensorrt-repo-ubuntu1604-ga-cuda9.0-trt3.0.4-20180208/now 1-1 arm64 [installed,local]
tensorrt/unknown,now 3.0.4-1+cuda9.0 arm64 [installed]
nvidia@tegra-ubuntu:~$
DevKit 1
nvidia@iceman:~$ apt list --installed | grep cuda
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
cuda-command-line-tools-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-core-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cublas-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cublas-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cudart-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cudart-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cufft-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cufft-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-curand-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-curand-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cusolver-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cusolver-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cusparse-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-cusparse-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-documentation-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-driver-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-libraries-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-license-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-misc-headers-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-npp-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-npp-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvgraph-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvgraph-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvml-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvrtc-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-nvrtc-dev-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-repo-l4t-9-0-local/now 9.0.252-1 arm64 [installed,local]
cuda-samples-9-0/unknown,now 9.0.252-1 arm64 [installed,automatic]
cuda-toolkit-9-0/unknown,now 9.0.252-1 arm64 [installed]
libcudnn7/now 7.1.5.14-1+cuda9.0 arm64 [installed,local]
libcudnn7-dev/now 7.1.5.14-1+cuda9.0 arm64 [installed,local]
libcudnn7-doc/now 7.1.5.14-1+cuda9.0 arm64 [installed,local]
libgie-dev/now 4.1.3-1+cuda9.0 arm64 [installed,local]
libnvinfer-dev/now 4.1.3-1+cuda9.0 arm64 [installed,local]
libnvinfer-samples/now 4.1.3-1+cuda9.0 arm64 [installed,local]
libnvinfer4/now 4.1.3-1+cuda9.0 arm64 [installed,local]
tensorrt/now 4.0.2.0-1+cuda9.0 arm64 [installed,local]
nvidia@iceman:~$