The link to the inference benchmarks simply says to use ./trtexec (with options). This is not helpful. Unless I missed something, or perhaps it was different in previous Jetpack versions (I’m using 4.2), I had to figure out where this executable was located.
I found /usr/src/tensorrt/samples/trtexec and did “sudo make”, without first checking if there was already a /usr/src/tensorrt/bin/ . When I did the make, it complained about:
../Makefile.config:5: CUDA_INSTALL_DIR variable is not specified, using /usr/local/cuda by default, use CUDA_INSTALL_DIR=<cuda_directory> to change.
../Makefile.config:8: CUDNN_INSTALL_DIR variable is not specified, using $CUDA_INSTALL_DIR by default, use CUDNN_INSTALL_DIR=<cudnn_directory> to change.
but it finished the compile and now the executable is available in /bin. So I don’t know if it compiled correctly.
Perhaps it would be great if this was pre-compiled (with the correct links to the cuda and cuDNN install paths), and the command was already in the PATH so that “trtexec” could just be used in the terminal from anywhere, like nvpmodel and jetson_clocks are (now? they are for me in 4.2).
EDIT: well actually it looks like it doesn’t matter, because every time I try to run the ./trtexec from /usr/src/tensorrt/bin/ , it always says
Could not open file XXXX
CaffeParser: Could not parse deploy file
where I’ve tried for example “…/data/googlenet.prototxt”. I can definitely see some things in /usr/src/tensorrt/data/, but it doesn’t seem to be working.
Would be great if the inference benchmark documentation could be updated for Jetpack 4.2. Or maybe now it’s supposed to be run from the Python API?? Again, documentation …