I tried to run the pick and place example application in a docker container (built using the steps given here: https://docs.nvidia.com/isaac/isaac/doc/getting_started.html#using-docker) using the following command:
bazel run //apps/samples/pick_and_place – --arm ur10
I got the following error:
2021-08-06 22:44:54.406 WARN external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet '_engine_tcp_udp_asio_context/isaac.alice.AsioContext' was not added to scheduler because no tick method is specified.
2021-08-06 22:44:54.406 INFO packages/sight/WebsightServer.cpp@247: Sight webserver is loaded
2021-08-06 22:44:54.406 INFO packages/sight/WebsightServer.cpp@248: Please open Chrome Browser and navigate to http://<ip address>:3000
2021-08-06 22:44:54.406 WARN external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet 'websight/isaac.sight.AliceSight' was not added to scheduler because no tick method is specified.
2021-08-06 22:44:54.407 WARN external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet 'controller.kinematic_tree/KinematicTree' was not added to scheduler because no tick method is specified.
2021-08-06 22:44:54.407 DEBUG external/com_nvidia_isaac_engine/engine/alice/backend/event_manager.cpp@40: Stopping node 'pose_initializer' because it reached status 'SUCCESS'
2021-08-06 22:44:54.407 WARN external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
2021-08-06 22:44:54.407 WARN external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
2021-08-06 22:44:54.417 WARN external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet '_check_operating_system/isaac.alice.CheckOperatingSystem' was not added to scheduler because no tick method is specified.
2021-08-06 22:44:54.417 WARN external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
2021-08-06 22:44:54.418 WARN external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
2021-08-06 22:44:54.424 WARN packages/ml/TensorRTInference.cpp@463: Could not read from .plan (path is set to 'external/sortbot_pose_estimation_models/resnet18_detector_kltSmall.plan'). Falling back to building the .plan from frozen model external/sortbot_pose_estimation_models/resnet18_detector_kltSmall.etlt. Note: this process may take up to several minutes.
2021-08-06 22:45:04.403 INFO external/com_nvidia_isaac_engine/engine/alice/backend/allocator_backend.cpp@57: Optimized memory CPU allocator.
2021-08-06 22:45:04.403 INFO external/com_nvidia_isaac_engine/engine/alice/backend/allocator_backend.cpp@66: Optimized memory CUDA allocator.
2021-08-06 22:46:10.778 WARN packages/ml/TensorRTInference.cpp@174: TRT WARNING: Half2 support requested on hardware without native FP16 support, performance will be negatively affected.
2021-08-06 22:47:55.626 ERROR packages/ml/TensorRTInference.cpp@171: TRT ERROR: ../rtSafe/cuda/caskUtils.cpp (98) - Assertion Error in trtSmToCask: 0 (Unsupported SM.)
2021-08-06 22:47:55.626 PANIC packages/ml/TensorRTInference.cpp@258: Failed to build TensorRT engine from external/sortbot_pose_estimation_models/resnet18_detector_kltSmall.etlt.
I found out that there is no TensorRT installed and I thought that if I install TensorRT, the error will go away. I was wrong. Then, I found out that for the graphics processor I have, which is NVIDIA GeForce RTX 3080 Laptop GPU, I must install CUDA 11.1 and TensorRT 7.2.1 (as per the solution here: RTX 3070 / TensorRT Internal Error: Assertion failed: Unsupported SM and here: 3080 support · Issue #800 · NVIDIA/TensorRT · GitHub). After a lot of struggle over the course of a few days, I finally managed to obtain exactly these versions of CUDA and TensorRT in a docker container along with the other dependencies required for working with Isaac SDK (I used this command to install other dependencies: engine/engine/build/scripts/install_dependencies.sh --config=x86_64_cuda_11_1
). No luck still. The error remains the same:
2021-08-06 23:44:33.986 WARN packages/ml/TensorRTInference.cpp@463: Could not read from .plan (path is set to 'external/sortbot_pose_estimation_models/resnet18_detector_kltSmall.plan'). Falling back to building the .plan from frozen model external/sortbot_pose_estimation_models/resnet18_detector_kltSmall.etlt. Note: this process may take up to several minutes.
2021-08-06 23:44:43.936 INFO external/com_nvidia_isaac_engine/engine/alice/backend/allocator_backend.cpp@57: Optimized memory CPU allocator.
2021-08-06 23:44:43.936 INFO external/com_nvidia_isaac_engine/engine/alice/backend/allocator_backend.cpp@66: Optimized memory CUDA allocator.
2021-08-06 23:47:34.139 WARN packages/ml/TensorRTInference.cpp@174: TRT WARNING: Half2 support requested on hardware without native FP16 support, performance will be negatively affected.
2021-08-06 23:49:55.527 ERROR packages/ml/TensorRTInference.cpp@171: TRT ERROR: ../rtSafe/cuda/caskUtils.cpp (98) - Assertion Error in trtSmToCask: 0 (Unsupported SM.)
2021-08-06 23:49:55.527 PANIC packages/ml/TensorRTInference.cpp@258: Failed to build TensorRT engine from external/sortbot_pose_estimation_models/resnet18_detector_kltSmall.etlt.
For reference, the output of the python3 ../engine/engine/build/scripts/version_checker.py
command in the docker container I have created looks like this:
---------------------------------------------------------------
|Package |Recommended Version |Current Version |
---------------------------------------------------------------
|OS |Ubuntu 18.04.2 LTS |Ubuntu 18.04.5 LTS |
|Bazel |3.1.0 |3.1.0 |
|GPU_Driver |>=440 |470.57.02 |
|Cuda |10.2.x |11.1.105 |
|Cudnn |8.0.3.x |8.0.5.39 |
|TensorRt |7.1.5 |7.2.1.6 |
|TensorFlow |1.15.0 |2.5.0 |
|pycapnp |>=0.6.3 |0.6.4 |
|librosa |>=0.6.3 |0.8.0 |
|SoundFile |>=0.10.2 |0.10.3.post1 |
|Python2 |2.7.x |2.7.17 |
|Python3 |3.6.x |3.6.9 |
---------------------------------------------------------------
Environment
TensorRT Version : 7.2.1.6
GPU Type : RTX3080 Laptop GPU
Nvidia Driver Version : 470.57.02
CUDA Version : 11.1.1
CUDNN Version : 8.0.5.39
Operating System + Version : Ubuntu 18.04.5
Python Version (if applicable) : 3.6.9