Please provide complete information as applicable to your setup.
• Hardware Platform (GPU)
• DeepStream 6.4
• JetPack Version (valid for Jetson only)
• TensorRT 8.6
**• NVIDIA GPU Driver Version Driver Version: 535.183.06 * *
• Issue Type( questions, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
docker run --gpus all -it --runtime=nvidia --name deepstream_container --net=host --privileged -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.4 nvcr.io/nvidia/deepstream:6.4-triton-multiarch
I did it with this.
NVIDIA-SMI 535.183.06 Driver Version: 535.183.06 CUDA Version: 12.2
root@arstest:/opt/nvidia/deepstream/deepstream-6.4/samples# ls
buildModelPrimary_Detector.log prepare_classification_test_video.sh streams trtis_model_repo
configs prepare_ds_triton_model_repo.sh triton_model_repo video
models prepare_ds_triton_tao_model_repo.sh triton_tao_model_repo
I can’t move on.
root@arstest:/opt/nvidia/deepstream/deepstream-6.4/samples# sh prepare_ds_triton_tao_model_repo.sh
Using tao-converter utility from this location: /root/bin/tao-converter/tao-converter
Downloading PeopleNet Transformer model from NGC TAO repository
–2024-08-20 07:03:03-- https://api.ngc.nvidia.com/v2/models/nvidia/tao/peoplenet_transformer/versions/deployable_v1.0/files/resnet50_peoplenet_transformer.etlt
Resolving api.ngc.nvidia.com (api.ngc.nvidia.com)… 54.68.89.11, 52.13.150.89
Connecting to api.ngc.nvidia.com (api.ngc.nvidia.com)|54.68.89.11|:443… connected.
HTTP request sent, awaiting response… 302 Found
Location: https://files.ngc.nvidia.com/org/nvidia/team/tao/models/peoplenet_transformer/versions/deployable_v1.0/files/resnet50_peoplenet_transformer.etlt?versionId=yS_53lHDfTQO5ZWKSBz4TTZYG1l200KO&Expires=1724223784&Signature=hwPYsREv~iDCzdxpZWm3YSOJCunQtmbAyhrDjnYKhB4-owF-WSgqqtZEOQiprO1yUee3uYLspRhSOszoaf5l~T5OPMMXg7V9Vu~4Lvomc-1trrvLLXPkOQPfgoXCFuPy0~OWh6gRgc-~AU72xgbpQr6tpGpci5vSihq3UjxInDIkaMm72c7RzomTebApcbxm7GG4YVIHL1Q~o2Pd0nk~-SDqG~knC0D2VMZV-WS3fYSYi3FiwqD8W4D4y4KuEs9hw2lYf5dMJ3B6-15XTJ-WEbAYb~UKZWkq45u-GYBwtqOVvqVTw5qHcJsrxsbNcviYd2O6zpHx4Tgs9o41vy1UVw__&Key-Pair-Id=KCX06E8E9L60W [following]
–2024-08-20 07:03:04-- https://files.ngc.nvidia.com/org/nvidia/team/tao/models/peoplenet_transformer/versions/deployable_v1.0/files/resnet50_peoplenet_transformer.etlt?versionId=yS_53lHDfTQO5ZWKSBz4TTZYG1l200KO&Expires=1724223784&Signature=hwPYsREv~iDCzdxpZWm3YSOJCunQtmbAyhrDjnYKhB4-owF-WSgqqtZEOQiprO1yUee3uYLspRhSOszoaf5l~T5OPMMXg7V9Vu~4Lvomc-1trrvLLXPkOQPfgoXCFuPy0~OWh6gRgc-~AU72xgbpQr6tpGpci5vSihq3UjxInDIkaMm72c7RzomTebApcbxm7GG4YVIHL1Q~o2Pd0nk~-SDqG~knC0D2VMZV-WS3fYSYi3FiwqD8W4D4y4KuEs9hw2lYf5dMJ3B6-15XTJ-WEbAYb~UKZWkq45u-GYBwtqOVvqVTw5qHcJsrxsbNcviYd2O6zpHx4Tgs9o41vy1UVw__&Key-Pair-Id=KCX06E8E9L60W
Resolving files.ngc.nvidia.com (files.ngc.nvidia.com)… 13.225.114.81, 13.225.114.5, 13.225.114.25, …
Connecting to files.ngc.nvidia.com (files.ngc.nvidia.com)|13.225.114.81|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 90730303 (87M) [application/octet-stream]
Saving to: ‘/tmp/tao_models/peoplenet_transformer/resnet50_peoplenet_transformer.etlt’
/tmp/tao_models/peoplenet_transform 100%[=================================================================>] 86.53M 102MB/s in 0.8s
2024-08-20 07:03:05 (102 MB/s) - ‘/tmp/tao_models/peoplenet_transformer/resnet50_peoplenet_transformer.etlt’ saved [90730303/90730303]
–2024-08-20 07:03:05-- https://api.ngc.nvidia.com/v2/models/nvidia/tao/peoplenet_transformer/versions/deployable_v1.0/files/labels.txt
Resolving api.ngc.nvidia.com (api.ngc.nvidia.com)… 54.68.89.11, 52.13.150.89
Connecting to api.ngc.nvidia.com (api.ngc.nvidia.com)|54.68.89.11|:443… connected.
HTTP request sent, awaiting response… 302 Found
Location: https://files.ngc.nvidia.com/org/nvidia/team/tao/models/peoplenet_transformer/versions/deployable_v1.0/files/labels.txt?versionId=uH2d9oFJ6X8VDi6GJ7hYZNwaKxpyz7cL&Expires=1724223786&Signature=ekWynS8QxtfSJgLm-KG-Axl9ZjjnpDcoyJQoRwou~kMf4hyvlhYNIh2~K8Svy2CsAqKIf7SObxNkvMgUQ8iKoaLTh~nF3OUElp9FzxUg6WQYT6jISyS66vrN7mf43~sPGKY4pszlgY~j2rzulJkJyer1O3KqGtBu0568wwCTtFEMAupRqpSqe5ykvUAdqRn9dTrox9o3PoxEVTmeZ6Q7bu6ZK7mSQvS~3yplj0u8Cl8bJ4K0XYL2LjpMmKjkFMW49AE439EEw8mpI3LkXSR2BFNQ8ZKjMqGENiUY0rIQprjMNToKb~RViGrYX-X2Xe3LZlQIfpTs4~LI1BIz3OY74g__&Key-Pair-Id=KCX06E8E9L60W [following]
–2024-08-20 07:03:06-- https://files.ngc.nvidia.com/org/nvidia/team/tao/models/peoplenet_transformer/versions/deployable_v1.0/files/labels.txt?versionId=uH2d9oFJ6X8VDi6GJ7hYZNwaKxpyz7cL&Expires=1724223786&Signature=ekWynS8QxtfSJgLm-KG-Axl9ZjjnpDcoyJQoRwou~kMf4hyvlhYNIh2~K8Svy2CsAqKIf7SObxNkvMgUQ8iKoaLTh~nF3OUElp9FzxUg6WQYT6jISyS66vrN7mf43~sPGKY4pszlgY~j2rzulJkJyer1O3KqGtBu0568wwCTtFEMAupRqpSqe5ykvUAdqRn9dTrox9o3PoxEVTmeZ6Q7bu6ZK7mSQvS~3yplj0u8Cl8bJ4K0XYL2LjpMmKjkFMW49AE439EEw8mpI3LkXSR2BFNQ8ZKjMqGENiUY0rIQprjMNToKb~RViGrYX-X2Xe3LZlQIfpTs4~LI1BIz3OY74g__&Key-Pair-Id=KCX06E8E9L60W
Resolving files.ngc.nvidia.com (files.ngc.nvidia.com)… 13.225.114.69, 13.225.114.25, 13.225.114.81, …
Connecting to files.ngc.nvidia.com (files.ngc.nvidia.com)|13.225.114.69|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 21 [application/octet-stream]
Saving to: ‘triton_tao_model_repo/peoplenet_transformer/labels.txt’
triton_tao_model_repo/peoplenet_tra 100%[=================================================================>] 21 --.-KB/s in 0s
2024-08-20 07:03:06 (1.63 MB/s) - ‘triton_tao_model_repo/peoplenet_transformer/labels.txt’ saved [21/21]
Creating TensorRT engine file for PeopleNet Transformer
Setting batch size to 16
[INFO] [MemUsageChange] Init CUDA: CPU +14, GPU +0, now: CPU 20, GPU 294 (MiB)
[INFO] [MemUsageChange] Init builder kernel library: CPU +889, GPU +174, now: CPU 985, GPU 468 (MiB)
[INFO] ----------------------------------------------------------------
[INFO] Input filename: /tmp/fileowPggk
[INFO] ONNX IR version: 0.0.8
[INFO] Opset version: 12
[INFO] Producer name: pytorch
[INFO] Producer version: 1.13.0
[INFO] Domain:
[INFO] Model version: 0
[INFO] Doc string:
[INFO] ----------------------------------------------------------------
[WARNING] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[WARNING] onnx2trt_utils.cpp:514: Your ONNX model has been generated with double-typed weights, while TensorRT does not natively support double. Attempting to cast down to float.
[WARNING] onnx2trt_utils.cpp:400: One or more weights outside the range of INT32 was clamped
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] No importer registered for op: MultiscaleDeformableAttnPlugin_TRT. Attempting to import as plugin.
[INFO] Searching for plugin: MultiscaleDeformableAttnPlugin_TRT, plugin_version: 1, plugin_namespace:
[INFO] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRT
[INFO] Detected input dimensions from the model: (-1, 3, 544, 960)
[INFO] Model has dynamic shape. Setting up optimization profiles.
[INFO] Using optimization profile min shape: (1, 3, 544, 960) for input: inputs
[INFO] Using optimization profile opt shape: (4, 3, 544, 960) for input: inputs
[INFO] Using optimization profile max shape: (16, 3, 544, 960) for input: inputs
[INFO] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[WARNING] Detected layernorm nodes in FP16: Sub_508, Pow_510, ReduceMean_511, Add_513, Sqrt_514, Div_515, Mul_516, Add_517, Sub_527, Pow_529, ReduceMean_530, Add_532, Sqrt_533, Div_534, Mul_535, Add_536, Sub_625, Pow_627, ReduceMean_628, Add_630, Sqrt_631, Div_632, Mul_633, Add_634, Sub_644, Pow_646, ReduceMean_647, Add_649, Sqrt_650, Div_651, Mul_652, Add_653, Sub_742, Pow_744, ReduceMean_745, Add_747, Sqrt_748, Div_749, Mul_750, Add_751, Sub_761, Pow_763, ReduceMean_764, Add_766, Sqrt_767, Div_768, Mul_769, Add_770, Sub_859, Pow_861, ReduceMean_862, Add_864, Sqrt_865, Div_866, Mul_867, Add_868, Sub_878, Pow_880, ReduceMean_881, Add_883, Sqrt_884, Div_885, Mul_886, Add_887, Sub_976, Pow_978, ReduceMean_979, Add_981, Sqrt_982, Div_983, Mul_984, Add_985, Sub_995, Pow_997, ReduceMean_998, Add_1000, Sqrt_1001, Div_1002, Mul_1003, Add_1004, Sub_1093, Pow_1095, ReduceMean_1096, Add_1098, Sqrt_1099, Div_1100, Mul_1101, Add_1102, Sub_1112, Pow_1114, ReduceMean_1115, Add_1117, Sqrt_1118, Div_1119, Mul_1120, Add_1121, Sub_1232, Pow_1234, ReduceMean_1235, Add_1237, Sqrt_1238, Div_1239, Mul_1240, Add_1241, Sub_1330, Pow_1332, ReduceMean_1333, Add_1335, Sqrt_1336, Div_1337, Mul_1338, Add_1339, Sub_1349, Pow_1351, ReduceMean_1352, Add_1354, Sqrt_1355, Div_1356, Mul_1357, Add_1358, Sub_1538, Pow_1540, ReduceMean_1541, Add_1543, Sqrt_1544, Div_1545, Mul_1546, Add_1547, Sub_1640, Pow_1642, ReduceMean_1643, Add_1645, Sqrt_1646, Div_1647, Mul_1648, Add_1649, Sub_1659, Pow_1661, ReduceMean_1662, Add_1664, Sqrt_1665, Div_1666, Mul_1667, Add_1668, Sub_1770, Pow_1772, ReduceMean_1773, Add_1775, Sqrt_1776, Div_1777, Mul_1778, Add_1779, Sub_1872, Pow_1874, ReduceMean_1875, Add_1877, Sqrt_1878, Div_1879, Mul_1880, Add_1881, Sub_1891, Pow_1893, ReduceMean_1894, Add_1896, Sqrt_1897, Div_1898, Mul_1899, Add_1900, Sub_2002, Pow_2004, ReduceMean_2005, Add_2007, Sqrt_2008, Div_2009, Mul_2010, Add_2011, Sub_2104, Pow_2106, ReduceMean_2107, Add_2109, Sqrt_2110, Div_2111, Mul_2112, Add_2113, Sub_2123, Pow_2125, ReduceMean_2126, Add_2128, Sqrt_2129, Div_2130, Mul_2131, Add_2132, Sub_2234, Pow_2236, ReduceMean_2237, Add_2239, Sqrt_2240, Div_2241, Mul_2242, Add_2243, Sub_2336, Pow_2338, ReduceMean_2339, Add_2341, Sqrt_2342, Div_2343, Mul_2344, Add_2345, Sub_2355, Pow_2357, ReduceMean_2358, Add_2360, Sqrt_2361, Div_2362, Mul_2363, Add_2364, Sub_2466, Pow_2468, ReduceMean_2469, Add_2471, Sqrt_2472, Div_2473, Mul_2474, Add_2475, Sub_2568, Pow_2570, ReduceMean_2571, Add_2573, Sqrt_2574, Div_2575, Mul_2576, Add_2577, Sub_2587, Pow_2589, ReduceMean_2590, Add_2592, Sqrt_2593, Div_2594, Mul_2595, Add_2596
[WARNING] Running layernorm after self-attention in FP16 may cause overflow. Exporting the model to the latest available ONNX opset (later than opset 17) to use the INormalizationLayer, or forcing layernorm layers to run in FP32 precision can help with preserving accuracy.
[INFO] Graph optimization time: 0.257572 seconds.
[INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 1170, GPU 526 (MiB)
[INFO] [MemUsageChange] Init cuDNN: CPU +0, GPU +10, now: CPU 1170, GPU 536 (MiB)
[INFO] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[INFO] Local timing cache in use. Profiling results in this builder pass will not be stored.
I’m stopped.