I’m trying to use the ‘TLT CV Inference Pipeline Quick Start Scripts’. tlt_cv_start_server.sh seems to be unable to find the GPU.
Can you tell me what is causing this?
environment: Jetson Nano & JetPack 4.5.1
~/workspace/quickstart/scripts$ bash tlt_cv_start_server.sh
[INFO] Starting TLT CV Inference Triton Server...
I0607 08:31:00.043801 1 pinned_memory_manager.cc:199] Pinned memory pool is created at '0x100c60000' with size 268435456
E0607 08:31:00.057894 1 model_repository_manager.cc:1705] instance group emotionmlp_tlt_0 of model emotionmlp_tlt has kind KIND_GPU but no GPUs are available
I0607 08:31:00.072456 1 model_repository_manager.cc:810] loading: bodypose_320x224_tlt:1
I0607 08:31:00.072712 1 model_repository_manager.cc:810] loading: facedetect_tlt:1
I0607 08:31:00.073164 1 model_repository_manager.cc:810] loading: bodypose_320x224_postprocess_tlt:1
I0607 08:31:00.073768 1 model_repository_manager.cc:810] loading: facedetect_postprocess_tlt:1
I0607 08:31:00.075094 1 model_repository_manager.cc:810] loading: faciallandmarks_tlt:1
I0607 08:31:00.075152 1 custom_backend.cc:198] Creating instance bodypose_320x224_postprocess_tlt_0_0_cpu on CPU using libbppost.so
I0607 08:31:00.077148 1 model_repository_manager.cc:810] loading: gaze_facegrid_tlt:1
I0607 08:31:00.077952 1 custom_backend.cc:198] Creating instance facedetect_postprocess_tlt_0_0_cpu on CPU using libodpost.so
I0607 08:31:00.078408 1 model_repository_manager.cc:810] loading: hcgesture_tlt:1
I0607 08:31:00.084121 1 model_repository_manager.cc:810] loading: heartrate_two_branch_tlt:1
E0607 08:31:00.111168 1 model_repository_manager.cc:986] failed to load 'faciallandmarks_tlt' version 1: Invalid argument: instance group faciallandmarks_tlt of model faciallandmarks_tlt must be KIND_GPU and must specify at least one GPU id
E0607 08:31:00.324385 1 model_repository_manager.cc:986] failed to load 'heartrate_two_branch_tlt' version 1: Invalid argument: instance group heartrate_two_branch_tlt of model heartrate_two_branch_tlt must be KIND_GPU and must specify at least one GPU id
I0607 08:31:00.469827 1 custom_backend.cc:198] Creating instance bodypose_320x224_postprocess_tlt_0_1_cpu on CPU using libbppost.so
E0607 08:31:01.079156 1 model_repository_manager.cc:986] failed to load 'gaze_facegrid_tlt' version 1: Invalid argument: instance group gaze_facegrid_tlt of model gaze_facegrid_tlt must be KIND_GPU and must specify at least one GPU id
I0607 08:31:01.178349 1 custom_backend.cc:198] Creating instance facedetect_postprocess_tlt_0_1_cpu on CPU using libodpost.so
I0607 08:31:01.179624 1 model_repository_manager.cc:983] successfully loaded 'bodypose_320x224_postprocess_tlt' version 1
I0607 08:31:01.180368 1 model_repository_manager.cc:983] successfully loaded 'facedetect_postprocess_tlt' version 1
E0607 08:31:01.347254 1 model_repository_manager.cc:986] failed to load 'hcgesture_tlt' version 1: Invalid argument: instance group hcgesture_tlt of model hcgesture_tlt must be KIND_GPU and must specify at least one GPU id
E0607 08:31:02.232410 1 model_repository_manager.cc:986] failed to load 'facedetect_tlt' version 1: Invalid argument: instance group facedetect_tlt of model facedetect_tlt must be KIND_GPU and must specify at least one GPU id
E0607 08:31:02.368834 1 model_repository_manager.cc:986] failed to load 'bodypose_320x224_tlt' version 1: Invalid argument: instance group bodypose_320x224_tlt of model bodypose_320x224_tlt must be KIND_GPU and must specify at least one GPU id
E0607 08:31:02.369099 1 model_repository_manager.cc:1183] Invalid argument: ensemble 'bodypose_320x224_ensemble_tlt' depends on 'bodypose_320x224_tlt' which has no loaded version
E0607 08:31:02.369187 1 model_repository_manager.cc:1183] Invalid argument: ensemble 'facedetect_ensemble_tlt' depends on 'facedetect_tlt' which has no loaded version
I0607 08:31:02.370461 1 server.cc:141]
+---------+--------+------+
| Backend | Config | Path |
+---------+--------+------+
+---------+--------+------+
I0607 08:31:02.371147 1 server.cc:184]
+----------------------------------+---------+--------------------------------------------------------------------------------------+
| Model | Version | Status |
+----------------------------------+---------+--------------------------------------------------------------------------------------+
| bodypose_320x224_ensemble_tlt | - | Not loaded: No model version was found |
| bodypose_320x224_postprocess_tlt | 1 | READY |
| bodypose_320x224_tlt | 1 | UNAVAILABLE: Invalid argument: instance group bodypose_320x224_tlt of model bodypose |
| | | _320x224_tlt must be KIND_GPU and must specify at least one GPU id |
| facedetect_ensemble_tlt | - | Not loaded: No model version was found |
| facedetect_postprocess_tlt | 1 | READY |
| facedetect_tlt | 1 | UNAVAILABLE: Invalid argument: instance group facedetect_tlt of model facedetect_tlt |
| | | must be KIND_GPU and must specify at least one GPU id |
| faciallandmarks_tlt | 1 | UNAVAILABLE: Invalid argument: instance group faciallandmarks_tlt of model faciallan |
| | | dmarks_tlt must be KIND_GPU and must specify at least one GPU id |
| gaze_facegrid_tlt | 1 | UNAVAILABLE: Invalid argument: instance group gaze_facegrid_tlt of model gaze_facegr |
| | | id_tlt must be KIND_GPU and must specify at least one GPU id |
| hcgesture_tlt | 1 | UNAVAILABLE: Invalid argument: instance group hcgesture_tlt of model hcgesture_tlt m |
| | | ust be KIND_GPU and must specify at least one GPU id |
| heartrate_two_branch_tlt | 1 | UNAVAILABLE: Invalid argument: instance group heartrate_two_branch_tlt of model hear |
| | | trate_two_branch_tlt must be KIND_GPU and must specify at least one GPU id |
+----------------------------------+---------+--------------------------------------------------------------------------------------+
I0607 08:31:02.372181 1 tritonserver.cc:1620]
+----------------------------------+------------------------------------------------------------------------------------------------+
| Option | Value |
+----------------------------------+------------------------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.5.0 |
| server_extensions | classification sequence model_repository schedule_policy model_configuration system_shared_mem |
| | ory cuda_shared_memory binary_tensor_data statistics |
| model_repository_path[0] | /model_repository/repository |
| model_control_mode | MODE_NONE |
| strict_model_config | 1 |
| pinned_memory_pool_byte_size | 268435456 |
| min_supported_compute_capability | 6.0 |
| strict_readiness | 1 |
| exit_timeout | 30 |
+----------------------------------+------------------------------------------------------------------------------------------------+
I0607 08:31:02.372389 1 server.cc:280] Waiting for in-flight requests to complete.
I0607 08:31:02.372472 1 model_repository_manager.cc:837] unloading: bodypose_320x224_postprocess_tlt:1
I0607 08:31:02.372574 1 model_repository_manager.cc:837] unloading: facedetect_postprocess_tlt:1
I0607 08:31:02.372643 1 server.cc:295] Timeout 30: Found 2 live models and 0 in-flight non-inference requests
I0607 08:31:02.373438 1 model_repository_manager.cc:966] successfully unloaded 'bodypose_320x224_postprocess_tlt' version 1
I0607 08:31:02.373840 1 model_repository_manager.cc:966] successfully unloaded 'facedetect_postprocess_tlt' version 1
I0607 08:31:03.373064 1 server.cc:295] Timeout 29: Found 0 live models and 0 in-flight non-inference requests
error: creating server: Internal - failed to load all models
E0607 08:31:03.374727 1 cuda_memory_manager.cc:65] Failed to finalize CUDA memory manager: [3] CNMEM_STATUS_NOT_INITIALIZED