Please provide complete information as applicable to your setup.
**• Hardware Platform (Jetson / GPU)**RTX4080
• DeepStream Version7.0
My code is based on deepstream-pose-classification.
I just take peoplenet detection and remove all segs and preprocessing.
The intention is to add my own seg.
Currently I am testing peoplenet.
nvcr.io/nvidia/deepstream:7.0-triton-multiarch is used in my development.
I have two issues.
(1)First issue is, I can run peoplenet using nvinfer.
But I can’t display?
xhost + is used before docker run. But still doesn’t display. sink-type: 3 in main config file.
Why I can’t display using nvinfer in docker?
(2)I can not run peoplenet using triton server.
Main config file.
source-list:
list: file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4;
streammux:
width: 1920
height: 1080
batched-push-timeout: 40000
tracker:
enable: 1
ll-lib-file: /workspace/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
ll-config-file: /workspace/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvDCF_accuracy.yml
primary-gie:
plugin-type: 1
#config-file-path: ../nvinfer/peoplenet_tao/config_infer_primary_peoplenet.txt
config-file-path: ../triton/peoplenet_tao/config_infer_primary_peoplenet.yml
#config-file-path: ../triton-grpc/peoplenet_tao/config_infer_primary_peoplenet.yml
sink:
sink-type: 3
enc-type: 0
config_infer_primary_peoplenet.yml
infer_config {
unique_id: 1
gpu_ids: [0]
max_batch_size: 24
backend {
inputs: [ {
name: "input_1:0"
}]
outputs: [
{name: "output_bbox/BiasAdd:0"},
{name: "output_cov/Sigmoid:0"}
]
triton {
model_name: "peoplenet"
version: -1
model_repo {
root: "../../../models"
strict_model_config: true
}
}
}
preprocess {
network_format: MEDIA_FORMAT_NONE
tensor_order: TENSOR_ORDER_LINEAR
tensor_name: "input_1:0"
maintain_aspect_ratio: 0
frame_scaling_hw: FRAME_SCALING_HW_DEFAULT
frame_scaling_filter: 1
normalize {
scale_factor: 0.0039215697906911373
channel_offsets: [0, 0, 0]
}
}
postprocess {
labelfile_path: "../../../models/peoplenet/labels.txt"
detection {
num_detected_classes: 1
per_class_params {
key: 0
value { pre_threshold: 0.4 }
}
nms {
confidence_threshold:0.2
topk:20
iou_threshold:0.5
}
}
}
extra {
copy_input_to_host_buffers: false
output_buffer_pool_size: 2
}
}
input_control {
process_mode: PROCESS_MODE_FULL_FRAME
operate_on_gie_id: -1
interval: 0
}
The engine was converted using the following command.
trtexec --onnx=./models/peoplenet/resnet34_peoplenet_int8.onnx --int8 \
--calib=./models/peoplenet/resnet34_peoplenet_int8.txt --saveEngine=./models/peoplenet/1/resnet34_peoplenet_int8.onnx_b1_gpu0_int8.engine \
--minShapes="input_1:0":1x3x544x960 --optShapes="input_1:0":12x3x544x960 --maxShapes="input_1:0":24x3x544x960
The errors in running in triton are
root@34a59dc5f368:/workspace/opt/nvidia/deepstream/deepstream-7.0/sources/deepstream_tao_apps/apps/tao_others/deepstream-ava# ./deepstream-pose-classification-app ../../../configs/app/deepstream_ava.yaml
width 1920 hight 1080
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
video file:///workspace/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4
WARNING: Overriding infer-config batch-size (0) with number of sources (12)
sink_type:3, enc_type:0
Now playing!
gstnvtracker: Loading low-level lib at /workspace/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
[NvMultiObjectTracker] Loading TRT Engine for tracker ReID...
[NvMultiObjectTracker] Loading Complete!
ERROR: [TRT]: 3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[NvMultiObjectTracker] Initialized
0:00:00.255154213 856 0x5cb99b82e600 WARN nvinferserver gstnvinferserver_impl.cpp:360:validatePluginConfig:<primary-nvinference-engine> warning: Configuration file batch-size reset to: 12
WARNING: infer_proto_utils.cpp:145 auto-update preprocess.network_format to IMAGE_FORMAT_RGB
INFO: infer_trtis_backend.cpp:218 TrtISBackend id:1 initialized model: peoplenet
ERROR: infer_trtis_server.cpp:1146 Triton: Triton inferAsync API call failed, triton_err_str:Invalid argument, err_msg:[request id: 0] inference request batch-size must be <= 1 for 'peoplenet'
ERROR: infer_trtis_backend.cpp:594 TRT-IS async inference failed., nvinfer error:NVDSINFER_TRITON_ERROR
ERROR: infer_trtis_backend.cpp:363 failed to specify dims when running inference on model:peoplenet, nvinfer error:NVDSINFER_TRITON_ERROR
0:00:00.416438238 856 0x5cb99b82e600 ERROR nvinferserver gstnvinferserver.cpp:405:gst_nvinfer_server_logger:<primary-nvinference-engine> nvinferserver[UID 1]: Error in specifyBackendDims() <infer_trtis_context.cpp:205> [UID = 1]: failed to specify input dims triton backend for model:peoplenet, nvinfer error:NVDSINFER_TRITON_ERROR
0:00:00.416446185 856 0x5cb99b82e600 ERROR nvinferserver gstnvinferserver.cpp:405:gst_nvinfer_server_logger:<primary-nvinference-engine> nvinferserver[UID 1]: Error in createNNBackend() <infer_trtis_context.cpp:294> [UID = 1]: failed to specify triton backend input dims for model:peoplenet, nvinfer error:NVDSINFER_TRITON_ERROR
ERROR: infer_trtis_server.cpp:885 Triton: failed to stop repo server, triton_err_str:Internal, err_msg:Exit timeout expired. Exiting immediately.
0:00:30.442080669 856 0x5cb99b82e600 ERROR nvinferserver gstnvinferserver.cpp:405:gst_nvinfer_server_logger:<primary-nvinference-engine> nvinferserver[UID 1]: Error in initialize() <infer_base_context.cpp:80> [UID = 1]: create nn-backend failed, check config file settings, nvinfer error:NVDSINFER_TRITON_ERROR
0:00:30.442093787 856 0x5cb99b82e600 WARN nvinferserver gstnvinferserver_impl.cpp:592:start:<primary-nvinference-engine> error: Failed to initialize InferTrtIsContext
0:00:30.442095688 856 0x5cb99b82e600 WARN nvinferserver gstnvinferserver_impl.cpp:592:start:<primary-nvinference-engine> error: Config file path: /workspace/opt/nvidia/deepstream/deepstream-7.0/sources/deepstream_tao_apps/configs/triton/peoplenet_tao/config_infer_primary_peoplenet.yml
0:00:30.442128447 856 0x5cb99b82e600 WARN nvinferserver gstnvinferserver.cpp:515:gst_nvinfer_server_start:<primary-nvinference-engine> error: gstnvinferserver_impl start failed
[NvMultiObjectTracker] De-initialized
Running...
ERROR from element primary-nvinference-engine: Failed to initialize InferTrtIsContext
Error details: gstnvinferserver_impl.cpp(592): start (): /GstPipeline:deepstream_pose_classfication_app/GstNvInferServer:primary-nvinference-engine:
Config file path: /workspace/opt/nvidia/deepstream/deepstream-7.0/sources/deepstream_tao_apps/configs/triton/peoplenet_tao/config_infer_primary_peoplenet.yml
Returned, stopping playback
Deleting pipeline