Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
NVIDIA RTX A4500
• DeepStream Version
7.1
• JetPack Version (valid for Jetson only)
• TensorRT Version
default tensorrt in nvcr.io/nvidia/deepstream:7.1-triton-multiarch
• NVIDIA GPU Driver Version (valid for GPU only)
Driver Version: 535.161.08
CUDA Version: 12.2
• Issue Type( questions, new requirements, bugs)
bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
Hello
When using the test1_app in pipeline_api, if the model is inferred with nvinfer and output-tensor-meta is set to true, the Probe method can be used normally to clone the tensor or copy it to the CPU.
- type: nvinfer
name: infer
properties:
config-file-path: /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-test1/dstest1_pgie_config.yml
output-tensor-meta: true
Probe code:
class Outputtensor(BatchMetadataOperator):
def handle_metadata(self, batch_meta):
for frame_meta in batch_meta.frame_items:
for user_meta in frame_meta.tensor_items:
for n, tensor in user_meta.as_tensor_output().get_layers().items():
print(f"tensor name: {n}“)
print(f"tensor object: {tensor}”)
# operations on tensors:
# torch_tensor = torch.utils.dlpack.from_dlpack(tensor).to(‘cpu’)
torch_tensor = torch.utils.dlpack.from_dlpack(tensor.clone())
print(torch_tensor)
pipeline:
Pipeline(PIPELINE_NAME, file_path).attach(“infer1”, Probe(“output”, Outputtensor())).start().wait()
However, when using the nvinferserver approach and setting output_control {output_tensor_meta: true} in the configuration file, an error occurs when using the Probe method to clone the tensor.
dstest1_config.yaml:
- type: nvinferserver
name: infer
properties:
config-file-path: /workspace/DeepStream/yolo_triton_dp/config_infer_triton_yolov7.txt
config_infer_triton_yolov7.txt :
infer_config {
unique_id: 1
gpu_ids: [0]
max_batch_size: 1
backend {
triton {
model_name: “yolov7”
version: -1
model_repo {
root: “/workspace/DeepStream/yolo_triton_dp/models”
}
}
}
preprocess {
network_format: IMAGE_FORMAT_RGB
tensor_order: TENSOR_ORDER_LINEAR
maintain_aspect_ratio: 1
frame_scaling_hw: FRAME_SCALING_HW_DEFAULT
frame_scaling_filter: 1
normalize {
scale_factor: 0.0039215697906911373
}
}
postprocess {
labelfile_path: “/workspace/DeepStream/yolo_triton_dp/classes.txt”
detection {
num_detected_classes: 80
custom_parse_bbox_func: “NvDsInferParseCustomEfficientNMS”
nms {
confidence_threshold: 0.5
topk: 300
iou_threshold: 0.45
}
}
}
extra {
copy_input_to_host_buffers: false
}
custom_lib {
path : “/workspace/DeepStream/yolo_triton_dp/lib/libnvds_infercustomparser.so”
}
}
input_control {
process_mode : PROCESS_MODE_FULL_FRAME
interval : 0
}
output_control {
output_tensor_meta: true
}
error info：
tensor name: num_dets
tensor object: <pyservicemaker._pydeepstream.Tensor object at 0x7f1fc4246d70>
CUDA memcpy failed
Segmentation fault (core dumped)
OR:
{‘822’: <pyservicemaker._pydeepstream.Tensor object at 0x7f16386e41f0>, ‘823’: <pyservicemaker._pydeepstream.Tensor object at 0x7f16386e4270>, ‘output0’: <pyservicemaker._pydeepstream.Tensor object at 0x7f16386e42f0>}
<pyservicemaker._pydeepstream.Tensor object at 0x7f16386e41f0> <pyservicemaker._pydeepstream.Tensor object at 0x7f16386e4270>
ERROR: QueueThread:GstInferServImpl internal unexpected error, may cause stop
The two aforementioned error messages occurred in different scenarios: the first appeared in a pipeline I built by writing my own configuration file, while the second occurred in a pipeline built using the configuration file from pipeline_apps’s test1_app. However, both errors seem to point to torch_tensor = torch.utils.dlpack.from_dlpack(tensor.clone()) or torch_tensor = torch.utils.dlpack.from_dlpack(tensor).to('cpu').
Could there be an issue with my configuration file? Or am I using the method incorrectly? I would greatly appreciate your help in addressing these questions. Thank you very much!
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)