Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
GPU- RTX A5000
• DeepStream Version
6.1 via nvcr.io/nvidia/deepstream:6.1-triton
• JetPack Version (valid for Jetson only)
• TensorRT Version
version in nvcr.io/nvidia/deepstream:6.1-triton
• NVIDIA GPU Driver Version (valid for GPU only)
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.73.05 Driver Version: 510.73.05 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA RTX A5000 Off | 00000000:0A:00.0 On | Off |
| 30% 41C P8 21W / 230W | 2117MiB / 24564MiB | 8% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
• Issue Type( questions, new requirements, bugs)
Question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
I am trying to run the pytorch model Keypoint R-CNN from here: keypointrcnn_resnet50_fpn — Torchvision 0.12 documentation
I have tried and failed to use this with trtexec after exporting ONNX model: Link
I have also tried and failed to export this using torch2trt: Link
I was then recommended to try running using Deepstream with the Triton backend.
I am now attempting this but I have come across the first stumbling block, which is generating the config.pbtxt file.
The model is described as:
During inference, the model requires only the input tensors, and returns the post-processed predictions as a
List[Dict[Tensor]], one for each input image. The fields of theDictare as follows, whereNis the number of detected instances:
- boxes (
FloatTensor[N, 4]): the predicted boxes in[x1, y1, x2, y2]format, with0 <= x1 < x2 <= Wand0 <= y1 < y2 <= H.- labels (
Int64Tensor[N]): the predicted labels for each instance- scores (
Tensor[N]): the scores or each instance- keypoints (
FloatTensor[N, K, 3]): the locations of the predicted keypoints, in[x, y, v]format.
I can run the following:
import torch
import torchvision
# Instance of the model
model = torchvision.models.detection.keypointrcnn_resnet50_fpn(pretrained=True)
# Switch the model to eval model
model.eval()
# An example input you would normally provide to your model's forward() method.
example = torch.rand(2, 3, 800, 800)
result = model(example)
print("--------------------")
print(result)
print("--------------------")
and get the output:
--------------------
[{'boxes': tensor([], size=(0, 4), grad_fn=<StackBackward0>), 'labels': tensor([], dtype=torch.int64), 'scores': tensor([], grad_fn=<IndexBackward0>), 'keypoints': tensor([], size=(0, 17, 3)), 'keypoints_scores': tensor([], size=(0, 17))}, {'boxes': tensor([], size=(0, 4), grad_fn=<StackBackward0>), 'labels': tensor([], dtype=torch.int64), 'scores': tensor([], grad_fn=<IndexBackward0>), 'keypoints': tensor([], size=(0, 17, 3)), 'keypoints_scores': tensor([], size=(0, 17))}]
--------------------
However I do not know how to translate this to a config.pbtxt file.
Trying to follow the docs I have come up with the following:
platform: "pytorch_libtorch"
max_batch_size: 2
input {
name: "INPUT__0"
data_type: TYPE_FP32
format: FORMAT_NCHW
dims: [ 3, 800, 800 ]
}
output [
{
name: "boxes"
data_type: TYPE_FP32
dims: 4
},
{
name: "labels"
data_type: TYPE_INT64
dims: [ -1 ]
label_filename: "resnet50_labels.txt"
},
{
name: "scores"
data_type: TYPE_FP32
dims: [ -1 ]
},
{
name: "keypoints"
data_type: TYPE_FP32
dims: [ 17 ]
dims: [ 3 ]
},
{
name: "keypoints_scores"
data_type: TYPE_FP32
dims: [ 17 ]
}
]
However I am not sure it is correct. In the python output there are outputs without sizes like the labels and I have put -1 here which I’m not sure if it’s correct. Same for scores. I also found the label_filename in another example but I’m not sure if that is correct either.
My questions:
- Is it possible for someone to check my pbtxt and give feedback
- Are there any good pytorch examples for what I am trying to do?
Thanks!