I am trying to do inference on an onnx segmentation model which takes one images as input and outputs the detection boxes (output0) and the segmentation mask (output1).
model:
Inputs:
name: images
type: float32[1,3,640,640]
Outputs:
name: output0
type: float32[1,70,8400]
name: output1
type: float32[1,32,160,160]
It actually works fine when using my own inference operator written with cupy. So the onnx model and the converted trt model should be correct. When using the holoscan MultiAIInferenceOP however I get an error:
YAML:
inference: # MultiAIInference
backend: "trt"
pre_processor_map:
"yolo_segment": ["in_frame"]
inference_map:
"yolo_segment": "out_yolov8_segment"
in_tensor_names: ["in_frame"]
out_tensor_names: ["out_yolov8_segment"]
enable_fp16: false
input_on_cuda: true
output_on_cuda: true
transmit_on_cuda: true
error message:
[2023-05-21 09:54:10.273] [holoscan] [info] [utils.hpp:45] 3: [executionContext.cpp::enqueueInternal::330] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::enqueueInternal::330, condition: bindings[x] != nullptr
)
[2023-05-21 09:54:10.273] [holoscan] [error] [holoinfer_constants.hpp:71] TRT inference core: Inference failure.
[2023-05-21 09:54:10.273] [holoscan] [error] [holoinfer_constants.hpp:71] Inference manager, Inference failed in core for yolo_segment
[2023-05-21 09:54:10.274] [holoscan] [error] [gxf_wrapper.cpp:68] Exception occurred for operator: 'MultiAIInferenceOp' - Error in Multi AI Inference Codelet, Sub-module->Tick, Inference execution, Message->Error in Multi AI Inference Codelet, Sub-module->Tick, Inference execution, Inference manager, Inference failed in execution for yolo_segment
2023-05-21 09:54:10.274 ERROR gxf/std/entity_executor.cpp@525: Failed to tick codelet MultiAIInferenceOp in entity: MultiAIInferenceOp code: GXF_FAILURE
2023-05-21 09:54:10.274 ERROR gxf/std/entity_executor.cpp@556: Entity [MultiAIInferenceOp] must be in Lifecycle::kStarted or Lifecycle::kIdle stage before stopping. Current state is Ticking
2023-05-21 09:54:10.274 WARN gxf/std/greedy_scheduler.cpp@235: Error while executing entity 20 named 'MultiAIInferenceOp': GXF_FAILURE
2023-05-21 09:54:10.274 ERROR gxf/std/entity_executor.cpp@556: Entity [MultiAIInferenceOp] must be in Lifecycle::kStarted or Lifecycle::kIdle stage before stopping. Current state is Ticking
2023-05-21 09:54:10.274 INFO gxf/std/greedy_scheduler.cpp@367: Scheduler finished.
2023-05-21 09:54:10.274 ERROR gxf/std/program.cpp@497: wait failed. Deactivating...
2023-05-21 09:54:10.275 ERROR gxf/core/runtime.cpp@1251: Graph wait failed with error: GXF_FAILURE
2023-05-21 09:54:10.275 PANIC /workspace/holoscan-sdk/src/core/executors/gxf/gxf_executor.cpp@296: GXF operation failed: GXF_FAILURE
Is the config for the MultiAIInferenceOP correct? I did not find out how to declare two outputs for a single model and assign them by their real name (output0, output1).
Setup:
NVIDIA Clara AGX Developer Kit with Holoscan SDK v0.5.0.