Convert Input Tensor to InferenceOP in Holoscan

Healthcare & Life Sciences Holoscan SDK
,
1

Hey, I am integrating my model to make a Holoscan sdk based application for real-time inference. But I am having one issue which I am not sure how to solve efficiently. Currently my model inputs images/frames with shape: [BatchSize, Channels, Height, Width] but the InferenceOp Operator expects the input to be of the shape: [Batch_Size, Height, Width, Channel]. Is there a zero copy way to covert the output tensor from FormatconverterOp to my required shape before passing it into InferenceOP operator.

2

Hi Dan

Thanks for your question and interest in holoscan SDK .
Unfortunately the FormatconverterOp is not able to do this at this time and it does a copy.
May be you can try this code below that would to the conversion for you. It does use cupy to run on the gpu but I am not sure if all operations avoids copying.

Hope that helps

class My_TransposeOp(Operator):
    #Custom Holoscan operator to transpose a tensor.

    def __init__(self, fragment, *args, perm=(1, 0, 2), keyname_in_message="out_tensor", **kwargs):
        self.perm = perm  # The desired output dimension order
        self.keyname_in_message = keyname_in_message
        super().__init__(fragment, *args, **kwargs)

    def setup(self, spec: OperatorSpec):
        spec.input("in_tensor")
        spec.output("out_tensor")

    def compute(self, op_input, op_output, context):
        message = op_input.receive("in_tensor")
        keyname = self.keyname_in_message
        if isinstance(message, dict):
            in_holoscan_tensor = message[keyname]
        else:
            in_holoscan_tensor = message
        in_cupy_array = cp.asarray(in_holoscan_tensor)
        out_cupy_array = cp.transpose(in_cupy_array, self.perm)
        # Ensure row-major (C-contiguous) memory layout for Holoviz
        out_cupy_array = cp.ascontiguousarray(out_cupy_array)
        op_output.emit({keyname: out_cupy_array}, "out_tensor")