Hi, I want to cast a int32 tensor to float type. How can I do that in TensorRT. I tried the Identity layer but it is not supported.
Hi, Please refer to the below links to perform inference in INT8
https://github.com/NVIDIA/TensorRT/blob/master/samples/opensource/sampleINT8/README.md
Thanks!
Hi, I am not doing INT8 quantization. What I asked is how to convert a INT32 tensor to a float precision tensor. More specifically, I want to feed second output tensor of ITopKLayer (INT32) as the input of IResizeLayer.
Thanks.
Hi @v.hunglx2,
INT32 is for indices.
Could you please let us know is there a specific reason that you want to convert indices to float?
Thank you.
In some case of semantic segmentation, performing TopK (argmax) in full-size segmentation output may slow-down the pipeline. We want to perform TopK in small-size segmentation output and then scale-up the max indices mask to the origin size, all by TensorRT layers
Hi @v.hunglx2,
INT32->kFloat is actually supported operation by nvinfer1::IIdentityLayer
. But looks like document not included it. Unfortunately we couldn’t find sample to share.
https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/classnvinfer1_1_1_i_identity_layer.html
We request you to try and if you face any issues please share logs, issue related scripts for better assistance.
Thank you.
Hi, I tried again with a simple onnx with Cast operator which will be parsed as nvinfer1::IIdentityLayer by trtexec tool.
TensorRT 7.1.3 (Jetpack): The TRT engine built succesfully
TensorRT 6.3.1 (DriveOS 5.2 release): I get some error. Sample onnx model and log are attached below.
So, maybe INT32 and FLOAT conversion is not supported by TensorRT 6?temp.onnx (175 Bytes) TensorRT6.3.1-trtexec.log (6.8 KB)
Hi @v.hunglx2,
With latest TRT version we could successfully generate the engine.
Yes, convert between FP32 and INT is TRT 7.0+ feature. It’s not supported in TRT version 6.
Thank you.