Is it possible to feed unsigned char data into TensorRT rather than float32?

[alexey.kostin]

Hi All,

Images are 2D arrays of pixels. In most cases pixel is 0…255 (unsigned char) for grey scale or triplets of 0…255 values for colour images. To feed these into TensorRT as inputs we have to convert them to CHW format which is OK (but would still be nice to have an option to do this on GPU through TensorRT). But also each intensity value needs to be converted to float32 which leads to 4x increase in amount of data sent to GPU plus this needs to be done on CPU.

My question is there an option to feed array of unsigned chars as input to TensorRT without conversion to float? Even better if it would be possible to feed HWC array.

Many thanks
Alexey

[SunilJB]

Hi,

TensorRT only supports FP32, FP16, INT32, and INT8: DataType — NVIDIA TensorRT Standard Python API Documentation 8.6.1 documentation

Please refer to this similar topic:

tensorrt inference error while load onnx model

Unsupported ONNX data type: UINT8(2) [TensorRT] ERROR: Network must have at least one output

Thanks

[alexey.kostin]

Hi Sunil,

Thanks for your response and sorry I was not more specific. Unsigned char in C++ is possible to convert to INT8 in TensorRT (it is only shift by 127 which meant amount of memory for input is same and this could be done quite efficiently). Also shift on origin for input data is easy to compensate in the first leyer of CNN. So if it would be possible to feed INT8 into TensorRT as input that would be big help too.

To rephrase the question. Is it possible to feed INT8 in to TensorRT as input layer rather than float32?

Thanks
Alexey

[SunilJB]

Hi,

Please refer to below links:

Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

Input images with dtype int8

Is there any way to use TensorRT with input images with dtype int8? E.g. send images with RGB values in range [0, 255] and do normalisation on the GPU. All samples I’ve seen use dtype float32. In Tensorflow we can use cast but that layer doesn’t seem to be supported by TensorRT. The benefit of using int8 would be to reduce the size of the data being sent to the GPU/TensorRT.

Thanks