Input images with dtype int8

Is there any way to use TensorRT with input images with dtype int8? E.g. send images with RGB values in range [0, 255] and do normalisation on the GPU. All samples I’ve seen use dtype float32.

In Tensorflow we can use cast but that layer doesn’t seem to be supported by TensorRT. The benefit of using int8 would be to reduce the size of the data being sent to the GPU/TensorRT.


Assuming you are creating a Network definition using the C++ API, you’d add the Input layer to the network, with the input dimensions. via

network->addInput(INPUT_BLOB_NAME, dt, Dims3{1, INPUT_H, INPUT_W});

Please reference addINput API

which contains a DataType parameter.

virtual ITensor* nvinfer1::INetworkDefinition::addInput	(	const char * 	name,
DataType 	type,
Dims 	dimensions 

Is there any way to do this with the Python API and converting Tensorflow models?

please reference

Proposed call returns null with this varning:

        ITensor* data = network->addInput(
                m_InputBlobName.c_str(), nvinfer1::DataType::kINT8,
                DimsCHW{static_cast<int>(m_InputC), static_cast<int>(m_InputH),

[E] [TRT] Parameter check failed at: …/builder/Network.cpp::addInput::466, condition: type != DataType::kINT8

Did you find how to do this ? Converting my ints to floats/half is very computationnaly expensive.