TensorRT version: tensorrt 3.0.2
GPU: GeForceGTX1080Ti
//--------------------------using int8 to parse---------------------------
starting parser->parse
parsing: kint8: 2, kfp16: 1, kfp32: 0, current mModelDataType: 2
ending parser->parse
[GIE] building CUDA engine
[GIE] completed building CUDA engine
gieModelStream type: 2
//---------------------------using fp32 to parse------------------------------
parsing: kint8: 2, kfp16: 1, kfp32: 0, current mModelDataType: 0
ending parser->parse
[GIE] building CUDA engine
[GIE] completed building CUDA engine
gieModelStream type: 2 //here!, why generated is 2(kint8)
Hi,
Could you share which sample do you use?
The deepstreamSDK, TensorRT sample or customized implementation?
Thanks.
- just using the sample_mnist can reproduce the error:
int main(int argc, char** argv)
{
// create a GIE model from the caffe model and serialize it to a stream
IHostMemory *gieModelStream{nullptr};
caffeToGIEModel("mnist.prototxt", "mnist.caffemodel", std::vector < std::string > { OUTPUT_BLOB_NAME }, 1, gieModelStream);
std::cout <<"gieModelStream: "<< int(gieModelStream->type()) <<std::endl; //here! add this line
}
output::
xz@maxieye:~/work/TensorRT-3.0.2$ ./bin/sample_mnist
gieModelStream: 2
2.using sample_int8 can also reproduce the error:
add the following lines to the function scoreModel()
if(datatype == nvinfer1::DataType::kFLOAT){
std::cout <<"gieModelStream: kFLOAT: "<< int(gieModelStream->type()) <<std::endl;
}
if(datatype == nvinfer1::DataType::kHALF){
std::cout <<"gieModelStream: kHALF: "<< int(gieModelStream->type()) <<std::endl;
}
if(datatype == nvinfer1::DataType::kINT8){
std::cout <<"gieModelStream: kINT8: "<< int(gieModelStream->type()) <<std::endl;
}
output::
xz@maxieye:~/work/TensorRT-3.0.2$ ./bin/sample_int8 mnist
FP32 run:400 batches of size 100 starting at 100
gieModelStream: kFLOAT: 2
…
Top1: 0.9904, Top5: 1
Processing 40000 images averaged 0.00170958 ms/image and 0.170958 ms/batch.
FP16 run:400 batches of size 100 starting at 100
Engine could not be created at this precision
INT8 run:400 batches of size 100 starting at 100
gieModelStream: kINT8: 2
…
Top1: 0.9908, Top5: 1
Processing 40000 images averaged 0.00129897 ms/image and 0.129897 ms/batch.
Thanks.
We have feedback this issue to our internal team.
Will update information with you later.
Hi,
Sorry for the late reply.
From our documentation for IHostMemory:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/c_api/_nv_infer_8h_source.html
<i>532 virtual DataType type() const = 0; </i>
The memory type is allocated as an array of bytes(8bit values).
Thanks