How to do INT8 calibrate FRCNN model?

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,
I am facing some troubles for converting FRCNN model to int8 engine.

config->setFlag(BuilderFlag::kINT8);
builder->setInt8Mode(true);
nvinfer1::Dims dims{4, {8, 3, 600, 1000}, {}};
BatchStream calibrationStream(8,8,dims,"./data/calibraton/frcnn/list_images.txt", dataDirs);
calibrator.reset(new Int8EntropyCalibrator2<BatchStream>(
            calibrationStream, 0, "frcnn", "data"));
config->setInt8Calibrator(calibrator.get());

I am using the above code for setting up the calibrator for INT8.

But I am getting the following logs :

Calibrating with file 0041.ppm
Calibrating with file 0052.ppm
Calibrating with file 0030.ppm
Calibrating with file 0079.ppm
Calibrating with file 0051.ppm
Calibrating with file 0088.ppm
Calibrating with file 0012.ppm
Calibrating with file 0038.ppm
[08/10/2020-13:35:38] [E] [TRT] …/builder/cudnnCalibrator.cpp (724) - Cuda Error in add: 719 (unspecified launch failure)
[08/10/2020-13:35:38] [E] [TRT] …/builder/cudnnCalibrator.cpp (724) - Cuda Error in add: 719 (unspecified launch failure)
[08/10/2020-13:35:38] [E] [TRT] FAILED_ALLOCATION: std::exception
[08/10/2020-13:35:38] [E] [TRT] …/rtSafe/cuda/caskConvolutionRunner.cpp (317) - Cuda Error in allocateContextResources: 719 (unspecified launch failure)
[08/10/2020-13:35:38] [E] [TRT] FAILED_EXECUTION: std::exception
[08/10/2020-13:35:38] [E] [TRT] …/rtSafe/safeRuntime.cpp (32) - Cuda Error in free: 719 (unspecified launch failure)
terminate called after throwing an instance of ‘nvinfer1::CudaError’

I know I am missing something or making a mistake. The error is thrown after the getbatch function of Int8EntropyCalibrator2 has been called.

  1. What does this error mean ?
  2. Also, FRCNN has multiple inputs and outputs. But the calibrator needed only one input blob name. I provided “data” and skipped “im_info”. Is that correct ?
  3. Does calibration process require labels ?

Thanks in advance.

3

Hi,

The error indicates that GPU cannot launch the CUDA kernel.
That’s because you will need a GPU architecture > 7.x device to have the INT8 operation support, which is Xaiver or XaiverNX in Jetson series.

You can find this document for more details:

Thanks.

4

@AastaLLL how come in your blog https://developer.nvidia.com/deepstream-sdk INT8 precision has been used here for Jetson Nano because Nano has only compute of 5.3 per your earlier link ?

tagging @nssreenivasalu

5

Hi,

Sorry that the blog may make you feel confused.

The performance of Nano is measured with FP16 mode.
You can find this in the comment below the profiling table:

  • FP16 inference on Jetson Nano

Thanks.

1 Like
6

Thanks for clarifying and sorry if I did not notice the comment below.

7

@a

When I run it on a GEForce RTX 2070 Super GPU with compute capability=7.5 as per https://developer.nvidia.com/cuda-gpus

I get the following error

Blockquote

image
image1299×290 52.1 KB

This is being run on a docker nvcr.io/nvidia/tensorrt:20.06-py3 with

image

Please advise.

8

Hi,

illegal memory access is usually related to some incorrect parameter or setting.
Would you mind to check this issue first:

Do you add the background class to the class number correctly?

Thanks.