I’m trying to convert pytorch -->onnx -->tensorrt, and it can running successfully. But use the int8 mode, there are some errors as fallows. And I found the erroer is caused by keep = nms(boxes_for_nms, scores, iou_threshold)
[03/01/2023-19:51:41] [TRT] [E] 2: [helpers.h::divUp::70] Error Code 2: Internal Error (Assertion n > 0 failed. )
the environment are:
jetpack 5.1
And the int8 calibrater is:
class MyCalibrator(trt.IInt8EntropyCalibrator2):
def __init__(self, calibrationDataPath, nCalibration, inputShape, cacheFile):
trt.IInt8EntropyCalibrator2.__init__(self)
self.imageList = glob(calibrationDataPath + "*.jpg")[:100]
self.nCalibration = nCalibration
self.shape = inputShape # (N,C,H,W)
self.buffeSize = trt.volume(inputShape) * trt.float32.itemsize
self.cacheFile = cacheFile
_, self.dIn = cudart.cudaMalloc(self.buffeSize)
self.oneBatch = self.batchGenerator()
print(int(self.dIn))
def __del__(self):
cudart.cudaFree(self.dIn)
def batchGenerator(self):
for i in range(self.nCalibration):
print("> calibration %d" % i)
subImageList = np.random.choice(self.imageList, self.shape[0], replace=False)
yield np.ascontiguousarray(self.loadImageList(subImageList))
def loadImageList(self, imageList):
# transform = transforms.Compose([transforms.ToTensor()])
transform = transforms.Compose([transforms.Resize([800, 800]), transforms.ToTensor()])
imgs = []
for i in range(self.shape[0]):
original_img = Image.open(imageList[i]).convert('RGB')
# original_img.show()
trans_img = transform(original_img)
imgs.append(trans_img)
batch_img = torch.stack(imgs, dim=0)
data =batch_img.numpy()
return data
Hi,
Could you check if your model can work with our trtexec binary first?
$ /usr/src/tensorrt/bin/trtexec --onnx=<file> --int8
$ /usr/src/tensorrt/bin/trtexec --loadEngine=<file> --int8
Thanks.
Yes, It can works with trtexec.
I would like to know why I can use trtexec for conversion, but not my own rewritten Python program. Also, if I use trtexec for INT8 calibration, how can I use my own dataset for calibration?
Hi,
Could you check if your model can work with the python sample below?https://elinux.org/Jetson/L4T/TRT_Customized_Example#OpenCV_with_PLAN_model
TensorRT calibration tutorial can be found below:
# Int8 Calibration In TensorRT
## Introduction
Int8 calibration in TensorRT involves providing a representative set of input data
to TensorRT as part of the engine building process. The
[calibration API](https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/Int8/Calibrator.html)
included in TensorRT requires the user to handle copying input data to the GPU and
manage the calibration cache generated by TensorRT.
While the TensorRT API provides a higher degree of control, we can greatly simplify the
process for many common use-cases. For that purpose, Polygraphy provides a calibrator, which
can be used either with Polygraphy or directly with TensorRT. In the latter
case, the Polygraphy calibrator behaves exactly like a normal TensorRT int8 calibrator.
In this example, we'll look at how you can use Polygraphy's calibrator to calibrate a network
with (fake) calibration data, and how you can manage the calibration cache with just a single
parameter.
show original
Thanks.
I’m so sad that my model can’t work with it. It got error:
[E] 2: [helpers.h::divUp::70] Error Code 2: Internal Error (Assertion n > 0 failed. )
I just want to convert MaskRcnn with pytorch to tensorrt with int8, but there are kinds of errors. Could you give me a full example to do it ?
Hi,
Could you share the complete output log with us?trtexec , it’s expected to work with the eLinux sample above.
Thanks.
Hi,
Hi,
The output log from the console.
Thanks.
system
April 26, 2023, 6:26am
15
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.