failed to parser mobilenet v2 caffe model on Jetson Nano

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

I am trying to run mobilenet v2 model on Nano, but failed in kHALF mode. Error message just as below, it seems that weight value out of range [65504, -65504], how to solve it? please give me some advice.

bob@bob-nano:~/samba/source/MobileNet-v2-TensorRT/build/bin$ ./mobileNet
attempting to open cache file …/…/model/mobilenet_v2.caffemodel.1.tensorcache
cache file not found, profiling network model
…/…/model/mobilenet_v2.prototxt
…/…/model/mobilenet_v2.caffemodel
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 98641.7 is outside of [65504, -65504].
Error: Weight 68991.2 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
Error: Weight 99977.8 is outside of [65504, -65504].
mobileNet: /home/bob/samba/source/MobileNet-v2-TensorRT/tensorNet.cpp:107: bool TensorNet::caffeToTRTModel(const char*, const char*, const std::vector<std::__cxx11::basic_string >&, unsigned int, std::ostream&): Assertion `blobNameToTensor != nullptr’ failed.

2

Hi,

Sorry for the late reply.

May I know have you converted the model from .uff into TensorRT engine with the FP16 mode enabled?
It should look like this:

nvinfer1::IBuilder* builder = CREATE_INFER_BUILDER(logger);
builder->setFp16Mode(true);
nvinfer1::ICudaEngine* engine = builder->buildCudaEngine(*network);

Thanks.

3

Dear AastaLLL:
Thanks for you reply.
I have enabled FP16 mode as shown in below code, it seems that you use “setFp16Mode()” to enable FP16 mode. I think I am doing the same thing.

if(useFp16)
{
    builder->setHalf2Mode(true);
    printf("support FP16 mode\n");
}
else
    printf("unsupport FP16 mode\n");

If I use FP32 Mode, no range error happens. Everything works fine.

4

Hi,

Would you mind to check if this issue is model related.
Please run the same code with a model included in the TensoRT data foldoer: /usr/src/tensorrt/data/.

Thanks.

5

Dear AastaLLL:
Sorry for late reply.
Yes，it is model related.
Since the model is trained under FP32 model, how to ensure the weight within [65504, -65504].
Thus I assume that TensorRT will transform the weight within the range.

6

If you are able to re-train the model, maybe you could set a slightly higher “weight decay” value. That would encourage the model to converge to a local minimum with smaller weights.