I had trained and converted a ResNet50 model with my own dataset on TLT (image: nvcr.io/nvidia/tlt-streamanalytics:v3.0-dp-py3).
According to the last step in TLT(tlt-converter), it outputs an engine file, here I named it model.plan.
!tlt-converter $USER_EXPERIMENT_DIR/export/resnet50.etlt
-k $KEY
-c $USER_EXPERIMENT_DIR/export/resnet50_int8_cache.bin
-o predictions/Softmax
-d 3,512,512
-i nchw
-m 64 -t int8
-e $USER_EXPERIMENT_DIR/export/model.plan
-b 64
After that, I started my triton server (image: nvcr.io/nvidia/tritonserver:20.11-py3) and triton can successfully recognize my resnet50 model.
.
├── 1
│ └── model.plan
├── config.pbtxt
└── resnet50_labels.txt
Last, I started a triton client (image: nvcr.io/nvidia/tritonserver:20.11-py3-clientsdk) and ran the command
$ image_client -m resnet50 -c 3 -s INCEPTION /workspace/test/LED.jpg
However, it returns a wrong category such as
Image ‘/workspace/test/LED.jpg’:
0.924829 (5) = inductor
0.020683 (3) = resistor
0.013248 (6) = capacitor
In addition, no matter what input image I gave, the output will be the same.
ex: image_client -m resnet50 -c 3 -s INCEPTION /workspace/test/LED.jpg
image_client -m resnet50 -c 3 -s INCEPTION /workspace/test/power.jpg
image_client -m resnet50 -c 3 -s INCEPTION /workspace/test/diode.jpg
The output would all be
0.924829 (5) = inductor
0.020683 (3) = resistor
0.013248 (6) = capacitor
The model works fine when I tested on TLT and the accuracy was good.
I’m really confused about why the model isn’t working perfectly when I deploy onto Triton.