Hi in the hekp you say there are 2 options to calibrate:
Blockquote
Option 1: Using the training data loader to load the training images for INT8 calibration. This option is now the recommended approach to support multiple image directories by leveraging the training dataset loader. This also ensures two important aspects of data during calibration:
Data pre-processing in the INT8 calibration step is the same as in the training process.
The data batches are sampled randomly across the entire training dataset, thereby improving the accuracy of the INT8 model.
Option 2: Pointing the tool to a directory of images that you want to use to calibrate the model. For this option, make sure to create a sub-sampled directory of random images that best represent your training dataset.
I am interested in option 1. Thus my questions are:
1)how do I use the training data loader to load the training images for INT8 calibration?
Is it just changing the path in the -cal_image_dir from one to another?
2) are the arguments of batch size and batches also relevant in the Option 1:?
3)I tried to use also Option2 and pointed with the -cal_image_dir into some directory where there are 78 images and --batch_size 1 \ and --batches 10
And when tried to run this I have got some error message
“not enough images provided 78 <160”
Where did the160 come from and what does this message mean?
Inside the the specs file (retrain specs file) I have a links to docker folders for trainig and validation, and those are enough for succesful exporting in int8, and NO specific changes or additions are needed. Right?
what are numbers 585->271 mean?
same regarding the 10/10 ? (note: we have 8 datasets used in training)
Hey
Thanks for the answer, but though you say that we did everything fine in op1(without pointing to the library with the calibration images)
we get very strange results comparing to the fp16:
please see the attached image:
*I apologize for the images are black since the images are our client’s property
For now let us see the op 1. We have a big training dataset with 8 directories that include more than 2000 images. And to say the truth I have no I idea where does it get the batches =10
Please see the attached specs file we use for retrain and also we point ti while exporting in int8 (as you could see in a command in my previous reply)
