I just need some confirmation that I understand the doc correctly.
For FP32/FP16/INT8 deployment, I have 2 options: using
.engineis hardware specific so it need to be generated on the deployment hardware (i.e., a Jetson), this
.enginecan either be generated using
tlt-converter(for Jetson) or Deepstream will generate a
.enginefile on the fly using the provided
.etltfile. My question is: is there any difference in the model’s performance (speed & accuracy( between using
tlt-convertervs. letting Deepstream generate a
.enginefile on the fly?
For INT8 deployment, regarding the calibration cache file. What is its purpose? What is the general advice on creating this file (e.g., the number of images to use, using train or val set or both)?
I’m aware that the number of images to be used for calibration need the be at least
batch_size * batches, what happens if the number of images exceed that value? How should I set
batchesto get optimal performance from the model?
max_batch_sizewhen create the
tlt-converter, in any Deepstream model config file, there is a
batch-size(=number of source elements in the pipeline) parameter, what is the relationship between these two parameters? My understanding is
max_batch_size, is that correct?
input_dimensionsneed to match the inferred
tlt-export. When deploy with Deepstream, in the model config file, there is a parameter called
uff-input-dims, this value need to match the
input_dimensions, is that correct?