- GPU Tesla T4
- DeepStream 5.0.1 container
- CUDA 10.2
- TensorRT 7.0.0 and 7.1.3
- Driver 450.51.06
Image resizing method in pipeline yields different result with opencv/PIL, making it difficult to align offline train/eval and online deployment. Is it possible to reveal some detail of resizing of DeepStream implementation?
For some time we had deployed a detector engine with deepstream pipeline. When dealing with the (mis)detection results, it is noticed that different image resize implementation could introduce noticeable classification error in our case. And we couldn’t get the detail of image resizing in
interpolation-methodoptions, we tried
bilinearas others are seldom used in production.
nvstreammuxhas no such options, but it seems like nearest (which is dGPU default)
Some experiment suggests the resizing operation produces different result with other common libs, e.g. opencv or PIL, which are used in offline train/eval. We do know the fact that opencv and PIL themselves don’t align with each other. But without further knowledge we could not choose or implement proper resizing library to unify the preprocessing in all environments.
By digging into
nvinfer we found the image buffer transformation are done by
NvBufSurfTransform with default interpolation method (nearest). It also seems the
interpolation-method directly maps to
NvBufSurfTransformInter_XXX options. However the color conversion (NV12 to BGRA in our case) and resizing are done in the same function call to
NvBufSurfTransform. Are they implemented separately or separable?
libnvbufsurftransform.so depends on
libnppig.so which provides resizing functions, but not
libnppicc.so which provides
NV12ToBGR conversions etc. Is the resizing done by NPP? Is there corresponding color conversion function in NPP?
Specifically we are interested in aligning the following options:
And if possible, may
interpolation-method option added in future, since it could perform image resize/resample when batching.