Image Caption with Deepstream

[Fiona.Chen]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

@lowjethro

Can I just confirm that with “network-type=100”, there will not be post-processing involved?

No. There will not be any default internal post-processing with "network-type=100”.

@lowjethro

There is actually a 2 stage process. The first stage is the visual processor. This does some basic functions like scaling and normalising of the input image. The output is a tensor of shape [1, 3, 224, 224].

Such preprocessing can be converred by gst-nvinfer internal pre-processing. DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums

@lowjethro

It will take in the output tensor from the visual processor and output a tensor of shape [1, 257, 1408]. The image encoder is a deep learning model and model inference occurs at this stage.

This can be done by gst-nvinfer too.

@lowjethro

Can I check, the best way to structure the pipeline for this case would be to have a Gst-nvdspreprocess (visual processor) → Gst-nvinfer (image encoder)?

No. As your description. The gst-nvinfer + customized post-processing is enough. No gst-nvdspreprocess is needed.
Please refer to the sample /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-infer-tensor-meta-test