Issues using appsrc and nvjpegdec

Xavier AGX
Deepstream 6.3
Jetpack 5.1.1

The current pipeline, which works, is

appsrc → nvvideoconvert → nvstreammux → nvinfer …

I’m given a stream of jpeg file locations which inside of the appsrc callback I decode and copy the pixel data into a gstreamer buffer and send it to the pipeline. This works fine but I wanted to experiment with using nvjpegdec to maybe speed up that part a bit.

appsrc → nvjpegdec → nvvideoconvert → nvstreammux → nvinfer

Now inside the appsrc callback I load the bytes into memory and put that in a buffer and send it. Making sure to update the caps of appsrc to “image/jpeg”. The pipeline runs fine but only one image is ever sent. Like, if I process 20 different jpegs all the results are identical to the first jpeg. Which is really weird. I can’t share the code here but wanted to ask if this approach would even work? I dug up one previous discussion about this approach and the conclusion was to do my first approach Nvjpegdec lost buffer nvds meta - #12 by Fiona.Chen but why does it kind of work? That’s the confusing part.

Thank you

yes, nvjpegdec can decode jpg files. please refer to this pipeline.

gst-launch-1.0 multifilesrc  location=/opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-can-orientation-app/data/sample1/"test_sample_%04d.jpg" ! nvjpegdec gpu-id=2 ! mux.sink_0 nvstreammux name=mux ! nvinferserver config-file-path=$MODEL_CONFIG ! fakesink

can you use the following simplified pipeline to check if the output is fine.

appsrc → nvjpegdec  -> nv3dsink

After some more trial and error I’ve found a specific example

appsrc -> jpegparse -> jpegdec -> nvvideoconvert → nvstreammux → nvinfer ....

works as expected. If I send 20 images through I get 20 different results from nvinfer

appsrc -> jpegparse -> nvjpegdec -> nvvideoconvert → nvstreammux → nvinfer ....
and
appsrc -> nvjpegdec -> nvvideoconvert → nvstreammux → nvinfer ....
and
appsrc -> jpegparse -> nvv4l2decoder -> nvvideoconvert → nvstreammux → nvinfer ....

All have the same issue where only the first image is processed and all results after that are just copies of the first.

I will try to get a minimal repro working this weekend.