Two questions about jpeg encode?

When I use jpeg encoding function in DeepStream SDK,I I found the following two problems:
(1)When I use nvjpegenc plugin to do jpeg encoding in pipeline, the simple pipeline process is shown below:
filesrc location=video_decode_0.yuv → videdoparse → nvvideoconvert → nvstreammux → nvvideoconvert → nvjpegenc → filesrc location=www.jpg
If the nvstreammux plugin(batch_size=1) input one yuv image, the nvjpegenc plugin can work successfully and print the encoding time consuming.But If the nvstreammux plugin(batch_size=2) input two yuv image,the nvjpegenc plugin can’t work properly and don‘t have time consuming printing.I want to know if the nvjpegenc plugin supports multiple batch encoding.

(2)When I use nvds_obj_enc_process interface to do batch jpeg encode,I found that there might be a problem with this interface.
My hardware platform is Jetson Xavier NX,my pipeline is similar to the sample_apps(path:/opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-image-meta-test).If the nvstreammux plugin(batch_size=1) input one yuv image, the nvds_obj_enc_process interface can work successfully and save multiple object jpg image.But If the nvstreammux plugin(batch_size=2) input two yuv image,the nvinfer plugin can give correct obj area coordinates,the nvds_obj_enc_process interface can’t save correct object jpg image in second yuv image.The following are the two input images(two_image.jpg) and the corresponding detection area.The object in left image can do jpeg encoding normally(left_obj.zip).But the object in right image can’t get correct jpg image(right_obj.zip).The obj area coordinates are correct ,but the image used for encoding is the one on the left.
This problem appears in many tests. Please see if the nvds_obj_enc_process interface is defective.


left_obj.zip (60.1 KB)
right_obj.zip (44.2 KB)

1 Like

(1)nvjpegenc is not deepstream plugin( please check all deepstream plugins here GStreamer Plugin Overview — DeepStream DeepStream Version: 5.0 documentation), so nvjpegenc can not handle batch data. You need to either add nvmultistreamtiler or nvstreamdemux to extract buffers from batch before you using nvjpegenc.
(2) We will check the nvds_obj_enc_process interface

what is the value of batch-size you set in primary-gie and streammux group?

ex: if batch-size=2 in primary-gie, maybe you must set batch-size=4 in streammux group

it work with my project

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_FAQ.html#what-is-the-difference-between-batch-size-of-nvstreammux-and-nvinfer-what-are-the-recommended-values-for-nvstreammux-batch-size

Hello,Is this reply about question 2?

The link is to answer:

OK.Please check the nvds_obj_enc_process interface as quickly as possible, looking forward to your reply.Thank you.

The original deepstream-image-meta-test sample only support one input. Seems you have modified it to support two inputs. Can you share your code?

I just modified the create_source_bin function in deepstream-image-meta-test sample,and the corresponding code is as follows:
static GstElement *
create_source_bin (guint index, gchar * uri)
{
GstElement bin = NULL;
uint32_t width = 1920;
uint32_t height = 1080;
gchar bin_name[16] = { };
g_snprintf (bin_name, 15, “source-bin-%02d”, index);
/
Create a source GstBin to abstract this bin’s content from the rest of the

  • pipeline */
    bin = gst_bin_new (bin_name);
    GstElement *source,*videoparser,*nvvidconv,*decode;
    GstPad *srcpad = NULL;
bin = gst_bin_new (bin_name);

source = gst_element_factory_make ("filesrc", "source");

videoparser = gst_element_factory_make ("jpegparse", "jpeg-parser");

decode = gst_element_factory_make ("nvv4l2decoder", "decoder0");

if(!videoparser || !decode )
{
    cout << "gst_element_factory_make videoparser or encoder error\n" << endl;
    return NULL;
}

g_object_set (G_OBJECT (source), "location", uri, NULL);
gst_bin_add_many (GST_BIN (bin),source, videoparser, decode ,NULL);
gst_element_link_many (source, videoparser, decode ,NULL);

if (!gst_element_add_pad (bin, gst_ghost_pad_new_no_target ("src",GST_PAD_SRC)))
{
    cout << "Failed to add ghost pad in source bin" << endl;
    return NULL;
}

srcpad = gst_element_get_static_pad (decode, "src");
if (!srcpad)
{
    cout << "Failed to get src pad of source bin. Exiting." << endl;
    return NULL;
}

GstPad *bin_ghost_src_pad = gst_element_get_static_pad (bin, "src");
if (!gst_ghost_pad_set_target (GST_GHOST_PAD (bin_ghost_src_pad),srcpad))
{
    cout << "Failed to link encoder src pad to source bin ghost pad" << endl;
    return NULL;
}
return bin;

}

After modification,you can execute the order ‘./deepstream-image-meta-test image_0.jpg image_1.jpg’.It will set num_sources variable to 2.And the number of entries that the nvstreammux plugin accepts depends on the number of images you enter in the command.

Can you provide your jpg images either?


With your code and images, nvds_obj_enc_process can work with both images when there are two inputs.

0_0_1_Person_162x486

Are these two images input and processed at the same time?Can you share your sample code with me to compare?

0_0_1_Person_186x486

Here are the results of my tests.I guess it may be the difference caused by different versions.You can send me the nvds_obj_enc_process interface version library you use to test.

can you share config file

I’m using the latest deepstream SDK version 5.0.1. There is no special nvds_obj_enc_process interface version library, it is just a part of deepstream SDK

Hello,My config file is as follows.
ds_image_meta_pgie_config.txt (3.3 KB)

Hello.Can you share your sample code and config file with me and I want to run it in my test environment.

I just modify deepstream_image_meta_test.c directly. deepstream_image_meta_test.c (20.3 KB)

Hello,I use your sample code for testing, but the result is still incorrect, and the following error is reported.

0:00:18.677584866 15072 0x5597173ad0 WARN nvinfer gstnvinfer.cpp:1975:gst_nvinfer_output_loop: error: Internal data stream error.
0:00:18.677690402 15072 0x5597173ad0 WARN nvinfer gstnvinfer.cpp:1975:gst_nvinfer_output_loop: error: streaming stopped, reason error (-5)
ERROR from element primary-nvinference-engine: Internal data stream error.
Error details: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(1975): gst_nvinfer_output_loop (): /GstPipeline:ds-image-meta-test-pipeline/GstNvInfer:primary-nvinference-engine:
streaming stopped, reason error (-5)
Returned, stopping playback

The details are printed in the attachment file.I checked that my SDK version is the latest version 5.0.1. Can you add some printing in the nvds_obj_enc_process interface to see why I have wrong results.

log.txt (3.4 KB)