When I use jpeg encoding function in DeepStream SDK,I I found the following two problems：
（1）When I use nvjpegenc plugin to do jpeg encoding in pipeline, the simple pipeline process is shown below:
filesrc location=video_decode_0.yuv → videdoparse → nvvideoconvert → nvstreammux → nvvideoconvert → nvjpegenc → filesrc location=www.jpg
If the nvstreammux plugin(batch_size=1) input one yuv image, the nvjpegenc plugin can work successfully and print the encoding time consuming.But If the nvstreammux plugin(batch_size=2) input two yuv image,the nvjpegenc plugin can’t work properly and don‘t have time consuming printing.I want to know if the nvjpegenc plugin supports multiple batch encoding.
(2)When I use nvds_obj_enc_process interface to do batch jpeg encode,I found that there might be a problem with this interface.
My hardware platform is Jetson Xavier NX,my pipeline is similar to the sample_apps(path:/opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-image-meta-test).If the nvstreammux plugin(batch_size=1) input one yuv image, the nvds_obj_enc_process interface can work successfully and save multiple object jpg image.But If the nvstreammux plugin(batch_size=2) input two yuv image,the nvinfer plugin can give correct obj area coordinates,the nvds_obj_enc_process interface can’t save correct object jpg image in second yuv image.The following are the two input images(two_image.jpg) and the corresponding detection area.The object in left image can do jpeg encoding normally(left_obj.zip).But the object in right image can’t get correct jpg image(right_obj.zip).The obj area coordinates are correct ,but the image used for encoding is the one on the left.
This problem appears in many tests. Please see if the nvds_obj_enc_process interface is defective.
(1)nvjpegenc is not deepstream plugin( please check all deepstream plugins here GStreamer Plugin Overview — DeepStream 6.1.1 Release documentation), so nvjpegenc can not handle batch data. You need to either add nvmultistreamtiler or nvstreamdemux to extract buffers from batch before you using nvjpegenc.
(2) We will check the nvds_obj_enc_process interface
I just modified the create_source_bin function in deepstream-image-meta-test sample，and the corresponding code is as follows：
static GstElement *
create_source_bin (guint index, gchar * uri)
{
GstElement bin = NULL;
uint32_t width = 1920;
uint32_t height = 1080;
gchar bin_name[16] = { };
g_snprintf (bin_name, 15, “source-bin-%02d”, index);
/ Create a source GstBin to abstract this bin’s content from the rest of the
bin = gst_bin_new (bin_name);
source = gst_element_factory_make ("filesrc", "source");
videoparser = gst_element_factory_make ("jpegparse", "jpeg-parser");
decode = gst_element_factory_make ("nvv4l2decoder", "decoder0");
if(!videoparser || !decode )
{
cout << "gst_element_factory_make videoparser or encoder error\n" << endl;
return NULL;
}
g_object_set (G_OBJECT (source), "location", uri, NULL);
gst_bin_add_many (GST_BIN (bin),source, videoparser, decode ,NULL);
gst_element_link_many (source, videoparser, decode ,NULL);
if (!gst_element_add_pad (bin, gst_ghost_pad_new_no_target ("src",GST_PAD_SRC)))
{
cout << "Failed to add ghost pad in source bin" << endl;
return NULL;
}
srcpad = gst_element_get_static_pad (decode, "src");
if (!srcpad)
{
cout << "Failed to get src pad of source bin. Exiting." << endl;
return NULL;
}
GstPad *bin_ghost_src_pad = gst_element_get_static_pad (bin, "src");
if (!gst_ghost_pad_set_target (GST_GHOST_PAD (bin_ghost_src_pad),srcpad))
{
cout << "Failed to link encoder src pad to source bin ghost pad" << endl;
return NULL;
}
return bin;
}
After modification，you can execute the order ‘./deepstream-image-meta-test image_0.jpg image_1.jpg’.It will set num_sources variable to 2.And the number of entries that the nvstreammux plugin accepts depends on the number of images you enter in the command.
Here are the results of my tests.I guess it may be the difference caused by different versions.You can send me the nvds_obj_enc_process interface version library you use to test.
I’m using the latest deepstream SDK version 5.0.1. There is no special nvds_obj_enc_process interface version library, it is just a part of deepstream SDK
The details are printed in the attachment file.I checked that my SDK version is the latest version 5.0.1. Can you add some printing in the nvds_obj_enc_process interface to see why I have wrong results.
