How to change source pixel format from YUYV to MJPG

Hi,
I’m using deepstream_app of recent version 4.0, and using with yolo detector through logitech webcam(c930e).
When I run deepstream app with yolo config file, it runs with YUYV pixel format, but I would like to use with MJPG format, as its fps is faster than YUYV’s.(As shown in bottom)

:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 160x120
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 176x144
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 320x180
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 424x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 480x270
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 800x448
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 848x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 960x540
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1024x576
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1600x896
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 2304x1296
			Interval: Discrete 0.500s (2.000 fps)
		Size: Discrete 2304x1536
			Interval: Discrete 0.500s (2.000 fps)

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 160x120
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 176x144
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 320x180
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 352x288
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 424x240
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 480x270
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 640x360
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 800x448
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 848x480
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 960x540
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1024x576
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1600x896
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.042s (24.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.067s (15.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
			Interval: Discrete 0.133s (7.500 fps)
			Interval: Discrete 0.200s (5.000 fps)

Source input is set as below and I would like to also set for the pixel format but seems there is no option…

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=1
camera-width=1280
camera-height=720
camera-fps-n=30
camera-v4l2-dev-node=0
#uri=file://../../samples/streams/sample_1080p_h264.mp4
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

option for source input

37 #define CONFIG_GROUP_SOURCE_TYPE "type"
  38 #define CONFIG_GROUP_SOURCE_CAMERA_WIDTH "camera-width"
  39 #define CONFIG_GROUP_SOURCE_CAMERA_HEIGHT "camera-height"
  40 #define CONFIG_GROUP_SOURCE_CAMERA_FPS_N "camera-fps-n"
  41 #define CONFIG_GROUP_SOURCE_CAMERA_FPS_D "camera-fps-d"
  42 #define CONFIG_GROUP_SOURCE_CAMERA_CSI_SID "camera-csi-sensor-id"
  43 #define CONFIG_GROUP_SOURCE_CAMERA_V4L2_DEVNODE "camera-v4l2-dev-node"
  44 #define CONFIG_GROUP_SOURCE_URI "uri"
  45 #define CONFIG_GROUP_SOURCE_LATENCY "latency"
  46 #define CONFIG_GROUP_SOURCE_NUM_SOURCES "num-sources"
  47 #define CONFIG_GROUP_SOURCE_INTRA_DECODE "intra-decode-enable"
  48 #define CONFIG_GROUP_SOURCE_NUM_DECODE_SURFACES "num-decode-surfaces"
  49 #define CONFIG_GROUP_SOURCE_NUM_EXTRA_SURFACES "num-extra-surfaces"
  50 #define CONFIG_GROUP_SOURCE_DROP_FRAME_INTERVAL "drop-frame-interval"
  51 #define CONFIG_GROUP_SOURCE_CAMERA_ID "camera-id"

Is there any possible way to use deepstream with MJPG format, so that I can run in HD resolution with 30fps??
Any help would be grateful…

Thanks,
Chris Jung

Hi,
It is not supported in default deepstream-app. You have to modify the code and rebuild the application. The cpp file is

deepstream_sdk_v4.0_jetson\sources\apps\apps-common\src\deepstream_source_bin.c

Default source bin is

v4l2src ! videoconvert ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=NV12'

For MJPG source, it should be

v4l2src ! image/jpeg ! jpegdec ! videoconvert ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=NV12'

Hi, DaneLLL

Thanks for reply! But still need help.
I found the deepstream_source_bin.c file, but can’t get your words.

I thought it can be simply fixed by adding two words(image/jpg ! jpegdec !) at the line

v4l2src ! videoconvert ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=NV12'

but doesn’t exist in the file.
What I can see is following code for V4L2 Source.

if (config->type == NV_DS_SOURCE_CAMERA_V4L2) {
    GstElement *nvvidconv1, *nvvidconv2;
    GstCapsFeatures *feature = NULL;


    nvvidconv1 = gst_element_factory_make ("videoconvert", "nvvidconv1");
    if (!nvvidconv1) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'nvvidconv1'");
      goto done;
    }

    feature = gst_caps_features_new ("memory:NVMM", NULL);
    gst_caps_set_features (caps, 0, feature);
    g_object_set (G_OBJECT (bin->cap_filter), "caps", caps, NULL);

    nvvidconv2 = gst_element_factory_make (NVDS_ELEM_VIDEO_CONV, "nvvidconv2");
    if (!nvvidconv2) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'nvvidconv2'");
      goto done;
    }

    g_object_set (G_OBJECT (nvvidconv2), "gpu-id", config->gpu_id,
        "nvbuf-memory-type", config->nvbuf_memory_type, NULL);

    gst_bin_add_many (GST_BIN (bin->bin), bin->src_elem, bin->cap_filter,
        nvvidconv1, nvvidconv2, bin->cap_filter,
        NULL);

    NVGSTDS_LINK_ELEMENT (bin->src_elem, nvvidconv1);

    NVGSTDS_LINK_ELEMENT (nvvidconv1, nvvidconv2);

    NVGSTDS_LINK_ELEMENT (nvvidconv2, bin->cap_filter);

    NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter, "src");

  }

Did I misunderstand your answer?
If I have to add the codes about image/jpg and jpegdec,
is there any documents for guideline?

Thanks,
Chris Jung

Hi,
You have to create the element and link it. Below is psuedo code:

jpeg = gst_element_factory_make ("jpegenc", "jpeg-encoder");
NVGSTDS_LINK_ELEMENT (bin->src_elem, jpeg);

Hi DaneLLL,

I modified the code according to the pipeline which you suggested.

deepstream_source_bin.c : line 60 to 150
34-37 video/x-raw -> image/jpeg
47 + GstElement *jpg_dec
50-54 + jpg_dec gst element factory make & error message
65-70 + convert to NV12 format
81 + gst_bin_add_many(jpg_dec)
83 + NVGSTDS_LINK_ELEMENT (bin->src_elem, jpg_dec);
85 + NVGSTDS_LINK_ELEMENT (jpg_dec, nvvidconv1);

static gboolean
create_camera_source_bin (NvDsSourceConfig * config, NvDsSrcBin * bin)
{
  GstCaps *caps = NULL, *convertCaps = NULL;
  gboolean ret = FALSE;

  switch (config->type) {
    case NV_DS_SOURCE_CAMERA_CSI:
      bin->src_elem =
          gst_element_factory_make (NVDS_ELEM_SRC_CAMERA_CSI, "src_elem");
      g_object_set (G_OBJECT (bin->src_elem), "bufapi-version", TRUE, NULL);
      g_object_set (G_OBJECT (bin->src_elem), "maxperf", TRUE, NULL);
      break;
    case NV_DS_SOURCE_CAMERA_V4L2:
      bin->src_elem =
          gst_element_factory_make (NVDS_ELEM_SRC_CAMERA_V4L2, "src_elem");
      break;
    default:
      NVGSTDS_ERR_MSG_V ("Unsupported source type");
      goto done;
  }

  if (!bin->src_elem) {
    NVGSTDS_ERR_MSG_V ("Could not create 'src_elem'");
    goto done;
  }

  bin->cap_filter =
      gst_element_factory_make (NVDS_ELEM_CAPS_FILTER, "src_cap_filter");
  if (!bin->cap_filter) {
    NVGSTDS_ERR_MSG_V ("Could not create 'src_cap_filter'");
    goto done;
  }
  caps = gst_caps_new_simple ("image/jpeg",
      "width", G_TYPE_INT, config->source_width, "height", G_TYPE_INT,
      config->source_height, "framerate", GST_TYPE_FRACTION,
      config->source_fps_n, config->source_fps_d, NULL);

  if (config->type == NV_DS_SOURCE_CAMERA_CSI) {
    GstCapsFeatures *feature = NULL;
    feature = gst_caps_features_new ("memory:NVMM", NULL);
    gst_caps_set_features (caps, 0, feature);
  }

  //g_object_set (G_OBJECT (bin->cap_filter), "caps", caps, NULL);
  if (config->type == NV_DS_SOURCE_CAMERA_V4L2) {
    GstElement *jpg_dec, *nvvidconv1, *nvvidconv2;
    GstCapsFeatures *feature = NULL;

    jpg_dec = gst_element_factory_make ("jpegdec", "jpeg-decoder");
    if (!jpg_dec) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'jpg_dec'");
      goto done;
    }

    nvvidconv1 = gst_element_factory_make ("videoconvert", "nvvidconv1");
    if (!nvvidconv1) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'nvvidconv1'");
      goto done;
    }

    feature = gst_caps_features_new ("memory:NVMM", NULL);
    gst_caps_set_features (caps, 0, feature);

    GstCaps *lnvcaps = gst_caps_new_simple ("video/x-raw", "format", G_TYPE_STRING,"NV12", 
"width", G_TYPE_INT, config->source_width, "height", G_TYPE_INT, config->source_height, 
"framerate", GST_TYPE_FRACTION, config->source_fps_n, config->source_fps_d, NULL);

    g_object_set (G_OBJECT (bin->cap_filter), "caps", lnvcaps, NULL);
    gst_caps_unref (lnvcaps);

    nvvidconv2 = gst_element_factory_make ("nvvideoconvert", "nvvidconv2");
    if (!nvvidconv2) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'nvvidconv2'");
      goto done;
    }

    g_object_set (G_OBJECT (nvvidconv2), "gpu-id", config->gpu_id,
        "nvbuf-memory-type", config->nvbuf_memory_type, NULL);

    gst_bin_add_many (GST_BIN (bin->bin), bin->src_elem, bin->cap_filter, jpg_dec, nvvidconv1, nvvidconv2, bin->cap_filter, NULL);

    NVGSTDS_LINK_ELEMENT (bin->src_elem, jpg_dec);

    NVGSTDS_LINK_ELEMENT (jpg_dec, nvvidconv1);

    NVGSTDS_LINK_ELEMENT (nvvidconv1, nvvidconv2);

    NVGSTDS_LINK_ELEMENT (nvvidconv2, bin->cap_filter);

    NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter, "src");

  } else {

    g_object_set (G_OBJECT (bin->cap_filter), "caps", caps, NULL);

    gst_bin_add_many (GST_BIN (bin->bin), bin->src_elem, bin->cap_filter, NULL);

    NVGSTDS_LINK_ELEMENT (bin->src_elem, bin->cap_filter);

    NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter, "src");
  }

I rebuilt it and run yolo by config file with absolute path.
The following error occurs.

(deepstream-app:29194): GStreamer-CRITICAL **: 21:06:48.531: passed '0' as denominator for 'GstFraction'
(deepstream-app:29194): GStreamer-CRITICAL **: 21:06:48.534: passed '0' as denominator for `GstFraction'
(deepstream-app:29194): GStreamer-WARNING **: 21:06:48.536: Name 'src_cap_filter' is not unique in bin 'src_sub_bin0', not adding

Using winsys: x11 
Creating LL OSD context new
Deserialize yoloLayerV3 plugin: yolo_17
Deserialize yoloLayerV3 plugin: yolo_24

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.

** INFO: <bus_callback:163>: Pipeline ready

** INFO: <bus_callback:149>: Pipeline running

Creating LL OSD context new
ERROR from src_elem: Internal data stream error.
Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming stopped, reason error (-5)
Quitting
App run failed

The blackscreen window comes up in a sec and quits with failure.
It works with original source file.
There seems to have some mistake on modified code but can’t find it.
Desperately need your help.

Thanks,
Chris Jung

Hi,
It looks like caps created as ‘image/jpeg(memory:NVMM),width=1280,height=720,framrate=30/1’. It should not have (memory:NVMM) and be ‘image/jpeg,width=1280,height=720,framrate=30/1’. Also it is not set to v4l2src. Please call gst_element_link_filtered() to configure it.
https://developer.gnome.org/gstreamer/stable/GstElement.html#gst-element-link-filtered

Hi DaneLLL,
Finally, it works.
I really appreciate your help.

bin->cap_filter =
      gst_element_factory_make (NVDS_ELEM_CAPS_FILTER, "src_cap_filter");
  if (!bin->cap_filter) {
    NVGSTDS_ERR_MSG_V ("Could not create 'src_cap_filter'");
    goto done;
  }
  caps = gst_caps_new_simple ("image/jpeg",
      "width", G_TYPE_INT, config->source_width, "height", G_TYPE_INT,
      config->source_height, "framerate", GST_TYPE_FRACTION,
      config->source_fps_n, config->source_fps_d, NULL);
  
  if (config->type == NV_DS_SOURCE_CAMERA_CSI) {
    GstCapsFeatures *feature = NULL;
    feature = gst_caps_features_new ("memory:NVMM", NULL);
    gst_caps_set_features (caps, 0, feature);
  }

  //g_object_set (G_OBJECT (bin->cap_filter), "caps", caps, NULL);
  if (config->type == NV_DS_SOURCE_CAMERA_V4L2) {
    GstElement *jpg_dec, *nvvidconv1, *nvvidconv2;
    GstCapsFeatures *feature = NULL;


    jpg_dec = gst_element_factory_make ("jpegdec", "jpeg-decoder");
    if (!jpg_dec) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'jpg_dec'");
      goto done;
    }

    nvvidconv1 = gst_element_factory_make ("videoconvert", "nvvidconv1");
    if (!nvvidconv1) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'nvvidconv1'");
      goto done;
    }
    
    GstCaps *lnvcaps = gst_caps_new_simple ("video/x-raw", "format", G_TYPE_STRING,"NV12", "width", G_TYPE_INT, config->source_width, "height", G_TYPE_INT, config->source_height, "framerate", GST_TYPE_FRACTION, config->source_fps_n, config->source_fps_d, NULL);
    feature = gst_caps_features_new ("memory:NVMM", NULL);
    gst_caps_set_features (lnvcaps, 0, feature);
    g_object_set (G_OBJECT (bin->cap_filter), "caps", lnvcaps, NULL);
    

    nvvidconv2 = gst_element_factory_make ("nvvideoconvert", "nvvidconv2");
    if (!nvvidconv2) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'nvvidconv2'");
      goto done;
    }

    g_object_set (G_OBJECT (nvvidconv2), "gpu-id", config->gpu_id,
        "nvbuf-memory-type", config->nvbuf_memory_type, NULL);

    gst_bin_add_many (GST_BIN (bin->bin), bin->src_elem, jpg_dec, nvvidconv1, nvvidconv2, bin->cap_filter, NULL);

    gst_element_link_filtered (bin->src_elem, jpg_dec, caps);

    NVGSTDS_LINK_ELEMENT (jpg_dec, nvvidconv1);

    NVGSTDS_LINK_ELEMENT (nvvidconv1, nvvidconv2);

    NVGSTDS_LINK_ELEMENT (nvvidconv2, bin->cap_filter);

    NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter, "src");

  } else {

    g_object_set (G_OBJECT (bin->cap_filter), "caps", caps, NULL);

    gst_bin_add_many (GST_BIN (bin->bin), bin->src_elem, bin->cap_filter, NULL);

    NVGSTDS_LINK_ELEMENT (bin->src_elem, bin->cap_filter);

    NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter, "src");
  }

Thank you,
Chris Jung

I made the changes but I cant seem to get it to rebuild.
Honestly I am not sure how to do it.
All my attempts have failed.
How do I rebuild

deepstream_sdk_v4.0_jetson\sources\apps\apps-common\src\deepstream_source_bin.c

after I make the changes

1 Like

Hi,
Please go to

deepstream_sdk_v4.0_jetson\sources\apps\sample_apps\deepstream-app\

apps-common is part of deepstrea-app.

Hi,

I am using Deepstream 5.0 and having the same issue converting yuyv to mjpg, I am trying to modify the source_bin code but it is a bit different from deepstream 4, here are the changes:

case NV_DS_SOURCE_CAMERA_V4L2:
      bin->src_elem =
          gst_element_factory_make (NVDS_ELEM_SRC_CAMERA_V4L2, "src_elem");
      bin->cap_filter1 =
          gst_element_factory_make (NVDS_ELEM_CAPS_FILTER, "src_cap_filter1");
      if (!bin->cap_filter1) {
        NVGSTDS_ERR_MSG_V ("Could not create 'src_cap_filter1'");
        goto done;
      }
      caps1 = gst_caps_new_simple ("image/jpeg",
          "width", G_TYPE_INT, config->source_width, "height", G_TYPE_INT,
          config->source_height, "framerate", GST_TYPE_FRACTION,
          config->source_fps_n, config->source_fps_d, NULL);
      break;
    default:
      NVGSTDS_ERR_MSG_V ("Unsupported source type");
      goto done;
  }

  if (!bin->src_elem) {
    NVGSTDS_ERR_MSG_V ("Could not create 'src_elem'");
    goto done;
  }

  bin->cap_filter =
      gst_element_factory_make (NVDS_ELEM_CAPS_FILTER, "src_cap_filter");
  if (!bin->cap_filter) {
    NVGSTDS_ERR_MSG_V ("Could not create 'src_cap_filter'");
    goto done;
  }
  

  if (config->type == NV_DS_SOURCE_CAMERA_CSI) {
    caps = gst_caps_new_simple ("video/x-raw", "format", G_TYPE_STRING, "NV12", "width", G_TYPE_INT, config->source_width, "height", G_TYPE_INT, config->source_height, "framerate", GST_TYPE_FRACTION, config->source_fps_n, config->source_fps_d, NULL);
    GstCapsFeatures *feature = NULL;
    feature = gst_caps_features_new ("memory:NVMM", NULL);
    gst_caps_set_features (caps, 0, feature);
  }

  if (config->type == NV_DS_SOURCE_CAMERA_V4L2) {
      caps = gst_caps_new_simple ("image/jpeg",
      "width", G_TYPE_INT, config->source_width, "height", G_TYPE_INT,
      config->source_height, "framerate", GST_TYPE_FRACTION,
      config->source_fps_n, config->source_fps_d, NULL);
    GstElement *jpg_dec, *nvvidconv1, *nvvidconv2;
    GstCapsFeatures *feature = NULL;
    jpg_dec = gst_element_factory_make ("jpegdec", "jpeg-decoder");
    if (!jpg_dec) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'jpg_dec'");
      goto done;	
     }

    nvvidconv1 = gst_element_factory_make ("videoconvert", "nvvidconv1");
    if (!nvvidconv1) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'nvvidconv1'");
      goto done;
    }

    feature = gst_caps_features_new ("memory:NVMM", NULL);
    gst_caps_set_features (caps, 0, feature);
    g_object_set (G_OBJECT (bin->cap_filter), "caps", caps, NULL);

    g_object_set (G_OBJECT (bin->cap_filter1), "caps", caps1, NULL);

    nvvidconv2 = gst_element_factory_make (NVDS_ELEM_VIDEO_CONV, "nvvidconv2");
    if (!nvvidconv2) {
      NVGSTDS_ERR_MSG_V ("Failed to create 'nvvidconv2'");
      goto done;
    }

    g_object_set (G_OBJECT (nvvidconv2), "gpu-id", config->gpu_id,
        "nvbuf-memory-type", config->nvbuf_memory_type, NULL);

    gst_bin_add_many (GST_BIN (bin->bin), bin->src_elem, bin->cap_filter1, jpg_dec,
        nvvidconv1, nvvidconv2, bin->cap_filter,
        NULL);
    gst_element_link_filtered (bin->src_elem, jpg_dec, caps);

I am able to compile but am having this error:

/deepstream-app -c /opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt
** ERROR: <create_camera_source_bin:164>: Failed to link 'src_cap_filter1' (image/jpeg, width=(int)1280, height=(int)720, framerate=(fraction)10/1) and 'nvvidconv1' (video/x-raw, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string){ I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, .....}, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ])
** ERROR: <create_camera_source_bin:213>: create_camera_source_bin failed
** ERROR: <create_pipeline:1263>: create_pipeline failed
** ERROR: <main:632>: Failed to create pipeline
Quitting
App run failed

I am really not proficient in programming so help is much appreciated. Thanks!

Robi Gregorio

@DaneLLL @jhchris713
I had change the deepstream_sdk_v4.0_jetson\sources\apps\apps-common\src\deepstream_source_bin.c file code to the pipeline which you suggested like as jhchris713 post# 7.
I rebuilt it which you suggested like as post# 9.

my computer environment:
Jetson Nano
deepstream_sdk_v4.0.2

my camera information:

ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'MJPG' (compressed)
        Name        : Motion-JPEG
                Size: Discrete 640x400
                        Interval: Discrete 0.005s (210.000 fps)
                Size: Discrete 320x240
                        Interval: Discrete 0.005s (210.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.005s (210.000 fps)

        Index       : 1
        Type        : Video Capture
        Pixel Format: 'MJPG' (compressed)
        Name        : Motion-JPEG
                Size: Discrete 640x400
                        Interval: Discrete 0.005s (210.000 fps)
                Size: Discrete 320x240
                        Interval: Discrete 0.005s (210.000 fps)
                Size: Discrete 640x360
                        Interval: Discrete 0.005s (210.000 fps)

unexpectedly, I had error like below:

dlinano@jetson-nano:~/deepstream_sdk_v4.0.2_jetson$ deepstream-app -c samples/configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt

(deepstream-app:12143): GStreamer-WARNING **: 00:01:31.744: Name 'src_cap_filter' is not unique in bin 'src_sub_bin0', not adding
Creating LL OSD context new
0:00:01.431210334 12143     0x27feab00 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:useEngineFile(): Failed to read from model engine file
0:00:01.431304350 12143     0x27feab00 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:01.431603588 12143     0x27feab00 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): INT8 not supported by platform. Trying FP16 mode.
0:01:52.945888838 12143     0x27feab00 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /home/dlinano/deepstream_sdk_v4.0.2_jetson/samples/models/Primary_Detector/resnet10.caffemodel_b1_fp16.engine

Runtime commands:
        h: Print this help
        q: Quit

        p: Pause
        r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF: FPS 0 (Avg)
**PERF: 0.00 (0.00)
** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

ERROR from src_elem: Internal data stream error.
Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming stopped, reason not-negotiated (-4)
Quitting
App run failed

Any advise for me is much appreciated. Thanks!