Nveglrender does not display

[mjnhb20011]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) / Jetson
• DeepStream Version / 6.0
• JetPack Version (valid for Jetson only) / 4.6.0
• TensorRT Version / 10.2
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) / questions

Hi, I’m made a two pipelines which is for decoding and disply.

decoding pipeline is
appsrc ! h264parse ! nvv4l2decoder ! nvvideoconvert ! videoconvert ! capsfilter(video/x-raw,format=RGBA) ! appsink // (manually push h264 encoded buffer to appsrc)

display pipeline is
appsrc ! queue ! tee name=t ! fakesink
t. ! videoconvert ! capsfilter(video/x-raw,format=RGBA) ! nvvideoconvert ! capsfilter(video/x-raw(memory:NVMM),format=RGBA) ! nvdsosd ! nvegltransform ! nveglglessink

I send GstBuffer from decoding pipeline’s appsink to display pipeline’s appsrc using function which connected to new-sample signal in appsink (copied from on_new_sample function in deepstream_ipc_test_app.c)

static GstFlowReturn onNewSampleFromSink(_GstAppSink *sink, gpointer *userData)
        {
            GstSample *sample = nullptr;
            GstBuffer *srcBuffer = nullptr;
            GstMapInfo srcMap;
            GstElement *appsrc = (GstElement *)userData;
            GstFlowReturn ret;
            static GMutex sinkLock;


            if (gst_app_sink_is_eos(sink)) {
                g_printerr("analyze pipeline received EOS\n");
                return GST_FLOW_EOS;
            }

            if(!(sample = gst_app_sink_pull_sample(GST_APP_SINK(sink)))) {
                g_printerr("analyze pipeline can not receive sample\n");
                return GST_FLOW_OK;
            }

            if (!(srcBuffer = gst_sample_get_buffer(sample))) {
                g_printerr("No more buffers available from app sink element\n");
                return GST_FLOW_ERROR;
            }

            if (!gst_buffer_map(srcBuffer, &srcMap, GST_MAP_READ)) {
                g_printerr("Map the gst buffer failed\n");
                return GST_FLOW_ERROR;
            }

            g_mutex_lock(&sinkLock);
            GstBuffer *dstBuffer = gst_buffer_new_allocate(nullptr, srcMap.size, nullptr);
            GstMapInfo dstMap;

            gst_buffer_map(dstBuffer, &dstMap, GST_MAP_WRITE);

            int copyFlags = (GST_BUFFER_COPY_META | GST_BUFFER_COPY_FLAGS | GST_BUFFER_COPY_TIMESTAMPS);
            gst_buffer_copy_into(dstBuffer, srcBuffer, (GstBufferCopyFlags)copyFlags, 0, -1);

            memcpy(dstMap.data, srcMap.data, srcMap.size);
            dstMap.size = srcMap.size;

            gst_buffer_unmap(dstBuffer, &dstMap);
            GstMemory *mem = gst_buffer_peek_memory(dstBuffer, 0);
            gst_mini_object_weak_ref(GST_MINI_OBJECT(mem), (GstMiniObjectNotify)gst_mem_free_cb, sample);
            gst_sample_ref(sample);
            gst_app_src_push_buffer((GstAppSrc *)appsrc, dstBuffer);

            g_mutex_unlock(&sinkLock);

            gst_sample_unref(sample);

            return GST_FLOW_OK;
        }

I didn’t get any errors in this code but renderer does not display anything. Can somebody point out what’s wrong with my pipeline?

Thanks in advance.

[junshengy]

If you are working on Jetson, use nv3dsink. nveglglesink only supports dGPU (x86).

/* Finally render the osd output */
  if(prop.integrated) {
    sink = gst_element_factory_make("nv3dsink", "nv3d-sink");
  } else {
#ifdef __aarch64__
    sink = gst_element_factory_make ("nv3dsink", "nvvideo-renderer");
#else
    sink = gst_element_factory_make ("nveglglessink", "nvvideo-renderer");
#endif
  }

[mjnhb20011]

Hi, thanks for the reply.

nvd3dsink element is supported after deepstream 7.0 right?

I’m using deestream 6.0

I need to send buffer to another pipeline so I used function in example deepstream_ipc_test_app.c

thanks.

[junshengy]

Sorry, I didn’t notice that you are using jetson nano.nv3dsink is indeed only supported in newer versions.
Your problem doesn’t seem to be related to display.

It looks like your pipeline is correct, I have two questions.

1. In order to use appsink, you copied the data from GPU to CPU, Can you dump the RGBA data wrapped by srcBuffer to a file to make sure the pipeline is working properly ? Similarly, can you dump the data sent from appsrc?

2. The above two pipelines should only work in the same process, do you use multiple processes?
   The ipc function requires a higher version of BSP support. You may try deepstream-appsrc-test

[mjnhb20011]

Hi, Thanks for the reply.

I solved the problem. root cause was capsfilter setting was not equal between appsrc and appsink.

I used gst_sample_get_caps function to get GstCaps from appsink and set it to appsrc.

@junshengy

The above two pipelines should only work in the same process, do you use multiple processes?
The ipc function requires a higher version of BSP support. You may try deepstream-appsrc-test

No it’s single process. Can I ask one more question?

I think copy data from GPU to CPU and send it to another pipeline is inefficient.

Can I transfer GstBuffer in gpu memory to another pipeline?

[junshengy]

Glad to hear that.

@mjnhb20011

Can I transfer GstBuffer in gpu memory to another pipeline?

First, the gpu memory is holded by a handle called Nvbufsurface.

If it is a single process， try this code snippets.

Get Nvbufsurface from GstBuffer

  GstBuffer* recv_buffer = gst_sample_get_buffer(sample);
  if (!recv_buffer) {
    ERROR_MSG("Error samping buffer from GStreamer pipeline");
    return kErrUnknown;
  }

  gst_buffer_map(recv_buffer, &map, (GstMapFlags)GST_MAP_READ);
  vb->timestamp = GST_BUFFER_PTS(recv_buffer);
  vb->surfaceRef = reinterpret_cast<NvBufSurface*>(map.data);
  gstSampleMap_.insert(std::make_pair(vb->surfaceRef, sample));
  gst_buffer_unmap(recv_buffer, &map);

Build a GstBuffer from NvBufSurface

  /* Create a new buffer with NvSurfaceBuf */
  GstBuffer* buffer =
          gst_buffer_new_allocate(nullptr, sizeof(NvBufSurface), nullptr);
  GstMapInfo map;
  gst_buffer_map(buffer, &map, GST_MAP_WRITE);
  memcpy(map.data, vb->surfaceRef, sizeof(NvBufSurface));
  map.size = sizeof(NvBufSurface);
  gst_buffer_unmap(buffer, &map);
  /* Set its timestamp */
  GST_BUFFER_PTS(buffer) = vb->timestamp;

  /* Push buffer to app src */
  GstFlowReturn pushret = gst_app_src_push_buffer(GST_APP_SRC(appsrc_), buffer);