How to apply osd on yuv in the mmapi frontend example

Hi,
Please try MODE_CPU. Only this mode is supported in nvosd_put_text().

1 Like

Hello,


If you look at the preview screen of the frontend sample, you can see that the bounding box (indicated by the blue arrow) is displayed.
However, there is no bounding box in the encoded video.

How to encode video including bounding box?

Thank you.

Can you tell me how to encode video including bounding box ?

Hi,
It is a bit confusing and we would like to make clarification. In attached code, it looks like you would like to put text hello world gc camera to video frames. Do we misunderstand the request? Please help clarify this.

1 Like

Hello,

Attach the h.265 encoded video.
I want to encode basic shape osd like text or rectangle with yuv overlaid.
It seems to be applied only when the osd previews display.

output3.h265 (53.5 MB)
output1.h265 (53.6 MB)
output2.h265 (53.3 MB)

Thank you.

Hello, @DaneLLL

I want the osd to be applied to outputx.h.265 as output.

I want the osd to be applied to outputx.h.265 as output.
Is this possible?
The frontend sample description states as follows, and I understood that when the video is encoded in h.264, h.265, it is encoded with the osd drawn on the video.


Can I encode to h.264, h.265 with the osd as I want?
Can you tell me how if possible?

Thank you.

Hi,
By default it encodes to trt.h264. Please check the file. If you need to encode to h265, please modify here:

TRTStreamConsumer::TRTStreamConsumer(const char *name, const char *outputFilename,
        Size2D<uint32_t> size, NvEglRenderer *renderer, bool hasEncoding) :
    StreamConsumer(name, size),
    m_VideoEncoder(name, outputFilename, size.width(), size.height(), V4L2_PIX_FMT_H264),
    m_hasEncoding(hasEncoding),
    m_eglRenderer(renderer)
1 Like

Hello,

If I apply what you said, will the video be encoded with the osd applied?
That is, if you check the encoded video, can you check the osd(text, rectangle, etc…)?

Thank you.

Hello, @DaneLLL

When I run the frontend example, the program sometimes dies with an error like the following, can you tell what the problem is?

imx327@imx327-desktop:/usr/src/jetson_multimedia_api/samples/frontend_orig$ sudo ./frontend --deploy …/…/data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.prototxt --model …/…/data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.caffemodel
[INFO] (NvEglRenderer.cpp:110) Setting Screen width 640 height 480
Full resolution: 1920x1080
[86817.449607] trt: Creating TRT model…
Net has batch_size, channel, net_height, net_width:1 3 368 640
Using cached TRT model
Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[86821.977451] trt: Batch size: 1
Error generated. main.cpp, runArgusProducer:97 Failed to get ICaptureSession interface

Thank you.

Hi,
Rectangles are put in

bool TRTStreamConsumer::RenderThreadProc()

nvosd_draw_rectangles() is called in the function. You can add nvosd_put_text() for putting text.

1 Like

Hello,

How can I make the osd appear as drawn in the encoded output file?

Thank you.

Hi,
Do you mean you don’t see bounding boxes in trt.h264? Since the samples is open source we would suggest trace the code to get more information and understanding. So that you are able to do customization.

1 Like

Hello, @DaneLLL

yes.


the frame with the bounding box is rendered on the display and simultaneously encoded as an H.264 or H.265 file.

The sample description is as above, so I thought it was basically possible.

  1. Isn’t this a function provided by the original sample?
  2. Can you tell me how to approach it?

Thank you.

Hi,
Please take a look at the function:

bool TRTStreamConsumer::RenderThreadProc()

In the function it does rendering and encoding. We can see bounding boxes on screen and encoded trt.h264. Please confirm if you run the default sample. Probably you have done certain customization so see different result.

1 Like

Hello,

I confirmed that rectangle osd came out from trt.h264.
Can you tell what kind (ex, person, car) this cafe model class is?

Thank you

Hi,
The default model detects person, car and motorcycle.

1 Like

Hello,

I used the imx327 mipi csi camera to run the sample.

When I run the sample, the detection performance that can be checked on the preview screen is not good.

Can you run the sample directly and attach the resulting output image (except for sensitive information)?

Thank you.

Hello, @DaneLLL

Thank you for your effort.
What fd should I use when calling nvosd in the frontend sample to display the osd in output.h26x rather than trt.h264?
I encountered a segmentation fault when I called nvosd_put_text(nvosd_text_context, MODE_CPU, buffer->planes[0].fd, 1, &textParam); inside the VideoEncoder::encoderCapturePlaneDqCallback function in VideoEncoder.cpp as follows.

bool
VideoEncoder::encoderCapturePlaneDqCallback(
        struct v4l2_buffer *v4l2_buf,
        NvBuffer *buffer,
        NvBuffer *shared_buffer)
{
    if (!v4l2_buf)
    {
        m_VideoEncoder->abort();
        ORIGINATE_ERROR("Failed to dequeue buffer from capture plane");
    }

#ifdef SY_NVOSD
        std::cout << __FUNCTION__ << ", " << __LINE__ << std::endl;
	    // Draw text
        void *nvosd_text_context;

        // NvOSD_TextParams textParam = { 0 };
        NvOSD_TextParams textParam;
        textParam.display_text = strdup("flexwatch") ? : strdup("nvosd overlay text");
        textParam.x_offset = 230;
        textParam.y_offset = 230;
        textParam.font_params.font_name = strdup("Arial");
        textParam.font_params.font_size = 30;
        textParam.font_params.font_color.red = 1.0;
        textParam.font_params.font_color.green = 0.0;
        textParam.font_params.font_color.blue = 1.0;
        textParam.font_params.font_color.alpha = 1.0;
        // m_textParams.push_back(textParam);
	    nvosd_put_text(nvosd_text_context, MODE_CPU, buffer->planes[0].fd, 1, &textParam);
#endif // SY_NVOSD

    m_outputFile->write((char *) buffer->planes[0].data, buffer->planes[0].bytesused);

NvMMLiteOpen : Block : BlockType = 8
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
892744264
842091865
NVMEDIA: H265 : Profile : 1
[98.884385] enc2: Waiting until producer is connected…
PRODUCER: Starting repeat capture requests.
[98.919986] enc0: Producer has connected; continuing.
[98.931402] enc1: Producer has connected; continuing.
[98.961165] enc2: Producer has connected; continuing.
[98.987413] trt: Producer has connected; continuing.
NVMEDIA_ENC: bBlitMode is set to TRUE
NVMEDIA_ENC: bBlitMode is set to TRUE
NVMEDIA_ENC: bBlitMode is set to TRUE
encoderCapturePlaneDqCallback, 275
Segmentation fault

Thank you.

Hi,
Encoded buffers are compressed h264/h265 stream. You have to put text/rectangles on YUV frame data. Please move the code to apply to buffers before feeding to encoder.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.