About streammux and appsrc

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

Can nvstreamux convert received host memory(video/x-raw) frame data into device memory(video/x-raw(memory:NVMM)) data? For subsequent nvinfer use.

#2

From the description below, nvstreamux is acceptable for video/x-raw data, but it will crash when I set it in the test program.

gst-inspect-1.0 nvstreammux

Pad Templates:
SINK template: ‘sink_%u’
Availability: On request
Has request_new_pad() function: gst_nvstreammux_request_new_pad
Capabilities:
video/x-raw(memory:NVMM)
format: { NV12, RGBA }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { NV12, RGBA }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]

SRC template: ‘src’
Availability: Always
Capabilities:
video/x-raw(memory:NVMM)
format: { NV12, RGBA }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]

#3

nvvidconv can do this conversion.

#4

I am trying to use appsrc and nvinfer to perform inference:

cv::VideoWriter writer;
    cv::Size sz = cv::Size(640, 480);
    auto pipe="appsrc !nvvideoconvert ! m.sink_0 nvstreammux name=m batch-size=1 width=800 height=600 "
              "!  nvinfer config-file-path=yolo.txt ! fakesink";
    cout << "open:" << pipe<<endl;
    writer.open(pipe , 0, (double)30, sz , true);
    if (!writer.isOpened()) {
        printf("=ERR= can't create video writer\n");
        return -1;
    }
    int fc=0;
    while (true){
        Mat fr = Mat::zeros(sz,CV_8UC3);
        string str= "frame " + to_string(fc);
        putText(fr,str,Point(200,200),0,1,Scalar(255),2);
        fc++;
        writer<<fr;
        waitKey(30);
    }

but errors:

open:appsrc !nvvideoconvert ! m.sink_0 nvstreammux name=m batch-size=1 width=800 height=600 !  nvinfer config-file-path=yolo.txt ! fakesink
[ WARN:0] global /home/stiv/opencv-4.1.1/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module appsrc0 reported: Internal data stream error.
[ WARN:0] global /home/stiv/opencv-4.1.1/modules/videoio/src/cap_gstreamer.cpp (1663) writeFrame OpenCV | GStreamer warning: Error pushing buffer to GStreamer pipeline

how can I fix that?

#5

Could you try using gst-lauch-1.0 videotestsrc !nvvideoconvert ! m.sink_0 nvstreammux name=m batch-size=1 width=800 height=600 ! nvinfer config-file-path=yolo.txt ! fakesink and see if it can work? videotestsrc which gives raw output. Then convert to NVMM output using nvvideoconvert and then use it in the rest of the pipeline.

#6