Emulate a USB camera from computer vision output both internally (v4l2loopback) and externally (as usb slave device)

Autonomous Machines Jetson & Embedded Systems Jetson Nano
,
9

1. If you want to send opencv processed frame to v4l2loopback (assuming here you have a v4l2loopback dev node /dev/video10, you would use a videoWriter such as:

cv::VideoWriter gst_v4l2sink ("appsrc ! videoconvert ! video/x-raw,format=BGRx ! identity drop-allocation=true ! v4l2sink device=/dev/video10", cv::CAP_V4L2, 0, fps, cv::Size (width, height));
if (!gst_v4l2sink.isOpened ()) {
    std::cout << "Failed to open gst_v4l2sink writer." << std::endl;
    return (-8);
}

and in the loop push processed frames at specified fps:

gst_v4l2sink.write(frame_out);

2. For Android host, you may better tell how you would access to Jetson. If connected trough a UDP/IP stack, you may stream your output, encoded as H264 (or H265, VP9…) into RTP over UDP to multicast address :

cv::VideoWriter gst_udpsink("appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw, format=BGRx ! nvvidconv ! nvv4l2h264enc insert-vui=1 ! video/x-h264, stream-format=byte-stream ! h264parse ! rtph264pay pt=96 config-interval=1 ! udpsink host=224.1.1.1 port=5000 auto-multicast=true", cv::CAP_GSTREAMER, 0, fps, cv::Size (width, height));
if (!gst_udpsink.isOpened ()) {
    std::cout << "Failed to open gst_udpsink writer." << std::endl;
    return (-8);
}

and push in the loop with:

gst_udpsink.write(frame_in);

You would read on remote host with a sdp file such as:

m=video 5000 RTP/AVP 96
c=IN IP4 224.1.1.1
a=rtpmap:96 H264/90000

Such as:

cvlc -v test.sdp

3. Or you may try RTSP (this require package libgstrtspserver-1.0-dev and test-launch example being built). This would imply opencv sending to shmsink through gstreamer, and test-launch serving RTSP from shmsrc:
cv::VideoWiter to shmsink:

cv::VideoWriter h264_shmsink ("appsrc is-live=true ! queue ! videoconvert ! video/x-raw, format=RGBA ! nvvidconv ! omxh264enc insert-vui=true ! video/x-h264, stream-format=byte-stream ! h264parse ! shmsink socket-path=/tmp/my_h264_sock ", cv::CAP_GSTREAMER, 0, fps, cv::Size (width, height));
if (!h264_shmsink.isOpened ()) {
    std::cout << "Failed to open h264_shmsink writer." << std::endl;
    return (-2);
}

push in loop processed frame to h264_shmsink.

So now you would launch your RTSP server with:

./test-launch "shmsrc socket-path=/tmp/my_h264_sock do-timestamp=true ! video/x-h264, stream-format=byte-stream, width=640, height=480, framerate=30/1 ! h264parse ! video/x-h264, stream-format=byte-stream ! rtph264pay pt=96 name=pay0 "

Note that you would have to manage the socket cleaning if any when closing client and server.

and you should be able to receive on host:

cvlc -v rtsp:<jetson_IP>:8554/test
2 Likes