Xavier ZED gstreamer Web Streaming

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hi.

I’m trying simple web-brwoser(only FF) streaming service on Jetson Xavier with ZED camera.

It works with pure gstreamer pipeline scripts, but it took 100% of CPU on converting,
so I tried Accelrated_Resize function, it worked, but web-browser doesn’t get proper data.

I wonder what is different between two pipelines.

HTML Code (Only FF works)

<!DOCTYPE html>
<html>
        <head>
                <meta http-equiv="content-type" content="text/html; charset=utf-8">
                <title>gst-stream</title>
        </head>
        <body>
                <video width=320 height=180 autoplay>
                        <source src="http://192.168.10.102:8080">
                </video>
        </body>
</html>

Pure Gstreamer Code

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1344,height=376,pixelformat=YUYV -p15 

gst-launch-1.0 \
	v4l2src device=/dev/video0 \
	! videoconvert \
	! videoscale \
	! video/x-raw, width=320, height=180, format=I420 \
	! clockoverlay shaded-background=true font-desc="Sans 38" \
	! queue \
	! theoraenc \
	! oggmux \
	! tcpserversink host=0.0.0.0 port=8080 sync=true

Xavier Acceled Code

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1344,height=376,pixelformat=YUYV -p15 

gst-launch-1.0 v4l2src device=/dev/video0 \
	! "video/x-raw, format=(string)YUY2, width=(int)1344, height=(int)376" \
	! nvvidconv left=0 right=672 top=0 bottom=376 \
	! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)320, height=(int)180" \
	! tcpserversink host=0.0.0.0 port=8080 sync=true

Both pipeline get YUYV(YUY2) stream from ZED, convert to I420, resize, and send.
Both pipeline works, but html webpage only receive first one, the other can’t.(no decoder for the stream)

I’m not sure what’s wrong with 2nd code.

May I take any advice for this problem?

Best regards

2

Maybe tcpserversink fails to read from NVMM memory. You may try to add a second nvvidconv for copying to CPU memory:

gst-launch-1.0 v4l2src device=/dev/video0 \
	! "video/x-raw, format=(string)YUY2, width=(int)1344, height=(int)376" \
	! nvvidconv left=0 right=672 top=0 bottom=376 \
	! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)320, height=(int)180" \
	[b]! nvvidconv \
	! "video/x-raw, format=(string)I420, width=(int)320, height=(int)180" \[/b]
	! tcpserversink host=0.0.0.0 port=8080 sync=true
3

Thank you for reply, however it wasn’t enough for the problem.

I found a point the Xavier Acceled Code doesn’t have ‘theoraenc’ & ‘oggmux’.
It encode a I420 raw stream to ogg stream for web-streaming on browser.

So I changed code like,

nvpmodel -m 0

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1344,height=376,pixelformat=YUYV -p15 

gst-launch-1.0 v4l2src device=/dev/video0 \
	! "video/x-raw, format=(string)YUY2, width=(int)1344, height=(int)376" \
	! nvvidconv left=0 right=672 top=0 bottom=376 \
	! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)320, height=(int)240, pixel-aspect-ratio=1/1" \
	! nvvidconv \
[b]	! theoraenc bitrate=256 \
	! oggmux \[/b]
	! tcpserversink host=0.0.0.0 port=8080 sync=true

Then, it worked well…

Now I’m afraid that theroaencoder and oggmuxer take huge cpu usage right now.
The pipeline take almost 85% cpuusage of Xavier with BAD POWER MODE (nvpmodel -m 0)

[ZED] 1344 x 376 RAW YUYV 15fps
↓
[CROP] 672 x 376 RAW YUYV 15fps
↓
[RESIZE] 320 x 240 RAW I420 15fps
↓
[THEROA ENCODE] bitrate=256
↓
[OGG MUXER]

Is that normal, this encoding flow took 85% of Xavier CPU Usage…?

4

Hi,
We don’t have HW component of theoraenc on Xavier, so your case is pure CPU pipeline and has high CPU loading.

For encoding on Xavier, we have h264 and h265 HW encoders. If you have other usecases requiring h264/h265 stream, it shall work better.

5

I was going to say the same: Yes, Theora is CPU only.

I’ve used the H264 and H265 hardware codecs successfully. Just be aware of the macroblock size restrictions – I think you need the input image to be a multiple of 16x16 for H265, and you won’t get an error, you’ll just get garbage out if you get it wrong.

The cropping and resizing, meanwhile, is trivial – even if you did it on the CPU, it wouldn’t take a lot of CPU effort. However, if you can do it in hardware, then why not? :-)