Video Broadcasting - Latency/Lag reduction

Dear All

I am new on the NVIDIA JETSON and I would like to know how I can optimize video broadcasting over internet.

I am using gstreamer to accomplish this using RTP:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480,framerate=30/1 ! queue ! videoconvert ! videoscale ! videorate ! video/x-raw,width=352,height=288,framerate=30/1 ! queue ! x264enc bitrate=1024 speed-preset=fast tune=zerolatency ! mpegtsmux ! rtpmp2tpay ! queue ! udpsink host=192.168.1.2 port=8093

On the server side, I am running:
/usr/bin/vlc -I dummy rtp://@:8093 :sout=#http{dst=:8094/vlctest.mpg}

I see I can use NVIDIA encoding plugin, can you suggest the best method?

Also wondering if I can adapt the following gst-launch command to use a USB based camera.

gst-launch-1.0 -e nvarguscamerasrc ! ‘video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1’ ! nvv4l2h264enc bitrate=8000000 insert-sps-pps=true ! rtph264pay mtu=1400 ! udpsink host=$CLIENT_IP port=5000 sync=false async=false

You may try using HW encoder:

gst-launch-1.0 -v videotestsrc ! video/x-raw, width=640, height=480, framerate=30/1 ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM), width=352, height=288' ! nvv4l2h264enc maxperf-enable=1 ! mpegtsmux ! rtpmp2tpay ! queue ! udpsink host=192.168.1.2 port=8093

I am not familiar with VLC, but it seems ok from Jetson itself (using HW decoder):

gst-launch-1.0 -ev udpsrc port=8093 ! application/x-rtp, media=video, encoding-name=MP2T, clock-rate=90000, payload=33 ! queue ! rtpmp2tdepay ! tsdemux ! video/x-h264, stream-format=byte-stream, width=352, height=288, framerate=30/1 ! h264parse ! nvv4l2decoder ! nvvidconv ! videoconvert ! xvimagesink

Thanks. It worked. Need to work on the Sever/Client to reduce the latency.

Any suggestions to improve the encoding/RTP upload?

Hi,
Each plugin in the pipeline may give certain latency. You may break down the pipeline and see which plugin gives much and if it is possible to have improvement.

Please execute sudo jetson_clocks to run in max performance. And in nvv4l2h264enc, please configure

  maxperf-enable      : Enable or Disable Max Performance mode
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false

DaneLLL

Thanks for the feedback. I started testing with RTSP using a Logitech USB camera

./test-launch --gst-debug=3 ‘( v4l2src device=/dev/video0 ! video/x-raw, width=640, height=480, framerate=30/1 ! queue ! nvvidconv ! ‘‘video/x-raw(memory:NVMM), width=352, height=288’’ ! nvv4l2h264enc maxperf-enable=1 ! rtph264pay name=pay0 pt=96 )’

when Client connects: I am getting this odd message:

0:08:08.600315751 25383 0x7f64003000 WARN v4l2bufferpool gstv4l2bufferpool.c:1480:gst_v4l2_buffer_pool_dqbuf:nvv4l2h264enc1:pool:sink V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:08:08.600342261 25383 0x7f64003000 WARN v4l2bufferpool gstv4l2bufferpool.c:1480:gst_v4l2_buffer_pool_dqbuf:nvv4l2h264enc1:pool:sink V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:08:08.736227313 25383 0x7f64003000 WARN v4l2bufferpool gstv4l2bufferpool.c:1480:gst_v4l2_buffer_pool_dqbuf:nvv4l2h264enc1:pool:sink V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:08:08.736276792 25383 0x7f64003000 WARN v4l2bufferpool gstv4l2bufferpool.c:1480:gst_v4l2_buffer_pool_dqbuf:nvv4l2h264enc1:pool:sink V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:08:08.736326324 25383 0x7f64003000 WARN v4l2bufferpool gstv4l2bufferpool.c:1480:gst_v4l2_buffer_pool_dqbuf:nvv4l2h264enc1:pool:sink V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0

Questions:

Any suggestions to improve the encoding:

./test-launch --gst-debug=3 ‘( v4l2src device=/dev/video0 ! video/x-raw, width=640, height=480, framerate=30/1 ! queue ! nvvidconv ! ‘‘video/x-raw(memory:NVMM), width=352, height=288’’ ! nvv4l2h264enc maxperf-enable=1 ! rtph264pay name=pay0 pt=96 )’

When I tried to “watch” the stream locally with VLC I got a bus error.

Please advise

Also I tried this pipeline:

sudo ./test-launch --gst-debug=3 ‘( v4l2src device=/dev/video0 ! image/jpeg,width=1280,height=720,type=video,framerate=30/1 ! jpegdec ! videoscale ! videoconvert ! x264enc tune=zerolatency ! rtph264pay name=pay0 pt=96 )’

Latency was reduced to 2 seconds using the following on the Receiver side:

gst-launch-1.0 rtspsrc location=rtsp://XX.XX.XX.XX:8554/test ! rtph264depay ! h264parse ! decodebin ! videoconvert ! xvimagesink sync=false.

Suggestions to improve latency?

You may try latency option of rtspsrc (in ms, default is 2000):

gst-launch-1.0 rtspsrc location=rtsp://XX.XX.XX.XX:8554/test latency=200 ! ...

@Honey_Patouceul

I tried to reduce the latency parameter but not much difference. Still between 1 to 2 seconds lag.

Just checking how I can find/measure the lag? and Do you think if I use a Camera on the NVIDIA camera port will improve results?

I am using a JETSON Nano and the CPU ussage is pretty much high.

Sorry if my advice has not been successful so far…be aware that I have no Nano nor MJPG USB camera for trying, so this is just general advice and pure speculation. Someone with right hw may better advise.
I think CPU usage is heavy because you are using CPU plugins such as jpegdec and x264enc. You would try to use HW JPG decoder/H264 encoder with :src -> image/jpeg, format=MJPG, framerate=30/1 -> nvv4l2decoder mjpeg=1 -> video/x-raw(memory:NVMM), format=I420, framerate=30/1 -> nvv4l2h264enc -> video/x-h264, stream-format=byte-stream, framerate=30/1 -> ...

The simple answer I can give here is using a terminal displaying time in a loop:

while true; do date +%s.%N; done

and have camera recording it.
Then have a rtsp client gstreamer pipeline displaying in the same display (or scene). Make a screen or scene shot and compare times. Be aware it has jitter and average several measurements. Not sure what are the flaws with this methodology. Some better skilled user may further comment.

Not sure again, but I think that MJPG cameras on Nano may have USB hub/bandwith issue.
So using a CSI camera might avoid this.
On my Xavier R32.4.2 I get about 400 ms latency on lo interface with R32.4.1 (using MAXN, and jetson_clocks doesn’t help much). I think I’ve been able to get better number with older L4T releases on TX2, but not sure again, I may be missing some of the tricks for recent releases.

@Honey_Patouceul

I tried:

/gst-rtsp-server/examples$ ./test-launch --gst-debug=3 ‘( v4l2src device=/dev/video0 io-mode=2 ! image/jpeg,width=640,height=480, framerate=30/1,format=MJPG ! nvv4l2decoder mjpeg=1 ! video/x-raw(memory:NVMM),format=I420, framerate=30/1 ! nvv4l2h264enc ! video/x-h264, stream-format=byte-stream, framerate=30/1 ! rtph264pay name=pay0 pt=96 )’
stream ready at rtsp://127.0.0.1:8554/test
Opening in BLOCKING MODE
0:00:02.516064322 8891 0x7fa4013cf0 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7fa407a670 Failed to determine interlace mode
0:00:02.516164426 8891 0x7fa4013cf0 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7fa407a670 Failed to determine interlace mode
0:00:02.516206301 8891 0x7fa4013cf0 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7fa407a670 Failed to determine interlace mode
0:00:02.516332238 8891 0x7fa4013cf0 WARN v4l2 gstv4l2object.c:4410:gst_v4l2_object_probe_caps:nvv4l2h264enc0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE
0:00:02.518642343 8891 0x7fa4013cf0 WARN v4l2 gstv4l2object.c:4410:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:02.518696718 8891 0x7fa4013cf0 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7fa4072bf0 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 277
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 277
0:00:02.528438488 8891 0x7f9c0308a0 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:v4l2src0:pool:src Uncertain or not enough buffers, enabling copy threshold
0:00:02.897154165 8891 0x7f9c0308a0 WARN v4l2 gstv4l2object.c:4410:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:02.897209217 8891 0x7f9c0308a0 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7fa4072bf0 Failed to determine interlace mode
0:00:02.897643592 8891 0x7f9c0308a0 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:nvv4l2decoder0:src could not send sticky events
0:00:02.897969165 8891 0x7f9c0308a0 WARN v4l2videodec gstv4l2videodec.c:1609:gst_v4l2_video_dec_decide_allocation: Duration invalid, not setting latency
0:00:02.899198436 8891 0x7f9c0308a0 WARN v4l2bufferpool gstv4l2bufferpool.c:1057:gst_v4l2_buffer_pool_start:nvv4l2decoder0:pool:src Uncertain or not enough buffers, enabling copy threshold
0:00:02.905086405 8891 0x7f9c031140 WARN v4l2bufferpool gstv4l2bufferpool.c:1535:gst_v4l2_buffer_pool_dqbuf:nvv4l2decoder0:pool:src Driver should never set v4l2_buffer.field to ANY
0:00:02.905841040 8891 0x7f9c0308a0 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: Internal data stream error.
0:00:02.905873384 8891 0x7f9c0308a0 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: streaming stopped, reason not-negotiated (-4)
0:00:02.906009790 8891 0x7fa4013cf0 WARN rtspmedia rtsp-media.c:2725:default_handle_message: 0x7fa409a2a0: got error Internal data stream error. (gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:media-pipeline/GstBin:bin0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4))
0:00:02.906063280 8891 0x5595fa3d90 WARN rtspmedia rtsp-media.c:2994:wait_preroll: failed to preroll pipeline
0:00:02.906090467 8891 0x5595fa3d90 WARN rtspmedia rtsp-media.c:3298:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:02.942085728 8891 0x5595fa3d90 ERROR rtspclient rtsp-client.c:1054:find_media: client 0x5596144110: can’t prepare media
0:00:02.942635572 8891 0x5595fa3d90 ERROR rtspclient rtsp-client.c:2910:handle_describe_request: client 0x5596144110: no media

Not really familiar with those errors, any clues?

What are available modes for your camera on /dev/video0 ?

v4l2-ctl -d0 --list-formats-ext

Here is the output

jptalledo@JP-Nano:~$ v4l2-ctl -d0 --list-formats-ext

ioctl: VIDIOC_ENUM_FMT

Index : 0

Type : Video Capture

Pixel Format: ‘YUYV’

Name : YUYV 4:2:2

Size: Discrete 640x480

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 160x90

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 160x120

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 176x144

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 320x180

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 320x240

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 352x288

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 432x240

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 640x360

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 800x448

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 800x600

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 864x480

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 960x720

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1024x576

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1280x720

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1600x896

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1920x1080

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 2304x1296

Interval: Discrete 0.500s (2.000 fps)

Size: Discrete 2304x1536

Interval: Discrete 0.500s (2.000 fps)

Index : 1

Type : Video Capture

Pixel Format: ‘H264’ (compressed)

Name : H.264

Size: Discrete 640x480

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 160x90

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 160x120

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 176x144

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 320x180

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 320x240

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 352x288

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 432x240

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 640x360

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 800x448

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 800x600

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 864x480

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 960x720

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1024x576

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1280x720

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1600x896

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1920x1080

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Index : 2

Type : Video Capture

Pixel Format: ‘MJPG’ (compressed)

Name : Motion-JPEG

Size: Discrete 640x480

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 160x90

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 160x120

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 176x144

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 320x180

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 320x240

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 352x288

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 432x240

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 640x360

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 800x448

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 800x600

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 864x480

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 960x720

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1024x576

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1280x720

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1600x896

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Size: Discrete 1920x1080

Interval: Discrete 0.033s (30.000 fps)

Interval: Discrete 0.042s (24.000 fps)

Interval: Discrete 0.050s (20.000 fps)

Interval: Discrete 0.067s (15.000 fps)

Interval: Discrete 0.100s (10.000 fps)

Interval: Discrete 0.133s (7.500 fps)

Interval: Discrete 0.200s (5.000 fps)

Seems your camera should be able to capture 640x480@30 fps in MJPG. Not sure what fails, but from the error message you may add interlace-mode=progressive and pixel-aspect-ratio=1/1 to the caps.
You may also add h264parse after nvv4l2h264enc.

You may also try capturing in YUYV mode:

 ./test-launch "v4l2src device=/dev/video0 io-mode=2 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! nvvidconv ! nvv4l2h264enc ! video/x-h264, stream-format=byte-stream, framerate=30/1 ! h264parse ! rtph264pay name=pay0 pt=96 "

Since your camera can provide H264, it may be worth to try if uvch264 gstreamer plugin works, as this would avoid to encode. I have no H264 camera for trying, but you may try something like:

 ./test-launch "uvch264 device=/dev/video0 ! video/x-h264, width=640, height=480, framerate=30/1 ! h264parse ! video/x-h264, stream-format=byte-stream, framerate=30/1 ! rtph264pay name=pay0 pt=96 "

I’d advise to first get capture working to display for checking, and try RTSP streaming when you are sure that capture works.

#MJPG
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! image/jpeg,width=640,height=480, framerate=30/1,format=MJPG ! nvv4l2decoder mjpeg=1 ! nvvidconv ! videoconvert ! xvimagesink

#YUYV422
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! videoconvert ! xvimagesink

#H264
gst-launch-1.0 uvch264 device=/dev/video0 ! video/x-h264, width=640, height=480, framerate=30/1 ! h264parse ! nvv4l2decoder ! nvvidconv ! videoconvert ! xvimagesink

It seems the YUYV mode provided the best solution at this point. Thanks for the suggestion.