Send TX2 encoded stream over WiFi

Hi Folks,

I need to encode tow independent, 1080p@60 fps H.265 video from two Sony IMX274 CSI cameras, and send the stream over WiFi to a Android application.

I am thinking of encoding using gstreamer.

Has anyone attempted this before ? Please share pointers to ship encoded streams over WiFi/TCP stack. Furthermore, what is best way capture these streams on a app running on Android smart phone ?

I would like to preview both cameras on smart phone.

Thanks,

Please try RTSP
https://devtalk.nvidia.com/default/topic/1014789/jetson-tx1/-the-cpu-usage-cannot-down-use-cuda-decode-/post/5188538/#5188538
https://devtalk.nvidia.com/default/topic/1018689/jetson-tx2/vlc-playing-gstreamer-flow/post/5187270/#5187270
UDP
https://devtalk.nvidia.com/default/topic/1027423/jetson-tx2/gstreamer-issue-on-tx2/post/5225972/#5225972
or HLS
https://devtalk.nvidia.com/default/topic/992525/jetson-tx1/no-video-for-hls-on-ios/post/5084851/#5084851

Hi Folks

WOuld like to revive this.

Hi DaneLLL,

Upon, following your first link, I am stumbling across the point when I try use nvcamerasrc (instead of videotestsrc) in the gstreamer pipeline.

With following line in test-launch.c

gst_rtsp_media_factory_set_launch (factory,
      "( videotestsrc is-live=1 !  x264enc ! rtph264pay name=pay0 pt=96 )");

I am able to stream from server side and view the stream using vlc player on client side.

However with any of following lines I am getting error (on server side, upon clicking pay in vlcplayer)…

gst_rtsp_media_factory_set_launch (factory,
      "( nvcamerasrc sensor-id=1 fpsRange='30.0 30.0' ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' !   queue ! x264enc ! rtph264pay name=pay0 pt=96 )");
gst_rtsp_media_factory_set_launch (factory,
      "( nvcamerasrc sensor-id=1 fpsRange='30.0 30.0' ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' !   omxh264enc iframeinterval=24 bitrate=5000000 ! h264parse  ! matroskamux ! rtph264pay name=pay0 pt=96 )");

Error

stream ready at rtsp://127.0.0.1:8554/test

(test-hardwareOpt:17419): GStreamer-CRITICAL **: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed

Please help.

Thanks

You may try to remove quotes around caps.
These quotes are needed in shell with gst-launch, but it may be interpreted here as an uri.

Hi Honey_Patouceul

Thanks for quick help.

I tried following with seems to work - well almost.

gst_rtsp_media_factory_set_launch (factory,
      "(" "nvcamerasrc sensor-id=1 fpsRange=\"30.0 30.0\"   ! " "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1 ! "   "omxh265enc bitrate=20000000 !  rtph265pay name=pay0 pt=96 " ")");

However I am getting lots of warnings and the bitsream on client seem to switch between good looking video and very corrupted video, very 2 sec.

0:00:10.753038426 18833   0x7f7401f190 FIXME           videoencoder gstvideoencoder.c:606:gst_video_encoder_setcaps:<omxh265enc-omxh265enc0> GstVideoEncoder::reset() is deprecated
Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 8 
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 8 
0:00:10.773317026 18833   0x7f7401f190 WARN             omxvideoenc gstomxvideoenc.c:1860:gst_omx_video_enc_set_format:<omxh265enc-omxh265enc0> Error setting temporal_tradeoff 0 : Vendor specific error (0x00000001)
===== NVENC blits (mode: 1) into block linear surfaces =====
0:00:10.892565811 18833   0x7f580022d0 FIXME             rtph265pay gstrtph265pay.c:1023:gst_rtp_h265_pay_payload_nal:<pay0> Set RTP marker bit appropriately
0:00:10.892965939 18833       0x5454a0 FIXME              rtspmedia rtsp-media.c:3425:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:11.182725944 18833       0x5454a0 FIXME              rtspmedia rtsp-media.c:3425:gst_rtsp_media_suspend: suspend for dynamic pipelines needs fixing
0:00:11.182848504 18833       0x5454a0 WARN               rtspmedia rtsp-media.c:3451:gst_rtsp_media_suspend: media 0x7f7c0561c0 was not prepared
0:00:11.190596239 18833   0x7f580022d0 FIXME             rtph265pay gstrtph265pay.c:1023:gst_rtp_h265_pay_payload_nal:<pay0> Set RTP marker bit appropriately
0:00:11.190698959 18833   0x7f580022d0 FIXME             rtph265pay gstrtph265pay.c:1023:gst_rtp_h265_pay_payload_nal:<pay0> Set RTP marker bit appropriately
0:00:11.196726824 18833   0x7f580022d0 FIXME             rtph265pay gstrtph265pay.c:1023:gst_rtp_h265_pay_payload_nal:<pay0> Set RTP marker bit appropriately
0:00:11.200275555 18833   0x7f580022d0 FIXME             rtph265pay gstrtph265pay.c:1023:gst_rtp_h265_pay_payload_nal:<pay0> Set RTP marker bit appropriately
0:00:11.201836226 18833   0x7f580022d0 FIXME             rtph265pay gstrtph265pay.c:1023:gst_rtp_h265

Using test-launch, I have these pipelines working (with huge latency):

# H265
./test-launch "( nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1 ! omxh265enc bitrate=20000000 ! video/x-h265 ! rtph265pay name=pay0 pt=96 )"

# H264
./test-launch "( nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1 ! omxh264enc bitrate=20000000 ! video/x-h264, profile=baseline ! rtph264pay name=pay0 pt=96 )"

I’m vieing the stream in another shell with:

gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/test ! queue ! decodebin ! videoconvert ! xvimagesink

Only H265 seems to have this short lagging each 2s, no idea why.
I’m not familiar with the options of the encoders, someone else may advise for better parameters.

Hi HoneyP,
Do you also observe lagging if you run nvoverlaysink instead of ‘videoconvert ! xvimagesink’?

Hi DaneLLL,

Yes, it happens as well with nvoverlaysink and nveglglessink. Can you reproduce this ?

gst-launch-1.0  nvcamerasrc ! 'video/x-raw(memory:NVMM), width=1280, height=720, format=I420, framerate=30/1' ! omxh265enc ! h265parse ! omxh265dec ! nvoverlaysink

gst-launch-1.0  nvcamerasrc ! 'video/x-raw(memory:NVMM), width=1280, height=720, format=I420, framerate=30/1' ! omxh265enc ! h265parse ! omxh265dec ! nvvidconv ! nvegltransform ! nveglglessink

I think the problem is in H265 encoder, because when I play with omxh265dec a H265 video recorded without omxh265enc, there is no lagging.

With 640x480, the lagging is very short, but as resolution increase, the lagging is more noticeable.

I can also see this with omxh264enc:

gst-launch-1.0  nvcamerasrc ! 'video/x-raw(memory:NVMM), width=1280, height=720, format=I420, framerate=30/1' ! omxh264enc ! h264parse ! omxh264dec ! nvoverlaysink

Boosting clocks doesn’t solves this issue.

It turns out that increasing from default bitrate 4000000 to 50000000 makes it ok.

gst-launch-1.0  nvcamerasrc ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1' ! omxh265enc bitrate=50000000 ! h265parse ! omxh265dec ! nvoverlaysink

H264 bitrates
H265 bitrates

Hi HoneyP, Thanks for sharing the information.