Change Framerate Before nvv4l2h264enc Encode

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
, ,
1

Hello Everyone,

I am taking 4K@30 FPS raw stream from Camera Sensor and Putting it into the shmsink.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 do-timestamp=1 ! video/x-raw(memory:NVMM), width=3840, height=2160, format=(string)NV12, framerate=30/1 ! nvvidconv ! video/x-raw, width=(int)3840, height=(int)2160, format=(string)NV12 ! shmsink socket-path=abc_raw_stream.nv12 shm-size=50000000 sync=TRUE

I am using the same shmsink for multiple application. One of the Application is too stream camera over RTMP. But before stream I need to process every frame. So I passing that opencv capture and writer API.

OpenCV Capture Part from Shmsink
shmsrc socket-path=abc_raw_stream.nv12 do-timestamp=true ! video/x-raw, width=3840, height=2160, format=(string)NV12, framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM), width=3840, height=2160, format=(string)NV12, framerate=30/1 ! nvvidconv ! video/x-raw, width=3840, height=2160, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink sync=true

OpenCV Writer Part for RTMP Stream
appsrc is-live=true ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw,format=RGBA ! nvvidconv ! nvv4l2h264enc profile=4 insert-sps-pps=true bitrate=1000000 ! video/x-h264 , stream-format=byte-stream ! h264parse ! flvmux ! rtmpsink location=rtmp_url/abcd

Now the Question is,
I want to take the 30fps Video from Camera Sensor. But Before Streaming it to Server using RTMP protocol, I want to reduce the Frame Rate either on Capture Side or Writer Side. For example 10fps. Is there a way by which this can be achieved ?

I am aware following till now,

  • nvvidconv doesn’t support the framerate conversion options
  • there are option like videorate and videoscale are there. But none of worked till now for me.
  • Where shall I focus or debug more to put videorate or videoscale option ? Either on Capture Part of OpenCV or Writer Part of OpenCV ?

Any suggestions or pointers are appreciated in Advance.

Thanks & Regards,
xeqtr8041

2

If opencv capture can provide this resolution @ 30 fps, it should be easy to read and process one frame, read then 2 other frames that are not processed, so your processing will be at 10 fps, just set this framerate to your videoWriter and send the processed frame.

Be aware that videorate and videoscale are CPU only (as as videoconvert that is a bottleneck with high resolution*framerate). Only use videoconvert with integer division of framerate, it may be very slow on Jeston otherwise.
So you may try this opencv videoCapture pipeline :

shmsrc socket-path=abc_raw_stream.nv12 do-timestamp=true ! video/x-raw, width=3840, height=2160, format=(string)NV12, framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM), width=3840, height=2160, format=(string)NV12, framerate=30/1 ! nvvidconv ! video/x-raw, width=3840, height=2160, format=(string)BGRx ! videorate ! video/x-raw, framerate=10/1 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink sync=true

You may try lower resolutions and framerate, see what works and try increasing.
Be sure to boost your NX with :

sudo nvpmodel -m 2
sudo jetson_clocks