# WebRTC - low performances with Nvidia encoder **URL:** \ **Category:** Jetson Nano\ **Tags:** nvbugs\ **Created:** [March 12, 2020, 2:11am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324 "2020-03-12T02:11:15Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![Tangfrere](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Tangfrere](https://forums.developer.nvidia.com/u/Tangfrere)\ **Post date:** [March 12, 2020, 2:11am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/1 "2020-03-12T02:11:15Z")
Hello me and my team are trying to port a C++ application from Intel UP2 to Nvidia Jetson Nano. Our app use WebRTC to send a video stream to another device. To do so, the video datastream have to be encoded in the H264 format. We use the Nvidia Encoder Factory class in the WebRTC library to compress the raw video stream from a camera into the H264 format. ``` Basically we have two issues: 1) The framerate is too low, I'm aiming for 30 fps in 1920x1080p resolution. On intel board I can have this performance, but on Nano it's less than 10 fps. Inside the WebRTC package provided by Nvidia there is a test application for it, called `video_loopback`, and I can see the framerate is not enough. Also by using different USB cameras I noticed a slightly better framerate sometimes but still not enough. 2) There are several parameters for the H264 Encoder class: `Profile`, `Level`, `RTP Packetization mode`. The combination of various parameters make a lot of possible configurations, but only one seems to be supported by the Nvidia encoder class in webRTC (packetization-mode = 0, profile-level-id = 42e01f). However many Android devices only support `packetization-mode=1`, it means I cannot establish a call between the Nano and an Android device, which is one of the key features of our application. I believe these 2 issues could be fixed by tweaking the encoder code in the Webrtc library, unfortunately I could not find the sources anywhere. I'd like to have access to these sources, or at least discuss with the person who developed it so I don't have to code the whole encoding logic. Here is also some technical questions which could help me and my team: - Which type of camera should we use for getting the best performances with the Nvidia Encoder (interface, model, capability)? - When capturing the video, does the Encoder need a specific video format as input ? (NV12, YUY2, MPEG, I420, …) - From Gstreamer I noticed very good results in terms of framerate, but I wonder why the WebRTC class is lower, did someone managed to transmit a 30fps stream with it? ``` ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [March 12, 2020, 6:11am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/2 "2020-03-12T06:11:56Z")
Hi, We have RTC package in [https://developer.nvidia.com/embedded/dlc/r32-3-1\_Release\_v1.0/t186ref\_release\_aarch64/WebRTC\_R32.3.1\_aarch64.tbz2](https://developer.nvidia.com/embedded/dlc/r32-3-1_Release_v1.0/t186ref_release_aarch64/WebRTC_R32.3.1_aarch64.tbz2) Not sure if you use this one. If not, please download it and floow README to give it a try. Also we support two software frameworks with hardware acceleration: gstreamer and tegra\_multimedia\_api. Please also take a look at the documents: [https://developer.nvidia.com/embedded/dlc/l4t-accelerated-gstreamer-guide-32-2](https://developer.nvidia.com/embedded/dlc/l4t-accelerated-gstreamer-guide-32-2) [https://docs.nvidia.com/jetson/archives/l4t-multimedia-archived/l4t-multimedia-3231/index.html](https://docs.nvidia.com/jetson/archives/l4t-multimedia-archived/l4t-multimedia-3231/index.html) ---
**Author:** ![Tangfrere](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Tangfrere](https://forums.developer.nvidia.com/u/Tangfrere)\ **Post date:** [March 12, 2020, 7:04am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/3 "2020-03-12T07:04:51Z")
> [@DaneLLL](#): > > Hi, > We have RTC package in > [https://developer.nvidia.com/embedded/dlc/r32-3-1\_Release\_v1.0/t186ref\_release\_aarch64/WebRTC\_R32.3.1\_aarch64.tbz2](https://developer.nvidia.com/embedded/dlc/r32-3-1_Release_v1.0/t186ref_release_aarch64/WebRTC_R32.3.1_aarch64.tbz2) > > Not sure if you use this one. If not, please download it and floow README to give it a try. > > Also we support two software frameworks with hardware acceleration: gstreamer and tegra\_multimedia\_api. Please also take a look at the documents: > [https://developer.nvidia.com/embedded/dlc/l4t-accelerated-gstreamer-guide-32-2](https://developer.nvidia.com/embedded/dlc/l4t-accelerated-gstreamer-guide-32-2) > [L4T Multimedia API Reference: Main Page](https://docs.nvidia.com/jetson/archives/l4t-multimedia-archived/l4t-multimedia-3231/index.html) Hi, Yes we use this exact WebRTC package, but it only contains the header files and the compiled library, not the sources of the Nvidia Encoder class which I’m looking for, my goal is to modify/improve it. About Gstreamer, I managed to have good result with it independently of webRTC, but that’s not what I need. And using the multimedia API, is indeed useful if I redo the entire class from scratch but accessing the sources of what is already done would be more time saving. ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [March 13, 2020, 5:47am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/4 "2020-03-13T05:47:01Z")
Hi, Please execute _sudo jetson\_clocks_ and check if there is improvement. If the result still does not meet your requirement, please share steps so that we can reproduce your observation and do further investigation. It is an enhancement to WebRTC package and may take some time in development. If you can move your usecase to leverage gstreamer or tegra\_multimedia\_api, it would be great. ---
**Author:** ![Tangfrere](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Tangfrere](https://forums.developer.nvidia.com/u/Tangfrere)\ **Post date:** [March 13, 2020, 7:04am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/5 "2020-03-13T07:04:14Z")
Running jetson\_clocks did not improve the framerate unfortunately. Well I’ll use the multimedia for my usecase and try to reach better results, thanks for the suggestions however. ---
**Author:** ![Tangfrere](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Tangfrere](https://forums.developer.nvidia.com/u/Tangfrere)\ **Post date:** [June 1, 2020, 7:03am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/6 "2020-06-01T07:03:29Z")
> [@DaneLLL](#): > > If the result still does not meet your requirement, please share steps so that we can reproduce your observation and do further investigation. Basically just run the ‘video\_loopback’ application contained in the Webrtc Nvidia API package, with an usb Camera on Jetson Nano, you should observe that it’s not even 20 fps. ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [June 1, 2020, 11:28pm UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/8 "2020-06-01T23:28:21Z")
Hi, We will try video\_loopback with E-Con CU135 on r32.4.2 and share the result. For information, please share which USB camera you use and the resolution. ---
**Author:** ![Tangfrere](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Tangfrere](https://forums.developer.nvidia.com/u/Tangfrere)\ **Post date:** [June 2, 2020, 1:13am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/9 "2020-06-02T01:13:44Z")
Thanks for checking, the model we use is this one: [http://www.webcamerausb.com/elp-high-speed-120fps-pcb-usb20-webcam-board-2-mega-pixels-1080p-ov2710-cmos-camera-module-with-21mm-lens-elpusbfhd01ml21-p-78.html](http://www.webcamerausb.com/elp-high-speed-120fps-pcb-usb20-webcam-board-2-mega-pixels-1080p-ov2710-cmos-camera-module-with-21mm-lens-elpusbfhd01ml21-p-78.html) ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [June 2, 2020, 2:58am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/10 "2020-06-02T02:58:45Z")
Hi, > [@Tangfrere](#): > > Basically just run the ‘video\_loopback’ application contained in the Webrtc Nvidia API package, with an usb Camera on Jetson Nano, you should observe that it’s not even 20 fps. Please share the resolution you run in this test. 1920x1080 or 1280x720? Thanks. ---
**Author:** ![Tangfrere](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Tangfrere](https://forums.developer.nvidia.com/u/Tangfrere)\ **Post date:** [June 3, 2020, 2:12am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/11 "2020-06-03T02:12:30Z")
1280x720 ---
**Author:** ![carolyuu](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/carolyuu/32/14043_2.png) [@carolyuu](https://forums.developer.nvidia.com/u/carolyuu)\ **Post date:** [June 3, 2020, 5:11am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/12 "2020-06-03T05:11:41Z")
Hi Tangfrere, How do you check the fps when running video\_loopback? We tried to run video\_loopback with README command, but can’t see the fps output. > $ ./video\_loopback --codec H264 --width 1280 --height 720 --capture\_device\_index 0 ---
**Author:** ![Tangfrere](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Tangfrere](https://forums.developer.nvidia.com/u/Tangfrere)\ **Post date:** [June 3, 2020, 5:21am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/13 "2020-06-03T05:21:51Z")
Yes there is nothing indicating fps, but visually I can clearly notice it’s not 30 fps, you can try to compare with a a camera on a desktop machine. ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [June 4, 2020, 12:50am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/15 "2020-06-04T00:50:51Z")
Hi, The low performance is due to a known limitation. Please check [development guide](https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fhardware_acceleration_in_webrtc.html%23wwpID0E0OB0HA). We will check to improve it by using _NvBufferTransform()_ to leverage hardware VIC engine. ---
**Author:** ![Tangfrere](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Tangfrere](https://forums.developer.nvidia.com/u/Tangfrere)\ **Post date:** [June 4, 2020, 7:25am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/17 "2020-06-04T07:25:24Z")
> [@DaneLLL](#): > > Hi, > The low performance is due to a known limitation. Please check [development guide](https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fhardware_acceleration_in_webrtc.html%23wwpID0E0OB0HA). I see, so the input video format is the bottleneck. > [@DaneLLL](#): > > We will check to improve it by using _NvBufferTransform()_ to leverage hardware VIC engine. Alright, thanks. ---
**Author:** ![carolyuu](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/carolyuu/32/14043_2.png) [@carolyuu](https://forums.developer.nvidia.com/u/carolyuu)\ **Post date:** [June 17, 2020, 5:33am UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/19 "2020-06-17T05:33:47Z")
Hi Tangfrere, List our test result with E-Con CU135 usb camera for you reference: > $ ./video\_loopback --width 1280 --height 720 --capture\_device\_index 10 --codec H264 --fps 30 --duration 10 > Opening in BLOCKING MODE > NvMMLiteOpen : Block : BlockType = 4 > ===== NVMEDIA: NVENC ===== > NvMMLiteBlockCreate : Block : BlockType = 4 > H264: Profile = 66, Level = 0 > Opening in BLOCKING MODE > NvMMLiteOpen : Block : BlockType = 4 > ===== NVMEDIA: NVENC ===== > NvMMLiteBlockCreate : Block : BlockType = 4 > H264: Profile = 66, Level = 0 > Opening in BLOCKING MODE > NvMMLiteOpen : Block : BlockType = 4 > ===== NVMEDIA: NVENC ===== > NvMMLiteBlockCreate : Block : BlockType = 4 > H264: Profile = 66, Level = 0 > Opening in BLOCKING MODE > NvMMLiteOpen : Block : BlockType = 4 > ===== NVMEDIA: NVENC ===== > NvMMLiteBlockCreate : Block : BlockType = 4 > H264: Profile = 66, Level = 0 > Opening in BLOCKING MODE > NvMMLiteOpen : Block : BlockType = 4 > ===== NVMEDIA: NVENC ===== > NvMMLiteBlockCreate : Block : BlockType = 4 > H264: Profile = 66, Level = 0 > Farewell, sweet Concorde! > RESULT psnr: video= {24.594748,2.2867353} dB > RESULT ssim: video= {0.92835946,0.016521056} score > RESULT sender\_time: video= {6.6736111,3.2624332} ms > RESULT receiver\_time: video= {16.104167,4.9563634} ms > RESULT network\_time: video= {4.9826389,0.66904055} ms > RESULT total\_delay\_incl\_network: video= {27.760417,5.0841764} ms > **RESULT time\_between\_rendered\_frames: video= {34.045296,6.7303492} ms** > **RESULT encode\_frame\_rate: video= {27.181818,8.4618737} fps** > RESULT encode\_time: video= {1,0} ms > RESULT media\_bitrate: video= {679319.09,251986.61} bps > RESULT fec\_bitrate: video= {0,0} bps > RESULT send\_bandwidth: video= {1374213.3,88051.633} bps > RESULT time\_between\_freezes: video= {9771,0} ms > RESULT pixels\_per\_frame: video= {523636.36,272894.44} px > RESULT min\_psnr: video= 16.250608 dB > RESULT decode\_time: video= {2.1818182,1.1922615} ms > RESULT dropped\_frames: video= 12 frames > RESULT cpu\_usage: video= 21.066234 % ---
**Author:** ![nadeemm](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nadeemm/32/367020_2.png) [@nadeemm](https://forums.developer.nvidia.com/u/nadeemm)\ **Post date:** [October 18, 2021, 5:32pm UTC](https://forums.developer.nvidia.com/t/webrtc-low-performances-with-nvidia-encoder/115324/20 "2021-10-18T17:32:54Z")
Closed on October 18, 2021, 5:32pm UTC