please help find the bottleneck of a TX1/TX2 video streaming system

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hi all,

I am working on a video streaming system.
Ideally, my system need stream data from 2 CMOS sensors to the TX1/TX2 via the CSI, do some color tuning on ISP, do some image processing like overlaying using GPU, and then stream out to display. Each sensor should output FullHD1080, 10bits/pixel RAW data at 60FPS simultaneously to Tegra. After some overlaying process on the GPU, Tegra streams out FullHD1080, RGB24 images at 60FPS to a monitor via HDMI.

Here are some questions I want to confirm before I go further.

  1. Is Nvidia TX1/TX2 able to handle the bandwidth if every resource is well utilized? (e.g. optimized program for GPU, efficient pipeline…)

  2. If the answer is NO for question 1, where is the bottleneck of the streaming path? Is there any way to avoid it.

  3. If the answer is YES for question 1, again, there must still be a relative “bottleneck” which I need pay more attention on. Where is it?

  4. Is there any other way to access the Nvidia TX1/TX2’s on board ISP? Now, for me, the only way to control the ISP is using Gstreamer with Nvidia’s ‘nvcamerasrc’. Since I will use my own cameras with a different sensor, I can not use this ‘nvcamerasrc’.

  5. If the answer is YES for question 4, how should we distribute different image processing tasks to ISP and GPU? ISP for color tuning and GPU for other image processing? I am not sure.

  6. What is the best tool or framework to develop the software so the code can be optimized best for GPU and the pipeline can be most efficient? Could any one please list the pros and cons of Gstreamer, Argus, Visionworks? Also please help compare using Opencv4Cuda with using Cuda C.

I appreciate if some one can help answer all or part of these questions, or provide some links, benchmarks …

Thanks in advance.

2

Hi coolkid
1/3. Yes, there’s no problem to handle your use case. You may consider if you need yourself post process or not, if otherwise there’s no bottleneck problem for your case.

  1. gstreamer/argus both can support any of customize sensor. Beside, you can download the L4T Documentation for the sensor programing.

  2. You can check the sample code for all of the use case. However for the color tuning you need look for the scaling partner’s help if you have concern about image quality.

  3. Check the L4T Documentation may get the information you need.

Below is the download center.