Jetson Xavier Saving Image Data at High BandWidth

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hello,

Important Components
BlackFly S camera → PCIe Startech pexusb3s44v → Jetson Xavier → 500GB NVME Storage
Requirements
Ideally images should be at least 960x720, and at 80FPS → with BayerRG8 about 55MB/s, with BGR8 about 165MB/s.

Problem
I am using the Jetson Xavier connected to a FLIR BlackFly S camera and trying to create videos from the camera captures. FLIR’s BlackFly S cameras use an SDK called Spinnaker in order to control various camera and image format control properties. My overall goal is to collect images from the camera and append them into a video that I can pass to NVIDIA’s DeepStream SDK (My final output must be in video format, does it matter where the conversion happens?). Before I started this process I was planning on using almost the maximum USB3 bandwidth ~400MB/s, however, it seems that the Xavier is too slow at the process that I am trying to implement. I’m curious if there are any tools with the NVIDIA SDK Manager that could help me optimize this process on the Xavier.

Possible Solution 1
The way the camera works is you can access an image in the buffer using a Spinnaker command “GetNextImage”, then, you can save the image directly to disk as .jpg, .png, or .raw for example. Saving images to disk as .raw allows me to capture at high frame rate and resolution (above my requirements), however, this leads to a large amount of data written to disk. Would it be possible to use some tool to quickly grab these images as they are written to disk, create a video file, and then delete the images? Is there any software tool with Xavier that I can use to do this?

Possible Solution 2
Instead of saving images to the disk, I have also tried writing the video file in my loop. However, it seems like the process of appending the image to the video file takes too long, and frames end up being dropped. I am using OpenCV’s VideoWriter and converting the output of “GetNextimage” into an OpenCV Mat. Could this process be optimized anyway with Xavier? Or is solution 1 a better approach?

I’m pretty new to coding and overall I’m just curious if there is a way to optimize this process using the Jetson Xavier.

Thanks for the help,

Connor

3

Hi,
In Deepstream SDK, we have existing implementation listed in document

General usecases are RTSP source and nvarguscamerasrc. From your comment, if the source supports capturing frames in BGR format through v4l2 interface, you can set type=1 and give it a try. However, may need to try and see if it can achieve 960x720 80 fps.

source30_1080p_dec_infer-resnet_tiled_display_int8.txt is the config file we have verified for Xavier. It is multiple sources in 30fps. Yours is single source at high frame rate. There is slight difference. Need a real run to see whether it works or not.