[ZED Camera] Saving images from ZED camera using SDK API is very slow

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

I am using Jetson nano with Stereolabs ZED Camera on our porject. I have a Python script using SDK Python API and I am reading images from ZED. I know from API documentation, that grabbing image takes about 1/FPS, thats ok. But problem is, when I obtain image and I save it to memory, which takes too long for me (10+ ms for VGA, 40+ ms for 720p, 90+ ms for 1080p). This effectively limits FPS for 720p to less than 25 FPS, not even considering any image processing. I tried to save it as numpy array, which is almost twice faster, but resulting files are too big, since it is uncompressed.

Goal is to save image data with timestamps to some memory, so they can be later used. Ideally with 50-60 FPS.

I am not expert on Jetson nor ZED SDK so I would appreciate any tips and reccomendation, how to deal with this problem. Thanks in advance!

Python code for measuring saving time:

#!/usr/bin/env python3

import pyzed.sl as sl
import time
import numpy as np

init = sl.InitParameters()
camera = sl.Camera()
if not camera.is_opened():
    print("Opening ZED Camera...")
status = camera.open(init)
if status != sl.ERROR_CODE.SUCCESS:
    print(repr(status))

mat = sl.Mat()

runtime = sl.RuntimeParameters()

init = sl.InitParameters()
init.camera_resolution = sl.RESOLUTION.RESOLUTION_VGA
#init.camera_resolution = sl.RESOLUTION.RESOLUTION_HD720
#init.camera_resolution = sl.RESOLUTION.RESOLUTION_HD1080
init.camera_linux_id = 0
init.camera_fps = 60  # The framerate is lowered to avoid any USB3 bandwidth issues

camera = sl.Camera()
cam_status = camera.open(init)

mat = sl.Mat()

while True:
    # Obtain camera image and get timestamps
    t1 = time.process_time()
    err = camera.grab(runtime)

    # If grabbing image successfull, save to buffer
    if err == sl.ERROR_CODE.SUCCESS:
        camera.retrieve_image(mat, sl.VIEW.VIEW_SIDE_BY_SIDE)
        img = mat.get_data()

        img_name = 'images/filename_{}.jpg'.format(time.time())

        # Saving as numpy array
        # t2 = time.process_time()
        # np.save(img_name, img)
        # print("np.save took {} ms".format((time.process_time()-t2)*1e3))
    
        t1 = time.process_time()
        img = sl.ERROR_CODE.ERROR_CODE_FAILURE
        countdown = 5
        while img != sl.ERROR_CODE.SUCCESS or countdown > 0:
            img = mat.write(img_name)
            countdown -= 1
            if img == sl.ERROR_CODE.SUCCESS:
                break

        print("Saving file took {} ms".format((time.process_time()-t1)*1e3))
2

Does boost the system performance by jetson_clocks help on this?

3

Hi,
SDK Python API is 3rdparty software implementation. Please run jetson_clocks to get max performance.
Also we suggest you try tegra_multimedia_api to use hardware components:
https://developer.nvidia.com/embedded/dlc/l4t-multimedia-api-reference-32-1

4

Thanks for the answers! I did boost the system performance by ‘sudo nvpmodel -m 0’ (I ran jetson_clocks with many errors on my nano), but it didn’t help at all. I’ve been able to speed up image writing by disabling computation of depth images in ZED SDK, but it didn’t solve the core of my problem.

I wonder, if the problem really is low memory write speed. If so, may using external SSD memory help? Or does Jetson TX2 write to memory (card) faster?

I also noticed interesting behavior of threads (module threading) in python scripts. As soon as I start to use them, even just for creating one more thread, it slows down significantly. Parent thread is sleeping and other thread does job, which takes much lower while not using threads.

5

The problem is that the Python SDK uses software (CPU based) compression.
The API is not supposed to be used to create a video using capture-and-compress for each image.
If you want to store a video, there is a separate API for creating re-playable video files in the ZED SDK. I know that this can use hardware video compression on the bigger Jetsons; I haven’t tried it on the Nano, so you’d have to look into that.

Separately, you may be better served by properly threading your application; have one thread just capture the images, and hand off to another thread that does the processing, as this will scale to a second CPU core.
Unfortunately, the Python interpreter has the GIL which means Python itself won’t scale across cores, so you will have to use the multiprocessing module to create a subprocess that does the processing.