Jetson-utils skipping out frames when using multithreading/multiprocessing - CPU power too low?

Hi,

I am trying to make a recording script using jetson-utils, reading from the camera sensor with a VideoSource object and writing out to a file with a VideoOutput object. The video input resolution is 3896x2192.

I first managed doing that quite simply with a single thread using the following code:

try:
    while True:
        frame = self.video_source.Capture()
        self.video_output.Render(frame)
except KeyboardInterrupt:
    pass

However, I would like to be able to split each file in chunks and I don’t want to miss any of the input data while the file is being written out. So I tried out multiprocessing and multithreading, having one thread with an infinite loop that reads frames and places them in a queue, and another thread that reads those frames from the queue and writes them to the output. I can provide some code if needed.

My problem is that my output videos are skipping out a lot of frames at random points and since the writer thread reads all the frames to the queue, my guess is that the VideoSource object doesn’t get all the frames from the sensor but I am not sure why that would happen. Could it be that writing/reading to/from queues is too resource hungry for the jetson nano or what else could be causing this?

Any help would be appreciated. Thank you!

Hi @mattstyl, the frames coming from videoSource.Capture() are in a ringbuffer, so these get overwritten with the latest frame from the camera after the ringbuffer loops around. They aren’t intended to be stored for an indefinite period of time.

You can try increasing the number of buffers here and see if it improves the behavior: https://github.com/dusty-nv/jetson-utils/blob/ebab1914877a51d4d33fa9b1f01b168adb712a32/video/videoOptions.cpp#L36

After changing that, re-run make && sudo make install from your build directory.

I haven’t tried this at 4K resolution, so YMMV. Also, are you writing to a compressed H264/H265-encoded video file? If so, that is already multi-threaded underneath in the videoOutput interface, so I’m not sure further threading on the Python side would benefit it.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.