Jetson AGX Xavier - most efficient way to capture a rbg or grayscale raw frame (C++)?

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hi All,
I need to save into a file a raw capture from jetson xavier camera (rgb or greyscale, not a compressed format like jpg).
I’m using C++, I first tryied wit v4lc multimedia APIs, trying to modifying multimedia APIs examples, but something seems to be wrong, probably in my v4lc camera driver, because the frame I save is always a “11111111…111” file. I faced the same problem using v4lc tool ( see below), so I gave up with this approach

nvidia@JetsonAgxXavier:~$ v4l2-ctl --device /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT

  • Index : 0*
  • Type : Video Capture*
  • Pixel Format: ‘BG10’*
  • Name : 10-bit Bayer BGBG/GRGR*
  •    Size: Discrete 2592x1944*

  •        Interval: Discrete 0.033s (30.000 fps)*

  •    Size: Discrete 2592x1458*

  •        Interval: Discrete 0.033s (30.000 fps)*

  •    Size: Discrete 1280x720*

  •        Interval: Discrete 0.008s (120.000 fps)*

nvidia@JetsonAgxXavier:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=BG10 --stream-mmap --stream-count=1 --set-ctrl bypass_mode=0 --stream-to=gb10Frame.raw

(bg10FRame.raw is a 111111111111…1111111111111" file…

Now I’m trying to use libARGUS, I found oneShot example and I can see It’s really easy to save a jpg frame into a file ( and the example works in my environment, the image saved is ok), but I have not understood if there is a simple way to save the raw RGB image ( or greyscale).
I don’t even understand if in this case It can be better use a EGLStream consumer or a CUDA consumer.

I can’t see a simple way to save the raw frame, but it seems strange to me, as it is so easy to save a compressed jpg image.

Thanks in advance,
Susanna

2

The bayer frames from V4L API would have to be debayered, so yes Argus can do that with ISP HW.
Rather than customizing Argus samples, you may use gstreamer and plugin nvarguscamerasrc that provides debayered frames in NV12 format into NVMM memory.
So you would also use nvvidconv plugin for color conversion and copying to CPU memory before saving to file.
Because nvvidconv doesn’t support BGR format (but supports BGRx or RGBA) for the color case we need to use videoconvert on CPU.
num-buffers property of nvarguscamerasrc plugin allows to capture a given number of frames. So basically:

# GRAY8
gst-launch-1.0 nvarguscamerasrc num-buffers=1 ! 'video/x-raw(memory:NVMM),width=1280,height=720' ! nvvidconv ! video/x-raw,format=GRAY8 ! filesink location=test_1280x720_gray8.raw

# BGR
gst-launch-1.0 nvarguscamerasrc num-buffers=1 ! 'video/x-raw(memory:NVMM),width=1280,height=720' ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! filesink location=test_1280x720_BGR.raw

Note that by default Argus will also auto-tune different parameters for improving quality (gain, wb, exposure…). So the first frames may not be as expected. You may get the 100th frame with:

gst-launch-1.0 nvarguscamerasrc num-buffers=100 ! 'video/x-raw(memory:NVMM),width=1280,height=720' ! nvvidconv ! video/x-raw,format=GRAY8 ! multifilesink location=test_1280x720_gray8.raw

gst-launch-1.0 nvarguscamerasrc num-buffers=100 ! 'video/x-raw(memory:NVMM),width=1280,height=720' ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! multifilesink location=test_1280x720_BGR.raw
1 Like
3

Hi Honey_Patoucel, first of all thank You so much for your prompt reply.
I have to make the capture and save the frame from a C++ application, so I can’t use nvarguscamerasrc from shell.

I think I misleaded you as I posted the result of a test I made using v4l2-ctl from shell, to double check the behaviour of v4lc C++ APIs, to be sure that the wrong result wasn’t depending on my code, but I have to save the frame using C++…

Any hint?
Anyway thank You a lot,
Regards,
Susanna

4

You can use a shell command from a C++ application using system() call, if this can fit your need. This would do the job of capturing and saving to file.

If you want to get more control on this, you may program gstreamer pipeline in C++ (or python).

If you want to do some processing, you may use OpenCV using gstreamer capture and gstreamer videoWriter encoding to file.

For jetson MM API, someone more skilled would better advise.

1 Like
5

Hi Honey_Patoucel, thank You again for the reply, I know about system() call,but it doesn’t work for my needs.
I’m looking for the most efficient solution to do this task from C++ code, It seems strange to me that APIs like LibArgus offer a great solution to obtain a jpeg file, but nothing to save the frame without compression, and I have not found an aswer in documentation and examples.
Still trying… thanks,
Susanna

7

hello susanna.fonti,

that’s correct, raw dump is not supported when going through ISP.
please also check Camera Architecture Stack for reference.

please use the quickest way to dump raw file by using v4l2 utility.
for example,
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=2592,height=1944,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.raw

8

Hi JerryChang, thank you for reply.
I already tried to use v4l2-ctl and v4lc C++ APIs, it was my first choice, but it didn’t work ( see my first post). I don’t know the reason, but all the tests I performed with v4lc failed, I always get a file with “1111111…1111” values ( even from shell, using v4lc-ctl or from C++ program, starting from the examples). I have noticed that I can save good images (in jpg format) with libArgus examples, so I’m sure the camera is working fine, but I still don’t know how to save the raw file…
Any hint, excluding v4lc?
Thanks in advance,
Susanna

9

hello susanna.fonti,

it happened to some sensors that 1st frame were abnormal, please try to increase the stream-counts to dump two or three more frames for analysis.
I usually using 3rdparty tools, such as 7yuv to view the raw data. it could also show the hex values of each pixel.
please have a try again,
finger-crossed.

10

Hi JerryChang, I tried with an higher stream-counts number and the second part of the file seems to contain data ( not all the bytes have the same value). I have to find a solution with C++ APIs, so now I’ll try to reproduce this test from my program, dropping the first frame.
finger-crossed, I hope to give you a positive update ASAP…
Thanks again,
Susanna

13

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

hello susanna.fonti,

do you have any further updates, are we able to close this thread?

15

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.