Intel Realsense Camera still does not work with v4l2_camera issac application

According to ISAAC SDK doc., the Realsense camera should work with v4l2_camera application. Instead the bazel run got coredump.

Can someone let me know what kind of camera I should use in order to run this application?

ZED camera is also suggested by the doc. to be used for this application but it costs very high.

Thanks
Jimmy

Hi,
any UVC camera will work, use deepstrem to decode MJPEG or such compressions, I think you could make realsense also work but it will take more work as you need to make your own driver like nvidia did for zed.

Regards,
Planktos

@Planktos, thanks. I am going to buy a UVC camera to try…

I’m using this one it’s working great and it’s very cheap however it’s a rolling shutter and image quality is not incredible still very good quality/price ratio for a stereo camera :D

@ Planktos, I have ordered: Creative Live! Cam Sync 1080p since it is a UVC compliant camera. I will try this to see. If this one doesn’t work, then I will order your one.

BTW, does your camera works for the ISAAC ZED_camera application as well?

Thanks a lot

The zed_camera app is just a test to see zed camera output and is usless if you don’t use a zed camera, Any uvc camera will work with any app exept for the zed camera test (but some apps requires a stereo camera, a IMU or a lidar)
What are you planning to use Isaac for ?

You’r welcom

Hi Nathan,

Actually I just got a real UVC compliant camera. But when I run the v4l2_camera application, always get the coredump problem.

In my system, I have the RTX 2080 gaming laptop with Ubuntu 18.04 which has the ISAAC ADK/SIM installed. Also the Jetson AGX Xavier as the device.

Also I try to blacklist the uvcvideo to disable the webcam on the laptop. After that I still get the coredump by using the new UVC camera.

Actually I want to learn how to use the ISAAC to control the robot arm and robot navigation.

Any further suggestion how to make the v4l2_camera application work?

If not, I am planning to buy the same camera as your one.

One more question, when you use your camera, did you hookup with your host machine or device? Also if it is on host machine, does the host machine has the webcam also?

Thanks
Jimmy

Work with host and device

My don’t but you just need to specify the video source in your pipeline to select the camera you want.

Can you see your camera output with an app like VLC ? If so try using deepstream with a pipeline like this one :
"pipeline": "gst-launch-1.0 -v v4l2src device=/dev/video0 ! avdec_mjpeg ! videoconvert ! video/x-raw,format=RGB,height=480,framerate=30/1 ! appsink name = acquired_image "

If you want more details on how a pipeline is implemented take a look at one of my old post

Tell me if you have issues understanding some lines of my code or have more errors, you’r welcome :)

Hi Nathan,

  1. when I run : vlc v4l2:///dev/video0, the webcam on the laptop shows (works)
  2. when I run: gst-launch-1.0 -v v4l2src device=/dev/video0 ! videoconvert ! video/x-raw,format=RGB,height=480,framerate=30/1 ! appsink name = acquired_image (also need to remove ! avdec_mjpeg sine since it shows:
    no element "avdec_mjpeg"
    )
    I can see the webcam light is on except that there is camera window show.
  3. After I run: bazel run //apps/samples/v4l2_camera

ERROR packages/sensors/V4L2Camera.cpp@150: [V4L2Camera] Unsupported width, 800, driver changed to 640

Hi

your camera output 1280 x 720 @ 30fps frames so change 480 into 720, if your camera is MJPEG encoded this should work if not you will get a error like “wrong pixel encoding” meaning it’s not MJPEG
so try
gst-launch-1.0 -v v4l2src device=/dev/video0 ! avdec_mjpeg ! videoconvert ! video/x-raw,format=RGB,height=720,framerate=30/1 ! appsink name = acquired_image
If not working look in vlc the camera properties to get the encoding (or run : v4l2-ctl --list-formats-ext in terminal) you should get something like that : Codec MJPEC Resolution 2560x960 Decoded format Planar 4:2:2 YUV full scale
and replace avdec_mjpeg with the encoding corresponding to your camera (lists of all decoding supported in GStreamer somewhere on the internet=)

Also change the config parameters acordingly :

"v4l2_camera": {
  "driver": {
    "device_id": 0,
    "rows": 720,
    "cols": 1280,
    "rate_hz": 30
  }
},

wierd do you have “deepstream” included in your build and app ? maybe if you share you code I could spot why it’s show that

Hi Nathan,

Actually I have returned my Creative Live! camera.

Now per your suggestion, I have changed the config block with following setting.

Now when I run: bazel run //apps/samples/v4l2_camera, it works except that the frame rate 30 will need be changed into 10 since I got error. From the localhost:3000, I can the camera picture.

Thanks a lot for you suggestions
Jimmy

“config”: {
“camera”: {
“V4L2Camera”: {
“device_id”: 0,
“rows”: 720,
“cols”: 1280,
“rate_hz”: 10
}
},

I meant: From the localhost:3000, I can see the camera picture.

Good call, didn’t seem great, are you using your laptop camera then ?

Except if your camera run at 10fps max in vlc you should get at least 20 fps decoding 720p MJPEG with your configuration , can you share the error to see if we could get something done about the frame rate ?
Maybe try with 60fps in pipeline and config see if it works with your laptop camera ?

You are welcom =)