Realsense D435, Jetpack 4.6 issues with realsense-viewer and pointcloud

[weber.patrick]

Hello,

I’m trying to get a Intel Realsense D435 camera working with Jetson Nano B01 for some days now and still have not much luck. I tried different mentioned installation types, build from source, using jetsonhacks script, official instructions from intel and so on.

I’m running on Jetpack 4.6, librealsense is 2.50 as well as latest realsense-ros (which states to be operational with librealsense v.2.50).

When I start realsense-viewer, I can see stereo depth information as well as rgb camera. It seems to be fine but there are two issues:

1. performance is incredible poor. This might because it gets detected as USB2.1 device but its attached to USB3.0 (it’s a B01 Nano). I purchased a new cable to hopefully fix this.
2. which is not related to cable are all these error messages I get like lots of frame drops and even “out of frame resources” messages
   
   

I tried further with ROS and things getting even worse.
First of all, during launch of ros nodes, I receive lots of error messages (see below)
When I start RVIZ, I can visualize camera image and /camera/depth/image_rect_raw but there is no pointcloud.
I checked with rostopic echo and it is a registered topic but simply there are no messages.

I know there are lots of related issues and I tried to follow them but it didn’t work at all.

[ INFO] [1643659545.852162712]: Initializing nodelet with 4 worker threads.
[ INFO] [1643659546.184069188]: RealSense ROS v2.3.2
[ INFO] [1643659546.184165805]: Built with LibRealSense v2.50.0
[ INFO] [1643659546.184271068]: Running with LibRealSense v2.50.0
[ INFO] [1643659546.256456755]:  
[ INFO] [1643659546.455477783]: Device with serial number 939622072486 was found.

[ INFO] [1643659546.455685237]: Device with physical ID 1-2.3-7 was found.
[ INFO] [1643659546.455802011]: Device with name Intel RealSense D435 was found.
[ INFO] [1643659546.457359030]: Device with port number 1-2.3 was found.
[ INFO] [1643659546.457730290]: Device USB type: 2.1
[ WARN] [1643659546.457938316]: Device 939622072486 is connected using a 2.1 port. Reduced performance is expected.
[ INFO] [1643659546.458089362]: Resetting device...
[ INFO] [1643659552.584323880]:  
[ INFO] [1643659552.773468978]: Device with serial number 939622072486 was found.

[ INFO] [1643659552.773594971]: Device with physical ID 1-2.3-8 was found.
[ INFO] [1643659552.773659816]: Device with name Intel RealSense D435 was found.
[ INFO] [1643659552.774807451]: Device with port number 1-2.3 was found.
[ INFO] [1643659552.774888286]: Device USB type: 2.1
[ WARN] [1643659552.774946673]: Device 939622072486 is connected using a 2.1 port. Reduced performance is expected.
[ INFO] [1643659552.780889849]: getParameters...
[ INFO] [1643659553.103543539]: setupDevice...
[ INFO] [1643659553.103669324]: JSON file is not provided
[ INFO] [1643659553.103736878]: ROS Node Namespace: camera
[ INFO] [1643659553.103938914]: Device Name: Intel RealSense D435
[ INFO] [1643659553.103988239]: Device Serial No: 939622072486
[ INFO] [1643659553.104038448]: Device physical port: 1-2.3-8
[ INFO] [1643659553.104166577]: Device FW version: 05.13.00.50
[ INFO] [1643659553.104213765]: Device Product ID: 0x0B07
[ INFO] [1643659553.104262256]: Enable PointCloud: On
[ INFO] [1643659553.104303872]: Align Depth: Off
[ INFO] [1643659553.104352102]: Sync Mode: On
[ INFO] [1643659553.104480699]: Device Sensors: 
[ INFO] [1643659553.203845900]: Stereo Module was found.
[ INFO] [1643659553.261978513]: RGB Camera was found.
[ INFO] [1643659553.262179924]: (Confidence, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1643659553.262361179]: Add Filter: pointcloud
[ INFO] [1643659553.264224717]: num_filters: 1
[ INFO] [1643659553.264370919]: Setting Dynamic reconfig parameters.
[ INFO] [1643659558.398382767]: Done Setting Dynamic reconfig parameters.
[ INFO] [1643659558.399333000]: depth stream is enabled - width: 640, height: 480, fps: 15, Format: Z16
[ INFO] [1643659558.400273024]: color stream is enabled - width: 640, height: 480, fps: 15, Format: RGB8
[ INFO] [1643659558.400361725]: setupPublishers...
[ INFO] [1643659558.407450764]: Expected frequency for depth = 15.00000
[ INFO] [1643659558.422272298]: Expected frequency for color = 15.00000
[ INFO] [1643659558.434075939]: setupStreams...
 31/01 21:05:58,731 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[ INFO] [1643659558.787591850]: SELECTED BASE:Depth, 0
[ INFO] [1643659558.840737250]: RealSense Node Is Up!
[ WARN] [1643659558.978464080]: 
 31/01 21:05:58,980 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,030 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,085 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,135 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,186 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,243 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,293 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,350 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,401 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,452 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,502 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,553 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,609 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,659 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,710 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,760 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,811 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,861 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,912 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:05:59,963 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:00,013 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:00,062 ERROR [546501013888] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 130
 31/01 21:06:00,072 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:00,124 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:00,175 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:00,225 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:00,282 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:00,337 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[ WARN] [1643659560.399582391]: Hardware Notification:Depth stream start failure,1.64366e+12,Error,Hardware Error
 31/01 21:06:01,202 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:01,264 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:01,314 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:01,365 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:01,517 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:01,568 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:01,728 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:01,782 WARNING [547103707520] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 31/01 21:06:02,628 ERROR [546526192000] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 132

EDIT: To verify there is no issue with the camera or firmware itself, I tried realsense-viewer on my Windows machine. Here it works without any issue, no frame drops and overall performance is way better it is also connected to USB2.

What I observed also is that Jetson Nano doesn’t use GPU but CPU only. Shouldn’t it use GPU for processing?

[Andrey1984]

Could you extend what is the issue exactly?
You did build the SDK without cuda support enabled?
Messages seems warnings? Or It halts the system? Or it continues to work, just showing the warnings? In the latter case you could just ignore .

[weber.patrick]

It was a long journey to get ROS wrapper running on jetson. At the end, I had success by building librealsense 2.43.0 from source with CUDA support and use ros-wrapper 2.23.0 with firmware 5.12.12.100.
I tried other versions as well without any success.

However, even with CUDA support enabled (I can verify GPU activity in jtop when camera is active), the overall performance is poor, Still only 1-2 fps which isn’t enougth to do anything with it. It doesn’nt matter if I visualize Pointcloud in RVIZ locally or on a different PC. Realsense- Viewer performance is also poor on Jetson Nano.

Right now, the combination of Jetson Nano and D435 is useless in my setup and I hope someone can help to figure things out.

[Andrey1984]

@weber.patrick you may like to check JetScan - Hackster.io
folks were able to benefit from use of nano + d435