Hi guys,
I’d like to develop a solution with AGX Orin for bin picking with cobots.
I was thinking to use ROS2 and a 3d camera for 6D pose estimation with some of the packages in Isaac ROS 3.0 Manipulator repositories.
It’s an industrial application, so reliability and the possibility to use cables that can withstand some rough handling would be nice.
I was looking to start with Zed stereo cameras but I’d like some opinion on which other cameras I may be wanting to start with, I would like to have as few issues as possible with setting this up. I also know of Intel, Orbbec and Leopard cameras. Any suggestions about compatibility with Jetson ecosystem and ease of setup?
Thank you!
PS: The obvious one I didn’t mention is the Intel Realsense series of cameras, I can’t understand if they are usable in an industrial environment.
Hi DaneLLL,
I’m already in contact with a couple of jetson ecosystem partners, I woud have liked to have some opinion from other developers since the range of possible choices is so vast.
For example, just staying in the RealSense product range, I would consider:
D435f : this one has a IR filter and it may help with reflections on shiny pieces, but is it ok for indoor use?
D405 : this is the most close-range oriented camera, if I want to mount it on the flange of a small size cobot it would be perfect, but it’s the only one not mentioned in the IntelRealSense repo and I don’t know if it may be harder to set up properly to work with Isaac ROS
So if anyone has some suggestion on which one could be better to start experimenting with some setup ease it would be great!
For industrial envorinments i would suggest considering GMSL camera models over USB, which will also support the Argus camera API for enhanced performance.
If you want to stay in RealSense Product range, an option would be the RealSense D455, but I don’t have any insight on driver support here. Leopard Imaging Hawk is the reference for Isaac ROS (e.g. Isaac Nova uses them), but you will only get drivers for AGX Orin Dev Kit and the 8 Camera deserializer out of the box. Stereolabs Zed X (Mini) could be a good alternative because of the Polarization Filter option, if reflections are a real problem to you. In addition, Stereolabs supports most Jetson Modules and more GMSL deserializer options right away. For ZED, you will also get lots of examples based on ZED SDK, for HAWK you will have to rely on Nvidia examples.
I also want to note that there are operational differences; for instance, RealSense utilizes Time of Flight (ToF) technology which may be limited by object reflectivity and distance, whereas Hawk and Zed X rely solely on image information.
I absolutely agree on GMSL for production, I woud be willing to use USB only to develop and test the solution first with one of the cameras that are already mentioned in the isaac_ros_pose_estimation documentation. This way I’d be able to focus on getting the solution working rather than adapting the pipeline to other cameras. But this would have sense only if the GMSL cameras have similar specs.
Another pro of the ZED cameras is that you can get from Stereolabs the whole package, including the Orin (AGX Dev or NX box) and all the required cables/adapters, which would ease my mind about compatibility issues and assembly of the test model.
The D4xx series of RealSense should all be stereo cameras like the Zed X and the HAWK, the ToF ones sould be RealSense L5xx series, if I’m not wrong.
Anyways I had some conflicting instructions from vendors about IR filter: some state that it’s only for outdoor use, some other that it’s great to use indoor with reflective objects. I suppose the only way is try two similar cameras with and without IR filter and find out which one work better!
For the HAWK camera, I’m a bit worried about the indication in the isaac_ros_pose_estimation documentation stating that “the Hawk pipeline may take more than 10s for pose estimation on some complicated objects”.
It would be a bit too long for my application.