Realsense Camera fails on Joystick and mapping application - RealSense error calling rs2_pipeline_start_with_config: Couldn't resolve requests

Hi, I’m testing the joystick application for Jetson Nano, and even though the control works, the Intel Realsense camera doesn’t seem to function. This is the log that I’m getting:

2020-07-21 13:56:23.033 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘websight/isaac.sight.AliceSight’ was not added to scheduler because no tick method is specified.
2020-07-21 13:56:23.131 WARN packages/realsense/RealsenseCamera.cpp@42: Realsense recommended firmware version is 05.11.15.00, currently using firmware version 05.12.05.00
2020-07-21 13:56:23.511 ERROR engine/alice/components/Codelet.cpp@229: Component ‘2d_kaya.kaya_hardware.camera/realsense’ of type ‘isaac::RealsenseCamera’ reported FAILURE:

RealSense error calling rs2_pipeline_start_with_config(pipe:0x7f88068d40, config:0x7f880e8d20): Couldn't resolve requests

2020-07-21 13:56:23.511 ERROR engine/alice/backend/event_manager.cpp@42: Stopping node ‘2d_kaya.kaya_hardware.camera’ because it reached status ‘FAILURE’

I have tried also on the mapping application, and is the same issue (web interface doesn’t show video). The camera works fine if I start the realsense-viewer manually.

Has anyone experienced this issue?

Hi @gerardom and thank you for reaching out to us.
Please make sure you’re using the RealSense D435 (not D435i) camera with firmware 5.11.15

Thanks Mikhail, I’m using the Kaya original BOM (including the RealSense 435), and have downrev to FW 5.11.15. The warning to use this specific version went away, but the error is still the same.

log:
2020-07-22 11:50:26.507 INFO packages/sight/WebsightServer.cpp@217: Please open Chrome Browser and navigate to http://:3000

2020-07-22 11:50:26.507 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘websight/isaac.sight.AliceSight’ was not added to scheduler because no tick method is specified.

2020-07-22 11:50:27.040 ERROR engine/alice/components/Codelet.cpp@229: Component ‘2d_kaya.kaya_hardware.camera/realsense’ of type ‘isaac::RealsenseCamera’ reported FAILURE:

RealSense error calling rs2_pipeline_start_with_config(pipe:0x7f6c06ef90, config:0x7f6c0e9690): Couldn’t resolve requests

2020-07-22 11:50:27.040 ERROR engine/alice/backend/event_manager.cpp@42: Stopping node ‘2d_kaya.kaya_hardware.camera’ because it reached status 'FAILURE’

2020-07-22 11:50:27.040 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘2d_kaya.kaya_hardware.camera/realsense’ was not added to scheduler because no tick method is specified.

Any other ideas?

Thank you for providing the details on this matter.
Does the apps/samples/realsense_camera work normally on the robot?
Try running sudo jetson_clocks and put Jetson Nano into maximum power mode with sudo nvpmodel -m 0 before starting an Isaac application.
Please share the JetPack version you’re running and the output of v4l2-ctl --list-devices

I’m running Jetpack 4.3, and the realsense-camera app from the sample directory has similar issues

kaya@Kaya:~ **sudo jetson_clocks** [sudo] password for kaya: kaya@Kaya:~ sudo nvpmodel -m 0
kaya@Kaya:~ cd deploy/media/realsense_camera-pkg/ kaya@Kaya:~/deploy/media/realsense_camera-pkg ./apps/samples/realsense_camera/realsense_camera
2020-08-05 16:24:01.467 INFO engine/alice/tools/websight.cpp@166: Loading websight…
2020-08-05 16:24:01.468 WARN engine/alice/backend/application_json_loader.cpp@174: This application does not have an explicit scheduler configuration. One will be autogenerated to the best of the system’s abilities if possible.
2020-08-05 16:24:01.469 INFO engine/alice/backend/redis_backend.cpp@40: Successfully connected to Redis server.

2020-08-05 16:24:01.469 WARN engine/alice/backend/backend.cpp@201: This application does not have an execution group configuration. One will be autogenerated to the best of the systems abilities if possible.
2020-08-05 16:24:01.469 WARN engine/gems/scheduler/scheduler.cpp@337: No default execution groups specified. Attempting to create scheduler configuration for 4 remaining cores. This may be non optimal for the system and application.
2020-08-05 16:24:01.469 INFO engine/gems/scheduler/scheduler.cpp@290: Scheduler execution groups are:
2020-08-05 16:24:01.469 INFO engine/gems/scheduler/scheduler.cpp@299: BlockerGroup: Cores = [3], Workers = No
2020-08-05 16:24:01.469 INFO engine/gems/scheduler/scheduler.cpp@299: WorkerGroup: Cores = [0, 1, 2], Workers = Yes
2020-08-05 16:24:01.508 INFO engine/alice/backend/modules.cpp@226: Loaded module ‘packages/realsense/librealsense_module.so’: Now has 45 components total
2020-08-05 16:24:01.527 INFO engine/alice/backend/modules.cpp@226: Loaded module ‘packages/rgbd_processing/librgbd_processing_module.so’: Now has 51 components total
2020-08-05 16:24:01.544 INFO engine/alice/backend/modules.cpp@226: Loaded module ‘packages/sight/libsight_module.so’: Now has 54 components total
2020-08-05 16:24:01.567 INFO engine/alice/backend/modules.cpp@226: Loaded module ‘packages/viewers/libviewers_module.so’: Now has 83 components total
2020-08-05 16:24:01.567 DEBUG engine/alice/application.cpp@348: Loaded 83 components: isaac::RealsenseCamera, isaac::alice::BufferAllocatorReport, isaac::alice::ChannelMonitor, isaac::alice::CheckJetsonPerformanceModel, isaac::alice::CheckOperatingSystem, isaac::alice::Config, isaac::alice::ConfigBridge, isaac::alice::ConfigLoader, isaac::alice::Failsafe, isaac::alice::FailsafeHeartbeat, isaac::alice::InteractiveMarkersBridge, isaac::alice::JsonToProto, isaac::alice::LifecycleReport, isaac::alice::MessageLedger, isaac::alice::MessagePassingReport, isaac::alice::NodeStatistics, isaac::alice::Pose, isaac::alice::Pose2Comparer, isaac::alice::PoseFromFile, isaac::alice::PoseInitializer, isaac::alice::PoseMessageInjector, isaac::alice::PoseToFile, isaac::alice::PoseToMessage, isaac::alice::PoseTree, isaac::alice::PoseTreeJsonBridge, isaac::alice::PoseTreeRelink, isaac::alice::ProtoToJson, isaac::alice::PyCodelet, isaac::alice::Random, isaac::alice::Recorder, isaac::alice::RecorderBridge, isaac::alice::Replay, isaac::alice::ReplayBridge, isaac::alice::Scheduling, isaac::alice::Sight, isaac::alice::SightChannelStatus, isaac::alice::Subgraph, isaac::alice::Subprocess, isaac::alice::TcpPublisher, isaac::alice::TcpSubscriber, isaac::alice::Throttle, isaac::alice::TimeOffset, isaac::alice::TimeSynchronizer, isaac::alice::UdpPublisher, isaac::alice::UdpSubscriber, isaac::map::Map, isaac::map::ObstacleAtlas, isaac::map::OccupancyGridMapLayer, isaac::map::PolygonMapLayer, isaac::map::WaypointMapLayer, isaac::navigation::DistanceMap, isaac::navigation::NavigationMap, isaac::navigation::RangeScanModelClassic, isaac::navigation::RangeScanModelFlatloc, isaac::rgbd_processing::DepthEdges, isaac::rgbd_processing::DepthImageFlattening, isaac::rgbd_processing::DepthImageToPointCloud, isaac::rgbd_processing::DepthNormals, isaac::rgbd_processing::DepthPoints, isaac::rgbd_processing::FreespaceFromDepth, isaac::sight::AliceSight, isaac::sight::SightWidget, isaac::sight::WebsightServer, isaac::viewers::BinaryMapViewer, isaac::viewers::ColorCameraViewer, isaac::viewers::DepthCameraViewer, isaac::viewers::Detections3Viewer, isaac::viewers::DetectionsViewer, isaac::viewers::FiducialsViewer, isaac::viewers::FlatscanViewer, isaac::viewers::GoalViewer, isaac::viewers::ImageKeypointViewer, isaac::viewers::LidarViewer, isaac::viewers::MosaicViewer, isaac::viewers::ObjectViewer, isaac::viewers::OccupancyMapViewer, isaac::viewers::PointCloudViewer, isaac::viewers::PoseTrailViewer, isaac::viewers::SegmentationCameraViewer, isaac::viewers::SegmentationViewer, isaac::viewers::SkeletonViewer, isaac::viewers::TensorViewer, isaac::viewers::TrajectoryListViewer,
2020-08-05 16:24:01.570 WARN engine/alice/application.cpp@164: The function Application::findComponentByName is deprecated. Please use getNodeComponentOrNull instead. Note that the new method requires a node name instead of a component name. (argument: ‘websight/isaac.sight.AliceSight’)
2020-08-05 16:24:01.570 INFO engine/alice/application.cpp@255: Starting application ‘realsense_camera’ (instance UUID: ‘bebf82d2-d772-11ea-aee2-5d24f609e236’) …
2020-08-05 16:24:01.570 DEBUG engine/gems/scheduler/execution_groups.cpp@476: Launching 0 pre-start job(s)
2020-08-05 16:24:01.570 DEBUG engine/gems/scheduler/execution_groups.cpp@485: Replaying 0 pre-start event(s)
2020-08-05 16:24:01.570 DEBUG engine/gems/scheduler/execution_groups.cpp@476: Launching 0 pre-start job(s)
2020-08-05 16:24:01.570 DEBUG engine/gems/scheduler/execution_groups.cpp@485: Replaying 0 pre-start event(s)
2020-08-05 16:24:01.570 INFO engine/alice/backend/asio_backend.cpp@33: Starting ASIO service
2020-08-05 16:24:01.572 INFO packages/sight/WebsightServer.cpp@216: Sight webserver is loaded
2020-08-05 16:24:01.572 INFO packages/sight/WebsightServer.cpp@217: Please open Chrome Browser and navigate to http://:3000
2020-08-05 16:24:01.572 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘websight/isaac.sight.AliceSight’ was not added to scheduler because no tick method is specified.
2020-08-05 16:24:01.573 WARN engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick paramater
2020-08-05 16:24:01.573 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘_check_operating_system/isaac.alice.CheckOperatingSystem’ was not added to scheduler because no tick method is specified.
2020-08-05 16:24:01.573 WARN engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick paramater
2020-08-05 16:24:01.573 WARN engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick paramater
2020-08-05 16:24:02.015 ERROR engine/alice/components/Codelet.cpp@229: Component ‘camera/realsense’ of type ‘isaac::RealsenseCamera’ reported FAILURE:

** RealSense error calling rs2_pipeline_start_with_config(pipe:0x7f640572f0, config:0x7f640d4950): Couldn’t resolve requests**

2020-08-05 16:24:02.015 ERROR engine/alice/backend/event_manager.cpp@42: Stopping node ‘camera’ because it reached status 'FAILURE’
2020-08-05 16:24:02.015 WARN engine/alice/backend/codelet_canister.cpp@225: Codelet ‘camera/realsense’ was not added to scheduler because no tick method is specified.
2020-08-05 16:24:11.570 INFO engine/alice/backend/allocator_backend.cpp@57: Optimized memory CPU allocator.
2020-08-05 16:24:11.570 INFO engine/alice/backend/allocator_backend.cpp@66: Optimized memory CUDA allocator.
2020-08-05 16:24:20.997 INFO packages/sight/WebsightServer.cpp@113: Server connected / 1
^C2020-08-05 16:25:03.648 INFO engine/alice/application.cpp@274: Stopping application ‘realsense_camera’ (instance UUID: ‘bebf82d2-d772-11ea-aee2-5d24f609e236’) …
2020-08-05 16:25:03.648 INFO engine/alice/backend/backend.cpp@134: Backend is shutting down…
2020-08-05 16:25:03.659 INFO packages/sight/WebsightServer.cpp@124: Disconnected
2020-08-05 16:25:03.659 INFO packages/sight/WebsightServer.cpp@450: =====================================================================================
2020-08-05 16:25:03.659 INFO packages/sight/WebsightServer.cpp@451: | Websight Statistics Report |
2020-08-05 16:25:03.659 INFO packages/sight/WebsightServer.cpp@452: =====================================================================================
2020-08-05 16:25:03.659 INFO packages/sight/WebsightServer.cpp@453: | Name | Bandwidth | Total | Count |
2020-08-05 16:25:03.659 INFO packages/sight/WebsightServer.cpp@454: -------------------------------------------------------------------------------------
2020-08-05 16:25:03.659 INFO packages/sight/WebsightServer.cpp@473: =====================================================================================
2020-08-05 16:25:03.669 INFO engine/alice/backend/asio_backend.cpp@49: Stopping ASIO service
2020-08-05 16:25:03.679 DEBUG engine/gems/scheduler/execution_groups.cpp@505: Stopping all threads for execution group BlockerGroup
2020-08-05 16:25:03.679 DEBUG engine/gems/scheduler/execution_groups.cpp@509: Stopping all threads DONE
2020-08-05 16:25:03.679 DEBUG engine/gems/scheduler/execution_groups.cpp@505: Stopping all threads for execution group WorkerGroup
2020-08-05 16:25:03.679 DEBUG engine/gems/scheduler/execution_groups.cpp@509: Stopping all threads DONE
2020-08-05 16:25:03.679 INFO engine/alice/backend/backend.cpp@149: Backend is shutting down… DONE
2020-08-05 16:25:03.679 INFO engine/alice/backend/backend.cpp@153: codelets:
|=====================================================================================================================|
| Job Statistics Report (regular) |
|=====================================================================================================================|
| Name | Count | Time (Median - 90% - Max) [ms] | Load (%) | Late (%) |
|---------------------------------------------------------------------------------------------------------------------|
| isaac.alice.BufferAllocatorReport | 21 | 11.10 | 16.55 | 24.13 | 0.3 % | 0.0 % |
| isaac.alice.ConfigBridge | 1 | 1.19 | 1.19 | 1.19 | 0.0 % | 0.0 % |
| InteractiveMarkersBridge | 1242 | 0.04 | 0.06 | 2.29 | 0.1 % | 0.0 % |
| isaac.alice.LifecycleReport | 249 | 0.02 | 0.03 | 9.21 | 0.0 % | 0.0 % |
| isaac.alice.MessagePassingReport | 21 | 28.60 | 45.33 | 76.96 | 1.0 % | 0.0 % |
| PoseTreeJsonBridge | 1242 | 0.15 | 0.18 | 1.61 | 0.2 % | 0.0 % |
| NodeStatistics | 63 | 3.17 | 3.67 | 4.91 | 0.2 % | 0.0 % |
| depth_to_pointcloud | 0 | 0.00 | 0.00 | -inf | 0.0 % | 0.0 % |
| viewer | 0 | 0.00 | 0.00 | -inf | 0.0 % | 0.0 % |
| color | 0 | 0.00 | 0.00 | -inf | 0.0 % | 0.0 % |
| depth | 0 | 0.00 | 0.00 | -inf | 0.0 % | 0.0 % |
| left_ir | 0 | 0.00 | 0.00 | -inf | 0.0 % | 0.0 % |
| right_ir | 0 | 0.00 | 0.00 | -inf | 0.0 % | 0.0 % |
| WebsightServer | 11745 | 5.23 | 5.32 | 60.31 | 98.2 % | 0.0 % |
|=====================================================================================================================|
kaya@Kaya:~/deploy/media/realsense_camera-pkg$

Got to work the v4l2-ctl command, and this is the output:

kaya@Kaya:~$ v4l2-ctl --list-devices
Intel® RealSense™ Depth Ca (usb-70090000.xusb-2.3):
/dev/video0
/dev/video1
/dev/video2

kaya@Kaya:~$

any other advice?

Thanks

Updated the Jetpack to 4.4 and still the same issue.