Unable to register shared memory region for input_1 ....., Running the Gaze Estimation Sample

Hi
I am following guides to use Gaze estimation. I started using this quick_start TLT CV Inference Pipeline Quick Start Scripts ā€” Transfer Learning Toolkit 3.0 documentation .

However when I start the client: ./tlt_cv_start_client.sh and I execute :./samples/tlt_cv/demo_gaze/gaze samples/tlt_cv/demo_gaze/demo.conf I get the follow error:

root@ml01:/workspace/tlt_cv-pkg# ./samples/tlt_cv/demo_gaze/gaze samples/tlt_cv/demo_gaze/demo.conf
[INFO] Skipping line with key '# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.'
[INFO] Skipping line with key '#'
[INFO] Skipping line with key '# NVIDIA CORPORATION and its licensors retain all intellectual property'
[INFO] Skipping line with key '# and proprietary rights in and to this software, related documentation'
[INFO] Skipping line with key '# and any modifications thereto. Any use, reproduction, disclosure or'
[INFO] Skipping line with key '# distribution of this software and related documentation without an express'
[INFO] Skipping line with key '# license agreement from NVIDIA CORPORATION is strictly prohibited.'
[INFO] Skipping line with key '#'
[INFO] Skipping line with key '## Do not use spaces in this file'
[INFO] Skipping line with key '# Path to device handle; ensure config.sh can see this handle'
[INFO] Skipping line with key '# Boolean indicating whether handle is pre-recorded'
[INFO] Skipping line with key '# Desired resolution (width, height, channels)'
[INFO] Skipping line with key '# Show visualization'
[INFO] Skipping line with key '# Leverage API to send single decoded frames to Pipeline'
[INFO] Skipping line with key '# If gaze_vec_length !'
[INFO] Skipping line with key '# Camera calib for default webcam'
[INFO] Skipping line with key '# Row major camera matrix'
[INFO]: Pipeline create direct
2021-05-28 09:46:01.567 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/application_json_loader.cpp@174: This application does not have an explicit scheduler configuration. One will be autogenerated to the best of the system's abilities if possible.
2021-05-28 09:46:01.567 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/application_json_loader.cpp@344: Modules requested by subgraph 'facedetect': //modules/generic:inferencer, //modules/vision/facedetect:preprocessor_module, //modules/vision/facedetect:postprocessor_module, //modules/vision/facedetect:redactor_module, //modules/opencv:bbox_renderer, 
2021-05-28 09:46:01.568 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/application_json_loader.cpp@344: Modules requested by subgraph 'faciallandmarks': (//modules/generic:inferencer), //modules/vision/faciallandmarks:preprocessor_module, //modules/vision/faciallandmarks:postprocessor_module, //modules/opencv:landmarks_renderer, 
2021-05-28 09:46:01.568 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/application_json_loader.cpp@344: Modules requested by subgraph 'gaze': (//modules/generic:inferencer), //modules/vision/gaze:preprocessor_module, //modules/vision/gaze:payload_producer_module, 
2021-05-28 09:46:01.569 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/redis_backend.cpp@33: 
**
**  Could not connect to redis server: Connection refused.
**
**  Metadata will not be available for this run.
**
**  You can start a redis server by running: external\redis\redis-server
**
2021-05-28 09:46:01.569 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/backend.cpp@200: This application does not have an execution group configuration. One will be autogenerated to the best of the systems abilities if possible.
2021-05-28 09:46:01.569 WARN  external/com_nvidia_isaac_engine/engine/gems/scheduler/scheduler.cpp@341: No default execution groups specified. Attempting to create scheduler configuration for 32 remaining cores. This may be non optimal for the system and application.
2021-05-28 09:46:01.569 INFO  external/com_nvidia_isaac_engine/engine/gems/scheduler/scheduler.cpp@291: Scheduler execution groups are:
2021-05-28 09:46:01.569 INFO  external/com_nvidia_isaac_engine/engine/gems/scheduler/scheduler.cpp@300: __BlockerGroup__: Cores = [8, 9, 10, 11, 12, 13, 14, 15], Workers = No
2021-05-28 09:46:01.569 INFO  external/com_nvidia_isaac_engine/engine/gems/scheduler/scheduler.cpp@300: __WorkerGroup__: Cores = [31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 5, 4, 3, 2, 0, 1, 6, 7], Workers = Yes
2021-05-28 09:46:01.580 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/generic/libinferencer_module.so': Now has 28 components total
2021-05-28 09:46:01.583 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/opencv/libbbox_renderer_module.so': Now has 29 components total
2021-05-28 09:46:01.586 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/opencv/libcamera_module.so': Now has 30 components total
2021-05-28 09:46:01.588 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/opencv/libencoder_module.so': Now has 31 components total
2021-05-28 09:46:01.591 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/opencv/liblandmarks_renderer_module.so': Now has 32 components total
[Jarvis] [E] Failed to open dynamic library /workspace/tlt_cv-pkg/_solib_k8/_U_S_Sprivate_Stypereg_C_Utypereg_Ulink___Uprivate_Stypereg/typereg_holder.so
2021-05-28 09:46:01.595 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/opencv/libpayload_producer_frame_module.so': Now has 33 components total
2021-05-28 09:46:01.598 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/vision/facedetect/libpostprocessor_module_module.so': Now has 34 components total
2021-05-28 09:46:01.603 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/vision/facedetect/libpreprocessor_module_module.so': Now has 35 components total
2021-05-28 09:46:01.607 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/vision/facedetect/libredactor_module_module.so': Now has 36 components total
2021-05-28 09:46:01.609 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/vision/faciallandmarks/libpostprocessor_module_module.so': Now has 37 components total
2021-05-28 09:46:01.613 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/vision/faciallandmarks/libpreprocessor_module_module.so': Now has 38 components total
2021-05-28 09:46:01.615 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/vision/gaze/libpayload_producer_module_module.so': Now has 39 components total
2021-05-28 09:46:01.619 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/vision/gaze/libpreprocessor_module_module.so': Now has 40 components total
2021-05-28 09:46:01.623 INFO  external/com_nvidia_isaac_engine/engine/alice/backend/modules.cpp@271: Loaded module 'modules/vision/payload_consumer_frame/libpayload_consumer_frame_module.so': Now has 41 components total
2021-05-28 09:46:01.623 DEBUG external/com_nvidia_isaac_engine/engine/alice/application.cpp@318: Loaded 41 components: isaac::alice::BufferAllocatorReport, isaac::alice::ChannelMonitor, isaac::alice::CheckJetsonPerformanceModel, isaac::alice::CheckOperatingSystem, isaac::alice::Config, isaac::alice::ConfigBridge, isaac::alice::ConfigLoader, isaac::alice::Failsafe, isaac::alice::FailsafeHeartbeat, isaac::alice::InteractiveMarkersBridge, isaac::alice::LifecycleReport, isaac::alice::MessageLedger, isaac::alice::MessagePassingReport, isaac::alice::NodeStatistics, isaac::alice::Pose, isaac::alice::PoseInitializer, isaac::alice::PoseTree, isaac::alice::PyCodelet, isaac::alice::Random, isaac::alice::Scheduling, isaac::alice::Sight, isaac::alice::SightChannelStatus, isaac::alice::Subgraph, isaac::alice::Subprocess, isaac::alice::Throttle, isaac::alice::TimeOffset, isaac::alice::TimeSynchronizer, nvidia::jrt::InferencerModule, nvidia::jrt::cvext::BBoxRenderer, nvidia::jrt::cvext::Camera, nvidia::jrt::cvext::Encoder, nvidia::jrt::cvext::LandmarksRenderer, nvidia::jrt::cvext::PayloadProducerFrame, nvidia::jrt::vision::PayloadConsumerFrame, nvidia::jrt::vision::Redactor, nvidia::jrt::vision::facedetect::Postprocessor, nvidia::jrt::vision::facedetect::Preprocessor, nvidia::jrt::vision::faciallandmarks::Postprocessor, nvidia::jrt::vision::faciallandmarks::Preprocessor, nvidia::jrt::vision::gaze::PayloadProducer, nvidia::jrt::vision::gaze::Preprocessor, 
2021-05-28 09:46:01.624 INFO  external/com_nvidia_isaac_engine/engine/alice/application.cpp@292: Starting application 'jarvis_vision_looking_at' (instance UUID: '831f283a-bf99-11eb-a157-6dc1f13a283a') ...
2021-05-28 09:46:01.624 DEBUG external/com_nvidia_isaac_engine/engine/gems/scheduler/execution_groups.cpp@476: Launching 0 pre-start job(s)
2021-05-28 09:46:01.624 DEBUG external/com_nvidia_isaac_engine/engine/gems/scheduler/execution_groups.cpp@485: Replaying 0 pre-start event(s)
[INFO] Logging with CSV Format
[INFO] timestamp, number of faces found
2021-05-28 09:46:01.637 DEBUG external/com_nvidia_isaac_engine/engine/gems/scheduler/execution_groups.cpp@476: Launching 0 pre-start job(s)
2021-05-28 09:46:01.637 DEBUG external/com_nvidia_isaac_engine/engine/gems/scheduler/execution_groups.cpp@485: Replaying 0 pre-start event(s)
2021-05-28 09:46:01.637 WARN  external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
2021-05-28 09:46:01.646 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@225: Codelet '_check_operating_system/isaac.alice.CheckOperatingSystem' was not added to scheduler because no tick method is specified.
2021-05-28 09:46:01.646 WARN  external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
2021-05-28 09:46:01.646 WARN  external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
[Jarvis] [E] operator() 137: unable to register shared memory region for input_1: TRTIS Error 'Unable to open shared memory region: '/832779a4-bf99-11eb-a157-6dc1f13a283a:input_1''
terminate called after throwing an instance of 'std::runtime_error*'
====================================================================================================
|                            Isaac application terminated unexpectedly                             |
====================================================================================================
#01 ./samples/tlt_cv/demo_gaze/gaze(+0xa2b2a) [0x56444f545b2a]
#02 google_breakpad::ExceptionHandler::GenerateDump(google_breakpad::ExceptionHandler::CrashContext*) /workspace/tlt_cv-pkg/samples/tlt_cv/demo_gaze/../../../_solib_k8/_U_S_Sprivate_Sapi_C_Ujrt_Ushared_Ulink___Uprivate_Sapi/libjrt.so(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3f0) [0x7f6d7c92d790]
#03 google_breakpad::ExceptionHandler::SignalHandler(int, siginfo_t*, void*) /workspace/tlt_cv-pkg/samples/tlt_cv/demo_gaze/../../../_solib_k8/_U_S_Sprivate_Sapi_C_Ujrt_Ushared_Ulink___Uprivate_Sapi/libjrt.so(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xc0) [0x7f6d7c92db00]
#04 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f6d7a1ca980]
#05 /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7) [0x7f6d79864fb7]
#06 /lib/x86_64-linux-gnu/libc.so.6(abort+0x141) [0x7f6d79866921]
#07 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8c957) [0x7f6d79ebb957]
#08 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92ae6) [0x7f6d79ec1ae6]
#09 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92b21) [0x7f6d79ec1b21]
#10 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92d54) [0x7f6d79ec1d54]
#11 nvidia::jrt::TritonShmInferencer::TritonShmInferencer(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, bool, bool, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /workspace/tlt_cv-pkg//modules/generic/libinferencer_module.so(_ZN6nvidia3jrt19TritonShmInferencerC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKlS9_S9_bbbiS9_+0x3871) [0x7f6d61984441]
#12 nvidia::jrt::InferencerModule::start() /workspace/tlt_cv-pkg//modules/generic/libinferencer_module.so(_ZN6nvidia3jrt16InferencerModule5startEv+0x1d3) [0x7f6d61973f03]
#13 isaac::alice::CodeletCanister::start() ./samples/tlt_cv/demo_gaze/gaze(_ZN5isaac5alice15CodeletCanister5startEv+0x22a) [0x56444f53c35a]
#14 isaac::alice::NodeCanister::startCodelets() ./samples/tlt_cv/demo_gaze/gaze(_ZN5isaac5alice12NodeCanister13startCodeletsEv+0x99) [0x56444f559a39]
#15 isaac::alice::NodeCanister::start() ./samples/tlt_cv/demo_gaze/gaze(_ZN5isaac5alice12NodeCanister5startEv+0x13e) [0x56444f559bee]
#16 isaac::alice::NodeBackend::processQueue() ./samples/tlt_cv/demo_gaze/gaze(_ZN5isaac5alice11NodeBackend12processQueueEv+0x91) [0x56444f554091]
#17 isaac::alice::NodeBackend::queueMain() ./samples/tlt_cv/demo_gaze/gaze(_ZN5isaac5alice11NodeBackend9queueMainEv+0x55) [0x56444f5541e5]
#18 isaac::scheduler::Job::run() /workspace/tlt_cv-pkg/samples/tlt_cv/demo_gaze/../../../_solib_k8/_U_S_Sprivate_Sapi_C_Ujrt_Ushared_Ulink___Uprivate_Sapi/libjrt.so(_ZN5isaac9scheduler3Job3runEv+0x36) [0x7f6d7c92cd16]
#19 isaac::scheduler::ExecutionGroup::BlockingWorker::main(isaac::scheduler::ExecutionGroup*, isaac::scheduler::Job*, bool) /workspace/tlt_cv-pkg/samples/tlt_cv/demo_gaze/../../../_solib_k8/_U_S_Sprivate_Sapi_C_Ujrt_Ushared_Ulink___Uprivate_Sapi/libjrt.so(_ZN5isaac9scheduler14ExecutionGroup14BlockingWorker4mainEPS1_PNS0_3JobEb+0x1b5) [0x7f6d7c8280a5]
#20 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xbd6df) [0x7f6d79eec6df]
#21 /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f6d7a1bf6db]
#22 /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f6d7994771f]
====================================================================================================
Minidump written to: /tmp/d406b39d-1350-4f5d-af53af92-77e2c404.dmp
Aborted (core dumped)

Can anyone help me?
Thanks

Have you run tlt_cv_init.sh and tlt_cv_start_server.sh?

yes, and when I started tlt_cv_start_server.sh I get:
I0527 16:20:40.448704 1 grpc_server.cc:3979] Started GRPCInferenceService at 0.0.0.0:8001
I0527 16:20:40.449715 1 http_server.cc:2717] Started HTTPService at 0.0.0.0:8000
I0527 16:20:40.501744 1 http_server.cc:2736] Started Metrics Service at 0.0.0.0:8002

Can you share the demo.conf ?

# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
#
# NVIDIA CORPORATION and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto. Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA CORPORATION is strictly prohibited.
#

## Do not use spaces in this file

# Path to device handle; ensure config.sh can see this handle
video_path=file://workspace/video/vid_20210527_143423.mp4
fps=29.96
# Boolean indicating whether handle is pre-recorded
is_video_path_file=true

# Desired resolution (width, height, channels)
resolution_whc=1920,1080,3

# Show visualization
visualization=true

# Leverage API to send single decoded frames to Pipeline
use_decoded_image_api=false

# If gaze_vec_length != 0, rendering of the vector will be enabled
gaze_vec_length=200

landmarks_3d_file_path=samples/tlt_cv/demo_gaze/anthropometic_3D_landmarks.txt

# Camera calib for default webcam
# Row major camera matrix
camera_matrix=960.5037478246579,0.0,643.5179717135431,0.0,960.1507947013808,365.1361007229173,0.0,0.0,1.0
distortion_coeffs=0.020923245479571907,-0.06576092071229227,0.0034316109115200603,-0.002976025265968983,-0.13518484381212603

Can you modify video_path=file://workspace/video/vid_20210527_143423.mp4 to
video_path=/workspace/video/vid_20210527_143423.mp4 and retry?

I made this modification and I got the same error.

To narrow down, could you also try other network too, for example, emotionnet?

I got the same error, using the same demo.conf.

This is the first time end user meets this error.
If you get the same error with gazenet or emotionnet, Iā€™m afraid there should be related to the environment setup.
Did you ever modify something, in other word, did you use the default setting in the tlt_cv scripts?

Sorry, I changed one thing in tlt_cv_start_server.sh. I use docker run with a new user. Using the default file is works fine. I am sorry, I forgot this change :/

Thanks for the info. May I know which line did you modify?

Instead of use:

docker run --rm -it --net host --runtime nvidia \
    --name image_tlt_cv_server \
    --ipc "host" \
    -v ${models_location}/triton_model_repository:/model_repository \
    --gpus device=1 \
    ${image_tlt_cv_server} /opt/tritonserver/bin/tritonserver \
        --model-store /model_repository/repository --log-info=true

I used :

docker run --rm -it --user="1002" --net host --runtime nvidia \
    --name image_tlt_cv_server \
    --ipc "host" \
    -v ${models_location}/triton_model_repository:/model_repository \
    --gpus device=1 \
    ${image_tlt_cv_server} /opt/tritonserver/bin/tritonserver \
        --model-store /model_repository/repository --log-info=true

Removing the user, that is, using the default, it works fine.

1 Like

Great, thanks for the info.