Camera does not work with TAO CV pipelines l4t

Hello
I tried to use a Raspberry Pi 2.1 camera (on IMX219 controller) along with the TAO Toolkit CV Inference Pipeline L4T ( NVIDIA NGC ).

I am making a pass of the nvargus daemon into the container:

docker run --rm -it --net host --runtime nvidia \
    ${VIDEO_DEVICES} \
    --name image_tao_cv_client \
    --ipc "host" \
    --env="DISPLAY" \
    --env="QT_X11_NO_MITSHM=1" \
    ${DOCKER_VOLUME_MNT_SAMPLES} \
    --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \
    -v /tmp/argus_socket:/tmp/argus_socket:rw \
    -v ${HOME}/.Xauthority:/root/.Xauthority:rw \
    -v /home/professorx/gaze/:/root/workspace/gaze/:rw \
    -v /home/professorx/gaze/samples/:/workspace/tao_cv-pkg/samples:rw \
    ${GPUS} \
    ${image_tao_cv_client} /bin/bash

Unfortunately, the camera does not start, I get an error:

[INFO] Logging with CSV Format
[EMOTION]	timestamp	faces	emotion
[GAZE]		timestamp	faces	looking
2021-09-24 09:11:36.443 WARN  external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
2021-09-24 09:11:36.461 WARN  external/com_nvidia_isaac_engine/engine/alice/backend/codelet_canister.cpp@229: Codelet '_check_operating_system/isaac.alice.CheckOperatingSystem' was not added to scheduler because no tick method is specified.
2021-09-24 09:11:36.461 WARN  external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
2021-09-24 09:11:36.471 WARN  external/com_nvidia_isaac_engine/engine/alice/components/Codelet.cpp@53: Function deprecated. Set tick_period to the desired tick parameter
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Available Sensor modes :
Resolution: 3264 x 2464 ; Framerate = 21.000000; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 3264 x 1848 ; Framerate = 28.000001; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1920 x 1080 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1640 x 1232 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1280 x 720 ; Framerate = 59.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

DEFAULT no IOCTL called
Opening in BLOCKING MODE
(NvCameraUtils) Error InvalidState: Mutex already initialized (in Mutex.cpp, function initialize(), line 41)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function open(), line 54)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 258)
(Argus) Error InvalidState: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
ArgusV4L2_Open failed: Illegal seek
(NvCameraUtils) Error InvalidState: Mutex already initialized (in Mutex.cpp, function initialize(), line 41)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function open(), line 54)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 258)
(Argus) Error InvalidState: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
ArgusV4L2_Open failed: Invalid argument
Opening in BLOCKING MODE 
Unsupported buffer type
VIDEOIO ERROR: V4L: device /dev/video0: Unable to query number of channels
Opening in O_NONBLOCKING MODE 
libv4l2: error setting pixformat: Device or resource busy
VIDEOIO ERROR: libv4l unable to ioctl S_FMT
Opening in BLOCKING MODE
(NvCameraUtils) Error InvalidState: Mutex already initialized (in Mutex.cpp, function initialize(), line 41)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function open(), line 54)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 258)
(Argus) Error InvalidState: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
ArgusV4L2_Open failed: Device or resource busy
Opening in BLOCKING MODE 
VIDEOIO ERROR: libv4l unable to ioctl VIDIOCSPICT

[Jarvis] [E] Could not open video source /dev/video0
terminate called after throwing an instance of 'std::runtime_error*'
[Jarvis] [E] Could not open video source /dev/video0
terminate called recursively
====================================================================================================
|                            Isaac application terminated unexpectedly                             |
====================================================================================================
#01 /workspace/tao_cv-pkg/private/api/libjrt_tao_cv_api.so(+0xa1290) [0x7f86ecf290]
#02 google_breakpad::ExceptionHandler::GenerateDump(google_breakpad::ExceptionHandler::CrashContext*) /workspace/tao_cv-pkg/private/api/../../_solib_k8/_U_S_Sprivate_Sapi_C_Ujrt_Ushared_Ulink___Uprivate_Sapi/libjrt.so(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3e8) [0x7f845a4e10]
#03 google_breakpad::ExceptionHandler::SignalHandler(int, siginfo_t*, void*) /workspace/tao_cv-pkg/private/api/../../_solib_k8/_U_S_Sprivate_Sapi_C_Ujrt_Ushared_Ulink___Uprivate_Sapi/libjrt.so(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xe0) [0x7f845a5118]
#04 linux-vdso.so.1(__kernel_rt_sigreturn+0) [0x7f877b16b0]
#05 /lib/aarch64-linux-gnu/libc.so.6(raise+0xb0) [0x7f86a964f8]
====================================================================================================
Minidump written to: /tmp/052f38c0-8b3c-4a5a-c1b3c884-18c16569.dmp
Aborted (core dumped)

On the host and in the container, the camera works great:

gst-launch-1.0 nvarguscamerasrc sensor_mode=0 ! ‘video/x-raw(memory:NVMM),width=3820, height=2464, framerate=21/1, format=NV12’ ! nvvidconv flip-method=0 ! ‘video/x-raw,width=960, height=616’ ! nvvidconv ! nvegltransform ! nveglglessink -e

How can I resolve this error?

System Information:

professorx@x-mansion:~/gaze/jetsonUtilities$ sudo python3 jetsonInfo.py
[sudo] password for professorx: 
NVIDIA Jetson Xavier NX (Developer Kit Version)
 L4T 32.5.1 [ JetPack 4.5.1 ]
   Ubuntu 18.04.6 LTS
   Kernel Version: 4.9.201-tegra
 CUDA 10.2.89
   CUDA Architecture: 7.2
 OpenCV version: 4.1.1
   OpenCV Cuda: NO
 CUDNN: 8.0.0.180
 TensorRT: 7.1.3.0
 Vision Works: 1.6.0.501
 VPI: ii libnvvpi1 1.0.15 arm64 NVIDIA Vision Programming Interface library
 Vulcan: 1.2.70

Camera Information:

root@x-mansion:/workspace/tao_cv-pkg# v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB
		Size: Discrete 3264x2464
			Interval: Discrete 0.048s (21.000 fps)
		Size: Discrete 3264x1848
			Interval: Discrete 0.036s (28.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1640x1232
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.017s (60.000 fps)

root@x-mansion:/workspace/tao_cv-pkg# v4l2-ctl --device=/dev/video0 -D --list-formats
Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, imx219 10-0010
	Bus info      : platform:15c10000.vi:2
	Driver version: 4.9.201
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB

Full error output, scripts for launching the TAO CV Client container and the source code in the attached files.

p.s. I got a similar error when trying to run examples to register gaze and emotions.
camera.conf (1.3 KB)
config.sh (5.1 KB)
emotional_gaze.cpp (19.5 KB)
emotional_gaze_crash-full_output.txt (22.7 KB)
tao_cv_start_client.sh (1.9 KB)

Do you have below devices ?
See Requirements and Installation — TAO Toolkit 3.0 documentation

  • Webcam
  • Logitech C920 Pro HD
  • Logitech C922
  • Logitech C310

I’m afraid the Pixel Format of IMX219 is not supported.

1 Like

Thanks for the answer.
Unfortunately, I have no webcams from the list.

Is the “Arducam HQ Camera” (on the IMX477 controller) suitable?

I need a compact camera, the webcam is relatively large.

I think it is not large. For example,

I have a question (for the future to keep in mind) TAO supports other cameras?

  • other webcams besides those listed in the list of requirements for HW;
  • CSI cameras
    p.s. And what is the reason for this limitation in choosing a camera as a video source?

Other webcams should meet below minimum requirement. See Requirements and Installation — TAO Toolkit 3.0 documentation

  • 720p Webcam

Above-mentioned cameras are just recommended ones. See Requirements and Installation — TAO Toolkit 3.0 documentation

For CSI camera, I will sync with internal team for their supporting.

1 Like