Deepstream cudaErrorInitializationError error on Xavier AGX

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Good day,
I’m setting up Deepstream 5.0 on Xavier AGX platform according to this guide:
https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html#page/DeepStream_Development_Guide/deepstream_quick_start.html#

Apparently I have faced an issue when I was running sample apps to verify that installation is valid.
nvidia@nvidia-desktop:$ ./deepstream-test1-app …/…/…/…/samples/streams/sample_720p.h264
NVMAP_IOC_QUERY_HEAP_PARAMS failed [Inappropriate ioctl for device]
NVMAP_IOC_QUERY_HEAP_PARAMS failed [Inappropriate ioctl for device]
Error: Could not get cuda device count (cudaErrorInitializationError)
Failed to parse group property
** ERROR: <gst_nvinfer_parse_config_file:1242>: failed
Now playing: …/…/…/…/samples/streams/sample_720p.h264

Using winsys: x11
NVMAP_IOC_QUERY_HEAP_PARAMS failed [Inappropriate ioctl for device]
Unable to set device in gst_nvstreammux_change_state
Unable to set device in gst_nvstreammux_change_state
Running…

In each and every application I’ve got ‘Error: Could not get cuda device count (cudaErrorInitializationError)’
And after a while when application is running it also crashes with cudaErrorInitializationError.

System specs are provided below:
Device: Nvidia Jetson AGX Xavier
$ jetson_release:

  • NVIDIA Jetson UNKNOWN
    • Jetpack 4.4 [L4T 32.4.3]
    • NV Power Mode: MODE_30W_ALL - Type: 3
    • jetson_stats.service: active
  • Libraries:
    • CUDA: 10.2.89
    • cuDNN: 8.0.0.180
    • TensorRT: 7.1.3.0
    • Visionworks: 1.6.0.501
    • OpenCV: 4.1.1 compiled CUDA: YES
    • VPI: 4.4-b186
    • Vulkan: 1.2.70
    • Deepstream: 5.0

As a background information:

  1. I have manually updated jetpack using ‘sudo apt-get update && sudo apt-get install nvidia-jetpack’ command. Previous version was Jetpack 4.4 DP.
  2. When I’m running ‘sudo apt-get update’ I got this error:
    W: GPG error: https://repo.download.nvidia.com/jetson/common r32.4 InRelease: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY 0D296FFB880FB004
    W: GPG error: https://repo.download.nvidia.com/jetson/t194 r32.4 InRelease: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY 0D296FFB880FB004

Could you please help me with this issue? Thanks in advance.

3

Hi,
Please clean gstreamr cache and check if you can run deepstream-app:

nvidia@nvidia-desktop:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/destream-app$ rm ~/.cache/gstreamer-1.0/registry.aarch64.bin
nvidia@nvidia-desktop:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app$ export DISPLAY=:0
nvidia@nvidia-desktop:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app$ deepstream-app -c source30_1080p_dec_infer-resnet_tiled_display_int8.txt

If not, we would suggest clean re-flash through SDKManager.

4

Good day DaneLLL,
Thanks for reply. I have followed instruction and recieved this output:
////////////////////////
nvidia@nvidia-desktop:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app$ sudo rm ~/.cache/gstreamer-1.0/registry.aarch64.bin
nvidia@nvidia-desktop:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app$ export DISPLAY=:0
nvidia@nvidia-desktop:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app$ deepstream-app -c source30_1080p_dec_infer-resnet_tiled_display_int8.txt
NVMAP_IOC_QUERY_HEAP_PARAMS failed [Inappropriate ioctl for device]

(gst-plugin-scanner:21496): GLib-GObject-WARNING **: 11:04:20.214: cannot register existing type ‘GstInterpolationMethod’

(gst-plugin-scanner:21496): GLib-GObject-CRITICAL **: 11:04:20.215: g_param_spec_enum: assertion ‘G_TYPE_IS_ENUM (enum_type)’ failed

(gst-plugin-scanner:21496): GLib-GObject-CRITICAL **: 11:04:20.215: validate_pspec_to_install: assertion ‘G_IS_PARAM_SPEC (pspec)’ failed
2020-08-20 11:04:21.108891: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.2

(gst-plugin-scanner:21496): GStreamer-WARNING **: 11:04:21.555: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstlibav.so’: /usr/lib/aarch64-linux-gnu/libgomp.so.1: cannot allocate memory in static TLS block

(gst-plugin-scanner:21496): GStreamer-WARNING **: 11:04:21.712: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstchromaprint.so’: /usr/lib/aarch64-linux-gnu/libgomp.so.1: cannot allocate memory in static TLS block
NVMAP_IOC_QUERY_HEAP_PARAMS failed [Inappropriate ioctl for device]
NVMAP_IOC_QUERY_HEAP_PARAMS failed [Inappropriate ioctl for device]
Error: Could not get cuda device count (cudaErrorInitializationError)
Failed to parse group property
** ERROR: <gst_nvinfer_parse_config_file:1242>: failed

Using winsys: x11
NVMAP_IOC_QUERY_HEAP_PARAMS failed [Inappropriate ioctl for device]
Unable to set device in gst_nvstreammux_change_state
Unable to set device in gst_nvstreammux_change_state
Unable to set device in gst_nvstreammux_change_state
** ERROR: main:655: Failed to set pipeline to PAUSED
Quitting
Unable to set device in gst_nvstreammux_change_state
App run failed
////////////////////////

Will reflash the device.