Cuda failure : status=801 - Nvidia vGPU - Gstreamer

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) - Tesla T4
• DeepStream Version - Deepstream 4.0.2
• JetPack Version (valid for Jetson only)
• TensorRT Version - TensorRT-6.0.1.5
• NVIDIA GPU Driver Version (valid for GPU only) - 450.102.04
• Issue Type( questions, new requirements, bugs) - Gstreamer gstlaunch1.0 pipeline

Outputs -

I have setup Nvidia vGPU for T4 card using VMWare vsphere ESXI. I have added the PCI device GRID T4-16Q to a newly create VM and installed cuda 10.1 into it along with the Nvidia GRID driver 450.

Now when I have configured the machine with gstreamer TensorRT and deepstream, I tried to run the pipeline below to capture the video stream from RTSP cam which we use in our application and is tested and verified earlierly in the HPE server with t4 card.

Pipeline : gst-launch-1.0 rtspsrc location=rtsp://root:Glueck321@10.0.1.36:554/axis-media/media.amp? ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

This pipeline throws the below error,

gst-launch-1.0 rtspsrc location=rtsp://root:Glueck321@10.0.1.36:554/axis-media/media.amp? ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://root:Glueck321@10.0.1.36:554/axis-media/media.amp?
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request

(gst-launch-1.0:3607): GStreamer-CRITICAL **: 16:41:59.029: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed
Cuda failure: status=801

Can anyone help?

You can try without h264parse.

@Fiona.Chen Still the same error pops up after removing h264parse.

rtspsrc location=rtsp://root:root@10.101.49.11/axis-media/media.amp?streamprofile=H264 latency=100 ! rtph264depay ! nvv4l2decoder ! nvvideoconvert ! autovideosink

GPU pipeline:

rtspsrc location=rtsp://root:root@10.101.49.11/axis-media/media.amp?streamprofile=H264 latency=100 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

But while I run the CPU pipeline it works.

rtspsrc location=rtsp://root:root@10.101.49.11/axis-media/media.amp?streamprofile=H264 latency=0 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! autovideosink

Can you use “eglglessink” instead of “autovideosink”?

@Fiona.Chen Still the same. The Progess : Sent PLAY request is displayed and immediately the Cuda failure pops up.

rtspsrc location=rtsp://root:root@10.0.1.36/axis-media/media.amp?streamprofile=H264 latency=100 ! rtph264depay ! nvv4l2decoder ! nvvideoconvert ! nveglglessink

Can deepstream-test1-app work on your platform?

@Fiona.Chen FYI, i tried out running the deepstream-test1

Running the deepstream-test1-app, doesnot display anything. The script just stays freezed as nothing happens.

Seems there is something wrong with your display driver.

@Fiona.Chen What is the recommended driver that I have to use while using the card as vGPU ?

Currently I have installed the Nvidia GRID driver available in the vGPU software pack for VMWare ESXI6.7.

Can you upgrade to latest DS version?DS4.0 is not supported now.

The sample deepstream apps all use eglglessink for display. It needs openGL and X11 be installed correctly. OpenGL | NVIDIA Developer

@Fiona.Chen I have also tried this out in the latest DS 5.1 with all the compatible Cuda, Nvidia, Cudnn and TensorRT versions but this shows up nothing except the gst-launch screen pops up and shows scattered background display of whatever is present at the time in background.

> With nvgleglessink:

> With autovideosink:

Is openGL installed correctly? The drivers are available OpenGL | NVIDIA Developer

@Fiona.Chen I completely removed the installed nvidia drivers and tried to run the pipeline again but still the same problem. The screen appears with no visuals.

Is GPU driver installed correctly? NVIDIA DRIVERS Linux x64 (AMD64/EM64T) Display Driver

@Fiona.Chen Installed the driver 460.32 from the shared link. But the output is still the same.

So it may be X11 issue with your vmware enviroment.