Jetson NANO and USB 5.8G UVC Camera Receiver

Hi all,
I’m trying how to use a 5.8G UVC USB receiver with Jetson NANO.
In particular I’m interested to redirect the 640x480 video received to the “detectnet-camera.py”.

Using a USB-CAM or a CSI-CAM everything works fine.
While running the detectnet-camera-py when the 5.8G UVC Receiver is connected to the USB here is what happened:

./detectnet-camera.py --camera=/dev/video0
jetson.inference.init.py
jetson.inference – initializing Python 2.7 bindings…
jetson.inference – registering module types…
jetson.inference – done registering module types
jetson.inference – done Python 2.7 binding initialization
jetson.utils.init.py
jetson.utils – initializing Python 2.7 bindings…
jetson.utils – registering module functions…
jetson.utils – done registering module functions
jetson.utils – registering module types…
jetson.utils – done registering module types
jetson.utils – done Python 2.7 binding initialization
jetson.inference – PyTensorNet_New()
jetson.inference – PyDetectNet_Init()
jetson.inference – detectNet loading build-in network ‘pednet’

detectNet – loading detection network model from:
– prototxt networks/ped-100/deploy.prototxt
– model networks/ped-100/snapshot_iter_70800.caffemodel
– input_blob ‘data’
– output_cvg ‘coverage’
– output_bbox ‘bboxes’
– mean_pixel 0.000000
– mean_binary NULL
– class_labels networks/ped-100/class_labels.txt
– threshold 0.500000
– batch_size 1

[TRT] TensorRT version 5.1.6
[TRT] loading NVIDIA plugins…
[TRT] Plugin Creator registration succeeded - GridAnchor_TRT
[TRT] Plugin Creator registration succeeded - NMS_TRT
[TRT] Plugin Creator registration succeeded - Reorg_TRT
[TRT] Plugin Creator registration succeeded - Region_TRT
[TRT] Plugin Creator registration succeeded - Clip_TRT
[TRT] Plugin Creator registration succeeded - LReLU_TRT
[TRT] Plugin Creator registration succeeded - PriorBox_TRT
[TRT] Plugin Creator registration succeeded - Normalize_TRT
[TRT] Plugin Creator registration succeeded - RPROI_TRT
[TRT] Plugin Creator registration succeeded - BatchedNMS_TRT
[TRT] completed loading NVIDIA plugins.
[TRT] detected model format - caffe (extension ‘.caffemodel’)
[TRT] desired precision specified for GPU: FASTEST
[TRT] requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT] native precisions detected for GPU: FP32, FP16
[TRT] selecting fastest native precision for GPU: FP16
[TRT] attempting to open engine cache file networks/ped-100/snapshot_iter_70800.caffemodel.1.1.GPU.FP16.engine
[TRT] loading network profile from engine cache… networks/ped-100/snapshot_iter_70800.caffemodel.1.1.GPU.FP16.engine
[TRT] device GPU, networks/ped-100/snapshot_iter_70800.caffemodel loaded
[TRT] device GPU, CUDA engine context initialized with 3 bindings
[TRT] binding – index 0
– name ‘data’
– type FP32
– in/out INPUT
– # dims 3
– dim #0 3 (CHANNEL)
– dim #1 512 (SPATIAL)
– dim #2 1024 (SPATIAL)
[TRT] binding – index 1
– name ‘coverage’
– type FP32
– in/out OUTPUT
– # dims 3
– dim #0 1 (CHANNEL)
– dim #1 32 (SPATIAL)
– dim #2 64 (SPATIAL)
[TRT] binding – index 2
– name ‘bboxes’
– type FP32
– in/out OUTPUT
– # dims 3
– dim #0 4 (CHANNEL)
– dim #1 32 (SPATIAL)
– dim #2 64 (SPATIAL)
[TRT] binding to input 0 data binding index: 0
[TRT] binding to input 0 data dims (b=1 c=3 h=512 w=1024) size=6291456
[TRT] binding to output 0 coverage binding index: 1
[TRT] binding to output 0 coverage dims (b=1 c=1 h=32 w=64) size=8192
[TRT] binding to output 1 bboxes binding index: 2
[TRT] binding to output 1 bboxes dims (b=1 c=4 h=32 w=64) size=32768
device GPU, networks/ped-100/snapshot_iter_70800.caffemodel initialized.
detectNet – number object classes: 1
detectNet – maximum bounding boxes: 2048
detectNet – loaded 1 class info entries
detectNet – number of object classes: 1
jetson.utils – PyCamera_New()
jetson.utils – PyCamera_Init()
[gstreamer] initialized gstreamer, version 1.14.5.0
[gstreamer] gstCamera attempting to initialize with GST_SOURCE_NVARGUS, camera /dev/video0
[gstreamer] gstCamera pipeline string:
v4l2src device=/dev/video0 ! video/x-raw, width=(int)1280, height=(int)720, format=YUY2 ! videoconvert ! video/x-raw, format=RGB ! videoconvert !appsink name=mysink
[gstreamer] gstCamera successfully initialized with GST_SOURCE_V4L2, camera /dev/video0
jetson.utils – PyDisplay_New()
jetson.utils – PyDisplay_Init()
[OpenGL] glDisplay – X screen 0 resolution: 1920x1080
[OpenGL] glDisplay – display device initialized
[gstreamer] opening gstCamera for streaming, transitioning pipeline to GST_STATE_PLAYING
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> videoconvert1
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1
[gstreamer] gstreamer changed state from NULL to READY ==> videoconvert0
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0
[gstreamer] gstreamer changed state from NULL to READY ==> v4l2src0
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> videoconvert1
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1
[gstreamer] gstreamer changed state from READY to PAUSED ==> videoconvert0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src0
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer msg new-clock ==> pipeline0
[gstreamer] gstreamer msg stream-start ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videoconvert1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videoconvert0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src0
[gstreamer] gstCamera onEOS
[gstreamer] gstreamer v4l2src0 ERROR Internal data stream error.
[gstreamer] gstreamer Debugging info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink

Seems that Gstreame does not recognize the stream data format.
Any ideas on how resolve this error?

Thanks to anyone will help
Plauto

Hi pfrigerio,

You may need to get the 5.8G UVC USB receiver work well with Jetson Nano, be sure to check the output format if can be supported.
Please also refer to below topic if can help:
https://devtalk.nvidia.com/default/topic/1063717/jetson-nano/usb-camera-mjpeg-to-h265/post/5386572/#5386572

Please also check
https://devtalk.nvidia.com/default/topic/1057681/jetson-tx1/logitech-c930e-on-jetson-tx1-very-slow-and-choppy-video/post/5363417/#5363417

hi and thanks for your support.
looking at v4l2-ctl output seems that this cam is supported:

$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘MJPG’ (compressed)
Name : Motion-JPEG
Size: Discrete 640x480
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 352x288
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.200s (5.000 fps)

I tried implemenying the pipeline suggested in the topic and it works. I can see video-in on the monitor

What’s next? how to modify the “detectnet-camera.cpp” or .py to have the same result?
Any reference is welcome :-)

Hi,
You need to replace the pipeline in gstCamera.cpp with yours.

Hi DaneLLL,
thanks for your answer, following your suggestion I’ve modified the gstCamera.cpp replacing the pipeline in function “buildLaunchStr” passing to the next function a fixed string like this:

mLaunchStr = "v4l2src device=/dev/video0 ! image/jpeg, width=640, height=480, framerate=30/1, format=MJPG ! jpegdec ! appsink name=mysink";

It works… but after few frames I encontered another error from CUDA cores here is the log :

nvidia@nvidia-nano:~/jetson-inference/build/aarch64/bin$ ./imagenet-camera --camera=/dev/video0 
[gstreamer] initialized gstreamer, version 1.14.5.0
[gstreamer] gstCamera attempting to initialize with GST_SOURCE_NVARGUS, camera /dev/video0
[gstreamer] gstCamera pipeline string:
v4l2src device=/dev/video0 ! image/jpeg, width=640, height=480, framerate=30/1, format=MJPG ! jpegdec ! appsink name=mysink
[gstreamer] gstCamera successfully initialized with GST_SOURCE_V4L2, camera /dev/video0

imagenet-camera:  successfully initialized camera device
    width:  1280
   height:  720
    depth:  24 (bpp)


imageNet -- loading classification network model from:
         -- prototxt     networks/googlenet.prototxt
         -- model        networks/bvlc_googlenet.caffemodel
         -- class_labels networks/ilsvrc12_synset_words.txt
         -- input_blob   'data'
         -- output_blob  'prob'
         -- batch_size   1

[TRT]   TensorRT version 5.1.6
[TRT]   loading NVIDIA plugins...
[TRT]   Plugin Creator registration succeeded - GridAnchor_TRT
[TRT]   Plugin Creator registration succeeded - NMS_TRT
[TRT]   Plugin Creator registration succeeded - Reorg_TRT
[TRT]   Plugin Creator registration succeeded - Region_TRT
[TRT]   Plugin Creator registration succeeded - Clip_TRT
[TRT]   Plugin Creator registration succeeded - LReLU_TRT
[TRT]   Plugin Creator registration succeeded - PriorBox_TRT
[TRT]   Plugin Creator registration succeeded - Normalize_TRT
[TRT]   Plugin Creator registration succeeded - RPROI_TRT
[TRT]   Plugin Creator registration succeeded - BatchedNMS_TRT
[TRT]   completed loading NVIDIA plugins.
[TRT]   detected model format - caffe  (extension '.caffemodel')
[TRT]   desired precision specified for GPU: FASTEST
[TRT]   requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT]   native precisions detected for GPU:  FP32, FP16
[TRT]   selecting fastest native precision for GPU:  FP16
[TRT]   attempting to open engine cache file networks/bvlc_googlenet.caffemodel.1.1.GPU.FP16.engine
[TRT]   loading network profile from engine cache... networks/bvlc_googlenet.caffemodel.1.1.GPU.FP16.engine
[TRT]   device GPU, networks/bvlc_googlenet.caffemodel loaded
[TRT]   device GPU, CUDA engine context initialized with 2 bindings
[TRT]   binding -- index   0
               -- name    'data'
               -- type    FP32
               -- in/out  INPUT
               -- # dims  3
               -- dim #0  3 (CHANNEL)
               -- dim #1  224 (SPATIAL)
               -- dim #2  224 (SPATIAL)
[TRT]   binding -- index   1
               -- name    'prob'
               -- type    FP32
               -- in/out  OUTPUT
               -- # dims  3
               -- dim #0  1000 (CHANNEL)
               -- dim #1  1 (SPATIAL)
               -- dim #2  1 (SPATIAL)
[TRT]   binding to input 0 data  binding index:  0
[TRT]   binding to input 0 data  dims (b=1 c=3 h=224 w=224) size=602112
[TRT]   binding to output 0 prob  binding index:  1
[TRT]   binding to output 0 prob  dims (b=1 c=1000 h=1 w=1) size=4000
device GPU, networks/bvlc_googlenet.caffemodel initialized.
[TRT]   networks/bvlc_googlenet.caffemodel loaded
imageNet -- loaded 1000 class info entries
networks/bvlc_googlenet.caffemodel initialized.
[OpenGL] glDisplay -- X screen 0 resolution:  1920x1080
[OpenGL] glDisplay -- display device initialized
[gstreamer] opening gstCamera for streaming, transitioning pipeline to GST_STATE_PLAYING
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> jpegdec0
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0
[gstreamer] gstreamer changed state from NULL to READY ==> v4l2src0
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> jpegdec0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src0
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[gstreamer] gstreamer msg new-clock ==> pipeline0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> jpegdec0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src0
[gstreamer] gstreamer msg stream-start ==> pipeline0

imagenet-camera:  camera open for streaming
[gstreamer] gstCamera onPreroll
[gstreamer] gstCamera -- allocated 16 ringbuffers, 460800 bytes each
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer msg async-done ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0
[gstreamer] gstCamera -- allocated 16 RGBA ringbuffers
class 0446 - 0.011103  (binder, ring-binder)
class 0498 - 0.027481  (cinema, movie theater, movie theatre, movie house, picture palace)
class 0508 - 0.032381  (computer keyboard, keypad)
class 0530 - 0.027374  (digital clock)
class 0549 - 0.022961  (envelope)
class 0620 - 0.017674  (laptop, laptop computer)
class 0664 - 0.038906  (monitor)
class 0681 - 0.077073  (notebook, notebook computer)
class 0688 - 0.012053  (oscilloscope, scope, cathode-ray oscilloscope, CRO)
class 0769 - 0.061208  (rule, ruler)
class 0782 - 0.066701  (screen, CRT screen)
class 0818 - 0.092968  (spotlight, spot)
class 0851 - 0.017813  (television, television system)
class 0854 - 0.033278  (theater curtain, theatre curtain)
class 0881 - 0.012195  (upright, upright piano)
class 0904 - 0.015537  (window screen)
class 0905 - 0.025120  (window shade)
class 0916 - 0.025218  (web site, website, internet site, site)
imagenet-camera:  9.29679% class #818 (spotlight, spot)
[OpenGL]   creating 640x480 texture
[cuda]   registered 4915200 byte openGL texture for interop access (640x480)

[TRT]   ----------------------------------------------
[TRT]   Timing Report networks/bvlc_googlenet.caffemodel
[TRT]   ----------------------------------------------
[TRT]   Pre-Process   CPU  0.09297ms  CUDA  0.44698ms
[TRT]   Network       CPU 48.76760ms  CUDA 44.12193ms
[TRT]   Post-Process  CPU  0.41078ms  CUDA  0.40984ms
[TRT]   Total         CPU 49.27135ms  CUDA 44.97875ms
[TRT]   ----------------------------------------------

[TRT]   note -- when processing a single image, run 'sudo jetson_clocks' before
                to disable DVFS for more accurate profiling/timing measurements

class 0410 - 0.026373  (apiary, bee house)
class 0489 - 0.015383  (chainlink fence)
class 0498 - 0.016764  (cinema, movie theater, movie theatre, movie house, picture palace)
class 0508 - 0.014339  (computer keyboard, keypad)
class 0599 - 0.047570  (honeycomb)
class 0611 - 0.035489  (jigsaw puzzle)
class 0620 - 0.061801  (laptop, laptop computer)
class 0632 - 0.011167  (loudspeaker, speaker, speaker unit, loudspeaker system, speaker system)
class 0681 - 0.048319  (notebook, notebook computer)
class 0703 - 0.016569  (park bench)
class 0782 - 0.011841  (screen, CRT screen)
class 0815 - 0.111036  (spider web, spider's web)
class 0818 - 0.047570  (spotlight, spot)
class 0819 - 0.014395  (stage)
class 0854 - 0.013576  (theater curtain, theatre curtain)
class 0904 - 0.104309  (window screen)
class 0971 - 0.023093  (bubble)
imagenet-camera:  11.10360% class #815 (spider web, spider's web)

few time frames are elaborated then here is the error:

[TRT]   ----------------------------------------------
[TRT]   Timing Report networks/bvlc_googlenet.caffemodel
[TRT]   ----------------------------------------------
[TRT]   Pre-Process   CPU  0.05724ms  CUDA  0.29047ms
[TRT]   Network       CPU 16.85818ms  CUDA 15.20141ms
[TRT]   Post-Process  CPU  0.61323ms  CUDA  0.00047ms
[TRT]   Total         CPU 17.52864ms  CUDA 15.49234ms
[TRT]   ----------------------------------------------

class 0410 - 0.023757  (apiary, bee house)
class 0488 - 0.034837  (chain)
class 0489 - 0.036939  (chainlink fence)
class 0490 - 0.137246  (chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour)
class 0599 - 0.222773  (honeycomb)
class 0611 - 0.016650  (jigsaw puzzle)
class 0632 - 0.018358  (loudspeaker, speaker, speaker unit, loudspeaker system, speaker system)
class 0695 - 0.012519  (padlock)
class 0815 - 0.181822  (spider web, spider's web)
class 0904 - 0.022057  (window screen)
class 0907 - 0.016138  (wine bottle)
class 0971 - 0.154310  (bubble)
imagenet-camera:  22.27725% class #599 (honeycomb)

[TRT]   ----------------------------------------------
[TRT]   Timing Report networks/bvlc_googlenet.caffemodel
[TRT]   ----------------------------------------------
[TRT]   Pre-Process   CPU  0.09198ms  CUDA  0.28406ms
[TRT]   Network       CPU 17.16062ms  CUDA 13.83359ms
[TRT]   Post-Process  CPU  0.55469ms  CUDA  0.52755ms
[TRT]   Total         CPU 17.80729ms  CUDA 14.64521ms
[TRT]   ----------------------------------------------

[TRT]   engine.cpp (764) - Cuda Error in execute: 4 (unspecified launch failure)
[TRT]   engine.cpp (764) - Cuda Error in execute: 4 (unspecified launch failure)
[TRT]   imageNet::Process() -- failed to execute TensorRT network
[TRT]   imageNet::Process() failed
[cuda]  cudaGetLastError()
[cuda]     unspecified launch failure (error 4) (hex 0x04)
[cuda]     /home/nvidia/jetson-inference/utils/cuda/cudaNormalize.cu:65
[cuda]  cudaNormalizeRGBA((float4*)img, make_float2(0.0f, 255.0f), (float4*)mNormalizedCUDA, make_float2(0.0f, 1.0f), width, height)
[cuda]     unspecified launch failure (error 4) (hex 0x04)
[cuda]     /home/nvidia/jetson-inference/utils/display/glDisplay.cpp:343
[cuda]  cudaGraphicsMapResources(1, &mInteropCUDA)
[cuda]     unspecified launch failure (error 4) (hex 0x04)
[cuda]     /home/nvidia/jetson-inference/utils/display/glTexture.cpp:259
[cuda]  cudaEventElapsedTime(&cuda_time, mEventsGPU[evt], mEventsGPU[evt+1])
[cuda]     unspecified launch failure (error 4) (hex 0x04)
[cuda]     /home/nvidia/jetson-inference/build/aarch64/include/jetson-inference/tensorNet.h:494

any idea on what happened ?

Hi,
There are cudaNV12ToRGBA32() and cudaRGB8ToRGBA32(). You may check which one you want to use and add videoconvert plugin before appsink for converting to the format.

Hi and thanks again,
can you please provide me an example of pipeline implementation?
More looking at Dusty source code I’m not able to find the functions cudaNV12ToRGBA32() and cudaRGB8ToRGBA32() for color space convertion, where i can find them? and eventually how to install?

Thank you in advance for your support.

Hello, I would like to get some example as well. Can you send me your gstCamera.ccp code please?

Hi,
The code is open source and we encourage users to integrate into their usecases.

You may also try DS4.0.1.
https://devtalk.nvidia.com/default/topic/1063437/deepstream-sdk/announcing-deepstream-sdk-4-0-1/

Hello

I have been trying to the jetson-inference however I have been I keep geting this error

I am using a USB cam

./imagenet-camera --camera=dev/video0
[gstreamer] initialized gstreamer, version 1.14.5.0
[gstreamer] gstCamera::Create(‘dev/video0’) – invalid camera device requested

imagenet-camera: failed to initialize camera device

hi, Im trying the same, use a UVC 5.8ghz receiver as video input, could you ge it work?, I’m beginner and I can’t understand what is the possible solutions or wre and how make code modifications
Thanks