TensorRT 10.3 does not support legacy caffe models for Jetpack6.2

Any ideas?

  • version
commit c038530ebf718e6867c4458c3e439406020732ff (HEAD -> master, origin/master, origin/HEAD)
Author: Dustin Franklin <dustinf@nvidia.com>
Date:   Wed Oct 16 06:56:03 2024 -0400

    updates for TRT10

------ 
Software part of jetson-stats 4.3.1 - (c) 2024, Raffaello Bonghi
Model: NVIDIA Jetson Orin Nano Developer Kit - Jetpack 6.2 [L4T 36.4.3]
NV Power Mode[0]: 15W
Serial Number: [XXX Show with: jetson_release -s XXX]
Hardware:
 - P-Number: p3767-0005
 - Module: NVIDIA Jetson Orin Nano (Developer kit)
Platform:
 - Distribution: Ubuntu 22.04 Jammy Jellyfish
 - Release: 5.15.148-tegra
jtop:
 - Version: 4.3.1
 - Service: Active
Libraries:
 - CUDA: 12.6.68
 - cuDNN: 9.3.0.75
 - TensorRT: 10.3.0.30
 - VPI: 3.2.4
 - OpenCV: 4.11.0 - with CUDA: YES
DeepStream C/C++ SDK version: 7.1

Python Environment:
Python 3.10.12
    GStreamer:                   YES (1.20.3)
  NVIDIA CUDA:                   YES (ver 12.6, CUFFT CUBLAS FAST_MATH)
         OpenCV version: 4.11.0  CUDA True
           YOLO version: 8.3.65
         PYCUDA version: 2024.1.2
          Torch version: 2.5.1+l4t36.4
    Torchvision version: 0.20.0a0+afc54f7
 DeepStream SDK version: 1.2.0
onnxruntime-gpu version: 1.19.2

  • imagenet log
$ imagenet rtp://@:5600
[gstreamer] initialized gstreamer, version 1.20.3.0
[gstreamer] gstDecoder -- creating decoder for 0.0.0.0
[gstreamer] gstDecoder -- resource discovery not supported for RTP/WebRTC streams
[gstreamer] gstDecoder -- defaulting to H264 codec (you can change this with the --input-codec option)
[gstreamer] gstDecoder -- pipeline string:
[gstreamer] udpsrc port=5600 multicast-group=0.0.0.0 auto-multicast=true caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" ! rtph264depay ! nvv4l2decoder name=decoder enable-max-performance=1 ! video/x-raw(memory:NVMM) ! nvvidconv name=vidconv ! video/x-raw ! appsink name=mysink sync=false
[video]  created gstDecoder from rtp://@:5600
------------------------------------------------
gstDecoder video options:
------------------------------------------------
  -- URI: rtp://@:5600
     - protocol:  rtp
     - location:  0.0.0.0
     - port:      5600
  -- deviceType: ip
  -- ioType:     input
  -- codec:      H264
  -- codecType:  v4l2
  -- frameRate:  0
  -- numBuffers: 4
  -- zeroCopy:   true
  -- flipMethod: none
  -- loop:       0
  -- latency     10
------------------------------------------------
[OpenGL] glDisplay -- X screen 0 resolution:  1920x1080
[OpenGL] glDisplay -- X window resolution:    1920x1080
[OpenGL] glDisplay -- display device initialized (1920x1080)
[video]  created glDisplay from display://0
------------------------------------------------
glDisplay video options:
------------------------------------------------
  -- URI: display://0
     - protocol:  display
     - location:  0
  -- deviceType: display
  -- ioType:     output
  -- width:      1920
  -- height:     1080
  -- frameRate:  0
  -- numBuffers: 4
  -- zeroCopy:   true
------------------------------------------------
[TRT]    downloading model Googlenet.tar.gz...
cd /usr/local/bin/networks ; wget --quiet --show-progress --progress=bar:force:noscroll --no-check-certificate https://nvidia.box.com/shared/static/u28j5jm4hnf1ex94dnhsuyu8p799l5d5.gz -O Googlenet.tar.gz ; tar -xzvf Googlenet.tar.gz ; rm Googlenet.tar.gz
Googlenet.tar.gz                          100%[=====================================================================================>]  47.28M   727KB/s    in 56s
Googlenet/
Googlenet/bvlc_googlenet.caffemodel
Googlenet/googlenet_noprob.prototxt
Googlenet/googlenet.prototxt
[TRT]    downloaded model Googlenet.tar.gz

imageNet -- loading classification network model from:
         -- prototxt     networks/Googlenet/googlenet.prototxt
         -- model        networks/Googlenet/bvlc_googlenet.caffemodel
         -- class_labels networks/ilsvrc12_synset_words.txt
         -- input_blob   'data'
         -- output_blob  'prob'
         -- batch_size   1

[TRT]    TensorRT version 10.3.0
[TRT]    loading NVIDIA plugins...
[TRT]    Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[TRT]    Registered plugin creator - ::BatchedNMS_TRT version 1
[TRT]    Registered plugin creator - ::BatchTilePlugin_TRT version 1
[TRT]    Registered plugin creator - ::Clip_TRT version 1
[TRT]    Registered plugin creator - ::CoordConvAC version 1
[TRT]    Registered plugin creator - ::CropAndResizeDynamic version 1
[TRT]    Registered plugin creator - ::CropAndResize version 1
[TRT]    Registered plugin creator - ::DecodeBbox3DPlugin version 1
[TRT]    Registered plugin creator - ::DetectionLayer_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_TRT version 1
[TRT]    Could not register plugin creator -  ::FlattenConcat_TRT version 1
[TRT]    Registered plugin creator - ::GenerateDetection_TRT version 1
[TRT]    Registered plugin creator - ::GridAnchor_TRT version 1
[TRT]    Registered plugin creator - ::GridAnchorRect_TRT version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 2
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 3
[TRT]    Registered plugin creator - ::LReLU_TRT version 1
[TRT]    Registered plugin creator - ::ModulatedDeformConv2d version 1
[TRT]    Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[TRT]    Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[TRT]    Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[TRT]    Registered plugin creator - ::NMSDynamic_TRT version 1
[TRT]    Registered plugin creator - ::NMS_TRT version 1
[TRT]    Registered plugin creator - ::Normalize_TRT version 1
[TRT]    Registered plugin creator - ::PillarScatterPlugin version 1
[TRT]    Registered plugin creator - ::PriorBox_TRT version 1
[TRT]    Registered plugin creator - ::ProposalDynamic version 1
[TRT]    Registered plugin creator - ::ProposalLayer_TRT version 1
[TRT]    Registered plugin creator - ::Proposal version 1
[TRT]    Registered plugin creator - ::PyramidROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::Region_TRT version 1
[TRT]    Registered plugin creator - ::Reorg_TRT version 2
[TRT]    Registered plugin creator - ::Reorg_TRT version 1
[TRT]    Registered plugin creator - ::ResizeNearest_TRT version 1
[TRT]    Registered plugin creator - ::ROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::ROIAlign_TRT version 2
[TRT]    Registered plugin creator - ::RPROI_TRT version 1
[TRT]    Registered plugin creator - ::ScatterElements version 2
[TRT]    Registered plugin creator - ::ScatterElements version 1
[TRT]    Registered plugin creator - ::ScatterND version 1
[TRT]    Registered plugin creator - ::SpecialSlice_TRT version 1
[TRT]    Registered plugin creator - ::Split version 1
[TRT]    Registered plugin creator - ::VoxelGeneratorPlugin version 1
[TRT]    completed loading NVIDIA plugins.
[TRT]    detected model format - caffe  (extension '.caffemodel')
[TRT]    desired precision specified for GPU: FASTEST
[TRT]    requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT]    [MemUsageChange] Init CUDA: CPU +2, GPU +0, now: CPU 43, GPU 2234 (MiB)
[TRT]    Trying to load shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    Loaded shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    [MemUsageChange] Init builder kernel library: CPU +927, GPU +759, now: CPU 1013, GPU 3034 (MiB)
[TRT]    CUDA lazy loading is enabled.
[TRT]    native precisions detected for GPU:  FP32, FP16, INT8
[TRT]    selecting fastest native precision for GPU:  FP16
[TRT]    could not find engine cache /usr/local/bin/networks/Googlenet/bvlc_googlenet.caffemodel.1.1.100300.GPU.FP16.engine
[TRT]    cache file invalid, profiling network model on device GPU
[TRT]    [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 87, GPU 3034 (MiB)
[TRT]    Trying to load shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    Loaded shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    [MemUsageChange] Init builder kernel library: CPU +926, GPU +138, now: CPU 1013, GPU 3172 (MiB)
[TRT]    CUDA lazy loading is enabled.
[TRT]    device GPU, loading /usr/local/bin/networks/Googlenet/googlenet.prototxt /usr/local/bin/networks/Googlenet/bvlc_googlenet.caffemodel
[TRT]    TensorRT 10.3 does not support legacy caffe models
[TRT]    device GPU, failed to load networks/Googlenet/bvlc_googlenet.caffemodel
[TRT]    failed to load networks/Googlenet/bvlc_googlenet.caffemodel
[TRT]    imageNet -- failed to initialize.
imagenet:  failed to initialize imageNet
  • detectnet log
daniel@daniel-nvidia:~/Work/jetson-fpv$ detectnet rtp://@:5600
[gstreamer] initialized gstreamer, version 1.20.3.0
[gstreamer] gstDecoder -- creating decoder for 0.0.0.0
[gstreamer] gstDecoder -- resource discovery not supported for RTP/WebRTC streams
[gstreamer] gstDecoder -- defaulting to H264 codec (you can change this with the --input-codec option)
[gstreamer] gstDecoder -- pipeline string:
[gstreamer] udpsrc port=5600 multicast-group=0.0.0.0 auto-multicast=true caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" ! rtph264depay ! nvv4l2decoder name=decoder enable-max-performance=1 ! video/x-raw(memory:NVMM) ! nvvidconv name=vidconv ! video/x-raw ! appsink name=mysink sync=false
[video]  created gstDecoder from rtp://@:5600
------------------------------------------------
gstDecoder video options:
------------------------------------------------
  -- URI: rtp://@:5600
     - protocol:  rtp
     - location:  0.0.0.0
     - port:      5600
  -- deviceType: ip
  -- ioType:     input
  -- codec:      H264
  -- codecType:  v4l2
  -- frameRate:  0
  -- numBuffers: 4
  -- zeroCopy:   true
  -- flipMethod: none
  -- loop:       0
  -- latency     10
------------------------------------------------
[OpenGL] glDisplay -- X screen 0 resolution:  1920x1080
[OpenGL] glDisplay -- X window resolution:    1920x1080
[OpenGL] glDisplay -- display device initialized (1920x1080)
[video]  created glDisplay from display://0
------------------------------------------------
glDisplay video options:
------------------------------------------------
  -- URI: display://0
     - protocol:  display
     - location:  0
  -- deviceType: display
  -- ioType:     output
  -- width:      1920
  -- height:     1080
  -- frameRate:  0
  -- numBuffers: 4
  -- zeroCopy:   true
------------------------------------------------

detectNet -- loading detection network model from:
          -- model        networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
          -- input_blob   'Input'
          -- output_blob  'NMS'
          -- output_count 'NMS_1'
          -- class_labels networks/SSD-Mobilenet-v2/ssd_coco_labels.txt
          -- threshold    0.500000
          -- batch_size   1

[TRT]    TensorRT version 10.3.0
[TRT]    loading NVIDIA plugins...
[TRT]    Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[TRT]    Registered plugin creator - ::BatchedNMS_TRT version 1
[TRT]    Registered plugin creator - ::BatchTilePlugin_TRT version 1
[TRT]    Registered plugin creator - ::Clip_TRT version 1
[TRT]    Registered plugin creator - ::CoordConvAC version 1
[TRT]    Registered plugin creator - ::CropAndResizeDynamic version 1
[TRT]    Registered plugin creator - ::CropAndResize version 1
[TRT]    Registered plugin creator - ::DecodeBbox3DPlugin version 1
[TRT]    Registered plugin creator - ::DetectionLayer_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_TRT version 1
[TRT]    Could not register plugin creator -  ::FlattenConcat_TRT version 1
[TRT]    Registered plugin creator - ::GenerateDetection_TRT version 1
[TRT]    Registered plugin creator - ::GridAnchor_TRT version 1
[TRT]    Registered plugin creator - ::GridAnchorRect_TRT version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 2
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 3
[TRT]    Registered plugin creator - ::LReLU_TRT version 1
[TRT]    Registered plugin creator - ::ModulatedDeformConv2d version 1
[TRT]    Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[TRT]    Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[TRT]    Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[TRT]    Registered plugin creator - ::NMSDynamic_TRT version 1
[TRT]    Registered plugin creator - ::NMS_TRT version 1
[TRT]    Registered plugin creator - ::Normalize_TRT version 1
[TRT]    Registered plugin creator - ::PillarScatterPlugin version 1
[TRT]    Registered plugin creator - ::PriorBox_TRT version 1
[TRT]    Registered plugin creator - ::ProposalDynamic version 1
[TRT]    Registered plugin creator - ::ProposalLayer_TRT version 1
[TRT]    Registered plugin creator - ::Proposal version 1
[TRT]    Registered plugin creator - ::PyramidROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::Region_TRT version 1
[TRT]    Registered plugin creator - ::Reorg_TRT version 2
[TRT]    Registered plugin creator - ::Reorg_TRT version 1
[TRT]    Registered plugin creator - ::ResizeNearest_TRT version 1
[TRT]    Registered plugin creator - ::ROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::ROIAlign_TRT version 2
[TRT]    Registered plugin creator - ::RPROI_TRT version 1
[TRT]    Registered plugin creator - ::ScatterElements version 2
[TRT]    Registered plugin creator - ::ScatterElements version 1
[TRT]    Registered plugin creator - ::ScatterND version 1
[TRT]    Registered plugin creator - ::SpecialSlice_TRT version 1
[TRT]    Registered plugin creator - ::Split version 1
[TRT]    Registered plugin creator - ::VoxelGeneratorPlugin version 1
[TRT]    completed loading NVIDIA plugins.
[TRT]    detected model format - UFF  (extension '.uff')
[TRT]    desired precision specified for GPU: FASTEST
[TRT]    requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT]    [MemUsageChange] Init CUDA: CPU +13, GPU +0, now: CPU 43, GPU 2229 (MiB)
[TRT]    Trying to load shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    Loaded shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    [MemUsageChange] Init builder kernel library: CPU +927, GPU +744, now: CPU 1013, GPU 3018 (MiB)
[TRT]    CUDA lazy loading is enabled.
[TRT]    native precisions detected for GPU:  FP32, FP16, INT8
[TRT]    selecting fastest native precision for GPU:  FP16
[TRT]    could not find engine cache /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.1.1.100300.GPU.FP16.engine
[TRT]    cache file invalid, profiling network model on device GPU
[TRT]    [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 87, GPU 3019 (MiB)
[TRT]    Trying to load shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    Loaded shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    [MemUsageChange] Init builder kernel library: CPU +926, GPU +147, now: CPU 1013, GPU 3166 (MiB)
[TRT]    CUDA lazy loading is enabled.
[TRT]    device GPU, loading /usr/local/bin/ /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
[TRT]    TensorRT 10.3 does not support legacy caffe models
[TRT]    device GPU, failed to load networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
[TRT]    detectNet -- failed to initialize.
detectnet:  failed to load detectNet model

Hi,

Yes, the Caffe support is removed from TensorRT 10.
We recommend to try our generative AI tutorial instead:

Thanks.

Is it mean we can’t use jetson-util’s imagenet/detectnet anymore?

Hi,
We will check if there is imagenet/detectnet model based on TensorRT.

Sorry, it’s jetson-inference.

Here is another thread about this error: Can't use any model with jetson-inference - #3 by AastaLLL

I hope imagenet/detect can work as it is used to work. And we are not care about if it’s cafffe model or onnx model, just get those commands to work.

Hi,

You will need an ONNX model to run with TensorRT 10.
However, due to the API changes, the resnet18-tagging-voc model still causes a segmentation fault as some corresponding modifications are required.

To run jetson-inference without changing, please set up your device with JetPack 6.0/TensorRT 8.6.
A newer version can work but some modifications are required and the Caffe/UFF-based models are not supported anymore.

As we now focus on the Jetson AI lab tutorial, the resources for maintaining jetson-inference are relatively limited.
However, the GitHub is open-sourced, you can do it directly.

Below are the API changes for TensorRT 10 for your reference:
https://docs.nvidia.com/deeplearning/tensorrt/latest/api/migration-guide.html#c

If you are looking for the detection model, Ultralytics YOLO can work on TensorRT 10.
Please find a tutorial below:

Thanks.