Segnet/poseNet Segmentation fault (core dumped) for Jetpack6.2

Any ideas?

  • version
commit c038530ebf718e6867c4458c3e439406020732ff (HEAD -> master, origin/master, origin/HEAD)
Author: Dustin Franklin <dustinf@nvidia.com>
Date:   Wed Oct 16 06:56:03 2024 -0400

    updates for TRT10

------ 
Software part of jetson-stats 4.3.1 - (c) 2024, Raffaello Bonghi
Model: NVIDIA Jetson Orin Nano Developer Kit - Jetpack 6.2 [L4T 36.4.3]
NV Power Mode[0]: 15W
Serial Number: [XXX Show with: jetson_release -s XXX]
Hardware:
 - P-Number: p3767-0005
 - Module: NVIDIA Jetson Orin Nano (Developer kit)
Platform:
 - Distribution: Ubuntu 22.04 Jammy Jellyfish
 - Release: 5.15.148-tegra
jtop:
 - Version: 4.3.1
 - Service: Active
Libraries:
 - CUDA: 12.6.68
 - cuDNN: 9.3.0.75
 - TensorRT: 10.3.0.30
 - VPI: 3.2.4
 - OpenCV: 4.11.0 - with CUDA: YES
DeepStream C/C++ SDK version: 7.1

Python Environment:
Python 3.10.12
    GStreamer:                   YES (1.20.3)
  NVIDIA CUDA:                   YES (ver 12.6, CUFFT CUBLAS FAST_MATH)
         OpenCV version: 4.11.0  CUDA True
           YOLO version: 8.3.65
         PYCUDA version: 2024.1.2
          Torch version: 2.5.1+l4t36.4
    Torchvision version: 0.20.0a0+afc54f7
 DeepStream SDK version: 1.2.0
onnxruntime-gpu version: 1.19.2

[TRT]    binding to input 0 input_0  binding index:  0
[TRT]    binding to input 0 input_0  dims (b=1 c=3 h=320 w=320) size=1228800
[TRT]    binding to output 0 output_0  binding index:  1
[TRT]    binding to output 0 output_0  dims (b=1 c=1 h=21 w=10) size=8400
Segmentation fault (core dumped)

You can give this fix a try - More fixes for TensorRT 10 by ligaz · Pull Request #1925 · dusty-nv/jetson-inference · GitHub

1 Like

Sorry, it worked for segnet/posenet.

There is another issue with imagenet: TensorRT 10.3 does not support legacy caffe models for Jetpack6.2


It didn’t work. It seems have other issues.

daniel@daniel-nvidia:~/Work/jetson-fpv/module/jetson-inference$
daniel@daniel-nvidia:~/Work/jetson-fpv/module/jetson-inference$
daniel@daniel-nvidia:~/Work/jetson-fpv/module/jetson-inference$
daniel@daniel-nvidia:~/Work/jetson-fpv/module/jetson-inference$
daniel@daniel-nvidia:~/Work/jetson-fpv/module/jetson-inference$
daniel@daniel-nvidia:~/Work/jetson-fpv/module/jetson-inference$
daniel@daniel-nvidia:~/Work/jetson-fpv/module/jetson-inference$ imagenet rtp://@:5600

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.687: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_dsanalytics.so': libyaml-cpp.so.0.7: cannot open shared object file: No such file or directory

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.688: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_3dbridge.so': libyaml-cpp.so.0.7: cannot open shared object file: No such file or directory

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.688: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_msgconv.so': libyaml-cpp.so.0.7: cannot open shared object file: No such file or directory

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.690: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_multistream.so': libyaml-cpp.so.0.7: cannot open shared object file: No such file or directory

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.691: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_3dfilter.so': libyaml-cpp.so.0.7: cannot open shared object file: No such file or directory

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.696: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_infer.so': libyaml-cpp.so.0.7: cannot open shared object file: No such file or directory

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.701: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_deepstream_bins.so': libyaml-cpp.so.0.7: cannot open shared object file: No such file or directory

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.704: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.704: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_3dmixer.so': libyaml-cpp.so.0.7: cannot open shared object file: No such file or directory

(gst-plugin-scanner:5550): GStreamer-WARNING **: 08:08:32.727: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
(Argus) Error EndOfFile: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 350)
(Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 366)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
(Argus) Error EndOfFile: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error EndOfFile:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 92)
[gstreamer] initialized gstreamer, version 1.20.3.0
[gstreamer] gstDecoder -- creating decoder for 0.0.0.0
[gstreamer] gstDecoder -- resource discovery not supported for RTP/WebRTC streams
[gstreamer] gstDecoder -- defaulting to H264 codec (you can change this with the --input-codec option)
[gstreamer] gstDecoder -- pipeline string:
[gstreamer] udpsrc port=5600 multicast-group=0.0.0.0 auto-multicast=true caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" ! rtph264depay ! nvv4l2decoder name=decoder enable-max-performance=1 ! video/x-raw(memory:NVMM) ! nvvidconv name=vidconv ! video/x-raw ! appsink name=mysink sync=false
[video]  created gstDecoder from rtp://@:5600
------------------------------------------------
gstDecoder video options:
------------------------------------------------
  -- URI: rtp://@:5600
     - protocol:  rtp
     - location:  0.0.0.0
     - port:      5600
  -- deviceType: ip
  -- ioType:     input
  -- codec:      H264
  -- codecType:  v4l2
  -- frameRate:  0
  -- numBuffers: 4
  -- zeroCopy:   true
  -- flipMethod: none
  -- loop:       0
  -- latency     10
------------------------------------------------
[OpenGL] glDisplay -- X screen 0 resolution:  1920x1080
[OpenGL] glDisplay -- X window resolution:    1920x1080
[OpenGL] glDisplay -- display device initialized (1920x1080)
[video]  created glDisplay from display://0
------------------------------------------------
glDisplay video options:
------------------------------------------------
  -- URI: display://0
     - protocol:  display
     - location:  0
  -- deviceType: display
  -- ioType:     output
  -- width:      1920
  -- height:     1080
  -- frameRate:  0
  -- numBuffers: 4
  -- zeroCopy:   true
------------------------------------------------

imageNet -- loading classification network model from:
         -- prototxt     networks/Googlenet/googlenet.prototxt
         -- model        networks/Googlenet/bvlc_googlenet.caffemodel
         -- class_labels networks/ilsvrc12_synset_words.txt
         -- input_blob   'data'
         -- output_blob  'prob'
         -- batch_size   1

[TRT]    TensorRT version 10.3.0
[TRT]    loading NVIDIA plugins...
[TRT]    Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[TRT]    Registered plugin creator - ::BatchedNMS_TRT version 1
[TRT]    Registered plugin creator - ::BatchTilePlugin_TRT version 1
[TRT]    Registered plugin creator - ::Clip_TRT version 1
[TRT]    Registered plugin creator - ::CoordConvAC version 1
[TRT]    Registered plugin creator - ::CropAndResizeDynamic version 1
[TRT]    Registered plugin creator - ::CropAndResize version 1
[TRT]    Registered plugin creator - ::DecodeBbox3DPlugin version 1
[TRT]    Registered plugin creator - ::DetectionLayer_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_TRT version 1
[TRT]    Could not register plugin creator -  ::FlattenConcat_TRT version 1
[TRT]    Registered plugin creator - ::GenerateDetection_TRT version 1
[TRT]    Registered plugin creator - ::GridAnchor_TRT version 1
[TRT]    Registered plugin creator - ::GridAnchorRect_TRT version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 2
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 3
[TRT]    Registered plugin creator - ::LReLU_TRT version 1
[TRT]    Registered plugin creator - ::ModulatedDeformConv2d version 1
[TRT]    Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[TRT]    Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[TRT]    Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[TRT]    Registered plugin creator - ::NMSDynamic_TRT version 1
[TRT]    Registered plugin creator - ::NMS_TRT version 1
[TRT]    Registered plugin creator - ::Normalize_TRT version 1
[TRT]    Registered plugin creator - ::PillarScatterPlugin version 1
[TRT]    Registered plugin creator - ::PriorBox_TRT version 1
[TRT]    Registered plugin creator - ::ProposalDynamic version 1
[TRT]    Registered plugin creator - ::ProposalLayer_TRT version 1
[TRT]    Registered plugin creator - ::Proposal version 1
[TRT]    Registered plugin creator - ::PyramidROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::Region_TRT version 1
[TRT]    Registered plugin creator - ::Reorg_TRT version 2
[TRT]    Registered plugin creator - ::Reorg_TRT version 1
[TRT]    Registered plugin creator - ::ResizeNearest_TRT version 1
[TRT]    Registered plugin creator - ::ROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::ROIAlign_TRT version 2
[TRT]    Registered plugin creator - ::RPROI_TRT version 1
[TRT]    Registered plugin creator - ::ScatterElements version 2
[TRT]    Registered plugin creator - ::ScatterElements version 1
[TRT]    Registered plugin creator - ::ScatterND version 1
[TRT]    Registered plugin creator - ::SpecialSlice_TRT version 1
[TRT]    Registered plugin creator - ::Split version 1
[TRT]    Registered plugin creator - ::VoxelGeneratorPlugin version 1
[TRT]    completed loading NVIDIA plugins.
[TRT]    detected model format - caffe  (extension '.caffemodel')
[TRT]    desired precision specified for GPU: FASTEST
[TRT]    requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT]    [MemUsageChange] Init CUDA: CPU +2, GPU +0, now: CPU 44, GPU 2689 (MiB)
[TRT]    Trying to load shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    Loaded shared library libnvinfer_builder_resource.so.10.3.0
d[TRT]    [MemUsageChange] Init builder kernel library: CPU +927, GPU +763, now: CPU 1014, GPU 3496 (MiB)
[TRT]    CUDA lazy loading is enabled.
[TRT]    native precisions detected for GPU:  FP32, FP16, INT8
[TRT]    selecting fastest native precision for GPU:  FP16
[TRT]    could not find engine cache /usr/local/bin/networks/Googlenet/bvlc_googlenet.caffemodel.1.1.100300.GPU.FP16.engine
[TRT]    cache file invalid, profiling network model on device GPU
[TRT]    [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 88, GPU 3496 (MiB)
[TRT]    Trying to load shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    Loaded shared library libnvinfer_builder_resource.so.10.3.0
[TRT]    [MemUsageChange] Init builder kernel library: CPU +926, GPU +141, now: CPU 1014, GPU 3637 (MiB)
[TRT]    CUDA lazy loading is enabled.
[TRT]    device GPU, loading /usr/local/bin/networks/Googlenet/googlenet.prototxt /usr/local/bin/networks/Googlenet/bvlc_googlenet.caffemodel
[TRT]    TensorRT 10.3 does not support legacy caffe models
[TRT]    device GPU, failed to load networks/Googlenet/bvlc_googlenet.caffemodel
[TRT]    failed to load networks/Googlenet/bvlc_googlenet.caffemodel
[TRT]    imageNet -- failed to initialize.
imagenet:  failed to initialize imageNet
daniel@daniel-nvidia:~/Work/jetson-fpv/module/jetson-inference$ 

model file is where it should be:

daniel@daniel-nvidia:~/Work/jetson-fpv$ ls /usr/local/bin/networks/Googlenet/googlenet.prototxt
/usr/local/bin/networks/Googlenet/googlenet.prototxt
daniel@daniel-nvidia:~/Work/jetson-fpv$ ls /usr/local/bin/networks/Googlenet/bvlc_googlenet.caffemodel
/usr/local/bin/networks/Googlenet/bvlc_googlenet.caffemodel

Hi,
For the imagenet issue, please update the topic thread:
TensorRT 10.3 does not support legacy caffe models for Jetpack6.2