Please provide the following info (check/uncheck the boxes after creating this topic): Software Version
DRIVE OS Linux 5.2.6
[*] DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other
Host Machine Version
[*] native Ubuntu 18.04
other
Hi, The model loading using tensorRT_optimization tool conversion has the following blobName not found problem, please help provide solution information, thanks
Dear @Marin_ma,
Could you share your model? If you are using inputBlobs, outputblobs parameters, please double check if it has those blobs in the model.
The model is too large to upload.
optimization command:
./tensorRT_optimization --modelType=onxx --onnxFile=~/myNetwork.onnx
How does the optimized model check for blob?
Dear @Marin_ma,
You may share model via any drive to reproduce the issue locally. Could you check if trtexec is working with out any issue to generate TensorRT model?
I want to replace the tensorRT model in the sample_object_detector_tracker with my own through tensorRT_optimization converted model, but model parsing blobName not found. How can I define BlobName based on the model structure?
sample_object_detector_tracker/main.cpp
// Get coverage and bounding box blob indices
const char* coverageBlobName = "coverage";
const char* boundingBoxBlobName = "bboxes";
CHECK_DW_ERROR(dwDNN_getOutputIndex(&m_cvgIdx, coverageBlobName, m_dnn));
CHECK_DW_ERROR(dwDNN_getOutputIndex(&m_bboxIdx, boundingBoxBlobName, m_dnn));
Dear @Marin_ma,
Please check for the final output array names of your model. In the sample, they are denoted as coverage, bboxes. Does your model also outputs bounding boxes and confidence values as output?
Dear @SivaRamaKrishnaNV
Yeah, bounding boxes and confidence values have output. yolov3.onnx model output array names is “082_convolutional 094_convolutional 106_convolutional”
Model conversion process
yolov3-416.weights —> yolov3.onnx —> yolov3.trt(tensorRT_optimization)
Does tensorRT have a visualization tool that can view the structure of the model? For example tensorboard, netron
Warning: SamplesDataPath::getBasePathFor(): 'DATA_ROOT' not found in search paths [/home/nvidia/mls/nvidia/dw4.0/data | /home/nvidia/mls/nvidia/dw4.0.runfiles/av/data | /home/nvidia/mls/nvidia/dw4.0/data | /home/nvidia/mls/nvidia/data | /home/nvidia/mls/data | /home/nvidia/data | /home/data]
Warning: SamplesDataPath::getBasePathFor(): 'DATA_ROOT' not found in search paths [/home/nvidia/mls/nvidia/dw4.0/data | /home/nvidia/mls/nvidia/dw4.0.runfiles/av/data | /home/nvidia/mls/nvidia/dw4.0/data | /home/nvidia/mls/nvidia/data | /home/nvidia/mls/data | /home/nvidia/data | /home/data]
[07-06-2022 14:37:38] Platform: Detected Drive Xavier P3479
[07-06-2022 14:37:38] TimeSource: monotonic epoch time offset is 1654287362845722
[07-06-2022 14:37:38] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to eth0
[07-06-2022 14:37:38] TimeSource Eth: Lost PTP time synchronizaton. Synchronized time will not be available from this timesource.
[07-06-2022 14:37:38] TimeSource: Could not detect valid PTP time source at 'eth0'. Fallback to CLOCK_MONOTONIC.
[07-06-2022 14:37:38] Platform: number of GPU devices detected 1
[07-06-2022 14:37:38] Platform: currently selected GPU device integrated ID 0
[07-06-2022 14:37:38] Context::mountResourceCandidateDataPath resource FAILED to mount from '/home/nvidia/mls/nvidia/dw4.0/data/': VirtualFileSystem: Failed to mount '/home/nvidia/mls/nvidia/dw4.0/data/[.pak]'
[07-06-2022 14:37:38] Context::findDataRootInPathWalk: Could not find ./data/DATA_ROOT in upto 7 parent directories from /usr/local/driveworks-4.0/targets/aarch64-Linux/lib/libdw_base.so.4.0
[07-06-2022 14:37:38] Context::findDataRootInPathWalk: Could not find ./data/DATA_ROOT in upto 7 parent directories from /usr/local/driveworks-4.0/targets/aarch64-Linux/lib/libdw_base.so.4.0
[07-06-2022 14:37:38] SDK: No resources(.pak) mounted, some modules will not function properly
[07-06-2022 14:37:38] SDK: Create NvMediaDevice
[07-06-2022 14:37:38] SDK: Create NvMedia2D
[07-06-2022 14:37:38] egl::Display: found 1 EGL devices
[07-06-2022 14:37:38] egl::Display: use drm device: /dev/dri/card0
[07-06-2022 14:37:38] TimeSource: monotonic epoch time offset is 1654287362845722
[07-06-2022 14:37:38] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to eth0
[07-06-2022 14:37:38] TimeSource Eth: Lost PTP time synchronizaton. Synchronized time will not be available from this timesource.
[07-06-2022 14:37:38] TimeSource: Could not detect valid PTP time source at 'eth0'. Fallback to CLOCK_MONOTONIC.
[07-06-2022 14:37:38] Initialize DriveWorks SDK v4.0.0
[07-06-2022 14:37:38] Release build with GNU 7.3.1 from no-gitversion-build against Drive PDK v5.2.6.0
[07-06-2022 14:37:38] SensorFactory::createSensor() -> camera.virtual, camera-group=a,camera-index=0,camera-type=ar0231-rccb-bae-sf3324,input-type=video,offscreen=2,profiling=1,slave=0,tensorRT_model=../../model/yolov3_rt.bin,video=../../dds/nvidia/dw4.0/video_0.h264
[07-06-2022 14:37:38] CameraVirtual: defaulting to non SIPL
[07-06-2022 14:37:38] CameraBase: pool size set to 8
[07-06-2022 14:37:38] DecoderNvMedia: the format of the decoder YUV420 planar will be replaced with yuv420 semiplanar in the next release
[07-06-2022 14:37:38] CameraVirtualNvMedia: no seek table found at ../../dds/nvidia/dw4.0/video_0.h264.seek, seeking is not available.
SimpleCamera: Camera image: 1280x800
Camera image with 1280x800 at 30 FPS
[07-06-2022 14:37:43] Loaded engine size: 573 MB
[07-06-2022 14:37:43] [MemUsageSnapshot] deserializeCudaEngine begin: CPU 340 MB, GPU 8883 MB
[07-06-2022 14:37:49] [MemUsageChange] Init cuDNN: CPU +922, GPU +1123, now: CPU 1262, GPU 10744 (MB)
[07-06-2022 14:37:50] [MemUsageChange] Init cuBlas: CPU +227, GPU +298, now: CPU 1489, GPU 11042 (MB)
[07-06-2022 14:37:50] Deserialize required 7232008 microseconds.
[07-06-2022 14:37:50] [MemUsageSnapshot] deserializeCudaEngine end: CPU 1488 MB, GPU 11042 MB
[07-06-2022 14:37:50] [MemUsageChange] Init cuDNN: CPU +1, GPU +0, now: CPU 1489, GPU 11042 (MB)
[07-06-2022 14:37:50] [MemUsageChange] Init cuBlas: CPU +0, GPU +0, now: CPU 1489, GPU 11042 (MB)
[07-06-2022 14:37:53] DNN: Missing or incompatible parameter in metadata (tonemapType). Parameter is set to default value. See dwDataConditioner for default values.
m_largeIdx= 0m_mediumIdx= 1m_smallIdx= 2
[07-06-2022 14:37:53] DataConditioner: Scale transformation has been configured with 1.0000000.
[07-06-2022 14:37:53] DataConditioner: Mean value subtract transformation has been configured with {0.00000, 0.00000, 0.00000}.
[07-06-2022 14:37:53] DataConditioner: Standard deviation has been configured with {1.00000, 1.00000, 1.00000}.
m_detectionRegion.x= 0
[07-06-2022 14:37:53] ../rtExt/cask/caskConvolutionRunner.cpp (30) - Cask Error in checkCaskExecError<false>: 1 (Cask Convolution execution)
[07-06-2022 14:37:53] FAILED_EXECUTION: std::exception
[07-06-2022 14:37:53] Driveworks exception thrown: DW_INTERNAL_ERROR: TensorRT: Inference failed.
terminate called after throwing an instance of 'std::runtime_error'
what(): [2022-06-07 14:37:53] DW Error DW_INTERNAL_ERROR executing DW function:
dwDNN_inferRaw(m_dnnOutputsDevice, &m_dnnInputDevice, 1U, m_dnn)
at /home/dcuman/mls/nvidia/xavier/driveworks-4.0/samples/src/dnn/sample_object_detector_tracker_yolov3/main.cpp:374
Aborted
Dear @victor.f.perez,
The error indicate the name mismatch of output blob. Please file a seperate topic with model details and output bob names used in sample to double check.