Basic Object Detector and Tracker Sample：Yellow bounding box is very small so i can't do Object Detector and Tracker in DriveWorks

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
[√] DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
[√] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
[√] DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
[√] other

Host Machine Version
[√] native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Hi，
When i use “Basic Object Detector and Tracker Sample”, i find that the yellow bounding box in the window is very small, so i can’t do Object Detector and Tracker.

image
image1770×1328 201 KB

I use the Entron F008A120RM0AES,  and the command is following:

image
image1311×54 4.8 KB

./sample_object_detector_tracker --input-type=camera --camera-type=F008A120RM0A --camera-group=c --camera-index=1

The log is following:
       nvidia@tegra-ubuntu:/usr/local/driveworks/samples/bin$ sudo ./sample_object_detector_tracker --input-type=camera --camera-type=F008A120RM0A --camera-group=c --camera-index=1

ProgramArguments: Missing argument ‘dwTracePath’ requested
[22-10-2022 12:16:29] Platform: Detected Drive Orin P3710
[22-10-2022 12:16:29] TimeSource: monotonic epoch time offset is 1666437705065534
[22-10-2022 12:16:29] TimeSourceVibranteLinux: detect valid PTP interface mgbe2_0
[22-10-2022 12:16:29] TimeSource Nvpss : PTP ioctl returned error. Synchronized time will not be available from this timesource.
[22-10-2022 12:16:29] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to mgbe2_0
[22-10-2022 12:16:29] PTP Time is available from Eth Driver
[22-10-2022 12:16:29] Adding variable DW_Base:DW_Version
[22-10-2022 12:16:29] Addded variable DW_Base:DW_Version
[22-10-2022 12:16:29] Platform: number of GPU devices detected 1
[22-10-2022 12:16:29] Platform: currently selected GPU device 0, Resource Data Dir: trt_08_04_12_05, Arch: ga10b
[22-10-2022 12:16:29] Platform: currently selected GPU device integrated ID 0
[22-10-2022 12:16:29] CUDLAEngine:getDLACountandCheck: CUDLA version is = 1002001
[22-10-2022 12:16:29] CUDLAEngine:getDLACountandCheck: Number of DLA devices = 2
[22-10-2022 12:16:29] ========== DLA Device 0 Attributes ==========
[22-10-2022 12:16:29] UVA supported = 0
[22-10-2022 12:16:29] Device version = 1
[22-10-2022 12:16:29] Device 0 Check PASSED
[22-10-2022 12:16:29] ========== DLA Device 1 Attributes ==========
[22-10-2022 12:16:29] UVA supported = 0
[22-10-2022 12:16:29] Device version = 1
[22-10-2022 12:16:29] Device 1 Check PASSED
[22-10-2022 12:16:29] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘./resources’: VirtualFileSystem: Failed to mount ‘./resources/resources.pak’
[22-10-2022 12:16:29] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks-5.8/samples/bin/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks-5.8/samples/bin/data/resources.pak’
[22-10-2022 12:16:29] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks/data
[22-10-2022 12:16:29] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks/data/resources.pak’
[22-10-2022 12:16:29] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.8/data
[22-10-2022 12:16:29] Context::mountResourceCandidateDataPath resource FAILED to mount from ‘/usr/local/driveworks-5.8/data’: VirtualFileSystem: Failed to mount ‘/usr/local/driveworks-5.8/data/resources.pak’
[22-10-2022 12:16:29] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks/lib/libdw_base.so.5.8
[22-10-2022 12:16:29] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.8/targets/aarch64-Linux/lib/libdw_base.so.5.8
[22-10-2022 12:16:29] SDK: No resources(.pak) mounted, some modules will not function properly
[22-10-2022 12:16:29] SDK: Create NvMediaDevice
[22-10-2022 12:16:29] SDK: use EGL display as provided
[22-10-2022 12:16:29] TimeSource: monotonic epoch time offset is 1666437705065534
[22-10-2022 12:16:29] TimeSourceVibranteLinux: detect valid PTP interface mgbe2_0
[22-10-2022 12:16:29] TimeSource Nvpss : PTP ioctl returned error. Synchronized time will not be available from this timesource.
[22-10-2022 12:16:29] TimeSource: Could not detect valid PTP time source at nvpps. Fallback to mgbe2_0
[22-10-2022 12:16:29] PTP Time is available from Eth Driver
[22-10-2022 12:16:29] Initialize DriveWorks SDK v5.8.82
[22-10-2022 12:16:29] Release build with GNU 9.3.0 from buildbrain-branch-0-g33d6f4a1440 against Drive PDK v6.0.5.0
[22-10-2022 12:16:29] SensorFactory::createSensor() → time.nvpps, nvpps-device=/dev/nvpps0
[22-10-2022 12:16:29] TimeSensor: initialized with no UTC time reference.
[22-10-2022 12:16:29] EndpointNVPPS: started on /dev/nvpps0
[22-10-2022 12:16:29] Initialize DriveWorks VisualizationSDK v5.8.82
[22-10-2022 12:16:29] Initialize DriveWorksGL SDK v5.8.82
[22-10-2022 12:16:29] SensorFactory::createSensor() → camera.gmsl, camera-type=F008A120RM0A,camera-group=c,slave=0,serialize=false,camera-count=4,camera-mask=0010
[22-10-2022 12:16:29] CameraGMSLSelector: cameraCount is a deprecated parameter, please remove as it has no more effect
[22-10-2022 12:16:29] CameraGMSLSelector: camera-mask is a deprecated parameter, please remove from your rig and replace with parameter link
[22-10-2022 12:16:29] CameraGMSLSelector: camera-mask 0010 was automatically translated to link 1. Note that it will be overridden by parameter link if present
[22-10-2022 12:16:29] CameraGMSLSelector: missing siblingIndex to be able to map to new camera.gmsl parameter called link, assuming link 0
[22-10-2022 12:16:29] CameraGMSLSelector: note the name doesn’t correspond to the old supported cameras, DW will pass the name as is to the SIPL library. Please consider replacing camera-type with camera-name
[22-10-2022 12:16:29] CameraGMSLSelector: creating CameraClient from remapped old gmsl parameters to camera-type=F008A120RM0A,camera-group=c,slave=0,serialize=false,camera-count=4,camera-mask=0010,camera-name=F008A120RM0A,interface=csi-ef,link=1
[22-10-2022 12:16:29] CameraGMSLSelector: missing ‘output-format=processed/yuv’ in the sensor parameters, frames might be captured with bad sensor settings(AE/AWB) and reprocess may result image quality issues
[22-10-2022 12:16:29] CameraBase: pool size set to 8
[22-10-2022 12:16:29] SensorFactory::createSensor() → camera.gmsl.master,
[22-10-2022 12:16:29] CameraMaster::parseDevBlock Getting device info list.
[22-10-2022 12:16:29] devBlock: 0 Slave = 0 Interface = csi-ef Camera_name = F008A120RM0A Link = 1
[22-10-2022 12:16:29] Camera Match Name: F008A120RM0A Description: Entron F008A120RM0A module - 120-deg FOV, 24-bit capture, MIPI-AR0820, MAX9295 linkIndex: 4294967295 serInfo.Name: MAX9295
[22-10-2022 12:16:29] Client, Setting up information for camera ID 0
[22-10-2022 12:16:29] Client, successfully found info for camera ID 0 bound to id 0
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
[22-10-2022 12:16:29] CameraClient: using NITO found at /opt/nvidia/nvmedia/nit/F008A120RM0A.nito
SimpleCamera: Camera image: 3848x2168
[22-10-2022 12:16:29] ImageStreamer(NvMedia → CUDA)
Camera image with 3848x2168 at 30 FPS
[22-10-2022 12:16:29] FileHeader: file header created with development build (patch = -1), assumingcompatibility
[22-10-2022 12:16:29] FileHeader: file header created with development build (patch = -1), assumingcompatibility
[22-10-2022 12:16:30] [MemUsageChange] Init CUDA: CPU +263, GPU +0, now: CPU 453, GPU 5765 (MiB)
[22-10-2022 12:16:30] Loaded engine size: 23 MiB
[22-10-2022 12:16:30] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[22-10-2022 12:16:30] Deserialization required 15391 microseconds.
[22-10-2022 12:16:30] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +23, now: CPU 0, GPU 23 (MiB)
[22-10-2022 12:16:30] Total per-runner device persistent memory is 719360
[22-10-2022 12:16:30] Total per-runner host persistent memory is 122368
[22-10-2022 12:16:30] Allocated activation device memory of size 69010432
[22-10-2022 12:16:30] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +67, now: CPU 0, GPU 90 (MiB)
[22-10-2022 12:16:30] DNN: Missing or incompatible parameter in metadata (tonemapType). Parameter is set to default value. See dwDataConditioner for default values.
[22-10-2022 12:16:30] DNN: Missing or incompatible parameter in metadata (channelIdx). Parameter is set to default value. See dwDataConditioner for default values.
[22-10-2022 12:16:30] DNN: Missing or incompatible parameter in metadata (channelIdx). Parameter is set to default value. See dwDataConditioner for default values.
[22-10-2022 12:16:30] DataConditioner: Scale transformation has been configured with 1.000000.
[22-10-2022 12:16:30] DataConditioner: Mean value subtract transformation has been configured with {0.0000, 0.0000, 0.0000}.
[22-10-2022 12:16:30] DataConditioner: Standard deviation has been configured with {1.0000, 1.0000, 1.0000}.
[22-10-2022 12:16:30] 1
[22-10-2022 12:16:30] SIPLMaster::SIPLMaster: Setting up master camera
Could not create NVCCP shared memory nvccpmutex
[22-10-2022 12:16:30] Platform:
[22-10-2022 12:16:30] Platform Config:
[22-10-2022 12:16:30] Description:
[22-10-2022 12:16:30] Number of device blocks: 1
[22-10-2022 12:16:30] Device Block : 0
[22-10-2022 12:16:30] csiPort: 8
[22-10-2022 12:16:30] i2cDevice: 2
[22-10-2022 12:16:30] Deserializer Name: MAX96712
[22-10-2022 12:16:30] Deserializer Description: Maxim 96712 Aggregator
[22-10-2022 12:16:30] Deserializer i2cAddress: 41
[22-10-2022 12:16:30] Simulator Mode: 0
[22-10-2022 12:16:30] Passive Mode: 0
[22-10-2022 12:16:30] Phy Mode: 0
[22-10-2022 12:16:30] Number of camera modules: 1
[22-10-2022 12:16:30] CameraModule index: 1
[22-10-2022 12:16:30] Name :F008A120RM0A
[22-10-2022 12:16:30] Description: Entron F008A120RM0A module - 120-deg FOV, 24-bit capture, MIPI-AR0820, MAX9295
[22-10-2022 12:16:30] Serializer name: MAX9295
[22-10-2022 12:16:30] Serializer description: Maxim 9295 Serializer
[22-10-2022 12:16:30] Serializer i2cAdress: 98
[22-10-2022 12:16:30] EEPROM name: N24C64
[22-10-2022 12:16:30] EEPROM i2cAddress: 84
[22-10-2022 12:16:30] EEPROM description: N24C64 EEPROM
[22-10-2022 12:16:30] Sensor ID: 0
[22-10-2022 12:16:30] Sensor name: AR0820
[22-10-2022 12:16:30] Sensor description: OnSemi AR0820 Sensor
[22-10-2022 12:16:30] Sensor i2cAddress: 16
[22-10-2022 12:16:30] Sensor isTPGEnabled: 0
[22-10-2022 12:16:30] Sensor isTriggerMode: 1
[22-10-2022 12:16:30] cfa: 35
[22-10-2022 12:16:30] embeddedTopLines: 6
[22-10-2022 12:16:30] embeddedBottomLines: 0
[22-10-2022 12:16:30] inputFormat: 8
[22-10-2022 12:16:30] height: 2168
[22-10-2022 12:16:30] width: 3848
[22-10-2022 12:16:30] fps: 30.000000
[22-10-2022 12:16:30] Embedded Data: 1
[22-10-2022 12:16:30] CameraMaster available outputs: 3
[22-10-2022 12:16:30] CameraClient: setting pipeline config for camera ID 0
[22-10-2022 12:16:30] CameraMaster: master initiation
MAX96712: Revision 5 detected
MAX96712 Link 1: PHY optimization was enabled
MAX9295: Revision 8 detected!
Sensor AR0820 GRBG Rev 2.1 detected!
[22-10-2022 12:16:30] CameraMaster: Bootstrap NvSIPL and ClientPipelines complete
[22-10-2022 12:16:30] CameraClient::readEEPROMInfo:
[22-10-2022 12:16:30] mmIntrinsicModelID: 27
[22-10-2022 12:16:30] mmIntrinsicCoeff[0]: -0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[1]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[2]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[3]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[4]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[5]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[6]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[7]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[8]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[9]: 1.892143e+11
[22-10-2022 12:16:30] mmIntrinsicCoeff[10]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[11]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[12]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[13]: 0.000000
[22-10-2022 12:16:30] mmIntrinsicCoeff[14]: 0.000170
[22-10-2022 12:16:30] mmIntrinsicCoeff[15]: 8.000000
[22-10-2022 12:16:30] mmIntrinsicMeasuredFlagID: 237
[22-10-2022 12:16:30] fwVersionMajor: 0
[22-10-2022 12:16:30] fwVersionMinor: 0
[22-10-2022 12:16:30] fwVersionPatch: 0
[22-10-2022 12:16:30] CameraMaster: starting…
[22-10-2022 12:16:30] CameraClient: allocating image pool for ICP
[22-10-2022 12:16:30] Allocating NvSci group
[22-10-2022 12:16:30] CameraClient: allocate image pool for 0
[22-10-2022 12:16:30] Allocating NvSci ISP0 images
[22-10-2022 12:16:30] CameraClient: format not specified. Using h264 in Codec Header.
[22-10-2022 12:16:31] SIPLClient: Acquisition started
[22-10-2022 12:16:31] SIPLClient: Acquisition started
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] CameraClient: Notification received from pipeline index:0 of type: NOTIF_WARN_ICP_FRAME_DROP
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] CameraClient: Notification received from pipeline index:0 of type: NOTIF_WARN_ICP_FRAME_DROP
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] CameraClient: Notification received from pipeline index:0 of type: NOTIF_WARN_ICP_FRAME_DROP
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] CameraClient: Notification received from pipeline index:0 of type: NOTIF_WARN_ICP_FRAME_DROP
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] CameraClient: Notification received from pipeline index:0 of type: NOTIF_WARN_ICP_FRAME_DROP
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] DNN: run infer internal batchsize 1 is not needed after 8.4.10.4.
[22-10-2022 12:16:31] SIPLDeviceBlockNotificationHandler: Queue timeout

2

Dear @zhaohm8,
May I know why you are using DRIVE OS 6.0.5? Is possible to flash latest release 6.0.8.1 using docker?

4

Hi @SivaRamaKrishnaNV ,
The entire team is using 6.0.5， and some vendors provide us with camera drivers based on 6.0.5, so maybe I can’t switch to 6.0.8 for a while.

5

Dear @zhaohm8,
Got it. You can modify the detection region params as per your need in the sample inside onInitialize() function

 m_detectionRegion.width = std::min(static_cast<uint32_t>(m_networkInputDimensions.width),
                                               m_imageWidth);
            m_detectionRegion.height = std::min(static_cast<uint32_t>(m_networkInputDimensions.height),
                                                m_imageHeight);
            m_detectionRegion.x = (m_imageWidth - m_detectionRegion.width) / 2; 
            m_detectionRegion.y = (m_imageHeight - m_detectionRegion.height) / 2;
7

Hi @SivaRamaKrishnaNV ,
Thanlks, i will try!

8

Hi @SivaRamaKrishnaNV ,
By modifying the detection region params, I get the same result as before, mainly because the F008A120RM0AES is 8MP 38482168, the ‘m_networkInputDimensions.width’ and “m_networkInputDimensions.height” of tensorRT_model are fixed and less than F008A120RM0AES, so the yellow bounding box is shrunk very small.
How do I scale the image（38482168 - > 640 *480） before it is fed into the tensorRT_model ?

9

Dear @zhaohm8 ,
The m_detectionRegion select the ROI from the image. Please modify the {x,y,width,height} to change the ROI. You may set ROI as {0,0,m_imageWidth,m_ImageHeight} if you want to perform object detection on whole image.

10

Hi @SivaRamaKrishnaNV ，
I can expand the ROI (Yellow bounding box) by modify the {x,y,width,height}, but the result of Object Detector and Tracker is very bad, the car can’t be identified in most cases. Maybe the ‘m_networkInputDimensions.width’ and “m_networkInputDimensions.height” of tensorRT_model are fixed and less than F008A120RM0AES, so …
Maybe the image（3848*2168) also need be scaled before it is fed into the tensorRT_model? What shoule i do ?

11

Dear @zhaohm8,
ROI region gets scaled automatically before feeding into TensorRT model. Sample object detector just demonstrate the integration of TRT model into DW framework. You may use any other model which gives more accurate results on your dataset.

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.