Running Egomotion Module with realtime sensor data (iteration of the sample)

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.10.0

Target Operating System
Linux

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)

Host Machine Version

native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers

Issue Description

The egomotion module is the only module for which we have got all the supported sensors, while I cannot populate the VehicleIO structs completely the Egomotion header takes in internally, I parse the Vehicle data from the canbus via separate parser function to fill the data structure values just as needed for Egomotion module to work.

However there are still some challenges to be tackled here :

CASE ONE } I need verification that the Global Egomotion module which takes in GPS sensors data, can it be used without RTK subscription (context : we dont have RTK, and hence GPS logger sample produces “GPS not working on most accurate “)

What I find is this →

dwGlobalEgomotionResult globalResult{}; // → this struct somewhat looks like this ::

typedef struct dwGlobalEgomotionUncertainty
{
    dwConfidence3f position;    //!< Position uncertainty (easting [m], northing [m], altitude [m]).
    dwConfidence3f orientation; //!< Orientation uncertainty (roll [rad], pitch [rad], yaw [rad]).
    bool validPosition;         //!< Indicates validity of `position` uncertainty estimate.
    bool validOrientation;      //!< Indicates validity of `orientation` uncertainty estimate.
    dwTime_t timestamp;         //!< Estimate timestamp.
} dwGlobalEgomotionUncertainty;

/**
 * @brief Holds global egomotion state estimate.
 */
typedef struct dwGlobalEgomotionResult
{
    dwGeoPointWGS84 position;  //!< Position in WGS-84 reference system.
    dwQuaternionf orientation; //!< Rotation from rig coordinate system to ENU coordinate system.
    bool validPosition;        //!< Indicates validity of `position` estimate.
    bool validOrientation;     //!< Indicates validity of `orientation` estimate.
    dwTime_t timestamp;        //!< Estimate timestamp.
    dwGlobalEgomotionUncertainty uncertainty;
} dwGlobalEgomotionResult;

Question 1 → If CASE 1 is correct deduction from my side, do I need to override the bool values as True in the result and uncertainty structs?

Question 2 → A parser module embedded into this creates latency and synchronisation issue, I am aware that egomotion module takes the sensors data in ordered timestamps, can it be pointed out explicitly how this happens? Is synchronisation done via hosttimestamps? sensortimestamps? Please see my log below to understand the timestamping issues

Question 3 → I want a proper parametric string with Novatel device to be able to run on ethernet connection type.

Can I get some insights here? Happy to PM source code only to mods here.

Logs

pomo@tegra-ubuntu:/usr/local/driveworks/bin$ sudo ./sygnalpomo_egomotion --rig=/usr/local/driveworks/data/samples/recordings/cloverleaf2/rig_ego.json
ProgramArguments: Missing argument 'dwTracePath' requested
Starting real-time egomotion processing...
[26-08-2025 21:59:17] Platform: Detected Drive Orin P3710
[26-08-2025 21:59:17] Adding variable DW_Base:DW_Version
[26-08-2025 21:59:17] Added variable DW_Base:DW_Version
[26-08-2025 21:59:17] Platform: number of GPU devices detected 1
[26-08-2025 21:59:17] Platform: currently selected GPU device 0, Resource Data Dir: trt_08_06_12_04, Arch: ga10b
[26-08-2025 21:59:17] Platform: currently selected GPU device integrated ID 0
[26-08-2025 21:59:17] CUDLAEngine:getDLACount: CUDLA version is = 1004000
[26-08-2025 21:59:17] CUDLAEngine:getDLACount: Number of DLA devices = 2
[26-08-2025 21:59:17] Context::mountResourceCandidateDataPath resource FAILED to mount from './resources': VirtualFileSystem: Failed to mount './resources/resources.pak'
[26-08-2025 21:59:17] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/bin/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/bin/data/resources.pak'
[26-08-2025 21:59:17] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[26-08-2025 21:59:17] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[26-08-2025 21:59:17] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[26-08-2025 21:59:17] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[26-08-2025 21:59:17] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[26-08-2025 21:59:17] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[26-08-2025 21:59:17] SDK: No resources(.pak) mounted, some modules will not function properly
[26-08-2025 21:59:17] [26-08-2025 21:59:17] Initialize DriveWorks SDK v5.20.24
[26-08-2025 21:59:17] [26-08-2025 21:59:17] Release build with GNU 9.3.0 from buildbrain-branch-0-gb4c0b405b15 against Drive PDK v6.0.10.0
[26-08-2025 21:59:17] Initialize DriveWorks VisualizationSDK v5.20.24
[26-08-2025 21:59:17] Initialize DriveWorksGL SDK v5.20.24
[26-08-2025 21:59:17] egl::Display: found 1 EGL devices
[26-08-2025 21:59:17] egl::Display: use drm device: /dev/dri/card0
[26-08-2025 21:59:17] fromFile: Loading rig file: /usr/local/driveworks/data/samples/recordings/cloverleaf2/rig_ego.json
[26-08-2025 21:59:17] createFromVehicleVIN: Failed to open a file '/tmp/car_vin' - file likely does not exist - skipping overlay
[26-08-2025 21:59:17] No valid data file found for can in parameter string: device=can3,bitrate=500000,type=native (using configuration folder /usr/local/driveworks/data/samples/recordings/cloverleaf2/)
[26-08-2025 21:59:17] No valid data file found for imu:novatel in parameter string: connection-type=ethernet,fifo-size=1024 (using configuration folder /usr/local/driveworks/data/samples/recordings/cloverleaf2/)
[26-08-2025 21:59:17] No valid data file found for gps:novatel in parameter string: connection-type=ethernet,fifo-size=1024 (using configuration folder /usr/local/driveworks/data/samples/recordings/cloverleaf2/)
[26-08-2025 21:59:17] No valid data file found for camera_front in parameter string: camera-name=F008A120RM0AV2,interface=csi-ab,CPHY-mode=1,link=0,output-format=processed,async-record=1,file-buffer-size=16777216 (using configuration folder /usr/local/driveworks/data/samples/recordings/cloverleaf2/)
[26-08-2025 21:59:17] createFromVehicleVIN: Failed to open a file '/tmp/car_vin' - file likely does not exist - skipping overlay
[SENSORS INIT] Vehicle CAN interface found: can
[SENSORS INIT] IMU Sensor found: imu:novatel
[SENSORS INIT] GPS Sensor found: gps:novatel
[SENSORS INIT] Camera Sensor found, Rig intake name: camera_front
[26-08-2025 21:59:17] Egomotion: gyroscope measurement filter factor 0.50000000, gravity filter factor in rotational filter 0.09090909
[26-08-2025 21:59:17] Egomotion: slip angle coefficient computed from vehicle params: -0.014475
[26-08-2025 21:59:17] Egomotion: likely invalid slip angle coefficient, should not be lower than -0.010000
[26-08-2025 21:59:17] Using default slip angle coefficient: -0.002830
[26-08-2025 21:59:17] SensorFactory::createSensor() -> time.nvpps, nvpps-device=/dev/nvpps0
[26-08-2025 21:59:17] Found driver: time.nvpps
[26-08-2025 21:59:17] TimeSensor: failed to set NVPPS parameters.  Error: Invalid argument
[26-08-2025 21:59:17] This platform does not support NVPPS GPIO mode. Fallback to timer mode
[26-08-2025 21:59:17] TimeSensor: initialized with no UTC time reference.
[26-08-2025 21:59:17] SensorFactory: Override default timeSensor with the user passed one
[26-08-2025 21:59:17] EndpointNVPPS: started on /dev/nvpps0
[26-08-2025 21:59:17] SensorFactory::loadPluginLibraryHelper:Succeed to load dynamic lib - /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libsensor_plugin_public_radar.so
[26-08-2025 21:59:17] SensorFactory::createSensor() -> can.socket, device=can3,bitrate=500000,type=native
[26-08-2025 21:59:17] Found driver: can.socket
[26-08-2025 21:59:17] CANSocket: Cannot get current state of hardware time stamping: ioctl(SIOCGHWTSTAMP, can3) -> Operation not supported
[26-08-2025 21:59:17] CANSocket: software based timestamps will be used for can3
[26-08-2025 21:59:17] SensorFactory::createSensor() -> imu.novatel, connection-type=ethernet,fifo-size=1024
[26-08-2025 21:59:17] Found driver: imu.novatel
[26-08-2025 21:59:17] SensorManager::addSensor DW_INVALID_ARGUMENT: IMUNovatelProprietary: invalid ip parameter
[26-08-2025 21:59:17] SensorManager::addSensorsFromRig() failed to add sensor from rig: imu:novatel imu.novatel connection-type=ethernet,fifo-size=1024 (Error: DW_INVALID_ARGUMENT)
[26-08-2025 21:59:17] Driveworks exception thrown: DW_INVALID_ARGUMENT: SensorManager::SensorManager() Failed to add sensors from rig

Error initializing SensorManager for real-time processing: DW_INVALID_ARGUMENT
pomo@tegra-ubuntu:/usr/local/driveworks/bin$
2

@SivaRamaKrishnaNV dint get an update on this thread, so decided to bypass VehicleIO in some way here, I need an urgent insight on this for our vehicles localization model.

4

Hi @ashwin.nanda ,
I received feedback from engineering team on the previous issue. Could you please check it in the topic thread.

Can you share the code and rig file via private message. Let me check and get back to you.

5

Thanks for that, ill continue about it there, can you please check your PM regarding this? sent you the full rig and source code.

6

Dear @ashwin.nanda ,

Global egomotion don’t require RTK

We expect all data to be synchronized to the same timing source. We normally don’t use hosttimestamp (time of reception), but rely on the sensorTimestamp which indicates the time of data validity.

Is it the same Novatel sensor listed in DRIVE AGX Orin Ecosystem Vendors | NVIDIA Developer ?

7

hi @SivaRamaKrishnaNV yes the sensor in inventory is supported (novatel pwrpak7-imu+gnss span system). we are currently on usb connection, however ethernet type connection needs some offset values in parameters as well? this am unsure of atm.

plus please note that we are working on the module while we post it here as well, so sensors timestamping and synchronisation is what our concern is at, where Im trying to align a temporal buffer capacity being filled and fed into egomotion module (effect of live CAN parsing)

I need to know that does egomotion take sensors data as sensorstimestamp from their packets and arrange them internally?

much thanks,

ashwin

8

An example of Novatel sensor params is below
connection-type=ethernet ,ip=192.168.0.210,port=2000,sensor2AntennaX=-0.27789,sensor2AntennaY=0.99984,sensor2AntennaZ=0.84171

We take the measurement by their timestamp and if the timestamp doesn’t increase or decreases we’ll ignore it.
Between GPS and IMU data, if the measurement from GPS comes with more latency (example: IMU data comes up to timestamp 10s and then a GPS measurement comes at timestamp 9.8s), we will still use the GPS measurement and correctly perform the calculation based on the timestamp from the GPSFrame.

10

@SivaRamaKrishnaNV just be clear here, so the fusion depends on sensordatatimestamp which contains a measured timestamp in the payload at an instance, not the arrival time.

in ref to the example: the gps is lagging by 0.2s because IMU data already has the payload exactly for timestamp and time at t=10s?

11

hi @SivaRamaKrishnaNV I was hoping if you could aid with my conceptual understanding in last response please?

regards,

Ashwin