Custom Sensor: Radar Decoder Plugin issues

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.6.0.8170
other

Host Machine Version
native Ubuntu 18.04
other

Hello,

I have an issue with creating a decoder plugin for radar sensor.
My goal is to connect an ethernet based radar, which is simulated in a virtual environment. The software provides the data through TCP/IP and is locally simulated on my workstation (hence the local IP you will see).

I’m executing the following command for the radar_replay sample and get the following output:

“any-workstation”:~/driveworks/build/src/sensors/radar/radar_replay$ ./sample_radar_replay --protocol=radar.socket --params=ip=192.168.2.125,port=2210,device=CUSTOM,protocol=tcp,decoder=/home/“any-user”/driveworks/build/src/sensors/plugins/radar/libsample_radar_decoder.so
[15-09-2021 18:01:13] Platform: Detected Generic x86 Platform
[15-09-2021 18:01:13] TimeSource: monotonic epoch time offset is 1631691975691083
[15-09-2021 18:01:13] Platform: number of GPU devices detected 1
[15-09-2021 18:01:13] Platform: currently selected GPU device discrete ID 0
[15-09-2021 18:01:13] SDK: Resources mounted from /usr/local/driveworks/samples/…/data/
[15-09-2021 18:01:13] TimeSource: monotonic epoch time offset is 1631691975691083
[15-09-2021 18:01:13] Initialize DriveWorks SDK v2.2.3136
[15-09-2021 18:01:13] Release build with GNU 7.4.0 from heads/buildbrain-branch-0-gca7b4b26e65
[15-09-2021 18:01:13] SensorFactory::createSensor() → radar.socket, ip=192.168.2.125,port=2210,device=CUSTOM,protocol=tcp,decoder=/home/“any-user”/driveworks/build/src/sensors/plugins/radar/libsample_radar_decoder.so
scansPerSecond = 15
[15-09-2021 18:01:13] RadarSocket::RadarSocket, connected to 192.168.2.125:2210
terminate called after throwing an instance of ‘std::runtime_error’
what(): In Radar Properties - packetsPerSecond is 0
Aborted (core dumped)

My decoder plugin is nearly empty, since I’m just trying to see whether this will work and I’m trying to understand, how the decoder has to look like:

#include <dw/sensors/plugins/radar/RadarDecoder.h>
#include <dw/sensors/plugins/radar/RadarPlugin.h>
#include <iostream>

_dwRadarDecoder_constants DecoderConstants;

dwStatus _dwRadarDecoder_initialize () {

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_release() {
	
	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_decodePacket 	( 	
	dwRadarScan *  	output,
	const uint8_t *  buffer,
	const size_t  	length,
	const dwRadarScanType  	scanType )
{

	output = 0;
	std::cout << buffer << std::endl;
	std::cout << length << std::endl;
	(void) scanType;

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_getConstants(_dwRadarDecoder_constants * constants)
{
	DecoderConstants.properties.scansPerSecond=15;
	DecoderConstants.properties.isDecodingOn=1;
	DecoderConstants.properties.numScanTypes=1;
	DecoderConstants.properties.supportedScanTypes[0][0] = 1;
	DecoderConstants.properties.packetsPerScan = 20;
	DecoderConstants.properties.maxReturnsPerScan[0][0] = 1;
	DecoderConstants.properties.inputPacketsPerSecond = 20;
	DecoderConstants.headerSize = 64;
	DecoderConstants.maxPayloadSize = 2048;
	DecoderConstants.vehicleStateSize = 64;
	DecoderConstants.mountSize = 64;

	constants = &DecoderConstants;

	std::cout << "scansPerSecond = " << constants->properties.scansPerSecond << std::endl;

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_synchronize ( 	
	const uint8_t * buffer,
	const size_t  	length,
	size_t *  	remaining)
{

	(void) buffer;
	(void) length;
	(void) remaining;

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_validatePacket ( 
	const uint8_t*   buffer,
	const size_t  	 length,
	dwRadarScanType* scanType) 
{

	(void) buffer;
	(void) length;
	(void) scanType;

	return DW_SUCCESS;
}


bool _dwRadarDecoder_isScanComplete ( 	
	dwRadarScanType  scanType,
	const uint8_t ** buffer,
	size_t *  	 length,
	size_t  	 numPackets 
	) 
{
	(void) scanType;
	(void) buffer;
	(void) length;
	(void) numPackets;

	return true;
}


dwStatus _dwRadarDecoder_encodeVehicleState ( 	
	uint8_t *  	buffer,
	const size_t  	maxOutputSize,
	const dwRadarVehicleState * packet 
	) 
{
	(void) buffer;
	(void) maxOutputSize;
	(void) packet;

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_encodeMountPosition ( 	
	uint8_t *  	buffer,
	const size_t  	maxOutputSize,
	const dwRadarMountPosition * packet 
	)
{
	(void) buffer;
	(void) maxOutputSize;
	(void) packet;

	return DW_SUCCESS;
}

but it seems like I’m not able to pass on the properties data as part of the “_dwRadarDecoder_constants” struct to the created sensor through “dwSAL_createSensor” in the radar_replay sample executed in the function “initializeSensor”.

As you can see in the above forwarded output, I’ve set up an output data stream with “cout” in order to see if this function got called by “CreateSensor” and if so, how the value looks like, because I assumed, that “dwSAL_createSensor” is executing exactly this function to get the properties.

So, what exactly am I doing wrong here? Besides, a good manual how to build such a decoder would be really helpful… and please don’t forward the webinar to me, since it’s describing how to build a “comprehensive” plugin and not the decoder, which is totally sufficient for my requirements.

Best regards,
Kerby

Radar (Decoder Only): Accessing data through "_decodePacket"
2

Hi @k.bodi2,

Please refer to “Custom Radars (Comprehensive)” document (/usr/local/driveworks/doc/nvsdk_html/sensorplugins_radarsensor.html on your host system) and Radar Plugin in DriveWorks. Let us know if it helps.

4

Hey @VickNV,

thanks for the informations! I can’t open the first link you provided, but I guess you are forwarding me to the “Custom Radars (Comprehensive)” topic as part of the DriveWorks SDK Reference manual. I read through it and also checked the second link, where the following code was provided by you:

dwStatus _dwSensorRadarPlugin_getDecoderConstants(_dwSensorRadarDecoder_constants* constants, dwSensorPluginSensorHandle_t sensor)
{
...
    for (size_t i = 0; i < DW_RADAR_RETURN_TYPE_COUNT; ++i)
    {
        for (size_t j = 0; j < DW_RADAR_RANGE_COUNT; ++j)
        {
            dwRadarReturnType returnType = static_cast<dwRadarReturnType>(i);
            dwRadarRange range = static_cast<dwRadarRange>(j);
            constants->maxPointsPerPacket[i][j] = ...;
            constants->packetsPerScan[i][j] = ...;
        }
    }

Guess, I will try this code snippet on my side.

Another question raised up after trying to understand the custom sensor interface.

Our virtual radar sensor is sending data depending on the amount of detected objects. Hence, the data length received will change for every packet and we usually use a self-developed socket client, which is trying to receive as many bytes as required. Let me briefly explain it:

First of all, we send a header which is always 64 bytes. This header includes informations about, how many detections are available and how big is the next packet size in order to differentiate between subsequent packages.

Nevertheless, after reading through most likely every article it seems like the received bytes through socket.h in DriveWorks (I guess it is done by socket.h) has to be always identical for each packet/scan, right? If this is the case for DriveWorks, there is no suitable interface for our virtual radar with DriveWorks. In this case, we would have to integrate our own socket connection and integrate the data received and decoded.

What is the main benefit of using your custom sensor interface with radar? As far as I can see, the only advantage connecting a custom radar sensor to DriveWorks is the Self-Calibration approach. Another one is providing a framework for vendor-specific decoding. Everything else like renderer should be also usable with user-defined data structures from my point of view.

PS: I should mention, that our virtual radar sensor is not requiring any data from the AGX. It’s just sending data to the platform to be processed there.

5

I checked the above quoted code for the constants and implemented it in the following way:

dwStatus _dwRadarDecoder_getConstants(_dwRadarDecoder_constants * constants)
{

	constants = &DecoderConstants;

    	for (size_t i = 0; i < DW_RADAR_RETURN_TYPE_COUNT; ++i)
    	{
        	for (size_t j = 0; j < DW_RADAR_RANGE_COUNT; ++j)
        	{
	    		constants->properties.supportedScanTypes[i][j] = 1;
 	    		constants->properties.maxReturnsPerScan[i][j]  = 1;
        	}
    	}

	constants->properties.scansPerSecond=15;
	constants->properties.isDecodingOn=1;
	constants->properties.numScanTypes=1;
	constants->properties.packetsPerScan = 20;
	constants->properties.inputPacketsPerSecond = 20;

	constants->headerSize = 64;
	constants->maxPayloadSize = 2048;
	constants->vehicleStateSize = 64;
	constants->mountSize = 64;

	std::cout << "scansPerSecond = " << constants->properties.scansPerSecond << std::endl;

	return DW_SUCCESS;
}

However, all your proposals are referencing to “Custom Radars (Comprehensive)” and as I told you, I just want to use the Decoder Plugin. It should be sufficiently good enough for my first tests. My properties struct is still empty as you can see:

image

The links you have referred to have no solution for me. Am I understanding it correctly, that you are not supporting the decoder plugin and hence wants to propose to use the comprehensive plugin?

6

Because I have experience in supporting the comprehensive plugin, I suggested you use it.

Per the log and the code snippet you provided, your _dwRadarDecoder_getConstants() already got called successfully.
I double-checked our code. You should be able to get the properties via dwSensorRadar_getProperties().
Did it return any error from any other API call before calling dwSensorRadar_getProperties()?

7

Thanks @VickNV for your reply!

Sounds reasonable to me and thanks for double-checking.

Unfortunately it’s still not working for me. Here is the debug output I get when running sample_radar_replay with my decoder:

image
image1232×577 124 KB

I’ve set up a breakpoint on the exact line where the if-condition checks if “scansPerSecond” is equal to zero and if so, throw a runtime error with the corresponding message visible in the log above.

Since all other functions before were running fine with dwStatus = “DW_SUCCESS” (otherwise it would have never reached this if-condition), I guess something else have to be wrong here. Something related to dwSAL_createSensor (…).

Because, while debugging my _dwRadarDecoder_getConstants function gets called by dwSAL_createSensor visible by the log output with the syntax “scansPerSecond = 15”, which is printed exactly after executing dwSAL_createSensor.

I don’t know what exactly is wrong since we can’t see inside the functions how it’s accessing the properties.
At last I’m just providing a pointer, so what else do I have to take care of? :\

I had no error from any other API call. Everything stopped exactly with the above highlighted line (Line no. 453).

8

Please share your libsample_radar_decoder.so and we see if we can reproduce it. Thanks.

9

I’m curious about what kind of simulation you are doing? What AV stack are you going to use? Is it based on DriveWorks? Thanks.

10

This function is a call by reference so assigning the address of DecoderConstants to the local variable, constants, won’t let the caller get your settings.
You need to fix it as below.

*constants = DecoderConstants

1 Like
11

We are using CarMaker in combination with your AGX platform and DriveWorks SDK to process virtually calculated Radar detections from CarMaker and feed the post-processed data back to CarMaker for closed-loop virtual vehicle regulation.

Thanks a lot - this fixed the issue!

12

Another question from my side. I’ve set up all I need and the first header send by CarMaker was received by DriveWorks with a content, that CarMaker is connected as a data transmitter. However, the length variable is greater zero (=64 bytes) only in the functions _dwRadarDecoder_validatePacket and _dwRadarDecoder_synchronize, in _dwRadarDecoder_decodePacket length is always zero. I would like to confirm the received data and process it in _dwRadarDecoder_decodePacket, but I don’t how it’s developed in the background to forward the buffer and length values from _dwRadarDecoder_validatePacket or _dwRadarDecoder_synchronize to _dwRadarDecoder_decodePacket?

Following code is currently implemented (see comments for details):

#include <dw/sensors/plugins/radar/RadarDecoder.h>
#include <dw/sensors/plugins/radar/RadarPlugin.h>
#include <iostream>

dwStatus _dwRadarDecoder_initialize () {

	// Indicator, which function got executed:
	std::cout << "1" << std::endl;

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_release() {
	
	// Indicator, which function got executed:
	std::cout << "2" << std::endl;

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_decodePacket 	( 	
	dwRadarScan *  	 output,
	const uint8_t *  buffer,
	const size_t  	 length,
	const dwRadarScanType  	scanType )
{
	(void) output;
	(void) scanType;

	// For unknown reasons "buffer" is always empty declared by "length" equal to zero for every loop?
	std::cout << "Buffer: " << *buffer << " - Length: " << length << std::endl;

	// Indicator, which function got executed:
	std::cout << "3" << std::endl;

	// When length is greater than zero, output the buffer content to screen
	// and abort the application before filling out the shell content:
	if (length > 0)
	{
		for (size_t i = 0; i < length; i++)
		{
			std::cout << buffer[i];
		}

		std::cout << std::endl;
		abort();
	}

	return DW_SUCCESS;
}


// Setting up all constants and properties of the sensor.
// Only detections will be received, hence no other data is required to be set up.

dwStatus _dwRadarDecoder_getConstants(_dwRadarDecoder_constants * constants)
{
	constants->properties.supportedScanTypes[DW_RADAR_RETURN_TYPE_DETECTION][DW_RADAR_RANGE_LONG] = 1;
	constants->properties.maxReturnsPerScan[DW_RADAR_RETURN_TYPE_DETECTION][DW_RADAR_RANGE_LONG]  = 2000;

	constants->properties.scansPerSecond=1;
	constants->properties.isDecodingOn=1;
	constants->properties.numScanTypes=1;
	constants->properties.packetsPerScan = 1;
	constants->properties.inputPacketsPerSecond = 0;

	constants->headerSize = 64;
	constants->maxPayloadSize = 2048;
	constants->vehicleStateSize = 0;
	constants->mountSize = 0;

	// Indicator, which function got executed:
	std::cout << "4" << std::endl;

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_synchronize ( 	
	const uint8_t * buffer,
	const size_t  	length,
	size_t *  	remaining)
{
	(void) buffer; // Currently not used

	// Set remaining data to zero, when first message of 64 bytes is received.
	// Is this required to provide the data to the "_dwRadarDecoder_decodePacket(...)" function?
	if (length == 64) {
		*remaining = 0;
	}

	// Indicator, which function got executed:
	std::cout << "5" << std::endl;

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_validatePacket ( 
	const uint8_t*   buffer,
	const size_t  	 length,
	dwRadarScanType* scanType) 
{
	(void) buffer; // Currently not used
	(void) length; // Currently not used

	scanType->range = DW_RADAR_RANGE_LONG;
	scanType->returnType = DW_RADAR_RETURN_TYPE_DETECTION;

	// Indicator, which function got executed:
	std::cout << "6" << std::endl;

	return DW_SUCCESS;
}


bool _dwRadarDecoder_isScanComplete ( 	
	dwRadarScanType  scanType,
	const uint8_t ** buffer,
	size_t *  	 length,
	size_t  	 numPackets 
	) 
{
	(void) scanType; // Currently not used
	(void) buffer;	 // Currently not used
	(void) length;	 // Currently not used

	// Tried to set up the numPackets in the hope that
	// _dwRadarDecoder_decodePacket(...) will be provided with data:
	numPackets = 1;

	// Indicator, which function got executed:
	std::cout << "7" << std::endl;

	return true;
}


dwStatus _dwRadarDecoder_encodeVehicleState ( 	
	uint8_t *  	buffer,
	const size_t  	maxOutputSize,
	const dwRadarVehicleState * packet 
	) 
{
	(void) buffer;			// Currently not used
	(void) maxOutputSize;	// Currently not used
	(void) packet;			// Currently not used

	// Indicator, which function got executed:
	std::cout << "8" << std::endl;

	return DW_SUCCESS;
}


dwStatus _dwRadarDecoder_encodeMountPosition ( 	
	uint8_t *  	buffer,
	const size_t  	maxOutputSize,
	const dwRadarMountPosition * packet 
	)
{
	(void) buffer;			// Currently not used
	(void) maxOutputSize;	// Currently not used
	(void) packet;			// Currently not used

	// Indicator, which function got executed:
	std::cout << "9" << std::endl;

	return DW_SUCCESS;
}

When exactly will the data located in buffer with the buffer length indicator “length” be forwarded to the _dwRadarDecoder_decodePacket function? What are the requirements?

I’m trying to understand the concept and structure of the decoder step by step, since the documentation doesn’t explain this in detail.

13

Please help to raise another topic for this. Thanks.

14

Done. See Radar (Decoder Only): Accessing data through "_decodePacket"