Integrating custom radar into Driveworks visualization

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

We have been attempting to use the DRIVE PX 2 AutoChauffeur to collect live data with our radar and use one of the Driveworks samples for visualization of the targets.

What we have done so far:

  1. Following the documentation outlined in the “Integrating with Custom Sensors” (Page 17) in DriveWorks Develpment Guide we:
  • wrote a custom plugin decoder
  • compiled the decoder into an .so
  • added the radar to the information into the recorder-config.json file
  • attempted to run ./sample_record
    Result:
  • When the radar was connected to the AURIX CAN bus (CAN1-CAN4) it appeared to fail due to a clock slip issue between the Tegra And AURIX (documented in forum topics 1022173, 1032283, 1030774, 1024268 and bug reports 2108249, 2103064) which appears to still be an unresolved issue. Per the advice in the forums, we followed the instructions in DRIVE PX2 TIME SYNC GUIDE and DRIVE PX2 EASYCAN SETUP GUIDE.
  • We then connected the radar to the TegraA CAN port and used a stand alone driver to record the radar output and write it to a local IP address. Following the instructions in the sample_record README for live recording, we were unable to figure out the correct parameter setting for the sample to run. The error is with setting the ‘protocol’ parameter which is not mentioned in the README.
  • We do not see another method for creating an Ethernet Radar (dwSensorRadar object) and have thus been unable to test our custom decoder.
  1. As a workaround, we have made a custom version of the radar_replay sample that takes in our radar output from the Tegra CAN port, parses the data, and sends the data to the renderer. Since we are not able to make a dwSensorRadar object, we are doing this by circumventing the calls to to dwSensorRadar functions (_readScan, _toggleScanType, _getProperties, _returnScan) and ‘manually’ filling the dwRadarScan nextPacket struct.
    Result:
  • we are able to see the targets visualized in the GUI but have not been able to figure out the correct way to convert our radar readings (radius and azimuth) into the x and y coordinates that it appears the renderer uses.

We would appreciate any actionable advice for dealing with any of these issues.

2

Hi jgs2185,

Why did you say there is a problem in “sample_record” but not “radar_replay” sample when initializing sensor? Both samples use “dwSAL_createSensor” to create sensor handle.

Could you give a more clear explanation?

3

I just saw your previous topic-> https://devtalk.nvidia.com/default/topic/1036363

Could you try to use “udp” for protocol?

4

Thanks Wayne.

We tried the two following commands, using --protocol=udp and --protocol=radar.udp, both with the same result.

Is this what you meant for us to do?

nvidia@nvidia:/usr/local/driveworks/bin$ ./sample_radar_replay params=ip=177.2.2.1,port=21234,device=CUSTOM,decoder=/home/nvidia/dw_moving/libradardecoder.so --protocol=udp
nvrm_gpu: Bug 200215060 workaround enabled.
Initialize DriveWorks SDK v0.6.67
Release build with GNU 4.9.2 from v0.6.0-rc7-0-g51bd3aa against Vibrante PDK v5.0.5.0
TimeSource: monotonic epoch time offset is 1529501989791261
TimeSource: PTP ioctl returned error. Synchronized time will not be available.
TimeSource: Could not detect valid PTP time source at 'eth0'. Fallback to CLOCK_MONOTONIC.
SDK: Resources mounted from ../data/resources
SDK: Create NvMediaDevice
SDK: use EGL display as provided
SDK: number of GPU devices detected 2
SDK: currently selected GPU device discrete ID 0
SAL: identified board as DrivePX2-TegraA
SensorFactory::createSensor() -> udp, file=../data/samples/sensors/radar/conti/radar_0.bin
Driveworks exception thrown: DW_SAL_NO_DRIVER_FOUND: udp

terminate called after throwing an instance of 'std::runtime_error'
  what():  Sensor Initialization Failed
Aborted (core dumped)
nvidia@nvidia:/usr/local/driveworks/bin$ ./sample_radar_replay params=ip=177.2.2.1,port=21234,device=CUSTOM,decoder=/home/nvidia/dw_moving/libradardecoder.so --protocol=radar.udp
nvrm_gpu: Bug 200215060 workaround enabled.
Initialize DriveWorks SDK v0.6.67
Release build with GNU 4.9.2 from v0.6.0-rc7-0-g51bd3aa against Vibrante PDK v5.0.5.0
TimeSource: monotonic epoch time offset is 1529501989791261
TimeSource: PTP ioctl returned error. Synchronized time will not be available.
TimeSource: Could not detect valid PTP time source at 'eth0'. Fallback to CLOCK_MONOTONIC.
SDK: Resources mounted from ../data/resources
SDK: Create NvMediaDevice
SDK: use EGL display as provided
SDK: number of GPU devices detected 2
SDK: currently selected GPU device discrete ID 0
SAL: identified board as DrivePX2-TegraA
SensorFactory::createSensor() -> radar.udp, file=../data/samples/sensors/radar/conti/radar_0.bin
Driveworks exception thrown: DW_SAL_NO_DRIVER_FOUND: radar.udp

terminate called after throwing an instance of 'std::runtime_error'
  what():  Sensor Initialization Failed
Aborted (core dumped)
5

How about

./sample_radar_replay --protocol=radar.socket --params=ip=177.2.2.1,port=21234,device=CUSTOM,protocol=udp,decoder=/home/nvidia/dw_moving/libradardecoder.so
6

jgs2185,

I think the current status is still not clear to me. Below are what I understand now. Could you confirm if it is correct?

  1. AURIX CAN cannot use due to clock slip issue. Try TegraA CAN as alternative.
  2. radar_replay is able to run sensor while “sample_record” cannot.
    If so, what happened to sample_record? and what did you add to radar_replay so that it can fetch data?

I am also asking the solution to problem 1, but it needs some time to get reply.

7

Hi Wayne,

for the two points:

  1. Correct, we cannot run a device through the AURIX CAN. We have already been using the TegraA CAN as an alternative.

  2. With the versions provided by NVIDIA, we can not run sample_radar_reply or sample_record.

We modified radar_replay to use a CAN sensor (through TegraA CAN) instead of a radar sensor. We are bypassing the dwSensorRadar_readScan and implemented local functions that parse the CAN messages and fill the dwRadarScan that populates the gPointCloudBuffer. With this we are now able to visualize our radar data in the radar_replay GUI.

We did not try doing anything similar in sample_record because our main goals are data visualization and testing our custom decoder. At this point, we are happy to be getting visualizations, but would still like to complete development of the decoder.

So with our radar connected to the TegraA CAN and publishing to an internal IP, following your suggestion above for calling sample_radar_replay:

./sample_radar_replay --protocol=radar.socket --params=ip=177.2.2.1,port=21234,device=CUSTOM,protocol=udp,decoder=/home/nvidia/dw_moving/libradardecoder.so

we are able to use the original sample_radar_replay to initialize a dwSensorRadar object. We have confirmed that the sample is accessing our custom decoder for the object initialization and to get the radar properties via the _dwRadarDecoder_getConstants function. However, in the call to the dwSensorRadar_readScan function, we always get a DW_TIME_OUT error and see no evidence that it is trying to access any of the other functions within our custom decoder.

In one of my previous forum posts

https://devtalk.nvidia.com/default/topic/1036363/running-sample_record-or-sample_radar_replay-with-custom-radar-and-decoder/?offset=3#5265240

the response was that sample_radar_replay only supports CONTINENTAL_ARS430 and DELPHI_ESR2_5. Since we now know that we are able to at least somewhat use sample_radar_replay with a custom radar, we need to know if this is ultimately a dead end (hard for us to troubleshoot when we don’t have source code for the dwSensorRadar functions) or if it should be possible to do.

If the issue with running a custom radar is ultimately that the functionality is not supported in the dwSensorRadar functions, then I will again ask, what is the intended method for testing a custom radar decoder within the DriveWorks framework? This does not seem to be outlined anywhere in the documentation, and again, without having access to the source code, is seemingly impossible for us to determine on our own.

Much appreciation for your responses so far and thanks in advance for any more insight you can provide.

8

jgs2185,

I need further check with our internal team. Once getting something, I’ll update this thread.

Thanks for sharing all the details I want to know.

9

jgs2185,

I wonder if UDP/IP is working for your case since you are using TegraA CAN instead of AURIX CAN and there seems no IP under such case and thus no “socket” is available.

10

Hi Wayne,

My understanding is that the only way to initialize sensors connected through the AURIX CAN is to do so through IP since that is how the AURIX and Tegra communicate? This is my assumption based on the section “Integrating with Custom Sensors” (Page 17) in DriveWorks Development Guide because it only references an “Ethernet Radar” and requires that the sensors have IP addresses and ports specified in the json file.

And just to try to be clear about what we are doing: we are running our radar with a standalone driver (separate from DriveWorks) with the radar connected to TegraA CAN. Our standalone driver takes the CAN messages from the Tegra and publishes them to a local network IP that is also setup in the driver. Then we use the DriveWorks sample_radar_replay to setup a dwSensorRadar object and access the published IP messages.

Thanks

11

jgs2185,

Could you share what sensor you are using?

For our supported sensor- Continental ARS430, we have rad-moon converter between it and PX2.

12

jgs2185,

After confirmed with team, driveworks only accept ethernet sensors but no CAN sensor.

13

jgs2185,

I would like to make a summary for this topic. Drive-av radar_replay sample now only supports ethernet-based radar. The hardware connection is similar to (px2 ethernet port) <–> RAD-MOON converter <–> (You radar)

With some ethernet configuration as below:

sudo ifconfig eth0:900 10.1.0.81 || true
sudo route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0:900 || true
sudo route add -net 10.1.0.0 netmask 255.255.0.0 dev eth0:900 || true

and the command to run dw was mentioned in #7.