Good morning everyone,
Myself and one of the other engineers at my company have been investigating DriveWorks for the purposes of improving on our current autonomy solutions for industrial material handling vehicles (forklifts). Some of this was inspired by ZF’s CES press release ( - Archives Page 1 | NVIDIA Blog ) which mentions (for the first time) that NVIDIA and/or ZF may be interested in supporting material handling vehicle applications in addition to automotive applications.
We were granted DriveWorks SDK access through the developer program application system in late January. So far, our effort has led to more questions than answers.
First - while it is understandable that DriveWorks is heavily biased towards running with DRIVE PX 2 hardware, this makes it difficult for companies who are “testing the water” for evaluating DriveWorks’ suitability. The PX 2 is a significant investment (I’ve heard a number around $15k) into an unknown technology. Given the question posed at https://devtalk.nvidia.com/default/topic/991426/drive-platforms/how-to-record-h264-compatible-with-drivenet-lanenet-object-tracker-from-a-usb-camera/ we’re not the only ones that are looking to do an initial evaluation on more “commodity” hardware (such as a laptop) prior to moving to a more robust embedded solution. In addition, NVIDIA is relatively unresponsive when it comes to questions regarding the PX 2. (I’m getting the impression that, despite ZF’s press release, NVIDIA is not actually particularly interested in material handling applications, given how hard it has been to obtain further information.)
Second - why are sensor drivers baked in to the monolithic DriveWorks library, as opposed to being handled by some sort of plugin system? This makes it impossible for any potential DriveWorks user to add support for additional sensors without NVIDIA’s involvement. As we’re an application that is not in NVIDIA’s core target market for this technology, this is a showstopper for us that makes DriveWorks too risky to investigate further. Specifically - Laser scanners from SICK such as the S300 series are dominant in our industry, but DriveWorks has no support for anything in SICK’s product lines (such as the RS-422 output from an S300 Professional - https://www.sick.com/us/en/product-portfolio/opto-electronic-protective-devices/safety-laser-scanners/s300-professional/c/g187237 ) and there’s no way for us to add the support ourselves.
Third - In our application, we requested Linux SDK access as we anticipated that most of our work would be done on Linux systems. However, occasionally being able to run the demo samples on a Windows machine would be beneficial. The DriveWorks developer page states to contact our account manager for other variants of the SDK - however anyone who applied via the developer program route does not have an account manager (Enless you have an account manager named noreply? I find this highly unlikely…). Access to these forums is the first time in nearly a month that we’ve had anything resembling a human we might be able to talk to.
Thanks for your interest in DriveWorks and the feedback you have provided. We are constantly working to make DriveWorks better and expect the upcoming releases in next few months will reflect that.
First - while it is understandable that DriveWorks is heavily biased towards running with DRIVE PX 2 hardware, this makes it difficult for companies who are “testing the water” for evaluating DriveWorks’ suitability.
DriveWorks is targeted for autonomous driving applications. It is supported both on x86+ dGPU and Drive PX 2 (DPX2) however some functionality (e.g. Automotive cameras) are available only on DPX2 as a generic PC cannot support it. If you prefer, you can still evaluate a range of functionalities in the current version of DriveWorks on the PC (with discrete GPU) and then move to DPX2
In addition, NVIDIA is relatively unresponsive when it comes to questions regarding the PX 2.
We’ve answered the question. Please refer to the link.
Second - why are sensor drivers baked in to the monolithic DriveWorks library, as opposed to being handled by some sort of plugin system? This makes it impossible for any potential DriveWorks user to add support for additional sensors without NVIDIA’s involvement.
DPX2 and DriveWorks are open platform. Future versions will allow integrating additional sensors without NVIDIA’s involvement
Third - In our application, we requested Linux SDK access as we anticipated that most of our work would be done on Linux systems. However, occasionally being able to run the demo samples on a Windows machine would be beneficial. The DriveWorks developer page states to contact our account manager for other variants of the SDK - however anyone who applied via the developer program route does not have an account manager
We are working on offering Linux SDK through the same portal (developer.nvidia.com)
“DriveWorks is targeted for autonomous driving applications. It is supported both on x86+ dGPU and Drive PX 2 (DPX2) however some functionality (e.g. Automotive cameras) are available only on DPX2 as a generic PC cannot support it. If you prefer, you can still evaluate a range of functionalities in the current version of DriveWorks on the PC (with discrete GPU) and then move to DPX2” - How can we evaluate functionalities beyond NVIDIA’s canned demos with demo data? jdahan26 was told that a DPX2 is required for vision capture (as USB cameras are not supported) at https://devtalk.nvidia.com/default/topic/991426/drive-platforms/how-to-record-h264-compatible-with-drivenet-lanenet-object-tracker-from-a-usb-camera/ - but you are implying otherwise and that there is some sort of supported method of collecting vision data on an x86 with discrete GPU?
Evaluating the algorithms on an x86 with discrete GPU would be our ideal starting point - but we cannot evaluate them for our uses with NVIDIA’s demo data as our autonomous driving environment (a warehouse) is significantly different from outdoor roads and likely any neural networks would require retraining on alternative data.
As to answers to questions - Thank you for finally responding to questions. Note that this was not possible before the additional contact email provided in the response to jdahan’s question was made available last week. We have submitted information requests via Self-Driving Cars Technology & Solutions | NVIDIA Automotive in the past and received no response whatsoever.
“Future versions will allow integrating additional sensors without NVIDIA’s involvement”
It looks like we may have to wait for this before continuing our evaluation.
"We are working on offering Linux SDK through the same portal (developer.nvidia.com) " - We have the Linux SDK, we were looking to additionally obtain the Windows version in order to run demos on a number of machines that don’t have Linux available. (I’ve seen multiple pieces of documentation implying a Windows version which is comparatively feature-limited exists - most notably the calibration tool screenshots on page 9 of the SDK overview chart deck were taken on a Windows machine. I also remember Windows being an option in our initial SDK access request application. Or does a Windows version suitable for only executing the demos not exist? It’s not required, but does happen to be a nice-to-have.)
Slide 29 of the DriveWorks SDK overview implies that the SfM engine supports visual odometry and can somehow output a vehicle pose matrix (“Vehicle Matrix” in the lower right hand corner of the slide).
However, I cannot see where the calculated vehicle pose from the visual odometry component is output.
dwReconstructor_updateHistoryAsync() takes a rig2world transformation as input, and in the demos this is obtained from the Ackermann motion model. The demo explicitly states: “a car pose is estimated entirely from CAN data using the egomotion module”. The only output related to vehicle position is rig2WorldHistoryIdx, which is an index into the pose history list - but the pose history list is internal so what is the point of obtaining an index that can’t be used to reference anything?