Sharing data between CGF and non CGF nodes

[ytai.ben.tsvi]

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Hi! I’m getting started with DriveOS and have some high-level architecture questions regarding the intended use of CGF and the serialization / replay features for sensors.

1. What is the recommended way to share information between realtime (CGF) and non-realtime nodes in the system? For example, I want to do my realtime pipeline from sensor to actuator using CGF, but then share the same data streams with non-realtime components, e.g. diagnostics. It is not obvious to me how this would work.
2. Can I handle logging and replay of all data in a centralized place, rather than have every sensor node be responsible for logging / replay of its own data? I’m tryin to achieve separation of concerns, where the decision on log / replay would be external to the actual graph nodes.

[VickNV]

Commonly, the data flow between “realtime” and “non-realtime” components is done through a message queue. In CGF the scheduler STM supports separate hyperepochs which are executed independently. So if the “realtime” nodes are in one hyperepoch they will be schedule independent of any delays in “non-realtime” nodes in another hyperepoch. The CGF channel in between the two communication endpoints should provide a large enough queue to buffer messages when the consumers are not receiving data fast enough for a period of time.

For logging the data produced by one node can be also passed to a node which records the data - this is in parallel to the other consumers of the data. For replay the producing node is commonly replaced with a component which sends out the previously recorded data on CGF channels to the “normal” consumers of the sensor data.