Data Logging on DRVIE Xavier Platform

Many developers choose to use rosbag as a data collection tool, which can create a recording node and call the subscriber method to receive specific or all the topics, and then write messages to the (.bag) file.

Things need to know that rosbag record process writes a large amount of data (e.g. LiDAR, Camera, Radar, IMU, …) and saves it (.rosbag) into disk, which means that buffers require a certain memory before being written to disk, the caches would also increase during the process (buffers -> caches). Both buffers and caches would occupy a lot of memory and if recording on an external disk (e.g. USB flash drive, SSD), the process might exits abruptly due to reach of the maximum of system memory.

To prevent it from happening, it is crucial that determine the write speed of the mounted disk before recording any data on it, and the following of the parameters would suggest to optimize target platform (e.g. Xavier) to store data.

  • dirty_background_ratio
  • dirty_ratio
  • dirty_background_bytes
  • dirty_bytes

Those parameters could also fix the problem (symptom) of Linux is freezing for a while when copying data or even just plugging an external drive on the machine.

Please feel free to contact me with further questions on this topic.
I might post more testing results or reports on NVIDIA Xavier Platform in the future.
e.g. each gmsl camera bandwidth -> image compression -> @ full resolution (1920*1208) (< 9 MB/s)

Email: twbabyduck@gmail.com

Tags: I/O, rosbag, 64-bit machine, Linux, data logging

1 Like