Lots of paper about one-sided RDMA [e.g. Sherman@SIGMOD22, Flock@SOSP21] claim RNIC read/write data in increasing address order, so the version-based consistency check is effective.
Data layout: [version | … payload … | version]
One thread updates the data record, while other threads read the data record. if left version == right version, the payload is expected consistent. However in my platform (CX-6, OFED5.0, RDMA RC one-sided operation), if the payload is large than 2 cachelines, middle of the payload may remain unchanged but other parts haved been updated. (mixture of old data and new data, but versions are matched)
I am wondering about the following questions:
- What kind of memory read/write order does CX-6 guaranteed, especially if the work request that contains multiple cachelines?
- If RNIC could read/write data in increasing address order, what kind of parameters should be tuned?