Create client proxy for native protocols
Using psm2 and verbs in a reconnecting fashion is clearly impossible. Therefore, lets try a client proxy for data transfers.
That is, metadata can go through ofi+tcp
or na+sm
(auto_sm).
Inter-node data transfers first go to the on-node proxy (via mmap/cross memory attach, supported by Mercury na+sm
) and from there to the daemon via the native protocol. Intra-node data transfers will not go to the proxy and can be moved to the daemon directly. See: GekkoFS_native_proxy_copy.pdf
Hopefully, we no longer have native plugin reconnection issues that break everything and still can retain most of the performance of the native interface and not go slower than ofi+tcp
directly.