Commit b881bd89 authored by Ramon Nou's avatar Ramon Nou
Browse files

Commit pending feature and test updates

parent 7173842a

README.md

+17 −0
Original line number Diff line number Diff line
@@ -988,6 +988,19 @@ read/write/error/retry totals and latency maxima, migration progress, and queue 
depth, daemon operation totals, and read/write byte totals. Legacy client

MessagePack/libzmq metrics remain unchanged.



### Distributed tracing (startup-only protocol upgrade)

- `GKFS_ENABLE_TRACE` - Enable sampled distributed RPC trace logging (default: OFF).

- `GKFS_TRACE_SAMPLE_RATE` - Trace one in every N root RPCs; must be a positive

  integer (default: `1`).



Tracing configuration is read once when each client, proxy, or daemon starts;

changing these variables afterward has no effect. A sampled trace carries a

correlation ID through client, proxy, and daemon RPC payloads and emits

structured debug records without paths or file names. This is a wire protocol

upgrade: **all communicating client, proxy, and daemon binaries must use this

version; mixed old/new deployments are unsupported.** Legacy MessagePack/libzmq

client metrics are unaffected.



### Client

#### Core

- `LIBGKFS_HOSTS_FILE` - Path to the hostsfile (created by the daemon and mandatory for the client).
@@ -995,6 +1008,10 @@ MessagePack/libzmq metrics remain unchanged. 
- `LIBGKFS_OFI_INTERFACE` - Force the client-side libfabric interface to use (equivalent to `FI_SOCKETS_IFACE`). For `ofi+sockets`, GekkoFS validates that the named local interface exists before initializing Margo.

- `LIBGKFS_RPC_LOOKUP_RETRIES` - Endpoint lookup attempts during client startup and malleability control RPC setup, in `[1,20]` (default: `3`).

- `LIBGKFS_RPC_LOOKUP_BACKOFF_MS` - Initial endpoint-lookup retry backoff, in milliseconds, in `[1,10000]`; retries use bounded exponential backoff with jitter (default: `50`).

- `LIBGKFS_RPC_TIMEOUT_MS` - Optional per-RPC deadline, in milliseconds, in

  `[0,600000]` (default: `0`, disabled). The value is read once during client

  startup. A deadline cancels the local Thallium request and reports a timeout;

  it does not retry writes or other potentially non-idempotent operations.

  This prevents clients from binding to loopback (`127.0.0.1`) when daemons are on real NICs (e.g., `ib0`).

  Required in malleable/HPC environments where the client may resolve to a loopback address.

  Example: `export LIBGKFS_OFI_INTERFACE=ib0`

docs/sphinx/users/running.md

+18 −0
Original line number Diff line number Diff line
@@ -311,6 +311,24 @@ RPC latency/errors, backend latency/errors/retries, migration fields, daemon 
operation totals, and read/write byte totals before the human-readable report.

Legacy client MessagePack/libzmq metrics remain unchanged.



### Distributed tracing



Set `GKFS_ENABLE_TRACE=ON` to enable sampled cross-component RPC trace logging.

`GKFS_TRACE_SAMPLE_RATE=N` selects one in every `N` root RPCs and must be a

positive integer (`1` is the default). Both values are read once at client,

proxy, and daemon startup; changing the environment later has no effect.



Tracing adds a correlation ID to Thallium RPC input payloads. Therefore it is a

protocol upgrade: client, proxy, and daemon binaries in a communicating

deployment must all be upgraded together. Mixed old/new binaries are not

supported. Trace logs intentionally omit paths and file names. The legacy

MessagePack/libzmq client metrics protocol is unaffected.



Client RPC deadlines are optional and startup-only. Set

`LIBGKFS_RPC_TIMEOUT_MS` to a value in `[1,600000]` to impose a Thallium

deadline; its default is `0` (disabled). A timed-out request is not retried

automatically, preventing duplicate non-idempotent writes.



### Advanced experimental features



#### Rename

include/client/env.hpp

+1 −0
Original line number Diff line number Diff line
@@ -114,6 +114,7 @@ static constexpr auto OFI_INTERFACE = ADD_PREFIX("OFI_INTERFACE"); 
static constexpr auto RPC_LOOKUP_RETRIES = ADD_PREFIX("RPC_LOOKUP_RETRIES");

static constexpr auto RPC_LOOKUP_BACKOFF_MS =

        ADD_PREFIX("RPC_LOOKUP_BACKOFF_MS");

static constexpr auto RPC_TIMEOUT_MS = ADD_PREFIX("RPC_TIMEOUT_MS");



} // namespace gkfs::env

include/client/preload_context.hpp

+7 −0
Original line number Diff line number Diff line
@@ -187,6 +187,7 @@ private: 
    bool async_write_stop_{false};



    std::string ofi_interface_;

    std::chrono::milliseconds rpc_timeout_{0};





public:
@@ -408,6 +409,12 @@ public: 
    void

    ipc_engine(std::shared_ptr<thallium::engine> engine);



    std::chrono::milliseconds

    rpc_timeout() const;



    void

    rpc_timeout(std::chrono::milliseconds timeout);



    bool

    use_metadata_batch() const;

include/client/preload_util.hpp

+6 −0
Original line number Diff line number Diff line
@@ -80,6 +80,12 @@ struct LookupRetryPolicy { 
LookupRetryPolicy

lookup_retry_policy();



std::chrono::milliseconds

rpc_timeout();



thallium::endpoint

lookup_endpoint(const std::string& uri, bool use_proxy = false);



template <typename E>

constexpr typename std::underlying_type<E>::type

to_underlying(E e) {