loopback on client - need a Gekko ENV variable or option as the server

Found by DMR testing

Misdiagnosed twice. First as a Mercury RPC concurrency bug (it is not), then as the nested Slurm reaping the daemons (it is not that either — see "ruled out" below).

The actual chain (job 44301178, found with per-10s GKFSPROBE instrumentation) 

GKFSPROBE t=60s daemons_alive=8 hostsfile=EXISTS lines=8
GKFSPROBE t=70s daemons_alive=7 hostsfile=EXISTS lines=8     <- ONE dies first
GKFSPROBE t=80s daemons_alive=0 hostsfile=MISSING            <- rest follow

mercury->rpc [error] hg_core_respond_na() Could not post send for output buffer
    (NA_NOENTRY, dest="ofi+sockets://127.0.0.1:33167")
HG_Respond() Could not respond (HG_NOENTRY)
srun: error: gs26r1b07: task 2: Aborted
srun: Terminating StepId=44301178.0
  1. A client endpoint comes up on loopback (127.0.0.1), while daemons advertise real IPs (10.2.x.x) in the hosts file.
  2. A daemon cannot route its RPC response back to that client → NA_NOENTRY → the daemon aborts.
  3. srun terminates the whole step (default kill-on-bad-exit) → all 8 daemons die.
  4. The departing daemons remove the shared hosts file.
  5. Ranks spawned at the DMR resize read LIBGKFS_HOSTS_FILE — a path that no longer exists — resolve no daemons, and get exactly 8 NA_NOENTRY failures, one per dead daemon. That count never fitted the "~224 ranks flood the RPC layer" concurrency story.
  6. GekkoFS's own logging path throws an uncaught fmt::v11::format_error while formatting that error → SIGABRT on rank 0.

the client endpoint binds to loopback, the daemon that cannot answer it aborts, and srun takes the other seven down with it. The client side has no interface control: preload.cpp:209 hands Thallium the bare protocol string | Pinned with FI_SOCKETS_IFACE=ib0 and --kill-on-bad-exit=0 so one abort is not fatal. Still needs an upstream knob — there is no client-side interface option |

Edited by Ramon Nou