fixes and refactor: preparing for 0.9.6 release

Ramon Nourequested to merge
rnou/p0 into master

Summary

This merge request improves GekkoFS reliability, test coverage, malleability support, client error handling, and CI tooling.

The changes also harden sfind against malformed directory records and make ThreadSanitizer builds work reliably in the CI environment.

Main changes

Reliability and RPC handling

  • Contain exceptions from daemon and proxy RPC handlers.
  • Make RPC responses safe when clients disconnect.
  • Improve error propagation from migration and mutation operations.
  • Add stronger validation around migration state transitions.
  • Improve endpoint lookup retries and client interface selection.
  • Improve daemon readiness detection and hostfile lifecycle handling.
  • Make hostfile cleanup atomic.

Malleability and random slicing

  • Add validation and fingerprinting for random-slicing layouts.
  • Improve migration status reporting.
  • Handle migration failures explicitly.
  • Reject invalid mutation transitions.
  • Improve migration write verification and retry handling.
  • Add coverage for expansion, shrinking, mutation failures, and migration behavior.
  • Improve runtime metrics and Prometheus type handling.

Client and syscall behavior

  • Add fchownat interception support and syscall coverage.
  • Return explicit errors for unsupported metadata operations.
  • Align flock handling between syscall and libc paths.
  • Improve internal symlink traversal safety.
  • Improve client logging and reduce shared log contention.
  • Add RAII handling for daemon and migration file descriptors.

Testing and CI

  • Add compatibility corpus workflows and tests.
  • Add RPC consistency checker tooling and CI integration.
  • Improve deterministic integration-test seed handling.
  • Remove the obsolete BATS test system.
  • Improve daemon crash, startup, disconnect, and resilience tests.
  • Install clang-tidy in the testing image.
  • Improve static-analysis and ABI-checking workflows.
  • Stabilize statistics tests under TSAN.

Sanitizer support

  • Avoid executing sanitizer-instrumented filesystem probes during CMake configuration.
  • Build TSAN executables without PIE.
  • Run TSAN Catch2 discovery and tests with ASLR disabled using: setarch x86_64 -R.
  • Prevent TSAN shadow-memory mapping failures during Catch2 test discovery.

sfind hardening

  • Validate directory record sizes before reading fields.
  • Reject truncated records.
  • Handle missing NUL terminators safely.
  • Use bounded directory-entry names for regex and path operations.
  • Protect empty working paths during path construction.

Summary:

  • Added runtime, operation, I/O, backend, migration, and latency metrics for Prometheus. - Added GKFS_DAEMON_PROMETHEUS_AUTH=user:pass for optional Pushgateway basic authentication. - Added unit coverage for metrics output and invalid authentication values.
  • Distributed RPC tracing and client RPC deadlines.
    • Added sampled trace correlation IDs across client, proxy, and daemon RPCs with GKFS_ENABLE_TRACE and GKFS_TRACE_SAMPLE_RATE.
    • Added optional client RPC deadlines through LIBGKFS_RPC_TIMEOUT_MS without retrying potentially non-idempotent operations.
  • Hostfile and malleability lifecycle metadata.
    • Added atomic hostfile updates, marker preservation, Random Slicing layout records, and layout hash validation across restart and mutate operations.
    • Added RAII file-descriptor handling and shared hostfile-management utilities.
  • Expanded diagnostics and engineering tooling.
    • Added RPC consistency checking, static-analysis helpers, compatibility-corpus workflows, logging-scale measurements, CI report indexing, and metadata reporting tools.
    • - Reduced client logging prefixes and routed client startup summaries to the configured client log destination.
    • Standardized client and daemon log levels to full uppercase names such as INFO, DEBUG, and ERROR.
    • Periodic daemon statistics can be written to the configured daemon log file with GKFS_DAEMON_STATS_LOG=ON.
    • Updated external dependencies, including fmt, spdlog, CLI11, GSL, and MessagePack, and raised the minimum CMake version for current test tooling.
    • Added CMake presets and configuration reports for debug, release, coverage, and sanitizer builds.
    • Reworked integration and CI execution with deterministic workflows, pytest timeouts/load distribution, sanitizer coverage, and RPC consistency checks.
    • Removed the legacy Bats-based dependency-script test setup in favor of the Python testing workflows.
    • Improved client operation diagnostics, readiness handling, retry/disconnect behavior, descriptor management, and per-process/sample-based logging controls.
    • - Fixed buffer-size validation issues in filtered directory listing paths.
    • Prometheus operation counters now update when Prometheus is enabled, including IOPS_WRITE and IOPS_READ.
    • Reduced noisy daemon errors for expected missing sparse chunk files; ENOENT reads are logged at debug level.
    • Hardened RPC and proxy error propagation, startup readiness, daemon disconnect handling, and malleability/migration cleanup.
    • Fixed client syscall and symlink edge cases, descriptor lifecycle issues, and compatibility-corpus snapshot/report generation.
    • Hardened sfind directory-record parsing and filtered-directory handling.
Edited by Ramon Nou

Merge request reports

CMake/FindFilesystem.cmake

+12 −3
Changes for CMake/FindFilesystem.cmake: 12 added lines, 3 removed lines.
Original line number Diff line number Diff line
@@ -107,9 +107,18 @@ cmake_minimum_required(VERSION 3.10) 
include(CMakePushCheckState)

include(CheckIncludeFileCXX)



# If we're not cross-compiling, try to run test executables.

# Otherwise, assume that compile + link is a sufficient check.

if (CMAKE_CROSSCOMPILING)

# If we're not cross-compiling, try to run test executables. Sanitizer runtime

# checks are not reliable during configuration (for example, ThreadSanitizer

# may reject the build environment), so use compile and link checks there too.

string(CONCAT _filesystem_check_flags

    " "

    "${CMAKE_CXX_FLAGS}"

    " ${CMAKE_CXX_FLAGS_${CMAKE_BUILD_TYPE}}"

    " ${CMAKE_EXE_LINKER_FLAGS}"

    " ${CMAKE_EXE_LINKER_FLAGS_${CMAKE_BUILD_TYPE}}"

)

if (CMAKE_CROSSCOMPILING OR

    _filesystem_check_flags MATCHES "(^|[ ;])-fsanitize=[^ ;]+")

    include(CheckCXXSourceCompiles)

    macro(_cmcm_check_cxx_source code var)

        check_cxx_source_compiles("${code}" ${var})

CMake/gkfs-config-report.txt.in

0 → 100644
+81 −0
Changes for CMake/gkfs-config-report.txt.in: 81 added lines, 0 removed lines.
Original line number Diff line number Diff line 
GekkoFS CMake configuration summary

===================================



Project

-------

name=@PROJECT_NAME@

version=@PROJECT_VERSION@

source_dir=@CMAKE_SOURCE_DIR@

binary_dir=@CMAKE_BINARY_DIR@

generator=@CMAKE_GENERATOR@

build_type=@CMAKE_BUILD_TYPE@

system=@CMAKE_SYSTEM_NAME@ @CMAKE_SYSTEM_PROCESSOR@

cmake_version=@CMAKE_VERSION@



Compilers

---------

c_compiler=@CMAKE_C_COMPILER@

c_compiler_id=@CMAKE_C_COMPILER_ID@

c_compiler_version=@CMAKE_C_COMPILER_VERSION@

cxx_compiler=@CMAKE_CXX_COMPILER@

cxx_compiler_id=@CMAKE_CXX_COMPILER_ID@

cxx_compiler_version=@CMAKE_CXX_COMPILER_VERSION@



Build options

-------------

GKFS_BUILD_DOCUMENTATION=@GKFS_BUILD_DOCUMENTATION@

GKFS_BUILD_FUSE=@GKFS_BUILD_FUSE@

GKFS_BUILD_LIBC_INTERCEPTION=@GKFS_BUILD_LIBC_INTERCEPTION@

GKFS_BUILD_PERFORMANCE=@GKFS_BUILD_PERFORMANCE@

GKFS_BUILD_TESTS=@GKFS_BUILD_TESTS@

GKFS_BUILD_TOOLS=@GKFS_BUILD_TOOLS@

GKFS_BUILD_USER_LIB=@GKFS_BUILD_USER_LIB@

GKFS_INSTALL_TESTS=@GKFS_INSTALL_TESTS@

GKFS_TESTS_FORWARDING=@GKFS_TESTS_FORWARDING@

GKFS_TESTS_GUIDED_DISTRIBUTION=@GKFS_TESTS_GUIDED_DISTRIBUTION@

GKFS_TESTS_INTERFACE=@GKFS_TESTS_INTERFACE@



Feature options

---------------

GKFS_ENABLE_AGIOS=@GKFS_ENABLE_AGIOS@

GKFS_ENABLE_CLIENT_LOG=@GKFS_ENABLE_CLIENT_LOG@

GKFS_ENABLE_CLIENT_METRICS=@GKFS_ENABLE_CLIENT_METRICS@

GKFS_ENABLE_PARALLAX=@GKFS_ENABLE_PARALLAX@

GKFS_ENABLE_PROMETHEUS=@GKFS_ENABLE_PROMETHEUS@

GKFS_ENABLE_ROCKSDB=@GKFS_ENABLE_ROCKSDB@

GKFS_ENABLE_UNUSED_FUNCTIONS=@GKFS_ENABLE_UNUSED_FUNCTIONS@

GKFS_FOLLOW_EXTERNAL_SYMLINKS=@GKFS_FOLLOW_EXTERNAL_SYMLINKS@

GKFS_USE_GUIDED_DISTRIBUTION=@GKFS_USE_GUIDED_DISTRIBUTION@

GKFS_USE_GUIDED_DISTRIBUTION_PATH=@GKFS_USE_GUIDED_DISTRIBUTION_PATH@

GKFS_USE_LEGACY_PATH_RESOLVE=@GKFS_USE_LEGACY_PATH_RESOLVE@



Limits and reporting

--------------------

GKFS_MAX_INTERNAL_FDS=@GKFS_MAX_INTERNAL_FDS@

GKFS_MAX_OPEN_FDS=@GKFS_MAX_OPEN_FDS@

GKFS_CLIENT_LOG_MESSAGE_SIZE=@GKFS_CLIENT_LOG_MESSAGE_SIZE@

GKFS_GENERATE_COVERAGE_REPORTS=@GKFS_GENERATE_COVERAGE_REPORTS@



Dependencies

------------

Mercury_VERSION=@Mercury_VERSION@

Argobots_VERSION=@Argobots_VERSION@

Thallium_VERSION=@Thallium_VERSION@

ZStd_VERSION=@ZStd_VERSION_STRING@

RocksDB_VERSION=@RocksDB_VERSION_STRING@

RocksDB_DIR=@RocksDB_DIR@

Prometheuscpp_VERSION=@Prometheuscpp_VERSION@

pkg_config_executable=@PKG_CONFIG_EXECUTABLE@

dependency_source_dir=@GKFS_DEPENDENCIES_PATH@

fmt_pinned_version=12.2.0

spdlog_pinned_version=1.17.0

cli11_pinned_version=2.7.2

catch2_pinned_version=3.16.0

gsl_pinned_version=5.0.0



Flags

-----

CMAKE_C_FLAGS=@CMAKE_C_FLAGS@

CMAKE_CXX_FLAGS=@CMAKE_CXX_FLAGS@

CMAKE_EXE_LINKER_FLAGS=@CMAKE_EXE_LINKER_FLAGS@

CMAKE_SHARED_LINKER_FLAGS=@CMAKE_SHARED_LINKER_FLAGS@
 
 No newline at end of file

CMake/gkfs-options.cmake

+2 −2
Changes for CMake/gkfs-options.cmake: 2 added lines, 2 removed lines.
Original line number Diff line number Diff line
@@ -193,7 +193,7 @@ function(gkfs_define_variable varName value type docstring) 
    _add_feature_info(${varName} ${varName} ${docstring})



    if (ARGS_ADVANCED)

        mark_as_advanced(varName)

        mark_as_advanced(${varName})

    endif ()



endfunction()
@@ -303,7 +303,7 @@ execute_process(COMMAND getconf OPEN_MAX 
    OUTPUT_STRIP_TRAILING_WHITESPACE

    ERROR_QUIET)

if (NOT _GETCONF_MAX_FDS)

    set(_GETCONF_MAX_FDS=512)

    set(_GETCONF_MAX_FDS 512)

endif ()



gkfs_define_variable(

docker/0.9.6/testing/Dockerfile

+1 −0
Changes for docker/0.9.6/testing/Dockerfile: 1 added line, 0 removed lines.
Original line number Diff line number Diff line
@@ -15,6 +15,7 @@ RUN apt-get update && \ 
    libjson-xs-perl \

    # Other system tools

    binutils \

    clang-tidy \

    cppcheck \

    curl \

    git \

docs/sphinx/devs/analysis.md

0 → 100644
+44 −0
Changes for docs/sphinx/devs/analysis.md: 44 added lines, 0 removed lines.
Original line number Diff line number Diff line 
# Sanitizer and static-analysis checks



Sanitizer and static-analysis checks are explicit and reproducible. They do not

change the normal `default-debug` or release builds.



## Sanitizers



Configure and build the unit test target with each sanitizer:



```console

$ cmake --preset default-asan

$ cmake --build builds/default-asan --target unit_tests

$ ctest --test-dir builds/default-asan --output-on-failure -L unit::all



$ cmake --preset default-ubsan

$ cmake --build builds/default-ubsan --target unit_tests

$ ctest --test-dir builds/default-ubsan --output-on-failure -L unit::all



$ cmake --preset default-tsan

$ cmake --build builds/default-tsan --target unit_tests

$ ctest --test-dir builds/default-tsan --output-on-failure -L unit::all

```



The CI sanitizer matrix uses separate `gkfs/build-asan`, `gkfs/build-ubsan`,

and `gkfs/build-tsan` directories. Integration tests are not run under these

presets because the current dependency and FUSE environments do not provide a

portable sanitizer runtime contract.



## Static analysis



The compile database is generated by CMake. Run cppcheck and clang-tidy locally:



```console

$ scripts/check_static_analysis.sh --all --require-tools \

    --build-dir builds/debug-local --output-dir analysis

```



Reports are written to `analysis/cppcheck.xml` and `analysis/clang-tidy.txt`.

Cppcheck runs warning, performance, and portability checks with system-header

noise suppressed. Clang-tidy runs the clang static-analyzer checks using the

checked-in `.clang-tidy` policy and the repository compile database. The CI job

publishes both reports for triage. To keep CI bounded, CI runs clang-tidy on the

RPC, mutation, and endpoint files most exposed by the current hardening work. A

full source scan remains available locally with `--clang-tidy` and no file list.