Commit 57cf4c17 authored by Ramon Nou's avatar Ramon Nou
Browse files

refactor: centralize hostfile management

parent 0ad6100f

WIP.md

+13 −7
Original line number Diff line number Diff line
@@ -388,15 +388,21 @@ an unrelated operation. 


## P2 — code quality and simplification



### 20. Remove duplicated hostfile and configuration parsing

### 20. Remove duplicated hostfile and configuration parsing — done



`MalleableManager` notes duplicated hostfile logic from preload utilities. Create

one common typed parser/validator for endpoints, markers, canonical sorting, and

errors. Preserve comments/unknown fields when possible. Add round-trip tests for

normal, add, remove, malformed, and versioned files.

Added one common typed hostfile parser/validator in

`common/hostfile_management` and migrated the client, daemon, proxy, and

malleability tool to use it. The shared implementation handles normal, `+`, and

`-` entries; canonical sorting; rootdir suffix normalization; schema checks;

duplicate host/URI checks; comments; unknown fields;

and round-trip clean writes. Random-slicing version/layout comments remain

validated by the same hostfile module.



**Acceptance:** Client, daemon, proxy, and tools produce identical topology and

layout inputs.

Added unit coverage for normal, add, remove, malformed, duplicate, boundary,

comment, and round-trip/versioned hostfiles.



**Validation:** Client, daemon, proxy, and tools now consume the same parsed

topology model. Full unit suite passes: 82 test cases and 3,464,810 assertions.



### 21. Generate RPC registration and consistency checks

include/common/common_defs.hpp

+0 −5
Original line number Diff line number Diff line
@@ -40,11 +40,6 @@ 
#define GEKKOFS_COMMON_DEFS_HPP



namespace gkfs {

namespace client {

// This must be equivalent to the line set in the gkfs script

constexpr auto hostsfile_end_str = "#FS_INSTANCE_END";



} // namespace client

namespace rpc {

// These constexpr set the RPC's identity and which handler the receiver end

// should use

include/common/malleability_markers.hpp→include/common/hostfile_management.hpp

+51 −3
Original line number Diff line number Diff line
@@ -26,8 +26,8 @@ 
  SPDX-License-Identifier: GPL-3.0-or-later

*/



#ifndef GEKKOFS_MALLEABILITY_MARKERS_HPP

#define GEKKOFS_MALLEABILITY_MARKERS_HPP

#ifndef GEKKOFS_HOSTFILE_MANAGEMENT_HPP

#define GEKKOFS_HOSTFILE_MANAGEMENT_HPP



#include <string>

#include <vector>
@@ -55,6 +55,12 @@ struct HostEntry { 
    std::string uri;      // Second field (e.g., "fi+sockets://...")

    std::vector<std::string> extra_fields; // Remaining 10 fields

    std::string raw_line; // Original line without marker prefix



    std::size_t

    field_count() const;



    const std::string&

    field(std::size_t index) const;

};



// Parsed result from a marker-based hostfile
@@ -126,6 +132,48 @@ parse_hostfile_markers(const std::string& path); 
void

validate_hostfile_markers(const HostfileMarkers& markers);



/**

 * @brief Validate host identities and URIs in every hostfile state.

 *

 * Unlike validate_hostfile_markers(), this also accepts a removing-only view,

 * which is needed while a client bootstraps during a shrink mutation.

 */

void

validate_hostfile_entries(const HostfileMarkers& markers);



/**

 * @brief Validate the number of fields shared by hostfile consumers.

 * @param markers Parsed hostfile markers

 * @param minimum_fields Minimum number of whitespace-separated fields

 * @throws std::runtime_error for malformed or conflicting topology entries

 */

void

validate_hostfile_schema(const HostfileMarkers& markers,

                         std::size_t minimum_fields = 2);



/**

 * @brief Return host entries for the requested mutation view.

 *

 * Active entries are always included. Adding and removing entries are included

 * when requested. If no active entries exist and fallback_removing is true,

 * removing entries are used as the bootstrap view.

 */

std::vector<HostEntry>

select_hostfile_entries(const HostfileMarkers& markers,

                        bool include_adding = false,

                        bool include_removing = false,

                        bool fallback_removing = false);



/**

 * @brief Return canonical hostname/URI pairs for a hostfile view.

 *

 * Entries are sorted by hostname and URI. A trailing rootdir suffix (`#...`)

 * is removed from the returned hostname after sorting.

 */

std::vector<std::pair<std::string, std::string>>

hostfile_pairs(const HostfileMarkers& markers, bool include_adding = false,

               bool include_removing = false, bool fallback_removing = false);



/**

 * @brief Strip marker prefix from a hostfile line

 * @param line The line (possibly with marker prefix)
@@ -162,4 +210,4 @@ write_rs_interval_comments( 
} // namespace malleable

} // namespace gkfs



#endif // GEKKOFS_MALLEABILITY_MARKERS_HPP
 
 No newline at end of file
 
#endif // GEKKOFS_HOSTFILE_MANAGEMENT_HPP
 
 No newline at end of file

include/daemon/malleability/malleable_manager.hpp

+0 −2
Original line number Diff line number Diff line
@@ -51,8 +51,6 @@ private: 
    // Tracks old hosts_size before expansion/shrink

    unsigned int old_hosts_size_{0};



    // TODO next 3 functions are mostly copy paste from preload_util. FIX



    std::vector<std::pair<std::string, std::string>>

    load_hostfile(const std::string& path);

scripts/run/gkfs

+0 −3
Original line number Diff line number Diff line
@@ -16,9 +16,6 @@ C_AST_GREEN="${C_GREEN}*${C_NONE} [gkfs] " 
C_AST_YELLOW="${C_BYELLOW}*${C_NONE} [gkfs] "

C_AST_RED="${C_BRED}*${C_NONE} [gkfs] "



# Important const globals

FS_INSTANCE_MARKER_CONST="#FS_INSTANCE_END"



count_host_lines() {

    local file="$1"

    grep -v '^#' "${file}" 2>/dev/null | grep -cv '^[[:space:]]*$'