Ramon Nou authored 1 year ago and Marc Vef committed 1 year ago

Added NUM_REPL env variable. (0 no replicas)

NUM_REPL  num replicas  (Replicas < servers)

Remove and truncate

Metadata replication

Metadata replication - Reattempt on stat

minimal compilation issues (c++20), srand for repl

Bitset

Bit set proposal (WIP)

Read - Write with bitset (<1024 chunks)

Changed bitset to vector

Added get_fs_config reattempt

Some more resilience on create

Added Replica_Check on write (disabled)

Added helper vector-bitset functions

calculates blocksize by dividing size / 512 as it is legacy in the linux kernel, only if renaming is deactivated

Marc Vef authored 3 years ago and Ramon Nou committed 3 years ago

A subdirectory is no longer created by default. Therefore, a server restart uses same directory for both data and metadata. Note, that the rootdir can be cleaned with the -c argument. For multiple daemons on one machine, the new argument --rootdir-suffix can be used which creates a subdirectory within the rootdir with a user-defined name. This also allows restarting multiple daemons on one node without losing data.

Ramon Nou authored 4 years ago and Ramon Nou committed 3 years ago

Moved distributor instantation to RPC_DATA and daemon.cpp on the server side. Added interval_map to GuidedDistributor for better performance

Moving reads extended log to normal log module.

Update logging.hpp

Added GKFS prefix to CMake options

Catch2 Guided Distributor test

Added creation of /tmp/guided.txt in the test

updated README.MD and protected Distributor for duplicated inserts

Added metadata + data optimization with #

Client now reads host file to determine auto_sm and RPC protocol

- more error handling
- removed runtime exception from client
- Streamlining error handling in client and making it consistent. Adding comments
- Improved readdir logging

Marc Vef authored 5 years ago and Alberto Miranda committed 4 years ago

The `ChunkStorage` backend class on the daemon was throwing `system_errors` without being caught, crashing the server in the process. `ChunkStorage` now uses a designated error class for errors that might occur. In addition the dependency to Argobots was removed which was used to trigger `ABT_eventuals`, laying ground work for future non-Argobots IO implementations. Further, the whole class was refactored for consistency and failure resistance.

A new class `ChunkOperation` is introduced which wraps Argobots' IO task operations which allows the removal of IO queue specific code within RPC handlers, i.e., read and write handlers. The idea is to separate eventuals, tasks and their arguments from handler logic into a designated class. Therefore, an object of an inherited class of `ChunkOperation` is instantiated within the handlers that drives all IO tasks. The corresponding code was added to the read and write RPC handlers. Note, `ChunkOperation` is not thread-safe and is supposed to be called by a single thread.

In addition, truncate was reworked for error handling (it crashed the server on error) and that it uses the IO queue as well since truncate causes a write operation and should not overtake IO tasks in the queue.

The chunk stat rpc handler was refactored for error handling and to use error codes as well. 

Further minor changes:
- dead chunk stat code has been removed
- some namespaces were missing: `gkfs::rpc`
- more flexible handler cleanup and response code
- fixed a bug where the chunk dir wasn't removed when the metadata didn't exist on the same node

gkfs::hg_tag -> gkfs::rpc::tag
gkfs::utils -> gkfs::util
gkfs::client -> gkfs::util
gkfs::logging -> gkfs::log

Reformatted code w.r.t. unifying code styles as of the official
CPPCoreGuidelines by Stroustrup and Sutter

Restructuring code w.r.t. configurations and definitions:
- #defines have been mostly removed from configurations
- a dedicated config file has been added for configurations with constexpr
- past configure file is now only a cmake wrapper
- wrapping global functions into namespaces

Removed all adafs and ifs occurrences. Now called gkfs