Closes #88 
Closes #125 

The distributor will read a shared file including [filename] [host] [size] [offset] and distribute the files accordingly.
We assume, as it should happen in SLURM that the nodes are in alphabetical order.

Distributors are instantiated per read/write operation. They should be moved to FsData to avoid the overhead. It is also necessary to reduce the creation cost when the distributor reads the data mappings from a file or similar.

See merge request !39

Ramon Nou authored Oct 01, 2020 and Ramon Nou committed Jun 16, 2021

Moved distributor instantation to RPC_DATA and daemon.cpp on the server side. Added interval_map to GuidedDistributor for better performance

Moving reads extended log to normal log module.

Update logging.hpp

Added GKFS prefix to CMake options

Catch2 Guided Distributor test

Added creation of /tmp/guided.txt in the test

updated README.MD and protected Distributor for duplicated inserts

Added metadata + data optimization with #

Resolve "Add functions for socketcall system call"

Closes #150

See merge request !91

Resolve "Rename 'global' directory to 'common'"

Closes #127

See merge request !84

Resolve "Change license to GPL"

Closes #146

See merge request !88

Resolve "Fix clang-format pipeline typo"

Closes #144

See merge request !86

This MR deletes the outdated scripts used for managing copyright headers and provides a new one in `scripts/maintainer-tools/copyright-headers`. Also, similarly to before, it also provides a Dockerfile for a `gekkofs/copyright-header` image that encapsulates a version of Osterman's [copyright-header](https://github.com/osterman/copyright-header/) Gem appropriately patched for our needs. Please refer to the README.md file in `scripts/maintainer-tools/copyright-headers` for more details.

Closes #142

See merge request !83

This MR enables the generation of code coverage reports for the source code. More specifically, it introduces the following changes:

- Adds the `gkfs-code-coverage.cmake` CMake script under `{PROJECT_ROOT}/CMake/` and includes it in the project's main `CMakeLists.txt`. This script defines the `GKFS_ENABLE_CODE_COVERAGE` option to control whether code coverage should be enabled when building, as well as the `target_code_coverage()` function to ensure that the appropriate flags are added to a CMake target.
- Adds `gcovr` to GekkoFS docker build image.
- Adds a `coverage.sh` script under `{PROJECT_ROOT}/scripts/ci` that provides both a `capture` and a `merge` work modes. This script abstracts away the direct usage of `gcovr` and allows the appropriate generation of coverage reports when running tests **in parallel** in the CI. The script relies on a `.coverage-exclusions` file (also under `{PROJECT_ROOT}/scripts/ci`) to control which paths should be excluded from coverage reporting (e.g. `/usr/include/.*`).
- Adds a new `report` stage in the CI pipeline that upon completion generates artifacts both for a Cobertura XML and a HTML code coverage reports.

See merge request !77

Resolve "Add path resolution tests"

Closes #108

See merge request !48

Optimization of create, stat, and remove operations. The following changes have been made:

- `gkfs_open()` logic refactored.
- create: Previously a stat RPC was send before each create to make sure the file doesn't exist. This logic is now implicit in the create operation on the daemon side.
- `get_metadata()` was used on the client for all stat operation, creating a `shared_ptr` for the `Metadata` object in the process which was not needed. A new function `get_metadata_attr()` was created to only return the metadata binary string, which will only create the metadata object if actually required.
- Remove logic was separated into two operations: `remove_metadata()` and `remove_data()`. 
- `gkfs::config::metadata::implicit_data_removal` setting: If `true`, will remove data on the same node during the `remove_metadata()` RPC. This is mainly an optimization, but will be useful for future asynchronous removal implementations.

Previously, the code path looked like this:
1. `stat()` to get `size` and `mode`.
2. Use these fields to determine if data needs to be removed or just metadata.
3. `remove()` is called which, first, sends a single RPC to the daemon with the metadata. Afterwards, data is removed.
4. The daemon used one handler for both cases.

It now looks like this: 
1. `remove()` is called which, first, sends a single RPC to the daemon with the metadata. Before, removing the metadata, the daemon fetches `mode` and `size`. If `implicit_data_removal` is set as a configuration, the data is removed in this RPC as well. `mode` and `size` are returned to the client.
2. The client determines if data needs to be removed as well.
3. If yes, sends a `remove_data()` RPC as it was previously.

Depends on !66 and !74.

Closes #94

See merge request !60

Removes 1 stat RPC per remove operation

Resolve "Don't use popen directly in integration tests"

Closes #141

See merge request hpc/gekkofs!80