Merge branch '49-daemon-add-option-to-not-use-a-separate-pid-directory-for-the-rootdir' into 'master'

Resolve "[daemon] Allow daemon restart without losing the file system state"

Closes #49

See merge request !85

Marc Vef authored Jan 26, 2022 and Ramon Nou committed Feb 04, 2022

A subdirectory is no longer created by default. Therefore, a server restart uses same directory for both data and metadata. Note, that the rootdir can be cleaned with the -c argument. For multiple daemons on one machine, the new argument --rootdir-suffix can be used which creates a subdirectory within the rootdir with a user-defined name. This also allows restarting multiple daemons on one node without losing data.

Resolve "faster data integrity test / faster directory test"

Closes #186

See merge request !121

Resolve "Extend directory tests"

Closes #194

See merge request !120

Resolve "Logging line from harness should be reduced"

Closes #192

See merge request !119

Resolve "Add daemon doxygen documentation"

See merge request !109

Resolve "Bump all dependencies"

Closes #179, #145, and #46

See merge request !112

8 xstreams is not necessary. might even be lowered to 2 eventually. Further, using 8 xstreams decreased io performance on psm2 severely

Also, add procps package to gekkofs/core

Resolve "Replace boost::program_options with CLI11"

Closes #149

See merge request !90

Boost preprocessor is header-only and supplied by the Mercury installation. Nevertheless, we never tested for it although it needs to exist for testing.

Merge branch 'amiranda/176-using-d-option-in-dl_dep-sh-ends-up-downloading-all-dependencies' into 'master'

Resolve "Various issues with dependency scripts"

This MR adds some tests for the scripts based on [bats-core](https://github.com/bats-core/bats-core), and enables its automatic execution by the CI.

It also changes the separator for dependency flavors from `:` to `%` to prevent clashing with the version separator for profiles (e.g. `libfabric:experimental@mogon2:latest` was ambiguous to `bash`).

Two new bash functions `determine_compiler()` and `profile_has_dependency()` are added to `compile_dep.sh` that are made available to `install` scripts:

- The `determine_compiler()` function makes its best to determine the default compiler available in the system. Installation scripts can access this information via the `COMPILER_NAME`, `COMPILER_FULL_VERSION`, and `COMPILER_MAJOR_VERSION` variables. 
- The `profile_has_dependency()` function allows installation scripts to check whether another dependency is provided by the currently loaded profile, and act accordingly. The function accepts a bash regex with the search argument.

Additionally, this MR fixes the following problems in `dl_dep.sh` and `compile_dep.sh`:

- [x] Using `-d` option in `dl_dep.sh` ends up downloading all dependencies (#176)
- [x] Specifying a full profile in `dl_dep.sh` with `-d` does not work (#177)
- [x] `dl_deps.sh` can download the same dependency in several versions but unpacks them into the same directory (#178)
- [x] ~~rocksdb install of newer versions needs PREFIX for `make install` to specify install path (#180)~~ (Moving to CMake for 0.9.0
- [x] When `clonedeps` is using a tag/branchname to clone, e.g., `HEAD@v1.12.1`, `clonedeps_args` for the dependency needs explicit space, e.g., `["mercury"]=" --recurse-submodules"` otherwise the script fails.
- [x] Dependency patches are being skipped (#184).
- [x] Script should use script directory as `pwd` instead of `bash_pwd`?


Closes #176 #177 #178 #180 #184

See merge request !111