Installing GekkoFS

This section describes how to build and install GekkoFS.

Base dependencies

GekkoFS relies on CMake for building and testing the daemon and client binaries, which are written in C++17. As such, GekkoFS has the following base dependencies for its build system:

  • A compiler with C++17 support:

    • GCC version 8.0 and newer

    • Clang 5.0 and newer.

  • CMake version 3.6 or newer to build the main code.

    • Version 3.11 or newer to build the tests.

GekkoFS also needs curl and git to be installed in your system if you plan on using the dependency management scripts provided with the code, as well as Python 3.6 or later to run some of the tests.

Additionally, since some GekkoFS dependencies rely on GNU Automake as their build system, it is also recommended to install the latest version of autoconf, automake, and libtool.

This set of dependencies are typically available in the system repositories. We show below how to install them for some of the most popular Linux distributions:

Debian/Ubuntu

# Base dependencies
$ apt install git curl cmake autoconf automake libtool libconfig-dev

# Testing support:
$ apt install python3-dev python3 python3-venv

CentOS/Red Hat

# Base dependencies:
$ yum install gcc-c++ git curl cmake autoconf automake libtool libconfig

# Testing support:
$ yum install python38-devel

Software dependencies

GekkoFS requires several software packages to be available in your system in order to function properly. We list them here for informational purposes.

Warning

Though it is possible to install all dependencies manually, GekkoFS provides some dependency management scripts that automate and greatly simplify this task. We strongly suggest that you take read through the step-by-step installation section before attempting a manual install.

  • RocksDB version 6.2.2 or newer and its used compression library:

    • lz4 version 1.8.0 or newer.

  • Margo version 0.6.3 and its dependencies:

  • syscall_intercept (commit f7cebb7 or newer) and its dependencies:

  • Date (commit e7e1482 or newer).

Important

Mercury may need additional plugins depending on the fabric technology used in the cluster. For instance, OmniPath requires the opa-psm2 library to work.

Optional dependencies

Additionally, some GekkoFS optional execution modes have additional dependencies:

  • AGIOS (commit c26a654 or newer) to enable the GekkoFWD I/O forwarding mode.

  • PARALLAX There are two different metadata backends in GekkoFS. The default one uses rocksdb, however

an alternative based on PARALLAX from FORTH is available. To enable it, use the -DGKFS_ENABLE_PARALLAX:BOOL=ON option, you can also disable rocksdb with -DGKFS_ENABLE_ROCKSDB:BOOL=OFF. Once it is enabled, --dbbackend option will be functional.

Step-by-step installation

  1. Make sure that the GekkoFS base dependencies are available on your machine.

  2. Clone GekkoFS:

    $ git clone --recurse-submodules https://storage.bsc.es/gitlab/hpc/gekkofs.git

    (Optional) If you checked out the sources using git without the :code`–recursive` option, you need to execute the following command from the root of the source directory:

    $ git submodule update --init

  3. Set up the necessary environment variables where the compiled GekkoFS software dependencies should be installed at. Throughout this example we assume dependencies will live in the

    /home/foo/gekkofs_deps/install directory):

    $ export GKFS_INSTALL_PATH=/home/foo/gekkofs_deps/install
$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${GKFS_INSTALL_PATH}/lib:${GKFS_INSTALL_PATH}/lib64

  4. Download all the required dependencies using dl_dep.sh, one of GekkoFS’ dependency management scripts. We will use the /home/foo/gekkofs_deps/git directory to keep all the source files:

    $ gekkofs/scripts/dl_dep.sh /home/foo/gekkofs_deps/git

Important

The /home/foo/gekkofs_deps/git directory containing the source code for all our downloaded dependencies can be safely removed once installation is complete.

  1. Build and install the dependencies into GKFS_INSTALL_PATH using compile_dep.sh, the second GekkoFS’ dependency management script:

    $ gekkofs/scripts/compile_dep.sh /home/foo/gekkofs_deps/git /home/foo/gekkofs_deps/install

  2. Now let’s configure the GekkoFS build by setting the appropriate options. GekkoFS makes use of the CMake build system and requires that you do an out-of-source build. In order to do that, you must create a new build directory and run the cmake command from it:

    # Create the build directory:
$ cd gekkofs
$ mkdir build && cd build
$ cmake \
     -DCMAKE_BUILD_TYPE:STRING=Release \
     -DCMAKE_PREFIX_PATH:STRING=/home/foo/gekkofs_deps/install \
     -DCMAKE_INSTALL_PREFIX:STRING=/home/foo/gekkofs_deps/install \
     -DGKFS_BUILD_TESTS:BOOL=ON \
     ..

    For this example, we set the CMAKE_BUILD_TYPE variable to Release to let CMake know that we need an optimized build. It is very important to make CMake aware of where GekkoFS dependencies are installed, which is why we set CMAKE_PREFIX_PATH to ${GKFS_INSTALL_PATH}. We also set CMAKE_INSTALL_PREFIX to the same directory, because we want the GekkoFS binaries to be installed in the same location. Finally, we also enable the compilation of GekkoFS tests (which are not enabled by default) by setting GKFS_BUILD_TESTS.

Attention

If you prefer a more interactive approach, it is also possible to use ccmake or cmake-gui to configure the package.

  1. We are finally ready to build, test and install GekkoFS:

    $ make -j8
$ make test
$ make install

After following this guide, GekkoFS binaries should now be available in the appropriate subdirectories of GKFS_INSTALL_PATH:

  • GekkoFS daemon (server): ${GKFS_INSTALL_PATH}/bin/gkfs_daemon

  • GekkoFS client interception library: ${GKFS_INSTALL_PATH}/lib/libgkfs_intercept.so

Install GekkoFS via Spack (alternative)

Spack is a package manager for supercomputers and Linux. It makes it easy to install scientific software for regular users. Spack is another method to install GekkoFS where Spack handles all the dependencies and setting up the environment.

Install Spack

First, install Spack. You can find the instructions here: https://spack.readthedocs.io/en/latest/getting_started.html

git clone https://github.com/spack/spack.git
. spack/share/spack/setup-env.sh

Attention

Note that the second line needs to be executed every time you open a new terminal. It sets up the environment for Spack

and the corresponding environment variables, e.g., $PATH.

Install GekkoFS with Spack

To install GekkoFS with Spack, the GekkoFS repository needs to be added to Spack as it is not part of the official Spack repository.

spack repo add gekkofs/scripts/spack

When added, the GekkoFS package is available. Its installation variants and options can be checked via:

spack info gekkofs

Then install GekkoFS with Spack:

spack install gekkofs
# for installing tests dependencies and running tests
spack install -v --test=root gekkofs

Finally, GekkoFS is loaded into the currently used environment:

spack load gekkofs

This installs the latest release version including its required Git submodules. The installation directory is $SPACK_ROOT/opt/spack/linux-<arch>/<compiler>/<version>/gekkofs-<version>. The GekkoFS daemon (gkfs_daemon) is located in the bin directory and the GekkoFS client (libgkfs_intercept.so) is located in the lib directory.

Note that loading the environment adds the GekkoFS daemon to the $PATH environment variable. Therefore, the GekkoFS daemon is started by running gkfs_daemon. Loading GekkoFS in Spack further provides the $GKFS_CLIENT environment variable pointing to the interception library.

Therefore, the following commands can be run to use GekkoFS:

# Consult `-h` or the Readme for further options
gkfs_daemon -r /tmp/gkfs_rootdir -m /tmp/gkfs_mountdir &
LD_PRELOAD=$GKFS_CLIENT ls -l /tmp/gkfs_mountdir
LD_PRELOAD=$GKFS_CLIENT touch /tmp/gkfs_mountdir/foo
LD_PRELOAD=$GKFS_CLIENT ls -l /tmp/gkfs_mountdir

When done using GekkoFS, unload it from the environment:

spack unload gekkofs

Miscellaneous

Use GekkoFS’s latest version (master branch) with Spack:

spack install gekkofs@latest

Use a specific compiler on your system, e.g., gcc-11.2.0:

spack install gekkofs@latest%gcc@11.2.0