Cargo
[](https://storage.bsc.es/gitlab/hpc/cargo/-/pipelines)
[](https://storage.bsc.es/gitlab/hpc/cargo/-/releases)
[](https://storage.bsc.es/gitlab/hpc/cargo/-/blob/main/COPYING)
[](https://en.wikipedia.org/wiki/C%2B%2B20)
A parallel data staging service for HPC clusters
Cargo is a HPC data staging service that runs alongside applications helping
them to transfer data in parallel between local and shared storage tiers.
> **Note**
>
> This software was partially supported by the EuroHPC-funded project ADMIRE
> (Project ID: 956748, https://www.admire-eurohpc.eu).
## Building Cargo
### Building Cargo and its dependencies with Spack
Cargo and its dependencies can be built using
[Spack](https://github.com/spack/spack). If you already have Spack, make sure
you have the latest release. If you use a clone of the Spack `develop`
branch, be sure to pull the latest changes.
#### Install Spack
If you haven't already, install Spack with the following commands:
```shell
$ git clone -c feature.manyFiles=true https://github.com/spack/spack
```
This will create a directory called `spack` in your machine. Once you have
cloned Spack, we recommend sourcing the appropriate script for your shell.
This will add Spack to your PATH and enable the use of the `spack` command:
```shell
# For bash/zsh/sh
$ . spack/share/spack/setup-env.sh
# For tcsh/csh
$ source spack/share/spack/setup-env.csh
# For fish
$ . spack/share/spack/setup-env.fish
```
Since `Cargo` is not yet available in the official Spack repositories, you need
to add the Cargo Spack repository to the Spack namespace in your machine. To do
that, download the `spack/` directory in the `Cargo` repository's root to your
machine (e.g. under `~/projects/cargo/spack`) and execute the following:
```shell
spack repo add ~/projects/cargo/spack/
```
You should now be able to fetch information from the `Cargo` package using
Spack:
```shell
spack info cargo
```
You are now ready to install `Cargo`:
```shell
spack install cargo
```
Include or remove variants with Spack when a custom `Cargo` build is desired.
The available variants are listed below:
| Variant | Command | Default | Description |
|---------|-------------|---------|------------------------------------|
| OFI | `cargo+ofi` | True | Use libfabric as transport library |
| UCX | `cargo+ucx` | False | Use ucx as transport library |
> **Attention**
>
> The initial install could take a while as Spack will install build
> dependencies (autoconf, automake, m4, libtool, and pkg-config) as well as
> any dependencies of dependencies (cmake, perl, etc.) if you don’t already
> have these dependencies installed through Spack.
After the installation completes, remember that you first need to load
`Cargo` in order to use it:
```shell
spack load cargo
```
### Building Cargo manually
If you prefer to build and install `Cargo` from sources, you can also do so.
For the build process to work correctly, the dependencies below will
need to be available in your system:
| Dependency | Version |
|----------------------------------------------------|---------------------------|
| Margo | v0.9.8+ |
| Argobots | v1.1+ |
| Mercury | v2.1.0+ |
| Thallium | v0.10.1+ |
| libfabric (if `CARGO_TRANSPORT_LIBRARY=libfabric`) | v0.10.1+ |
| ucx (if `CARGO_TRANSPORT_LIBRARY=ucx`) | v0.10.1+ |
| boost program_options | v1.71.0+ |
| boost mpi | v1.71.0+ |
| boost iostreams (optional, for testing) | v1.71.0+ |
| MPI | tested with OpenMPI 4.0.3 |
Once all dependencies are available, you can download build and install
`Cargo` with the following commands:
```shell
## clone the repository
git clone https://storage.bsc.es/gitlab/hpc/cargo.git
cd cargo
## prepare the CMake build
#
# PREFIX="some_dir_where_dependencies_can_be_found"
# INSTALL_DIR="some_dir_where_you_want_cargo_installed"
mkdir build && cd build
## build and install
cmake \
-DCMAKE_PREFIX_PATH:STRING="${PREFIX};${CMAKE_PREFIX_PATH}" \
-DCMAKE_INSTALL_PREFIX:STRING="${INSTALL_DIR}" \
-DCARGO_TRANSPORT_LIBRARY:STRING=libfabric \
-DCARGO_BUILD_TESTS:BOOL=ON \
..
make -j8 install
```
These commands will generate and install the `Cargo` server binary
(`${INSTALL_DIR}/bin/cargo`) as well as the `Cargo` interface
library (`${INSTALL_DIR}/lib/libcargo.so`) and its headers
(`${INSTALL_DIR}/include/cargo/*`).
## Testing
Tests can be run automatically with CTest:
```shell
cd build
ctest -VV --output-on-failure --stop-on-failure -j 8
```
When this happens, a Cargo server with 3 workers is automatically started
(via `mpirun`/`mpiexec`) and stopped (via RPC) so that tests can progress.
Alternatively, during development one may desire to run the Cargo server
manually and then the tests. In this case, the following commands can be used:
```shell
# start the Cargo server with 3 workers. The server will be listening on
# port 62000 and will communicate with workers via MPI messages. The server can
# be stopped with Ctrl+C, `kill -TERM