Newer
Older
# ADA-FS
This is a file system.
# Dependencies
## Rocksdb
### Debian/Ubuntu - Dependencies
- Upgrade your gcc to version at least 4.8 to get C++11 support.
- Install gflags. First, try: `sudo apt-get install libgflags-dev` If this doesn't work and you're using Ubuntu, here's
a nice tutorial: (http://askubuntu.com/questions/312173/installing-gflags-12-04)
- Install snappy. This is usually as easy as: `sudo apt-get install libsnappy-dev`
- Install zlib. Try: `sudo apt-get install zlib1g-dev`
- Install bzip2: `sudo apt-get install libbz2-dev`
- Install zstandard: `sudo apt-get install libzstd-dev`
- Install lz4 `sudo apt-get install liblz4-dev`
### CentOS/Red Hat - Dependencies
- Upgrade your gcc to version at least 4.8 to get C++11 support: yum install gcc48-c++
- Install gflags:
```bash
git clone https://github.com/gflags/gflags.git
cd gflags
git checkout v2.0
./configure && make && sudo make install
```
__Notice:__ Once installed, please add the include path for gflags to your CPATH environment variable and the lib path
to LIBRARY_PATH. If installed with default settings, the include path will be /usr/local/include and the lib path will
be /usr/local/lib.
- Install snappy:
`sudo yum install snappy snappy-devel`
- Install zlib:
`sudo yum install zlib zlib-devel`
- Install bzip2:
`sudo yum install bzip2 bzip2-devel`
- Install ASAN (optional for debugging):
`sudo yum install libasan`
- Install zstandard:
```bash
wget https://github.com/facebook/zstd/archive/v1.1.3.tar.gz
mv v1.1.3.tar.gz zstd-1.1.3.tar.gz
tar zxvf zstd-1.1.3.tar.gz
cd zstd-1.1.3
make && sudo make install
```
# Usage
## Clone and compile direct ADA-FS dependencies
- Go to the subfolder `ifs/scripts` and first clone all dependencies projects. You can choose the according na_plugin
(execute the script for help):
```bash
usage: dl_dep.sh [-h] [-n <NAPLUGIN>] [-c <CLUSTER>]
source_path
This script gets all ADA-FS dependency sources (excluding the fs itself)
positional arguments:
source_path path where the dependency downloads are put
optional arguments:
-h, --help shows this help message and exits
-n <NAPLUGIN>, --na <NAPLUGIN>
network layer that is used for communication. Valid: {bmi,cci,ofi,all}
defaults to 'all'
-c <CLUSTER>, --cluster <CLUSTER>
additional configurations for specific compute clusters
supported clusters: {mogon1,fh2}
```
- Now use the install script to compile them and install them to the desired directory. You can choose the according
na_plugin (execute the script for help):
```bash
usage: compile_dep.sh [-h] [-n <NAPLUGIN>] [-c <CLUSTER>] [-j <COMPILE_CORES>]
source_path install_path
This script compiles all ADA-FS dependencies (excluding the fs itself)
positional arguments:
source_path path to the cloned dependencies path from clone_dep.sh
install_path path to the install path of the compiled dependencies
optional arguments:
-h, --help shows this help message and exits
-n <NAPLUGIN>, --na <NAPLUGIN>
network layer that is used for communication. Valid: {bmi,cci,ofi,all}
defaults to 'all'
-c <CLUSTER>, --cluster <CLUSTER>
additional configurations for specific compute clusters
supported clusters: {mogon1,mogon2,fh2}
-j <COMPILE_CORES>, --compilecores <COMPILE_CORES>
number of cores that are used to compile the depdencies
defaults to number of available cores
```
You need to decide what Mercury NA plugin you want to use. The following NA plugins are available, although only BMI is considered stable at the moment.
- `ofi+tcp` for using the libfabric plugin with TCP
- `ofi+verbs` for using the libfabric plugin with Infiniband verbs (not threadsafe. Do not use.)
- `ofi+psm2` for using the libfabric plugin with Intel Omni-Path
- `cci+verbs` for using the cci plugin with Infiniband verbs
- `bmi+tcp` for using the bmi plugin with the tcp protocol
In addition you can add a specific directory where all dependencies are located, i.e., headers and libraries. This can
be done by using `-DADAFS_DEPS_INSTALL=<path>`. If the variable is not set this path points to `/usr/local`.
cmake -DADAFS_DEPS_INSTALL=<path> -DCMAKE_BUILD_TYPE={Release, Debug}} ..
First on each node a daemon has to be started. This can be done in two ways using the `adafs_daemon` binary directly or
the corresponding startup and shutdown scripts. The latter is recommended for cluster usage. It requires pssh (or
parallel-ssh) with python2.
### Start and shut down daemon directly
`./build/bin/adafs_daemon -r <fs_data_path> -m <pseudo_mount_dir_path> --hosts <hosts_comma_separated>`
Shut it down by gracefully killing the process.
### Startup and shutdown scripts
The scripts are located in `ifs/scripts/{startup_adafs.py, shutdown_adafs.py}`. Use the -h argument for their usage.
## Miscellaneous
Metadata and actual data will be stored at the `<fs_data_path>`. The path where the application works on is set with
`<pseudo_mount_dir_path>`
Run the application with the preload library: `LD_PRELOAD=<path>/build/lib/libiointer.so ./application`. In the case of
an MPI application use the `{mpirun, mpiexec} -x` argument.
Please consult `configure_public.hpp` for log locations and verbosity, etc. `include/configure.hpp` contains file system
specific configurations normal user should not be needed to modify.