Verified Commit 006c7d4e authored by Alberto Miranda's avatar Alberto Miranda ♨️
Browse files

Prepare to merge back into 'dev_toto'

parents 4f6e0409 866b8016
Loading
Loading
Loading
Loading
Loading
+27 −1
Original line number Diff line number Diff line
@@ -6,6 +6,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.6.2] - 2019-10-07
## Added
 - Paths inside kernel pseudo filesystems (`/sys`, `/proc`) are forwarded directly to the kernel and internal path resolution will be skipped. Be aware that also paths  like  `/sys/../tmp/gkfs_mountpoint/asd` will be forwarded to the kernel
 - Added new Cmake flag `CREATE_CHECK_PARENTS` to controls if the existance of the parent node needs to be checked during the creation of a child node.
## Changed
 - Daemon logs for RPC handlers have been polished
 - Updated Margo, Mercury and Libfabric dependencies
## Fixed
 - mk_node RPC wasn't propagating errors correctly from daemons
 - README has been improoved and got some minor fixes
 - fix wrong path in log call for mk_symlink function

## [0.6.1] - 2019-09-17
## Added
 - Added new Cmake flag `LOG_SYSCALLS` to enable/disable syscall logging.
 - Intercept the 64 bit version of `getdents`.
 - Added debian-based docker image.
## Changed
 - Disable syscalls logging by default
 - Update Mercury, RocksDB and Libfabric dependencies
## Fixed
 - Fix read at the end of file.
 - Don't create log file when using `--version`/`--help` cli flags.
 - On some systems LD_PRELOAD used on /bin/bash binary was not working.
 - Missing definition of `loff_t` on new version of GCC.

## [0.6.0] - 2019-07-26
## Added
- Add compile time option to disable shared memory communication `-DUSE_SHM:BOOL=OFF`
+6 −4
Original line number Diff line number Diff line
@@ -102,15 +102,17 @@ optional arguments:

## Compile GekkoFS
You need to decide what Mercury NA plugin you want to use. The following NA plugins are available, although only BMI is considered stable at the moment.
 - `ofi+tcp` for using the libfabric plugin with TCP
 - `ofi+sockets` for using the libfabric plugin with TCP
 - `ofi+tcp` for using the libfabric plugin with TCP (new version)
 - `ofi+verbs` for using the libfabric plugin with Infiniband verbs (not threadsafe. Do not use.)
 - `ofi+psm2` for using the libfabric plugin with Intel Omni-Path
 - `bmi+tcp` for using the bmi plugin with the tcp protocol 

```bash
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake -DCMAKE_BUILD_TYPE=Release -DRPC_PROTOCOL='ofi+sockets' ..
make
make install
```

## Run GekkoFS
@@ -121,7 +123,7 @@ parallel-ssh) with python2.

### Start and shut down daemon directly

`./build/bin/gkfs_daemon -r <fs_data_path> -m <pseudo_mount_dir_path> --hosts <hosts_comma_separated>`
`./build/bin/gkfs_daemon -r <fs_data_path> -m <pseudo_mount_dir_path>`
 
Shut it down by gracefully killing the process.
 
@@ -134,7 +136,7 @@ The scripts are located in `scripts/{startup_gkfs.py, shutdown_gkfs.py}`. Use th
Metadata and actual data will be stored at the `<fs_data_path>`. The path where the application works on is set with
`<pseudo_mount_dir_path>`
 
Run the application with the preload library: `LD_PRELOAD=<path>/build/lib/libiointer.so ./application`. In the case of
Run the application with the preload library: `LD_PRELOAD=<path>/build/lib/libgkfs_intercept.so ./application`. In the case of
an MPI application use the `{mpirun, mpiexec} -x` argument.
 
### Logging
+48 −0
Original line number Diff line number Diff line
FROM debian:stable-slim

LABEL Description="This is a debian based environment to build GekkoFS"

ENV GKFS_PATH	/opt/gkfs

ENV SCRIPTS_PATH	${GKFS_PATH}/scripts
ENV DEPS_SRC_PATH	${GKFS_PATH}/deps_src
ENV INSTALL_PATH	${GKFS_PATH}/build_deps

RUN apt-get update && apt-get install -y --no-install-recommends \
		git \
		curl \
		ca-certificates \
		libtool \
		pkg-config \
		make \
		automake \
		cmake \
		gcc \
		g++ \
		# Mercury dependencies
		libltdl-dev \
		lbzip2 \
		# RocksDB
		libsnappy-dev \
		liblz4-dev \
		libzstd-dev \
		libbz2-dev \
		zlib1g-dev \
		# syscall_intercept dependencies
		libcapstone-dev \
		# GekkoFS
		libboost-filesystem-dev \
		libboost-program-options-dev \
		valgrind \
		uuid-dev \
# Clean apt cache to reduce image layer size
&& rm -rf /var/lib/apt/lists/*

# Download dependencies source
COPY scripts/dl_dep.sh		$SCRIPTS_PATH/
RUN /bin/bash $SCRIPTS_PATH/dl_dep.sh $DEPS_SRC_PATH all

# Compile dependencies
COPY scripts/compile_dep.sh $SCRIPTS_PATH/
COPY scripts/patches        $SCRIPTS_PATH/patches
RUN /bin/bash $SCRIPTS_PATH/compile_dep.sh $DEPS_SRC_PATH $INSTALL_PATH
+2 −0
Original line number Diff line number Diff line
@@ -21,6 +21,8 @@ std::shared_ptr<Metadata> adafs_metadata(const std::string& path, bool follow_li

int adafs_open(const std::string& path, mode_t mode, int flags);

int check_parent_dir(const std::string& path);

int adafs_mk_node(const std::string& path, mode_t mode);

int check_parent_dir(const std::string& path);
+1 −0
Original line number Diff line number Diff line
@@ -17,6 +17,7 @@
#include <sys/types.h>
#include <fcntl.h>


int hook_openat(int dirfd, const char *cpath, int flags, mode_t mode);
int hook_close(int fd);
int hook_stat(const char* path, struct stat* buf);
Loading