Prepare to merge back into 'dev_toto'

CHANGELOG.md

@@ -6,6 +6,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.6.2] - 2019-10-07
+## Added
+ - Paths inside kernel pseudo filesystems (`/sys`, `/proc`) are forwarded directly to the kernel and internal path resolution will be skipped. Be aware that also paths  like  `/sys/../tmp/gkfs_mountpoint/asd` will be forwarded to the kernel
+ - Added new Cmake flag `CREATE_CHECK_PARENTS` to controls if the existance of the parent node needs to be checked during the creation of a child node.
+## Changed
+ - Daemon logs for RPC handlers have been polished
+ - Updated Margo, Mercury and Libfabric dependencies
+## Fixed
+ - mk_node RPC wasn't propagating errors correctly from daemons
+ - README has been improoved and got some minor fixes
+ - fix wrong path in log call for mk_symlink function
+
+## [0.6.1] - 2019-09-17
+## Added
+ - Added new Cmake flag `LOG_SYSCALLS` to enable/disable syscall logging.
+ - Intercept the 64 bit version of `getdents`.
+ - Added debian-based docker image.
+## Changed
+ - Disable syscalls logging by default
+ - Update Mercury, RocksDB and Libfabric dependencies
+## Fixed
+ - Fix read at the end of file.
+ - Don't create log file when using `--version`/`--help` cli flags.
+ - On some systems LD_PRELOAD used on /bin/bash binary was not working.
+ - Missing definition of `loff_t` on new version of GCC.
+
 ## [0.6.0] - 2019-07-26
 ## Added
 - Add compile time option to disable shared memory communication `-DUSE_SHM:BOOL=OFF`
@@ -23,7 +49,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## Fixed
 - Errors on get_dirents RPC are now reported back to clients
 - Write errors happenig on deamons are now reported back to clients
-- number overflow on lseek didn't allow to use seek on huge files 
+- number overflow on lseek didn't allow to use seek on huge files
 
 ## [0.5.0] - 2019-04-29
 ## Changed

README.md

@@ -102,15 +102,17 @@ optional arguments:
 
 ## Compile GekkoFS
 You need to decide what Mercury NA plugin you want to use. The following NA plugins are available, although only BMI is considered stable at the moment.
- - `ofi+tcp` for using the libfabric plugin with TCP
+ - `ofi+sockets` for using the libfabric plugin with TCP
+ - `ofi+tcp` for using the libfabric plugin with TCP (new version)
  - `ofi+verbs` for using the libfabric plugin with Infiniband verbs (not threadsafe. Do not use.)
  - `ofi+psm2` for using the libfabric plugin with Intel Omni-Path
  - `bmi+tcp` for using the bmi plugin with the tcp protocol 
 
 ```bash
 mkdir build && cd build
-cmake -DCMAKE_BUILD_TYPE=Release ..
+cmake -DCMAKE_BUILD_TYPE=Release -DRPC_PROTOCOL='ofi+sockets' ..
 make
+make install
 ```
 
 ## Run GekkoFS
@@ -121,7 +123,7 @@ parallel-ssh) with python2.
 
 ### Start and shut down daemon directly
 
-`./build/bin/gkfs_daemon -r <fs_data_path> -m <pseudo_mount_dir_path> --hosts <hosts_comma_separated>`
+`./build/bin/gkfs_daemon -r <fs_data_path> -m <pseudo_mount_dir_path>`
  
 Shut it down by gracefully killing the process.
  
@@ -134,7 +136,7 @@ The scripts are located in `scripts/{startup_gkfs.py, shutdown_gkfs.py}`. Use th
 Metadata and actual data will be stored at the `<fs_data_path>`. The path where the application works on is set with
 `<pseudo_mount_dir_path>`
  
-Run the application with the preload library: `LD_PRELOAD=<path>/build/lib/libiointer.so ./application`. In the case of
+Run the application with the preload library: `LD_PRELOAD=<path>/build/lib/libgkfs_intercept.so ./application`. In the case of
 an MPI application use the `{mpirun, mpiexec} -x` argument.
  
 ### Logging

docker/debian_build_env.docker

+FROM debian:stable-slim
+
+LABEL Description="This is a debian based environment to build GekkoFS"
+
+ENV GKFS_PATH	/opt/gkfs
+
+ENV SCRIPTS_PATH	${GKFS_PATH}/scripts
+ENV DEPS_SRC_PATH	${GKFS_PATH}/deps_src
+ENV INSTALL_PATH	${GKFS_PATH}/build_deps
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+		git \
+		curl \
+		ca-certificates \
+		libtool \
+		pkg-config \
+		make \
+		automake \
+		cmake \
+		gcc \
+		g++ \
+		# Mercury dependencies
+		libltdl-dev \
+		lbzip2 \
+		# RocksDB
+		libsnappy-dev \
+		liblz4-dev \
+		libzstd-dev \
+		libbz2-dev \
+		zlib1g-dev \
+		# syscall_intercept dependencies
+		libcapstone-dev \
+		# GekkoFS
+		libboost-filesystem-dev \
+		libboost-program-options-dev \
+		valgrind \
+		uuid-dev \
+# Clean apt cache to reduce image layer size
+&& rm -rf /var/lib/apt/lists/*
+
+# Download dependencies source
+COPY scripts/dl_dep.sh		$SCRIPTS_PATH/
+RUN /bin/bash $SCRIPTS_PATH/dl_dep.sh $DEPS_SRC_PATH all
+
+# Compile dependencies
+COPY scripts/compile_dep.sh $SCRIPTS_PATH/
+COPY scripts/patches        $SCRIPTS_PATH/patches
+RUN /bin/bash $SCRIPTS_PATH/compile_dep.sh $DEPS_SRC_PATH $INSTALL_PATH

include/client/adafs_functions.hpp

@@ -21,6 +21,8 @@ std::shared_ptr<Metadata> adafs_metadata(const std::string& path, bool follow_li
 
 int adafs_open(const std::string& path, mode_t mode, int flags);
 
+int check_parent_dir(const std::string& path);
+
 int adafs_mk_node(const std::string& path, mode_t mode);
 
 int check_parent_dir(const std::string& path);

include/client/hooks.hpp

@@ -17,6 +17,7 @@
 #include <sys/types.h>
 #include <fcntl.h>
 
+
 int hook_openat(int dirfd, const char *cpath, int flags, mode_t mode);
 int hook_close(int fd);
 int hook_stat(const char* path, struct stat* buf);

include/global/configure.hpp.in

@@ -71,4 +71,6 @@
 #define DEFAULT_CLIENT_LOG_LEVEL "info,errors,critical,mercury"
 #define DEFAULT_DAEMON_LOG_LEVEL 4 // info
 
+#cmakedefine01 LOG_SYSCALLS
+
 #endif //FS_CONFIGURE_H