Commit 380b8ca7 authored by Marc Vef's avatar Marc Vef
Browse files

Merge branch...

Merge branch 'marc/240-syscall_intercept-compilation-fails-if-clone3-syscall-does-not-exist' into 'master'

Resolve "Syscall_intercept: compilation fails if clone3 syscall does not exist"

Closes #240

See merge request !157
parents e32fd199 97667ab2
Loading
Loading
Loading
Loading
Loading
+189 −114
Original line number Diff line number Diff line
@@ -6,29 +6,43 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### New

- Additional tests to increase code coverage ([!141](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/141)).
- GKFS_ENABLE_UNUSED_FUNCTIONS added to disable code to increase code coverage.

### Changed
- Support parallelism for path resolution tests ([!145](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/145)).

- Support parallelism for path resolution tests ([!145](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/145))
  .
- Support parallelism for symlink tests ([!147](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/147)).

### Removed

### Fixed
- Using `unlink` now fails if it is a directory unless the `AT_REMOVEDIR` flag is used (POSIX compliance) ([!139](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/139)).
- Support glibc-2.34 or newer with syscall_intercept [!146](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/146)).
- Additional `#ifdef` to remove unused code ([!141](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/141))
### Removed
### Fixed
- Fixed a bug which caused GekkoFS tests to
  fail ([!235](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/235)).
- Fixed a compilation error for testing ([!234](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/234)).

- Updated daemon log level for tests ([!138](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/138)).
- Using `unlink` now fails if it is a directory unless the `AT_REMOVEDIR` flag is used (POSIX
  compliance) ([!139](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/139)).
- fchdir generate a SIGSEV in debug mode (due to
  log) ([!141](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/141))
- Support glibc-2.34 or newer with
  syscall_intercept [!146](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/146)).
- Fixed segfault in sfind/gfind ([!148](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/148)
- Fixed fstatat to be able to understand `AT_EMPTY_PATH` flag used in coreutils (`cat`
  ...) ([!149](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/149)).
- Fixed segfault in sfind/gfind ([!227](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/227)
- Updated CI pipeline for Gitlab version 15.1.1 ([!150](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/150))
  .
- Fixed a compilation error for testing ([!153](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/153)).
- Fixed a bug which caused GekkoFS tests to
  fail ([!154](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/154)).
- Fixed RocksDB compilation for future GCC
  versions ([!155](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/155)).
- Fixed an issue where GekkoFS would halt when applications would create child processes, e.g., with fork, clone,
  pthread_create ([!156](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/156)
- Fixed an issue where compilation of syscall_intercept would fail for newer kernels using the `clone3()` system
  call ([!157](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/157)

## [0.9.1] - 2022-04-29

@@ -41,7 +55,8 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
    - `--enable-collection` collects normal statistics.
    - `--enable-chunkstats` collects extended chunk statistics.
- Statistics output to file is controlled by `--output-stats <filename>`
- Added Prometheus support for outputting statistics ([!132](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/132)):
- Added Prometheus support for outputting
  statistics ([!132](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/132)):
  - Prometheus dependency optional and enabled at compile time with the CMake argument `GKFS_ENABLE_PROMETHEUS`.
  - `--enable-prometheus` enables statistics pushing to Prometheus if statistics are enabled.
  - `--prometheus-gateway` sets an IP and port for the Prometheus connection.
@@ -57,16 +72,22 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- Environment variable to change Daemon log levels was changed from `GKFS_LOG_LEVEL`
  to `GKFS_DAEMON_LOG_LEVEL` ([!135](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/135)).
- Update Catch2 to support newer glibc library ([!131](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/131)).
- Adding support for `faccessat2()` and `getxattr()` system calls ([!209](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/209)).
- Adding support for `faccessat2()` and `getxattr()` system
  calls ([!209](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/209)).

### Removed

- Removed old initialization code in the GekkoFS
  client ([!136](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/136)).
- Removed boost interval dependencies from guided distributor ([!122](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/122)).
- Removed boost interval dependencies from guided
  distributor ([!122](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/122)).

### Fixed
- Documentation: Doxygen now includes private struct and class members ([!125](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/125)).
- Guided distributor tests are no longer run when they are turned off ([!129](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/129)).

- Documentation: Doxygen now includes private struct and class
  members ([!125](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/125)).
- Guided distributor tests are no longer run when they are turned
  off ([!129](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/129)).

## [0.9.0] - 2022-02-22

@@ -105,7 +126,9 @@ on a per directory or file basis ([!39](https://storage.bsc.es/gitlab/hpc/gekkof
    logged, among others ([!98](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/98)).
  - A convenience library has been added for `path_util`
    ([!102](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/102)).

### Changed

- GekkoFS license has been changed to GNU General Public License version 3
  ([!88](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/88))
- Create, stat, and remove operation have been refactored and improved, reducing
@@ -122,27 +145,40 @@ and improved ([!111](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/
  ([!112](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/112)).

### Removed
- Boost is no longer used for the client and daemon ([!90](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/90), 

- Boost is no longer used for the client and
  daemon ([!90](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/90),
  [!122](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/122),
  [!123](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/123)).
  Note that tests still require `Boost_preprocessor`.
- Unneeded sources in CMake have been removed ([!101](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/101)).

### Fixed
- Building tests no longer proceeds if virtualenv creation fails ([!68](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/68)).
- An error where unit tests could not be found has been fixed ([!79](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/79)).
- The daemon can now be restarted without losing its namespace ([!85](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/85)).
- An issue has been resolved that required AGIOS even if it wasn't been used ([!104](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/104)).
- Several issues that caused docker images to fail has been resolved ([!105](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/105), 
[!106](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/106), [!107](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/107), 

- Building tests no longer proceeds if virtualenv creation
  fails ([!68](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/68)).
- An error where unit tests could not be found has been
  fixed ([!79](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/79)).
- The daemon can now be restarted without losing its
  namespace ([!85](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/85)).
- An issue has been resolved that required AGIOS even if it wasn't been
  used ([!104](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/104)).
- Several issues that caused docker images to fail has been
  resolved ([!105](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/105),
  [!106](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/106)
  , [!107](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/107),
  [!114](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/114)).
- An CMake issue in `path_util` that caused the compilation to fail was fixed ([!115](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/115)). 
- An CMake issue in `path_util` that caused the compilation to fail was
  fixed ([!115](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/115)).
- Fixed an issue where `ls` failed because newer kernels use `fstatat()` with `EMPTY_PATH`
  ([!116](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/116)).
- Fixed an issue where `LOG_OUTPUT_TRUNC` did not work as expected ([!118](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/118)).
- Fixed an issue where `LOG_OUTPUT_TRUNC` did not work as
  expected ([!118](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/118)).

## [0.8.0] - 2020-09-15

### New

- Both client library and daemon have been extended to support the ofi+verbs
  protocol.
- A new Python testing harness has been implemented to support integration
@@ -168,13 +204,17 @@ Note that tests still require `Boost_preprocessor`.
  daemons, the requests are scheduled using the AGIOS scheduling library before
  they are dispatched to the shared backend parallel file system.
- The `fsync()` system call is now fully supported.

### Improved

- Argobots tasks in the daemon are now wrapped in a dedicated class,
  effectively removing the dependency. This lays ground work for future
  non-Argobots I/O implementations.
- The `readdir()` implementation has been refactored and improved.
- Improvements on how to the installation scripts manage dependencies.

### Fixed

- The server sometimes crashed due to uncaught system errors in the storage
  backend. This has now been fixed.
- Fixed a bug that broke `ls` on some architectures.
@@ -182,9 +222,13 @@ Note that tests still require `Boost_preprocessor`.
  client applications via `errno` propagation.

## [0.7.0] - 2020-02-05

### Added

- Added support for `eventfd()`and `eventfd2()` system calls.

### Changed

- Replaced Margo with Mercury in the client library in order to increase
  application compatibility: the Argobots ULTs used by Margo to send and
  process RPCs clashed at times with applications using pthreads.
@@ -204,7 +248,9 @@ Note that tests still require `Boost_preprocessor`.
  can be used to further control messages. Run the client with
  `LIBGKFS_LOG=help` for more details.
- Improved dependency management in CMake.

### Fixed

- Relocate internal file descriptors to a private range to avoid interfering
  with client application file descriptors.
- Handle internal file descriptors created by `fcntl()`.
@@ -212,60 +258,89 @@ Note that tests still require `Boost_preprocessor`.
  `recvmsg()`.

## [0.6.2] - 2019-10-07

### Added
 - Paths inside kernel pseudo filesystems (`/sys`, `/proc`) are forwarded directly to the kernel and internal path resolution will be skipped. Be aware that also paths  like  `/sys/../tmp/gkfs_mountpoint/asd` will be forwarded to the kernel
 - Added new Cmake flag `CREATE_CHECK_PARENTS` to controls if the existance of the parent node needs to be checked during the creation of a child node.

- Paths inside kernel pseudo filesystems (`/sys`, `/proc`) are forwarded directly to the kernel and internal path
  resolution will be skipped. Be aware that also paths like  `/sys/../tmp/gkfs_mountpoint/asd` will be forwarded to the
  kernel
- Added new Cmake flag `CREATE_CHECK_PARENTS` to controls if the existance of the parent node needs to be checked during
  the creation of a child node.

### Changed

- Daemon logs for RPC handlers have been polished
- Updated Margo, Mercury and Libfabric dependencies

### Fixed

- mk_node RPC wasn't propagating errors correctly from daemons
- README has been improoved and got some minor fixes
- fix wrong path in log call for mk_symlink function

## [0.6.1] - 2019-09-17

### Added

- Added new Cmake flag `LOG_SYSCALLS` to enable/disable syscall logging.
- Intercept the 64 bit version of `getdents`.
- Added debian-based docker image.

### Changed

- Disable syscalls logging by default
- Update Mercury, RocksDB and Libfabric dependencies

### Fixed

- Fix read at the end of file.
- Don't create log file when using `--version`/`--help` cli flags.
- On some systems LD_PRELOAD used on /bin/bash binary was not working.
- Missing definition of `loff_t` on new version of GCC.

## [0.6.0] - 2019-07-26

### Added

- Add compile time option to disable shared memory communication `-DUSE_SHM:BOOL=OFF`

### Changed

- Deamons does not store anymore information about the others deamons.
- Improoved error handling on deamon initialization
- Decreased RPC timeout 3min -> 3sec
- Update 3rd party dependencies

### Removed

- PID file is not used anymore, we use only the new `hosts file` for out of bound communication
- Dropped CCI plugin support
- Dropped hostname-suffix cli option
- Dropped port cli option (use `--listen` instead)
- It is not needed anymore to pass hosts information to deamons, thus the `--hosts` cli have been removed

### Fixed

- Errors on get_dirents RPC are now reported back to clients
- Write errors happenig on deamons are now reported back to clients
- number overflow on lseek didn't allow to use seek on huge files

## [0.5.0] - 2019-04-29

### Changed
- Intercept I/O syscalls instead of GlibC function using [syscall intercept library](https://github.com/pmem/syscall_intercept)

- Intercept I/O syscalls instead of GlibC function
  using [syscall intercept library](https://github.com/pmem/syscall_intercept)

## [0.4.0] - 2019-04-18

First GekkoFS public release

This version provides a client library that uses GLibC I/O function interception.

## [0.3.1] - 2018-03-04

### Changed

- Read-write process improved. @Marc vef
- Improved Filemap. @Marc Vef
+5 −2
Original line number Diff line number Diff line
@@ -2,13 +2,16 @@ diff --git a/src/intercept.c b/src/intercept.c
index 41fd95d..c0cd865 100644
--- a/src/intercept.c
+++ b/src/intercept.c
@@ -689,7 +689,8 @@ intercept_routine(struct context *context)
@@ -689,7 +689,12 @@ intercept_routine(struct context *context)
 		 * the clone_child_intercept_routine instead, executing
 		 * it on the new child threads stack, then returns to libc.
 		 */
-		if (desc.nr == SYS_clone && desc.args[1] != 0)
+#ifdef SYS_clone3
+		if ((desc.nr == SYS_clone || desc.nr == SYS_clone3) &&
+				desc.args[1] != 0)
+#else
 		if (desc.nr == SYS_clone && desc.args[1] != 0)
+#endif
 			return (struct wrapper_ret){
 				.rax = context->rax, .rdx = 2 };
 		else