Commit 02ea850a authored by Ramon Nou's avatar Ramon Nou
Browse files

changelog and readme for libc interception

parent 8675113b

CHANGELOG.md

+1 −0
Original line number Diff line number Diff line
@@ -14,6 +14,7 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). 
  - Lock system (server level) ([!245](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/245))

    - Use PROTECT_FILES_GENERATOR=1 and PROTECT_FILES_CONSUMER=1 to enable. Generator, creates transparent .lockgekko files that blocks the open (for some seconds) of any consumer. Multiple opens / closes for generator are managed.

  - Basic mmap support ([!247](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/245))

  - LIBC interception support, with extra tests ([!202](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/202))

    

### Changed 

  - Tests check ret for -1 instead of 10000 fd ([!320](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/320))

README.md

+22 −0
Original line number Diff line number Diff line
@@ -22,6 +22,7 @@ to I/O, which reduces interferences and improves performance. 
    - [GekkoFS daemon orchestration via the gkfs script (recommended)](#gekkofs-daemon-orchestration-via-the-gkfs-script-recommended)

  - [The GekkoFS client library](#the-gekkofs-client-library)

    - [Interposition library via system call interception](#interposition-library-via-system-call-interception)

    - [Interposition library via libc call interception](#interposition-library-via-libc-call-interception)

    - [User library via linking against the application](#user-library-via-linking-against-the-application)

  - [Logging](#logging)

- [Advanced and experimental features](#advanced-and-experimental-features)
@@ -239,6 +240,24 @@ to be empty. 


For MPI application, the `LD_PRELOAD` variable can be passed with the `-x` argument for `mpirun/mpiexec`.





### Interposition library via libc call interception



tl;dr example:



```bash

export LIBGKFS_ HOSTS_FILE=<hostfile_path>

LD_PRELOAD=<install_path>/lib64/libgkfs_libc_intercept.so cp ~/some_input_data <pseudo_gkfs_mount_dir_path>/some_input_data

LD_PRELOAD=<install_path>/lib64/libgkfs_libc_intercept.so md5sum ~/some_input_data <pseudo_gkfs_mount_dir_path>/some_input_data

```



Works as the syscall intercept, but it is not hardware dependent (as does not use syscall interception).



This interposition library, may work where syscall intercepts does not work. However, often, some libc calls are not 

correctly intercepted, causing failures.



In order to debug the missing call, ltrace, LD_DEBUG=symbols,libs or other techniques may be needed.



### User library via linking against the application



GekkoFS offers a user library that can be linked against the application, which is built by default:
@@ -526,11 +545,14 @@ Once it is enabled, `--dbbackend` option will be functional. 
- `GKFS_CREATE_CHECK_PARENTS` - Enable checking parent directory for existence before creating children (default: ON)

- `GKFS_MAX_INTERNAL_FDS` - Number of file descriptors reserved for internal use (default: 256)

- `GKFS_MAX_OPEN_FDS` - Maximum number of open file descriptors supported (default: 1024)

- `GKFS_SYMLINK_SUPPORT` - Enable support for rename (default: OFF)

- `GKFS_RENAME_SUPPORT` - Enable support for rename (default: OFF)

- `GKFS_FOLLOW_EXTERNAL_SYMLINKS` - Enable support for following external links into the GekkoFS namespace (default: OFF)

- `GKFS_USE_LEGACY_PATH_RESOLVE` - Use the legacy implementation of the resolve function, deprecated (default: OFF)

- `GKFS_USE_GUIDED_DISTRIBUTION` - Use guided data distributor (default: OFF)

- `GKFS_USE_GUIDED_DISTRIBUTION_PATH` - File Path for guided distributor (default: /tmp/guided.txt)

- `GKFS_BUILD_USER_LIB` - Build the user library (default: ON)

- `GKFS_BUILD_LIBC_INTERCEPTION` - Builds the libc interception .so (default: ON)



#### Logging

- `GKFS_ENABLE_CLIENT_LOG` - Enable logging messages in clients (default: ON)