Commit 1b31ed66 authored by Ramon Nou's avatar Ramon Nou
Browse files

fuse robustness

parent e14998dc

CHANGELOG.md

+4 −0
Original line number Diff line number Diff line
@@ -31,6 +31,10 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). 
  - Added FUSE support([!264](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/264))

    - fuse_client mounting a virtual path exists, avoiding usage of LD_PRELOAD.

    - Added tests for FUSE support

  - Optimized and fixed fuse ([!293](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/293))

    - New optimization options

    - Less contention with threading

README.md

+7 −2
Original line number Diff line number Diff line
@@ -283,18 +283,19 @@ The `fuse_client` requires the `LIBGKFS_HOSTS_FILE` environment variable to be s 


Common launch arguments:

- `-f`: Run in foreground.

- `-s`: Single-threaded operation (recommended).

- `-s`: Single-threaded operation. With the recent FUSE optimizations, multi-threaded operation is now the default and recommended for performance. Use `-s` only for debugging.

- `<mount_dir_path>`: The directory where GekkoFS will be mounted.



FUSE-specific options can be passed via `-o`:

- `writeback`: Enable writeback cache.

- `direct_io`: Enable direct I/O (can boost performance significantly).

- `max_threads=<N>`: Limit the maximum number of background threads FUSE can spawn to serve requests (supported in FUSE 3.12+).

- `timeout=<seconds>`: Caching timeout (default: 1.0).

- `cache={never,auto,always}`: Cache policy.



Example with options:

```bash

<install_path>/bin/fuse_client -f -s <mount_dir_path> -o writeback,direct_io,timeout=5.0

<install_path>/bin/fuse_client -f <mount_dir_path> -o writeback,direct_io,timeout=5.0,max_threads=32

```



## Logging
@@ -628,6 +629,10 @@ Client-metrics require the CMake argument `-DGKFS_ENABLE_CLIENT_METRICS=ON` (see 
- `LIBGKFS_READ_INLINE_PREFETCH` - Prefetch inline data when opening files (default: OFF).

- `LIBGKFS_USE_DIRENTS_COMPRESSION` - Enable compression for directory entries (default: OFF).

- `LIBGKFS_DIRENTS_BUFF_SIZE` - Buffer size for directory entries (default: 8MB).

- `GKFS_FUSE_ENTRY_TIMEOUT` - Caching timeout for dentry entries in the FUSE client (default: 1.0).

- `GKFS_FUSE_ATTR_TIMEOUT` - Caching timeout for file attributes in the FUSE client (default: 1.0).

- `GKFS_FUSE_NEGATIVE_TIMEOUT` - Caching timeout for negative lookups in the FUSE client (default: 1.0).

- `GKFS_FUSE_WRITEBACK` - Enable writeback cache in the FUSE client (default: OFF).



#### Caching

##### Dentry cache

include/client/fuse/fuse_client.hpp

+16 −1
Original line number Diff line number Diff line
@@ -72,6 +72,7 @@ extern "C" { 
#include <string>

#include <mutex>

#include <cstdlib>

#include <atomic>



#ifdef __FreeBSD__

#include <sys/socket.h>
@@ -111,7 +112,18 @@ struct _uintptr_to_must_hold_fuse_ino_t_dummy_struct { 


struct Inode {

    std::string path;

    uint64_t lookup_count;

    std::atomic<uint64_t> lookup_count;



    Inode() : lookup_count(0) {}

    Inode(const std::string& p, uint64_t lc) : path(p), lookup_count(lc) {}

    Inode(const Inode& other) : path(other.path), lookup_count(other.lookup_count.load()) {}

    Inode& operator=(const Inode& other) {

        if (this != &other) {

            path = other.path;

            lookup_count.store(other.lookup_count.load());

        }

        return *this;

    }

};



enum {
@@ -135,6 +147,9 @@ struct u_data { 
    int xattr;

    char* mountpoint;

    double timeout;

    double entry_timeout;

    double attr_timeout;

    double negative_timeout;

    int cache;

    int timeout_set;

};

src/client/fuse/fuse_client.cpp

+245 −143

File changed.

Preview size limit exceeded, changes collapsed.

src/client/gkfs_metadata.cpp

+13 −1
Original line number Diff line number Diff line
@@ -1165,6 +1165,19 @@ gkfs_opendir(const std::string& path) { 
        ret.second->add("..", gkfs::filemap::FileType::directory);

        for(auto& fut : dcache_futures) {

            auto res = fut.get(); // Wait for the RPC result



            if(res.first != 0) {

                ret.first = res.first;

                LOG(ERROR, "{}() RPC failed with error: {}", __func__,

                    res.first);

                continue;

            }

            if(!res.second) {

                LOG(ERROR, "{}() RPC returned null entries vector", __func__);

                ret.first = EIO;

                continue;

            }



            auto& open_dir = *res.second;

            for(auto& dentry : open_dir) {

                // type returns as unsigned char
@@ -1186,7 +1199,6 @@ gkfs_opendir(const std::string& path) { 
                                                      get<3>(dentry)});

                cnt++;

            }

            ret.first = res.first;

        }

        LOG(DEBUG, "{}() Unpacked dirents for path '{}' counted '{}' entries",

            __func__, path, cnt);