Commit ae8c58a3 authored by Ramon Nou's avatar Ramon Nou
Browse files

Merge branch... 

Merge branch 'rnou/357-missing-libc-interception-functions-may-lead-to-silently-working-calls' into 'master'

Resolve "missing libc interception functions may lead to silently working calls"

Closes #357

Closes #357

See merge request !251
parents 02ea850a 946b4619

CHANGELOG.md

+2 −0
Original line number Diff line number Diff line
@@ -11,6 +11,8 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). 
  - Tests to cover proxy and (malleability) ([!222](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/222))

  - New fd generation method ([!225](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/202))

    - Use LIBGKFS_PROTECT_FD=1 to enable the original method of assignation and protection.

    - Use LIBGKFS_RANGE_FD=1 to enable virtual fds from 10000, but without protecting and relocating fds. 

      - ([!251](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/251))

  - Lock system (server level) ([!245](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/245))

    - Use PROTECT_FILES_GENERATOR=1 and PROTECT_FILES_CONSUMER=1 to enable. Generator, creates transparent .lockgekko files that blocks the open (for some seconds) of any consumer. Multiple opens / closes for generator are managed.

  - Basic mmap support ([!247](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/245))

README.md

+1 −0
Original line number Diff line number Diff line
@@ -610,6 +610,7 @@ until the file is closed. The cache does not impact the consistency of the file 
##### Protecting FDs

When the user creates a fd, this is protected from normal fds with a recolocation. This theoretically protects the fd from being closed from outside. However a new fd assignation system has been developed and is activated by default.

- `LIBGKFS_PROTECT_FD=1` - Enable the original method of assignation and protection.

- `LIBGKFS_RANGE_FD=1`- Enables FDs from 10000, virtual, but does not protect them.



##### Lightweight File-Locking (server side)

Using two environment variables

include/client/env.hpp

+2 −0
Original line number Diff line number Diff line
@@ -73,6 +73,8 @@ static constexpr auto PROTECT_FILES_GENERATOR = 
        ADD_PREFIX("PROTECT_FILES_GENERATOR");

static constexpr auto PROTECT_FILES_CONSUMER =

        ADD_PREFIX("PROTECT_FILES_CONSUMER");

static constexpr auto RANGE_FD = ADD_PREFIX("RANGE_FD");



static constexpr auto NUM_REPL = ADD_PREFIX("NUM_REPL");

static constexpr auto PROXY_PID_FILE = ADD_PREFIX("PROXY_PID_FILE");

namespace cache {

include/client/preload_context.hpp

+8 −0
Original line number Diff line number Diff line
@@ -142,6 +142,8 @@ private: 
    bool protect_fds_{false};

    bool protect_files_generator_{false};

    bool protect_files_consumer_{false};

    bool range_fd_{false};





    std::shared_ptr<gkfs::messagepack::ClientMetrics> write_metrics_;

    std::shared_ptr<gkfs::messagepack::ClientMetrics> read_metrics_;
@@ -336,6 +338,12 @@ public: 
    void

    protect_files_consumer(bool protect);



    bool

    range_fd() const;



    void

    range_fd(bool fd);



    const std::shared_ptr<gkfs::messagepack::ClientMetrics>

    write_metrics();

src/client/open_file_map.cpp

+16 −3
Original line number Diff line number Diff line
@@ -162,7 +162,16 @@ OpenFileMap::safe_generate_fd_idx_() { 
            }

        }

    } else {

        // Some architectures do not support SYS_open

        // Return a virtual fd from 10000, but avoid doing all the FD movements

        if(CTX->range_fd()) {

            if(fd_validation_needed) {

                while(exist(fd)) {

                    fd = generate_fd_idx();

                }

            }

            return fd;

        }



        fd = syscall_no_intercept(SYS_openat, AT_FDCWD, "/dev/null", O_RDWR,

                                  S_IRUSR | S_IWUSR);

    }
@@ -185,9 +194,13 @@ OpenFileMap::remove(const int fd) { 
        return false;

    }

    files_.erase(fd);



    if(!CTX->protect_fds()) {

        if(!CTX->range_fd()) {

            // We close the dev null fd

            close(fd);

            return true;

        }

    }

    if(fd_validation_needed && files_.empty()) {

        fd_validation_needed = false;