Commit 66398571 authored by sevenuz's avatar sevenuz
Browse files

benchmark hash functions in examples directory, build option and documentation

parent 1bd4bb13

CMake/gkfs-options.cmake

+9 −1
Original line number Diff line number Diff line
@@ -331,6 +331,14 @@ gkfs_define_variable( 
    "File Path for guided distributor"

)



gkfs_define_option(

		GKFS_USE_RAPIDHASH_DISTRIBUTION

    HELP_TEXT "Use rapidhash data distributor"

    DEFAULT_VALUE OFF

    DESCRIPTION "Use the rapidhash function as wide striping distributer instead of standard hash"

		EXTRA_INFO "Rapidhash performs best in SMHasher benchmark and ours, which is in the examples"

)



gkfs_define_option(

    GKFS_USE_GUIDED_DISTRIBUTION

    HELP_TEXT "Use guided data distributor"

CMakePresets.json

+1 −0
Original line number Diff line number Diff line
@@ -70,6 +70,7 @@ 
      "binaryDir": "${sourceDir}/gkfs/build",

      "cacheVariables": {

        "CMAKE_INSTALL_PREFIX": "${sourceDir}/gkfs/install",

        "GKFS_USE_RAPIDHASH_DISTRIBUTION": true,

        "GKFS_USE_GUIDED_DISTRIBUTION": true,

        "GKFS_ENABLE_PARALLAX": false,

        "GKFS_BUILD_TESTS": true,

README.md

+6 −0
Original line number Diff line number Diff line
@@ -470,6 +470,11 @@ The data distribution can be selected at compilation time, we have 2 distributor 


Chunks are distributed randomly to the different GekkoFS servers.



#### Rapidhash



Chunks are distributed randomly to the different GekkoFS servers

using [rapidhash](https://github.com/Nicoshev/rapidhash) instead of the standard libary hash function.



### Guided Distributor



The guided distributor allows defining a specific distribution of data on a per directory or file basis.
@@ -526,6 +531,7 @@ Once it is enabled, `--dbbackend` option will be functional. 
- `GKFS_RENAME_SUPPORT` - Enable support for rename (default: OFF)

- `GKFS_FOLLOW_EXTERNAL_SYMLINKS` - Enable support for following external links into the GekkoFS namespace (default: OFF)

- `GKFS_USE_LEGACY_PATH_RESOLVE` - Use the legacy implementation of the resolve function, deprecated (default: OFF)

- `GKFS_USE_RAPIDHASH_DISTRIBUTION` - Use rapidhash function instead of standard library hash data distributor (default: OFF)

- `GKFS_USE_GUIDED_DISTRIBUTION` - Use guided data distributor (default: OFF)

- `GKFS_USE_GUIDED_DISTRIBUTION_PATH` - File Path for guided distributor (default: /tmp/guided.txt)

docs/sphinx/users/running.md

+6 −1
Original line number Diff line number Diff line
@@ -235,6 +235,11 @@ The data distribution can be selected at compilation time, we have 2 distributor 


Chunks are distributed randomly to the different GekkoFS servers.



#### Rapidhash



Chunks are distributed randomly to the different GekkoFS servers

using [rapidhash](https://github.com/Nicoshev/rapidhash) instead of the standard libary hash function.



#### Guided Distributor



The guided distributor allows defining a specific distribution of data on a per directory or file basis.

examples/CMakeLists.txt

+2 −1
Original line number Diff line number Diff line
@@ -26,5 +26,6 @@ 
# SPDX-License-Identifier: GPL-3.0-or-later                                    #

################################################################################



add_subdirectory(distributors)

add_subdirectory(gfind)

add_subdirectory(user_library)