Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
hpc
gekkofs
Commits
2a28c444
Commit
2a28c444
authored
Mar 17, 2022
by
Marc Vef
Browse files
Changelog formatting, changed kreon appearances to parallax
parent
40d499ee
Changes
7
Hide whitespace changes
Inline
Side-by-side
CHANGELOG.md
View file @
2a28c444
# Changelog
All notable changes to GekkoFS project will be documented in this file.
The format is based on
[
Keep a Changelog
](
https://keepachangelog.com/en/1.0.0/
)
,
and this project adheres
to
[
Semantic Versioning
](
https://semver.org/spec/v2.0.0.html
)
.
The format is based on
[
Keep a Changelog
](
https://keepachangelog.com/en/1.0.0/
)
,
and this project adheres
to
[
Semantic Versioning
](
https://semver.org/spec/v2.0.0.html
)
.
## [Unreleased]
-
Parallax experimental integration
Support for different databases backend
New Docker Images and scripts (0.9.1)
(
[
!110
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/110
)
).
-
Added
`--clean-rootdir-finish`
argument to remove rootdir/metadir at the end.
### New
-
Added new experimental metadata backend:
Parallax (
[
!110
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/110
)
).
-
Added support to use multiple metadata backends.
-
Added
`--clean-rootdir-finish`
argument to remove rootdir/metadir at the end when the daemon finishes.
### Changed
-
`-c`
argument has been moved to
`--clean-rootdir-finish`
and is now used to clean rootdir/metadir on daemon
shutdown (
[
!110
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/110
)
).
### Removed
### Fixed
## [0.9.0] - 2022-02-22
### New
-
GekkoFS now uses C++17 (!74).
-
Added a new
`dirents_extended`
function which can improve
`find`
operations.
A corresponding example how to use this
function can be found at
`examples/gfind/gfind.cpp`
with a non-mpi version at
`examples/gfind/sfind.cpp`
(
[
!73
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/73
)
).
-
Code coverage reports for the source code are now generated and tracked
(
[
!77
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/77
)
).
-
Added a new
`dirents_extended`
function which can improve
`find`
operations.
A corresponding example how to use this
function can be found at
`examples/gfind/gfind.cpp`
with a non-mpi version at
`examples/gfind/sfind.cpp`
(
[
!73
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/73
)
).
-
Code coverage reports for the source code are now generated and tracked
(
[
!77
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/77
)
).
-
Considerable overhaul and new features of the GekkoFS testing facilities
(
[
!120
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/120
)
,
[
!121
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/121
)
).
...
...
@@ -82,8 +96,6 @@ Note that tests still require `Boost_preprocessor`.
(
[
!116
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/116
)
).
-
Fixed an issue where
`LOG_OUTPUT_TRUNC`
did not work as expected (
[
!118
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/118
)
).
-
Added new metadata backend, kreon.
## [0.8.0] - 2020-09-15
### New
-
Both client library and daemon have been extended to support the ofi+verbs
...
...
README.md
View file @
2a28c444
...
...
@@ -110,7 +110,7 @@ Options:
--dbbackend
'rocksdb'
(
default
)
or
'parallaxdb'
can be specified as
metadata backend,
in
that
case
a file
in
'metadir'
named
rocksdbx is created. Parallaxdb support is experimental.
--
kreon
size
'parallaxdb'
specific, size of the metadata file
in
GB. Minimal is 8 GB
--
parallax_
size
'parallaxdb'
specific, size of the metadata file
in
GB. Minimal is 8 GB
(
default 8, 8 GB
)
--version
Print version and exit.
```
...
...
docs/sphinx/users/running.md
View file @
2a28c444
...
...
@@ -78,7 +78,7 @@ Further options are available
--dbbackend
'rocksdb'
(
default
)
or
'parallaxdb'
can be specified as
metadata backend,
in
that
case
a file
in
'metadir'
named
rocksdbx is created. Parallaxdb support is experimental.
--
kreon
size
'parallaxdb'
specific, size of the metadata file
in
GB. Minimal is 8 GB
--
parallax_
size
'parallaxdb'
specific, size of the metadata file
in
GB. Minimal is 8 GB
(
default 8, 8 GB
)
--version
Print version and exit.
```
`
...
...
include/daemon/classes/fs_data.hpp
View file @
2a28c444
...
...
@@ -71,10 +71,9 @@ private:
// Database
std
::
shared_ptr
<
gkfs
::
metadata
::
MetadataDB
>
mdb_
;
std
::
string
dbbackend_
;
bool
keep_rootdir_
=
true
;
// Kreon
unsigned
long
long
size_md_
=
8589934592ull
;
// Parallax
unsigned
long
long
parallax_size_md_
=
8589934592ull
;
// Storage backend
std
::
shared_ptr
<
gkfs
::
data
::
ChunkStorage
>
storage_
;
...
...
@@ -206,10 +205,10 @@ public:
blocks_state
(
bool
blocks_state
);
unsigned
long
long
kreon
_size_md
()
const
;
parallax
_size_md
()
const
;
void
kreon
_size_md
(
unsigned
int
size_md
);
parallax
_size_md
(
unsigned
int
size_md
);
};
}
// namespace daemon
...
...
src/daemon/backend/metadata/parallax_backend.cpp
View file @
2a28c444
...
...
@@ -90,7 +90,7 @@ ParallaxBackend::ParallaxBackend(const std::string& path)
}
if
(
size
==
0
)
{
size
=
GKFS_DATA
->
kreon
_size_md
();
size
=
GKFS_DATA
->
parallax
_size_md
();
lseek
(
fd
,
size
-
1
,
SEEK_SET
);
std
::
string
tmp
=
"x"
;
...
...
@@ -334,8 +334,6 @@ ParallaxBackend::decrease_size_impl(const std::string& key, size_t size) {
std
::
vector
<
std
::
pair
<
std
::
string
,
bool
>>
ParallaxBackend
::
get_dirents_impl
(
const
std
::
string
&
dir
)
const
{
auto
root_path
=
dir
;
// lock_guard<recursive_mutex> lock_guard(kreon_mutex_);
struct
par_key
K
;
str2par
(
root_path
,
K
);
...
...
src/daemon/classes/fs_data.cpp
View file @
2a28c444
...
...
@@ -211,14 +211,14 @@ FsData::blocks_state(bool blocks_state) {
}
unsigned
long
long
FsData
::
kreon
_size_md
()
const
{
return
size_md_
;
FsData
::
parallax
_size_md
()
const
{
return
parallax_
size_md_
;
}
void
FsData
::
kreon
_size_md
(
unsigned
int
size_md
)
{
FsData
::
size_md_
=
(
unsigned
long
long
)
size_md
*
1024ull
*
1024ull
*
1024ull
;
FsData
::
parallax
_size_md
(
unsigned
int
size_md
)
{
FsData
::
parallax_
size_md_
=
static_cast
<
unsigned
long
long
>
(
size_md
*
1024ull
*
1024ull
*
1024ull
)
;
}
}
// namespace gkfs::daemon
src/daemon/daemon.cpp
View file @
2a28c444
...
...
@@ -80,7 +80,7 @@ struct cli_options {
string
hosts_file
;
string
rpc_protocol
;
string
dbbackend
;
string
kreon
size
;
string
parallax_
size
;
};
/**
...
...
@@ -641,8 +641,8 @@ parse_input(const cli_options& opts, const CLI::App& desc) {
}
else
GKFS_DATA
->
dbbackend
(
gkfs
::
metadata
::
rocksdb_backend
);
if
(
desc
.
count
(
"--
kreon
size"
))
{
// Size in GB
GKFS_DATA
->
kreon
_size_md
(
stoi
(
opts
.
kreon
size
));
if
(
desc
.
count
(
"--
parallax
size"
))
{
// Size in GB
GKFS_DATA
->
parallax
_size_md
(
stoi
(
opts
.
parallax_
size
));
}
}
...
...
@@ -708,7 +708,7 @@ main(int argc, const char* argv[]) {
"Metadata database backend to use. Available: {rocksdb, parallaxdb}'
\n
"
"RocksDB is default if not set. Parallax support is experimental.
\n
"
"Note, parallaxdb creates a file called rocksdbx with 8GB created in metadir."
);
desc
.
add_option
(
"--
kreon
size"
,
opts
.
kreon
size
,
desc
.
add_option
(
"--
parallax
size"
,
opts
.
parallax_
size
,
"parallaxdb - metadata file size in GB (default 8GB), "
"used only with new files"
);
desc
.
add_flag
(
"--version"
,
"Print version and exit."
);
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment