Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
hpc
gekkofs
Commits
33e7ce65
Commit
33e7ce65
authored
Mar 24, 2022
by
Ramon Nou
Browse files
ifdef Prometheus, and CMAKE
parent
142115a8
Changes
12
Hide whitespace changes
Inline
Side-by-side
CHANGELOG.md
View file @
33e7ce65
...
...
@@ -11,12 +11,15 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### New
-
Added Stats (
[
!128
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/128
)
) gathering in servers
-
GKFS_CHUNK_STATS enables chunk usage output
-
Stats output can be enabled with --output-stats
<filename>
-
GKFS_CHUNK_STATS enables chunk usage output
-
Stats output can be enabled with --output-stats
<filename>
-
Added new experimental metadata backend:
Parallax (
[
!110
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/110
)
).
-
Added support to use multiple metadata backends.
-
Added
`--clean-rootdir-finish`
argument to remove rootdir/metadir at the end when the daemon finishes.
-
Added Prometheus Ouput (
[
!132
](
https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/132
)
)
-
New option to define gateway --prometheus-gateway
<gateway:port>
-
Prometheus output is optional with "GKFS_ENABLE_PROMETHEUS"
### Changed
...
...
CMakeLists.txt
View file @
33e7ce65
...
...
@@ -201,6 +201,12 @@ if (GKFS_CHUNK_STATS)
endif
()
message
(
STATUS
"[gekkofs] Gather Chunk Stats:
${
GKFS_CHUNK_STATS
}
"
)
option
(
GKFS_ENABLE_PROMETHEUS
"Enable Prometheus Push "
OFF
)
if
(
GKFS_ENABLE_PROMETHEUS
)
add_definitions
(
-DGKFS_ENABLE_PROMETHEUS
)
endif
()
message
(
STATUS
"[gekkofs] Prometheus Output:
${
GKFS_ENABLE_PROMETHEUS
}
"
)
configure_file
(
include/common/cmake_configure.hpp.in include/common/cmake_configure.hpp
)
...
...
README.md
View file @
33e7ce65
...
...
@@ -110,6 +110,7 @@ Options:
Note, parallaxdb creates a file called rocksdbx with 8GB created
in
metadir.
--parallaxsize
TEXT parallaxdb - metadata file size
in
GB
(
default 8GB
)
, used only with new files
--output-stats
TEXT Enables the output of the stats on the FILE
(
each 10s
)
for
debug
--prometheus-gateway
TEXT Defines the ip:port of the Prometheus Push gateway
--version
Print version and exit.
```
...
...
@@ -238,6 +239,9 @@ To enable it use the `-DGKFS_ENABLE_PARALLAX:BOOL=ON` option, you can also disab
Once it is enabled,
`--dbbackend`
option will be functional.
### Stats
Pushing stats to Prometheus is enabled with the
`-DGKFS_ENABLE_PROMETHEUS`
and the setup of the
`--output-stats <FILE>`
.
Without the last one, the push to the gateway is disabled.
### Acknowledgment
...
...
docs/sphinx/users/running.md
View file @
33e7ce65
...
...
@@ -80,6 +80,7 @@ Options:
Note, parallaxdb creates a file called rocksdbx with 8GB created
in
metadir.
--parallaxsize
TEXT parallaxdb - metadata file size
in
GB
(
default 8GB
)
, used only with new files
--output-stats
TEXT Outputs the stats to the file each 10s.
--prometheus-gateway
TEXT Defines the ip:port of the Prometheus Push gateway
--version
Print version and exit.
```
`
...
...
include/common/statistics/stats.hpp
View file @
33e7ce65
...
...
@@ -44,7 +44,8 @@
#include
<fstream>
// PROMETHEUS
// PROMETHEUS includes
#ifdef GKFS_PROMETHEUS
#include
<prometheus/counter.h>
#include
<prometheus/summary.h>
#include
<prometheus/exposer.h>
...
...
@@ -52,6 +53,9 @@
#include
<prometheus/gateway.h>
using
namespace
prometheus
;
#endif
/**
* Provides storage capabilities to provide stats about GekkoFS
* The information is per server.
...
...
@@ -171,16 +175,15 @@ private:
dump
(
std
::
ofstream
&
of
);
// Prometheus structs
std
::
shared_ptr
<
Exposer
>
exposer
;
///< Push
std
::
shared_ptr
<
Gateway
>
gateway
;
std
::
shared_ptr
<
Registry
>
registry
;
Family
<
Counter
>*
family_counter
;
Family
<
Summary
>*
family_summary
;
// Prometheus Push structs
#ifdef GKFS_PROMETHEUS
std
::
shared_ptr
<
Gateway
>
gateway
;
///< Prometheus Gateway
std
::
shared_ptr
<
Registry
>
registry
;
///< Prometheus Counters Registry
Family
<
Counter
>*
family_counter
;
///< Prometheus IOPS counter
Family
<
Summary
>*
family_summary
;
///< Prometheus SIZE counter
std
::
map
<
IOPS_OP
,
Counter
*>
IOPS_Prometheus
;
std
::
map
<
SIZE_OP
,
Summary
*>
SIZE_Prometheus
;
#endif
public:
/**
...
...
@@ -188,7 +191,8 @@ public:
* @param output_thread creates an aditional thread that outputs the stats
* @param filename file where to write the output
*/
Stats
(
bool
output_thread
,
std
::
string
filename
);
Stats
(
bool
output_thread
,
std
::
string
filename
,
std
::
string
prometheus_gateway
);
/**
* @brief Destroys the class, and any associated thread
...
...
@@ -196,6 +200,16 @@ public:
*/
~
Stats
();
/**
* @brief Set the up Prometheus gateway and structures
*
* @param gateway_ip
* @param gateway_port
*/
void
setup_Prometheus
(
std
::
string
gateway_ip
,
std
::
string
gateway_port
);
/**
* @brief Adds a new read access to the chunk/path specified
*
...
...
include/daemon/classes/fs_data.hpp
View file @
33e7ce65
...
...
@@ -95,6 +95,9 @@ private:
bool
output_stats_
=
false
;
std
::
string
stats_file_
;
// Prometheus
std
::
string
prometheus_gateway_
=
"127.0.0.1:9091"
;
public:
static
FsData
*
getInstance
()
{
...
...
@@ -240,6 +243,12 @@ public:
void
stats_file
(
std
::
string
stats_file
);
std
::
string
prometheus_gateway
()
const
;
void
prometheus_gateway
(
std
::
string
prometheus_gateway_
);
};
...
...
scripts/profiles/0.9.1/install/curl.install
0 → 100644
View file @
33e7ce65
################################################################################
# Copyright 2018-2022, Barcelona Supercomputing Center (BSC), Spain #
# Copyright 2015-2022, Johannes Gutenberg Universitaet Mainz, Germany #
# #
# This software was partially supported by the #
# EC H2020 funded project NEXTGenIO (Project ID: 671951, www.nextgenio.eu). #
# #
# This software was partially supported by the #
# ADA-FS project under the SPPEXA project funded by the DFG. #
# #
# This file is part of GekkoFS. #
# #
# GekkoFS is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# GekkoFS is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with GekkoFS. If not, see <https://www.gnu.org/licenses/>. #
# #
# SPDX-License-Identifier: GPL-3.0-or-later #
################################################################################
# vi: ft=bash
################################################################################
## The installation script must define both a pkg_install function and
## pkg_check function that, as their name implies, must specify how
## a dependency package should be installed and tested. ## ## The following
## variables can be used in the installation script:
## - CMAKE: a variable that expands to the cmake binary
## - SOURCE_DIR: the directory where the sources for the package were
## downloaded
## - INSTALL_DIR: the directory where the package should be installed
## - CORES: the number of cores to use when building
## - COMPILER_NAME: the name of the compiler being used (e.g. g++, clang, etc.)
## - COMPILER_FULL_VERSION: the compiler's full version (e.g. 9.3.0)
## - COMPILER_MAJOR_VERSION: the compiler's major version (e.g. 9)
## - PERFORM_TEST: whether tests for the package should be executed
################################################################################
pkg_install
()
{
ID
=
"curl"
CURR
=
"${SOURCE_DIR}/${ID}"
cd
"${CURR}"
autoreconf
-
fi
.
/
configure
--
prefix
=
"${INSTALL_DIR}"
--
without
-
ssl
make
-
j
"${CORES}"
make
install
}
pkg_check
()
{
:
}
scripts/profiles/0.9.1/install/prometheus-cpp.install
0 → 100644
View file @
33e7ce65
################################################################################
# Copyright 2018-2022, Barcelona Supercomputing Center (BSC), Spain #
# Copyright 2015-2022, Johannes Gutenberg Universitaet Mainz, Germany #
# #
# This software was partially supported by the #
# EC H2020 funded project NEXTGenIO (Project ID: 671951, www.nextgenio.eu). #
# #
# This software was partially supported by the #
# ADA-FS project under the SPPEXA project funded by the DFG. #
# #
# This file is part of GekkoFS. #
# #
# GekkoFS is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# GekkoFS is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with GekkoFS. If not, see <https://www.gnu.org/licenses/>. #
# #
# SPDX-License-Identifier: GPL-3.0-or-later #
################################################################################
# vi: ft=bash
################################################################################
## The installation script must define both a pkg_install function and
## pkg_check function that, as their name implies, must specify how
## a dependency package should be installed and tested. ## ## The following
## variables can be used in the installation script:
## - CMAKE: a variable that expands to the cmake binary
## - SOURCE_DIR: the directory where the sources for the package were
## downloaded
## - INSTALL_DIR: the directory where the package should be installed
## - CORES: the number of cores to use when building
## - COMPILER_NAME: the name of the compiler being used (e.g. g++, clang, etc.)
## - COMPILER_FULL_VERSION: the compiler's full version (e.g. 9.3.0)
## - COMPILER_MAJOR_VERSION: the compiler's major version (e.g. 9)
## - PERFORM_TEST: whether tests for the package should be executed
################################################################################
pkg_install
()
{
ID
=
"prometheus-cpp"
CURR
=
"${SOURCE_DIR}/${ID}"
prepare_build_dir
"${CURR}"
cd
"${CURR}/build"
$
{
CMAKE
}
-
DCMAKE_INSTALL_PREFIX
=
"${INSTALL_DIR}"
\
-
DBUILD_SHARED_LIBS
:
BOOL
=
ON
..
make
-
j
"${CORES}"
install
}
pkg_check
()
{
:
}
src/common/CMakeLists.txt
View file @
33e7ce65
...
...
@@ -48,16 +48,20 @@ target_sources(statistics
${
CMAKE_CURRENT_LIST_DIR
}
/statistics/stats.cpp
)
if
(
GKFS_ENABLE_PROMETHEUS
)
find_package
(
prometheus-cpp REQUIRED
)
target_link_libraries
(
statistics
PRIVATE
set
(
PROMETHEUS_LIB
prometheus-cpp-pull
prometheus-cpp-push
prometheus-cpp-core
curl
)
curl
)
endif
()
target_link_libraries
(
statistics
PRIVATE
${
PROMETHEUS_LIB
}
)
if
(
GKFS_ENABLE_CODE_COVERAGE
)
target_code_coverage
(
distributor AUTO
)
...
...
src/common/statistics/stats.cpp
View file @
33e7ce65
...
...
@@ -33,39 +33,15 @@ using namespace std;
namespace
gkfs
::
utils
{
static
std
::
string
GetHostName
()
{
char
hostname
[
1024
];
if
(
::
gethostname
(
hostname
,
sizeof
(
hostname
)))
{
return
{};
}
return
hostname
;
}
Stats
::
Stats
(
bool
output_thread
,
std
::
string
stats_file
)
{
// Init clocks
start
=
std
::
chrono
::
steady_clock
::
now
();
// To simplify the control we add an element into the different maps
// Statistaclly will be negligible... and we get a faster flow
for
(
auto
e
:
all_IOPS_OP
)
{
IOPS
[
e
]
=
0
;
TIME_IOPS
[
e
].
push_back
(
std
::
chrono
::
steady_clock
::
now
());
}
for
(
auto
e
:
all_SIZE_OP
)
{
SIZE
[
e
]
=
0
;
TIME_SIZE
[
e
].
push_back
(
pair
(
std
::
chrono
::
steady_clock
::
now
(),
0.0
));
}
// Prometheus
// exposer = std::make_shared<Exposer>("127.0.0.1:8080");
void
Stats
::
setup_Prometheus
(
std
::
string
gateway_ip
,
std
::
string
gateway_port
)
{
// Prometheus Push model. Gateway
#ifdef GKFS_ENABLE_PROMETHEUS
const
auto
labels
=
Gateway
::
GetInstanceLabel
(
GetHostName
());
gateway
=
std
::
make_shared
<
Gateway
>
(
"127.0.0.1"
,
"9091"
,
"GekkoFS"
,
labels
);
gateway
=
std
::
make_shared
<
Gateway
>
(
gateway_ip
,
gateway_port
,
"GekkoFS"
,
labels
);
registry
=
std
::
make_shared
<
Registry
>
();
family_counter
=
&
BuildCounter
()
.
Name
(
"IOPS"
)
...
...
@@ -81,6 +57,7 @@ Stats::Stats(bool output_thread, std::string stats_file) {
.
Name
(
"SIZE"
)
.
Help
(
"Size of OPs"
)
.
Register
(
*
registry
);
for
(
auto
e
:
all_SIZE_OP
)
{
SIZE_Prometheus
[
e
]
=
&
family_summary
->
Add
(
{{
"operation"
,
SIZE_OP_S
[
static_cast
<
int
>
(
e
)]}},
...
...
@@ -88,6 +65,37 @@ Stats::Stats(bool output_thread, std::string stats_file) {
}
gateway
->
RegisterCollectable
(
registry
);
#endif /// GKFS_ENABLE_PROMETHEUS
}
Stats
::
Stats
(
bool
output_thread
,
std
::
string
stats_file
,
std
::
string
prometheus_gateway
)
{
// Init clocks
start
=
std
::
chrono
::
steady_clock
::
now
();
// To simplify the control we add an element into the different maps
// Statistaclly will be negligible... and we get a faster flow
for
(
auto
e
:
all_IOPS_OP
)
{
IOPS
[
e
]
=
0
;
TIME_IOPS
[
e
].
push_back
(
std
::
chrono
::
steady_clock
::
now
());
}
for
(
auto
e
:
all_SIZE_OP
)
{
SIZE
[
e
]
=
0
;
TIME_SIZE
[
e
].
push_back
(
pair
(
std
::
chrono
::
steady_clock
::
now
(),
0.0
));
}
#ifdef GKFS_ENABLE_PROMETHEUS
try
{
auto
pos_separator
=
prometheus_gateway
.
find
(
":"
);
setup_Prometheus
(
prometheus_gateway
.
substr
(
0
,
pos_separator
),
prometheus_gateway
.
substr
(
pos_separator
+
1
));
}
catch
(
const
std
::
exception
&
e
)
{
setup_Prometheus
(
"127.0.0.1"
,
"9091"
);
}
#endif
output_thread_
=
output_thread
;
...
...
@@ -164,7 +172,9 @@ Stats::add_value_iops(enum IOPS_OP iop) {
TIME_IOPS
[
iop
].
pop_front
();
TIME_IOPS
[
iop
].
push_back
(
std
::
chrono
::
steady_clock
::
now
());
#ifdef GKFS_ENABLE_PROMETHEUS
IOPS_Prometheus
[
iop
]
->
Increment
();
#endif
}
void
...
...
@@ -177,8 +187,9 @@ Stats::add_value_size(enum SIZE_OP iop, unsigned long long value) {
TIME_SIZE
[
iop
].
pop_front
();
TIME_SIZE
[
iop
].
push_back
(
pair
(
std
::
chrono
::
steady_clock
::
now
(),
value
));
#ifdef GKFS_ENABLE_PROMETHEUS
SIZE_Prometheus
[
iop
]
->
Observe
(
value
);
#endif
if
(
iop
==
SIZE_OP
::
READ_SIZE
)
add_value_iops
(
IOPS_OP
::
IOPS_READ
);
else
if
(
iop
==
SIZE_OP
::
WRITE_SIZE
)
...
...
@@ -310,10 +321,11 @@ Stats::output(std::chrono::seconds d, std::string file_output) {
if
(
times
%
4
==
0
)
output_map
(
of
);
#endif
#ifdef GKFS_ENABLE_PROMETHEUS
// Prometheus Output
auto
res
=
gateway
->
Push
();
std
::
cout
<<
"result "
<<
res
<<
std
::
endl
;
#endif
while
(
running
and
a
<
d
)
{
a
+=
1s
;
std
::
this_thread
::
sleep_for
(
1s
);
...
...
src/daemon/classes/fs_data.cpp
View file @
33e7ce65
...
...
@@ -257,4 +257,14 @@ FsData::stats_file(std::string stats_file) {
FsData
::
stats_file_
=
stats_file
;
}
std
::
string
FsData
::
prometheus_gateway
()
const
{
return
prometheus_gateway_
;
}
void
FsData
::
prometheus_gateway
(
std
::
string
prometheus_gateway
)
{
FsData
::
prometheus_gateway_
=
prometheus_gateway
;
}
}
// namespace gkfs::daemon
src/daemon/daemon.cpp
View file @
33e7ce65
...
...
@@ -83,6 +83,7 @@ struct cli_options {
string
dbbackend
;
string
parallax_size
;
string
stats_file
;
string
prometheus_gateway
;
};
/**
...
...
@@ -295,7 +296,8 @@ init_environment() {
// Initialize Stats
GKFS_DATA
->
stats
(
std
::
make_shared
<
gkfs
::
utils
::
Stats
>
(
GKFS_DATA
->
output_stats
(),
GKFS_DATA
->
stats_file
()));
GKFS_DATA
->
output_stats
(),
GKFS_DATA
->
stats_file
(),
GKFS_DATA
->
prometheus_gateway
()));
// Initialize data backend
auto
chunk_storage_path
=
fmt
::
format
(
"{}/{}"
,
GKFS_DATA
->
rootdir
(),
...
...
@@ -658,6 +660,15 @@ parse_input(const cli_options& opts, const CLI::App& desc) {
GKFS_DATA
->
spdlogger
()
->
debug
(
"{}() Stats Enabled: '{}'"
,
__func__
,
stats_file
);
}
#ifdef GKFS_ENABLE_PROMETHEUS
if
(
desc
.
count
(
"--prometheus_gateway"
))
{
auto
gateway
=
opts
.
prometheus_gateway
;
GKFS_DATA
->
prometheus_gateway
(
gateway
);
GKFS_DATA
->
spdlogger
()
->
debug
(
"{}() Prometheus Gateway: '{}'"
,
__func__
,
gateway
);
}
#endif
}
/**
...
...
@@ -729,6 +740,10 @@ main(int argc, const char* argv[]) {
"--output-stats"
,
opts
.
stats_file
,
"Creates a thread that outputs the server stats each 10s, to the file specified"
);
desc
.
add_option
(
"--prometheus-gateway"
,
opts
.
prometheus_gateway
,
"Defines the prometheus gateway, default is 127.0.0.1:9091, experimental enable at compilation"
);
desc
.
add_flag
(
"--version"
,
"Print version and exit."
);
// clang-format on
try
{
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment