Commit d4583896 authored by Ramon Nou's avatar Ramon Nou
Browse files

Stats for GekkoFS 

updated CMake

Sampling output Thread

First stats test

Added Stats argument to enable output

Added DIRENTS and STATS stats, remover mkdir, rmdir, metadata and data

Added Ifdef feature, chunk distribution statistics

Added output to file

Added documentation

Testing stats

Started prometheus stats
parent e6b87245

CHANGELOG.md

+3 −2
Original line number Diff line number Diff line
@@ -7,11 +7,12 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). 


## [Unreleased]





### New



- Added Stats ([!128](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/128)) gathering in servers

- GKFS_CHUNK_STATS enables chunk usage output

- Stats output can be enabled with --output-stats <filename>

### New



- Added new experimental metadata backend:

  Parallax ([!110](https://storage.bsc.es/gitlab/hpc/gekkofs/-/merge_requests/110)).

    - Added support to use multiple metadata backends.

include/common/statistics/stats.hpp

+16 −0
Original line number Diff line number Diff line
@@ -42,6 +42,14 @@ 
#include <iostream>

#include <iomanip>

#include <fstream>





// PROMETHEUS

#include <prometheus/counter.h>

#include <prometheus/exposer.h>

#include <prometheus/registry.h>



using namespace prometheus;

/**

 * Provides storage capabilities to provide stats about GekkoFS

 * The information is per server.
@@ -160,6 +168,14 @@ private: 
    void

    dump(std::ofstream& of);





    // Prometheus structs

    std::shared_ptr<Exposer> exposer;

    Registry registry;

    Family<Counter>* family_counter;

    Counter* IOPS_create;





public:

    /**

     * @brief Starts the Stats module and initializes structures

src/common/CMakeLists.txt

+9 −0
Original line number Diff line number Diff line
@@ -48,6 +48,15 @@ target_sources(statistics 
    ${CMAKE_CURRENT_LIST_DIR}/statistics/stats.cpp

    )



    find_package(prometheus-cpp REQUIRED)

    target_link_libraries(statistics

  PRIVATE

  

    prometheus-cpp-pull

    prometheus-cpp-push

    prometheus-cpp-core

    )



if(GKFS_ENABLE_CODE_COVERAGE)

  target_code_coverage(distributor AUTO)

endif()

src/common/statistics/stats.cpp

+15 −0
Original line number Diff line number Diff line
@@ -51,6 +51,21 @@ Stats::Stats(bool output_thread, std::string stats_file) { 
        TIME_SIZE[e].push_back(pair(std::chrono::steady_clock::now(), 0.0));

    }





    // Prometheus 

    exposer = std::make_shared<Exposer>("127.0.0.1:8080");

    

    family_counter = &BuildCounter()

                        .Name("IOPS")

                        .Help("Number of IOPS")

                        .Register(registry);

    

    IOPS_create = &family_counter->Add({{"operation","Create"}});





    IOPS_create->Increment();





    output_thread_ = output_thread;



    if(output_thread_) {