Class Stats

Class Documentation

class gkfs::utils::Stats

Number of operations (Create, write/ read, remove, mkdir…) Size of database (metadata keys, should be not needed, any) Size of data (+write - delete) Server Bandwidth (write / read operations)

mean, (lifetime of the server) 1 minute mean 5 minute mean 10 minute mean

To provide the stats that we need, we need to store the info and the timestamp to calculate it A vector should work, with a maximum of elements,

Public Types

enum class IopsOp

Values:

enumerator iops_create
enumerator iops_write
enumerator iops_read
enumerator iops_stats
enumerator iops_dirent
enumerator iops_remove
enum class SizeOp

Values:

enumerator write_size
enumerator read_size

Public Functions

Stats(bool enable_chunkstats, bool enable_prometheus, const std::string &filename, const std::string &prometheus_gateway)

Starts the Stats module and initializes structures.

Parameters

  • enable_chunkstats – Enables or disables the chunk stats

  • enable_prometheus – Enables or disables the prometheus output

  • filename – file where to write the output

  • prometheus_gateway – ip:port to expose the metrics

~Stats()

Destroys the class, and any associated thread.

void setup_Prometheus(const std::string &gateway_ip, const std::string &gateway_port)

Set the up Prometheus gateway and structures.

Parameters

  • gateway_ip – ip of the prometheus gateway

  • gateway_port – port of the prometheus gateway

void add_read(const std::string &path, unsigned long long chunk)

Adds a new read access to the chunk/path specified.

Parameters

  • path – path of the chunk

  • chunk – chunk number

void add_write(const std::string &path, unsigned long long chunk)

Adds a new write access to the chunk/path specified.

Parameters

  • path – path of the chunk

  • chunk – chunk number

void add_value_iops(enum IopsOp)

Add a new value for a IOPS, that does not involve any size No value needed as they are simple (1 create, 1 read…) Size operations internally call this operation (read,write)

Parameters

IopsOp – Which operation to add

void add_value_size(enum SizeOp, unsigned long long value)

Store a new stat point, with a size value. If it involves a IO operations it will call the corresponding operation.

Parameters

  • SizeOp – Which operation we refer

  • value – to store (SizeOp)

double get_mean(enum IopsOp)

Get the total mean value of the asked stat This can be provided inmediately without cost.

Parameters

IopsOp – Which operation to get

Returns

mean value

double get_mean(enum SizeOp)

Get the total mean value of the asked stat This can be provided inmediately without cost.

Parameters

SizeOp – Which operation to get

Returns

mean value

Returns

mean value

std::vector<double> get_four_means(enum SizeOp)

Get all the means (total, 1,5 and 10 minutes) for a SIZE_OP Returns precalculated values if we just calculated them 1 minute ago.

Parameters

SizeOp – Which operation to get

Returns

std::vector< double > with 4 means

std::vector<double> get_four_means(enum IopsOp)

Get all the means (total, 1,5 and 10 minutes) for a IOPS_OP Returns precalculated values if we just calculated them 1 minute ago.

Parameters

IopsOp – Which operation to get

Returns

std::vector< double > with 4 means

Private Functions

void output(std::chrono::seconds d, std::string file_output)

Sends all the stats to the screen Debug Function.

Parameters

  • d – is the time between output

  • file_output – is the output file

void output_map(std::ofstream &output)

Called by output to generate CHUNK map.

Parameters

output – is the output stream

void dump(std::ofstream &of)

Dumps all the means from the stats.

Parameters

of – Output stream

Private Members

const std::vector<std::string> IopsOp_s = {"IOPS_CREATE", "IOPS_WRITE", "IOPS_READ", "IOPS_STATS", "IOPS_DIRENTS", "IOPS_REMOVE"}

Stats Labels.

const std::vector<std::string> SizeOp_s = {"WRITE_SIZE", "READ_SIZE"}

Stats Labels.

std::chrono::time_point<std::chrono::steady_clock> start

When we started the server.

std::map<IopsOp, std::atomic<unsigned long>> iops_mean

Stores total value for global mean.

std::map<SizeOp, std::atomic<unsigned long>> size_mean

Stores total value for global mean.

std::mutex time_iops_mutex
std::mutex size_iops_mutex
std::map<IopsOp, std::deque<std::chrono::time_point<std::chrono::steady_clock>>> time_iops

Stores timestamp when an operation comes removes if first operation if > 10 minutes Different means will be stored and cached 1 minuted

std::map<SizeOp, std::deque<std::pair<std::chrono::time_point<std::chrono::steady_clock>, unsigned long long>>> time_size

For size operations we need to store the timestamp and the size

std::thread t_output

Thread that outputs stats info.

bool output_thread_

Enables or disables the output thread.

bool enable_prometheus_

Enables or disables the prometheus output.

bool enable_chunkstats_

Enables or disables the chunk stats output.

bool running = true

Controls the destruction of the class/stops the thread.

std::map<std::pair<std::string, unsigned long long>, std::atomic<unsigned int>> chunk_reads

Stores the number of times a chunk/file is read.

std::map<std::pair<std::string, unsigned long long>, std::atomic<unsigned int>> chunk_writes

Stores the number of times a chunk/file is write.

Private Static Attributes

static constexpr const std::initializer_list<Stats::IopsOp> all_IopsOp = {IopsOp::iops_create, IopsOp::iops_write, IopsOp::iops_read, IopsOp::iops_stats, IopsOp::iops_dirent, IopsOp::iops_remove}

Enum IOPS iterator.

static constexpr const std::initializer_list<Stats::SizeOp> all_SizeOp = {SizeOp::write_size, SizeOp::read_size}

Enum SIZE iterator.