Skip to content

Push stats to prometheus

Ramon Nou requested to merge rnou/stats_prometheus into master

This MR pushes the stats to Prometheus, a pull model can easily be implemented.

It is a follow-up of the !128 (closed) MR :

This MR will provide the mechanism to store and check stats inside GekkoFS

There are two types of Stats: Number of operations (i.e., Create) and Operations with Size (i.e. write / read )

The stats are stored to allow the calculation of averages (and other stats) total, 1 min, 5 min and 10 minutes.

Other stats included: File-Chunk more accessed (write-read), using GKFS_CHUNK_STATS in the CMAKE. The structures can show the next info: accesses -- <file> // Chunk_id.

READ CHUNK MAP
1 -- /top/file_a // 4
2 -- /top/file_a // 3
3 -- /top/file_a // 1
/top/file_a // 2
5 -- /top/file_a // 0
WRITE CHUNK MAP
1 -- /top/file_a // 2
3 -- /top/file_a // 1
8 -- /top/file_a // 0

There is a thread for output the data each 10s to the console. The thread can be activated using a command line option, but the normal behaviour should be to allow an external RPC to gather the stats (i.e., a monitoring system).

Using --output-stats:

Stats IOPS_CREATE IOPS/s (avg, 1 min, 5 min, 10 min)                91.66 -         0 -         0 -         0 - 
Stats IOPS_WRITE IOPS/s (avg, 1 min, 5 min, 10 min)                 45.83 -         0 -         0 -         0 - 
Stats IOPS_READ IOPS/s (avg, 1 min, 5 min, 10 min)                  45.83 -         0 -         0 -         0 - 
Stats IOPS_STATS IOPS/s (avg, 1 min, 5 min, 10 min)                 320.8 -         0 -         0 -         0 - 
Stats IOPS_DIRENTS IOPS/s (avg, 1 min, 5 min, 10 min)               45.83 -         0 -         0 -         0 - 
Stats IOPS_REMOVE IOPS/s (avg, 1 min, 5 min, 10 min)                91.66 -         0 -         0 -         0 - 
Stats WRITE_SIZE MB/s (avg, 1 min, 5 min, 10 min)                  0.1353 -         0 -         0 -         0 - 
Stats READ_SIZE MB/s (avg, 1 min, 5 min, 10 min)                   0.1353 -         0 -         0 -         0 - 
Edited by Ramon Nou

Merge request reports