Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • G gekkofs
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 83
    • Issues 83
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 10
    • Merge requests 10
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • hpc
  • gekkofs
  • Merge requests
  • !132

Push stats to prometheus

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Ramon Nou requested to merge rnou/stats_prometheus into master Mar 23, 2022
  • Overview 28
  • Commits 20
  • Pipelines 26
  • Changes 23

This MR pushes the stats to Prometheus, a pull model can easily be implemented.

It is a follow-up of the !128 (closed) MR :

This MR will provide the mechanism to store and check stats inside GekkoFS

There are two types of Stats: Number of operations (i.e., Create) and Operations with Size (i.e. write / read )

The stats are stored to allow the calculation of averages (and other stats) total, 1 min, 5 min and 10 minutes.

Other stats included: File-Chunk more accessed (write-read), using GKFS_CHUNK_STATS in the CMAKE. The structures can show the next info: accesses -- <file> // Chunk_id.

READ CHUNK MAP
1 -- /top/file_a // 4
2 -- /top/file_a // 3
3 -- /top/file_a // 1
/top/file_a // 2
5 -- /top/file_a // 0
WRITE CHUNK MAP
1 -- /top/file_a // 2
3 -- /top/file_a // 1
8 -- /top/file_a // 0

There is a thread for output the data each 10s to the console. The thread can be activated using a command line option, but the normal behaviour should be to allow an external RPC to gather the stats (i.e., a monitoring system).

Using --output-stats:

Stats IOPS_CREATE IOPS/s (avg, 1 min, 5 min, 10 min)                91.66 -         0 -         0 -         0 - 
Stats IOPS_WRITE IOPS/s (avg, 1 min, 5 min, 10 min)                 45.83 -         0 -         0 -         0 - 
Stats IOPS_READ IOPS/s (avg, 1 min, 5 min, 10 min)                  45.83 -         0 -         0 -         0 - 
Stats IOPS_STATS IOPS/s (avg, 1 min, 5 min, 10 min)                 320.8 -         0 -         0 -         0 - 
Stats IOPS_DIRENTS IOPS/s (avg, 1 min, 5 min, 10 min)               45.83 -         0 -         0 -         0 - 
Stats IOPS_REMOVE IOPS/s (avg, 1 min, 5 min, 10 min)                91.66 -         0 -         0 -         0 - 
Stats WRITE_SIZE MB/s (avg, 1 min, 5 min, 10 min)                  0.1353 -         0 -         0 -         0 - 
Stats READ_SIZE MB/s (avg, 1 min, 5 min, 10 min)                   0.1353 -         0 -         0 -         0 - 
Edited Mar 25, 2022 by Ramon Nou
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: rnou/stats_prometheus