Class ChunkStorage

Class Documentation

class gkfs::data::ChunkStorage

ChunkStorage class handles all interaction with node-local storage system and is run as a single instance within the GekkoFS daemon.

Public Functions

ChunkStorage(std::string &path, size_t chunksize)

Initializes the ChunkStorage object on daemon launch.

Parameters
  • path – Root directory where all data is placed on the local FS.

  • chunksize – Used chunksize in this GekkoFS instance.

Throws

ChunkStorageException – on launch failure

void destroy_chunk_space(const std::string &file_path) const

Removes chunk directory with all its files which is a recursive remove operation on the chunk directory.

Parameters

file_path – Chunk file path, e.g., /foo/bar

Throws

ChunkStorageException

ssize_t write_chunk(const std::string &file_path, gkfs::rpc::chnk_id_t chunk_id, const char *buf, size_t size, off64_t offset) const

Writes a single chunk file and is usually called by an Argobots tasklet.

Parameters
  • file_path – Chunk file path, e.g., /foo/bar

  • chunk_id – Number of chunk id

  • buf – Buffer to write to chunk

  • size – Amount of bytes to write to the chunk file

  • offset – Offset where to write to the chunk file

Throws

ChunkStorageException – with its error code

Returns

The amount of bytes written

ssize_t read_chunk(const std::string &file_path, gkfs::rpc::chnk_id_t chunk_id, char *buf, size_t size, off64_t offset) const

Reads a single chunk file and is usually called by an Argobots tasklet.

Parameters
  • file_path – Chunk file path, e.g., /foo/bar

  • chunk_id – Number of chunk id

  • buf – Buffer to read to from chunk

  • size – Amount of bytes to read to the chunk file

  • offset – Offset where to read from the chunk file

Throws

ChunkStorageException – with its error code

Returns

The amount of bytes read

void trim_chunk_space(const std::string &file_path, gkfs::rpc::chnk_id_t chunk_start)

Delete all chunks starting with chunk a chunk id.

Parameters
  • file_path – Chunk file path, e.g., /foo/bar

  • chunk_start – Number of chunk id

Throws

ChunkStorageException – with its error code

void truncate_chunk_file(const std::string &file_path, gkfs::rpc::chnk_id_t chunk_id, off_t length)

Truncates a single chunk file to a given byte length.

Parameters
  • file_path – Chunk file path, e.g., /foo/bar

  • chunk_id – Number of chunk id

  • length – Length of bytes to truncate the chunk to

Throws

ChunkStorageException

ChunkStat chunk_stat() const

Calls statfs on the chunk directory to get statistic on its used storage space.

Throws

ChunkStorageException

Returns

ChunkStat struct

Private Functions

inline std::string absolute(const std::string &internal_path) const

Converts an internal gkfs path under the root dir to the absolute path of the system.

Parameters

internal_path – E.g., /foo/bar

Returns

Absolute path, e.g., /tmp/rootdir/<pid>/data/chunks/foo:bar

void init_chunk_space(const std::string &file_path) const

Initializes the chunk space for a GekkoFS file, creating its directory on the local file system.

Parameters

file_path – Chunk file path, e.g., /foo/bar

Private Members

std::shared_ptr<spdlog::logger> log_

Class logger.

std::string root_path_

Path to GekkoFS root directory.

size_t chunksize_

File system chunksize. TODO Why does that exist?

Private Static Functions

static inline std::string get_chunks_dir(const std::string &file_path)

Returns the chunk dir directory for a given path which is expected to be absolute.

Parameters

file_path

Returns

Chunk dir path

static inline std::string get_chunk_path(const std::string &file_path, gkfs::rpc::chnk_id_t chunk_id)

Returns the backend chunk file path for a given internal path.

Parameters
  • file_path – Internal file path, e.g., /foo/bar

  • chunk_id – Number of chunk id

Returns

Chunk file path, e.g., /foo/bar /tmp/rootdir/<pid>>/data/chunks/foo:bar/0