Now that we use only relative path on daemon side,
also the initial creation of the root directory should be
use a relative path.

Acknowledge the main thread that the IO operation had terminate before
to actually close the file.

This is a "dirty" fix that waits to be refactored.

WARNING: Error on close will not be reported to the client that issued the IO
operation.

Data operation has been grouped under a ChunkStorage class
that will manage all the data backend operations.

The instance of this object is stored into the ADAFS_DATA sigleton.

The metadataDB class was using integer to return errors.
New exceptions classes has been introduced in order to be used as error
propagation method.

In this commit the MetadataDB::get function has been modified in order
to make use of exceptions.

During daemon setup, multiple spdloggers are configured.

Using spdlog::get("<LOGNAME>") is possible to retrieve those and use
them in other part of the code.

Paths are now made relative to the mountdir at client side.

The new relativize_path fucntion in the PreloadContext is used to check
if a path is relative to the pseudo-mount directory and to make it
relative

Metadata::serialize() was appending result instead of ovewriting it.

The new approach is to make serialize function return a new constructed
string instead of accepting a string reference to fill.

The rpc stat handler was the only one calling directly a db function,
bypassing the intermediate adafs_ops interface.

A new function on the adafs_ops interface has been added to be used by
the rpc stat handler.

Setting the properly the JMALLOC_LIBRARIES variable into the find module
made possible to avoid the conditonal inclusion of the library

Tommaso Tocci authored Apr 19, 2018 and Marc Vef committed Apr 27, 2018

The creation was performed through a simple rocksDB PUT operation.

Multiple creation attempts on the same file would override the
previously inserted metadata, in particular the file size.

The creation is now handled with a new MergeOperand.
Now a creation attampt on an already existing file will be converted in
a NOP.

Tommaso Tocci authored Apr 19, 2018 and Marc Vef committed Apr 27, 2018

The new MergeOperand base class can be subclassed in order to implement
new merge-operands with their specific parameters and semantics.

Tommaso Tocci authored Apr 19, 2018 and Marc Vef committed Apr 27, 2018

The UpdateSizeOperand has been renamed IncreaseSizeOperand.

The operand was using three parameters size,offset and append_flag.

Since we only need the (size + offset) sum in order to apply the
operand, the offset field has been removed.

Tommaso Tocci authored Apr 17, 2018 and Marc Vef committed Apr 27, 2018

In order to make the file-size-update operation atomic it is necessary
to make the following action atomic:

 - READ old size value from rocksDB
 - COMPUTE new size based on the write operation
 - WRITE back the new value rocksDB

We use rocksDB merge operation [1] in order to implement the atomic update.

[1]: https://github.com/facebook/rocksdb/wiki/Merge-Operator

Tommaso Tocci authored Apr 17, 2018 and Marc Vef committed Apr 27, 2018

with the current storage back-end, chunks of a file are stored in
several files in a folder on a real kernel-space filesystem.

Removing a file means that we need to remove the entire directory and
all its sub-files (rm -r).
This operation is not atomic and it can raise errors if performed
concurrently..

This commit handles properly this kind of errors.

Since the introduction of required parameters wasn't possible to show
the help message.
In fact the --help flag was checked after all other parameters.

Previously, clients would look for the daemon by name which was not robust.
If the daemon was started with valgrind for example, clients would not find
the daemon.

Similar to other applications a daemon now creates a pid file with
information about its pid and its started mountdir.

Proper errorhandling on server and client side was added.

Multiple daemons per node are explicitly not allowed for now!

Review and fixes that died when rebasing all commits

Nafi3 authored Mar 12, 2018 and Marc Vef committed Mar 23, 2018

This commit gives new options regarding separation of metadir and rootdir location. If only rootdir destination is given both metadata and data is stored at the same path

When rocksdb is build with jemalloc linking with adafs was failing

The LRU map caused multiple issues with memory mappings and caused
severe errors and hanging situations. It has been replaced with a
std::map.

An Argobots mutex was added for rpc address lookups as many threads
might insert and lookup this map in parallel. This caused bulk_transfer
seqfauls as well as other file system crashes.

Cleanup of file system logging.