Resolve "Separate metadata and data directories"

Closes #36

See merge request zdvresearch_bsc/adafs!15

Review and fixes that died when rebasing all commits

Nafi3 authored Mar 12, 2018 and Marc Vef committed Mar 23, 2018

This commit gives new options regarding separation of metadir and rootdir location. If only rootdir destination is given both metadata and data is stored at the same path

Adding optional JeMalloc dependency + rocksdb version bump

See merge request zdvresearch_bsc/adafs!27

Fix misaligned operations

See merge request zdvresearch_bsc/adafs!28

target was the index of the cycle and target[target] was the actual
identifier of the target. In some part of the code target was considered
as the identifier directly.

Now the index of the cycle is i and target is the actual identifier of
the target.

The calculus (on clients) about the last chunk involved into a
write/read operation was bugged.

A new blocks_calc_util library has been introduced in order to make
easier to perform this kind of calculus.

startup script: do not check for local daemon

See merge request zdvresearch_bsc/adafs!30

In the case the host running the startup script is not one of the node
that will run adafs it does not make sense to check for the binary of
the daemon locally.

When rocksdb is build with jemalloc linking with adafs was failing

Preload: Better naming in IO process + chnk_end target fix

See merge request zdvresearch_bsc/adafs!25

Resolve "Remove all chunks when a file is removed"

Closes #34

See merge request zdvresearch_bsc/adafs!22

bugfix: dl_script with non-existing output path

See merge request zdvresearch_bsc/adafs!23

Client io cleanup

See merge request zdvresearch_bsc/adafs!24

Instead use margo_iforward. Also only two bulk_handles are now used:
1 for margo_ipc and one for margo_rpc RPC requests.

The download script do not work if the specified output path
contains more then 1 non-existing folder.
This is due to the -f flag used in the readlink command.
We now use the less strict -m one.

Resolve "IO exceeding segment count"

Closes #32

See merge request zdvresearch_bsc/adafs!13

The LRU map caused multiple issues with memory mappings and caused
severe errors and hanging situations. It has been replaced with a
std::map.

An Argobots mutex was added for rpc address lookups as many threads
might insert and lookup this map in parallel. This caused bulk_transfer
seqfauls as well as other file system crashes.

Cleanup of file system logging.

Before data was pulled first then written or read first and then pushed back

Only use 1 segment now but with multiple transfers

ensure folder creation on download script

See merge request zdvresearch_bsc/adafs!20

Source folder was removed if existed before but never recreated

test: add simple Write/Read test

See merge request zdvresearch_bsc/adafs!19

This is a very simple test that just write a file and read it, checking
for content match.