Resolve "Increase transfer status to see the status of all the files"

Closes #38

See merge request !26

has the output filename
BW shows mean BW for the file.

Fix fallocate error and add support for adhoc backends

fallocate is different from posix_fallocate, the first fails if the fallocate operation is not supported by the FS, like BEEGFS. However, the second one has a fail-safe and implements a write '0's to create the complete file.

In this MR , we also include support for different adhoc fs (parallel, posix, gekkofs, hercules, dataclay and expand)

We remove mpio in the ccp options and change it to parallel. Now is also posible to have a pure posix - posix transfer, which seems more performant than MPIIO

Closes #35, #36, #37

See merge request !25

function

is set

posix_file::file::fallocate

Resolve "If a final '/' is used in --input  then filenames in output are cut"

Closes #33 
Closes #34 
Release 0.3.3

See merge request !24

alternative for beegfs

Release 0.3.2

Closes #31

See merge request !23

added backends type

Resolve "Update Spack recipe to 0.3.1"

Closes #30

See merge request !22

Resolve "Change seq data transfers to progress under tests"

Closes #29

See merge request !21

Resolve "Update Spack recipe v0.3.0"

Closes #28

See merge request !20

Resolve "Release Cargo v0.3.0"

Closes #27

See merge request !19

Directory support

This MR adds two features:

- Creation of directories on the output. As cargo is normally running in the prolog the user has zero capabilities to create directories on the adhoc filesystem (as it is ephemeral). This MR creates the needed directories.

- If the input contains a directory, we get all the files recursively. Before this MR only files can be used as input. The behaviour is that the input is used as prefix, i.e., relative directory, and will be put inside the output.

- Prepares for 0.3.0 release 

 The Merge request goes over !16

See merge request !18

Ramon Nou authored Oct 30, 2023 and Ramon Nou committed Nov 07, 2023

updated thread

Moving info from master to worker progress

loop update

rebase task

added chrono fmt

Progress loop extracted

Reduce simultaneous transfers

wrong Error code in write

error control

Missing size for write

Added BW, solved sleep bug (intialization)

BW workflow finished

Support for creating directories

Directory Support

Reduce CPU usage with dynamic wait

Only call create directories, if the path is not 0

Updated Spack for 0.4.0

bw shaping workflow

This MR :
- Breaks pread / pwrite processes in progressive steps so we can control dataflow
- Adds BW_shaping RPC and MPI messages 
- Adds BW information inside the transfer_update message.
-

See merge request !16