This MR adds the ccp
tool to Cargo so that data transfers can be started
from the command line. The tool supports the following arguments:
$ ccp -h
Cargo parallel copy tool
Usage: ccp [OPTIONS]
Options:
-h,--help Print this help message and exit
-s,--server ADDRESS Address of the Cargo server (can also be
provided via the CCP_SERVER environment
variable).
-i,--input SRC... Input dataset(s)
-o,--output DST... Output dataset(s)
--if FLAGS Flags for input datasets. Accepted values
- posix: read data using POSIX (default)
- mpio: read data using MPI-IO
--of FLAGS Flags for output datasets. Accepted values
- posix: write data using POSIX (default)
- mpio: write data using MPI-IO
Once a Cargo server is up and running, ccp
can be used as follows:
$ srun -N4 cargo -l ofi+tcp://192.18.0.10:62000 -o cargo.log
[...]
$ ccp -s ofi+tcp://192.18.0.10:62000 -i input0.dat input1.dat -o output0.dat output1.dat
Modes of operation that control how transfers should be executed can be selected
through the the --if
and --of
options. For now, posix
and mpio
modes are
supported. Note however that not all combinations are supported yet, and that
performance will depend on whether the underlying file system supports parallel
I/O through MPI-IO.
Closes #19 (closed)