Skip to content

Resolve "Add a CLI tool for requesting data transfers."

This MR adds the ccp tool to Cargo so that data transfers can be started from the command line. The tool supports the following arguments:

$ ccp -h
Cargo parallel copy tool
Usage: ccp [OPTIONS]

Options:
  -h,--help                   Print this help message and exit
  -s,--server ADDRESS         Address of the Cargo server (can also be
                              provided via the CCP_SERVER environment
                              variable).
  -i,--input SRC...           Input dataset(s)
  -o,--output DST...          Output dataset(s)
  --if FLAGS                  Flags for input datasets. Accepted values
                                - posix: read data using POSIX (default)
                                - mpio: read data using MPI-IO
  --of FLAGS                  Flags for output datasets. Accepted values
                                - posix: write data using POSIX (default)
                                - mpio: write data using MPI-IO

Once a Cargo server is up and running, ccp can be used as follows:

$ srun -N4 cargo -l ofi+tcp://192.18.0.10:62000 -o cargo.log
[...]
$ ccp -s ofi+tcp://192.18.0.10:62000 -i input0.dat input1.dat -o output0.dat output1.dat

Modes of operation that control how transfers should be executed can be selected through the the --if and --of options. For now, posix and mpio modes are supported. Note however that not all combinations are supported yet, and that performance will depend on whether the underlying file system supports parallel I/O through MPI-IO.

Closes #19 (closed)

Edited by Alberto Miranda

Merge request reports