Verified Commit 15e7ad8d authored by Alberto Miranda's avatar Alberto Miranda ♨️
Browse files

Add README.md's in scripts/maintainer-tools/

parent d5737a7c
Loading
Loading
Loading
Loading
+9 −0
Original line number Diff line number Diff line
# Maintainer Tools

This folder contains several tools designed to help with the maintenance of the
project.

- **Copyright headers:** The [`copyright-headers`](https://storage.bsc.es/gitlab/hpc/gekkofs/-/tree/master/scripts/maintainer-tools/copyright-headers) folder contains a
script to add/remove copyright headers from source files in the project.
The script relies on a Docker container that encapsulates a patched version of
[osterman/copyright-header](https://github.com/osterman/copyright-header).
+197 −0
Original line number Diff line number Diff line
# Copyright Headers
This folder contains the necessary files to allow maintainers to quickly
**add** and **remove** copyright headers from source files.

## TL;DR

General steps for updating copyright headers in the project:

0. Ensure that the `COPYRIGHT_YEARS` variable in `project.config` matches the
copyright headers currently in source files.

  ```shell
  COPYRIGHT_YEARS="{:bsc => [2018, 2020], :jgu => [2015, 2020]}"
                                      ^                     ^
  ```
1. Remove outdated copyright headers from source files:
  ```console
  cd <GEKKOFS-PROJECT-ROOT>/scripts/maintainer-tools/copyright-headers
  manage-headers.sh --remove --project-dir /projects/gekkofs/source
  ```
2. Edit the `project.config` file and update the `COPYRIGHT_YEARS` variable as
needed.
  ```shell
  COPYRIGHT_YEARS="{:bsc => [2018, 2021], :jgu => [2015, 2021]}"
                                      ^                     ^
  ```
3. Add the new copyright headers to source files:
  ```console
  manage-headers.sh --add --project-dir /projects/gekkofs/source
  ```

If done correctly and the templates have not been modified further, only the
updated fields should show in a diff view:

  ```diff
  diff --git a/examples/gfind/gfind.cpp b/examples/gfind/gfind.cpp
  index 1370e528..116b1344 100644
  --- a/examples/gfind/gfind.cpp
  +++ b/examples/gfind/gfind.cpp
  @@ -1,6 +1,6 @@
   /*
  -  Copyright 2018-2021, Barcelona Supercomputing Center (BSC), Spain
  -  Copyright 2015-2021, Johannes Gutenberg Universitaet Mainz, Germany
  +  Copyright 2018-2020, Barcelona Supercomputing Center (BSC), Spain
  +  Copyright 2015-2020, Johannes Gutenberg Universitaet Mainz, Germany

     This software was partially supported by the
     EC H2020 funded project NEXTGenIO (Project ID: 671951, www.nextgenio.eu).
   ```

## Further Details

### manage-headers.sh
The main script to manage copyright headers. It can be instructed to
auto-detect targets for imprinting or to work on specific files:

```shell
###########################
## Target auto-detection ##
###########################
# 1. add a copyright header to all source files in `--project-dir` that
#    don't currently have one
manage-headers.sh --add --project-dir /projects/gekkofs/source

# 2. remove the copyright header from all source files in `--project-dir` that
#    currently have one
manage-headers.sh --remove --project-dir /projects/gekkofs/source

###############################
## Specific target selection ##
###############################
# 1. add a copyright header to '/projects/gekkofs/source/scripts/ci/coverage.sh'
#    if it doesn't already have one
manage-headers.sh --add /projects/gekkofs/source/scripts/ci/coverage.sh

# 2. remove the copyright header from '/projects/gekkofs/source/scripts/ci/coverage.sh'
#    if it already has one
manage-headers.sh --remove /projects/gekkofs/source/scripts/ci/coverage.sh
```

#### Controlling the generated headers
The copyright headers generated by the scripts can be controlled by editing the
following files:
* `gekkofs-syntax.yml`: A YAML file that describes the details of how each source
file type should be handled, e.g. which extensions should be considered for a
certain file type, which types of comments should be used, etc.
* `gekkofs-template.erb`: The main template for the copyright header. It supports
limited Ruby expansion from the variables defined in `project.config`.
* `project.config`: The main project configuration file. It contains variables
that control how the copyright header is produced, such as the name of the
software, its description or the years for which copyright applies, as well as
which templates should be used to expand these variables.

#### NOTES
1. To prevent undesired modifications to source files, the `manage-headers.sh`
script only considers files that are already under version control.
2. Due to limitations on how the `manage-headers.sh` script currently handles
Docker volumes, manually selecting targets may fail if the target itself is
not a sub-path of the current project directory, which defaults to `$PWD`
unless explicitly set with `--project-dir`.

### Osterman's 'copyright-headers' Ruby gem
The `manage-header.sh` relies on a [Ruby gem by Erik Osterman](https://github.com/cloudposse/copyright-header) to do the header imprinting. To make it more suitable for
our needs, the gem has been patched with several modifications and encapsulated
into the `gekkofs/copyright-header` Docker image for ease of use. The following
files are used in this process.

 * `Dockerfile`: The Docker file for the `gekkofs/copyright-header` image. It
 basically downloads the latest version of Osterman's gem, patches it and
 defines an entrypoint for it.
 * `patch.diff`: Our specific modifications to the original Ruby gem.

### Tips and Tricks

The following techniques may be useful to find out what's happening if the
`manage-headers.sh` script is not behaving as expected:

1. Checking which targets will be considered by the script:
  ```shell
  manage-headers.sh --add --project-dir /projects/gekkofs/source --show-targets
  -- Project directory: '/projects/gekkofs/source' --
  Checking '/projects/gekkofs/source/test/main_temp.cpp'                                      [OK]
  Checking '/projects/gekkofs/source/test/symlink_test.cpp'                                   [OK]
  Checking '/projects/gekkofs/source/test/main_IO_testing.cpp'                                [OK]
  Checking '/projects/gekkofs/source/test/CMakeLists.txt'                                     [OK]
  ...
  Checking '/projects/gekkofs/source/test/main.cpp'                                           [OK]
  Checking '/projects/gekkofs/source/test/dir_test.cpp'     [SKIPPED -- not under version control]
  ```

2. Executing a dry run:

  ```shell
  manage-headers.sh --add --project-dir /projects/gekkofs/source -n
  -- Project directory: '/projects/gekkofs/source' --
  Checking '/projects/gekkofs/source/test/main_temp.cpp'                                      [OK]
  Checking '/projects/gekkofs/source/test/symlink_test.cpp'                                   [OK]
  Checking '/projects/gekkofs/source/test/main_IO_testing.cpp'                                [OK]
  Checking '/projects/gekkofs/source/test/CMakeLists.txt'                                     [OK]
  ...
  Checking '/projects/gekkofs/source/test/main.cpp'                                           [OK]
  Checking '/projects/gekkofs/source/test/dir_test.cpp'     [SKIPPED -- not under version control]
  -- DRY RUN --
  SKIP /projects/gekkofs/source/test/main_temp.cpp; exception=detected existing license
  SKIP /projects/gekkofs/source/test/symlink_test.cpp; exception=detected existing license
  SKIP /projects/gekkofs/source/test/main_IO_testing.cpp; exception=detected existing license
  SKIP /projects/gekkofs/source/test/CMakeLists.txt; exception=detected existing license
  SKIP /projects/gekkofs/source/test/main.cpp; exception=detected existing license
  SKIP /projects/gekkofs/source/test/dir_test.cpp; exception=detected existing license
  ```

1. Invoking the docker image directly:

  ```shell
  docker run -ti --rm gekkofs/copyright-header:latest --help
  Usage: /usr/local/bundle/bin/copyright-header options [file]
    -n, --dry-run                    Output the parsed files to STDOUT
    -o, --output-dir DIR             Use DIR as output directory
        --license-file FILE          Use FILE as header (instead of using --license argument)
        --license [AGPL3|ASL2|BSD-2-CLAUSE|BSD-3-CLAUSE|BSD-4-CLAUSE|Boost|GPL3|LGPL3|MIT]
                                     Use LICENSE as header
        --copyright-software NAME    The common name for this piece of software (e.g. "Copyright Header")
        --copyright-software-description DESC
                                     The detailed description for this piece of software (e.g. "A utility to manipulate copyright headers on source code files")
        --copyright-holder NAME      The legal owner of the copyright for the software. (e.g. "Erik Osterman <e@osterman.com>"). Repeat argument for multiple names.
        --copyright-year YEAR        The years for which the copyright exists (e.g. "2012"). Repeat argument for multiple years.
    -w, --word-wrap LEN              Maximum number of characters per line for license (default: 80)
    -a, --add-path PATH              Recursively insert header in all files found in path (allows multiple paths separated by platform path-separator ":")
    -r, --remove-path PATH           Recursively remove header in all files found in path (allows multiple paths separated by platform path-separator ":")
    -g, --guess-extension            Use the GitHub Linguist gem to guess the extension of the source code when no extension can be determined (experimental).
    -c, --syntax FILE                Syntax configuration file
    -V, --version                    Display version information
    -h, --help                       Display this screen
  ```

2. Invoking the docker image directly (with full volume substitution):
  ```shell
  docker run -ti --rm \
    --volume=/projects/gekkofs/source/scripts/maintainer-tools/copyright-headers:/projects/gekkofs/source/scripts/maintainer-tools/copyright-headers \
    --volume=/projects/gekkofs/source/scripts/maintainer-tools/copyright-headers/gekkofs-template.erb:/usr/src/gekkofs-template.erb \
    --volume=/projects/gekkofs/source/scripts/maintainer-tools/copyright-headers/gekkofs-syntax.yml:/usr/src/gekkofs-syntax.yml \
    gekkofs/copyright-header:latest [COPYRIGHT_HEADER_ARGS]
  ```
3. Starting an interactive shell inside the docker image:
  ```shell
  docker run -ti --rm --entrypoint /bin/bash gekkofs/copyright-header:latest
  ```

4. Starting an interactive shell inside the docker image (with full volume substitution):
  ```shell
  docker run -ti --rm \
    --volume=/projects/gekkofs/source/scripts/maintainer-tools/copyright-headers:/projects/gekkofs/source/scripts/maintainer-tools/copyright-headers \
    --volume=/projects/gekkofs/source/scripts/maintainer-tools/copyright-headers/gekkofs-template.erb:/usr/src/gekkofs-template.erb \
    --volume=/projects/gekkofs/source/scripts/maintainer-tools/copyright-headers/gekkofs-syntax.yml:/usr/src/gekkofs-syntax.yml \
    --entrypoint /bin/bash \
    gekkofs/copyright-header:latest
  ```