Resolve "Various issues with dependency scripts"

Alberto Mirandarequested to merge
amiranda/176-using-d-option-in-dl_dep-sh-ends-up-downloading-all-dependencies into master

This MR adds some tests for the scripts based on bats-core, and enables its automatic execution by the CI.

It also changes the separator for dependency flavors from : to % to prevent clashing with the version separator for profiles (e.g. libfabric:experimental@mogon2:latest was ambiguous to bash).

Two new bash functions determine_compiler() and profile_has_dependency() are added to compile_dep.sh that are made available to install scripts:

  • The determine_compiler() function makes its best to determine the default compiler available in the system. Installation scripts can access this information via the COMPILER_NAME, COMPILER_FULL_VERSION, and COMPILER_MAJOR_VERSION variables.
  • The profile_has_dependency() function allows installation scripts to check whether another dependency is provided by the currently loaded profile, and act accordingly. The function accepts a bash regex with the search argument.

Additionally, this MR fixes the following problems in dl_dep.sh and compile_dep.sh:

  • Using -d option in dl_dep.sh ends up downloading all dependencies (#176 (closed))
  • Specifying a full profile in dl_dep.sh with -d does not work (#177 (closed))
  • dl_deps.sh can download the same dependency in several versions but unpacks them into the same directory (#178 (closed))
  • libfabric experimental not working (#180 (closed))
  • psm2 is viewed as a dependency when in fact it is build as part of libfabric. Therefore if [[ "$((${#wgetdeps[@]}+${#clonedeps[@]}))" -ne "${#order[@]}" ]]; then fails (#180 (closed))
  • rocksdb experimental not working (#180 (closed))
  • rocksdb install of newer versions needs PREFIX for make install to specify install path (#180 (closed)) (Moving to CMake for 0.9.0
  • When clonedeps is using a tag/branchname to clone, e.g., HEAD@v1.12.1, clonedeps_args for the dependency needs explicit space, e.g., ["mercury"]=" --recurse-submodules" otherwise the script fails.
  • Dependency patches are being skipped (#184 (closed)).
  • Script should use script directory as pwd instead of bash_pwd?

Closes #176 (closed) #177 (closed) #178 (closed) #180 (closed) #184 (closed)

Edited by Alberto Miranda

Merge request reports

scripts/profiles/0.8.0/install/agios.install

+5 −1
Changes for scripts/profiles/0.8.0/install/agios.install: 5 added lines, 1 removed line.
Original line number Diff line number Diff line
@@ -37,11 +37,15 @@ 
##               downloaded

## - INSTALL_DIR: the directory where the package should be installed

## - CORES: the number of cores to use when building

## - COMPILER_NAME: the name of the compiler being used (e.g. g++, clang, etc.)

## - COMPILER_FULL_VERSION: the compiler's full version (e.g. 9.3.0)

## - COMPILER_MAJOR_VERSION: the compiler's major version (e.g. 9)

## - PERFORM_TEST: whether tests for the package should be executed

################################################################################



pkg_install() {

    CURR="${SOURCE_DIR}/agios"

    ID="agios"

    CURR="${SOURCE_DIR}/${ID}"

    prepare_build_dir "${CURR}"

    cd "${CURR}/build"

    ${CMAKE} -DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" ..

scripts/profiles/0.8.0/install/argobots.install

+5 −1
Changes for scripts/profiles/0.8.0/install/argobots.install: 5 added lines, 1 removed line.
Original line number Diff line number Diff line
@@ -37,11 +37,15 @@ 
##               downloaded

## - INSTALL_DIR: the directory where the package should be installed

## - CORES: the number of cores to use when building

## - COMPILER_NAME: the name of the compiler being used (e.g. g++, clang, etc.)

## - COMPILER_FULL_VERSION: the compiler's full version (e.g. 9.3.0)

## - COMPILER_MAJOR_VERSION: the compiler's major version (e.g. 9)

## - PERFORM_TEST: whether tests for the package should be executed

################################################################################



pkg_install() {

    CURR="${SOURCE_DIR}/argobots"

    ID="argobots"

    CURR="${SOURCE_DIR}/${ID}"

    prepare_build_dir "${CURR}"

    cd "${CURR}"

    ./autogen.sh

scripts/profiles/0.8.0/install/bmi.install

+5 −1
Changes for scripts/profiles/0.8.0/install/bmi.install: 5 added lines, 1 removed line.
Original line number Diff line number Diff line
@@ -37,11 +37,15 @@ 
##               downloaded

## - INSTALL_DIR: the directory where the package should be installed

## - CORES: the number of cores to use when building

## - COMPILER_NAME: the name of the compiler being used (e.g. g++, clang, etc.)

## - COMPILER_FULL_VERSION: the compiler's full version (e.g. 9.3.0)

## - COMPILER_MAJOR_VERSION: the compiler's major version (e.g. 9)

## - PERFORM_TEST: whether tests for the package should be executed

################################################################################



pkg_install() {

    CURR="${SOURCE_DIR}/bmi"

    ID="bmi"

    CURR="${SOURCE_DIR}/${ID}"

    prepare_build_dir "${CURR}"

    cd "${CURR}"

    ./prepare

scripts/profiles/0.8.0/install/bzip2.install

+5 −1
Changes for scripts/profiles/0.8.0/install/bzip2.install: 5 added lines, 1 removed line.
Original line number Diff line number Diff line
@@ -37,11 +37,15 @@ 
##               downloaded

## - INSTALL_DIR: the directory where the package should be installed

## - CORES: the number of cores to use when building

## - COMPILER_NAME: the name of the compiler being used (e.g. g++, clang, etc.)

## - COMPILER_FULL_VERSION: the compiler's full version (e.g. 9.3.0)

## - COMPILER_MAJOR_VERSION: the compiler's major version (e.g. 9)

## - PERFORM_TEST: whether tests for the package should be executed

################################################################################



pkg_install() {

    CURR="${SOURCE_DIR}/bzip2"

    ID="bzip2"

    CURR="${SOURCE_DIR}/${ID}"

    cd "${CURR}"

    make install PREFIX="${INSTALL_DIR}"

}

scripts/profiles/0.8.0/install/capstone.install

+5 −1
Changes for scripts/profiles/0.8.0/install/capstone.install: 5 added lines, 1 removed line.
Original line number Diff line number Diff line
@@ -37,11 +37,15 @@ 
##               downloaded

## - INSTALL_DIR: the directory where the package should be installed

## - CORES: the number of cores to use when building

## - COMPILER_NAME: the name of the compiler being used (e.g. g++, clang, etc.)

## - COMPILER_FULL_VERSION: the compiler's full version (e.g. 9.3.0)

## - COMPILER_MAJOR_VERSION: the compiler's major version (e.g. 9)

## - PERFORM_TEST: whether tests for the package should be executed

################################################################################



pkg_install() {

    CURR="${SOURCE_DIR}/capstone"

    ID="capstone"

    CURR="${SOURCE_DIR}/${ID}"

    prepare_build_dir "${CURR}"

    cd "${CURR}/build"

    ${CMAKE} -DCMAKE_INSTALL_PREFIX="${INSTALL_DIR}" -DCMAKE_BUILD_TYPE:STRING=Release ..