Commit dc51e1c1 authored by Alberto Miranda's avatar Alberto Miranda ♨️
Browse files

Update README.md

parent 723c7ef7
Loading
Loading
Loading
Loading
Loading
+56 −19
Original line number Diff line number Diff line
# Norns
# NORNS
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![pipeline status](https://storage.bsc.es/gitlab/hpc/norns/badges/master/pipeline.svg)](https://storage.bsc.es/gitlab/hpc/norns/commits/master)
[![coverage report](https://storage.bsc.es/gitlab/hpc/norns/badges/master/coverage.svg)](https://storage.bsc.es/gitlab/hpc/norns/commits/master)

Norns is an open-source data scheduling service that orchestrates asynchronous
NORNS is an open-source data scheduling service that orchestrates asynchronous
data transfers between different storage backends in an HPC cluster. Through
its API, Norns provides three different functions. First, it allows system
its API, NORNS provides three different functions. First, it allows system
administrators to expose the storage architecture of an HPC cluster by creating
**dataspaces** associated to different storage backends such as node-local NVMs 
and POSIX file systems, or system-wide parallel file systems and object stores, 
@@ -17,20 +17,20 @@ objects. Third, it arbitrates requests by managing a queue of pending work and
evaluating requests to maximize dataspace throughput while minimizing the 
interferences with normal application I/O.

Norns has currently been tested only under GNU/Linux.
NORNS has currently been tested only under GNU/Linux.

## Building and installing from source

Distribution tarballs are available from the [releases](releases) tab. If you 
are building Norns from a developer Git clone, you must first run the
`bootstrap.sh` script, which will invoke the GNU Autotools to bootstrap Norns'
configuration and build mechanisms. If you are building Norns from an official
are building NORNS from a developer Git clone, you must first run the
`bootstrap.sh` script, which will invoke the GNU Autotools to bootstrap NORNS'
configuration and build mechanisms. If you are building NORNS from an official
distribiution tarball, there is no need to run the `bootstrap.sh` script, since
all distribution tarballs are already boostrapped.

### Dependencies

Compiling and running Norns requires up-to-date versions of the following
Compiling and running NORNS requires up-to-date versions of the following
software packages (note that, though it may compile and run, using excessively
old versions of these packages can cause indirect errors that are very
difficult to track down):
@@ -53,15 +53,50 @@ difficult to track down):
- [Hermes](https://storage.bsc.es/gitlab/hpc/hermes) (our own C++ wrapper for
  Mercury. It should be automatically downloaded when cloning with the
  `--recursive` option).
- [libarchive](https://github.com/libarchive/libarchive) (3.1.2 or higher).

#### Installation in CentOS

TODO
```bash
# Installing dependencies available through package manager
$ yum install -y boost-devel protobuf protobuf-devel protobuf-c protobuf-c-devel \
                 yaml-cpp yaml-cpp-devel libyaml-devel libarchive

# Building and installing libfabric (required for Mercury's OFI/libfabric plugin)
$ git clone https://github.com/ofiwg/libfabric.git && cd libfabric
$ ./autogen.sh
$ mkdir build && cd build 
$ ../configure && make && make install

# Building and installing Mercury with OFI/libfabric plugin
$ git clone https://github.com/mercury-hpc/mercury.git && cd mercury
$ mkdir build && cd build
$ cmake -DCMAKE_BUILD_TYPE:STRING=Debug -DBUILD_TESTING:BOOL=OFF \
        -DMERCURY_USE_SM_ROUTING:BOOL=OFF -DMERCURY_USE_SELF_FORWARD:BOOL=OFF \
        -DMERCURY_USE_CHECKSUMS:BOOL=OFF -DMERCURY_USE_BOOST_PP:BOOL=ON \
        -DMERCURY_USE_EAGER_BULK:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=ON \
        -DNA_USE_OFI:BOOL=ON \
        ..
$ make && make install

# Building, testing and installing NORNS under '/usr/local/', with configuration
# files under '/etc/norns/' and temporary files under '/var/run/norns/'
$ git clone --recursive https://storage.bsc.es/gitlab/hpc/norns.git && cd norns
$ ./bootstrap.sh
$ mkdir build && cd build 
$ ../configure \
    --enable-tests \
    --prefix=/usr/local \
    --sysconfdir=/etc/norns \
    --localstatedir=/var/run/norns
$ make && make check && make install

```

#### Installation in Ubuntu

```bash
# Installing dependencies avaiable through package manager
# Installing dependencies available through package manager
$ apt-get install -y libboost-system-dev libboost-filesystem-dev \
                     libboost-program-options-dev libboost-thread-dev \
                     libboost-regex-dev libprotobuf-dev protobuf-compiler \
@@ -69,15 +104,13 @@ $ apt-get install -y libboost-system-dev libboost-filesystem-dev \
                     libyaml-cpp-dev libyaml-dev libarchive-dev

# Building and installing libfabric (required for Mercury's OFI/libfabric plugin)
$ git clone https://github.com/ofiwg/libfabric.git &&
$ cd libfabric
$ git clone https://github.com/ofiwg/libfabric.git && cd libfabric
$ ./autogen.sh
$ mkdir build && cd build 
$ ../configure && make && make install

# Building and installing Mercury with OFI/libfabric plugin
$ git clone https://github.com/mercury-hpc/mercury.git
$ cd mercury
$ git clone https://github.com/mercury-hpc/mercury.git && cd mercury
$ mkdir build && cd build
$ cmake -DCMAKE_BUILD_TYPE:STRING=Debug -DBUILD_TESTING:BOOL=OFF \
        -DMERCURY_USE_SM_ROUTING:BOOL=OFF -DMERCURY_USE_SELF_FORWARD:BOOL=OFF \
@@ -87,10 +120,9 @@ $ cmake -DCMAKE_BUILD_TYPE:STRING=Debug -DBUILD_TESTING:BOOL=OFF \
        ..
$ make && make install

# Building, testing and installing Norns under '/usr/local/', with configuration
# Building, testing and installing NORNS under '/usr/local/', with configuration
# files under '/etc/norns/' and temporary files under '/var/run/norns/'
$ git clone --recursive https://storage.bsc.es/gitlab/hpc/norns.git
$ cd norns
$ git clone --recursive https://storage.bsc.es/gitlab/hpc/norns.git && cd norns
$ ./bootstrap.sh
$ mkdir build && cd build 
$ ../configure \
@@ -99,8 +131,13 @@ $ ../configure \
    --sysconfdir=/etc/norns \
    --localstatedir=/var/run/norns
$ make && make check && make install
```


# Optional: providing file system permission override capabilities to Norns
#### Configuration

```bash
# Optional: providing file system permission override capabilities to NORNS
# control daemon
$ setcap cap_sys_ptrace,cap_chown=+ep /usr/local/bin/urd
```
 No newline at end of file