RADIUSS Shared CI

Documentation of the CI infrastructure developed for RADIUSS projects.

RADIUSS Shared CI is a sub-project from the RADIUSS initiative focusing on sharing resource and documentation regarding Continuous Integration among RADIUSS projects.

Note

LLNL’s RADIUSS project (Rapid Application Development via an Institutional Universal Software Stack) aims to broaden usage across LLNL and the open source community of a set of libraries and tools used for HPC scientific application development.

Background and Motivation

Projects belonging to the RADIUSS scope are targeting the same machines and use Spack as a packaging system. We want to ensure that they build successfully with similar toolchains and we leverage Spack to do so.

We designed an automated CI infrastructure based on GitLab that we meant to be universal enough to be shared among RADIUSS projects. This infrastructure involves using Spack to setup the project dependencies and generate a configuration file. This allows projects to easily share the full context of their builds. The project is then built and tested as usual and most of the CI infrastructure is shared to avoid duplication and reduce maintenance cost.

Cool features provided by Radiuss-Shared-CI

  • A shared configuration, for reduced maintenance.

  • A core set of specs to build coherently with other projects.

  • Adding extra specs is supported.

  • Uses GitLab child pipelines to keep the pipeline readable.

  • Each child pipeline reports independently to GitHub.

  • Project can extend their pipeline at will.

Overview

We split the documentation in three steps necessary to adopt RADIUSS Shared CI methodology. Those actions will be documented in the user_guide.

  1. Use Spack to install dependencies and configure the project build. Spack provides a single context to express toolchains, machines setup and build sequence. It is increasingly used to install the dependency tree of large simulation codes.

  2. Build and test without breaking your habits. We do not require the adoption of Spack to build your code but we require that your build system accepts the configuration file generated by Spack as an input (CMakeCache.txt for CMake build system). That way, dependencies and options are already set coherently with the spec provided to build the dependency tree.

  3. Setup the CI using the shared template. Once you have put in the effort to implement the first two steps, you should be able to benefit from the shared CI infrastructure. In very complex scenario, you would still be able to use the RADIUSS Shared CI template as a starting point for a custom implementation.

In the dev_guide, we discuss the layout of the RADIUSS Shared CI infrastructure and how the different pieces work with one another. Technical choices are also explained there.