How To

Add a new machine

Adding a new machine can be done directly in this project so that the configuration is shared with all.

The sub-pipeline definition

To add a new machine, create a corresponding pipelines/<machine>.yml file describing the sub-pipeline. There are two main cases: whether the machine uses Slurm/Flux or LSF Spectrum as a scheduler.

Machines using Slurm/Flux scheduler

For machines using Slurm (or Flux) scheduler, use ruby (or corona) as a starting point. Then replace all the instances of “RUBY” and “ruby” with the new machine name.

Then go to customization/custom-jobs-and-variables.yml and add the variables:

  • <MACHINE>_SHARED_ALLOC with allocation options sufficient for the shared allocation (salloc, flux --parent alloc) to cover for all the jobs.

  • <MACHINE>_JOB_ALLOC with allocation options for any of the jobs (srun) the machine will take. The job will run under the shared allocation. Lastly, do not prescribe a number of cores here, as they should be defined at Spack and Make/CMake level.

Note

Use the values we have for ruby and corona as guides, but adapt the partition, number of cpus per task and duration coherently with the machine. The job allocation specied should cover for most job. If you have only a few jobs that need longer allocation, you should override them in your local jobs and set specific allocation options for them.

Machines using LSF Spectrum scheduler

For machines using LSF Spectrum scheduler, use lassen as a starting point. Then replace all the instances of “LASSEN” and “lassen” with the new machine name.

Then go to customization/custom-jobs-and-variables.yml and add the variables:

  • <MACHINE>_JOB_ALLOC with allocation options covering for any of the jobs the machine will take.

Note

Use the values we have for lassen as guides, but adapt the partition and duration coherently with the machine.

Reference the new sub-pipeline

In customization/subscribed-pipelines.yml, add a new section corresponding to the new machine. This is used by customization/gitlab-ci.yml to control which sub-pipelines are effectively generated.

Changelog

Don’t forget to provide a quick description of your changes in the CHANGELOG.md.

New tag

Once the new machine setup is tested and valid, submit a PR. We will review it and merge it. We may create a new tag although it is not required for a new machine. Indeed, using a new machine is a opt-in change for users: they will have to activate it in customization/subscribed-pipelines.yml the same way you did above (which only is a suggested template).