Investigation: Multi-target Manifest Files

[CodeGat]

Background

We need to come to a decision on how the humble spack.yaml will fare in multiple targets HPCs. Specifically, we currently have a single spack.yaml for Gadi, but if we can't make it sufficiently generic (compilers, target, etc), we need a different solution.

Potential Implementations

• Simplest, cleanest, most naive: The single spack.yaml can be made sufficiently generic for all (future) deployment targets
• CI-enabled templating: Single spack.yaml, have sections that require modification per-target to be templated somehow - maybe as an environment variable, or some other marker in the spack.yaml that notes that the section is to be injected at install time. The injected spack.yaml is still part of a release artifact. But, as some incredibly wise person noted once: "Every line of code we write is a line of code we have to maintain".
• More complex (EDIT: Not that complex now due to work in Fix Up Multi-Target TODOs build-cd#199): Have a spack.yaml per target - eg. gadi.spack.yaml, setonix.spack.yaml. Has duplication of sections.

[aidanheerdegen]

Turns out we can use some approaches in spack environments to select, say compilers, based on an environment variable or even arch

https://spack.readthedocs.io/en/latest/environments.html#spec-list-references

[CodeGat]

AKA Option 1 looks feasible!

[CodeGat]

After testing, these variables are only interpreted on spack.specs rather than spack.packages.all.requires - we put our compiler and target definitions (the things we were looking to change) in the packages section. And I'm not even sure that we can put the target in the specs section, anyway. Seems like option 1 is a no...

[CodeGat]

I take it back, Option 1 works. It works with the following (and we can still point people to changing stuff in a single place, via:

spack:
  definitions:
    # This is the root-sbd
    - root_package:
      - [email protected]

    # Compiler and Target definitions
    - when: re.match(r'.+\.gadi\.nci\.org\.au', hostname)
      compiler_targets: ['%[email protected] target=x86_64']
    - when: re.match(r'setonix-.+', hostname)
      compiler_targets: ['%[email protected] target=x86_64_v4']

    # Root Spec Definition
    - root-spec:
      - matrix:
        - [$root_package]
        - [$compiler_targets]
  # add package specs to the `specs` list
  specs:
    - $root-spec