WP7: HPC Resources for Exa‑MA

Lydie Grospellier, Christophe Prud’homme — version 1.1, 2025‑04‑22

1. Introduction & WP7 Overview

Work Package 7 (WP7) provides the “glue” for Exa‑MA, delivering the CI/CD framework, containerized environments, packaging standards, benchmarking suite, training materials, and automated deployment pipelines that bind together the methodological developments from WP1-WP6 into reproducible, high‑performance demonstrators on supercomputers.

2. Objectives and Deliverables

  • Demonstrators

    • Level 1 mini‑apps (single‑WP exercises)

    • Level 2 mini‑apps (cross‑WP exercises)

    • Level 3 proxy‑apps (full WP integration)

  • Benchmarking suite & KPI reports

    • Performance, stability, deployment time, user adoption

  • Automated deployment platform

    • Uniform Docker/Singularity images

    • SPAC & Guix‑HPC packages

  • Training materials & showroom

    • Tutorials, FAQs, recorded workshops

    • Web‑based portal showcasing live demos

3. Governance and Roles

  • Steering Committee

    • Representatives from WP1-WP6 + WP7 leads

  • Technical Referents

    • CI/CD pipelines

    • Containerization & packaging

    • Benchmarking & KPI tracking

    • Training & documentation

  • Demonstrator Leads

    • Assigned binômes (developer + WP expert) per demonstrator

4. CI/CD Pipelines

  • Version Control & Artifact Management

    • Git repository structure, tagging conventions, artifact registry

  • Automated Build & Test Workflows

    • Build → unit tests → integration tests → container image builds

  • Deployment to Test Environments

    • Staging cluster for full‑scale smoke tests

  • Pipeline Templates & Best Practices

    • Examples in .github/workflows/ci.yml

5. Containerization & Packaging

  • Docker & Singularity Images

    • Base images for common HPC stacks (MPI, CUDA, etc.)

  • SPAC & Guix‑HPC Packaging

    • Declarative package definitions for reproducibility

  • Image Registry & Versioning

    • Tagging by WP, demonstrator level, software version

6. Benchmarking & KPI Tracking

  • Key Performance Indicators

    • Execution time, throughput, memory footprint, scalability

  • Automated Data Collection

    • Scripts to extract logs, metrics, energy consumption

  • Dashboards & Reports

    • Shared Grafana/Prometheus dashboards

    • Quarterly KPI summary reports

7. Demonstrator Development Process

7.1. Demonstrator Levels

  • Level 1 (WP‑Specific Mini‑Apps)

  • Level 2 (Cross‑WP Mini‑Apps)

  • Level 3 (Proxy‑Apps covering ≥3 WPs)

7.2. Project Template & Responsibilities

  • Repository Skeleton

    • src/, tests/, ci/, containers/, docs/

  • CI/CD Configuration

    • Pre‑configured pipeline definitions

  • Binôme Assignment

    • Developer + WP expert collaborate on test definitions, benchmarks, packaging

8. Communication & Collaborative Tools

  • GitHub Project

    • Kanban board with issue templates, milestones, labels

  • Slack

    • Channels: #wp7-ci, #wp7-containers, #wp7-benchmarks, #wp7-training

  • Documentation Hub

    • GitHub Pages or internal Wiki

9. Training Plan & Capacity Building

  • Workshops & Tutorials

    • Monthly hands‑on sessions: CI/CD, containers, benchmarking

  • On‑Demand Materials

    • Recorded videos, step‑by‑step guides, FAQ

  • Mentorship & Office Hours

    • Weekly drop‑in sessions with technical referents

10. Maintenance, Support & Risk Management

  • Issue Tracking & Support

    • Dedicated Slack channel + GitHub issue queue

  • Risk Register

    • Tool/version compatibility, HPC site differences, network policies

  • Mitigation Plans

    • Regular upgrade cycles, multi‑site test matrix, fallback container images

11. Annexes & References

11.1. Annex A: Resource Request Form

For Level 1 demonstrator resource estimates, please use our [Resource Request Form](forms.gle/iBs6YNgb8ZXCLk8L6).

11.2. Annex B: Glossary of Performance Terms

  • CPU‑bound:: Performance limited by processor speed or insufficient parallelism.

  • Memory‑bound:: Performance limited by memory bandwidth or latency.

  • Latency‑bound:: Performance limited by communication or I/O delays.