Workshop Overview
How can AI workloads be engineered for optimal performance in modern HPC environments?
The rapid advancement of Artificial Intelligence (AI) and Machine Learning (ML) has positioned High-Performance Computing (HPC) systems as indispensable platforms for developing, training, and executing these workloads. However, the architectural complexity and batch-oriented design of traditional HPC systems pose unique challenges distinct from those encountered in resource-elastic environments such as clouds.
The parallelization characteristics, input/output requirements, and dynamic workflows of AI workloads demand innovative techniques for efficient utilization of HPC resources. Moreover, the performance engineering of such workloads is crucial to achieve scalability, portability, and reproducibility across diverse system architectures.
This workshop aims to bring together researchers, practitioners, and system developers to discuss engineering challenges, performance optimization, and emerging opportunities at the intersection of AI and HPC. It invites among others, papers that present experimental results, architectural insights, performance studies, and best practices advancing the convergence of these domains.
Program (draft)
| Time | Topic | Presenter |
|---|---|---|
| 9:00-11:00 | Invited Talk: performance inefficiencies in LLM fine-tuning | Gokcen Kestor, Barcelona Supercomputing, Spain |
| PreLoRA: Hybrid Pre-training of Vision Transformers with Full Training and Low-Rank Adapters | Murali Emani, Argonne National Laboratory, USA | |
| MCast: Generalizing HPC Application Runtime Prediction | Avani Wildani, Cloudflare | |
| Panel Discussion: The Rise of HPC-Driven AI Factories Worldwide Panel Moderator: Vijeta Sharma, MIMER AI Factory, Sweden |
|
|
| 11:00-11:30 | ☕ Coffee Break | |
| 11:30-13:00 | An Autotuning-based Hyperparameter Optimization Framework for Mixed-kernel SVM Classifications in Science and Engineering | Xingfu Wu, Argonne National Laboratory, USA |
| Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads | Minyu Cui, Chalmers University of Technology, Sweden | |
| Optimizing Teacher-Student Partitioning for Scalable Knowledge Distillation on HPC Systems | Adrian Perez Dieguez, Qualcomm, USA | |
| LLMs in Performance Engineering: New Workflows for Autonomous Code Optimization | Anja Gerbes, GWDG, Germany | |
| 13:00-14:00 | 🍲 Lunch Break | |
| 14:00-16:00 | Keynote: Challenges and Opportunities in Training Next-Generation Mixture of Expert Models on HPC Platforms | Sajal Dash, Oak Ridge National Laboratory, USA |
| TiledAttention: a CUDA Tile SDPA Kernel for PyTorch | Taimur Khan, UFZ, Germany | |
| Performance Engineering for the SEODA Project for CASPIr (Lightning Talk) | Buket Benek Gursoy, Irish Centre for High-End Computing, Ireland | |
| JumpLM: Simultaneous Visualization of Hardware and LLM Metrics for a Joint Configuration Tuning | Lena Jurkschat, ScaDS.AI, Germany | |
| AI Application Benchmarking: Power-Aware Performance Analysis for Vision and Language Models | Lukas Schröder, NHR@FAU, Germany | |
| 16:00-18:00 | ☕ Coffee Break / open discussions |
Call for Papers
We invite submissions of original research papers, case studies, and experience reports that address the challenges and opportunities at the intersection of AI/ML and HPC.The papers submitted to this workshop will be published in LNCS, Springer.
Submission Guidelines
- Format: LNCS (Lecture Notes in Computer Science) format
- Length: 6 to 12 pages for full papers (including a bibliography and appendices)
- Lightning Talks: Maximum 1 page abstract (talks excluded from publication)
- Portal: ISC Submission portal
- Submissions should be original and not previously published
- Download LNCS template from Springer
Topics of Interest
We welcome submissions on the following topics, including but not limited to:
Workload Characterization
- Characterizing AI/ML workloads on HPC systems
- Data preparation for AI/ML workload on HPC
- Hybrid workloads on HPC systems
Performance & Optimization
- Parallelization strategies for AI/ML
- Performance optimization of AI/ML frameworks on HPC
- Efficient inference of LLMs on HPC
- Cross-platform portability and reproducibility
Infrastructure & Systems
- AI factories and end-to-end pipelines
- Next-generation HPC systems for AI/ML
- Best practices for integrating ML/AI into HPC
- Specialized AI/ML frameworks for HPC
Resource Management
- Resource allocation and scheduling for AI/ML workloads
- Energy efficiency and power management
- DevOps and MLOps for HPC-AI/ML
Applications
- HPC-AI/ML convergence for scientific applications
- AI-enhanced HPC simulations
- Industrial AI/ML on HPC
- Collaborative and interactive AI/ML on HPC
Evaluation & Benchmarking
- HPC-AI/ML benchmarking and evaluation
- Performance studies and best practices
Important Dates
Submission Information
Submission Portal: Scientific contributions can be submitted via the ISC Submission portal.
Submissions may be either single-blind or double-blind and authors can decide whether to anonymise their names and affiliations.
For questions about submissions, please contact the General chairs.
Workshop Organization
General Chairs
Program Chairs
Dr. Paramita Mirza
Fraunhofer IIS, Germany
Dr. Taras Lazariv
ScaDS.AI Dresden/Leipzig, Germany
Prof. Dr. Faouzi Alaya Cheikh
Norwegian University of Science and Technology, Norway
Dr. Ernst Gunnar Gran
Norwegian University of Science and Technology, Norway
Program Committee
Media Chair
Contact
For questions, please contact the General Chairs:
- Dr. Siavash Ghiasvand: siavash.ghiasvand@tu-dresden.de
- Dr. Vijeta Sharma: vijeta.sharma@ri.se
- Dr. Ajeet Ram Pathak: ajeet.r.pathak@ntnu.no
Follow us on LinkedIn