BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Denver
X-LIC-LOCATION:America/Denver
BEGIN:DAYLIGHT
TZOFFSETFROM:-0700
TZOFFSETTO:-0600
TZNAME:MDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0600
TZOFFSETTO:-0700
TZNAME:MST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260422T000713Z
LOCATION:601
DTSTART;TZID=America/Denver:20231113T115000
DTEND;TZID=America/Denver:20231113T123000
UID:submissions.supercomputing.org_SC23_sess443_misc153@linklings.com
SUMMARY:Unlocking the Potential of Large Language Models for High-Performa
 nce Computing Code
DESCRIPTION:Gal Oren (Technion - Israel Institute of Technology)\n\nHigh-P
 erformance Computing (HPC) has long been the driving force behind advancem
 ents in science, engineering, and beyond. Yet, realizing the full potentia
 l of HPC applications has often been hampered by the intricate nature of p
 rogramming for the underlying parallel systems. In this keynote, we explor
 e a transformative approach that bridges the gap between human ingenuity a
 nd computational power using the capabilities of large language models (LL
 Ms).\n\nOur research is an exploration of how cutting-edge LLMs can be tai
 lored to the demanding domain of HPC, where computational speed and effici
 ency reign supreme. While LLMs have showcased remarkable proficiency in un
 derstanding and generating code, their training data primarily comes from 
 general-purpose codebases. In stark contrast, HPC code involves intricate 
 mathematical modeling, parallelism, and optimization, demanding customized
  adaptations.\n\nThat is why our journey for ‘HPC LLMs’ began with the col
 lection of an extensive dataset, HPCorpus, that represents the culmination
  of HPC code in C, C++, and Fortran from diverse domains. Armed with this 
 invaluable resource, we embarked on an ambitious mission to enhance the ca
 pabilities of language models in the realm of HPC. The creation of Tokompi
 ler, a pioneering HPC-specific code tokenizer, marked a pivotal turning po
 int. Tokompiler, designed to preprocess code for language models, introduc
 ed a revolutionary approach that harnessed abstract syntax trees (ASTs) in
 to the source code itself and reshaped the way language models comprehend 
 and generate code, resembling how compilers perceive our codes, not humans
 . Building upon this innovation, we undertook comprehensive pre-training e
 fforts with CompCoder, adapting transformer-based language models to the i
 ntricacies of HPC. This journey has culminated in novel downstream tasks, 
 including the generation of OpenMP and MPI code, where our models shine by
  transforming serial code into efficient parallel one. Together, these mil
 estones represent a great leap forward in the convergence of AI and HPC fr
 om a different perspective, promising to redefine the landscape of computa
 tional science.\n\nAs we stand at the crossroads of AI and HPC, the possib
 ilities are boundless. Our journey is merely the prologue, unveiling a mul
 titude of untapped opportunities in HPC code comprehension, generation, an
 d optimization. From refining domain-specific code to tackling complex sim
 ulations and accelerating scientific breakthroughs, the horizons are vast.
  The symbiotic partnership between LLMs and HPC promises to revolutionize 
 how HPC practitioners write code. Looking ahead, we envision a future wher
 e LLMs for HPC become indispensable tools for researchers and developers i
 n their quest for unprecedented speed, accuracy, and efficiency.\n\nTag: A
 rtificial Intelligence/Machine Learning, Software Engineering\n\nRegistrat
 ion Category: Workshop Reg Pass\n\nSession Chairs: Giorgis Georgakoudis (L
 awrence Livermore National Laboratory (LLNL)), Ignacio Laguna (Lawrence Li
 vermore National Laboratory (LLNL)), and Konstantinos Parasyris (Lawrence 
 Livermore National Laboratory (LLNL))\n\n
END:VEVENT
END:VCALENDAR