BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Denver
X-LIC-LOCATION:America/Denver
BEGIN:DAYLIGHT
TZOFFSETFROM:-0700
TZOFFSETTO:-0600
TZNAME:MDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0600
TZOFFSETTO:-0700
TZNAME:MST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260422T000712Z
LOCATION:710
DTSTART;TZID=America/Denver:20231112T120600
DTEND;TZID=America/Denver:20231112T123000
UID:submissions.supercomputing.org_SC23_sess419_ws_prot101@linklings.com
SUMMARY:Extra-Deep:  Automated Empirical Performance Modeling for Distribu
 ted Deep Learning
DESCRIPTION:Marcus Ritter and Felix Wolf (Technical University of Darmstad
 t)\n\nWith the rapidly increasing size and complexity of DNNs, equally sop
 histicated methods are needed to train them efficiently, including distrib
 uted training and various model/hybrid parallelism approaches. Even though
  developers heavily rely on state-of-the-art frameworks such as PyTorch an
 d TensorFlow, these provide little insight into an application's training 
 behavior at scale, leading to latent performance bottlenecks and inefficie
 nt training configurations. We propose Extra-Deep, an automated empirical 
 performance modeling approach for distributed deep learning. We leverage t
 he created models to analyze a training task's performance, scalability, e
 fficiency, and cost. Using an efficient sampling strategy that reduces the
  profiling time for the required empirical measurements by, on average, ab
 out 94.9%, we can identify cost-effective training configurations even for
  large-scale applications. We evaluated our approach on three parallelizat
 ion strategies, with four DNN models and five datasets. The results show t
 hat Extra-Deep has an average prediction accuracy of 93.6% when compared t
 o empirical results.\n\nTag: Performance Measurement, Modeling, and Tools,
  Programming Frameworks and System Software\n\nRegistration Category: Work
 shop Reg Pass\n\nSession Chairs: David Boehme (Lawrence Livermore National
  Laboratory (LLNL)), Anthony Danalis (AMD), and Josef Weidendorfer (Dresde
 n University of Technology)\n\n
END:VEVENT
END:VCALENDAR