BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Denver
X-LIC-LOCATION:America/Denver
BEGIN:DAYLIGHT
TZOFFSETFROM:-0700
TZOFFSETTO:-0600
TZNAME:MDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0600
TZOFFSETTO:-0700
TZNAME:MST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260422T000712Z
LOCATION:704-706
DTSTART;TZID=America/Denver:20231112T161400
DTEND;TZID=America/Denver:20231112T163400
UID:submissions.supercomputing.org_SC23_sess437_ws_shda101@linklings.com
SUMMARY:Benchmarking and In-Depth Performance Study of Large Language Mode
 ls on Habana Gaudi Processors
DESCRIPTION:Chengming Zhang and Baixi Sun (Indiana University); Xiaodong Y
 u (Stevens Institute of Technology, Argonne National Laboratory (ANL)); Zh
 en Xie, Weijian Zheng, Kamil A. Iskra, and Pete Beckman (Argonne National 
 Laboratory (ANL)); and Dingwen Tao (Indiana University)\n\nTransformer mod
 els suffer from high computational complexity. Habana GAUDI architecture o
 ffers a promising solution to tackle these issues. GAUDI features a Matrix
  Multiplication Engine (MME) and a cluster of fully programmable Tensor Pr
 ocessing Cores (TPC). This paper explores the untapped potential of using 
 GAUDI processors to accelerate Transformer-based models, addressing key ch
 allenges in the process. First, we provide a performance comparison betwee
 n the MME and TPC components, illuminating their relative strengths and we
 aknesses. Second, we explore strategies to optimize MME and TPC utilizatio
 n, offering practical insights to enhance computational efficiency. Third,
  we evaluate the performance of Transformers on GAUDI, particularly in han
 dling long sequences and uncovering performance bottlenecks. Last, we eval
 uate the end-to-end performance of two Transformer-based large language mo
 dels (LLM) on GAUDI. The contributions of this work encompass practical in
 sights for practitioners and researchers alike. We delve into GAUDI's capa
 bilities for Transformers through systematic profiling, analysis, and opti
 mization exploration.\n\nTag: Accelerators, Codesign, Heterogeneous Comput
 ing, Task Parallelism\n\nRegistration Category: Workshop Reg Pass\n\nSessi
 on Chairs: Tong Shu (University of North Texas), Seung-Hwan Lim (Oak Ridge
  National Laboratory (ORNL)), Pavan Balaji (Meta AI), and Sanjukta Bhowmic
 k (University of North Texas)\n\n
END:VEVENT
END:VCALENDAR