BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Denver
X-LIC-LOCATION:America/Denver
BEGIN:DAYLIGHT
TZOFFSETFROM:-0700
TZOFFSETTO:-0600
TZNAME:MDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0600
TZOFFSETTO:-0700
TZNAME:MST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260422T000713Z
LOCATION:704-706
DTSTART;TZID=America/Denver:20231112T171800
DTEND;TZID=America/Denver:20231112T172800
UID:submissions.supercomputing.org_SC23_sess437_ws_shda103@linklings.com
SUMMARY:Accelerating Hyperparameter Optimization Algorithms with Mixed Pre
 cision
DESCRIPTION:Marcel Aach (Forschungszentrum Jülich, University of Iceland);
  Rakesh Sarma and Eray Inanc (Forschungszentrum Jülich); Morris Riedel (Un
 iversity of Iceland, Juelich Supercomputing Centre (JSC)); and Andreas Lin
 termann (Forschungszentrum Jülich)\n\nHyperparameter Optimization (HPO) of
  Neural Networks is a computationally expensive procedure, that has the po
 tential to benefit from the use of novel accelerator capabilities. This pa
 per investigates the performance of three popular HPO algorithms in terms 
 of the achieved speed-up and model accuracy, utilizing early stopping, Bay
 esian, and genetic optimization approaches, in combination with mixed prec
 ision functionalities on NVIDIA A100 GPUs with Tensor Cores. The benchmark
 s are performed on 64 GPUs in parallel on three datasets: two from the vis
 ion and one from the CFD domain. The results show that, depending on the a
 lgorithm, larger speed-ups can be achieved for mixed precision compared to
  full precision HPO if the checkpoint frequency is kept low. In addition t
 o the reduced runtime, also small gains in generalization performance on t
 he test set are observed.\n\nTag: Accelerators, Codesign, Heterogeneous Co
 mputing, Task Parallelism\n\nRegistration Category: Workshop Reg Pass\n\nS
 ession Chairs: Tong Shu (University of North Texas), Seung-Hwan Lim (Oak R
 idge National Laboratory (ORNL)), Pavan Balaji (Meta AI), and Sanjukta Bho
 wmick (University of North Texas)\n\n
END:VEVENT
END:VCALENDAR