BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Denver
X-LIC-LOCATION:America/Denver
BEGIN:DAYLIGHT
TZOFFSETFROM:-0700
TZOFFSETTO:-0600
TZNAME:MDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0600
TZOFFSETTO:-0700
TZNAME:MST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260422T000712Z
LOCATION:405-406-407
DTSTART;TZID=America/Denver:20231116T143000
DTEND;TZID=America/Denver:20231116T150000
UID:submissions.supercomputing.org_SC23_sess175_pap347@linklings.com
SUMMARY:FORGE: Pre-Training Open Foundation Models for Science
DESCRIPTION:Junqi Yin, Sajal Dash, Feiyi Wang, and Mallikarjun Shankar (Oa
 k Ridge National Laboratory (ORNL))\n\nLarge language models (LLMs) are po
 ised to revolutionize the way we conduct scientific research, yet their co
 mplexity and cost hinder adoption by the wider science community. Identify
 ing suitable scientific use cases, optimizing model and data sizes, and sc
 aling up training are among the most pressing issues. Here we provide prac
 tical solutions for building and using LLM-based foundation models targeti
 ng scientific use cases. We present an end-to-end examination of the effec
 tiveness of LLMs in scientific research, including their scaling behavior 
 and computational requirements on Frontier, the first exascale supercomput
 er. We have also developed for release to the scientific community a suite
  of open foundation models called FORGE with up to 26B parameters using 25
 7B tokens from over 200M scientific articles. We have demonstrated the use
  and effectiveness of FORGE on scientific downstream tasks. Our research e
 stablishes best practices that can be applied across various fields to uti
 lize LLMs for scientific discovery.\n\nTag: Artificial Intelligence/Machin
 e Learning, Applications, Modeling and Simulation, State of the Practice\n
 \nRegistration Category: Tech Program Reg Pass\n\nSession Chair: Wei Xu (B
 rookhaven National Laboratory)\n\n
END:VEVENT
END:VCALENDAR