In 2015, Asmus was working in Colorado State University’s Information Technology division as a security analyst, focused on compliance, network firewalls, and routing. In this role, she met WINS founder Marla Meehl, who was then running the Front Range GigaPoP. With Meehl’s encouragement, Asmus applied for a WINS scholarship and, to her surprise, was named a 2016 WINS awardee. 

“I hadn’t really worked with network switches until I came to SCinet that year,” Asmus recalled. “Kevin Hayden [2024 Exhibits Chair] was my mentor, and he handed me a set of 65 switches and told me to upgrade and configure them. He had to walk me through the first one. Then it started really clicking and making sense. Along with the team, I did all the edge switches and wireless for the conference—and loved it.” 

Asmus formed a tight bond with the other WINS awardees and the SCinet team that year. 

“I met 200 great people who felt like a new family, and that made me want to come back.”

— Angie Asmus, SC24 SCinet Chair

From WINS to Career Wins

Asmus has come back every year since, working her way up to becoming the Edge Team Chair. At SC23, she served as SCinet Deputy Chair, and at SC24, she will be SCinet Chair in Atlanta. From WINS awardee to SCinet chair in a mere eight years is a pretty powerful trajectory, and Asmus gives the credit to WINS.

“WINS means everything to me,” she said. “It’s given me the confidence and experience to rise through the ranks in my career, not to mention the chance to present and speak on the importance of diversity in the tech field. I’ve gained skills I never knew I needed, from soft skills to tech skills.” 

To wit, Asmus was recently promoted to Interim Director of the Network and Telecommunications team at Colorado State University, where she manages three teams with 20 full-time employees. 

Ultimately, Asmus has this to say to women considering WINS: “Definitely apply for the WINS scholarship. It is life changing. It’s truly so impactful for what it can do for your career.” 

SC24 SCinet Chair Angie Asmus (left) and SC24 SCinet Technical Director Brenna Meade (right)

SC23 WINS Cohort in Denver

Learn More & Apply

Considering SCinet is the fastest and most powerful network in the world for the duration of the SC conference, this is no easy feat. Each year, the Help Desk assists SC Conference attendees and SCinet teams in resolving network problems quickly and smoothly by correct identification, complete documentation, appropriate handoff, and excellent communication, according to Karmen Goddard-Wall, SCinet Help Desk Chair, who is leading this effort along with Deputy Chair, Remy Scott. 

Remy Scott

Engineering Scientist Associate at Texas Advanced Computing Center (TACC)

Remy on LinkedIN

“This is where it can get challenging because this is a global conference. We have students, volunteers, vendors, and exhibitors from all around the world. Language barriers can occur, and, to add another layer on top of that, there can be miscommunications as well,” Scott noted.

Still, none of these potential challenges fazes the Help Desk team. “A lot of the Help Desk job is interacting and making connections with the people as much as the equipment,” Goddard-Wall noted. “That is the part that I personally like the best!”

Help Desk to the Rescue

New this year, the Help Desk will have its own space on the exposition floor, Booth 987, located next to the SCinet Network Operations Center (NOC). According to Goddard-Wall, there are a few top issues that can affect connectivity at SC, such as:

• Broken Fiber
• Incorrect Assembly of Equipment: Fiber is laid with a loop/coupler at the end or connectors are plugged into the DNOCs backwards
• Faulty Equipment: Sometimes, it gets damaged in transit
• For Wired or Wireless: Host has been quarantined due to a security problem

Karmen Goddard-Wall

SC23 SCinet Help Desk Chair

“There are people who love to steal passwords, insert malware, and do things that are illegal, for example, downloading copyrighted information or movies,” Goddard-Wall said.

The Help Desk team employs a variety of tools to diagnose network issues, from power meters to determine if fiber is intact to dashboards displaying all of the physical equipment, whether it is working, and if all the routing information is correct.

“A lot of times, just laying eyes on the equipment is the best way to go,” Goddard-Wall added. “If we have the volunteers, we physically walk over to the booth to make sure that the equipment is put together correctly, talk to the people involved, and see what lights are on/not on the switch.”

It is this kind of high-touch, human-friendly interaction that keeps the SCinet network humming along, year after year.

Contacting the SCinet Help Desk

For this year’s SCinet crew, that village is extensive and generous with more than 90 contributors donating $43 million worth of hardware, software, services, and nearly 200 SCinet volunteers. 

That’s what it takes to plan and execute the SCinet network, whose creation supports the revolutionary applications and experiments that are a hallmark of the SC Conference. SCinet enables exhibitors to demonstrate the advanced computing capabilities of their solutions and services while delivering wireless capability to more than 10,000 conference attendees. 

In short, the 2023 SCinet contributors and volunteers are the heart of the village that makes the SCinet magic happen.

“The equipment, WAN connectivity, software, and support personnel provided every year to SCinet by our contributors are literally what make the fastest network on Earth—for a week—possible.”

Brian Smith

SCinet Contributor Relations Co-Lead

“The equipment, WAN connectivity, software, and support personnel provided every year to SCinet by our contributors are literally what make the fastest network on Earth—for a week—possible,” said Brian Smith, SCinet Contributor Relations Co-Lead and Director of Technology at Cornelis Networks. SCinet contributors allow the network to pull in multiple terabits of connectivity, test new gear in a semi-production environment, and demonstrate the practicality of technologies, such as IPv6, to more than 10,000 attendees. 

“SCinet volunteers are able to take knowledge and experience they get working with loaned contributor gear back to their home institutions and help influence purchase decisions and potentially improve their home institution’s network speeds, reliability, and security,” Smith said. “All of this is possible because of the generosity of our contributors.”

Hans Addleman is equally effusive about SCinet’s contributor village. As 2023 SCinet Chair, he’s had a front-row seat to the difference their generosity—in both money and time—means for everyone at SC. With several levels of contribution available, from Bronze to Diamond, there’s something for everyone interested in supporting SCinet.

“I cannot overstate the critical role that our dedicated contributors play in the success of the SCinet network at SC23, notably this year’s Diamond contributors, Cisco and PIER Group.”

“I cannot overstate the critical role that our dedicated contributors play in the success of the SCinet network at SC23, notably this year’s Diamond contributors, Cisco and PIER Group,” Addleman said. “Their expertise, innovation, and unwavering commitment are the very backbone of this network, and together, we transform the impossible into reality, ensuring that SC remains a beacon of technological advancement and collaboration.”

Diamonds Are Forever

Every contribution is important, of course, but Diamond contributors go above and beyond with minimum contributions of $5 million each in materials, services, and volunteers. This year’s Diamond contributors are Cisco and PIER Group.

Cisco’s SCinet contributions this year include core routing (Cisco Series 8808 Router) and spine switch (Nexus 9232E) equipment, dense wavelength division multiplexing (NCS 1010, 1002, and 1004) equipment, optical DCI (NCS 1004) equipment, and Cisco NSO.

“At Cisco, the research community is near and dear. As a matter of fact, if it were not for Sandy Lerner and Leonard Bosack, students at Stanford University, we would not exist,” said Marcus Moffett, Cisco Vice President of Sales System Engineering for the U.S. public sector. “For the past 38 years, the research community has played a critical role as partners in shaping technology and how it positively impacts everything in our daily lives. Our mission is to securely connect everything, to make anything possible, and nowhere is that truer than in research. We at Cisco are proud to continue this long tradition of partnership through the support of SCinet and Supercomputing 2023.”

Marcus Moffett

Vice President of Sales System Engineering,
Cisco Systems

Caleb Williams

Account Manager and Contract Development Lead, PIER Group

For its part, PIER Group is providing about 400 wireless access points weighing over five tons (Aruba AP-635), Aruba Mobility Controller Software (Aruba OS 8.10) with Aruba 9240xm Controllers, Aruba ClearPass Policy Manager for wireless authentication and authorization, and 140 Aruba 6100 and 40 Aruba 6300M switches.

It’s money well spent, according to Caleb Williams, Account Manager and Contract Development lead at PIER Group: “The value PIER Group gets from SCinet is immense. We get to collaborate with our peers from universities and research institutions all over the country—all over the world—for weeks on end. We get to test new technologies and ways of networking that become realities a lot of times shortly after SCinet. Also, the bonds/friendships that are formed at SCinet between all the volunteers are truly bonds for life.”

Get Involved in SCinet 2024

It’s a little too late to become part of the SCinet village for this year, but 2024 is just around the corner.

Learn more about the SCinet contributor program, including details about how to participate and its benefits. Then start making next year’s plans today.

Participate in SCinet

Fall has arrived, and the SCinet team has begun its takeover of the Colorado Convention Center to start constructing the SCinet network for the SC23 conference. Volunteers of SCinet are among the earliest to arrive in Denver for the event called “Staging” week. At Staging, SCinet volunteers begin installing equipment borrowed through the SCinet contributors program under a tight deadline to enable basic network connectivity that will support the SC conference. However, for SC23, SCinet will not simply construct an ordinary network in Denver. This year’s version will provide an astonishing 6.71 Tbps (terabits per second) of network bandwidth to the convention center. This speed is sufficient to download the Library of Congress’ complete 21 petabyte (equivalent to 21000 terabytes) catalog in just over seven hours¹.

Achieving this goal necessitates forging robust partnerships with a multitude of external stakeholders, encompassing Research and Education (R&E) networks, commercial network providers, and researchers spanning the globe.

Wide Area Networks Need Wide Social Networks

Carlos Rojas-Torres

SCinet WAN Team Member

Leadership & Teamwork

The existence of the SCinet wide area network, or WAN, and collaborative efforts of SCinet contributors are absolutely essential for the realization of the conference network.

“The WAN team requires a strong and committed leadership team. The preparation and coordination of contributors, volunteers, and the SCinet committee is a multi-year endeavor,” explained Carlos Rojas-Torres, SCinet WAN team member from the University Corporation for Atmospheric Research (UCAR) and Front Range GigaPoP. “The team is composed of some of the most talented people in the industry, and it’s practically impossible to walk away from there without learning something new every single day.”

Relationships Matter

The WAN team’s ability to stay ahead of the numerous challenges they encounter each year can be attributed to the cultivation of these enduring multi-year partnerships and relationships. 

“The culture and relationships are what makes the WAN team successful,” said Chris Skaar, SCinet WAN team co-lead from University of Illinois Urbana-Champaign. “The leads for the WAN team have, over the years, fostered a culture of hard work, supporting one another, learning from each other, and developing long-lasting connections. We also try to help our contributors be as involved as possible and ensure that our team really learns about the products that they are donated.”

Chris Skaar

SCinet WAN Team Co-Chair

Big Tbps Requires Big Creativity

SCinet’s WAN design is intricately linked to the specific destinations researchers wish to connect to and the use cases that drive the presentation of their research at SC.

Cody Rotermund

SCinet WAN Team Deputy Chair

Putting the Puzzle Together

“Leading up to Staging week, it’s almost like we’re putting together a puzzle,” explained Cody Rotermund, SCinet WAN Team Deputy Chair from the Energy Sciences Network (ESnet). “We get some of the pieces through the end researchers or through the SCinet Network Research Exhibition (NRE) process. However, many of the puzzle pieces you need to form a complete picture aren’t really known until later in the year.  Essentially, to start putting an initial picture together, we do quite a bit of forecasting based in part on what we did in previous years. With that in mind, we’ll take a look at what fiber currently exists, and what we need to work with our contributors, to build out and go from there.”

A one-size-fits-all solution simply doesn’t exist. Each scenario demands a distinct technical approach and the judicious pairing of technology. The SCinet WAN leadership team invests a significant amount of time in assessing emerging technologies from WAN equipment contributors and aligning them with researcher use cases and connectivity requirements to meet the demanded network bandwidth.

“Once we have a handle on the initial fiber plant design, we then work with our hardware contributors and attempt to line up the best technical solution for the use case,” Rotermund added. “In some cases, a network carrier contributor might be able to provide end-to-end connectivity over their Dense Wavelength Division Multiplexing [DWDM] system. In other scenarios, we collaborate with our hardware contributors to connect into the carrier’s DWDM system using our own transponder modems.”

The World Is Our Stage: SCinet Global WAN Map

Finding the best technological solution often requires outside-the-box thinking. For example, Skaar noted that SCinet uses multiple contributors’ equipment to run “alien waves” across existing Line systems. An alien wave is when a third party uses their own transponder modems to transmit data over a carrier’s DWDM network. 

“This involves using Vendor A’s transponders across Vendor B’s line system and requires a lot of coordinated effort as many times these circuits must be manually configured on both vendors’ equipment,” Skaar described. “This year, we are utilizing ZR optics, which use tunable, optical frequencies in the same spectrum as DWDM to transmit data over shorter distances, under 10 km. This allows us to multiplex or ‘mux’ these wavelengths into our DWDM system from our local Point of Presence [POP] sites to the convention center, reducing the need for extra fiber.”

Testing, Testing, and More Testing

The performance tolerances for operating connections at speeds like 100 Gigabit Ethernet (GE), 400 GE, and beyond are exceptionally tight. The SCinet WAN team dedicates significant effort to collaborate with various network carriers and conducts thorough performance validation and testing throughout the Staging phase and in the days leading up to the SC23 conference. Time is of the essence during this critical period, and the WAN team’s innovative use of technology for remote testing has yielded significant benefits and results.

By employing a micro-electromechanical systems (MEMS) optical switch, the WAN team has the ability to implement dynamic fiber topology adjustments within the convention center, eliminating the need for manual fiber swapping in the field. This innovation enables them to programmatically establish connections between WAN circuits and test equipment. Once the connections have been validated, they can be seamlessly redirected to the appropriate router or switch, streamlining network operations and enhancing efficiency.

“This saves us valuable time of manually repatching fiber jumpers, as well as providing extra optical measurements, which is crucial in troubleshooting,” Skaar added.

Share Your World with Us

Are you seeking hands-on, experiential training in the construction and operation of a wide area network? Join SCinet and learn from some of the most brilliant minds in the field! The SCinet WAN Team includes optical engineers from diverse backgrounds, including R&E, government, industry, and HPC. As Rojas-Torres strongly encouraged, SCinet offers a valuable opportunity for knowledge and skill acquisition in the world of wide area networking. 

“Don’t hesitate: go for it!” he implored. “The friendships, professional networking, and exposure to new technologies you’ll experience while being part of the WAN team will stay with you forever.”

Learn more about how to participate in SCinet for SC24:

Participate in SCinet

More About WANs

What Is a WAN and What Makes It Work?

A Wide Area Network (WAN) encompasses a set of technologies and service provider networks designed to interconnect various Local Area Networks (LANs) over extensive distances. A LAN enables networked devices, e.g., computers, to communicate and share resources within a confined geographical area or under a single administrative entity, such as a school, data center, or business. If you think of the LAN as an isolated island, the WAN acts as the bridge that facilitates the exchange of data and traffic between these distinct islands.

To illustrate with a practical example, consider a scenario where a university located in New York operates its own LAN and seeks to establish a connection with a remote resource hosted in the Cloud, which resides within another LAN situated in Los Angeles. In this instance, the WAN plays a crucial role by facilitating long-distance communication between these two organizations. This communication is achieved using fiber-optic cables as the transmission medium and specialized Dense Wavelength Division Multiplexing (DWDM) equipment, which enables efficient multiplexing and transmission of data over various wavelengths, ensuring reliable connectivity between the two LANs separated by a significant geographical distance.

DWDM and Optical Networking

A device called a transponder modem takes one or multiple network Ethernet connections from a LAN and encodes each connection into a specific frequency range within the visible light spectrum, also referred to as a channel or wavelength. The DWDM equipment functions like a prism, taking these individual wavelengths generated by the transponder and amalgamating them into a composite signal for transmission over a single fiber. The bandwidth capacity of each of these wavelengths varies based on the optical properties of the fiber, such as distance, loss, and reflection. It is common to encounter DWDM equipment that can support up to 80 distinct wavelengths over a single pair of fibers. This capability allows the WAN provider network to pack a substantial amount of network bandwidth over a single fiber efficiently.

After the DWDM equipment transmits the composite signal onto the fiber, signal degradation becomes more pronounced the farther it has to travel. To counter this, amplification equipment is strategically placed along the fiber route to rejuvenate the signal. Eventually, it reaches the DWDM equipment at the opposite end, which then separates the composite signal into its original wavelengths. These wavelengths are received by the transponder modem, which converts them back into individual Ethernet connections for the LAN.

References

¹ https://www.loc.gov/programs/digital-collections-management/about-this-program/frequently-asked-questions/

Unsurpassed Speed

Fiber, Fiber, Fiber

Julie Locke

SCinet Fiber Team Chair

A major part of creating SCinet’s advanced network involves all things fiber—providing it, installing it, repairing and maintaining it, etc. Depending on the conference location, anywhere from 30-70 miles of fiber optic cables need to be installed to support internet connectivity during SC. SC22 required 37 miles of fiber, while SC19—which took place in Denver (same as this year)—needed 66 miles! Providing this level of fiber infrastructure requires an enormous amount of labor. Numerous volunteers are required during set-up week, each of which will work a minimum of 35 hours. Even after installation, maintenance and repair can require a lot of time and effort. When discussing the fiber requirements for SC22, Julie Locke, SCinet Fiber Team Chair, noted that they had at least 21 fiber repairs, some of which had more than one location broken that needed to be spliced. Dealing with this much fiber is certainly a labor-intensive process. But in the end, it’s worth it. The SCinet Fiber Team knows what it takes to support the world’s most powerful network, and they are willing to put in the effort to make it happen.

How Much Power?

Obviously, providing such a powerful network to the 10,000+ attendees of SC involves large levels of power consumption. Jay Harris, SCinet Power Team Co-Chair, broke it down like this: 

“The SCinet NOC (Network Operations Center) over the last several years consumes around 32-35kW of power at any given time. To put that in perspective, in less than 2 hours the NOC uses as much power as a 2000 square foot home would use in an entire 24-hour day, or reversing that, we use about as much power as 12 average-sized homes to power the network.”

Harris went on to note that this figure does not include the Deployable Network Operations Centers (DNOCs), the conference router, or the “Roady Rack,” which collectively consume another 5-8kW of power.

Jay Harris

SCinet Power Team Co-Chair

The Debut of INext

Greg Veldman

SCinet Architecture Team Co-Chair

SC23 is heralding in lots of change, but one of the most exciting new developments is the deployment of Intranet Next or INext. INext is a revision of a primary organizational tool used by SCinet known as the Intranet. This tool tracks many details of SCinet, such as information about pieces of physical equipment and where they should be installed, DNS names, and WAN circuits. Building SCinet without the Intranet (or a similar tool) would be next to impossible. That being said, the Intranet tool had begun showcasing major issues, and so INext was born. Greg Veldman, SCinet Architecture Team Co-Chair and long-time volunteer for SCinet, gives more details on the project:

“For SC23, we’re deploying Intranet Next, or INext. It is a complete rewrite of the Intranet tool from the ground up. The previous generation of this tool was not performing up to current technological standards. One of its major shortcomings was that it had no API nor any good way to interact with it programmatically. In an age of heavy automation, that was starting to present a real problem to the work of planning and building SCinet. It was also code that was more than fifteen years old, written using a framework that was no longer supported, and was starting to become very difficult to maintain after years of changes and updates based on shifting requirements.”

The SCinet team is excited to see how INext performs at SC23. Its implementation has been a multi-year effort involving members from multiple SCinet teams, all of whom are volunteers.

Get Involved with SCinet

SCinet happens thanks to individual volunteers giving their time, and HPC-related companies or institutions contributing materials and services. As a volunteer, you’ll join more than 180 professionals from around the globe working together to design, deliver, install, and operate SCinet at SC. Our team members come from educational institutions, government agencies, high-performance computing sites, research and education networks, equipment vendors, and telecommunications carriers.

As a SCinet contributor, your organization participates by donating equipment, software, or services needed to build and support the network each year for the conference. We invite new and returning contributors to participate in SCinet.

Learn more about how to get involved in SCinet for SC24 and beyond. You can volunteer, contribute, or both!

Participate in SCinet

SCinet Diamond Contributor

SCinet Routing & Cisco NSO

Over the years, the Routing Team at SCinet has piloted different automation solutions to enhance their collective ability to scale and accomplish more in less time. For SC23, the Routing Team has partnered with Cisco to use the NSO orchestrator application. This app helps standardize configurations, simplify the deployment of new devices, and minimize human error.

Corey Eichelberger

SCinet Routing Team Co-Chair

“By leveraging the programmatic interfaces of modern network hardware and automation technologies, the Routing Team can realize efficiencies in how we build and manage the SCinet network,” explained Corey Eichelberger, SCinet Routing Team Co-Chair from the Texas Advanced Computing Center (TACC). “This allows the Routing Team to free up valuable time and resources for our volunteers to work on the more experimental projects that SCinet is known for.”

Kalina Dunn

SCinet Routing Deputy Chair

There is also a skills gap across the industry for network engineers who understand how to write software. Routing Team Deputy Chair Kalina Dunn, from GlobalNOC at Indiana University, explained: “Using technologies like Cisco NSO, we are also looking to provide our volunteers with NetOps experience and training, which are typically hard to come by in the networking industry. By letting them learn and experience how these technologies work, they can bring their experience back to help solve similar challenges their home institutions are facing.”

Cisco NSO Architecture

Cisco NSO is a multi-vendor orchestration platform that provides a single programmatic API (application programming interface) to the network [1].

“Cisco NSO can provide turnkey solutions to automate almost any hardware with an IP address and login, providing the HPC community a multi-vendor tool that will manage different vendors’ nuances of managing devices via the Cisco NED (Network Element Driver),” said Patrick Finnegan, a Network & Security Architect at Cisco and SCinet Routing Team member.

Combining various technologies to address the permutations of vendor equipment is a longstanding challenge for SCinet. Routing Team member Bartosz Drogosiewicz, from ICM University of Warsaw, offered his take on the challenges posed by the multi-vendor issue.

“The industry is at this weird but perfectly understandable point where every vendor tries to have its own automation solution that is only compatible with their own products,” Drogosiewicz said. “If you’re working in a multi-vendor environment, it becomes a challenge to orchestrate all the different devices and services. There’s a lot of little things and quirks that pile up while discovering nooks and crannies of old and new devices. That becomes the hard part: the little things that need to be documented and passed on.”

Bartosz Drogosiewicz

SCinet Routing Team Member

Patrick Finnegan

Cisco Network & Security Architect

An Automation Journey

The SCinet Routing Team has long been a proponent of technologies to simplify management and provide more intelligent networks. While scripting against network elements is a concept that has been applied for decades, the Routing Team began adopting more robust techniques, such as Software Defined Networking (SDN) and OpenFlow, as far back as SC15.

OpenFlow, OpenDaylight, and Ryu (SC15–SC17)

OpenFlow was a transformative step toward adding more intelligence and programmability to the network’s dataplane. This provides more granular control over traffic flows in the network. OpenFlow was first introduced by the Open Networking Foundation in February 2011 [2].

The SCinet Routing Team leveraged the multi-vendor aspect of the SCinet network to test the robustness of equipment support for the OpenFlow protocol. At SC15, the Routing Team deployed the OpenDaylight OpenFlow controller. Network service that once required individual engineers logging into many devices, OpenDaylight could deploy between the core network and the booth at the push of a button [3]. This experiment was continued as part of SC17 using the Ryu SDN controller [4].

Ansible/NAPALM and Faucet (SC18)

Despite OpenFlow being adopted in Research and Education networks, industry support for OpenFlow among hardware vendors declined [5]. The Routing Team began to investigate alternative open-source projects being leveraged for server automation and orchestration. 

During this period, the SCinet network was split between two different automation/orchestration frameworks. The first half of the network services were deployed using a combination of Ansible and NAPALM. The NAPALM library provided a multi-vendor unified API for programmatic network equipment access. The Ansible application provided an orchestration engine and the ability to automate workflows [6].

A model of configuration intent was exported from the SCinet Database (called Intranet). The final piece of this solution was the Flask web container application developed by the Routing Team, which acted as a web user interface front end for Ansible/NAPALM and the SCinet Intranet export script called GenConfig.

The other half of network services were deployed using an OpenFlow controller called Faucet. Despite OpenFlow adoption being on the decline, Faucet provided a unique solution as vendor OpenFlow support must be certified as compatible by the Faucet Foundation. This approach helped ensure consistency of OpenFlow support between hardware vendors. 

Either the hardware is compliant with the required OpenFlow features, or it is not. Faucet also separates the SDN controller function from the operation of the network devices. An operator could update the software on the Faucet controller, and the network will continue operating using its current known good configuration [7].

Cisco NSO (SC19–SC23)

Despite previous success, the multi-vendor aspect of SCinet continued to present challenges for managing configurations and introducing new contributors in the network. Groundwork began in SC19 to build a virtual SCinet R&D network using the GNS3 hypervisor, allowing multi-user topology management. This allowed the Routing Team to expand their R&D cycle for SCinet from a few weeks to year-round. Using network design data from previous SCinet networks, the Routing Team leveraged this environment to pilot configuration management and orchestration with NSO.

Find Your Own “Route” with SCinet

In addition to research and education networks, getting hands-on experience in a multi-vendor network and learning from the brightest minds across educational institutions, government agencies, and HPC sites are among the many reasons volunteers are drawn to SCinet. Drogosiewicz offered his own reasoning about why others may want to join.

“SCinet builds something totally unique,” he added. “I call it ‘The Woodstock of IT.’ It’s a three-day festival with the world’s fastest temporary network built and disassembled within a week. Being a part of the Routing Team means that you’re doing all the different aspects of architecture, design, R&D, configuring, and troubleshooting. Everyone participates in every aspect of it, and it’s a unique chance to broaden your knowledge.”

Learn more about how to participate in SCinet for SC24:

Participate in SCinet

References:

[1] https://developer.cisco.com/docs/nso/guides/#!introduction-to-nso/architecture-overview

[2] https://web.archive.org/web/20110326024026/http://www.opennetworkingfoundation.org/?p=7 

[3] https://www.opendaylight.org/

[4] https://ryu-sdn.org/

[5] https://blog.ipspace.net/2022/05/openflow-still-kicking.html

[6] https://github.com/napalm-automation/napalm-ansible

[7] https://faucet.nz/

An IPv6 Proponent

SCinet has long been a proponent of IPv6, having made IPv6 available to attendees and participants since 2003, and has used the network’s experimental nature to progress the implementation of IPv6 across many network and software platforms over the years.

For SC22, SCinet continued its IPv6 efforts. The SCinet Wireless Team created a dual stacked (providing both IPv4 and IPv6) wireless network for the conference attendees. If you did not notice, you have the expert volunteers of SCinet to thank. 

“For SC22, the Wireless Team enabled IPv6 in the Aruba wireless infrastructure along with RA Guard for added protection,” explained Jeff Hagley of PIER Group, who is Co-Chair of SCinet Wireless/Edge. “This was a massive effort that required close collaboration between the Security/Routing/DevOps/Wireless teams. This collaboration made it possible for clients connecting to the SC wireless network to access the internet using IPv6.”

Trend Conference Traffic by IP Version at SC22

Routing Advertisement Guard (RA Guard) for IPv6 helps block malicious actors from changing another device’s IPv6 configuration when using Stateless Address Auto-Configuration (SLAAC) for its IPv6 address assignment.

Using technologies such as Dynamic Host Configuration Protocol Version 6 (DHCPv6) and Stateless Address Auto-Configuration (SLAAC), modern client devices could request IPv6 addresses if their software stack supported the standard. Results from this effort highlighted that many newer devices and operating systems conform to current standards, preferring IPv6 communications over legacy IPv4 when available.

Filling a Critical Need

Today marks a critical point for IPv6. By the end of fiscal year 2025, the U.S. government (per OMB M-21-07) is requiring all agencies to migrate 80 percent of their assets off of legacy IPv4. An ever-increasing number of Internet-connected devices, government requirements, and significant industry progress have fueled the surge in IPv6 use across the entire Internet. As of July 2023, Google’s IPv6 availability statistics illustrate that global IPv6 usage for its users sits at around 39-43 percent, depending on the weekday (greater on weekends). Adoption between countries remains uneven among Internet service providers. [1]

SC23 SCinet Chair Hans Addleman, from Indiana University, stated IPv6 adoption is a top priority for the SC23 conference network.

“The adoption and implementation of internet standards, like IPv6, fulfill a critical need for networks around the world. SCinet is pleased to be able to show leadership and encourage our volunteers to take these lessons back to their home institutions.”

— Hans Addleman, SC23 SCinet Chair

With this in mind, the SC23 SCinet team is working toward making IPv6 adoption even greater at this year’s SC Conference. Among the goals, SCinet will be implementing an IPv6-only management network and enable DHCP option 108 for wireless. Option 108 leverages capabilities published in the recent RFC 8925. A host requests an IPv4 address from a DHCP server, and the server responds over IPv4 with a DHCP packet that has an Option 108 set.

This option communicates to the host that the network supports IPv6 and to turn off its IPv4 network stack. If the host does not support Option 108, it will ignore the message and request an IPv4 address instead. This allows for IPv6-capable devices to get on the network using IPv6 only, while other devices can continue to operate using IPv4.

Historical Context

The Internet as known today became the information highway with the launch of the World Wide Web and the Web Browser in 1993. However, the foundational work conducted by the U.S. Department of Defense on a network called ARPANET later led to the introduction of the IPv4 communication standard in 1983. During this period, the Internet was a resource predominantly used by government agencies and academics for research collaboration. 

The IPv4 protocol is an open-standards-based set of rules that defines how machines can exchange information on the Internet using a unique identifier called the IP address. Similar to your home address, the IP address identifies where a networked computing device lives on the Internet. While IPv4 has made the Internet go round for decades, it was never intended to scale to the number of users and devices that leverage the information highway today. 

Through enhancements to the IPv4 standard and translation services, such as Network Address Translation (NAT) where an IP address is translated from a private RFC 1918 reserved address to a publicly routable Internet address, network operators have been able to extend the useful life of IPv4. All of these technologies add operational complexity, as well as barriers, to innovation. Further, the urgency to switch networks to IPv6 is increasing. As recently as 2019, the last Regional Internet Registry (RIR) RIPE exhausted its remaining IPv4 addresses.

Why Change Is Good

IPv6 was first introduced by the Internet Engineering Task Force as a standard in 1999. Among its enhancements, IPv6 expands the number of addresses from 4.29 billion (with IPv4) to 340 trillion addresses. Given that, why is IPv4 still the predominant standard in the Internet today? 

Nicholas Buraglio, an SCinet volunteer and the implementation lead at ESnet for IPv6 and compliance with the IPv6-only Federal OMB M-21-07 mandate, explained: “Today, the limiting factors are, by and large, two-fold: lack of resources and lack of knowledge. Lack of those two impediments, coupled with the ubiquitous existence of address translation and its incorrect conflation with security.” 

These limitations also may be aided by the lack of a perceived business driver.

“I think the biggest challenge is getting people to see the value in adopting IPv6, and the capabilities it can bring,” said Shannon Champion, an SCinet Volunteer from PIER Group. “This lack of customer requests has slowed the vendors down in implementing a one-for-one feature parity across the products.”

“The biggest challenge is getting people to see the value in adopting IPv6, and the capabilities it can bring.”

— Shannon Champion, PIER Group

One way to ameliorate this lack of value recognition is to give people an opportunity to use IPv6, which is exactly what SCinet intends to do at SC23. 

“SCinet is making a large push this year for IPv6-only management networks for contributor provided equipment,” Champion added. “This will help to ensure that in the future, this equipment is IPv6 ready. In addition to that, SCinet is providing IPv6 on the user-facing networks to demonstrate it is safe to deploy at home organizations.” 

Setting up IPv6 at SC23 and pushing for its adoption is demanding work, but it is crucial for moving the industry forward. According to Buraglio, “In working with IPv6, SCinet is allowing engineers, especially those in the federal space, to get hands-on IPv6 experience in a production environment. They can then carry that experience back to their home institutions to aid in their IPv6 implementations that further the OMB requirement for moving off of legacy IP.”

Although IPv6 is a solution to the problems facing IPv4, some people may still hesitate to adopt it due to the amount of perceived work involved. Regardless, the tides of change are coming—the federal mandate is in place, and the 2025 deadline will be here in the blink of an eye. A growing number of providers have put their support behind IPv6. Amazon Web Services (AWS) even held its own “IPv6 Day” event in June 2023. Amazon also announced it will begin charging an hourly rate for publicly routable IPv4 addresses in AWS beginning February 2024. [2]

Get Hands-On at SC23

SCinet’s work at SC23 will allow people to get hands-on experience with IPv6 and, hopefully, get the wheels spinning for how to implement it once they are back at their home institutions. For some, the change still may be overwhelming. For those wanting advice on how to get started, Champion has this to offer: “Start off with dual stacked networks across the organization. In addition to that, go into new projects with the mindset of an IPv6 service with legacy IPv4 support. This will make you have the conversation about IPv6 support for the project, and have you justify why to not have it instead of why to have it.” 

To learn more about the SCinet IPv6 implementation, visit SCinet Booth 1081 at SC23 in Denver.

References:

[1] https://www.google.com/intl/en/ipv6/statistics.html#tab=ipv6-adoption
[2] https://aws.amazon.com/blogs/aws/new-aws-public-ipv4-address-charge-public-ip-insights/

Immensely Powerful, insanely fast

This year’s sprint was an expansion of the usual DevOps meeting, including leads from the architecture, routing, network research, and contributor relations subcommittees. It was a chance for the team leads to do hands-on work in preparation for SCinet, collaborating across all of the teams for any updates or services they want to implement. 

With 28 people meeting in Austin and a few more participating remotely, it seemed a bit like organized chaos. But it was for a good cause: the teams were trying to make as much progress as possible toward bringing the IPV6 protocol to SCinet.

New Protocol, More Addresses

Internet Protocol Version 6 (IPV6) is the latest version of the Internet Protocol (IP), designed to replace the current version of the protocol, IPV4. The growing number of Internet-connected devices has caused a shortage of IPV4 addresses, which is why IPV6 was developed. Moving SCinet to IPV6 will increase the number of available addresses for exhibitors and attendees, enabling additional cybersecurity protocols over the network. 

Fortified with burgers, Texas BBQ, and donuts, the teams got to work, tackling specific projects in their areas and pushing to ensure they are prepared for any challenges that may arise in November. 

SC23 General Chair Dorian Arnold (center right) also briefly joined Hans Addleman (center left) and the teams, sharing his vision and meaning of this year’s conference theme, “I Am HPC.”

Laying the Groundwork

SCinet Chair Hans Addleman and Vice Chair Nathaniel Mendoza expanded the sprint this year to build camaraderie among the teams and avoid the last-minute panic that can occur in the days and weeks leading up to SC.

“If we lay the groundwork in advance, we might be able to avoid some rough patches in October,” said Addleman. “The teams shared ideas and expertise, which could get us to more efficient problem-solving and a better end result.”

Discover SCinet

Created in 2015, WINS seeks to address the gender gap within the information technology industry, particularly in network engineering. Each year WINS funds highly qualified women, selected through a competitive application process, to join the conference’s SCinet team. SCinet is a global multi-terabit network created each year for the SC Conference and built from the ground up to support the revolutionary participant demos that have become the hallmark of the high-tech conference.

WINS strives to have a diverse representation of organizations and applicant backgrounds in the cohort each year. WINS continues to receive highly competitive applications, and the program continues to show strong interest and support.

SC23 WINS Awardees

The WINS awardees are deeply immersed in a six- to nine-month engineering experience before SC and during the conference while SCinet is operating live. Awardees are integrated into one of over fifteen SCinet teams and are paired with a mentor to ensure a positive experience throughout the process.

Brittany Huff

Purdue University

Team: SCinet Network Security

Shannen McKenna

University of California, Davis

Team: SCinet Network Security

Hallie Mull

Bucknell University

Team: SCinet Routing

Suzanne Prentice

Oak Ridge National Laboratory

Team: SCinet DevOps

Pauline Sawadogo

Clemson University

Team: SCinet Edge

Alysha Toya

Jemez Health and Human Services

Team: SCinet Edge

Ola Gamal

CENIC

Team: SCinet WAN

“WINS helps to address the gender gap in information technology, especially in networking technology and high performance computing (HPC). Through WINS, the talented group of early to mid-career women engineers and technologists have a unique opportunity for direct access to cutting-edge network technology, working side-by-side with the SCinet volunteers, including the world’s leading network and software engineers from academia, government, and industry, to design and create SCinet. The women who participate in WINS are empowered to thrive and achieve success in their chosen career.”

From Awardee to Chair

When WINS was created in 2015, less than 14% of SCinet volunteers were women. Since then, that amount has tripled with a significant number of alumni ascending to SCinet management roles, such as Angie Asmus from Colorado State University.

As one of the original WINS awardees, Asmus was selected among peers to serve as the SC24 Chair of SCinet, the top position a volunteer can hold on the SCinet committee. “WINS was initially started to close the gender gap, but it has become so much more.  The opportunity WINS has given me and other women involved in SCinet has been pivotal in our careers.  It has provided hands-on networking experience, public-speaking practice, a trusted community, and leadership opportunities that have opened the doors for many of us to advance in our careers. Seeing WINS participants grow and develop into leaders in SCinet and their home institutions is exciting.  I would not have had the career advancement or the opportunity to lead the great SCinet organization without my experience in WINS.  I’m excited to lead all of the SCinet volunteers, especially the WINS participants and alumni at SC24.”

In addition to the SCinet experience, WINS provides ongoing support and career development opportunities for awardees before, during, and after the conference. This includes monthly calls for the community of WINS alumni participants, which now tops 45 women.

A Joint Effort

WINS is a joint effort between the University Corporation for Atmospheric Research (UCAR), the Department of Energy’s Energy Sciences Network (ESnet), and Indiana University. To date, financial support for the WINS program has come from a combination of funding, with the original core funding from the NSF (144064, 1640987, and 2129723) and additional funding from the joint partners as well as other contributors, including NREL, Internet2, Ciena, Meta, Juniper, KINBER, Pacific Northwest GigaPoP (PNWGP), Northern Tier Network Consortium (NTNC), The Quilt, and Purdue University. 

Members of the WINS management team include Marla Meehl and Scott Baily of UCAR, Kate Robinson and Jason Zurawski of ESnet, Brenna Meade from Indiana University, and Wendy Huntoon of AIHEC.

Abstracts Due June 2

Incentives for Participation

Ready to Submit?

Dispatch from the Desert

Regina Hartman, of Juniper Networks has participated in SCinet for the past six years.

Hartman shares “This is my first year participating in the inventory portion of SCinet” Hartman explained. “We tested fiber and organized our testing tools in preparation for a successful SC23.”

As part of this year’s inventory, more than 100 network switches; 300 wireless access points (and tripods); thousands of power cords; miles of fiber optic cables, power equipment, and equipment racks; and all of the necessary related tools were sorted, documented, cleaned, and neatly placed in their respective crates. Now, they are ready to be shipped to Denver in October ahead of SC23. Brian Smith, of Cornelis Networks, is serving as co-team lead for the SCinet Logistics and Contributor Relations teams, and is responsible for inventory, staging, setup, and teardown. This year also marked Smith’s first SCinet inventory. Wesley DeWitt, the team co-chair also from Cornelis Networks, is working with Smith. 

“During the inventory-stage process, I make sure that everything is accounted for and in a proper working state,” DeWItt explained.

Assuring the equipment is operational is especially important as there is little  relief from the triple-digit heat at  Freeman’s Mojave Desert storage facility. Every year, some of SCinet’s 106 equipment crates get left behind in the storage warehouse because they may only be needed to support certain conference centers.

“We also determine what gets shipped and when,” DeWitt added. “This year, we got rid of a lot of deprecated hardware that we decided to donate and recycle.”

In addition, SCinet tries to reduce their carbon footprint by shipping only the essentials each year. For SC23, five semi trucks will be needed to deliver SCinet equipment to Denver. Notably, while reusing as much as possible, SCinet recycled four pallets of e-waste, consisting of items such as switches, fiber, and old computers. These items may no longer be cutting-edge equipment that can operate the world’s fastest temporary network, but local schools often appreciate receiving the equipment, which can provide an upgrade for their facilities. SCinet and the SC Conference have long made such donations, which has created an avenue for fellowship and further promotes high-performance computing.

The Heart of Network Planning

Inventory is not just an exercise of counting and cleaning, it is also where the heart of the network planning starts to take shape. SC23 SCinet Chair Hans Addleman, of Indiana University, decided to perform inventory in the winter months to better view the hardware needs and initiate the creative process for booth design with Regina Martin from Freeman. Martin has been part of the SC Conference exhibits and infrastructure since 2007. According to Addleman, as SCinet is becoming more and more complex, it is better to start the process earlier.

In addition, having many team leaders represented during the inventory process enabled conversations toward new ideas. By bringing SCinet, Steering, and Executive team members together, they were able to draft the Network Operations Center (NOC) layout, sort the wireless network diagrams, plan the fiber optic cable routing, determine power needs for the equipment, and most importantly document and share the location of all of the equipment. With this informed pre-planning, every item can be easily found by any member of SCinet via a simple spreadsheet search.

“Back together again, the SCinet team worked together to get the job done,” Hartman concluded.

A Global Collaboration

Powered by People

During the week of SC, SCinet becomes the most powerful and advanced temporary network on Earth, connecting the SC community to the world. Every year, industry, academia, and government experts volunteer to design, build, and administer the cutting-edge SCinet infrastructure. SCinet not only provides internet connectivity for conference attendees, it pushes the boundaries of networking technologies and innovations through workshops, demos, and collaborative opportunities.

Over the years, thousands of people from the HPC community have participated in SCinet. Most recently, 175 dedicated and determined volunteers from academia, government agencies, and industry representing 5 countries, 30 states, and 78 organizations performed this tremendous undertaking. SCinet was supported by 29 contributing organizations who loaned software, services, and hardware valued at $70 million.

SCinet is also a mentoring and networking incubator for its volunteers, offering leadership training to senior-level participants. SCinet supported 20 student volunteers, including 2 lead student volunteers, and 6 WINS (Women in IT Networking at SC) awardees. Their collaboration not only generates personal growth that benefits participants’ home institutions, but also results in innovation which enables exhibitors and researchers at SC to demonstrate what is possible when they have access to cutting-edge resources.

A Colossal Achievement

SCinet takes two years to plan, one month to build, one week to run, and one day to tear down.

Five semi trailers transported 231 pallets of equipment to the convention center. SCinet deployed 12.77 miles of fiber, 10,400 pounds of WiFi 6E-enabled wireless equipment, 3,410 fiber patches, and 102 exhibitor booth connections, all of which consumed 32.0 kilowatts of power.

In order to achieve the installation, some volunteers averaged 30,000 steps a day during the week of SC!

Aggregate WAN Traffic as Measured at SC22

Global Collaboration