Subscribe to Blog Notification Emails

Latest Blog Post

Scalable Techniques for Multi-terabit Cloud Networking

Jayshree Ullal
by Jayshree Ullal on Nov 2, 2009 2:48:34 AM

Current data center network designs are very compromised. They are based on blocking multi-tier designs with 50:1 over-subscription using the Spanning Tree Protocol (STP) to prevent loops and enable device redundancy. Link Aggregation, LAG, was introduced to distribute network traffic across multiple links for more capacity. This made scalable bandwidth and device redundancy mutually exclusive. With previous architectures, flexible designs for new cloud applications such as map-reduce,Hadoop clusters, Video/CDN, and HPC applications were price and performance prohibitive.

Introducing MLAG for cross-sectional bandwidth capacity

Arista’s introduction of Multi-chassis Link Aggregation (MLAG) is an innovative and profound development in cloud networking to offer fully non-blocking scalable two-tier designs in the data center.

Multi-chassis Link Aggregation (MLAG) can be best described as a set of ports spread across two cooperating switches (known as "peer switches") that behave like an ordinary link aggregation group. The beauty of MLAG is the creation of large layer 2-based domains that offer simplicity, reliability and vastly improved capacity for intra-machine communications

MLAG overcomes the inherent limitations of legacy Spanning Tree mechanisms enabling “active-active” multi-path topologies with multiple uplinks. This enables very large Layer-2 networks to better support virtualization or high performance compute (HPC) cluster applications and also overlays Layer 3 options.

A real world case

Arista customers are deploying elegant topologies of MLAG in a rack server or blade server with built in blade switches running HPC applications. In this figure below, multiple blade server enclosures with integrated switches in each chassis, interconnected via 10 Gigabit Ethernet, are depicted as the “Switch 3” element. Each Switch 3 element can be connected with multiple uplinks for scalable server connectivity. The transparent connection of these links across a pair of upstream Arista 7148SX (cooperating peers Switch 1 and 2) ensures that the servers are not only guaranteed a maximum upstream bandwidth of up to 160Gb/s but also built as a two-tier topology devoid of manual and complex configurations. This creates an elegant design that is fully non-blocking and scales to thousands of ports.

MLAG Design

MLAG Design: Switch 3 is Blade Server + integrated Switch connected via MLAG to Switches 1 and 2 as peer Arista 7148SXs

MLAG achieves multi-terabit capacity making maximum use of each device and each link. It is an important tool in the network architect’s toolbox for building highly scalable and resilient cloud networks. I am excited about innovations like MLAG as they can be deployed on existing standards-based 1/10GbE switches without inventing proprietary mechanisms. At the same time MLAG options can co-exist with future standards for interoperability such as TRILL via IEEE/IETF/ ANSI committees. As always I welcome your comments at feedback@arista.com.

References:

Hadoop Cluster Applications - Technical Bulletin
Multi-Chassis Link Aggregation - Technical Bulletin
A Scalable Flexible Data Center Network

Opinions expressed here are the personal opinions of the original authors, not of Arista Networks. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Arista Networks or any other party.
Jayshree Ullal
Written by Jayshree Ullal
As CEO and Chairperson of Arista, Jayshree Ullal is responsible for Arista's business and thought leadership in AI and cloud networking. She led the company to a historic and successful IPO in June 2014 from zero to a multibillion-dollar business. Formerly Jayshree was Senior Vice President at Cisco, responsible for a $10B business in datacenter, switching and services. With more than 40 years of networking experience, she is the recipient of numerous awards including E&Y's "Entrepreneur of the Year" in 2015, Barron's "World's Best CEOs" in 2018 and one of Fortune's "Top 20 Business persons" in 2019. Jayshree holds a B.S. in Engineering (Electrical) and an M.S. degree in engineering management. She is a recipient of the SFSU and SCU Distinguished Alumni Awards in 2013 and 2016.

Related posts

The New AI Era: Networking for AI and AI for Networking*

As we all recover from NVIDIA’s exhilarating GTC 2024 in San Jose last week, AI state-of-the-art news seems fast and furious....

Jayshree Ullal
By Jayshree Ullal - March 25, 2024
The Arrival of Open AI Networking

Recently I attended the 50th golden anniversary of Ethernet at the Computer History Museum. It was a reminder of how familiar...

Jayshree Ullal
By Jayshree Ullal - July 19, 2023
Network Identity Redefined for Zero Trust Enterprises

The perimeter of networks is changing and collapsing. In a zero trust network, no one and no thing is trusted from inside or...

Jayshree Ullal
By Jayshree Ullal - April 24, 2023