The evolution of AI interconnects - Marvell Technology

Skyler: Welcome to Line Rate — the show tracking the chips, standards, and companies shaping datacenter fabric and A I high-speed interconnect. I'm Skyler Monroe, and if you care about what moves data between GPUs, across racks, or between datacenters at scale, you're in the right place. Big thanks to our sponsors: Ada, which helps organizations actually integrate A I into their real workflows, and Zen Semiconductor, an umbrella company with a portfolio of businesses spanning the silicon industry — including Ada for A I deployment. Today we've got Marvell on the evolution of A I interconnects, Elon Musk going all-in on mesh optical, Cisco making big datacenter promises, Ayar Labs and Wiwynn teaming up on co-packaged optics, and QTS tapping Lumen for next-gen A I connectivity. Let's get into it. Skyler: First up — Marvell Technology dropped a piece on the evolution of A I interconnects, and while it's partly marketing, there's real signal in what they're saying and how they're framing the problem. So let's unpack it. The core argument Marvell is making is that A I workloads have fundamentally broken the assumptions that traditional datacenter networking was built on. And they're right. Classical Ethernet and InfiniBand were designed for a world where east-west traffic was bursty, latency tolerances were measured in microseconds, and you had thousands of relatively independent compute nodes. A I training flips all of that. You've got massive, highly synchronized collective operations — all-reduce, all-gather, all-to-all — running across thousands of GPUs simultaneously, where a single straggler link can stall the entire job. The fabric is no longer just moving data. It's participating in the computation. Marvell's angle here is that the interconnect stack needs to be rethought at every layer — physical, electrical, and protocol — to match these new demands. They're specifically calling out the gap between what 400G and 800G Ethernet can deliver today and what frontier model training actually needs in terms of bandwidth, latency, and congestion behavior. One of the more interesting points they make is about congestion control. Traditional TCP-based congestion control is too slow and too coarse for A I collective traffic. You need mechanisms that react in nanoseconds, not milliseconds, and that understand the collective communication pattern — not just point-to-point flows. This is where things like DCQCN, RDMA, and now the emerging U E C stack come in, and Marvell is positioning its silicon as native to all of these. They're also talking about scale-up versus scale-out as distinct problem domains. Scale-up — connecting GPUs within a node or rack — is increasingly being handled by proprietary fabrics like NVLink. But scale-out — connecting racks, pods, and clusters — is where Marvell sees its opportunity, supplying the custom A S I C silicon that hyperscalers are using to build their own switching fabrics rather than buying merchant silicon from Broadcom. That's a significant strategic position. Marvell has been quietly winning custom silicon engagements with the major cloud providers, and pieces like this are part of how they signal to the market that they understand A I workload requirements at a deep level. The competitive context here is real. Broadcom dominates merchant switching silicon, but the hyperscalers increasingly want to differentiate their fabrics, and that means custom. Marvell is chasing that custom business hard, and the A I interconnect narrative is central to their pitch. Worth watching. Skyler: Next story — and this one has a lot of layers. Semivision published a deep dive on what they're calling Elon Musk's mesh optical deal, framing it as the opening shot in the next A I infrastructure war — one that will be fought with light. So what's actually going on here? The background is that xAI, Musk's A I company, has been building out Colossus, its massive GPU cluster in Memphis, at a pace that's pretty unprecedented. We're talking about going from zero to over a hundred thousand H100s in a matter of months. To do that, you need fabric that can scale just as fast, and xAI apparently made a significant move toward free-space optical mesh networking — essentially using directed laser beams through air to connect racks and pods without laying fiber between every endpoint. The semivision piece frames this as strategically significant for a few reasons. One, free-space optical, or FSO, eliminates the cabling bottleneck that plagues dense GPU clusters. When you're standing up tens of thousands of GPUs at speed, the physical act of running fiber is a real constraint. Optical wireless gives you reconfigurable, high-bandwidth links that you can point and shoot. Two, mesh optical topologies map well onto A I collective communication patterns. Instead of a fat-tree switching hierarchy where every packet has to traverse multiple hops, a mesh can give you more direct paths between communicating nodes, which matters for latency-sensitive collective operations. Three — and this is the competitive angle — if xAI can make FSO work at scale, it gives them a fabric architecture that others can't easily copy, because it depends on tight integration between the physical infrastructure, the optical hardware, and the network control plane. Now, there are real engineering challenges here. FSO is sensitive to physical alignment, thermal drift, and environmental factors inside datacenters — even though you don't have weather to contend with indoors, you have vibration, airflow, and the fact that racks move. Making this reliable at scale is non-trivial. But the broader point the semivision piece is making — that the next differentiation in A I compute isn't just GPU count, it's fabric architecture — that's absolutely correct. The companies that figure out how to move data between GPUs faster and more efficiently than their competitors will have a structural advantage in training large models, and that advantage compounds over time. Whether FSO is the answer or one of several answers, the direction of travel is clear: we're moving toward more photons and fewer electrons in the A I fabric, and the infrastructure war is very much underway. Skyler: Alright, story three — Cisco posted a blog titled "Building Data Center Infrastructure for the A I Revolution," and look, Cisco blogs are always going to have some amount of vendor positioning, but there's substance here worth engaging with seriously, because Cisco is making some specific architectural bets that tell you where they think the market is going. The headline claim is that A I is driving a fundamental redesign of datacenter infrastructure, not just an upgrade cycle. Cisco's argument is that the combination of GPU density, power requirements, and east-west bandwidth demands means that the old three-tier architecture — access, aggregation, core — is inadequate for modern A I workloads. They're pushing a flatter, spine-leaf topology built around high-radix switches, which is not a new idea, but Cisco is now specifically tying it to A I fabric requirements in a way that's more direct than their previous messaging. What's interesting is their position on Ethernet for A I. Cisco is firmly in the camp that Ethernet — specifically 400G and 800G Ethernet with proper RDMA and congestion management — is the right foundation for A I scale-out fabric, as opposed to InfiniBand. They're a founding member of the Ultra Ethernet Consortium, so this isn't surprising, but the blog doubles down on U E C as the path forward for open, interoperable A I networking. The U E C spec is still evolving — we're talking about a working group that's been active for a couple of years now, with the transport layer spec still being refined — so Cisco is making a forward-looking bet here. They also touch on the optics layer, which is relevant given the other stories we're covering today. Cisco has been investing in pluggable and co-packaged optical solutions, and the blog frames optics as integral to their A I fabric story, not just an add-on. That's meaningful because it signals they understand that the electrical-optical boundary is moving closer to the A S I C. On the competitive side, Cisco is in an interesting spot. They're not a GPU vendor, they're not building custom silicon for hyperscalers the way Marvell is, but they have massive installed base and relationships with enterprise and mid-market customers who are building out A I infrastructure for the first time. For those customers, the Cisco narrative — buy a validated, integrated stack from a vendor you already trust — is compelling. The question is whether that's enough to compete against the hyperscaler-designed fabrics that are increasingly setting the performance bar. This is the tension Cisco has to navigate, and this blog is part of how they're trying to do it. Skyler: Story four — and this one is closer to my heart because it's squarely about the physics of where the interconnect is heading. Ayar Labs and Wiwynn announced a partnership to develop co-packaged optical A I infrastructure. Let's talk about why this matters. Ayar Labs is one of the most interesting companies in the interconnect space right now. Their core technology is silicon photonics integrated directly into the compute package — not a pluggable transceiver hanging off the edge of a switch or a NIC, but optical I/O built into the same package as the A S I C. The promise is significant: you eliminate the electrical SerDes runs between the chip and the optical module, which is where a huge amount of power and signal integrity loss occurs at high data rates. With co-packaged optics, or CPO, you get lower power per bit, higher bandwidth density, and potentially lower latency compared to pluggable optics. Wiwynn is a major ODM — original design manufacturer — that builds servers and rack infrastructure for hyperscalers. Their partnership with Ayar is about integrating Ayar's optical I/O technology into actual production-ready rack designs, which is a necessary step to move CPO from lab demo to real deployment. This is a critical juncture for CPO. The technology has been discussed for years, and companies like Intel, Broadcom, and Marvell have all been investing in it, but getting it into production hardware is hard. You have to solve thermal management, reliability, and yield issues, and you have to build the supply chain. The Wiwynn partnership suggests Ayar is serious about the production path, not just the roadmap. The A I angle here is direct. As GPU clusters scale to hundreds of thousands of accelerators, the bandwidth requirements at the fabric level are staggering. Pluggable optics at 800G are available today, 1.6T is coming, but the power consumption of dense pluggable modules at those rates starts to become a real constraint inside a rack. CPO is potentially the answer to the bandwidth-per-watt problem at scale. Ayar specifically has an interesting architecture — their TeraPHY chiplet uses an external continuous-wave laser source rather than integrating the laser onto the silicon, which sidesteps one of the hardest problems in silicon photonics — getting reliable, efficient lasers on silicon. It's a practical engineering choice that makes their near-term production path more realistic. For the industry, this partnership is a signal that CPO is moving from concept to infrastructure. Watch for more ODM and hyperscaler engagements as the technology matures. Skyler: Last story of the day — QTS Realty Trust, which is one of the larger colocation and hyperscale data center operators, announced that they're tapping Lumen Technologies to connect what they're calling the next generation of A I datacenters. This is interesting for a few reasons, and it's a different layer of the stack than we've been talking about — this is about the wide-area network connectivity between datacenter facilities, not the fabric inside the datacenter. QTS has been on an aggressive expansion path, building out large campuses in multiple markets specifically to serve A I workloads. The challenge they face, and every large colo operator faces, is that A I training and inference increasingly span multiple facilities. You might have your primary GPU cluster in one campus, your storage in another, and your inference endpoints distributed across a third set of locations. Connecting all of that with enough bandwidth and low enough latency to make the distributed system behave coherently is a real networking problem. That's where Lumen comes in. Lumen has one of the largest fiber networks in North America, and they've been pivoting their business toward providing high-bandwidth, low-latency connectivity specifically for A I and cloud workloads. The QTS deal is about provisioning dedicated, high-capacity dark fiber or wavelength services between QTS campuses — essentially giving QTS and their customers a private wide-area fabric rather than relying on public internet or shared carrier capacity. For the A I use case, this matters because large-scale distributed training, or even just moving multi-petabyte datasets between facilities for preprocessing and training, requires sustained, predictable bandwidth at a level that commodity internet connections can't reliably provide. The latency requirements vary — for training, you generally need very high bandwidth but can tolerate somewhat more latency than the in-datacenter fabric. But for distributed inference, where you're serving live requests across geographically distributed model shards, latency becomes much more important. This deal is part of a broader trend of A I operators building private, dedicated wide-area networks — essentially their own WAN fabric — to support distributed A I workloads. The hyperscalers have been doing this for years with their own submarine cables and metro fiber rings. Now the colocation and cloud-adjacent operators are following suit by partnering with carriers like Lumen. It's the interconnect story scaling out to the geographic level, and it's a growing piece of the A I infrastructure picture. Skyler: Alright, that's a wrap on today's Line Rate. From Marvell's interconnect thesis to Musk going optical, Cisco's A I infrastructure play, Ayar Labs bringing co-packaged optics closer to production, and QTS building out private WAN fabric with Lumen — the theme across all of it is the same: A I is forcing every layer of the connectivity stack to be rethought, from the package level all the way to the wide-area network. None of this is moving slowly. Line Rate, brought to you by Ada and Zen Semiconductor — Ada helps organizations integrate A I into their real workflows, and Zen Semiconductor is an umbrella company with a portfolio across the silicon industry. If you found this useful, subscribe wherever you get your podcasts, share it with a colleague who's deep in the stack, and check the show notes for links to all the sources we covered today. See you next time.