Multiple AI Scale-Up Options Emerge - Semiconductor Engineering

Skyler: Hey, welcome to Line Rate — the show about datacenter fabric and A I high-speed interconnect. I'm Skyler Monroe, and if you care about the chips and standards moving data between accelerators, across racks, and between datacenters, this is your show. Big thanks to our sponsors, Ada, who helps clients integrate A I into their workflows, and Zen Semiconductor, an umbrella company with a portfolio of businesses across the silicon industry. Alright, we've got a packed episode today — scale-up interconnect is fragmenting fast, NVLink Fusion just got a massive new partner, and the U A Link consortium is opening its doors. Let's get into it. Skyler: First up, let's talk about the broader landscape of A I scale-up options, because Semiconductor Engineering just ran a piece that captures something I've been watching closely — the fact that we're no longer in a world where there's one obvious answer to how you connect accelerators inside a pod or a rack. A year ago, you could pretty much say NVLink for NVIDIA, and everything else is still figuring it out. That's not true anymore. The space has fractured, and in a lot of ways that's a good sign for the ecosystem. Skyler: So what are the actual options on the table? You've got NVLink, which is proprietary to NVIDIA and extremely high bandwidth — we're talking hundreds of gigabytes per second per GPU in an NVLink domain. Then you've got CXL, which is more of a coherent memory fabric play — it's found its home in CPU-to-accelerator and memory pooling scenarios more than pure scale-up GPU training. You've got PCIe, which is the universal baseline but not really a scale-up fabric — it's the floor, not the ceiling. And then you've got the open contenders: Ultra Ethernet Consortium, or U E C, and U A Link, which is the AMD-led open scale-up fabric. Skyler: What's interesting about the Semiconductor Engineering piece is the framing — multiple options are emerging not just because the market wants competition, but because different workloads genuinely have different requirements. Inference at scale looks different from training. A dense GPU pod looks different from a disaggregated architecture with a mix of accelerator types. So the pressure on the market isn't just "beat NVLink" — it's "fit the workload." Skyler: The piece also touches on something that doesn't get enough attention, which is the latency sensitivity of scale-up versus scale-out. Scale-up — connecting accelerators tightly inside a domain — demands ultra-low latency and extremely high bandwidth. That's where NVLink has an enormous head start. Scale-out is where Ethernet-based solutions like U E C are more natural — it's the network between pods, racks, datacenters. The blurring of those two categories is where things get really interesting. Skyler: For architects and engineers designing systems today, the key takeaway is that the answer to "what interconnect do I use" is genuinely becoming workload-dependent and vendor-dependent in a more nuanced way than before. If you're all-in on NVIDIA silicon, NVLink remains hard to beat for scale-up. But the moment you want flexibility, open interfaces, or a multi-vendor environment, the options are maturing fast. We'll see more of that in the next few stories. Skyler: Next, Nokia dropped a piece on optimizing datacenter networking for A I workloads, and I want to dig into this because Nokia's perspective here is interesting — they're coming at it from the network fabric side, not the accelerator side, and that gives them a different lens on the problem. Skyler: The core argument Nokia is making is that conventional datacenter networking — the stuff optimized for cloud applications and web traffic — isn't built for A I training workloads, and that gap matters more than most people realize. Traditional traffic is bursty in a different way. You have lots of small flows, lots of short TCP connections, elephant flows mixed with mice flows. A I training is dominated by collective communications — all-reduce, all-gather, broadcast operations that synchronize gradients across thousands of GPUs. That creates a traffic pattern that's much more synchronized, much more uniform, and far less tolerant of jitter. Skyler: Nokia's framing around this centers on a few key pain points. Congestion management is a big one. In an A I fabric, you cannot afford tail latency the way you can in a general-purpose cloud network. If one GPU stalls waiting for a gradient update that's sitting in a congested buffer somewhere, the whole training step is stalled. That's the "straggler" problem, and it kills efficiency. They're arguing — correctly, I think — that you need network-aware congestion control tuned specifically to collective comm patterns. Skyler: The other thing Nokia highlights is the importance of lossless or near-lossless transport. Retransmissions in an A I fabric aren't just inefficient — they destroy the synchronization properties that make collective operations work efficiently. This is exactly why RDMA over Converged Ethernet has been so important in this space, and why protocols like RoCEv2 paired with Priority Flow Control have become standard in serious A I deployments. Skyler: Now, Nokia isn't just doing generic commentary here — they have skin in the game with their IP routing and optical transport portfolio. They're pitching a vision where the WAN layer and the datacenter fabric layer are co-optimized, which makes sense if you're running distributed training across multiple datacenter sites. That's increasingly relevant as models get bigger and training runs span multiple physical locations. Skyler: The practical upshot for datacenter architects: the network is not a passive pipe in an A I cluster. It's an active participant in compute efficiency. Getting the fabric right — topology, congestion control, transport protocol — can meaningfully change your model flops utilization numbers. Nokia's piece is a vendor pitch, but the technical substrate is real and worth understanding. Skyler: Alright, Marvell published a piece on the evolution of A I interconnects, and this one is worth spending some time on because Marvell sits in a genuinely interesting position — they're a silicon vendor building custom A I silicon for hyperscalers, and they're also deep in the optical DSP and SerDes space. So when they talk about interconnect evolution, they're not just doing market commentary, they're describing the environment their chips have to live in. Skyler: The throughline of their piece is that interconnect requirements are scaling faster than traditional networking roadmaps can accommodate. That's not a new observation, but the specifics matter. We're moving from 400G to 800G to 1.6T per port speeds in the datacenter, and the cadence is accelerating, not slowing. The SerDes rates underpinning those speeds — we're at 112 gigabaud PAM4 today, moving to 224 gigabaud, and the signal integrity challenges at those speeds are serious. Skyler: Marvell's angle is that the transition from electrical to optical — particularly the shift toward co-packaged optics — is becoming less of a future option and more of a near-term necessity. At very high lane rates, the copper reach starts to shrink to the point where pluggable optics at the faceplate can't get the signals cleanly from the switch ASIC to the connector. Co-packaged optics puts the optical engine right next to the switch ASIC, dramatically reducing the electrical channel length and enabling much higher aggregate bandwidth. Skyler: They're also talking about the role of custom silicon — what they call XPUs, essentially customer-specific A I accelerators — and how the interconnect requirements for those chips drive design decisions all the way down to the die-to-die interface. If you're building a custom A I accelerator for a hyperscaler, you have to design the chip with the fabric in mind. You can't treat the network interface as an afterthought. Skyler: One thing Marvell is careful to highlight is the disaggregation trend — the idea that compute, memory, and networking are increasingly being designed as separable, composable elements rather than tightly integrated systems. That plays to their strengths as a merchant silicon supplier. If hyperscalers want to mix and match, they need chips from vendors who can deliver high-performance, standards-compliant silicon at scale. Skyler: The interconnect evolution framing from Marvell is a useful reminder that what we're watching isn't just a standards fight — it's a physics fight. The companies that figure out how to move bits at the required speed and reach, with acceptable power and cost, are the ones that win. And right now, the entire industry is racing hard on that front. Skyler: Okay, this is the big one this week — NVIDIA announced that Arm is joining as an NVLink Fusion partner, and this is a genuinely significant development that deserves some real unpacking. Skyler: So, NVLink Fusion. Let's start with what it is. NVIDIA announced NVLink Fusion earlier this year as a way to allow third-party chips — meaning non-NVIDIA silicon — to connect into an NVLink fabric. The pitch is that you can build custom C P U s, custom accelerators, custom A S I C s, and connect them into an NVLink domain with NVIDIA GPUs, getting the full benefit of NVLink's high bandwidth and low latency without having to use a traditional PCIe or Ethernet interface. It's NVIDIA opening up the interconnect — selectively, under their terms — to the broader ecosystem. Skyler: Now, Arm as a partner here is a big deal. Arm is the foundation of almost every custom silicon effort in the A I space right now. When hyperscalers build custom A I chips — whether it's Google's TPU successors, Amazon's Trainium and Inferentia, or any number of startups — a huge fraction of them are building on Arm C P U cores. If Arm's interconnect IP is designed to work natively with NVLink Fusion, that means any chip designer building on Arm can more easily interface with NVIDIA's GPU ecosystem. Skyler: The strategic read here is nuanced. On one hand, this looks like NVIDIA being generous and open. On the other hand, this is NVIDIA making NVLink the gravitational center of the A I compute ecosystem. If your custom silicon connects via NVLink Fusion, you're now dependent on NVIDIA's interconnect roadmap, NVIDIA's fabric products, NVIDIA's pricing. You've chosen to orbit NVIDIA rather than build an independent stack. Skyler: For Arm, the partnership makes sense commercially — their customers want to build chips that can interoperate with NVIDIA GPUs, and certifying NVLink Fusion support in Arm's interconnect IP makes that easier. But there's a tension here. Arm also has customers who are explicitly trying to build NVIDIA alternatives, and those customers might not love the idea of Arm getting cozy with NVIDIA's interconnect. Skyler: For the U E C and U A Link ecosystems, this is a competitive signal worth paying attention to. Every design win NVIDIA gets for NVLink Fusion is a design that won't go to open interconnect standards. The counter-argument from the open camp is that NVLink Fusion still requires NVIDIA GPUs somewhere in the topology — you can't build a pure NVLink Fusion cluster without NVIDIA silicon. Whether that's a dealbreaker depends on what you're building. For hyperscalers who want to avoid NVIDIA lock-in entirely, open standards remain the play. For everyone else who's going to have NVIDIA GPUs in the mix anyway, NVLink Fusion is increasingly compelling. Skyler: Finally, let's talk about U A Link, because the consortium just announced they're opening up membership, inviting new members to help shape the standard. And this is an important moment in the trajectory of what's shaping up to be the most serious open alternative to NVLink in the scale-up space. Skyler: Quick background for anyone who needs it: U A Link — Ultra Accelerator Link — was founded in 2023 with AMD as the lead, and founding members including Intel, Google, Microsoft, Meta, Qualcomm, and others. The goal is to define an open, vendor-neutral high-bandwidth low-latency interconnect for scale-up — specifically designed to connect A I accelerators within a pod or rack domain, which is exactly where NVLink dominates today. Skyler: U A Link 1.0 targets 1000 gigabytes per second per port, which is genuinely competitive in the scale-up bandwidth tier. And crucially, it's being designed from the ground up for collective communications — all-reduce, all-gather, the operations that dominate A I training. That's not an accident. The founding consortium understood that you can't just repurpose a general-purpose interconnect for this workload. Skyler: The decision to open membership is a strategic move, and it's the right one. U A Link needs broad industry participation to become a real standard. The value of an open standard comes from the ecosystem around it — the chips that implement it, the system vendors that design for it, the software stacks that optimize for it. The more companies have a stake in U A Link succeeding, the more likely it is to reach critical mass. Skyler: There's also a timing element here. The NVLink Fusion announcement with Arm puts pressure on U A Link to accelerate. If the narrative becomes "NVLink is open enough because Fusion exists," U A Link has a harder hill to climb. The open camp needs to demonstrate that their approach is genuinely more flexible and vendor-neutral, and that requires a broad and diverse membership roster. Skyler: What I'll be watching: who joins the consortium in this next wave. If you see major A I accelerator startups — the Cerebras, Groqs, SambaNova types — signing on, that's a signal the startup ecosystem sees U A Link as viable. If you see more hyperscalers committing, that's a signal U A Link is on the procurement roadmap. And if you see major system vendors, the ODMs and OEMs building A I servers, that closes the loop from silicon to shipping product. Skyler: The U A Link versus NVLink Fusion dynamic is going to be one of the defining storylines in A I infrastructure for the next two to three years. Today's membership announcement is a small step, but it's in the right direction for the open ecosystem. Skyler: Alright, that's a wrap on this week's Line Rate. We covered a lot of ground — the fragmentation of scale-up options, what smart datacenter fabric design actually looks like for A I workloads, Marvell's read on where interconnect physics is headed, the huge NVLink Fusion-Arm partnership, and U A Link opening its membership. Big week in A I interconnect, and honestly, it doesn't feel like things are slowing down. Line Rate, brought to you by Ada and Zen Semiconductor — Ada helps your team integrate A I into your workflows, and Zen Semiconductor is an umbrella company with a portfolio of companies across the silicon industry. If you're getting value from the show, please subscribe and share it with a colleague in the space — it genuinely helps. Show notes with links to everything we covered are in the episode description. See you next week.