Nokia’s Latency Reduction Leap for AI Data Centres - Data Centre Magazine

Skyler: Hey, welcome to Line Rate — the show that tracks the chips, standards, and consortia moving data across the modern A I datacenter. I'm Skyler Monroe, and if you care about what's happening at the fabric layer — rack-to-rack, appliance-to-appliance, the stuff that actually makes distributed A I training and inference work — you're in the right place. Big thanks to our sponsors Ada, who helps companies actually integrate A I into their real workflows, and Zen Semiconductor, an umbrella company with a portfolio across the silicon industry. Really appreciate their support keeping this show going. Alright, we've got a packed episode today — Nokia making moves on latency, Marvell opening up the co-packaged optics playbook, the Ultra Ethernet Consortium origin story, NVLink Fusion's ripple effects, and Meta's take on CPO in hyperscale. Let's get into it. Skyler: Okay, first up — Nokia. The headline is "Nokia's Latency Reduction Leap for A I Data Centres," and look, Nokia is not the first name most people in this room think of when someone says A I fabric. But that's kind of the interesting thing here. Nokia's been quietly positioning their routing and transport portfolio around the A I training and inference workload problem, and what they're talking about is meaningful if you understand the context. Skyler: So the core issue is this: A I training at scale — we're talking thousands of GPUs running a single job — is brutally sensitive to tail latency. Like, one slow link, one congested queue, and you've got stragglers. And in a synchronous training run, the whole job waits for the slowest node. That's not a theoretical problem, that is the day-to-day operational reality for anyone running large-scale training clusters. Skyler: What Nokia is pushing is the idea that their routing and transport layer can meaningfully reduce the latency floor and the latency variance in these environments. And they're specifically calling out A I data centres as a target vertical, which tells you something about where Nokia sees growth coming from. Skyler: Now, the technical substance here connects directly to what the Ultra Ethernet Consortium is working on — which we'll get to in a minute — because the core thesis of U E C is that Ethernet, with the right transport extensions, can compete with InfiniBand on the metrics that matter for A I: low latency, high bandwidth utilization under load, and congestion response that doesn't crater your throughput. Skyler: Nokia's angle is that the network fabric itself — the routing layer — is a variable that operators have underinvested in optimizing for A I workloads. They're not wrong about that. A lot of clusters were built with data center fabrics that were optimized for east-west storage traffic or cloud-native microservices, not for the all-to-all communication patterns you get in a transformer training run. Skyler: The all-to-all pattern is brutal on networks. Every GPU is trying to talk to every other GPU during the gradient synchronization phase. If your fabric isn't purpose-built or at least well-tuned for that, you're going to see head-of-line blocking, you're going to see buffer exhaustion, and your effective GPU utilization drops through the floor. Skyler: What's interesting about Nokia entering this conversation loudly is that it signals the incumbents in enterprise and carrier networking see A I infrastructure as a greenfield opportunity. Arista has been talking about this. Cisco has been talking about this. And now Nokia is making explicit plays. The question is always: how much of this is marketing alignment and how much is genuine technical differentiation? And honestly, the answer probably varies by deployment. Skyler: For operators evaluating their fabric options right now, the key thing to watch is whether Nokia is publishing actual benchmark data — queue depth behavior, flow completion times under realistic A I traffic profiles — or whether this is mostly positioning. Because in this market, everyone has a slide deck that says "optimized for A I." The ones who win are the ones who can show numbers. Skyler: Bottom line on Nokia: they're a credible player in transport infrastructure, and the direction they're pointing is right. Latency and congestion management for A I fabric is a real problem space with real dollars attached to it. Worth watching, but the proof will be in the benchmarks. Skyler: Alright, story two — Marvell and Open CPX. The headline is "Open CPX Sets the Stage for More Flexible, Scalable Connectivity," and this one is genuinely interesting from a standards and ecosystem perspective. Skyler: So co-packaged optics — CPO — is one of those technologies that has been "just around the corner" for what feels like a decade, but it's actually starting to materialize in real products now. The basic idea is that instead of having pluggable optical transceivers hanging off your ASIC on a front panel, you integrate the optical engine much closer to the switch silicon — ideally in the same package. The benefit is dramatic: you eliminate the SerDes runs that eat power and degrade signal integrity, you reduce latency, and you open up bandwidth headroom that pluggables fundamentally can't reach. Skyler: But here's the problem CPO has always had: it's a packaging and integration play, which means it historically required a very tight coupling between the switch ASIC vendor and the optical engine vendor. That's great if you're a vertically integrated company, but it creates a nightmare for the broader ecosystem. If you're an operator, you want to be able to mix and match, qualify components independently, and not be locked into a single vendor's CPO implementation. Skyler: That's what Open CPX is trying to solve. Marvell is proposing an open interface specification that defines how the optical chiplet connects to the host — standardizing the electrical interface, the management plane, the mechanical and thermal constraints. The "CPX" stands for co-packaged express, and the "open" part is the key word. The goal is to let optical engine vendors and switch ASIC vendors develop to a common interface, which means a switch designer isn't forced to use one specific optical supplier. Skyler: This is a classic chicken-and-egg problem in standards land. The spec only matters if both sides of the interface — silicon and optics — actually build to it. And that requires broad industry buy-in. Marvell publishing this is a first step, but you'd want to see buy-in from the optical chiplet vendors — think names like Ayar Labs, Ranovus, and others in that space — as well as other switch ASIC vendors actually committing to the interface. Skyler: The timing here is interesting because the bandwidth demands of next-generation A I switches are genuinely pushing against what pluggable optics can deliver. We're talking about switches with 51.2 terabits per second of capacity today, and the roadmap goes higher. At those bandwidth densities, pluggables run into real physical limits — power per port, thermal density on the faceplate, the loss budget on the traces between the ASIC and the cage. CPO solves those problems structurally. Skyler: For hyperscalers, this is a big deal. If Open CPX gains traction, it means they're not betting on a single vendor's CPO implementation. They can have multiple qualified sources for the optical engine, which is how you actually build a supply chain you can rely on at scale. If you're Google or Microsoft and you're trying to deploy hundreds of thousands of switch ports, single-source dependencies are a serious risk. Skyler: The competitive angle here is that Intel, Broadcom, and others in the switch ASIC space are all going to have opinions about an open interface specification proposed by Marvell. Marvell has been aggressive about positioning themselves as the CPO-friendly option — their Teralynx switch family has been specifically called out as CPO-ready. So there's both a genuine ecosystem play here and a competitive positioning play, and you should read it as both simultaneously. Skyler: Open CPX is not a ratified standard yet — it's a proposal from Marvell with an intent to drive it through an industry body. Watch for whether it gets picked up by OIF, which is the Optical Internetworking Forum, because that's where something like this would need to land to become truly multi-vendor. Until it's there, it's a vendor proposal. An interesting one, but still a proposal. Skyler: Okay, story three — and this one is a bit of a throwback, but it matters for context. The headline is "Leading Cloud Service, Semiconductor, and System Providers Unite to Form Ultra Ethernet Consortium," from the Linux Foundation. Now, the UEC founding announcement is not new news — this actually dropped in mid-2023 — but it keeps surfacing in feeds because U E C is very much still in the news as the spec work progresses. So let's use it as an anchor to talk about where U E C actually stands. Skyler: The founding of the Ultra Ethernet Consortium was a genuinely significant moment. You had AMD, Arista, Broadcom, Cisco, Eviden, HPE, Intel, Meta, Microsoft, and others all signing on to say: we believe Ethernet can be the substrate for high-performance A I fabric, but we need to evolve the transport to make it work. And they're doing this in a Linux Foundation home, which signals open governance, not a vendor-controlled spec. Skyler: The technical problem they're solving is real. Standard Ethernet with TCP or RDMA over Converged Ethernet — RoCE — has well-documented issues in A I training environments. Congestion control is one. The PFC — Priority Flow Control — mechanism that RoCE relies on to prevent packet drops can cause pause storms that propagate through the fabric and tank throughput. Anyone who's operated a large RoCE cluster has war stories about this. Skyler: U E C is proposing a new transport layer — the UET, Ultra Ethernet Transport — that reimagines how reliability and congestion are handled at the endpoint rather than relying on the network to be lossless. The idea is that you put smarter logic in the NIC and in the endpoint, and you let the network be slightly lossy in controlled ways, with the endpoint handling retransmit and reorder. This is philosophically closer to how some high-performance computing interconnects work, and it sidesteps the PFC storm problem. Skyler: They're also working on a scatter-gather model that's designed for the all-to-all communication pattern — again, this is the gradient synchronization problem I mentioned earlier. The goal is to let A I workloads express their communication intent in a way that the transport can optimize for, rather than trying to shoehorn A I traffic into primitives designed for client-server or storage workloads. Skyler: As of where we are now, U E C has published initial specs and is working through the messy middle of standards development — interoperability, edge cases, implementation agreements. This is where standards either get real or die. The fact that it has genuine hyperscaler engagement — Meta and Microsoft are not in this for the press release — is a positive signal. But we're still a ways from U E C silicon being a default choice in new cluster builds. Skyler: The competitive frame is U E C versus InfiniBand, which is still Nvidia's preferred fabric for their GPU clusters. InfiniBand with NVLink is a proven, integrated, and frankly well-optimized stack for A I training. U E C's pitch is openness, cost, and ubiquity — Ethernet everywhere, multiple vendors, no single-vendor lock-in. That's a compelling pitch for hyperscalers who want to build at scale without depending on Nvidia's supply chain. Whether the transport performance closes the gap is the open question. Skyler: Watch for silicon implementations — when you start seeing U E C-native NICs coming to market from multiple vendors, that's when this transitions from a standards story to an infrastructure story. Skyler: Story four — NVLink Fusion. The headline is from 24/7 Wall St.: "5 Stocks That Win Big From NVIDIA's NVLink Fusion A I Ecosystem Expansion." Look, I'm going to be honest with you — I will not be giving stock picks on this show. But the NVLink Fusion announcement is technically substantive, so let's dig into what's actually happening here. Skyler: NVLink Fusion is Nvidia's move to open up NVLink — previously a proprietary, Nvidia-only interconnect — to third-party silicon. Historically, NVLink has been the glue between Nvidia GPUs in a compute node. NVLink Bridge, NVSwitch — these are what give you the high-bandwidth, low-latency communication between GPUs in an NVL72 or NVL36 configuration. And it has been completely closed. You could not put a non-Nvidia chip on one side of an NVLink connection. Full stop. Skyler: NVLink Fusion changes that. Nvidia is now allowing third-party ASICs — think custom A I accelerators, networking chips, or even CPUs — to connect to a Hopper or Blackwell GPU over NVLink. The practical implication is that a company building a custom A I chip can now design it to speak NVLink natively and get first-class, high-bandwidth connectivity to Nvidia GPUs without going through PCIe. Skyler: That is a significant architectural unlock. PCIe is not terrible, but it's bandwidth-limited and latency-limited compared to NVLink. If you're building a custom inference accelerator or a domain-specific chip that needs to work alongside Nvidia GPUs, getting native NVLink access means you can actually build systems where the custom silicon isn't a bottleneck. You can really participate in the high-bandwidth memory fabric rather than being an awkward peripheral. Skyler: The companies the article calls out as beneficiaries are the ones who could build custom silicon that now has a viable path to deploying alongside Nvidia GPUs without performance penalties. Think custom networking ASICs, think inference accelerators from startups, think system builders who want to compose heterogeneous compute. Skyler: But here's the nuance. NVLink Fusion is Nvidia opening the door — on Nvidia's terms. You still need to license the NVLink spec from Nvidia. You still need to go through Nvidia's compliance and interoperability process. This is not open like Ethernet or PCIe. Nvidia controls the roadmap, the licensing, and ultimately who gets to play. That's a very different model from what U E C or CXL are trying to do. Skyler: So you're trading openness for performance. For some use cases — particularly high-value inference deployments where you want maximum bandwidth between a custom chip and Nvidia GPUs — that tradeoff might be completely worth it. For hyperscalers who are philosophically committed to open, vendor-neutral stacks, this is less interesting. Skyler: The competitive pressure this creates is interesting. It makes Nvidia's GPU ecosystem stickier, because now third-party silicon vendors are incentivized to build to NVLink rather than to open alternatives. If enough custom silicon ends up NVLink-native, it reinforces Nvidia's platform position even as the compute layer diversifies. It's a classic platform strategy — extend the ecosystem just enough to pull in more participants while keeping the key interfaces proprietary. Nvidia plays this game extremely well. Skyler: For the A I fabric community specifically, watch how this interacts with UALink — U A Link — which is AMD's open alternative for GPU-to-GPU coherent fabric. U A Link is trying to do what NVLink Fusion is doing but in an open, multi-vendor way. NVLink Fusion makes Nvidia's ecosystem more attractive; U A Link is the open answer. The battle between those two paradigms is going to define a lot of infrastructure choices over the next three to five years. Skyler: Last story — Meta and CPO. The headline is from a preview of ECOC 2025 coverage: "Meta Evaluates Co-Packaged Optics in Hyperscale Data Centers." ECOC is the European Conference on Optical Communications, and it's one of the major venues where the optics industry shows its serious technical work. If Meta is presenting CPO evaluation results there, that's worth paying attention to. Skyler: So we talked about CPO in the Marvell story from the packaging and standards angle. Let me give you the operator perspective, because that's what Meta brings. Meta has been one of the more publicly engaged hyperscalers on CPO. They've been involved in the Consortium for On-Board Optics — COBO — and they've been vocal about the fact that pluggable optics are approaching physical limits for what they need. Skyler: What Meta is presenting at ECOC is their evaluation framework and early results for CPO in actual hyperscale data center conditions. And that distinction matters enormously. CPO in a lab is one thing. CPO in a data center with real thermal environments, real mechanical constraints, real operations workflows — that's where you find out whether a technology is actually deployable or whether it's still a research project. Skyler: The operational challenges for CPO are non-trivial. With pluggable transceivers, if an optical module fails, a technician pulls it out and puts a new one in. Field-replaceable, no rocket science. With co-packaged optics, the optical engine is integrated into or very close to the switch package. A failure scenario potentially requires replacing the entire switch, or at minimum requires more complex and expensive repair procedures. That has real implications for operating cost and availability. Skyler: There's also the supply chain question. Pluggable optics have a massive, mature, competitive supply chain. Multiple vendors, standard form factors, global qualification databases. CPO supply chains are nascent. Meta, at their scale, needs confidence that they can source CPO-enabled switches from multiple vendors at the volumes they need.