The Rise of Co-Packaged Optics in AI Data Centers: Video - IDTechEx

Skyler: Welcome to Line Rate — the show about datacenter fabric, A I interconnect, and the hardware that keeps the whole machine running at speed. I'm Skyler Monroe, and if you live in the world of silicon, switches, and scale-out infrastructure, you're in the right place. Big thanks to our sponsors — Ada, who helps organizations actually integrate A I into their workflows in ways that stick, and Zen Semiconductor, an umbrella company with a portfolio of businesses across the silicon industry, including Ada on the deployment side. Alright, today we've got a stacked episode — co-packaged optics is having a moment, Celestica just dropped a real CPO switch product, there's a massive semiconductor market forecast you need to hear, and we're going to talk about why Nvidia's NVSwitch might just be the InfiniBand of scale-up networks. Let's get into it. Skyler: Okay, so first up — co-packaged optics. IDTechEx put out a video piece on the rise of CPO in A I data centers, and look, I know CPO has been "the next big thing" for like five years now, but I genuinely think we're at an inflection point, and it's worth unpacking why. So the core problem CPO is solving is pretty fundamental. When you're running pluggable transceivers — your QSFP-DD, your OSFP modules — you're moving electrical signals from the switch ASIC across a PCB, into a pluggable module, converting to optical, and sending it down fiber. Every one of those electrical hops costs you power and signal integrity. At 400 gig that was manageable. At 800 gig it's getting painful. At 1.6 terabit per second, which is where the industry is headed, it may simply not be viable without heroic engineering. Co-packaged optics changes the geometry. You literally put the optical engines right next to the switch ASIC — on the same package substrate — so the electrical path from ASIC to laser is measured in millimeters instead of centimeters. That's not a marginal improvement. That's a different class of solution. The power savings estimates I've seen run anywhere from 30 to 50 percent reduction in transceiver power consumption compared to pluggable at equivalent bandwidth. In a hyperscale deployment where you might have hundreds of thousands of ports, that math gets very interesting very fast. Now the reason CPO has taken so long to actually ship is that it's hard in ways that are easy to underestimate. You're integrating photonics and electronics at the package level. Thermal management is a nightmare because you've got a hot ASIC right next to components that are sensitive to temperature. Serviceability is a real concern — if your optics fail, you can't just hot-swap a module. And manufacturing yield across both the photonic and electronic die is a challenge. But the industry has been grinding on these problems, and what IDTechEx is tracking is that the economic pressure from A I workloads is finally tipping the scales in CPO's favor. The bandwidth density requirements of GPU clusters are creating a forcing function that pluggable just can't meet at acceptable power budgets. Skyler: Building right on that — IEEE Spectrum did a piece framing the same shift as A I data centers getting a laser boost through optical networking, and I think it's worth zooming out a bit to the broader optical story here, not just CPO specifically. What A I training clusters need from their interconnect is fundamentally different from what traditional cloud workloads needed. You've got these massive all-to-all communication patterns — every GPU needs to talk to every other GPU with low latency and high bandwidth during gradient synchronization. And when you scale that to tens of thousands of accelerators, the network becomes as important as the compute. The optical angle here is interesting because it's not just about CPO at the switch — it's about the entire optical layer evolving. You've got silicon photonics maturing to a point where you can actually integrate laser sources in ways that were previously impossible or impractical. Intel, Cisco, Coherent, Marvell — a lot of serious money has gone into silicon photonics platforms over the last decade, and that investment is starting to pay off in the context of A I fabric. The IEEE piece also touches on something I think gets underappreciated — the latency profile of optical versus electrical. For scale-up networks, the ones connecting GPUs within a server or a rack, you need extremely low latency and electrical usually wins there. But for scale-out — rack to rack, pod to pod — optical is the only path that makes sense at the distances and bandwidths involved. And the interesting tension right now is that CPO is blurring that boundary. By getting optics closer to the ASIC, you're reducing the latency overhead that optical has traditionally carried. So the line between "close enough for electrical" and "far enough for optical" is moving, and that has real implications for how A I clusters get architected. The other thing IEEE flagged is the ecosystem question — you need the whole stack to evolve together. Lasers, modulators, detectors, DSPs, the packaging. It's a systems integration challenge as much as it's a component technology challenge. Skyler: Alright, let's talk money for a second, because Yole Group dropped a forecast that I think is worth sitting with. Data centers are on track to drive a 1.5 trillion dollar semiconductor market by 2031. Trillion with a T. That's the total semiconductor market — not just the data center slice — but Yole is specifically calling out data center demand as the primary growth engine. To put that in context, the semiconductor market was somewhere around 550 to 600 billion dollars in 2023. So we're talking about roughly a tripling in under a decade, and the thesis is that A I infrastructure buildout is the engine. Now why does this matter for what we cover on Line Rate? Because the capital flows here are going to shape every standard, every architecture decision, every ecosystem battle we talk about on this show. When you have that kind of market expansion forecast, you get massive investment in the adjacent layers — photonics, advanced packaging, high bandwidth memory, networking silicon. The companies that own the bottleneck in A I fabric stand to capture enormous value. And right now, the bottleneck is moving around. It was compute for a while — get the GPUs. Then it became memory bandwidth — hence HBM. Now there's a real argument that interconnect is the binding constraint for the next generation of scale-out clusters. Yole's numbers reflect a world where that constraint gets taken seriously and enormous capital gets deployed against it. The other thing this forecast signals is consolidation pressure. When markets get this big this fast, the companies with scale advantages — in manufacturing, in ecosystem, in standards influence — tend to pull away. You can already see that dynamic playing out in networking silicon, where Broadcom has an enormous position and everybody else is trying to find the angle that breaks their hold. For CPO specifically, the Yole forecast implies a silicon photonics market that could be worth tens of billions of dollars by the end of the decade. That's enough to sustain real competition, real R&D cycles, and real ecosystem development. Which brings me to the next story. Skyler: So Celestica just unveiled the DS6005 — a co-packaged optics switch for A I data centers — and this is the kind of product announcement that actually matters because it's a real thing shipping, not a roadmap slide. Let me tell you what we know about the DS6005. It's built around CPO technology, it's targeting A I fabric applications, and Celestica is positioning it as a production-grade solution rather than a reference design or a lab demo. Celestica is an interesting company to be making this move. They're primarily known as an EMS — electronics manufacturing services — player. They build stuff for other people. But they've been building out their own product portfolio in networking, and the DS6005 is a statement that they're serious about being in the CPO switch market as a product company, not just a contract manufacturer. The technical substance of a CPO switch like this is worth unpacking. The switch ASIC at the heart of the DS6005 — and this is almost certainly a Broadcom Tomahawk or Jericho-class device — has its optical engines integrated at the package level rather than through pluggable modules. The bandwidth density you can achieve this way is substantially higher than pluggable equivalents at the same power envelope. For A I fabric specifically, where you need to build spine layers that can handle massive east-west traffic from GPU clusters, that bandwidth density matters a lot. The competitive landscape for CPO switches right now is actually pretty thin in terms of production-available products. You've got announcements from a bunch of players — Arista has talked about CPO roadmaps, Cisco has silicon photonics plays through their Acacia acquisition, Juniper has research efforts — but actual shipping hardware is much rarer than the press releases suggest. If Celestica is genuinely delivering a production DS6005, that's a meaningful moment in the CPO transition. The serviceability question is going to come up with every customer conversation they have. The traditional answer — you replace the entire line card or even the whole switch unit — is a hard sell for operations teams used to hot-swapping pluggables. How Celestica addresses that in their system architecture will be telling. Watch this one. Skyler: Okay, last story, and this one's a bit more analytical — The Next Platform published a piece arguing that Nvidia's NVSwitch is essentially the InfiniBand of scale-up A I networks, and the longer you sit with that framing, the more interesting it gets. So let me set the stage. InfiniBand, if you're not steeped in the HPC world, was the high-performance interconnect that dominated supercomputing for decades. It had better latency, better bandwidth, better semantics for HPC workloads than Ethernet. It was also proprietary, complex, and expensive — and the conventional wisdom for twenty years was that Ethernet would eventually kill it because the economics of commodity networking would win. That didn't really happen. Mellanox — which Nvidia acquired in 2020 — kept InfiniBand alive and growing precisely because for certain workloads, the performance advantage was too large to give up. Now draw the parallel to NVSwitch. NVSwitch is the custom switching fabric that Nvidia uses inside its DGX systems and GB200 NVL racks — it's what enables NVLink across large numbers of GPUs. It's proprietary. It's optimized for A I training semantics in ways that commodity switch silicon isn't. It delivers bandwidth and latency characteristics that no third-party switch can match for GPU-to-GPU communication within a scale-up domain. And the argument The Next Platform is making is that this creates a structural lock-in dynamic that looks a lot like what InfiniBand created in HPC. The more your software stack — your NCCL collectives, your training frameworks — gets tuned to NVLink and NVSwitch semantics, the harder it is to substitute something else, even if that something else is technically competitive on raw specs. This is the scale-up versus scale-out distinction that I think is really critical for understanding the A I networking market. Scale-up is within a pod — GPU to GPU, with NVLink and NVSwitch. Scale-out is pod to pod, cluster to cluster — and that's where InfiniBand and Ethernet and all the Ethernet alternatives like U E C and U A Link compete. Nvidia owns scale-up in a way that's hard to challenge because NVLink is baked into the GPU die. The question is whether scale-up keeps expanding — whether the NVL rack concept just keeps getting bigger — or whether scale-out wins by making cross-node communication good enough that you don't need the proprietary domain. And the answer probably isn't binary. You'll have both, but the boundary matters enormously for the competitive dynamics of everyone making switches, NICs, and interconnect silicon. AMD with their Infinity Fabric, Intel with Gaudi interconnects, the whole open accelerator ecosystem — they're all trying to answer the NVSwitch question in their own way. Nobody has a fully convincing answer yet. Skyler: Alright, that's a wrap on today's episode of Line Rate. We went deep on co-packaged optics — the technology, the market context, and what a real shipping product looks like — and we ended with a big strategic frame on where Nvidia's NVSwitch fits in the scale-up versus scale-out story. A lot of moving pieces, and this is genuinely one of the fastest-moving corners of the semiconductor industry right now, so stay close. Line Rate, brought to you by Ada and Zen Semiconductor — Ada helps organizations integrate A I into their workflows, and Zen Semiconductor is an umbrella company with a portfolio across the silicon industry. If you found this useful, please subscribe and share it with the engineers and architects in your orbit — it really does help the show grow. Show notes with links to everything we covered today are available wherever you get your podcasts. I'll catch you next time.