Co-Packaged Optics Reaches Power Efficiency Tipping Point - Semiconductor Engineering

Skyler: Welcome to Line Rate — the show tracking the chips, standards, and infrastructure moving data at the speed of A I. I'm Skyler Monroe, and if it plugs into a datacenter and moves bits fast, we're talking about it. Huge thanks to our sponsors today — Ada, which helps clients integrate A I into their workflows, and Zen Semiconductor, an umbrella company with a portfolio of businesses across the silicon industry, including Ada. Really glad to have them on board. Alright — today we've got a full card: co-packaged optics hitting a real efficiency milestone, the market forces behind CPO growth, why copper isn't dead yet inside your scale-up network, a look at how A I interconnects are actually evolving, and a new edge compute play pushing capacity out to cell towers in Texas. Let's get into it. Skyler: First up — co-packaged optics, and a headline that actually means something: CPO has hit a power efficiency tipping point. Semiconductor Engineering ran a piece on this and it's worth unpacking because "tipping point" is a phrase that gets thrown around a lot in this industry, but there's real substance here. So let's talk about what's actually happening. Skyler: The core issue with pluggable optics — your traditional QSFP-DD, your OSFP modules sitting in the faceplate of a switch — is that the signal has to travel from the ASIC, across a PCB trace, through a retimer in many cases, and then finally into the optical module. Every one of those hops costs you power, and it costs you signal integrity. At 800 gig per lane and heading toward 1.6 terabit, that SerDes loss on the PCB is brutal. You're burning watts just fighting the physics of getting the signal to the edge of the board. Skyler: Co-packaged optics solves this by moving the optical engine right next to the switching ASIC — same package, dramatically shorter electrical path. You eliminate those long PCB traces, you kill the retimers, and you cut the SerDes power budget significantly. The question has always been: does the integration complexity and the yield risk of putting photonics next to silicon offset those gains? And what the Semiconductor Engineering piece is pointing to is that we're crossing the line where the answer is now clearly yes. Skyler: The efficiency numbers being thrown around are compelling. We're talking about CPO designs achieving something in the range of 5 picojoules per bit or better for the optical I/O portion — and when you're running a 51.2 terabit switch, that math adds up to hundreds of watts of difference versus pluggable. At hyperscaler scale, hundreds of watts per switch times tens of thousands of switches is a real number. That's not a rounding error in your PUE. Skyler: What makes this a tipping point moment rather than just incremental progress is the manufacturing ecosystem catching up. Silicon photonics foundry capacity has matured, the co-design tools between the ASIC teams and the photonics teams have gotten a lot better, and the connectorization problem — how do you actually get fiber into a package that's now sitting on a board differently than a faceplate module — that's being standardized. You're seeing groups like the CPO Alliance and contributions into OIF getting traction on the mechanical and electrical interfaces. Skyler: For datacenter architects, the practical implication is that your next-generation spine layer — the 51.2T and beyond switching platforms — will very likely be CPO-native. The pluggable era isn't over for everything, but for the highest-bandwidth, most power-constrained parts of the fabric, CPO is where the physics forces you. The efficiency tipping point is the industry saying: the bet has paid off. Now let's scale it. Skyler: Sticking with CPO — the second story is a market research angle from GlobeNewswire, and the headline is that co-packaged optics market growth is driven by A I. Which, honestly, is not a surprise to anyone in this room, but the numbers and the framing are worth talking about because they reveal something about the timing and the competitive dynamics. Skyler: The A I training and inference buildout is the single biggest demand driver for high-radix, high-bandwidth switching. When you're building a GPU cluster at the scale of a 100,000-accelerator pod — which is now a real thing that hyperscalers are deploying — the fabric connecting those GPUs has to move extraordinary amounts of data with minimal latency. The all-reduce operations in distributed training are brutal on the network. Every microsecond of added latency or every watt of inefficiency gets multiplied across thousands of links. Skyler: What the market analysis points to is that A I infrastructure spending is specifically pulling CPO forward on the adoption curve. Historically, new optical interconnect technologies take a long time to go from demonstration to production deployment — pluggable 400G had a multi-year ramp. CPO is being pushed harder because the A I buildout isn't waiting. The power density in an A I compute rack — you've got GPUs pulling 700, 900 watts each, NVLink switches, A I accelerator ASICs — the networking has to be equally efficient or the power budget of the whole rack doesn't close. Skyler: The market forecasts being cited put CPO revenue growing very aggressively through the late 2020s. I'd take the specific numbers with some grains of salt — market research reports have a history of being optimistic — but the directional signal is right. You've got Microsoft, Google, Meta, Amazon all running their own networking silicon programs, and every one of those programs is either already CPO or has CPO on the roadmap. That's not a coincidence. Skyler: There's also a geographic dimension here that's worth flagging. Taiwan's silicon photonics foundry ecosystem — TSMC's photonics offerings, the packaging houses — and the U.S. and European players like Intel and Ayar Labs are all racing to be the volume supplier for this transition. Who owns the CPO supply chain matters for the hyperscalers because it affects their ability to dual-source, and it matters for geopolitical resilience given where we are with semiconductor policy. Skyler: The other thing the A I demand pull does is compress the standardization timeline. When you've got a handful of hyperscalers all wanting CPO at volume at the same time, they have enough buying power to push the ecosystem to converge on interfaces. That's actually healthy for the industry — it means you're less likely to end up with five incompatible CPO implementations that fragment the market. The A I wave is, in a weird way, forcing standardization faster than it would have happened organically. Skyler: Alright, story three — and this one comes from Marvell, so it's explicitly vendor perspective, but it's technically grounded and worth taking seriously. The headline is co-packaged copper extending its reach inside scale-up networks. And this is the counterpoint to the CPO conversation — copper isn't going away, and Marvell is making a case for why co-packaged copper has a specific and meaningful role in the A I fabric. Skyler: Let's be precise about terminology here. When we say co-packaged copper, we're talking about integrating the SerDes and the electrical interface very close to the compute or switching ASIC — similar packaging philosophy to CPO, but you're staying in the electrical domain. No photons. The argument is that inside a scale-up domain — so the high-bandwidth, low-latency interconnect between GPUs within a pod, think NVLink-style topologies or UALink-style topologies — the distances are short enough that copper can work, and copper is cheaper, lower latency, and doesn't need lasers. Skyler: Marvell's argument is essentially that the reach of co-packaged copper is being extended by advances in SerDes equalization and signal processing. Where you previously might have needed to go optical at, say, a meter or two to maintain signal integrity at 112 gig PAM4, improvements in DSP and equalization are pushing that reach out. Not to the distances optical handles — we're not talking cross-datacenter — but within a rack or across adjacent racks, copper can stay viable. Skyler: For scale-up networks specifically, this matters a lot. The latency advantage of copper over optical is real — you don't have optical-electrical conversion delays, you don't have the laser driver power overhead. In a tight all-reduce loop where you're doing billions of gradient synchronization operations during a training run, that latency delta accumulates. If you can stay in copper within your scale-up domain and only go optical for scale-out — the inter-pod or inter-datacenter fabric — you've optimized the right parts of the network for the right cost and performance targets. Skyler: The competitive context here is that Marvell is clearly positioning against pure-play optical and against the assumption that CPO wins everywhere. They have a stake in the copper interconnect world — their PAM4 SerDes business is significant — and this piece is a reminder that the networking stack in a big A I cluster has multiple tiers, and not every tier needs to be optical. That's actually correct. The architecture of a well-designed A I fabric uses the right medium for the right hop. Skyler: What I'd watch for is where exactly the crossover point lands as lane rates climb toward 224 gig. At 112 gig PAM4, co-packaged copper can do a meter or two reasonably. At 224 gig, the physics get harder. The equalization DSP gets more power-hungry. So the reach extension argument has a ceiling, and the optical camp would argue that ceiling is coming faster than the copper proponents want to admit. The real answer is probably that both coexist in layered A I fabrics for the foreseeable future — copper inside the scale-up domain, optics everywhere else. Skyler: Story four — also from Marvell, and this one zooms out to the bigger picture: the evolution of A I interconnects. This is more of a perspective piece than a product announcement, but it's a useful framework for thinking about where the stack is going, and Marvell has a vantage point here because they're selling into both the compute-side and the networking-side of the A I infrastructure. Skyler: The framing Marvell uses is essentially a generational one. Early A I training deployments were about getting GPUs connected with whatever high-bandwidth interconnect existed — InfiniBand was dominant, Ethernet was catching up. The interconnect was important but it wasn't the bottleneck in the same way it is now. As model sizes exploded — we're talking about models with hundreds of billions of parameters, training runs that last weeks — the all-to-all communication patterns put enormous pressure on the fabric. The interconnect became the critical path. Skyler: What's interesting about where we are now is that the industry has bifurcated the interconnect problem into two distinct domains, and the solutions for each look quite different. Scale-up is the tight coupling between accelerators within a training pod — you want the highest possible bandwidth, the lowest possible latency, and you're willing to pay a premium for it because this is where the gradient synchronization happens. NVLink is the canonical example. UALink is the industry consortium trying to build an open standard for this space, and it's still evolving — I want to be clear, U A Link specs are still being developed and the ecosystem is not yet mature. Skyler: Scale-out is the connection between pods, between racks, between datacenters — this is where Ethernet and InfiniBand compete, where RoCE and RDMA matter, where U E C — the Ultra Ethernet Consortium — is trying to define a next-generation Ethernet transport that can handle A I workloads better than vanilla Ethernet. U E C is also still a work in progress, so anyone telling you it's fully baked should be taken with skepticism. Skyler: The evolution Marvell is pointing to is that these two domains are converging in terms of the demands placed on the hardware, but diverging in terms of the solutions. The bandwidth requirements for both are climbing toward terabits per port. But scale-up wants coherent-like latencies and tight integration with the accelerator, while scale-out wants routable, congestion-managed, resilient networking that can work across heterogeneous infrastructure. Those are genuinely different problems. Skyler: What this means for the companies building the interconnect silicon is that there's no single product that wins everywhere. You need a portfolio — high-speed SerDes for scale-up, switch ASICs with sophisticated congestion control for scale-out, and optics or copper optimized for the hop length of each domain. Marvell's implicit message is that they have products across all of these layers. But the broader takeaway for the industry is that the A I interconnect stack is more complex and more layered than it was even two or three years ago, and it's going to keep getting more complex as model architectures evolve. Skyler: One more thing worth flagging: the custom silicon angle. The hyperscalers aren't just buying from Marvell and Broadcom anymore — they're building their own network ASICs, their own optical transceivers, their own interconnect stacks. Google's TPU interconnect, Microsoft's MAIA ecosystem, Amazon's Trainium networking — these are real, production-scale, custom solutions. The merchant silicon guys have to keep running faster just to stay relevant, which is part of why you're seeing these Marvell perspective pieces — it's partly competitive positioning in a market that's getting more complicated for them. Skyler: Last story today — and this one is a bit different, more on the infrastructure deployment and investment side. Data Center Dynamics reported that Forum Markets, an investment firm, is partnering with Edge Node on a U.S. compute rollout. The joint venture is taking on datacenter capacity in Dallas and North Carolina, and the interesting piece is that they're also deploying compute capacity at Texas cell tower sites. Skyler: Let me talk about why this matters from a fabric and interconnect perspective, because at first glance it looks like a real estate and finance story. But the cell tower angle is actually technically interesting. What Edge Node and Forum Markets are doing by deploying compute at cell tower sites is pushing processing capacity to the edge of the network — geographically distributed, closer to end users or to the data sources. This is a different architecture than the mega-campus hyperscaler model. Skyler: For A I inference specifically — as opposed to training — latency to the end user matters. If you're running a real-time A I application, whether that's industrial automation, autonomous systems, or just low-latency inference for consumer applications, the speed of light is a real constraint. A request that has to travel to a centralized datacenter in Ashburn or Dallas and back adds tens of milliseconds. For some applications that's fine. For others, it's not. Cell tower sites give you geographic density that a traditional campus model can't match. Skyler: The infrastructure challenge is significant though. Cell tower sites are not datacenters — they have limited power, limited cooling, limited physical space, and they're geographically distributed in a way that makes management complex. The compute you can deploy there is constrained. You're not putting 10,000-GPU pods at a cell tower. You're putting inference nodes — purpose-built accelerator hardware that's optimized for power efficiency and small form factor. Skyler: From an interconnect standpoint, this distributed model creates interesting problems. How do you connect the edge inference nodes back to the core — where the models are trained and updated, where the data lakes live? You need reliable, high-bandwidth WAN connectivity from sites that were designed for cellular backhaul, not for datacenter interconnect. That's a non-trivial networking challenge, and it's an area where the standards and the product ecosystem are still catching up to the deployment ambition. Skyler: Dallas and North Carolina are smart choices for the core sites — both have strong fiber density, reasonable power costs, and established datacenter ecosystems. Dallas in particular has become a significant A I infrastructure hub, partly because of Texas power deregulation giving operators more flexibility on electricity sourcing. The cell tower overlay is the differentiating piece of this JV, and it signals that the edge compute market is real enough to attract institutional investment capital. Skyler: For the interconnect industry, edge compute proliferation is a demand driver for a different profile of networking hardware — lower power, smaller form factor, ruggedized for non-datacenter environments, but still needing to hit meaningful bandwidth back to the core. That's a product category that's going to grow as more of these distributed A I inference deployments come online. Skyler: Alright, that's the show for today. Co-packaged optics crossing the efficiency threshold, A I demand pulling CPO adoption forward, copper holding its ground in scale-up networks, the layered evolution of A I interconnect architecture, and edge compute starting to attract real capital and real infrastructure. Big week in the fabric space. Thanks again to our sponsors — Line Rate, brought to you by Ada and Zen Semiconductor. Ada helps teams integrate A I into their workflows, and Zen is an umbrella company with a portfolio across the silicon industry. If you got value from this episode, please subscribe wherever you get your podcasts, and share it with someone who cares about where datacenter infrastructure is headed — there are more of us than people think. Show notes with links to everything we covered today are in the feed. I'm Skyler Monroe. See you next time on Line Rate.