The Last Line of Defense for Copper: The Rise of Co-Packaged Copper (CPC) in AI Data Center Interconnect Architectures - semivision

Skyler: Welcome to Line Rate — the show tracking the standards, silicon, and systems moving data at the speed of A I. I'm Skyler Monroe, and if it connects a GPU to another GPU, a rack to another rack, or a datacenter to another datacenter, we're covering it. Huge thanks to our sponsors today — Ada, who helps organizations actually integrate A I into their real workflows, and Zen Semiconductor, an umbrella company with a portfolio of businesses across the silicon industry, including Ada. Really appreciate their support. Alright, today we've got a packed show — copper making a last stand inside the package, Marvell laying out their full interconnect vision across two pieces, optical circuit switching threatening to upend how we think about spine layers, and Cisco making the case that your next A I cluster might span three time zones. Let's get into it. Skyler: Okay, first up — copper. And before you roll your eyes and say "Skyler, we've been burying copper for years," hear me out, because this one is actually interesting. The framing here comes from a SemiVision analysis titled "The Last Line of Defense for Copper," and that title is doing a lot of work. Because yes, optical is winning at the link level — DAC cables are getting shorter, active optical cables are everywhere, co-packaged optics are coming. But there's this window, this very specific architectural window, where copper is fighting back in a pretty clever way, and it's called Co-Packaged Copper — CPC. Skyler: So what is CPC? The idea is that instead of routing signals off the package and then onto a board and then over some kind of cable — even a short one — you co-package the retimer or the connectivity chiplet right alongside the main compute or switch die. You're keeping the signal path extremely short and extremely controlled. We're talking millimeters, not centimeters. And at those distances, copper electrical signaling can still win on power, latency, and cost against optical alternatives. Skyler: Now why does this matter for A I fabric specifically? Think about what's happening inside a rack-scale AI system. You've got NVLink, you've got UALink being spec'd out, you've got Ultra Ethernet Consortium defining what the next layer looks like. All of these architectures have a near-field component — something that happens at the board or package level before you ever get to pluggable optics. CPC is trying to own that layer. It's the argument that says: don't go optical until you absolutely have to. Skyler: The power math is real. Every time you convert from electrical to optical and back, you burn energy. In a system where you might be doing that conversion thousands of times across a fabric, the watts add up fast. CPC proponents are saying: keep it electrical as long as physics allows, and co-packaging lets physics allow it a little longer. The key enabler here is advanced packaging — chiplet integration, tight die-to-die interconnect, the stuff coming out of places like TSMC's CoWoS and SoIC roadmaps. Skyler: The competitive tension is real too. Co-packaged optics — CPO — has enormous momentum from hyperscalers who want to pull the optical engine right up to the switch ASIC. Intel, Broadcom, Marvell all have CPO programs. But CPO is hard. Yield, thermal management, repairability — these are unsolved problems at scale. CPC is essentially the argument that says: CPO will get there eventually, but in the meantime, here's a path that uses the advanced packaging ecosystem we already have. It's a transitional architecture, but in this industry, transitional architectures can dominate for a decade. Watch this space closely. Skyler: Next up, Marvell. And Marvell actually dropped two pieces this cycle that I want to cover back to back because they're really telling the same story from two different angles. The first is a broader piece called "The Evolution of A I Interconnects," and it's essentially Marvell laying out their worldview — where the interconnect stack came from, where it's going, and implicitly, where Marvell fits into every layer of it. Skyler: What's interesting about how Marvell frames this is that they're not just talking about one link type. They're tracing the whole arc — from PCIe as the original scale-up fabric inside a server, through NVLink's dominance in tight GPU coupling, through the explosion of 400G and 800G Ethernet for scale-out, and now into this emerging world where the boundaries between scale-up and scale-out are getting blurry. That boundary blurring is the key insight. When you've got NVLink Switch — or when U A Link gets ratified and deployed — you're suddenly doing what used to be scale-up work over fabrics that look a lot more like Ethernet in their topology. The switch-based, multi-hop model is bleeding into the near-range domain. Skyler: Marvell's angle here is interesting because they play across that whole stack. They make Ethernet PHYs, they make optical DSPs, they have CPO work, they've got their Teralynx switch ASIC line. So when they write a piece about interconnect evolution, they're not a disinterested observer — they're telling you how their portfolio maps to the future they're describing. But the technical substance holds up regardless. The point about serialization and deserialization — SerDes — is worth dwelling on. Every generation of A I interconnect pushes SerDes to a higher rate, and right now we're at 112 gigabits per second per lane as the production workhorse, with 224G on the near-term roadmap. Each jump in SerDes rate is an inflection point because it changes what's feasible at the link and fabric level. Skyler: The evolution framing also touches on the protocol layer — and this is where things like U E C matter. Ultra Ethernet Consortium is still actively developing the spec. The transport layer, the congestion control mechanisms, the adaptive routing — none of that is fully locked. But the direction is set: the industry wants a loss-aware, AI-optimized Ethernet stack that doesn't need the full TCP overhead. Marvell's ASIC roadmap is being built to land in that world. The timing risk is real — if U E C slips or fragments, companies that bet heavily on it are exposed. But the bet seems broadly shared enough that it's more likely to converge than not. Skyler: The second Marvell piece is the one I'd actually encourage people to read if you're trying to understand where the photonics story is landing right now. It's called "Solving A I's Three Big Problems Through Photonic Fabric Technology" — and they're using their own trademarked term, Photonic Fabric, but the underlying problems they're describing are real and shared across the industry. The three problems they identify are bandwidth, reach, and power. Which, honestly, is a pretty clean framing. Skyler: Bandwidth is the obvious one — A I training workloads are all-to-all collective communication patterns, and they're insatiable. Every new model generation needs more aggregate bisection bandwidth in the fabric. Reach is maybe underappreciated — as GPU clusters scale past a single rack, past a single row, past a single building, you need to maintain low-latency high-bandwidth connectivity across increasing physical distances. And power is existential — datacenters are running into power caps, and every watt you burn in the fabric is a watt you can't give to compute. Skyler: Marvell's answer to all three is essentially: integrate the optical function much closer to the die. Their Photonic Fabric vision involves silicon photonics integrated at a very high level with their DSP and switch silicon. The argument is that if you can get a tight enough integration, you collapse the electrical-to-optical conversion overhead, you get the reach of optics without all the transceiver power penalty, and you scale bandwidth by going wide with wavelengths instead of burning power on electrical retimers and SerDes at every hop. Skyler: What I think is genuinely interesting here is the tension with the CPC story we just covered. CPC says: stay electrical as long as you can. Marvell's photonics pitch says: go optical as early and as deeply as you can. Both are real engineering positions. The answer probably varies by layer — CPC wins within a package or at extremely short reach, photonics wins the moment you need to cross any meaningful distance. The question is exactly where that crossover point is, and that's being actively contested right now in the labs and in the standards bodies. Skyler: Okay, story four — and this is one I've been wanting to talk about for a while because it gets at something that doesn't get covered enough: Optical Circuit Switching, or OXC, and what it could mean for A I infrastructure topology. This comes from Dell'Oro Group, and the title is "Beyond the GPU Arms Race — The Potential Role of OXC in Building Next Gen A I Infrastructure." Skyler: So let me back up and explain what OXC actually is, because it's not yet part of most datacenter conversations at the practitioner level. Traditional datacenter switching — your spine-leaf Ethernet fabric — is packet switching. Every packet gets forwarded individually based on its header. The switching happens in silicon, it's fast, it's flexible, but every hop burns power and adds latency, and the silicon area required to do terabits of packet switching is substantial. OXC takes a different approach: you switch optical signals directly, without converting them to electrical first. You're reconfiguring the physical light paths themselves. Skyler: The upside of OXC is significant. Because you're not doing packet processing, the power profile is dramatically lower per bit. And because you can dedicate entire optical paths to specific flows — say, all the traffic between two groups of GPUs during an allreduce — you can achieve very high utilization on those paths without the overhead of per-packet decisions. For A I workloads, which often have predictable, bursty collective communication patterns, that's an interesting fit. Skyler: The Dell'Oro piece is thoughtful about where OXC fits architecturally. They're not saying OXC replaces packet switching — they're saying it potentially sits above the packet layer as a reconfigurable optical layer that shapes the topology dynamically. So your leaf-spine fabric is still there, but above it — or around it — you have an OXC layer that can rewire which pods talk to which pods based on what the workload needs. For a training job that runs for days and has a known communication pattern, you can optimize the optical topology for exactly that job. Skyler: The companies in this space are a mix of startups and established players. Calient has been in optical circuit switching for a long time. Ayar Labs and others are pushing photonic integration angles. And hyperscalers are increasingly interested — Google has published work on optical circuit switching in their datacenter fabric, and Meta has experimented with it. The challenge is control plane complexity. Reconfiguring optical paths takes milliseconds to seconds compared to microseconds for packet forwarding. That means the traffic engineering has to be done at a coarser time granularity. For A I training where you have job-level scheduling, that's fine. For general-purpose cloud traffic, it's harder. Skyler: The Dell'Oro framing of "beyond the GPU arms race" is pointed. The argument is that the industry has been so focused on raw GPU compute that the interconnect — which is often the actual bottleneck in large training runs — hasn't gotten proportional investment. OXC is one of the architectural levers that could change the efficiency equation at the fabric level, independent of what NVIDIA or AMD or Google do on the compute side. If you're a datacenter architect or an investor in infrastructure, this is a technology trajectory worth tracking carefully. It's not mainstream yet, but the physics are compelling and the hyperscaler interest is real. Skyler: Last story, and this one is zooming out the furthest — Cisco dropping a piece on what they're calling "Scale-Across," which is their framing for the idea that future A I infrastructure won't live in a single datacenter. The subtitle says it clearly: "Why the Future of Distributed A I Isn't in One Data Center." Skyler: So let's be honest — some of this is Cisco planting a flag that positions their WAN and routing portfolio as essential A I infrastructure. That's a legitimate business move. But the underlying technical argument is worth engaging with seriously, because the scale-across challenge is real and increasingly urgent. We've spent the last few years talking about scale-up — tight coupling inside a node or a small cluster — and scale-out — Ethernet fabric across a rack or a pod. Scale-across is the next layer: how do you run a coherent A I job across multiple datacenters that might be hundreds of kilometers apart? Skyler: The reason this is becoming a real engineering problem and not just a marketing concept is physical constraints. Power availability. Land. Water. The sheer infrastructure required to put fifty thousand GPUs in one place is hitting real limits. Hyperscalers are already planning distributed training across multiple sites. OpenAI, Google, Microsoft — they're all thinking about this. And when you start doing that, the interconnect problem becomes fundamentally different. You're now dealing with speed-of-light latency at hundreds of miles, with different network operators, with potential asymmetry in bandwidth, with failure domains that span geographies. Skyler: Cisco's argument is that making this work requires rethinking how you do traffic engineering and routing at the WAN level — moving toward something that can do for distributed A I what the NVLink fabric does for a single node. You need deterministic latency, high bandwidth, and enough intelligence in the network to route around failures without disrupting a training job. That's a hard problem because traditional WAN routing was designed for resilience, not for microsecond-deterministic behavior. Skyler: There's a standards angle here too. Some of the work happening in the Ultra Ethernet Consortium around congestion control and adaptive routing is relevant even at WAN distances — the ideas about flow-level multipathing, about in-network telemetry, those translate. And there's interesting work on RDMA over long-haul links that's being explored. The latency tolerance of A I training workloads — particularly inference-time parallelism versus training — is going to drive different architectural choices. Inference is much less tolerant of jitter; training can absorb more variability if the synchronization points are managed carefully. Skyler: What I'd watch is how the big networking vendors — Cisco, Juniper, Nokia — position against the hyperscaler-owned WAN fabric. Companies like Google and Meta have increasingly built their own long-haul optical infrastructure, and they're not buying traditional telco gear for their backbone. The scale-across story is going to play out differently for enterprises and cloud customers versus hyperscalers who own their own fiber. But the fundamental engineering challenge — moving A I workloads efficiently across geographic distance — is real for everyone. This is going to be a major topic over the next three to five years. Skyler: Alright, that's a wrap on today's Line Rate. We went from copper hanging on inside the package, through Marvell's full-stack photonics pitch, through optical circuit switching reshaping spine topology, all the way out to multi-datacenter A I fabric. Lot of threads to pull on. Thanks again to our sponsors — Line Rate, brought to you by Ada and Zen Semiconductor. Ada helps organizations integrate A I into their actual workflows, and Zen Semiconductor is an umbrella company with a portfolio of businesses across the silicon industry. If you found this useful, subscribe wherever you get your podcasts, share it with a colleague who's in the weeds on interconnect, and check the show notes for links to all five pieces we covered today. See you next time.