Optics is the next big AI bottleneck. This company could be an underrated beneficiary. - MarketWatch

Skyler: Hey, welcome back to Line Rate — the show tracking the chips, standards, and infrastructure moves shaping the AI datacenter fabric. I'm Skyler Monroe, and if you care about how data moves at scale — rack to rack, switch to switch, datacenter to datacenter — you're in the right place. Big thanks to our sponsors Ada, who helps organizations actually integrate A I into their workflows in ways that stick, and Zen Semiconductor, an umbrella company with a portfolio of businesses across the silicon industry. Today we've got a packed one — co-packaged optics heating up fast, power infrastructure struggling to keep pace with AI density, Cisco making the case for distributed A I across datacenters, and Elon Musk placing a very large bet on mesh optical networking. Let's get into it. Skyler: Alright, story one. MarketWatch is running a piece flagging optics as the next major A I bottleneck — and pointing to what they're calling an underrated beneficiary in that space. And look, I'm not going to turn this into a stock tip segment, but the underlying technical argument here is absolutely worth unpacking, because they're right about the bottleneck. Skyler: Here's the deal. As A I clusters keep scaling — we're talking hundreds of thousands of GPUs and accelerators wired together — the electrical interconnect that's carried us this far is starting to hit a wall. Copper traces inside a package, or even over a backplane, lose signal integrity fast as you push bandwidth density up. You're fighting insertion loss, you're fighting crosstalk, and you're fighting power consumption that scales badly with distance and data rate. Skyler: Co-packaged optics — CPO — is the answer the industry is converging on. The idea is you move the optical engine as close to the switching ASIC as physically possible. Instead of a pluggable transceiver hanging off the front panel of a switch, you integrate the photonics right into the package with the switch die, or at least onto the same substrate. That dramatically cuts the length of the high-speed electrical SerDes lanes that are doing the heavy lifting between the optics and the switch logic, and that's where your power and signal headroom go. Skyler: The reason this is a bottleneck right now is timing. The hyperscalers need next-generation switches at 51.2 terabits per second and eventually 102.4 terabits per second to keep feeding their GPU clusters. At those speeds, the traditional pluggable optics approach gets really painful — the SerDes lanes running from ASIC to pluggable module are burning enormous power just to compensate for the signal loss on that short but lossy journey across the board. CPO sidesteps most of that. Skyler: The challenge is that CPO is not a drop-in swap. It changes thermal design, it changes serviceability — you can't just yank a bad transceiver anymore — and it requires extremely tight co-design between the photonics supplier and the switch ASIC vendor. That's why whoever is embedded early in those co-design relationships has a real structural advantage. This isn't a commodity component play. It's a deep integration play, and the companies that get designed in at the platform level are very hard to displace. Keep watching this space closely because the OFC roadmap and what the hyperscalers are actually qualifying right now will tell the real story. Skyler: Okay, story two comes from Data Center Dynamics, and it's about a problem that is frankly less glamorous than photonics but just as urgent — power. The headline is about retrofitting existing datacenter power infrastructure to handle A I-scale rack densities, and the constraints standing in the way. Skyler: So here's the context. Traditional datacenter racks were designed around maybe five to fifteen kilowatts per rack. That was the compute world for a long time. A I training clusters have absolutely blown that assumption up. An H100 server draws on the order of ten kilowatts by itself. A full rack of DGX H100 systems or equivalent dense GPU sleds can push sixty, eighty, even a hundred kilowatts per rack. That's not a marginal increase — that's a full order of magnitude shift from what a lot of existing facilities were built for. Skyler: And here's where it gets really sticky for operators trying to retrofit rather than greenfield build. The power delivery infrastructure — your UPS systems, your PDUs, your busways, your transformers — was all sized for the old world. You can't just plug in a hundred-kilowatt rack into a facility that was designed for fifteen. The breakers won't support it, the cooling definitely won't support it, and the physical busbar capacity may not support it either. Skyler: On the cooling side, air cooling starts becoming genuinely untenable at these densities. You can push it — people are running very high-velocity airflow, hot-aisle containment, rear-door heat exchangers — but at some point you're moving so much air through a rack that the mechanical engineering of the cooling system becomes its own problem. That's why direct liquid cooling — DLC — and immersion cooling are moving from niche to mainstream fast. But retrofitting liquid cooling into a facility designed for air is expensive and disruptive. Skyler: From a fabric perspective, this matters because power constraints directly limit how you can deploy your compute and your networking hardware. If you can't physically get the power to the rack, you can't run the switch at full port count or full speed. Some operators are seeing situations where they have the networking gear, they have the GPUs, but the facility literally cannot deliver the watts to make it all run simultaneously. That creates throttling and architectural compromises that hurt cluster performance. Skyler: The piece flags that there's no clean universal answer here — it's a site-by-site engineering exercise. What it does signal for the industry is that power delivery and thermal management are now first-class design constraints in A I fabric planning, not afterthoughts. If you're architecting a new A I cluster and you're not modeling your power topology at the same level of rigor as your network topology, you're going to have a bad time. Skyler: Story three — this one's from Cisco's blog, and the title is "Scale-Across: Why the Future of Distributed A I Isn't in One Datacenter." Now, Cisco publishing something like this obviously comes with a commercial motive — they sell the gear that would connect multiple datacenters — but the technical substance is worth taking seriously. Skyler: The argument Cisco is making is essentially this: the days of assuming your entire A I workload lives inside a single datacenter are numbered. There are a few forces driving this. Land and power availability near any given metro area is finite. Regulatory and data sovereignty requirements are pushing compute geographically closer to where data is generated. And the sheer scale of frontier model training is starting to exceed what any one facility can physically support. Skyler: So the "scale-across" framing is their way of describing distributed A I infrastructure that spans multiple physical sites — whether that's two buildings on a campus, multiple datacenters in a metro area, or genuinely geographically dispersed sites. And the challenge they're highlighting is the interconnect fabric binding all of that together. Skyler: Inside a single datacenter, you've got your RDMA fabrics — RoCE, InfiniBand — running at very low latency, sub-microsecond to low microsecond, with very high bandwidth. The moment you go across a dark fiber link between sites, your latency goes up, your jitter goes up, and your effective bandwidth for GPU-to-GPU communication takes a real hit. Distributed training algorithms are sensitive to that. Gradient synchronization across a high-latency link can seriously throttle your effective training throughput. Skyler: The interesting technical question is where the viable latency ceiling is for different workloads. Inference can often tolerate more latency than training. Certain parallelism strategies — pipeline parallelism versus tensor parallelism — have very different sensitivity profiles. What Cisco is really selling here is intelligent routing and traffic engineering across that wide-area fabric, and the ability to make the network transparent enough to the A I workload that the application layer doesn't have to be rewritten. Skyler: What I'd watch for here is how the emerging standards around wide-area A I fabric evolve. U E C — the Ultra Ethernet Consortium — is primarily focused on intra-datacenter right now, but the principles around congestion control, packet ordering, and low-latency transport are directly relevant to scale-across scenarios. And the vendors trying to own that inter-datacenter fabric layer are going to be in a very lucrative position as these multi-site clusters become the norm. Skyler: Story four, and this is where it gets really interesting — Digitimes has a piece analyzing what Nvidia and Broadcom's CPO bets actually reveal about where A I infrastructure is heading. We talked about CPO at a technology level in story one — now let's talk about the strategic dimension. Skyler: Nvidia and Broadcom are approaching co-packaged optics from pretty different positions, and that tells you something about how they each see their own competitive moat. Broadcom has been telegraphing CPO as a priority for its switch ASICs for a while. They're one of the dominant vendors in merchant switch silicon — Tomahawk, Jericho — and integrating optics tightly with those platforms is a way to extend their reach further up the value stack and make it harder for anyone to build a competing solution that just drops in a Broadcom chip. Skyler: Nvidia's angle is somewhat different. They come at this from the perspective of having an end-to-end A I infrastructure play with NVLink inside the server and InfiniBand for the scale-out fabric. Their interest in CPO is partly about making the ConnectX and the Quantum platforms more competitive as you push toward higher port counts and higher speeds, and partly about owning more of the optical layer that sits between their networking hardware and the broader datacenter. Skyler: What's telling is that both of them are making bets here simultaneously. That's not a coincidence — it reflects a shared read on where the industry constraint is. When two companies that compete in some dimensions and collaborate in others both independently prioritize the same technology, that's usually a strong signal the technology is transitioning from optional to mandatory. Skyler: The Digitimes piece also touches on the silicon photonics ecosystem question — who actually makes the photonic integrated circuits that go into CPO modules, and how concentrated is that supply chain. It's fairly concentrated right now. Intel has silicon photonics capabilities, there are a handful of merchant photonics players, and the foundry ecosystem for photonics is nowhere near as mature as for CMOS logic. That supply chain thinness is one of the reasons CPO timelines have historically slipped. Skyler: For datacenter architects and fabric engineers, the practical takeaway is this: your next-generation switch platform decisions are going to carry CPO assumptions embedded in them whether you explicitly plan for that or not. The lead times, the co-design requirements, and the serviceability model for CPO are fundamentally different from pluggables. Start thinking about that now, not when you're six months out from a deployment. Skyler: And story five — this one has a bit more color to it. Semivision is running a piece framed around Elon Musk's mesh optical deal and arguing that the next A I infrastructure war is going to be fought with light. There's obviously a narrative angle here, but let me pull out the actual technical thread. Skyler: What this is referencing is xAI's aggressive buildout of optical interconnect for Colossus — the Memphis supercluster — and Musk's reported interest in mesh optical architectures for linking clusters together at scale. The framing of "fought with light" is a bit dramatic, but it's not wrong as a directional statement. When you're trying to connect hundreds of thousands of GPUs across a campus or across multiple buildings and you need to do it with bandwidth in the terabits per second range, fiber is the only realistic medium. Skyler: The "mesh optical" framing is interesting because it implies something more flexible than a traditional point-to-point or hub-and-spoke fiber plant. A mesh optical architecture means you're building a reconfigurable optical network where traffic can be dynamically routed through the fiber layer without necessarily going back to a switching ASIC at every hop. Think ROADMs — Reconfigurable Optical Add-Drop Multiplexers — and optical switching fabrics that can steer wavelengths around the network based on traffic demand. Skyler: For A I training at scale, this matters because your communication pattern isn't static. Different phases of training — forward pass, backward pass, gradient aggregation — have very different traffic matrices. A network that can dynamically reconfigure its optical topology to match the traffic pattern in near-real-time is potentially much more efficient than one locked into a fixed wiring. This is sometimes called optical circuit switching, and it's been a research area for a long time, but the scale of A I workloads is finally making the business case compelling enough to invest in it seriously. Skyler: The competitive angle Semivision is highlighting is that whoever controls the optical layer at scale — not just the transceivers but the actual photonic switching and routing infrastructure — has a different kind of leverage than a traditional network vendor. You're not just selling ports, you're selling the physical topology. And that's much harder to rip and replace. Skyler: For the hyperscalers and large A I shops building at this scale, the question is becoming: do you buy an optical mesh solution from a vendor, or do you vertically integrate and build your own? Musk's xAI apparently leaning toward a more purpose-built approach is consistent with what we've seen from Google and Meta — at sufficient scale, the economics of rolling your own often win out over vendor solutions. The companies that should be worried are the traditional telecom optical equipment vendors who assumed the hyperscaler market was theirs to lose. Skyler: Alright, that's a wrap on today's stories. Optics is becoming the defining constraint in A I fabric scaling — both inside the rack with CPO and across the campus with mesh optical. Power infrastructure is the unglamorous but equally real bottleneck that's going to bite operators who haven't planned for it. And the perimeter of what we call "the datacenter" is expanding — scale-across architectures are going to be the norm, not the exception. A lot of light being thrown at hard problems this week, in every sense. Thanks so much for listening — Line Rate, brought to you by Ada and Zen Semiconductor. Ada helps organizations integrate A I into their workflows the right way, and Zen Semiconductor is an umbrella company with a portfolio of silicon-industry businesses. If you got value from this episode, subscribe wherever you get your podcasts, share it with a colleague who needs to know this stuff, and check the show notes for links to all the stories we covered today. I'll see you next time on Line Rate.