Apple Is Building AI Server Architecture That Cannot Scale Without Nvidia Networking - Tech Times

Skyler: Hey, welcome to Line Rate — the show that tracks what's actually happening at the silicon and standards layer of A I infrastructure. I'm Skyler Monroe, and if it moves bits between accelerators, racks, or datacenters, we're talking about it here. Big thanks to our sponsors: Ada, who helps organizations actually integrate A I into their real workflows, and Zen Semiconductor, an umbrella company with a portfolio across the silicon industry — including Ada for A I deployment. Alright, today we've got Apple's A I server plans running straight into Nvidia's network gravity, Celestica dropping a co-packaged optics switch that deserves more attention than it's getting, and Synopsys putting U E C and U A Link I P into the hands of chip designers everywhere. Plus, we're going to dig into a new security spec for Ultra Ethernet that quietly landed and matters a lot. Let's get into it. Skyler: So the first story is about Apple, and I want to be careful here because the framing in the press is a little sensational, but the underlying dynamic is genuinely interesting. The headline is that Apple is building A I server architecture that can't scale without Nvidia networking — and look, that's technically true but it's also true of basically everyone who isn't building their own fabric end-to-end. So let's unpack what's actually going on. Skyler: Apple has been quietly building out server infrastructure to support Apple Intelligence — their on-device and cloud A I stack. They're running what they call Private Cloud Compute, and they've published quite a bit about the security architecture. What's less discussed is the networking layer. And when you start asking how Apple scales that out across pods and racks, you run into the same wall everyone runs into: you need a high-performance fabric for the GPU or accelerator side of the workload. Skyler: Now Apple designs its own silicon — the M-series, the A-series, and presumably whatever they're running in their server nodes. They are genuinely world-class at chip design. But chip design and fabric design are two very different problems. Fabric is a full-stack discipline — it's the PHYs, the switches, the congestion control, the collective communication libraries, the whole system. Nvidia has spent years and billions building that out with NVLink and InfiniBand and now NVLink switches at scale. That's a moat. Skyler: What the reporting is pointing at is that for the scale-out side of Apple's A I compute — the part where you need to coordinate across many nodes to run large model inference or training — Apple would realistically be reaching for Nvidia's networking stack. InfiniBand or maybe Ethernet with Nvidia's SHARP offload. Because building your own scale-out fabric is an enormous undertaking. Google did it with TPU pods and their Jupiter network. Meta did it with their RoCE-based fabric. Microsoft is doing it with Azure's custom gear. But all of those are decade-long efforts with huge teams. Skyler: Here's why this matters for the people listening to this show. It illustrates a structural reality in the A I infrastructure market right now: the accelerator is only part of the story. The network is the other half of the system, and it's almost as hard to get right. Apple is one of the most sophisticated silicon companies on the planet, and even they are apparently not in a position to roll their own fabric at the scale needed for serious A I compute. That is a signal about how defensible Nvidia's networking position really is, separate from the GPU story entirely. Skyler: The counterargument is that Apple's workload is heavily inference-focused, runs at relatively lower parallelism than frontier model training, and their Private Cloud Compute architecture is specifically designed to keep workloads small and isolated for privacy reasons. So maybe they don't need the kind of 400 or 800 gigabit per second all-to-all fabric that a hyperscaler training run needs. But the reporting suggests they're planning to scale, and at some point scale means fabric. Worth watching. Skyler: Okay, moving on to something I'm genuinely excited about. Celestica just announced the DS6005 — a data center switch built with co-packaged optics, and they're positioning it explicitly for power-efficient A I fabrics. This is a product announcement, not a paper or a standard, but it's significant because it's a real shipping product in a space where a lot of the co-packaged optics conversation has been... let's say, forward-looking. Skyler: Let me back up for a second on what co-packaged optics actually means, because it's one of those terms that gets thrown around without enough grounding. Traditional switch architecture has the switch ASIC on a board, and then optical transceivers — the pluggable modules — sit in cages at the edge of the faceplate. The electrical signal has to travel from the ASIC across the PCB to those transceivers. At 400 gig and especially at 800 gig and beyond, that electrical trace is a significant source of power loss and signal integrity headaches. It also forces you to run a lot of SerDes at very high speed across that distance. Skyler: Co-packaged optics, or CPO, moves the optical engine right next to the switch ASIC — sometimes on the same package substrate. You eliminate most of that electrical trace. The signal from the ASIC goes a very short distance to the optical engine, and then it's photons from there to the fiber. The result is dramatically lower power per bit. We're talking potentially 30 to 50 percent lower power for the optical interface portion compared to pluggable. In a world where a 128-port 800 gig switch can have optical interface power measured in kilowatts, that is not a rounding error. Skyler: The DS6005 from Celestica is a 51.2 terabit switch — so we're in that top-of-rack or spine territory for high-density A I fabrics. And they're doing CPO to hit power efficiency targets that pluggable just can't reach at that port density and speed. Celestica is positioning this for A I fabric use cases specifically, which makes sense. A I training clusters are where the power and density constraints are most acute. Skyler: Now, Celestica is interesting here because they're an ODM — an original design manufacturer. They're not a pure silicon company, they're not a networking vendor in the traditional sense. They build hardware, often as white-box or branded gear for hyperscalers and large enterprises. So when Celestica puts out a product like this, it's a signal that CPO is maturing to the point where a sophisticated hardware integrator is comfortable productizing it. That's a different signal than a startup demo or a chipmaker showing off a reference design. Skyler: The competitive implication is real. Arista, Cisco, Juniper are all going to have to respond to CPO at the product level. Broadcom's Bailly switch ASIC is designed with CPO in mind. Marvell's been talking about it. Intel's Silicon Photonics group has been working the CPO angle for years. But Celestica shipping a product is a line in the sand. The efficiency argument for CPO in A I fabrics is now a product argument, not just a research argument. Skyler: The open question is ecosystem. Pluggable modules are universal — you buy a switch, you buy modules from multiple vendors, the market is liquid. CPO locks you into a tighter integration. Repair, replacement, and upgrades get more complicated. The industry is working on standards through bodies like the Co-Packaged Optics Alliance to address this, but it's a real consideration for anyone specifying a fabric today. Still, for power-constrained builds — and every A I datacenter is power-constrained right now — the efficiency case for CPO is getting very hard to ignore. Skyler: Alright, story three. Synopsys has introduced I P solutions for both Ultra Ethernet and U A Link. And this is one of those announcements that sounds like a chip company doing chip company things, but is actually a pretty meaningful signal about where these standards are in their maturity lifecycle. Skyler: Quick framing if you're new to these standards: Ultra Ethernet — or U E C, the Ultra Ethernet Consortium spec — is an Ethernet-based fabric designed specifically for A I and HPC workloads. It takes Ethernet's ubiquity and adds things like lossless transport, better congestion control, and collective communication support that standard Ethernet doesn't have. The goal is to give people an alternative to InfiniBand that doesn't require buying into a proprietary stack. The spec is still evolving — I want to be clear about that — but it's progressing and the consortium has real weight behind it with members like AMD, Cisco, Intel, Meta, Microsoft, and others. Skyler: U A Link is a different animal. It's a short-reach, chip-to-chip or accelerator-to-accelerator interconnect — think scale-up fabric inside a server or a tightly coupled pod. The U A Link Consortium is positioning it as an open alternative to NVLink for connecting accelerators within a node or a small cluster. Again, still evolving as a spec, but the consortium launched last year with AMD, Google, Intel, Meta, Microsoft, and Broadcom involved, which is a serious group of people who would all like to have an option that isn't proprietary to Nvidia. Skyler: So when Synopsys announces I P for both of these — silicon I P, meaning pre-designed, verified blocks that chip designers can license and drop into their designs — that's a maturity signal. EDA and I P companies like Synopsys don't go build I P for specs that are vaporware. They have to make bets on what their customers are going to be designing in the next one to three years, and those customers — chip companies, accelerator vendors, NIC makers — are apparently asking for U E C and U A Link I P. Skyler: What this does practically is lower the barrier to building silicon that speaks these protocols. Before Synopsys or Cadence or someone like them does this work, every chip team that wants to support U E C has to build the PHY and protocol logic themselves. That's expensive and time-consuming. Verified I P cuts that time significantly and reduces risk. It's a necessary step on the path to broad industry adoption of any interconnect standard. Skyler: The competitive read here is that this is another data point in the slow but real erosion of the "there's no alternative to Nvidia's stack" narrative. U E C and U A Link are the industry's answer to NVLink and InfiniBand, and they're backed by a coalition that collectively has more silicon design capacity than any single company. Synopsys putting I P behind them means the silicon for these standards is becoming more accessible. That's good for the ecosystem. Skyler: That said, I want to temper the excitement a little. I P availability doesn't mean ecosystem parity with NVLink or InfiniBand, which have years of production hardening and software stack maturity behind them. The software layer — collective communication libraries, runtime, MPI integration — is often where open standards struggle to match closed systems. But you have to start somewhere, and silicon I P is the right place to start. Skyler: Last story of the day, and this one is a little more niche but I think it's important: Ultra Ethernet Security, the spec being developed under the Ultra Ethernet Consortium designated as U E T dash T S S. Semiconductor Engineering ran a piece digging into what this is, and it deserves more attention than it's gotten. Skyler: So the core problem being addressed is this: when you build a fabric specifically for A I and HPC workloads, you're moving enormous amounts of data at very high speed between compute nodes. Traditional Ethernet security mechanisms — things like MACsec at Layer 2, or TLS at the application layer — either add latency and overhead that A I workloads can't tolerate, or they operate at the wrong layer of the stack to be effective in an RDMA or collective communication context. Skyler: The U E T-TSS spec is trying to define a security model that's tailored to the specific threat model of A I fabric. And the threat model is actually different from enterprise networking. In A I training or inference, you care a lot about things like: can a node in the fabric impersonate another node and inject bad gradients into a training run? Can someone sitting on the fabric exfiltrate model weights that are being shuffled around during a distributed compute job? Can you trust that the collective operation — the all-reduce, the all-gather — isn't being tampered with? Skyler: Those aren't threats that enterprise security models were built around, and plugging standard security mechanisms into an A I fabric without rethinking them can either leave you exposed or crush your performance. U E T-TSS is trying to thread that needle — define authentication, integrity, and confidentiality mechanisms that fit inside the latency and bandwidth budget of an A I workload. Skyler: The spec is apparently looking at things like per-message authentication, key management architectures that work at line rate, and protection mechanisms that can be offloaded to silicon without blowing up your latency budget. This is hard work. Line-rate cryptography at 400 or 800 gig per port, across thousands of ports in a fabric, is a significant engineering challenge. You're essentially asking for hardware-accelerated security that's transparent to the application. Skyler: Why does this matter for people building or specifying fabrics? Because enterprise and regulated customers — financial services, healthcare, government — are not going to run A I workloads on a fabric that doesn't have a credible security story. And hyperscalers doing multi-tenant A I compute need security guarantees at the fabric layer, not just above it. U E T-TSS is the U E C's answer to that requirement, and having a standard approach rather than every vendor doing their own proprietary security is genuinely important for interoperability and auditability. Skyler: The Semiconductor Engineering piece is worth reading in full if you want the technical depth — I'll put it in the show notes. The headline takeaway is that the Ultra Ethernet Consortium is not just thinking about performance. They're thinking about the full production deployment story, and security is a big part of that. It makes U E C a more serious contender for real enterprise and hyperscaler deployments. Skyler: Okay, that's the show. We went deep on Apple's fabric dependency problem, Celestica's very real CPO switch product, Synopsys bringing U E C and U A Link I P to the market, and the security spec that's trying to make Ultra Ethernet something you can actually deploy in a production environment you'd stake your reputation on. Lot of good stuff today. Line Rate, brought to you by Ada and Zen Semiconductor — Ada helps organizations integrate A I into their actual workflows, and Zen Semiconductor is an umbrella company with a portfolio across the silicon industry. If you got value out of this episode, please subscribe and share it with a colleague who cares about what happens below the application layer. Show notes with links to everything we talked about are in your podcast app. See you next time.