Beyond HBM: Samsung Targets the Next AI Battleground with Advanced Packaging and CPO - semivision

Skyler: Hey, welcome back to Line Rate — the show tracking datacenter fabric and A I high-speed interconnect. I'm Skyler Monroe, and if you care about the chips, standards, and systems moving data at ludicrous speed inside and between A I clusters, you're in the right place. Quick thanks to our sponsors: Ada, who helps organizations actually integrate A I into their real workflows, and Zen Semiconductor, an umbrella company with a portfolio of companies across the silicon industry. Alright, today we've got a packed show — co-packaged optics is everywhere, Samsung is making moves on advanced packaging, NVIDIA dropped a deep dive on NVLink, and Marvell is talking interconnect scaling. Let's get into it. Skyler: First up — Samsung. And this isn't a memory story, even though that's where most people's minds go when Samsung comes up in A I silicon conversations. This is about what Samsung is positioning as their next big play: advanced packaging and co-packaged optics. The SemiVision piece lays out the strategic logic pretty clearly. Samsung has been fighting hard in HBM — high-bandwidth memory — against SK Hynix, and it's been a rough couple of years for them there. Hynix got the Blackwell design wins, Samsung's been playing catch-up. So the question becomes: where does Samsung go to find differentiated value in the A I supply chain? And their answer, increasingly, is the package itself. Skyler: When we talk about advanced packaging in this context, we're talking about things like 2.5D integration — putting compute chiplets and memory on a shared silicon interposer — and eventually 3D stacking, where you're literally putting silicon on top of silicon. Samsung has their I-Cube and X-Cube packaging technologies, and they've been investing hard in expanding that capability. The argument is that as A I chips get more complex and the bandwidth demands between compute and memory keep climbing, the package becomes a first-class competitive surface. It's not just who makes the best die anymore — it's who can put the right dies together in the right way with the best yield. Skyler: But the co-packaged optics angle is where it gets really interesting for our audience. CPO — co-packaged optics — is about integrating optical components directly into the switch or compute package, rather than having pluggable optical modules hanging off the faceplate. The bandwidth density and power efficiency case for CPO is strong: you eliminate a lot of the SerDes loss you get with long PCB traces to a pluggable, you can run faster lane speeds, and you get much better watts-per-bit. Samsung sees CPO as a natural extension of their advanced packaging play because the manufacturing challenges — precision integration of heterogeneous components, thermal management, yield — are exactly in their wheelhouse as a packaging foundry. Skyler: The competitive read here is important. Samsung isn't just trying to be a memory supplier to NVIDIA or whoever wins the A I chip race — they're trying to be an indispensable part of the physical stack. If your CPO module is packaged by Samsung, if your interposer is from Samsung, you're now much more deeply tied into their ecosystem. It's a classic platform play, and it makes sense given how much the industry is thinking about who controls the integration layer. Watch Samsung's packaging business as its own competitive vector going forward — this is not a side project for them. Skyler: Okay, next story — NVIDIA dropped a pretty detailed technical blog post on NVLink, and even if you've been following NVLink for years, there's some useful framing in here about how NVIDIA thinks about scale-up networking inside what they're now calling A I factories. Let's unpack this. NVLink is NVIDIA's proprietary high-speed interconnect for tying GPUs together — this is the scale-up network, meaning it's for high-bandwidth, low-latency communication within a tightly coupled group of GPUs, as opposed to scale-out, which is your Ethernet or InfiniBand connecting racks and pods together. Skyler: With Blackwell, NVLink has taken a significant step up. We're talking NVLink 5, running at 1.8 terabits per second of total bandwidth per GPU. The new NVLink Switch — which is physically a separate chip in the NVLink Switch System — lets you build NVLink domains of up to 576 GPUs. That's a full NVLink fabric spanning multiple nodes, all sharing that low-latency, high-bandwidth interconnect. And the key technical point NVIDIA keeps making is that this is a flat, non-blocking fabric. Every GPU in that domain can talk to every other GPU at full bandwidth. No oversubscription. Skyler: Why does that matter so much for A I training? It comes back to how large model parallelism works. When you're doing tensor parallelism or pipeline parallelism across hundreds of GPUs, you are constantly doing all-reduce operations — collective communications where every GPU needs to share gradient data with every other GPU. The latency and bandwidth of that communication directly affects how fast your training step runs. If your interconnect is a bottleneck, you're leaving GPU utilization on the floor. NVLink's design is explicitly optimized to make those collectives as fast as possible. Skyler: The competitive framing here is also worth noting. NVIDIA is very deliberately drawing a line between scale-up — NVLink, proprietary, high-performance — and scale-out — where they play more nicely with InfiniBand and increasingly Ethernet. The scale-up domain is where NVIDIA has the most control, and frankly the most moat. No one else has 576-GPU NVLink domains. AMD has their Infinity Fabric, but the scale is nowhere near this. Google has their ICI interconnect inside TPU pods. But for third-party silicon trying to compete — this is one of the hardest walls to scale. NVLink is both a technical achievement and a lock-in mechanism, and NVIDIA is very aware of both functions. Skyler: The other thing worth flagging from the blog post is the NVLink Spine architecture — how you physically build out these large NVLink domains using switch trays in the rack. This is becoming infrastructure in the truest sense — it's not just a chip feature, it's a rack-level and facility-level design. When hyperscalers spec out their next-gen A I training clusters, NVLink topology is a fundamental input to the physical design of the building. That's a remarkable position for a single vendor's proprietary interconnect to occupy. Skyler: Alright, story three — and this one is probably my favorite of the week from a technical validation standpoint. Wiwynn, which is one of the major ODMs building servers and rack systems for hyperscalers, ran a live A I rack demo at OCP Summit using co-packaged optics. This is significant because it crosses a threshold that the CPO world has been working toward: it's not a lab demo of a CPO component, it's not a simulation — it's a full rack-level system, running A I inference workloads, with CPO as the optical interconnect technology. System-level proof, as the headline says. Skyler: Let's talk about why this milestone matters technically. CPO is a genuinely hard integration challenge. You're not just swapping out a pluggable module — you're rethinking how the optical interface is built into the package. Thermal management becomes much more complex because lasers and photonic components are sensitive in ways that pure CMOS logic is not. You have to solve for optical alignment at package assembly time and maintain it over the lifetime of the system under vibration and thermal cycling. The co-design between the switch ASIC and the optical engine has to be much tighter. These are not trivial problems. Skyler: So when Wiwynn demonstrates a live A I rack at OCP — in front of the hyperscaler engineers who are the actual customers — that's them saying: we've solved enough of these integration problems to build a real system. OCP Summit is specifically the venue where hyperscalers go to see what's production-ready versus what's a roadmap promise. Wiwynn running live inference on a CPO-based rack at that venue is a meaningful signal to the procurement teams in the audience. Skyler: The OCP angle is also strategically interesting. OCP — Open Compute Project — is where the hyperscalers do a lot of their collaborative pre-competitive work on hardware standards. The fact that CPO is getting serious OCP-level attention means the hyperscalers are actively trying to build an ecosystem around it rather than letting it be controlled by one or two vendors. If CPO gets fully standardized through OCP channels, that changes the competitive dynamics significantly — it means Wiwynn and other ODMs can source CPO components from multiple optical engine suppliers and build interoperable systems. That's the dream for the buyers, even if it's more complicated for the suppliers. Skyler: The broader CPO market trajectory also got a data point this week — Fortune Business Insights published market size projections for CPO in A I infrastructure out to 2034, and the numbers are substantial. I won't read out specific figures because market research projections in nascent technology spaces should always be taken with a grain of salt — the range of assumptions is enormous. But the directional story is clear: CPO is expected to go from early deployments today to a significant fraction of A I cluster optical interconnect within a decade. The Wiwynn demo and the market projections are telling the same story from different angles — this technology is transitioning from research project to production consideration. Skyler: Fourth story — Marvell published a piece making the case that scaling A I means scaling interconnects, and while this is obviously self-serving content from a company that sells interconnect silicon, the technical argument is worth engaging with on its merits. Marvell is in an interesting competitive position — they're one of the major players making custom silicon for hyperscalers, including A I accelerators and networking ASICs, and they have a significant CPO and optical DSP portfolio. Skyler: The core argument they're making is pretty straightforward: as A I models get larger and training clusters scale to hundreds of thousands of accelerators, the interconnect doesn't just have to scale in bandwidth — it has to scale in a way that keeps latency bounded and power consumption manageable. Every generation of A I scaling has hit an interconnect wall at some point, and the question is always whether the interconnect roadmap can keep pace with the compute roadmap. Marvell's answer, unsurprisingly, is that you need purpose-built interconnect silicon — not general-purpose networking gear repurposed for A I. Skyler: What I find technically interesting in their framing is the emphasis on what I'd call the three-layer interconnect problem. You have scale-up — the tight, low-latency fabric inside a training pod, like NVLink or U A Link territory. You have scale-out — connecting pods and racks, where Ethernet and InfiniBand are competing. And then you have what some people are starting to call scale-across — long-distance, potentially multi-datacenter connectivity for very large distributed training jobs. Marvell is making the case that each of these layers has different requirements and needs purpose-built solutions. Skyler: The competitive subtext here is that Marvell wants to be in all three layers. They have their Teralynx switch ASICs for scale-out Ethernet, they have optical DSPs and CPO components for the physical layer, and they're doing custom A I accelerator silicon for hyperscaler clients. That's a pretty broad bet on the interconnect stack. The risk is that it's also a broad surface area — NVIDIA owns scale-up, Broadcom is extremely strong in scale-out switching, and the custom silicon space is crowded with internal hyperscaler designs. Marvell has to execute across all of these simultaneously. Skyler: One more thing worth pulling out of the Marvell piece — they talk about the importance of U E C, the Ultra Ethernet Consortium, as the framework for evolving Ethernet to meet A I workload requirements. U E C is still a work in progress — the specs are still being finalized, the full interoperability story isn't there yet — but the direction is clear. The A I industry wants Ethernet to be a viable alternative to InfiniBand for scale-out, because Ethernet means competitive supply chains and open ecosystems. Marvell is a U E C member and they have obvious interest in that standard succeeding. But the technical gaps between where Ethernet is today and what A I collectives need in terms of congestion control and latency are real, and U E C is the industry's attempt to close them. Skyler: Alright, let's bring it home. This week was basically a CPO week with a side of scale-up networking, and that's not a coincidence — these are the two biggest active battlegrounds in A I infrastructure right now. On the CPO side, you've got Samsung making strategic packaging moves, Wiwynn providing system-level validation at OCP, and market projections showing serious long-term growth. The technology is real and it's moving from labs to racks. On the scale-up side, NVIDIA's NVLink deep dive is a reminder that their proprietary interconnect is one of their most durable competitive advantages — 576 GPUs in a flat, non-blocking NVLink domain is not something you replicate quickly. And Marvell's interconnect scaling piece ties it all together with the argument that the three-layer interconnect problem — scale-up, scale-out, scale-across — is the defining infrastructure challenge of the A I era. Skyler: That's it for this episode of Line Rate. Thanks for listening. Line Rate, brought to you by Ada and Zen Semiconductor — Ada helps organizations integrate A I into their real-world workflows, and Zen Semiconductor is an umbrella company with a portfolio across the silicon industry. If you found this useful, please subscribe wherever you get your podcasts, and share it with a colleague who should be paying attention to this space. Full show notes with links to everything we talked about today are available — links are in the episode description. I'll see you next time.