Rapid scale-up in data center: The case for distributed optical circuit switching

AI infrastructure is moving from server-scale acceleration to multi-rack supernodes that must behave like a single, highly available computer. Distributed optical circuit switching offers a practical way to extend scale-up fabrics with lower latency, lower power, flexible topology control, and limited disruption to existing data center architecture.

AI data centers are entering a phase where the hardest problem is not simply building a faster accelerator. It’s keeping thousands of accelerators fed, synchronized, and available as one machine. Training frontier models, serving long-context inference, and running agentic workloads all create a similar demand: more devices must communicate over longer distances without allowing the network to dominate cost, latency, power, or reliability.

That changes the definition of scale-up. It’s no longer only the set of links inside a server or rack. Increasingly, scale-up must span multiple racks while preserving the low-latency behavior programmers expect from a tightly coupled system.

Nvidia has framed the issue in similar terms, noting that as AI factories reach “extreme scale,” networking infrastructure “must be reinvented to keep pace.” The company’s recent silicon photonics announcements focus on exactly the metrics now becoming critical in AI infrastructure: power efficiency, signal integrity, resilience, and deployment speed.

That emphasis reflects a broader industry shift. The bottleneck is no longer just whether a link can move bits from one endpoint to another. It’s whether the entire fabric can support large, tightly synchronized accelerator domains while reducing power, limiting failure points, and remaining practical to deploy and service.

Copper has been the default for scale-up because it’s familiar, low latency, and economical at short reach, but physics is becoming less forgiving as per-lane data rates rise. Longer copper paths increase insertion loss and signal-integrity burden; thicker wires help the signal but work against density, airflow, and manufacturability.

Rack-scale cable trays become difficult to assemble and service, and a failure may require replacing large mechanical units rather than a small module. Retimers, equalization, and more complex board design can extend the life of copper, but they do not remove the underlying reach-density-power tradeoff.

Traditional pluggable optics solve part of this problem. They move data farther with less distance penalty than copper and are widely deployed in scale-out networks. But a scale-up fabric is different from a scale-out Ethernet fabric. Scale-up traffic often has stricter latency, synchronization, and collective-communication requirements, and in emerging architectures may carry memory-semantic traffic rather than ordinary packetized network traffic.

In that environment, the raw specifications of an optical link alone are not sufficient. The fabric also needs deterministic paths, fast reconfiguration, high serviceability, and smaller, more containable failure domains.

Figure 1 Next-generation scale-up and scale-out solutions extend the limits of traditional electrical interconnect and pluggable optical modules. Source: Lightelligence

Co-packaged optics (CPO) is an important long-term solution, but it’s not a simple drop-in replacement for today’s data center designs. Bringing optical engines into or near the package changes the thermal, packaging, manufacturing, serviceability, and qualification model. Recent industry discussions around CPO focus heavily on these operational issues: how to manufacture high-yield optical assemblies, make optical interfaces field-serviceable, and manage light sources and redundancy at scale.

Nvidia’s recent Spectrum-X Ethernet Photonics announcements also show where early CPO commercialization is gaining traction: scale-out and scale-across AI-factory networking. Scale-up CPO will follow, but it asks a different question: how do we preserve the behavior of a tightly coupled compute domain while changing the physical medium underneath it?

This is where distributed optical circuit switching, or dOCS, becomes important. Rather than concentrating the switching function in precious rack space, a dOCS architecture distributes compact silicon-photonic switching elements close to servers, GPU trays, or XPU clusters. This improves compute density by eliminating the switch chassis.

The goal is to keep data in the optical domain from port to port through the switch path, reducing unnecessary optical-electrical-optical conversions and avoiding the power and cost burden of very large centralized electrical switch ASICs. In practical terms, dOCS treats the scale-up network less like a fixed cable plant and more like a configurable optical fabric.

Figure 2 A distributed optical circuit switch (dOCS) integrates the optical interconnect and switching functions into one compact module. Source: Lightelligence

The distinction matters. A centralized switch can become expensive, power-intensive, and operationally painful as scale-up domains grow. It can also create a large failure domain. A distributed optical switch breaks the switching function into smaller modules, shrinking the blast radius of any one failure and enabling more granular service.

In one described implementation, the dOCS module integrates optical interconnect and optical circuit-switching functions in a compact module built around silicon photonics, controller, driver, and receiver circuitry. The architecture is intended to support millisecond-level failover, including substitution of a hot-standby GPU when a device fails.

For AI workloads, the value of that reconfigurability is not only resilience, it’s also topology control. Different phases of training and inference stress the fabric differently.

Dense all-reduce operations, mixture-of-experts routing, retrieval, key value (KV)-cache movement, and pipeline-parallel execution do not all benefit from the same topology. A circuit-switched optical layer can expose topologies such as ring, mesh, or dragonfly and allow the cluster manager to adjust the fabric according to specific workload-parallelism requirements.

Recent dOCS-based supernode work has described real-time topology reconfiguration and elastic expansion beyond 500 GPUs in a single logical domain. The broader architectural point is more important than the exact number: the network should become a schedulable resource, not a static constraint.

This also helps explain why dOCS is especially relevant to certain architectures. A supernode is not just a cluster with a marketing label. It is a tightly interconnected group of GPUs or other accelerators that attempts to behave like one large compute unit. In that context, effective model FLOPs utilization depends on how much time accelerators spend computing rather than waiting.

Optical scale-up links can extend reach across racks, while circuit switching can create predictable paths for high-volume collective traffic. The result is not simply more bandwidth on a datasheet; it’s a path to higher sustained utilization.

The same argument applies to memory. AI systems are increasingly constrained by memory capacity, memory bandwidth, and the location of memory relative to compute. High-bandwidth memory (HBM) on the accelerator remains essential, but it’s finite and expensive. Long-context inference and agentic workloads amplify the pressure by generating large KV caches and preserving more state across interactions.

CXL is important because it provides cache-coherent connectivity for processors, memory expansion, and accelerators, enabling resource sharing with lower software complexity. But coherent memory fabrics still need physical reach. Optical PCIe/CXL links and reconfigurable optical switching can help move memory from a stranded local resource toward a pooled resource available across boards, servers, and racks.

The practical advantage of dOCS is that it creates an intermediate deployment path. Data center operators do not need to redesign every server, change every protocol, or wait for full CPO maturity before gaining optical reach and switching flexibility in the scale-up domain. A dOCS-based fabric can be designed to remain protocol agnostic at the physical layer, supporting multiple higher-level fabrics while changing the transmission medium and switching mechanism underneath.

That matters because AI data centers are heterogeneous. They include GPUs, CPUs, XPUs, switches, storage, memory expansion devices, and management controllers sourced from multiple vendors and refreshed on different schedules.

Figure 3 As shown in this example, the dOCS architecture offers a practical advantage by creating an intermediate deployment path. Source: Lightelligence

A sensible roadmap is therefore not copper versus CPO, or pluggables versus CPO. It’s a staged migration. Linear pluggable optics can address immediate reach and power issues with limited architectural change. Distributed optical circuit switching can add fabric-level reconfiguration, availability, and scale-up reach.

Near-packaged optics can shorten electrical traces and improve density. CPO optics can eventually move optical I/O directly beside GPUs or switch ASICs for the highest bandwidth density and lowest electrical loss. Each step moves optics closer to the compute, but each step should also preserve serviceability and operational practicality.

The AI infrastructure industry is learning a familiar lesson from earlier computing transitions: the winning technology is rarely the one with the most elegant device physics alone. It’s the one that fits into systems, software, manufacturing, and operations at the right time.

The dOCS architecture fits that requirement because it attacks a real bottleneck now. It extends scale-up beyond the practical limits of copper, avoids inefficiencies of repeated electrical conversion, supports flexible accelerator-fabric topologies, and improves system uptime by localizing failures and rapidly rerouting traffic or switching to standby resources when hardware fails.

The future data center will almost certainly use several optical technologies at once: pluggable optics, linear drive optics, near-packaged optics, CPO, optical I/O chiplets, CXL-enabled memory fabrics, and optical circuit switches. The case for dOCS is that it gives operators a rapid-deploy scale-up option while the rest of the optical roadmap matures.

For AI infrastructure, that may be the most important kind of innovation: not a clean-sheet replacement for the data center, but a way to make the next supernode larger, more efficient, more available, and easier to deploy than the last one.

Maurice (Mo) Steinman is senior VP and U.S. GM at Lightelligence. He has enjoyed a 40-year career in the semiconductor industry, working for such companies as Digital, Compaq, HP, Intel and AMD, where he held the title of Senior Fellow before joining Lightelligence. Steinman has expertise in SoC architecture, SoC interconnect, memory subsystems, and energy management.

Related Content

• Optical switches primed for test debut
• The advent of co-packaged optics (CPO) in 2025
• Building Optical Control Planes: Challenges and Solutions
• Where co-packaged optics (CPO) technology stands in 2026
• Overcoming interconnect obstacles with co-packaged optics (CPO)

The post Rapid scale-up in data center: The case for distributed optical circuit switching appeared first on EDN.