Network engineering

AI Compute Fabric Planning: Choose the Network Around the Workload

Select an AI compute fabric by mapping communication patterns, server attachment, operational skills and measurable workload outcomes.

The right compute fabric is the one that supports the intended application and can be operated reliably by the team responsible for it. A protocol choice or switch generation is only one part of that decision. Begin with workload behavior and server integration, then evaluate complete supported configurations.

Describe communication before comparing protocols

Identify whether the environment is dominated by synchronized training collectives, distributed simulation, inference services, storage transfer or a mixture. Record expected job sizes and concurrency. Separate communication inside the server from traffic that must cross the network.

An illustrative cluster running many independent inference services may have a different traffic pattern from one running a single tightly coupled training job across the same number of GPUs. Equal GPU counts do not create equal network requirements.

Count supported endpoints

Inventory the exact server and adapter variants, host-interface placement and required operating modes. Ethernet and InfiniBand support must be established for the specific adapter, not inferred from a broad family name. ConnectX documentation is a starting point for this host qualification. R51

Keep storage and management requirements visible. A compute-fabric decision should not leave the team without a recovery path or assume that every service uses the same protocol.

Compare complete operating models

Decision area	Questions for an Ethernet/RoCE design	Questions for an InfiniBand design
Host integration	Is the supported RDMA and congestion stack available?	Is the supported HCA and fabric stack available?
Fabric control	Who owns routing, queue policy and congestion behavior?	Who owns subnet management and fabric policy?
Operations	Can the team diagnose host-to-fabric behavior?	Can the team maintain and diagnose the chosen fabric?
Expansion	Are additional endpoints and paths qualified?	Are topology, management and interconnects qualified?

This is a requirements comparison, not a universal ranking. Use the relevant vendor deployment guidance for each candidate architecture. R08 R52

Design topology from job placement and growth

Calculate endpoint-facing and inter-switch ports separately. State oversubscription assumptions and show the capacity available after an agreed fault. A small single-stage design and a multistage design consume ports differently; scaling the server count without revisiting the fabric can invalidate the original assumptions.

Reserve capacity for a defined expansion step. “Future-proof” is not an engineering quantity; a documented plan to add a known number of racks and endpoints is.

Validate a representative configuration

Build a pilot that includes actual host hardware, software and representative interconnects. Measure the application’s relevant outcome alongside communication tests. For collective workloads, retain NCCL test settings and host placement rather than presenting an isolated bandwidth figure as a general cluster result. R11

The final recommendation should identify the supported configuration, expected operating responsibilities, expansion boundary and evidence needed for acceptance. A useful sourcing request includes server inventory, application framework, job size, rack layout and operational constraints. That information supports a fabric decision far better than a request for the fastest switch.

Engineering illustrations

These figures illustrate the design concepts discussed here. They are not measured results, certified configurations or a supplied-equipment list.