AI & HPC networks
Why does an H100 SuperPOD need four separate network schedules?
Keep compute, storage, service access and OOB management distinct before counting equipment.
A single list of 400G equipment hides the distinctions that matter in NVIDIA’s H100 SuperPOD reference. A useful purchasing brief identifies which workload each connection serves before it assigns switches, cables or management ports.
The short answer
Keep compute, storage, in-band management and out-of-band management in separate schedules. This H100 reference uses InfiniBand for both compute and high-performance storage; Ethernet management is not a substitute storage architecture.
Keep compute, storage, in-band management and out-of-band management as distinct planning roles in the referenced DGX H100 SuperPOD architecture. Conceptual illustration; equipment shapes are illustrative.
Official technical sources · Independent analysis by SwitchInfra
This is a documented design analysis, not a customer deployment claim. Candidate roles require verification for your configuration.
4 — network fabrics
NDR — compute fabric reference
InfiniBand — high-performance storage
Ethernet — management fabrics
What the technical reference establishes
The selected reference is the H100 SuperPOD V11 architecture, not a statement about every NVIDIA platform. It separates training communication, storage I/O and management duties even where equipment families recur.
For a comparable proposal, begin with a connection register carrying the network role at both ends. This prevents a storage uplink, management bond or BMC connection from disappearing into a generic count of server ports.
DGX H100 SuperPOD reference architecture
Compute — The reference describes an NDR InfiniBand fabric.
Storage — High-performance storage uses a separate InfiniBand fabric.
In-band — Ethernet carries cluster services and user-accessible connectivity.
OOB — A distinct Ethernet fabric reaches management interfaces.
Scope of the analysis: Independent selection analysis of official NVIDIA documentation. Proposed examples are not a deployed customer project or a SwitchInfra delivery.
Work through the engineering choices
Start with the documented constraint, then state the conditions for the proposed equipment role.
Separate the two InfiniBand purchasing decisions
Compute and storage are separate fabrics.
Design exercise: Proposed task: give every endpoint a compute or storage role.
Count accelerator communication ports independently from storage-facing ports. Record the intended fabric, rail and peer for each connection. Similar connector shapes do not make two ports interchangeable in an application design, and moving a connection changes the topology rather than merely its cable label.
For storage, request the workload’s read, write and checkpoint requirements alongside capacity. An array quotation without its server-facing and uplink design leaves a gap in the network BOM. Preserve distinct acceptance tests for data delivery and distributed application communication.
H100 SuperPOD network fabrics · H100 SuperPOD key components
QM9700 family
Compute or storage InfiniBand switch
Separate fabric quantities; confirm exact managed switch configuration.
Keep service access separate from device control
The reference distinguishes in-band and OOB Ethernet.
Design exercise: Proposed task: inventory services separately from BMC and switch management.
An administrator reaching a server through its normal operating system is exercising a different path from recovery through a management controller. List provisioning, repositories, monitoring and storage access against their intended networks. Then identify which services remain reachable during an operating-system or fabric failure.
Count infrastructure management endpoints beyond the compute nodes: storage controllers, switches and power equipment may also need ports. Record uplinks and access policy with the endpoint count. A spare copper socket is useful only if the intended management interface and operational path are supported.
SN4600C family
In-band Ethernet reference role
Management and service connectivity; not the NDR compute fabric.
SN2201 family
OOB management reference role
Count actual BMC and infrastructure management connections.
Convert the architecture into a site-specific schedule
The reference has component counts for defined scalable units.
Design exercise: reconcile a proposed partial build against the chosen reference size.
Do not divide a full-cluster switch count by the number of purchased servers and assume the remainder is a valid smaller fabric. Preserve the design’s rail and uplink structure, then identify deliberately unpopulated connections. Ask the integrator to document deviations and their consequences.
For each cable, quote endpoints, protocol, port mode, routed length and connector form. Add optics, patching, software and support as explicit items where required. Review the resulting schedule against rack space, power and service access before declaring the purchasing package complete.
Review four schedules, then reconcile the totals
A common model family may appear in more than one fabric. Keep quantities, ownership and acceptance evidence attached to the role so that one allocation cannot silently consume another.
DGX H100 SuperPOD reference architecture
Workload
Compute communication, storage throughput and management access requirements.
Connections
Per-fabric endpoint map, rails, uplinks, lengths and exact port modes.
Acceptance
Independent tests for application communication, storage I/O and recovery access.
Candidate models and selection boundaries
These are conditional model or product-family inquiries for the documented equipment roles. Confirm the required rate and complete channel before finalizing any line.
| Model / family and status | Role and condition |
|---|---|
| QM9700 family Conditional model / family inquiry | Compute or storage InfiniBand switch Separate fabric quantities; confirm exact managed switch configuration. H100 SuperPOD key components |
| SN4600C family Conditional model / family inquiry | In-band Ethernet reference role Management and service connectivity; not the NDR compute fabric. H100 SuperPOD network fabrics |
| SN2201 family Conditional model / family inquiry | OOB management reference role Count actual BMC and infrastructure management connections. H100 SuperPOD key components |
Official model or family references only. Exact SwitchInfra listings, offered configuration, supply and project compatibility have not been verified; request confirmation by model.
Continue in the relevant product family
Use the analysis to define the role before comparing catalog options.
Discuss this project
Carry the route, media, endpoint and loss requirements into the inquiry. Leave unknown quantities blank; reference port and fiber counts are not an equipment BOM.
Follow the next selection question.
Compare related decisions while keeping each source’s scope, hardware and link conditions separate.
Browse more engineering analyses →
BlueField-3: include management and recovery in the BOM
Separate data ports, OOB access, RShim and server power requirements.
Read the engineering analysisOfficial technical source
ConnectX-6 Dx: qualify the card at both interfaces
Compare OPN, PCIe resources and complete Ethernet channels for a 25G or 100G refresh.
Read the engineering analysisOfficial technical source
ConnectX-7: a 400G upgrade starts inside the server
Qualify the host slot, exact connector and current feature support before buying interconnects.
Read the engineering analysisOfficial technical source
Before you finalize the channel
Is H100 SuperPOD storage always Ethernet?
Not in this selected reference: its high-performance storage fabric is InfiniBand.
Can one switch count cover both IB fabrics?
Only if the design explicitly allocates every port; this reference separates the fabrics.
Does a reference BOM prove the proposed build is validated?
No. Site changes and offered configurations still require review and acceptance.
Are these models confirmed available?
No exact SwitchInfra stock or offered configuration has been verified here.
Trace the project facts and technical limits
Source check: 19 October 2026. Catalog status is stated per candidate.
Official sources read 19 October 2026. Project-specific engineering review, firmware support, exact offered parts and supply remain unconfirmed.