AMD: Instinct MI355X and the ROCm Open-Source Ecosystem

1. Introduction

1.1. Vendor Positioning

AMD is the only vendor besides NVIDIA with the complete combination of a high-end AI accelerator + CPU + open-source software stack. Its market role can be summarized as the challenger and open-source-ecosystem standard-bearer: on the hardware side it counters NVIDIA's proprietary interconnect with large memory and open standards (UALink); on the software side it counters CUDA's closed ecosystem with the open-source ROCm route. The narrative thread of this section is that AMD uses "openness" as its differentiating weapon against NVIDIA's "full-stack closure," but software-stack maturity and benchmark-completeness remain its clear weaknesses.

1.2. Basic Information Card

1.3. Financial and Market Position

1.4. Position in the AI Harness System

Like NVIDIA, AMD is the underlying compute provider for the inference/training-side Harness, but its differentiation lies in outsourcing the key L2/L3 implementations (vLLM, SGLang, etc.) to the open-source community rather than building its own closed loop — ROCm's role is to "make the open-source inference engines run just as fast." This makes the Harness capability ceiling of the AMD stack depend on how quickly the open-source ecosystem matures on the ROCm backend.

2. Glossary

3. Features and Product Line

3.1. Instinct MI350 Series

Key specifications and official performance framing (AMD official blog 2025-06; all vendor self-tested figures):

All of the above comparisons are vendor self-tested; when citing, you must note "AMD official framing."

3.2. ROCm 7 Software Stack

1. Released 2025-06, available Q3 2025; official claims of over 4x inference and 3x training vs ROCm 6.0; on MI300X, ROCm 6→7 averages 3.5x inference and 3x training;
2. Supports 1.8+ million Hugging Face models out of the box;
3. Adds ROCm Enterprise AI: an enterprise MLOps platform covering training/inference operations management — a key move for AMD to complete L5;
4. Open-source frameworks (vLLM, SGLang, TensorRT-LLM ROCm backend) have official adaptation, with performance improving as ROCm iterates.

3.3. Hardware Roadmap: MI400 and Helios

1. MI400 (2026): 432 GB HBM4, 19.6 TB/s bandwidth, FP4 40 PFLOPS;
2. Helios: a double-wide rack-scale system of MI400 + Zen 6 EPYC (Venice) + Pensando Vulcano NIC (2026) — AMD's direct answer to the NVL72 supernode;
3. MI500 series: planned for 2027;
4. Open standards: champions the open UALink interconnect, diverging in direction from NVIDIA's proprietary NVLink.

4. Platform Architecture

4.1. ROCm Software Stack Layering

Figure 2-1 | AMD ROCm software stack and open-source ecosystem layering

Data sources: AMD official blog (2025-06), Data Center Dynamics (2025-06); diagram based on this document's analysis.

4.2. Deployment Forms and Compatibility

1. Baseboard compatibility: MI350 is compatible with the MI300 UBB universal baseboard, letting customers upgrade without replacing racks — lowering cloud vendors' card-swap costs; a pragmatic strategy countering NVIDIA's "full-rack delivery";
2. Cooling flexibility: air cooling scales to 64 cards, direct liquid cooling to 96/128 cards;
3. Supply channels: shipped through Dell, HPE, Supermicro, and mainstream CSPs and Neocloud.

5. Harness Design

5.1. Six-Layer Capability Overview

Structural characteristic: the strong and weak layers of the AMD stack are precisely complementary to NVIDIA's — strong at "official adaptation speed of open frameworks" (L2), weak at "system-level state management and governance" (L4/L6). L4 is its most obvious weak layer.

5.2. L1 Context Engineering Layer

The 288 GB HBM3E per card lets a 520B-parameter model reside on a single card (official framing), significantly reducing cross-card KV sharding — a direct benefit for multi-turn conversational agents (whose context grows linearly with each turn); ROCm 7's FlashAttention and Transformer Engine support optimize long-context workloads.

5.3. L2 / L3 Tooling and Orchestration Layer

vLLM, SGLang, and TensorRT-LLM (ROCm backend) all have official adaptation; AMD's strategy is to hand inference orchestration to the open-source community: L1/L3 capabilities such as vLLM prefix caching and SGLang RadixAttention iterate on ROCm with each open-source release. This has a double-edged effect: broad adaptation coverage (shared with the vLLM ecosystem), but the responsibility for version combinations and performance tuning shifts to the user side.

5.4. L4 Memory and State Layer

KV management and distributed inference rely on the open-source stack; vLLM prefix caching and PD disaggregation are less mature on ROCm than on NVIDIA (synthesized judgment, Grade C) — this is the weak layer most important to make explicit to readers in this Harness Design chapter.

5.5. L5 Evaluation and Observability Layer

ROCm Enterprise AI (released 2025-06) is positioned as an enterprise MLOps platform covering training/inference operations management; because it has shipped recently, its observability depth (agent-scenario capabilities such as trace tracing and regression suites) has no public validation cases yet (Grade B, release framing).

5.6. L6 Governance and Security Layer

Governance-layer material is scarce: AMD's narrative is that "not being locked in" thanks to the open ecosystem (UALink open standard, ROCm open-source license) is itself a governance option; enterprise delivery can combine partner models such as Red Hat (Grade C synthesized judgment). The enterprise-grade security support system corresponding to NVIDIA AI Enterprise has no publicly documented equivalent depth.

6. Real-World Cases

6.1. Customer Cases

Case analysis: AMD's customer strategy is to "gain endorsement from cloud vendors and top AI companies, and lower switching costs through open standards" — OCI's zettascale commitment and OpenAI's involvement in the MI400 design show the challenger has already won second-source orders from the most demanding customers.

6.2. Conflicting Performance Claims Side by Side

There are two vendor framings for the GB200 vs MI355X comparison; per this group's discipline, they are presented side by side rather than picking one:

The two framings are incomparable on model (DeepSeek-R1 0528 vs Llama 3.1 405B), precision (framing not fully disclosed vs FP4), and system form (NVL72 supernode vs single-card comparison against B200). There is only one objective conclusion: each leads in the scenario each side chose as its most favorable, and independent third-party data for the same scenario is missing.

7. Summary

Advantages:

1. Large-memory differentiation: 288 GB per-card memory and the tokens-per-dollar framing hit the core "unit cost" demand of the inference era;
2. Open route: ROCm open source + UALink open standard + UBB baseboard compatibility keep switching costs far lower than CUDA lock-in;
3. High-value customer endorsement: the most demanding customers such as OpenAI, Meta, xAI, and OCI have formed second-source / co-design relationships.

Disadvantages:

1. Software-stack maturity: L4 (KV state management) and L6 (enterprise governance) are clear weak layers, and ROCm Enterprise AI is still new;
2. Benchmark gap: no official MLPerf v5.x submission records were found for this generation's MI355X, so verifiability trails NVIDIA (see the Information Gap Statement);
3. Market-share gap: third-party estimates put revenue share at roughly 5–7%, with both economies of scale and community investment still in a catch-up position.

Applicability boundary: teams that are cost-performance sensitive on inference and willing to take on the responsibility of tuning the open-source stack; as a second source to NVIDIA under a multi-vendor strategy; scenarios with very large memory needs (long context, large models resident on a single card).

Selection recommendation: use "tokens-per-dollar" rather than peak compute as the procurement decision metric; before going live, you must bench-test the target framework (vLLM/SGLang) performance on the target ROCm version; watch how the 2026 Helios rack-scale system changes the supernode competitive landscape.

Information Gap Statement

1. No MI355X result was found in official MLPerf submissions (AMD has submitted historically); before adoption, check the result database at mlcommons.org;
2. No specific deployment instances of ROCm 7 at domestic (Chinese) cloud vendors were found (public cases concentrate on overseas CSPs);
3. The official whitepaper for the Helios rack's rack-level interconnect topology (UALink/Infinity Fabric) was not obtained; rack-level specifications await official addition.

8. References

1. AMD Instinct MI350 Series GPUs: A Game Changer — AMD official blog, 2025. https://www.amd.com/en/blogs/2025/amd-instinct-mi350-series-game-changer.html
2. AMD launches Instinct MI350 GPUs, unveils double-wide Helios AI rack-scale system — Data Center Dynamics, 2025. https://datacenterdynamics.com/en/news/amd-launches-instinct-mi350-gpus-unveils-double-wide-helios-ai-rack-scale-system/
3. AMD Unleashes Open AI Ecosystem Vision at Advancing AI 2025 — Hardware Busters, 2025. https://hwbusters.com?p=76153/
4. AMD's Su-premacy Begins — Physical AI News, 2025. https://physicalainews.com/amds-su-premacy-begins
5. AMD Launches New Instinct GPU Series to Challenge AI Market — Tech Icons, 2025. https://techicons.com/latest-news/amd-launches-new-instinct-gpu-series-to-challenge-ai-market
6. ROCm official documentation — AMD, 2025. https://rocm.docs.amd.com/
7. ROCm/ROCm open-source repository — AMD GitHub, 2025. https://github.com/ROCm/ROCm
8. Nvidia Blackwell Statistics 2026 (third-party AMD share framing) — Axis Intelligence, 2026. https://axis-intelligence.com/nvidia-blackwell-statistics