AI Infra Group Overview

1. Group Positioning: The Physical Foundation of the L2 Execution Layer

1.1. Full-Series Landscape

AI Infra (artificial intelligence infrastructure) is the physical and software foundation that carries large-model training and inference. Within the AI Harness six-layer capability model, this group covers the physical foundation of the L2 execution layer: chips and supernodes determine the physical upper limit of what an agent "can do", training and inference frameworks are the scheduling hub of L2/L3, and operations and observability correspond to L5/L6 (evaluation, observation and governance).

This group is written in a full-series format, organized around five stages: "chip → supernode → cluster interconnect and data center → training-inference framework → operations". The authoritative endorsement for this main line comes from the judgment of Huawei Connect 2025: "Supernodes have become the dominant product form and are becoming the new normal for AI infrastructure construction", and "a supernode is in fact a computer that can learn, think and reason — physically composed of multiple machines, but logically a single machine" (A-level, official Huawei).

As of end-2025, 42 10,000-GPU-scale AI computing clusters had been built nationwide; the national intelligent computing capacity (FP16) stood at 1590 EFLOPS at end-2025 and 1882 EFLOPS at end-March 2026 (per MIIT data, B-level). In 2025, China's data-center electricity consumption was 1960 TWh (up +18.1% year-on-year), expected to exceed 5000 TWh by 2030 (CAICT methodology, B-level). The simultaneous expansion of scale and energy consumption determines that every direction in this group must make an engineering trade-off between "computing fast" and "computing affordably".

Figure 0-1 | AI Infra Full-Series Stack and Harness Six-Layer Mapping

Schematic: drawn based on the analysis of this group's documents; the six-layer definitions are quoted from the AI Harness parameter card v1.1.

1.2. Relationship with the 04-Hardware R&D Group: A Closed Loop from Design to Operation

The existing document AI Infra (Design) already covers the design side of AI Infra (the full GB200 NVL72 hardware specification table, Moore Threads Kua'e, domestic 10,000-GPU landscape shipment data, and the "AI designing AI chips" flywheel such as AlphaChip/PrefixRL). This group covers the full series on the operational side: chip roadmaps and operating parameters, supernode operational forms, cluster interconnect and data-center operational data, training and inference frameworks, and operations fault tolerance.

The two groups together form a "design → operation" closed loop: packaging on the design side (CoWoS, HBM) determines the memory ceiling on the operational side; the failure profile on the operational side (Llama 3's 419 unexpected interruptions, of which GPU-related ones account for 58.7%) in turn drives reliability targets on the design side. When design-side details are involved, this group's documents uniformly cross-reference "see 02-行业赋能/04-硬件研发组" rather than elaborating again.

1.3. Functional Positioning Comparison of the Four Directions

1.4. Mapping of Directions to the Harness Six Layers

The table below is the unified mapping baseline used across this group (direction documents must not rewrite it on their own when citing):

2. Common Challenges and Countermeasures

2.1. Scale: Faults Change from Exception to Norm

At scales beyond 10,000 GPUs, hardware faults are no longer exceptional events. The 54-day snapshot of Meta Llama 3 405B training shows: of 466 job interruptions, 419 were unexpected — roughly once every 3 hours; GPU-related causes account for 58.7% of unexpected interruptions (A-level, official technical report). The group-wide countermeasure is a unified tolerance approach — degrading faults from global events to local events: MegaScale's five-step pipeline of "heartbeat detection → self-check → isolation → replacement → checkpoint recovery" (arXiv 2402.15627, A-level), Moore Threads' DP-group-level isolation and zero-interruption standby-machine insertion (B-level, vendor account), and Llama 3's combination of asynchronous checkpointing with hot-standby nodes. See 04-training-framework.md and 06-ops.md.

2.2. Energy: PUE Policy Red Line and Liquid Cooling

The "Special Action Plan for Green and Low-Carbon Development of Data Centers" (2024-07) from four national departments including the NDRC clearly states: by end-2025, the PUE of newly built and expanded/rebuilt large and super-large data centers must not exceed 1.25, national hub node projects must not exceed 1.2, and green power must account for over 80% of newly built national hub data centers (A-level, policy original text). This means power distribution and cooling are no longer "post-hoc problems" for the data-center discipline, but pre-constraints on chip selection and supernode form — Atlas 950 SuperPoD adopts cross-cabinet all-optical interconnect, and Vera Rubin NVL72 uses a "cable-less, water-pipe-less, fan-less" midplane design, both partly driven by the pressure of cooling and cabling. See 03-cluster.md.

2.3. Interconnect: A Standards Race with Multiple Parallel Paths

Scale-up interconnect is in a three-line competition: NVIDIA NVLink (closed + NVLink Fusion semi-open), the UALink consortium (open, released 2.0 in 2026-04 with added In-Network Compute), and OCP ESUN/SUE (Ethernet-based, with Broadcom Tomahawk 6 as the chip foundation). Domestically, EthLink (ByteDance), Eth-X (Tencent), OISA (China Mobile) and Huawei Lingqu (UnifiedBus) 2.0 open specification (A-level, HC2025) are advancing in parallel. This group's documents take a unified stance: each path is annotated with one of three states — "mass-produced / specification published / planned" — and the term "commercially scaled" is not used to describe paths lacking native silicon (such as UALink, which as of 2026-09 still transitions over 800GbE bridging). See 03-cluster.md.

2.4. Localization: System-Level Innovation under Process Constraints

Huawei made a clear judgment at HC2025: "China's semiconductor manufacturing process will lag behind for a long time, and sustainable compute can only be based on actually available process nodes" (A-level). Under this constraint, the common choice of the domestic route is to compensate for single points with systems: CloudMatrix 384 interconnects 384 Ascend 910C chips over a UB network, using system-dense BF16 of about 300 PFLOPS to benchmark against rack-level competitors; Ascend advances on a "one generation per year" roadmap (950PR/950DT → 960 → 970), and develops its own HiBL/HiZQ HBM and HiF8/HiF4 data formats. This group uniformly requires: domestic parameter citations must state whether they are paper measurements (e.g. arXiv 2506.12708) or launch-event statements, as the two carry different evidence levels.

2.5. Measurement: Three Core Metrics

Consistent with and inherited from the conclusion of 04-硬件研发组/05-ai-infra.md: MFU (model FLOPs utilization), ETTR (effective training time ratio) and the proportion of effective training time are the unified landing points for all efficiency conclusions in this group. On the inference side, TTFT/TPOT/cost per token are added. Any vague claim of "performance improvement" or "efficiency gain" is forbidden within this group's documents and AGENTS.md; performance figures must state their basis (official release / MLPerf / third-party measurement / vendor claim) and hardware generation.

3. Document Navigation

Suggested reading order: new members follow 01 → 02 → 03 → 04 → 05 → 06 bottom-up; readers with an operations/SRE background can enter in reverse starting from 06-ops.md.

4. Summary

This group's position in the AI Harness system is the physical foundation of the L2 execution layer: chips and supernodes determine the physical upper limit of what can be done, training and inference frameworks are the scheduling hub of L2/L3, and operations take on L5 observation and L6 governance. Together with the 04-硬件研发组, it forms a "design → operation" closed loop. Three main threads run through the whole group: first, the normalization of faults makes fault-tolerant orchestration (degrading faults from global to local) the primary engineering problem; second, the PUE red line (1.25/1.2) and the liquid-cooling transition make energy a pre-constraint on the same level as compute; third, the multi-path standards race in interconnect and system-level localization innovation make "basis annotation" (A/B/C levels, official/measured/claimed) the infrastructure of this group's document credibility.

5. References

1. Leading a new paradigm for AI infrastructure with pioneering supernode interconnect technology (Keynote speech of Eric Xu at HC2025) — Huawei, 2025-09-18. https://www.huawei.com/cn/news/2025/9/hc-xu-keynote-speech
2. Serving Large Language Models on Huawei CloudMatrix384 — Huawei/SiliconFlow, arXiv 2506.12708, 2025-06. https://arxiv.org/pdf/2506.12708
3. MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs — ByteDance, arXiv 2402.15627, 2024. https://arxiv.org/abs/2402.15627
4. The Llama 3 Herd of Models — Meta, 2024. https://arxiv.org/abs/2407.21783
5. MLPerf Training v5.0 results release — MLCommons/PR Newswire, 2025-06-04. https://prn.to/3Zlln3j
6. NVIDIA Vera Rubin NVL72 product page — NVIDIA, 2026-03. https://www.nvidia.com/en-us/data-center/vera-rubin-nvl72/
7. Research on key technologies and industrial development trends of supernodes — Chinese Society of Thermophysics (repost), 2025-2026. https://www.zgcsswdx.cn/info/24724.html
8. Special Action Plan for Green and Low-Carbon Development of Data Centers — four national departments including the NDRC, 2024-07. https://www.ndrc.gov.cn/
9. Data-center compute scale and energy-consumption data (policy-interpretation recount, per MIIT/CAICT methodology) — interpretation of the <Implementation Plan for Promoting the Coordinated Green and Digital Transformation (2026—2030)> by seven departments including the Central Cyberspace Affairs Commission, 2026-09.
10. AI Infra (design side, cross-reference) — 02-行业赋能/04-硬件研发组/05-ai-infra.md, 2026-09-12.