Google TPU: Ironwood and Cloud Providers' In-House Chip Roadmap

1. Introduction

1.1. Vendor Positioning

Google TPU (Tensor Processing Unit) is the representative of this group's "in-house development and use by cloud providers" roadmap: the TPU is developed by Google, deployed in its own data centers, and offered externally through Google Cloud as a service — it is not sold as a standalone chip. Google itself is the most important user — Gemini, Veo, Imagen, and Anthropic's Claude are all trained and inferred on TPUs (official statement).

The narrative thread of this article: a cloud provider's in-house development and use, and the originator of the "Age of Inference" narrative. When Google released Ironwood in 2025-04, it explicitly proposed the concept of the "Age of Inference," shifting the industry's focus from training to inference-side services — this narrative directly corresponds to the explosion of L1/L2 (context and tool execution) workloads in Harness's six layers.

1.2. Basic Information Card

1.3. Product Generations

CPU companion: Axion (Arm-architecture in-house CPU) forms Google's in-house compute combination with the TPU; the N4A instance is officially claimed to offer up to 2x better price/performance than comparable x86 VMs (as relayed from customer measurements: Vimeo core transcoding workloads +30%, ZoomInfo price/performance +60%).

1.4. Position in the AI Harness System

Google is the only vendor that is simultaneously a top-layer player in Harness (Gemini models, ADK agent framework, see the 02-AI-Agents group) and a bottom-layer compute player (TPU/Axion). The TPU stack's unique contribution to Harness's six layers lies in "integrated governance in a cloud-hosted form" (L6 fully carried by the GCP IAM/quota system) and the L2 openness brought by the "vLLM unified TPU backend."

2. Glossary

3. Functionality and Product Line

3.1. TPU Product Generations and Specs

See the generation table in Section 1.3. Spec data follows the Google Cloud official figures (cross-checked via xueshe compilation / telnyx / introl, A/B grade). MLPerf figures (MLCommons, A/B grade):

1. MLPerf Training 4.1: Trillium offers up to 1.8x the performance/dollar of v5p; multislice data center network cross-domain scaling efficiency of 99% (94% in a single ICI domain for v5p);
2. MLPerf Inference 5.0: Trillium's Stable Diffusion XL throughput is 3.5x that of v5e, at a cost of $0.22 per 1000 images (35% lower than v5e).

3.2. Ironwood: The First Inference-First TPU

Compiled based on Google's official blogs (the 2025-04 Next 25 announcement post and the 2025-11 GA announcement post, A grade):

Official performance comparison figures: 10x higher peak performance than v5p; over 4x higher single-chip training and inference performance than Trillium.

3.3. Axion CPU and AI Hypercomputer

1. AI Hypercomputer: Google's integrated supercomputing system (co-designed compute + network + storage + software); per the IDC report: average 353% three-year ROI for adopting customers, 28% IT cost reduction, 55% team efficiency gain (relayed figures);
2. Axion product portfolio: N4A (general-purpose VM preview), C4A, C4A metal (Arm bare-metal preview) — officially positioned as the general-purpose compute backbone for AI workloads (data preparation, retrieval, application serving);
3. GKE Cluster Director: advanced maintenance, topology-aware scheduling, and high-resilience cluster capabilities for TPU customers.

4. Platform Architecture

4.1. Pods Interconnect Topology

Figure 6-1 | Google TPU Pods Interconnect Topology: The Three Layers of ICI / OCS / Jupiter

Data sources: Google's official blogs (2025-04 Ironwood announcement post, 2025-11 GA announcement post), MLCommons MLPerf Training 4.1; the topology is a schematic illustration.

4.2. Software Stack

1. Pathways: Google DeepMind's ML runtime; distributed-compute orchestration across thousands of TPUs, which customers can use to combine hundreds of thousands of Ironwood chips;
2. MaxText: open-source LLM training reference implementation, newly strengthening training/reinforcement-learning techniques such as SFT and GRPO (generative reinforcement policy optimization);
3. vLLM TPU backend: officially announced strengthened support — developers need only minor configuration changes to switch between or run concurrently on GPUs and TPUs;
4. GKE Inference Gateway: intelligent load balancing among TPU servers; by official figures, up to 96% TTFT reduction and up to 30% service cost reduction.

5. Harness Design

5.1. Overview of Six-Layer Capabilities

5.2. L1 Context Engineering Layer

1. SparseCore: accelerates ultra-large Embedding; the Ironwood enhanced version extends the capability from AI to finance and scientific computing (official statement);
2. 192 GB per-card HBM3e: large-model KV Cache resident on a single card, reducing cross-card memory access;
3. JAX/XLA static-graph compilation: favorable for operator fusion and memory planning in large-batch inference — at the cost of lower flexibility than the PyTorch eager path for dynamic-control-flow workloads (a typical agent tool-calling pattern), so selection requires trade-offs.

5.3. L2 / L3 Tools & Orchestration Layer

The vLLM unified TPU backend (third parties cite a 2–5x improvement, C grade) significantly lowers L2 migration costs; L3's differentiation lies in cluster orchestration: the Multislice mechanism organizes training/inference across multiple Pods and even multiple data centers (MLPerf multislice scaling efficiency of 99%, A grade), and GKE Cluster Director provides topology-aware scheduling. MaxText supports SFT/GRPO, offering an open-source reference implementation for post-training and reinforcement learning (a source of agent capabilities).

5.4. L4 Memory & State Layer

The 1.77 PB shared HBM at the Super Pod level is the physical ceiling of state capacity; OCS optical switching instantly reorganizes the topology on failure, routing around interruptions to restore workloads — by official figures the service keeps running — which constitutes the core mechanism of L4 state continuity in the cloud-hosted form.

5.5. L5 Evaluation & Observability Layer

MLPerf's 99% multislice scaling efficiency can be regarded as third-party evidence of cluster stability and observability (A grade); runtime observability relies on GKE Inference Gateway's TTFT/cost metric system.

5.6. L6 Governance & Security Layer

The TPU is a cloud-hosted form (no authorization to build private clusters), so governance depends entirely on the GCP IAM, quota, and audit systems — an advantage for cloud-native enterprises (ready out of the box), but a hard constraint for private-deployment needs. On-prem TPUs are not yet broadly available; if the Meta negotiations materialize, they would become the first exception (B-grade reporting figure, see Section 6.2).

6. Case Studies

6.1. Anthropic: The One-Million-TPU Commitment

Background: In 2025-10/11, Anthropic announced access to up to 1 million TPUs with over 1 GW of capacity coming online in 2026, the largest TPU commitment in Google's history (A/B grade); previously there had been commitments of hundreds of thousands of Trillium chips.

Approach: Claude trains and infers across a multi-stack of NVIDIA GPUs + AWS Trainium + Google TPUs (cross-referenced with article 07: Anthropic is simultaneously training Claude on Project Rainier).

Official quote: Anthropic's head of compute James Bradbury said Ironwood's inference performance and training scalability help it scale efficiently and meet customer expectations for speed and reliability (quoted from the Ironwood announcement post).

Interpretation: Anthropic's "hardware-neutral multi-stack" strategy is a landmark event in this round of compute competition — leading model vendors are no longer betting solely on NVIDIA, but dynamically allocating across GPU / Trainium / TPU based on cost and availability.

6.2. Other Customer Cases

6.3. Conflicting Cost Figures Presented in Parallel

Alternating, conflicting figures exist regarding the TPU's cost advantage; per discipline, they are presented in parallel rather than picking one:

Conclusion: the TPU's cost advantage shows stronger evidence on stable, large-batch, XLA-compatible inference workloads; there is clear countervailing testimony on research agility and dynamic workloads. Before citing any single number, you must verify its workload assumptions.

7. Summary

Advantages:

1. System-level co-design: the cluster-orchestration capability (L3) built from OCS + Jupiter + Pathways is the strongest in this group;
2. Alignment of the Age-of-Inference narrative with products: Ironwood is the first inference-first TPU, and SparseCore/shared HBM/GKE Inference Gateway all point to inference workloads;
3. Cost evidence chain: MLPerf performance/dollar figures (Trillium 1.8x v5p) corroborate multiple customer cost-reduction cases.

Disadvantages:

1. Cloud lock-in: cannot be deployed privately (on-prem not available), and the XLA/JAX ecosystem is not friendly to dynamic-control-flow workloads;
2. Opaque pricing: no public list price, only a range of committed-use discounts;
3. Meta-level on-prem deals unannounced, leaving the roadmap with uncertainty.

Applicability boundary: teams already in the GCP ecosystem whose workloads are dominated by stable batch inference; XLA/JAX tech-stack teams; NVIDIA replacement/supplementary compute under a multi-stack strategy. Not suitable for strong private-deployment requirements or heavy dynamic-control-flow research workloads.

Selection recommendation: negotiate for a price range with committed-use discounts; first validate the target model's XLA compatibility on v6e before evaluating an Ironwood migration; treat "research iteration speed" as a first-class decision variable on par with cost (the Midjourney counterexample).

Information Gap Statement

1. No official price list for Ironwood / Trillium (Google does not publish list prices, only relayed committed-use discount ranges);
2. Official architecture documentation for Axion–TPU synergy is scattered across multiple blog posts, lacking a unified white paper;
3. The Meta–Google TPU deal has not been officially announced as of 2026-09 and can only be described as "negotiation/reporting";
4. Two figures exist for single-chip HBM bandwidth — 7.2 TB/s (official blog) and 7.37 TB/s (third-party compilation); this article follows the official blog figure of 7.2 TB/s.

8. References

1. Ironwood: The First Google TPU for the Age of Inference — Google official blog (Amin Vahdat), 2025-04-09. https://blog.google/intl/zh-tw/products/cloud/ironwood-the-first-google-tpu-for-the-age-of-inference/
2. Ironwood TPUs and New Axion-Based VMs (GA announcement) — Google Cloud Blog, 2025-11. https://cloud.google.com/blog/products/compute/ironwood-tpus-and-new-axion-based-vms-for-your-ai-workloads
3. The Diversification of AI Hardware: Beyond NVIDIA's Dominance — Ecitis (contains TPU generation specs and MLPerf data compilation), 2025. https://blog.ecitis.org/ai-hardware-landscape
4. Google TPU vs NVIDIA GPU: An Infrastructure Decision Framework — Introl, 2025. https://introl.com/blog/google-tpu-vs-nvidia-gpu-infrastructure-decision-framework-2025
5. TPU vs GPU Compared for AI Training and Inference — Telnyx Learn, 2025. https://telnyx.com/learn-ai/tpu-vs-gpu
6. Cloud TPU official documentation — Google Cloud, 2025. https://cloud.google.com/tpu/docs
7. AWS activates Project Rainier cluster of nearly 500,000 Trainium2 chips (includes cross-reporting on Anthropic's million-TPU expansion) — Data Center Dynamics, 2025-11. https://www.datacenterdynamics.com/en/news/aws-activates-project-rainier-cluster-of-nearly-500000-trainium2-chips/
8. MLPerf Benchmarks results library — MLCommons, 2025. https://mlcommons.org/benchmarks/inference/
9. Nvidia to Google TPU Migration 2025 (TCO model, D-grade analytical projection) — TinyComputers/baud.rs, 2025. https://baud.rs/4aoJ1v
10. Google Cloud releases its most powerful in-house TPU: a decade of R&D, 4x the performance of its predecessor — Cailianshe (财联社), 2025-11-06. https://www.cls.cn/detail/2193551