AI4S · Scientific Discovery

1. Introduction

1.1 Background

AI for Science (AI4S, AI-driven scientific research) is the direction with the highest scientific return on AI agent technology, and also the one most easily overhyped by narratives. On one hand, protein structure prediction since 2018, and weather foundation models and materials discovery since 2023, mark the first time AI has reached or surpassed the level of traditional methods on several core scientific tasks; on the other hand, autonomous "AI Scientist" systems have been independently evaluated and found to exhibit novelty misjudgment and numerical hallucination, and AI weather models have been shown to systematically underestimate extreme events—an AI4S without a strict validation layer is an amplifier of hallucination.

Policy is likewise accelerating. In 2023-03, the Ministry of Science and Technology and the National Natural Science Foundation of China launched the "AI-driven scientific research" special deployment, focusing on key areas such as drug R&D, gene research, biological breeding, and new materials R&D, advancing algorithm-model innovation, open sharing of scientific research data, and computing-platform construction, while supporting the heterogeneous integrated development of high-performance computing centers and intelligent computing centers. In 2026-05, the China Meteorological Administration launched the second AI weather forecast model demonstration program and released meteorological industry standards such as the Verification and Evaluation of Forecast Performance of AI Weather Forecast Models—"verification and evaluation" has become a precondition for operational admission, which is highly isomorphic with the role of L5 in the six-layer Harness model.

This direction has a fundamental difference from the other five directions that is worth declaring up front: here the ground truth is not a human-annotated dataset, but physical laws, experimental verification, and conservation-law constraints. A SQL query can be compared, a review's citations can be checked, but a crystal structure proposed by AI can only be adjudicated by physical laws and experimental synthesis.

1.2 Definition

The AI Harness for the scientific discovery direction refers to the engineering support layer that, around hypothesis generation, literature-evidence assembly, simulation and experiment planning, result verification, and conclusion grading, provides agents with domain-context assembly, simulation and experiment tool contracts, lab-in-the-loop orchestration, hypothesis and failure-record memory, and an evaluation basis centered on physical laws and experimental verification.

It does not replace the judgment of domain scientists, does not replace experiments themselves, and does not replace peer review. It carries the layer that "constrains each of the agent's scientific claims with verifiable evidence."

Three boundary points need to be clarified:

• The AI4S Harness is not a "machine that automatically produces papers." Independent evaluations of autonomous scientific discovery systems have shown that unvalidated automatic output carries errors in a respectable form (see section 3.3).
• The AI4S Harness does not adjudicate experiments. The execution authority for physical experiments and clinical validation rests with humans; agents are responsible for hypotheses, design, and interpretation.
• The AI4S Harness has a distinctive evaluation-cost structure: cheap verification (conservation laws, convergence, comparison against existing models) can be performed at high frequency, while expensive verification (experimental synthesis, clinical trials) must be human-in-the-loop and triggered on demand.

1.3 Positioning in the AI Harness System — L5 Is the First-Order Problem

图 1-1｜AI4S 六层能力模型：L5 第一性问题与 L3 lab-in-the-loop 编排

数据来源：基于本文分析绘制的示意图。

The emphasis of the scientific discovery direction in the six-layer capability model is as follows (this emphasis analysis is based on public case facts, not a standard):

The core lies in L5 and L3.

L5 is a first-order problem because all of the value in this direction is ultimately adjudicated by "whether it withstands physical laws and experiments." The verifiable forms of ground truth include:

• Physical constraints: conservation of energy, mass, and momentum; convergence; comparison against analytical solutions or mature physical models (e.g., ECMWF HRES / ENS in meteorology).
• Experimental verification: synthesis confirmation by independent laboratories (e.g., the 736 external independent syntheses of materials predicted by GNoME), or clinical or statistical significance (e.g., survival analysis in target evaluation).
• Reproducibility: the same result can be recomputed or reproduced by independent teams on different setups (corresponding to the ACM four-tier reproducibility definition).

The counterexample warning likewise comes from the absence of L5: Sakana's AI Scientist was found by an independent evaluation to misjudge existing concepts as novel and to produce hallucinated numerical results in 4 of 7 manuscripts (per public reporting). Without strict L5, the larger the scale of AI4S output, the greater the accumulated hallucination.

L3 is core because the field is converging on the lab-in-the-loop architecture: agents are responsible for hypothesis generation, experiment design, and data interpretation, while human researchers are responsible for posing questions, judging which AI outputs to adopt, and executing physical experiments. The instance in Stanford's virtual CSO system where "the science-review agent pointed out that a ligand–receptor inference lacked spatial context, thereby triggering spatial transcriptomics analysis" (see section 3.3) demonstrates the value of self-criticism among agents as an orchestration mechanism.

Bottleneck layer: L5's cost structure. Cheap verification (conservation laws, convergence, comparison against existing models) can be automated and executed at high frequency; expensive verification (experimental synthesis, wet experiments, clinical) cannot be run at high frequency and must be human-in-the-loop. Designing a tiered evaluation system of "surrogate metrics first, ground-truth calibration periodically" is the greatest engineering challenge for Harness implementation in this direction.

1.4 Value and Bottlenecks

Value is reflected in three places:

• Expanding the hypothesis space: models can systematically propose hypotheses among hundreds of millions of candidates (protein structures, crystal structures, drug–indication pairings), freeing humans from enumeration labor.
• Compressing cognitive labor: FutureHouse's Robin system is described by the institution as able to complete in one day part of the literature review and hypothesis generation work that human researchers would originally need months to do; another report claims it compressed the cognitive labor of a discovery cycle from an estimated 872–937 person-hours to under 2 hours (the two figures have different calibers; see the information gap statement).
• Turning verification standards from verbal into automatic: conservation-law checks, convergence determination, and citation verifiability—steps that originally depended on the quality of reviewers—can be solidified into gates that every output must pass.

Bottlenecks are likewise threefold:

• Novelty misjudgment and numerical hallucination: AI Scientist's 4/7 problem rate shows that the stronger the generative capability, the more dangerous the unvalidated output.
• Out-of-distribution failure: AI weather models systematically underestimate record-breaking extreme events, and the gap grows with extremity (see section 3.2)—the tail of the training distribution is precisely the most scientifically valuable region.
• Verification cost and the trust gap: expensive verification cannot be run at high frequency, making the "AI proposes, humans select" step the bottleneck; the controversy over GNoME's "tenfold" claim is an embodiment of this gap (see section 3.3).

2. Glossary

3. Case Studies

3.1 Case Study 1: Protein Structure Prediction — From CASP to Programmable Design

3.1.1 Background

Protein structure prediction is the first "theorem-level" success scenario of AI4S: a clearly defined task (from sequence to structure), a mature evaluation system (the CASP competition and the GDT metric), and a ground truth determined by experiment (X-ray / cryo-EM structures). In 2018, AlphaFold first demonstrated deep learning's potential on this task at CASP13.

3.1.2 Approach

The key milestones from 2020–2024 are as follows (multiple sources cross-agree; source grades are mainly A/B, see references):

As for adoption scale, per public reporting, 2 million researchers across 190 countries have used AlphaFold 2 or 3 (grade B source).

3.1.3 Results

This case establishes a path of the greatest reference value for Harness design—evaluation first, generation follows, open source amplifies:

1. The CASP competition provided physical-level ground truth: predictions were directly compared against experimentally determined structures, with no dispute over winners. This is exactly the ideal form of the L5 evaluation layer—first a trustworthy criterion, then model competition, and finally narrative convergence.
2. Open source and database-ification amplified marginal value: RoseTTAFold's open source and AlphaFold DB's coverage of 200 million structures let downstream researchers consume the results without training their own. In Harness terms: model results only become scientific productivity once solidified into searchable, reusable artifacts (L4).
3. The leap from prediction to design depends on a feedback loop: RFdiffusion's designs must be manufactured and validated in the lab—the closed loop of generation (L2/L3) and experimental verification (L5) is what makes a complete scientific discovery cycle.

Boundaries that must be stated honestly: AlphaFold 3's full model was not entirely open-sourced, and the community has discussed its reproducibility and fair use; "at least 50% improvement in accuracy" is the paper's conclusion against existing methods, limited in scope to the interaction types it evaluated. These boundaries do not affect the conclusions above, but should be preserved when citing.

3.2 Case Study 2: AI Weather Foundation Models — Surpassing Numerical Forecasting and Systematically Underestimating Extreme Events

3.2.1 Background

Medium-range weather forecasting is the field in AI4S where "AI first challenged, on a routine task, equations that physicists wrote for a century." Since 2023, models such as Pangu-Weather, GraphCast, FourCastNet, FuXi, and Fengwu have been published one after another, and reported to surpass operational numerical weather prediction in both accuracy and speed.

3.2.2 Approach

A chronology of key facts (multiple sources cross-agree):

Critical content (must be presented alongside the success narrative): according to research related to Science Advances, the major AI weather models systematically underestimate record-breaking heat, cold, and wind events, and the more extreme the event, the larger the gap; among them, AIFS especially underestimates storm peak wind speeds because its MSE training objective tends to smooth the atmospheric gradients that produce sharp peaks. The conclusion: physical models still prevail in the tail of the distribution—where extreme events live.

A deeper structural dependency must also be declared: AI weather models rely on ERA5 reanalysis data for both training and inference, and ERA5 is itself the output of physical models plus data assimilation (4D-Var). If the traditional physical models and assimilation pipeline were abandoned, AI models would lose the strictly structured initial conditions they need for training and routine inference. A pure AI model is essentially a highly complex pattern-matching interpolator that lacks a structural understanding of conservation laws and could, in principle, produce physically impossible atmospheric states when pushed beyond the boundary of its training distribution. The frontier direction is a deep hybrid of PINN and differentiable solvers with machine-learning parameterization schemes.

3.2.3 Results

This case yields three structural conclusions for AI4S (and indeed for all data-driven scientific directions):

1. Benchmark superiority does not equal task superiority: GraphCast beating HRES on 90.3% of verification targets is a fact; but extreme events fall precisely in the remaining tail, and the tail is where forecast value is highest. "90% superior" and "systematically underestimating extremes" are two sides of the same set of facts and must be stated together.
2. Training-data dependence sets the capability ceiling: the ERA5 dependency means AI weather models are methodologically not substitutes for physical models but consumers of them. The China Meteorological Administration's "effective supplement" positioning, and its use of verification-evaluation standards as an operational admission threshold, is a direct expression of Harness L5 thinking at the industry-governance layer.
3. The emergence of industry standardization marks the direction's maturity: the 2026 meteorological industry standard turned "verification and evaluation" into an admission condition, which equals turning L5 from a recommendation into a mandatory gate—a governance path all AI4S subfields can follow.

3.3 Case Study 3: Testing the Mettle of Autonomous Scientific Discovery Systems — Success, Controversy, and lab-in-the-loop

3.3.1 Background

In 2024–2026, "AI Scientist"-type systems entered a period of dense releases: Sakana AI's The AI Scientist, FutureHouse's Robin and Kosmos, Google's co-scientist, Stanford's virtual CSO, ChemAgents, Lila Sciences, and others. There is a significant gap between the field's promotional narratives and its actual mettle; independent evaluations and scholarly criticism provide valuable calibration data.

3.3.2 Approach

First the facts, then the challenges.

Positive facts:

• Stanford's virtual CSO B7-H3 case demonstrated a complete loop of multi-agent collaboration and self-criticism: when evaluating B7-H3 as a lung cancer treatment target, the statistical-genetics agent found no significant germline association but flagged regulatory activity; the single-cell atlas agent identified B7-H3 enrichment in cancer-associated fibroblasts; the science-review agent pointed out that the ligand–receptor inference lacked spatial context—this self-criticism triggered subsequent spatial transcriptomics analysis that confirmed an immune-exclusion microenvironment around B7-H3's high-expression regions; the clinical-trial agent ran a stratified survival analysis on the lung adenocarcinoma genome atlas (disease-specific survival HR = 1.82, p = 0.031); it ultimately recommended the antibody–drug conjugate (ADC) modality, and downgraded the priority of small molecules and PROTACs after the pharmacologist agent found no druggable pocket.
• Mathematics: AlphaGeometry2 solved 84% (42/50) of IMO geometry problems at gold-medal level; in 2026-02, DeepMind's Aletheia autonomously solved 4 open problems among the 700 Erdős conjectures and generated complete research papers.
• Physics control: in 2022-02, a reinforcement learning controller from DeepMind and EPFL ran on a real tokamak with 90 measurement inputs, 19 voltage commands, and a 10 kHz frequency, maintaining a plasma configuration that previously required weeks of expert hand-tuning—the first time ever.

Independent challenges (must be presented with as much space as the success cases):

1. AI Scientist's novelty misjudgment and numerical hallucination: Sakana's AI Scientist was found by an independent evaluation to misjudge existing concepts as novel in 4 of 7 manuscripts and to produce hallucinated numerical results. One paper generated by its improved version once reached the peer-review acceptance threshold at a workshop of ICLR 2025—this is at the conference-workshop level and is not equivalent to acceptance at the main top-conference track; this must be noted when citing.
2. The controversy over GNoME's "tenfold" claim: GNoME discovered 2.2 million new crystal structures, of which 381,000 were judged stable, and external laboratories independently synthesized and confirmed 736. But the implicit claim that "AI expanded the catalog of useful materials tenfold" has been retracted in spirit (if not in form); the Nobel-style excitement around GNoME has settled into a more restrained narrative: AI is good at proposing candidates, while human chemists still have to pick the few worth doing.
3. The challenge to Robin's ripasudil: FutureHouse's Robin proposed using the glaucoma drug ripasudil for dry age-related macular degeneration, mechanistically via ROCK inhibition—but this was challenged: Robin's contribution is the "drug–indication pairing," not the underlying mechanism itself; this is a meaningful contribution but more limited than the original framing suggested.
4. Humans still lead on complex scientific tasks: a recent Nature study further notes that humans still outperform the best AI agents on complex scientific tasks, though the gap is narrowing (this secondhand source has limited credibility; marked [To be verified]).
5. Lack of shared benchmarks: a Nature systematic selection guide notes that the field still lacks shared benchmarks to distinguish real progress from marketing, and that the "AI scientist" label varies greatly in meaning from vendor to vendor (secondhand source; recommended to check the original).

3.3.3 Results

With the positive and negative facts juxtaposed, three conclusions most central to Harness design in this direction can be distilled:

1. lab-in-the-loop is an architecture validated by both sides. The Stanford case shows that self-criticism among agents (the review agent triggering spatial analysis) can improve conclusion quality; the challenges to AI Scientist and Robin show that end-to-end autonomous output lacking external validation has limited contribution density. Convergent conclusion: agents handle hypotheses, design, and interpretation, while humans handle problem definition, adoption judgment, and physical experiments—this is exactly the standard form of the L3 orchestration layer in this direction.
2. Numbers must come from execution, and novelty must pass retrieval comparison. AI Scientist's two classes of failure (numerical hallucination and novelty misjudgment) each map to two engineering-able gates: numbers can only come from simulation / code-execution results; "first discovery"-type claims must first pass literature retrieval comparison. Both are written into the red lines in section 4.1.
3. Contribution narratives must be tied to verification grades. GNoME's 381,000 stable candidates are a computation-level conclusion, while the 736 independent syntheses are an experiment-level conclusion—nearly three orders of magnitude apart. Harness output norms should mandatorily distinguish the three evidence grades of "computational prediction / experimental confirmation / independent reproduction" and prohibit conflating them.

4. Practice Standards

4.1 AGENTS.md Specification

Standard-source statement: the following is a proposed standard draft of the AGENTS.md for the scientific discovery direction (AI4S) put forward in this document. As of now, there is no original AGENTS.md specification for the AI4S direction issued by any official body, industry association, or standards organization; AGENTS.md is a community convention, not a standard. This draft inherits all clauses of the data-science team-level AGENTS.md and tightens and extends them for the AI4S direction.

# AGENTS.md —— 科学发现（AI for Science）

> 继承数据科学组级 AGENTS.md 全部条款。本节为科学发现方向的收紧与扩展。
> 本文件为标准建议稿，业界尚无官方标准版本。

## 角色与边界

- 本 Agent 是**科学假说的起草者与验证的组织者**：可以综述文献、提出假说、设计模拟与实验、执行计算验证、解读结果。
- 不可以直接执行的操作：执行物理 / 化学 / 生物实验、进行临床或伦理裁决、单方面宣布「首次发现」、替人类决定采纳哪个假说。
- **ground truth 不是人类标注，而是物理定律、实验验证与守恒律**。任何与物理约束冲突的计算结果，默认为错误，除非有明确证据说明约束条件差异。
- 问题的定义权、假说的采纳权与实验的执行权归领域科学家，不归本 Agent。

## 环境假设

- 领域工具可用：模拟引擎（分子模拟、CFD、气象模型等）、领域数据库、文献检索接口。
- 模拟与实验工具经工具契约暴露（如 MCP 服务器）；工具不直接承载计算，而是产出可由工作流引擎 / 调度系统承载的作业描述。
- 验证判据可用：守恒律容差、收敛阈值、基准算例、（如适用）实验对照协议。
- 文献库与引用数据库可检索，且检索结果带可点击来源。
- 失败实验记录库可用（可为空集，但需有沉淀路径）。
- **任一验证判据缺失时，先与领域科学家共同定义判据，再开始计算。无判据的模拟产出一律标记为未验证。**

## 上下文加载顺序（Context Budget）

1. 科学问题与验收判据（必须）：假说、目标性质、判据类型与容差
2. 领域约束：守恒律、边界条件、适用范围（必须）
3. 文献证据：与假说直接相关的既有工作与引用（**新颖性判断前必须**）
4. 领域数据库条目：结构、性质、已验证记录
5. 同类既往模拟：参数、判据、结果与失败记录
6. 计算资源约束：精度、规模、时长与成本上限
7. 实验可行性与安全约束：涉及湿实验 / 生物 / 临床时的伦理与安全前提

- 文献证据必须携带可点击来源；无来源的领域知识不得作为新颖性或正确性的依据。

## 工具契约

| 工具 | 用途 | 模式 | 约束 |
|---|---|---|---|
| 文献检索 | 查询既有工作、引用核查 | 只读 | 结果必须带来源；新颖性比对结果留痕 |
| 领域数据库 | 查询结构、性质、既有记录 | 只读 | 不得修改公共数据库 |
| 模拟工具接口 | 发起模拟任务描述 | 受限写 | emit 作业描述给工作流引擎；不直接操控调度器 |
| 数值与符号计算 | 计算、推导、统计分析 | 受限执行 | 数值必须由执行产生并记录输入输出 |
| 验证判据执行器 | 守恒律检查、收敛判定、基准对照 | 自动 | 每次模拟产出必经；结果不可被跳过 |
| 实验记录本 | 读写实验设计与结果记录 | 受限写 | 实验执行状态由人更新，Agent 不得代填 |

- 工具参数必须做模式校验；格式错误在调用前拦截。
- 每次模拟记录：判据、参数、版本、运行时长、守恒律偏差与收敛残差。

## 任务执行流程（SOP）

1. **问题定义**：与科学家对齐科学问题、目标性质与验收判据；判据缺失则先补判据。
2. **文献综述**：检索并汇总既有工作，标注来源；识别可借鉴方法与已知失败。
3. **假说生成**：给出假说及其可检验推论；每个假说附验证判据草案。
4. **新颖性比对**：对「首次 / 新发现」类主张执行文献检索比对；比对结果留痕，未过比对不得声称新颖。
5. **计算设计**：选择方法与参数，声明适用范围与已知的近似。
6. **执行与物理验证**：经工作流引擎执行模拟；自动执行守恒律检查、收敛判定与基准对照。
7. **结果解读**：区分「计算预测 / 实验确认 / 独立复现」三档证据等级；列出与预期不符的反例。
8. **实验设计移交**：需要湿实验 / 物理 / 临床验证时，输出实验设计草案，交由人类执行；Agent 不得代执行。
9. **结论分级与移交评审**：结论按证据等级分级输出；所有引用可点击核查；进入人工评审与（如适用）同行评审。
10. **归档**：假说、判据、参数、结果、失败记录与文献比对记录归档；失败原因写入失败记录库。

## 验证与证据要求

- **每条数值必须由执行产生**：来自模拟、代码或统计计算的实际输出，禁止由模型凭记忆或估算生成。
- **物理约束优先于拟合优度**：结果违反守恒律或产生物理上不可能的状态时，判定为无效，不得以「整体误差小」辩护。
- **证据等级三档制**：计算预测（未经实验）→ 实验确认 → 独立复现；输出中必须显式标注当前档位。
- **新颖性主张必须先过文献比对**：比对关键词、命中结果与差异说明留痕。
- **引用必须可点击且逐条可核查**：无来源的引用禁止出现在任何产出物中。
- 主动报告反例：与假说不符的结果必须完整呈现，禁止只报告支持性结果。
- 规模结论与已验证结论分开陈述：如「候选 38 万」与「已合成确认 736」是两句话，不是一句话。

## 失败与升级策略

- 同类失败重试不超过 2 次；第 3 次改变策略或升级。
- **物理约束检查失败**：默认结果无效；先排查边界条件与参数，再考虑方法适用性；不得放宽判据。
- **文献比对发现已有工作**：更新假说定位，如实引用既有工作，禁止重复包装。
- **模拟与既有基准冲突**：报告冲突并升级给科学家，禁止选择性地忽略不利的基准结果。
- **工具参数格式错误**：停止重试，修正调用构造；多次出现上报工具契约缺陷。
- **涉及生物安全 / 伦理敏感的实验设计**：停止生成细节，升级给伦理与安全负责人。
- **成本或时长超预算**：暂停并报告，获批后继续。
- 升级时携带：科学问题、假说、判据、已执行验证、反例清单、建议下一步。

## 安全与合规红线

- 不得伪造、篡改或选择性报告数据。
- 不得伪造引用、编造文献或夸大既有工作的支持程度。
- 不得在未过文献比对的情况下使用「首次」「首创」类表述。
- 不得代替人类执行或声称已执行物理、化学、生物实验。
- 涉及病原体、危险化学品、临床与动物实验的设计，须先通过伦理与安全审查。
- 依《科学数据管理办法》，政府预算资助项目形成的科学数据须先汇交、再验收；不得私匿应汇交数据。
- 不得绕过同行评审流程直接对外发布科学结论。

## 禁止事项

- 禁止生成未经验证的数值结论。
- 禁止把计算预测表述为实验确认；禁止把单点确认表述为普遍规律。
- 禁止把会议研讨会层面的接收表述为主会录用或「顶会发表」。
- 禁止跳过新颖性比对直接宣称突破。
- 禁止在判据未定义时开始大规模模拟。
- 禁止删除或淡化失败记录。
- 禁止跨方向复制通用模板；AI4S 的验证判据与治理要求与数据分析、HPC 有实质差异。

## 输出格式

- 结论先行 → 证据等级（计算预测 / 实验确认 / 独立复现）→ 判据执行记录 → 反例与局限 → 建议。
- 数值带单位与判据上下文；范围用「~」连接；百分比数值与 % 之间无空格。
- 文献引用带可点击来源；「首次发现」类表述必须附比对记录编号。
- 候选规模与已验证数量分开陈述，用表格列出：证据等级、数量、验证方式、验证方。
- 失败记录使用固定格式：假说、判据、失败现象、根因分析、可复用教训。

## 评估与自检

- [ ] 验收判据在计算开始前已定义并双方确认
- [ ] 全部数值来自实际执行，输入输出留痕
- [ ] 守恒律检查与收敛判定已自动执行且通过，或失败已如实报告
- [ ] 新颖性主张已过文献比对，比对记录留痕
- [ ] 全部引用可点击且逐条核查
- [ ] 证据等级三档制已执行，候选与已验证分开陈述
- [ ] 反例与不利结果已完整呈现
- [ ] 需实验验证的部分已移交人类，未代执行
- [ ] 伦理与安全约束已确认（如适用）
- [ ] 假说、判据、结果与失败记录已归档

4.2 SKILL.md Specification

Standard-source statement: the following is a proposed standard draft of the SKILL.md for the scientific discovery direction put forward in this document; likewise, there is no original official standard.

---
name: ai4s-hypothesis-validation
description: 科学假说从生成到计算验证与实验移交的标准执行流程。适用于文献综述与新颖性比对、假说生成、模拟验证设计、守恒律与收敛性检查、证据分级与实验设计移交等任务。触发场景：任何以「提出并验证科学假设」为目标的智能体任务。
version: 1.0
created: 2026-09-12
---

# 科学假说验证标准流程

## 适用场景

- 文献综述与新颖性比对（某方向已知什么、还有什么空隙）。
- 科学假说的生成、形式化与可检验推论设计。
- 计算验证：模拟参数设计、守恒律与收敛性检查、基准对照。
- 实验设计草案编制与移交（湿实验 / 物理 / 临床）。
- 不适用场景：实验的实际执行与裁决、临床决策、伦理与安全审批。

## 前置条件

- 科学问题与目标性质已与领域科学家对齐。
- 验证判据（守恒律容差、收敛阈值、基准算例）已定义；未定义则先补。
- 文献检索接口与领域数据库可用且结果带来源。
- 模拟工具经工具契约可用；工作流引擎与资源预算已确认。
- 失败记录库可用（可为空集）。

## 输入

| 输入项 | 必需 | 说明 |
|---|---|---|
| 科学问题与目标性质 | 是 | 待发现或待验证的对象 |
| 验收判据 | 是 | 守恒律容差、收敛阈值、基准与对照 |
| 领域约束与适用范围 | 是 | 物理边界条件、方法近似声明 |
| 资源预算 | 是 | 计算规模、时长与成本上限 |
| 既有工作与失败记录 | 否 | 文献综述初稿、既往模拟记录 |

## 输出

| 输出项 | 必需 | 说明 |
|---|---|---|
| 假说书 | 是 | 假说、可检验推论、验证判据草案 |
| 新颖性比对记录 | 是 | 检索关键词、命中结果、差异说明 |
| 计算验证报告 | 是 | 参数、版本、守恒律偏差、收敛残差、基准对照 |
| 证据分级结论 | 是 | 计算预测 / 实验确认 / 独立复现三档 |
| 反例与局限 | 是 | 与预期不符的结果及解释 |
| 实验设计草案 | 否 | 交由人类执行的实验方案与安全前提 |

## 执行步骤

1. **对齐问题与判据**
   与科学家确认科学问题、目标性质与验收判据；判据缺失时先共同定义，禁止无判据开算。

2. **文献综述**
   检索既有工作并标注来源；汇总已知方法、已知结论与已知失败；形成新颖性空隙假设。

3. **假说生成**
   给出假说及其可检验推论；每个假说附验证判据草案与证伪条件。

4. **新颖性比对**
   对「首次 / 新发现」类主张执行文献检索比对：记录关键词、命中结果、差异说明；未过比对的主张降级为「改进 / 应用」。

5. **计算设计**
   选择方法与参数；声明近似与适用范围；设计对照算例与负对照；估算资源与时长。

6. **执行与自动验证**
   经工作流引擎提交模拟；自动执行守恒律检查、收敛判定与基准对照；检查不通过则结果无效并进入失败分析。

7. **结果解读与分级**
   区分计算预测 / 实验确认 / 独立复现三档；规模结论与已验证结论分开陈述；列出全部反例。

8. **实验设计移交**
   需要实验验证时输出实验设计草案，含安全与伦理前提；执行权归人类，Agent 不得代执行。

9. **人工评审**
   结论连同判据记录、比对记录与反例一并提交评审；评审意见回流入假说书。

10. **归档与失败沉淀**
    假说书、验证报告、比对记录、反例与失败原因归档；失败教训写入失败记录库。

## 质量标准（DoD）

判据与验证：

- [ ] 判据在计算开始前定义并经科学家确认
- [ ] 守恒律检查与收敛判定已自动执行并留痕
- [ ] 全部数值来自实际执行，禁止模型生成数值
- [ ] 与基准 / 解析解 / 既有模型的对照已完成

证据与引用：

- [ ] 新颖性主张已过文献比对，记录留痕
- [ ] 全部引用可点击且逐条核查
- [ ] 证据等级三档制已执行：计算预测 / 实验确认 / 独立复现
- [ ] 规模结论与已验证数量分开陈述

诚信与治理：

- [ ] 反例与不利结果完整呈现
- [ ] 实验设计已移交，未代执行任何实验
- [ ] 伦理与安全前提已确认（如适用）
- [ ] 假说、判据、结果与失败记录已归档

## 常见失败与处理

| 失败现象 | 根因 | 处理方式 |
|---|---|---|
| 结果违反守恒律 | 边界条件错误 / 方法不适用 / 时间步长过大 | 判定无效；排查条件与方法；禁止以整体误差小辩护 |
| 收敛残差不达标 | 判据过严或数值方法限制 | 报告残差与判据差距；升级给科学家；禁止放宽判据求完成 |
| 文献比对发现已有相同工作 | 综述不足 | 更新假说定位为改进 / 应用；如实引用既有工作 |
| 数值「过于完美」 | 数值由模型生成而非执行 | 回溯输入输出记录；补执行；报告诚信事件 |
| 候选规模被误读为已验证数量 | 证据等级未分开陈述 | 重写结论：计算预测 N 项、实验确认 M 项，分开成句 |
| 假说无法证伪 | 判据缺失或表述含糊 | 回到判据定义步骤；补证伪条件后再开算 |
| 实验团队拒绝移交方案 | 安全 / 伦理前提缺失 | 补齐安全与伦理评估；升级给负责人 |
| 模拟与既有基准冲突 | 方法近似或基准适用范围不同 | 如实报告冲突与两种解释；升级裁决 |

## 示例

**任务**：为某类多孔材料筛选吸附性能候选，并评估新颖性。

1. 对齐判据：目标性质为吸附容量；判据为能量守恒偏差小于 1e-6、与已发表基准材料偏差方向一致、统计显著性 p 小于 0.05。
2. 文献综述：检索既有筛选工作 24 篇，汇总已知方法与最佳已报道吸附容量。
3. 假说生成：提出「孔径分布与吸附容量存在非线性关系」的假说，附证伪条件。
4. 新颖性比对：以三个关键词组检索，命中 6 篇相近工作；差异说明留痕，主张降级为「在特定子族中的系统验证」。
5. 计算设计：蒙特卡洛模拟，声明力场近似；2,000 候选，预算内分批。
6. 执行与自动验证：经工作流引擎分批执行；守恒律检查全部通过；2 批收敛失败，进入失败分析。
7. 结果解读：计算预测前 20 候选（证据等级：计算预测）；与既有实验对照的 40 个已知材料偏差在判据内。
8. 实验设计移交：输出前 5 候选的合成与测定草案，含安全前提；执行归人类。
9. 人工评审：结论、判据记录、比对记录与反例一并提交。
10. 归档：全部工件归档；2 批收敛失败的原因写入失败记录库。

4.3 Implementation Checklist

4.3.1 Context Layer (L1)

• [ ] The literature evidence base is searchable, and results come with clickable sources
• [ ] Domain database entries (structures, properties, verified records) are included in the context
• [ ] Acceptance-criterion templates (conservation-law tolerance, convergence threshold, benchmark cases) are available
• [ ] Failed experiment records can be retrieved and injected into the context
• [ ] A method's approximations and scope of applicability are explicitly declared along with the context

4.3.2 Tools and Execution Layer (L2)

• [ ] Simulation and experiment planning tools are exposed through tool contracts, emitting job descriptions rather than directly manipulating the scheduler
• [ ] Tool parameters have schema validation, and malformed parameters are intercepted before invocation
• [ ] Numerical and symbolic computation leaves input/output traces
• [ ] Each simulation records criteria, parameters, version, duration, and constraint deviations

4.3.3 Orchestration and Control Layer (L3)

• [ ] The lab-in-the-loop architecture is in place: hypothesis adoption and experiment execution are decided by humans
• [ ] An inter-agent self-criticism mechanism is available (a review agent questions conclusions and triggers supplementary analysis)
• [ ] Novelty comparison is a mandatory checkpoint before generating "first discovery"-type claims
• [ ] Experiment design handoff has a clear state flow (draft → human execution → result feedback)

4.3.4 Memory and State Layer (L4)

• [ ] The hypothesis library and literature citation library can accumulate sustainably
• [ ] A failure-record library is established, and failure causes and lessons are searchable
• [ ] Simulation parameters and versions are traceable and recomputable
• [ ] The correspondence between review comments and new-version hypotheses is recorded

4.3.5 Evaluation and Observation Layer (L5)

• [ ] Physical-level automated verification (conservation laws, convergence, benchmark comparison) is a mandatory gate for every simulation output
• [ ] The three-tier evidence grading (computational prediction / experimental confirmation / independent reproduction) is applied across all outputs
• [ ] Tiered evaluation: cheap physical constraints executed at high frequency, expensive experimental validation human-in-the-loop and triggered on demand
• [ ] Scale conclusions and verified counts are stated separately
• [ ] There is a complete channel for presenting counterexamples and adverse results

4.3.6 Governance and Safety Layer (L6)

• [ ] Numbers must come from execution and citations must be verifiable—the two integrity red lines are technically enforced
• [ ] "First"-type claims are constrained by comparison-record identifiers
• [ ] Designs involving biosafety, hazardous chemicals, and clinical or animal experiments have an ethics and safety review checkpoint
• [ ] The scientific-data deposit obligation (deposit before acceptance) is incorporated into the process
• [ ] Unverified conclusions must not be released externally, and the peer-review process cannot be bypassed

5. Summary

The core demand of the scientific discovery direction on AI Harness can be summed up in one sentence: ground truth is not human annotation, but physical laws, experimental verification, and conservation-law constraints—so L5 becomes the first-order problem.

The evidence chain in this direction shows a rare symmetry:

1. Every successful case stands on strong verification. AlphaFold2's persuasiveness comes from comparison against experimentally determined structures at CASP; GraphCast / GenCast's persuasiveness comes from comparison against ECMWF HRES / ENS; Stanford's virtual CSO's persuasiveness comes from spatial analysis and statistical significance triggered by the review agent. Every AI4S result that holds up is backed by an executable, reproducible verification system.
2. Every case that failed lacked the same piece of verification. AI Scientist had 4 of 7 manuscripts with novelty misjudgment and numerical hallucination; the controversy over GNoME's "tenfold" claim (381,000 stable candidates versus 736 independent syntheses, nearly three orders of magnitude apart); Robin's ripasudil was pointed out to contribute a "drug–indication pairing" rather than the underlying mechanism; AI weather models systematically underestimate extreme events, and the gap grows with extremity—the failure forms differ, but the root cause is the same: generative capability exceeded verification capability.
3. Humans still lead on complex tasks, but the industry has found the right collaboration form: lab-in-the-loop. Agents do hypotheses, design, and interpretation; humans do problem definition, adoption judgment, and physical experiments. This is both a calibration of the "human vs AI" narrative and the standard answer of the L3 orchestration layer in this direction.

Therefore the focus of this direction falls on L5 and L3: L5 solidifies physical laws and experimental verification into automatic gates and an evidence-grading regime (the three tiers of computational prediction / experimental confirmation / independent reproduction, with scale conclusions stated separately from verified conclusions); L3 solidifies the human–machine division of labor into a lab-in-the-loop orchestration mechanism. As for the remaining four layers, they all serve these two.

To sum up this direction's claim in one sentence: attach an evidence grade to every scientific claim, turn every physical constraint into an automatic gate, run every "first discovery" through literature comparison first, and keep every experiment in human hands.

Information Gap Statement

1. There is no official or industry-recognized standard text for the scientific discovery direction's AGENTS.md / SKILL.md. Sections 4.1 and 4.2 are both standard drafts proposed in this document.
2. "AI Harness × data science" and the authoritative definitions of AI4S, market size, adoption rate, and dedicated funding amounts and project counts have no reliable public source; this document provides no such figures.
3. "Humans still outperform the best AI agents on complex scientific tasks" (a Nature study) comes from secondhand reporting, marked [To be verified]; it is recommended to check the original before citing.
4. The content of the Nature "Which AI scientist suits your lab?" selection guide (the field lacks shared benchmarks, and the "AI scientist" label varies greatly in meaning) comes from secondhand reporting, marked [To be verified].
5. The two efficiency figures for the Robin system have inconsistent calibers: "completing months of work in a day" and "compressing 872–937 person-hours to under 2 hours" come from different reports; this document only presents them side by side and does not adopt either as a conclusion.
6. Figures such as AlphaFold's user base (190 countries / 2 million researchers), the tokamak RL-control details (90 inputs / 19 commands / 10 kHz), and Aletheia solving 4 open Erdős problems come from multi-source cross-referenced reporting (grade B); it is recommended to check the primary papers when citing.
7. The conclusion that AI weather models underestimate extreme events comes from reporting of research related to Science Advances; the directional conclusion is consistent with scholarly consensus, and specific figures are not cited; the original is recommended for verification.
8. Some performance figures of domestic Chinese weather foundation models such as Fengwu and FuXi (e.g., effective forecasts of 11.25 days, 0.09° resolution) come from vendor and media reporting (grade B); it is recommended to check the papers or official releases when citing.
9. The statement about the Science Intelligence and Materials Creation Center's "investment exceeding ten billion yuan" and its hosting of the major AI4S special program comes from secondhand reporting and is not adopted by this document; no official public breakdown of the specific funding amounts and project counts of China's AI4S program was found.
10. The material systems, criterion thresholds, and candidate counts in the section 4.2 example are illustrative constructions and do not represent any real research.

6. References

1. AI for Science achievements chronology and key data — AIWiki. https://aiwiki.ai/wiki/ai_for_science
2. AI for Science capabilities and milestones compilation — Achievements AI. https://achievements.ai/type/capability
3. AI for Science knowledge tree (AlphaFold / GNoME / Aletheia entries, etc.) — BAAI Community. https://hub.baai.org.cn/knowledge-tree/8fd127c3-5f22-493d-8fa2-d23da3c3ace7
4. AI in Physics review page (AlphaGeometry / tokamak RL control / GNoME controversy, etc.) — Scale Physics. https://scalephysics.com/horizons/aiinphysics/
5. Research related to AI weather models underestimating extreme events (Science Advances / Earth & Environment) — Nature. https://www.nature.com/articles/s43247-025-02502-y
6. Machine Learning Weather Forecasting analysis page (ERA5 dependency and hybrid-modeling directions) — Research. https://research.mental-momentum.ai/r/machine-learning-weather-forecasting-ijl0v7
7. AI weather forecast entry (Fengqing / Fenglei / Fengshun / first tier / demonstration program) — Baidu Baike. https://baike.baidu.com/item/AI天气预报/68593824
8. Agents for R&D Science (AI Scientist 4/7 evaluation, Robin ripasudil challenge, Stanford virtual CSO B7-H3 case) — Orchestra Bio. https://orchestra.bio/blog/agents-for-r-d-science
9. Autonomous scientific discovery systems 2026 roundup (Co-Scientist / Kosmos / funding and tool ecosystem) — Mixflow. https://mixflow.ai/blog/the-ai-pulse-whats-new-in-autonomous-scientific-discovery-for-2026
10. Nature "Which AI scientist suits your lab?" retelling page (the field lacks shared benchmarks, autonomy-tier selection) — Sinapti. https://sinapti.ca/post/es/nature-compara-los-sistemas-de-ia-que-automatizan-la-investi-xd43nve2
11. Shanghai AI Laboratory's Sinan scientific-intelligence evaluation system (SciEvalKit's seven capabilities × six disciplines) — Shanghai AI Laboratory. https://www.shlab.org.cn/news/5444233
12. SciEvalKit official repository — InternScience (GitHub). https://github.com/InternScience/SciEvalKit
13. Awesome AI for Science (a full panorama of scientific-agent benchmarks: SciCode, MLAgentBench, NewtonBench, etc.) — GitHub. https://github.com/ai-boost/awesome-ai-for-science
14. Reporting on the "AI-driven scientific research" special deployment — China Journalists Association website. https://union.china.com.cn/cmdt/txt/2024-03/26/content_42737343.html
15. Agentic MOF Screening on Aurora (an AI4S and HPC crossover case) — supercomputing.news. https://www.supercomputing.news/hpc/agentic-mof-screening-aurora