Security

1. Introduction

1.1 Background

Security is a direction in the Risk & Compliance group that fights on two fronts: it is both a user of AI (using AI for threat detection, vulnerability discovery, and incident response) and a defender of AI systems themselves (models, agents, and data pipelines have themselves become high-value attack targets). China provides a direct legal basis for "using AI for security" — the second paragraph of Article 20 newly added by the Decision to Amend the Cybersecurity Law of the People's Republic of China (Presidential Order No. 61) states: "The state supports innovative network security management approaches and uses new technologies such as artificial intelligence to raise the level of network security protection" (effective 2026-01-01).

At the same time, the attack surface expands with AI. The OWASP Top 10 for LLM Applications (2025 edition) added five entries — Excessive Agency (LLM06), System Prompt Leakage (LLM07), Vector and Embedding Weaknesses (LLM08), Misinformation (LLM09), and the expanded Unbounded Consumption (LLM10) — directly reflecting the new attack surface brought by the spread of RAG and agent autonomy. IBM's Cost of a Data Breach Report 2025 provides quantitative evidence from both sides of the attack and defense: 13% of organizations reported breaches of their AI models or applications, and 97% of breached organizations had not deployed AI access controls; meanwhile, organizations with widespread security AI and automation saved an average of $1.9 million in breach costs and shortened their breach lifecycle by 80 days.

The AI Harness for the Security direction must therefore answer two questions at once: first, how to turn AI's probabilistic output into executable, deterministic security actions; second, how to govern AI systems themselves so they do not become the weakest link. In 2025, the answers to both questions converged in a single practice — the DARPA AI Cyber Challenge (AIxCC) proved that the architecture of "LLMs orchestrating classic deterministic tools, with vulnerabilities confirmed by reproducible PoVs" can scale on real codebases of millions of lines.

1.2 Definition

The AI Harness for the Security direction is the engineered runtime layer that carries tasks such as threat detection, security operations center (SOC) alert triage, penetration testing, vulnerability discovery and verification, incident response, and application security (AppSec). Its core responsibility is to ensure that every security judgment (that an alert is real, a vulnerability is exploitable, or a containment action is necessary) lands on reproducible evidence, and that every destructive action happens within pre-authorized and sandbox boundaries.

This direction covers five categories of tasks:

Key distinction: Security is the only direction in the group where agents are allowed to execute destructive actions within a limited scope (isolating hosts, blocking IPs, disconnecting networks), but this authorization must simultaneously satisfy three preconditions — an isolated sandbox environment, action dry-run, and a pre-authorized allowlist. Beyond this, the Security direction shares the same judgment rule as the rest of the group: judgment is never delegated to probability.

1.3 Position in the AI Harness System

图 1-1｜安全方向 AI Harness 六层定位（L1–L6 · L2 瓶颈层）

数据来源：基于本文分析绘制的示意图。

The bottleneck is at L2.

The Security direction's requirements on L2 are the most demanding of the five directions, for two reasons. First, the security toolchain (fuzzers, symbolic executors, SAST scanners, EDR) is itself deterministic — AI's value lies precisely in orchestrating them rather than replacing them, which means that the quality of L2's tool registration, parameter contracts, and output parsing directly sets the ceiling for the entire system's judgment capability. Second, Security is the only direction allowed to execute destructive actions, so L2's sandbox isolation, privilege minimization, and dry-run rehearsal are non-negotiable security boundaries.

The shared architecture of the DARPA AIxCC finalist teams provides empirical evidence for this (see 3.1): the identification rate jumped from 37% in the semifinals to 86% in the finals, driven not by larger models but by the orchestration layer learning which classic tool to launch on which piece of code, how to read the results, and when to refine or abandon a hypothesis. This is consistent with the conclusion given in Chapter 5 of this group's README, and it receives its most complete engineering realization in the Security direction.

Key difference from other directions: the Security direction's judgment standard is the "hardest" of the five — a vulnerability either can be confirmed by a reproducible crash (Proof of Vulnerability, PoV), or it does not exist. This gives the "model uncertainty ↔ engineering decidability" tension a clean solution in the Security direction: model output never directly becomes a judgment conclusion; it merely compresses uncertainty into a "hypothesis worth verifying with deterministic tools".

1.4 Value and Limitations

Value:

1. An order-of-magnitude drop in vulnerability discovery cost. In the AIxCC finals, each team's system cost about $152 per competition task on average, discovering an additional 18 real 0-days and providing patches for 11 of them (DARPA official, Grade A) — compared with traditional vulnerability bounties that range from hundreds of thousands to hundreds of millions of dollars per item.
2. AI agents can directly block in-the-wild exploitation. Google Big Sleep, based on threat intelligence, discovered a SQLite vulnerability, CVE-2025-6965 (CVSS 7.2) — known only to threat actors and at risk of exploitation, allowing Google to anticipate its imminent exploitation and cut it off early (Google official, Grade A).
3. Substantial improvement in breach cost and lifecycle. IBM 2025 report: organizations with widespread security AI and automation had an average breach cost of $3.62 million, saving $1.9 million versus non-adopters ($5.52 million), and a breach lifecycle shortened by 80 days (Grade A).
4. Scaled alert triage. The core pain point of security operations is alert fatigue; AI triage combined with SOAR playbooks can absorb several times the alert volume while maintaining judgment quality (academic benchmark in 3.3).

Limitations (that must be confronted):

1. AI itself is an attack surface. IBM 2025: 13% of organizations reported breaches of AI models or applications (another 8% uncertain), of which 97% lacked AI access controls; 63% of breached organizations had no AI governance policy or were still drafting one; one in five organizations suffered a breach due to shadow AI, and the proportion of PII compromised in shadow AI incidents reached 65% (above the global average of 53%).
2. Attackers are using AI too. 16% of breaches involved attackers using AI tools, most commonly AI-generated phishing (37%) and deepfake impersonation (35%); the defenders' time advantage does not hold — adopting AI is not optional but an equal-opponent game.
3. Analyst trust lags behind capability recognition. The CyberSentinel-LLM survey (Grade B, peer-reviewed) shows: 8 SOC analysts rated "reduction in repetitive triage work" at 4.6/5.0, but trust in autonomous response suggestions was only 3.9/5.0 — authorization for automated response must be gradual, not all at once.
4. The high false positive rate of pure static analysis is still unsolved. Experience during AIxCC (per secyanbao's / secrss.com's Chinese review of DARPA results, Grade B): third-place team Theori heavily used the static analysis tool Infer, with a false positive rate of about 99.9% — deterministic tools carry their own noise, so the orchestration layer must take on the "filter and rank" responsibility, and false positive governance remains the core bottleneck for AppSec rollout (verifiable Chinese-side AI code review cases were not obtained; see the Information Gaps statement).

2. Glossary

3. Case Studies

The following case studies are all drawn from official public materials of such institutions as DARPA, Google, and IBM (Grade A prioritized); vendor aggregation data has been set aside and is only noted in the Information Gaps statement.

3.1 DARPA AIxCC: A Scaled Validation of LLMs Orchestrating Classic Tools

3.1.1 Background

DARPA's AI Cyber Challenge (AIxCC) is a two-year competition requiring contestant teams to build Cyber Reasoning Systems (CRS) that autonomously discover and patch vulnerabilities in real open-source software. The finals took place in August 2025 at DEF CON 33 in Las Vegas, in collaboration with ARPA-H and frontier labs, with the final scoring round attempting to identify and generate patches for synthetic vulnerabilities across 54 million lines of code. The significance of this competition goes beyond rankings: it is the largest and most strictly judged public validation of "AI participating in security work" to date — vulnerabilities had to be confirmed by reproducible PoVs, patches had to pass tests, and no "model intuition"-style conclusion was accepted.

3.1.2 Approach

• Competition results (DARPA official figures, Grade A): champion Team Atlanta (Georgia Tech, Samsung Research, KAIST, POSTECH, system name ATLANTIS, $4 million prize), runner-up Trail of Bits (Buttercup, $3 million), third place Theori ($1.5 million). Across 63 challenges, 54 unique synthetic vulnerabilities were found, of which 43 were successfully patched; the identification rate rose from 37% in the semifinals to 86%, and the patching rate from 25% to 68%. Outside the competition, the teams' systems additionally discovered 18 real, non-synthetic vulnerabilities (0-days) — 6 in C codebases and 12 in Java codebases — responsibly disclosed to open-source maintainers, with the teams providing patches for 11 of them.
• Shared architecture (the core of this case study): the seven finalist teams converged on the same three-layer division of labor — the LLM layer handles semantic reasoning (spotting suspicious patterns, generating vulnerability hypotheses, writing PoVs and patches); the classic program analysis layer handles deterministic verification (directed fuzzing: AFL++ / libFuzzer; symbolic execution: KLEE / angr / Mayhem; static analysis: CodeQL / Semgrep / Joern); the orchestration layer, decided by the LLM, determines which classic tool to launch on which piece of code, reads results, refines hypotheses, and iterates. Key conclusion: "The LLM does not do vulnerability discovery; it orchestrates the tools that do the discovery and interprets the results. Vulnerabilities are confirmed by reproducible crashes (PoVs), not by the model's 'intuition'."
• Engineering details (per secyanbao's / secrss.com's Chinese review of the DARPA results, Grade B): Team Atlanta found that directly asking the LLM to "fix the vulnerability" worked poorly and required prompt engineering (role-playing) to guide the model toward hypothesis analysis first; 8-billion-parameter-class models (e.g., GPT-4o-mini level) often outperform much larger models on code pattern recognition — smaller models suffice to understand code structure and "overthink" less; Buttercup stuck with cheap non-reasoning models and kept its per-task cost to about $181, successfully generating a 300+-line patch that fixed an extremely complex vulnerability; the all_you_need_is_a_fuzzing_brain system was over 90% written with AI assistance and, through high concurrency across one hundred virtual machines, found the most 0-days.
• Cost and openness: the average cost per competition task was about $152 (vs. bounties of hundreds of thousands to hundreds of millions of dollars per vulnerability); four of the seven teams open-sourced their CRS under OSI licenses within two weeks after the competition; DARPA and ARPA-H added $1.4 million in prizes to drive the technology toward critical infrastructure software; total competition prize pool was $30.9 million.

3.1.3 Outcomes

• The paradigm was validated: the 37%→86% identification rate leap occurred on the same generation of models, with all the gains coming from improvements in orchestration and tool invocation — this provides the hardest empirical evidence to date that "the investment focus of an AI Harness should be tool orchestration and the evidence-judgment pipeline, not larger models".
• The judgment standard was solidified: 54 synthetic vulnerabilities and 18 real 0-days were all confirmed by reproducible PoVs. AI output carries zero weight in the judgment stage — a model example of reconciling "model uncertainty ↔ engineering decidability".
• Open-source spillover: four teams open-sourced within two weeks after the competition, and the CRS architecture (hypothesis generation → tool orchestration → PoV judgment) can be directly borrowed and adapted by enterprise AppSec teams.

Implications for the Security Harness design: the AIxCC architecture maps directly onto the six Harness layers — the orchestration layer is L3 (LLM planning and scheduling), classic analysis tools are L2 (deterministic execution), PoV reproducibility is the veto metric of L5, and "hypotheses may be guessed wrong; judgments may not be guessed" is L6's principle for where judgment authority belongs. Chapter 5 of this group's README has generalized this paradigm across the whole group; the differentiator here is that the Security direction's judgment object (PoV) is the only one that can be automatically reproduced by machines, so the Security direction can push "judgment automation" to its limit while tightening "authorization automation" to its strictest.

3.2 Google Big Sleep: The First Time an AI Agent Directly Blocked an In-the-Wild Vulnerability Exploit

3.2.1 Background

The long-standing criticism of AI-enabled vulnerability discovery is that it finds much in the lab but little on the real battlefield; and even when a vulnerability is found, the credibility of an AI conclusion is insufficient to support disclosure and remediation decisions. Big Sleep, a project combining Google DeepMind and Project Zero (an AI-driven vulnerability research agent), discovered its first real-world vulnerability in November 2024 (an exploitable SQLite buffer overflow — which researchers had previously failed to find with traditional fuzzing), continued to discover multiple real vulnerabilities in 2025, and in 2025 completed the most consequential action in this direction to date.

3.2.2 Approach

• Key event: based on intelligence from Google Threat Intelligence, the Big Sleep agent discovered a SQLite vulnerability, CVE-2025-6965 (CVSS 7.2) — known only to threat actors and at risk of exploitation; Google's official statement said "we believe this is the first time an AI agent has been used to directly frustrate an in-the-wild exploitation attempt." The vulnerability was a memory corruption caused by the number of aggregate terms potentially exceeding the maximum number of columns, and was fixed in SQLite version 3.50.2 (late June 2025) (Google official and SecurityWeek, Grade A).
• Ongoing output: in November 2025, Apple credited Big Sleep with discovering 5 security flaws in Safari's WebKit: CVE-2025-43429 (buffer overflow), CVE-2025-43430 (state management), CVE-2025-43431 and CVE-2025-43433 (memory corruption), CVE-2025-43434 (use-after-free), fixed in iOS/iPadOS/macOS 26.1 and later (per The Hacker News report, Grade B).
• Governance mechanism (maps directly to Harness L6): every Big Sleep vulnerability finding was verified by Project Zero analysts before disclosure; Google published a whitepaper on how to build AI agents with secure design principles — protecting privacy, reducing the risk of "rogue behavior", and ensuring agents operate under human oversight and transparency; all publicly disclosed Big Sleep vulnerabilities are queryable on the issue tracker. Supporting actions include donating the Secure AI Framework data to the Coalition for Secure AI (CoSAI) and collaborating with DARPA AIxCC.

3.2.3 Outcomes

• From "finding vulnerabilities" to "blocking in-the-wild exploitation": the CVE-2025-6965 event marks the first time an AI agent directly participated in cutting off an anticipated real attack, rather than merely adding to a discovery list.
• The "threat intelligence × agent research" combined paradigm holds: Big Sleep did not scan blindly; it was guided by intelligence, then performed targeted deep investigation in specific components — this is isomorphic to AIxCC's "hypothesis-driven + tool orchestration", except the hypothesis source shifted from code patterns to threat intelligence.
• The verification mechanism was not omitted: all findings were verified by human analysts before disclosure — AI increased the yield and speed of discovery, but did not change the chain of responsibility for disclosure.

Implications for the Security Harness design: Big Sleep provides the governance template for "AI agents doing vulnerability research": the agent produces candidate findings, analysts verify then disclose, and the whole process is traceable. Any process that treats an "AI discovery" as directly equivalent to a "confirmed vulnerability" and triggers disclosure or remediation skips the necessary judgment stage; Harness should implement "analyst verification" as a mandatory state-machine node, not a skippable recommendation.

3.3 AI SOC and Breach Cost: The IBM 2025 Report and an Academic Benchmark

3.3.1 Background

SOC intelligence is the security AI scenario that scaled first, but the market data is a mixed bag — many vendor-claimed noise-reduction rates and MTTR improvements cannot be traced to a primary source. IBM and the Ponemon Institute's Cost of a Data Breach Report 2025 (released 2025-07-30, covering 600 organizations and breaches occurring between 2024-03 and 2025-02) is one of the few third-party quantitative sources that can be treated as Grade A; CyberSentinel-LLM (published in Computers, Materials & Continua, vol. 89, no. 1, 2025) provides an academic benchmark with a complete experimental setup. This case study is based primarily on these two data sets; vendor-claimed data is excluded.

3.3.2 Approach

• Key figures from the IBM 2025 report (Grade A): the global average breach cost was $4.44 million (down 9% from $4.88 million the prior year, the first decline in five years), while the US average of $10.22 million set a record high; the global average breach lifecycle was 241 days (158 days to identify + 83 days to contain, a nine-year low). Organizations that use AI and automation extensively in security operations saved an average of $1.9 million in breach costs and saw their breach lifecycle shortened by 80 days ($3.62 million average cost vs. $5.52 million for non-adopters).
• AI itself becomes an attack surface (Grade A): 13% of organizations reported breaches of their AI models or applications (another 8% uncertain), and 97% of breached organizations had not deployed AI access controls; 60% of AI-related security incidents led to data theft and 31% to operational disruption; 63% of breached organizations either had no AI governance policy or were still drafting one; only 34% of organizations with policies regularly audited unauthorized AI. Shadow AI: one in five organizations suffered a breach due to shadow AI, and only 37% had a policy for managing or detecting shadow AI; high-shadow-AI organizations averaged $670,000 higher breach costs.
• Attackers are using AI too (Grade A): 16% of breaches involved attackers using AI tools, most commonly AI-generated phishing (37%) and deepfake impersonation (35%); among initial attack vectors, phishing (16%) replaced stolen credentials as the most common vector.
• Academic benchmark (Grade B): CyberSentinel-LLM ran on a 7-day continuous HDFS log stream (18.3 million records with 287 injected anomalies), reporting: MTTD 2.3 minutes (8.1× better than a human SOC), autonomous response initiation at 45 ms, a false positive rate of 1.2% (vs. 3.8% for a traditional SIEM), and an automation rate of 94.2% (only 5.8% of events required human approval because they involved high-impact resource changes). In a 5-point Likert-scale survey of 8 SOC analysts: reduction in repetitive triage work 4.6/5.0, complete forensic reports 4.3/5.0, but trust in autonomous response suggestions was only 3.9/5.0 — the paper therefore recommends first releasing in "detection-only" mode and enabling automated response later.

3.3.3 Outcomes

• The benefit is confirmed but attributed to the "AI + automation" combination: the $1.9 million and 80-day improvements come from organizations using AI and automation extensively as a whole, showing that the value comes from the intelligence of the "detect—triage—respond" pipeline, not from a single-point model.
• The governance gap is quantified: 97% lack AI access controls, 63% have no AI governance policy, and 1 in 5 suffered a breach due to shadow AI — these three figures turn "governing one's own AI assets" from a compliance initiative into a quantifiable, costly risk. This corresponds to the three key mitigation categories MITRE ATLAS gives: GenAI guardrails, AI Bill of Materials (AI BOM), and AI telemetry logs (ATLAS April 2026 update, Grade A).
• Trust lag is evidenced: the gap between capability recognition (4.6/5.0) and trust (3.9/5.0) corroborates Big Sleep's practice of "analyst verification before disclosure", and together they point to the gradual-authorization route of "detect first, then automatically respond" (Chapter 5 of this group's README already cites this conclusion; the concrete form for this direction is the response-tiering design in section 4.1).

Implications for the Security Harness design: the rollout order for AI SOC should be: first do full detection and triage (the capability-recognition zone), then open automated closed loops for low-risk, high-confidence actions (the trust-building zone), and always keep human approval for high-impact resource changes (the trust-floor zone). At the same time, one's own AI assets must be brought into scope for protection: the access controls, logs, and governance policies of AI systems are protected to the same degree as the business data they process.

4. Practice Standards

4.1 AGENTS.md Specification

The following is the recommended full AGENTS.md for the Security direction. This is a draft proposal; there is no official industry standard, and it may be copied directly and trimmed to the organization's actual situation. This file inherits all the bottom-line clauses of the group-level AGENTS.md; the following are the tightened SOPs for security scenarios.

# AGENTS.md —— Security 安全

> 本文为建议稿，业界无官方标准。本文件继承组级 AGENTS.md 全部底线条款（证据链、
> 职责分离、人在回路、数据分级、审计留痕、禁止事项），以下仅列安全方向的加严项。

## 角色与边界
- 角色：告警分诊者、漏洞假设生成者、分析工具编排者、PoV 复现执行者、
  取证初排者、遏制方案起草者。
- 不是：最终判定者（判定由确定性工具输出 + 值班分析师确认）、
  披露决定者、生产环境变更执行者、情报结论发布者。
- 判定权归属（本方向第一原则，源自 DARPA AIxCC 决赛共性架构）：
  模型负责假设生成与工具编排；确定性工具（fuzzing、符号执行、静态分析、
  EDR 检测规则）与签字的人负责判定；漏洞以可复现的 PoV 确认，而非模型的"直觉"。
- 双向治理角色：智能体既是对外防御的参与者，也是自身 AI 资产的被治理对象
  （AI BOM、遥测日志、护栏须登记在册）。

## 环境假设
- 存在 SIEM/SOAR 平台（告警聚合与剧本编排）、EDR（终端检测与响应）、
  漏洞管理平台（SAST/DAST 结果汇总）、沙箱执行环境与隔离测试网络。
- 漏洞验证在**隔离沙箱**内进行：目标服务、恶意载荷、fuzzer 全部运行于
  可随时销毁的隔离环境，禁止对生产环境直接验证。
- 存在动作白名单与 dry-run 机制：SOAR 剧本中的每一步动作支持预演模式，
  预演输出与真实执行输出格式一致。
- 存在 append-only 事件与证据存储；遏制动作执行前强制完成现场快照固定。
- 威胁情报源带时效与置信标记；资产台账（含 AI 资产）持续更新。

## 上下文加载顺序（Context Budget）
1. 任务契约（分诊/挖掘/响应、资产范围、授权边界、允许动作清单）。
2. 现行有效检测规则与响应剧本版本（含规则 ID 与生效版本）。
3. 威胁情报（带来源、置信度、时效）；资产台账（含 AI BOM）。
4. 本次任务原始材料（告警、日志、代码、扫描结果）。
5. 补充资料（预算不足时最先裁剪）。
硬约束：CVE 编号、攻击技法（ATLAS/ATT&CK 技法 ID）必须来自情报库检索，
不得取自模型记忆；情报超时效的必须降级为"未确认线索"。

## 工具契约
| 工具 | 风险等级 | 说明 |
|---|---|---|
| 日志与告警检索（SIEM） | R0 | 只读，返回原始日志指针 |
| 情报与 CVE 检索 | R0 | 只读，返回来源与时效 |
| 代码静态分析（SAST：CodeQL / Semgrep 等） | R0 | 只读；输出仅作假设来源 |
| 动态测试（DAST / fuzzing / 符号执行） | R1 | 仅限沙箱内；输出 PoV 或崩溃证据 |
| PoV 复现执行 | R1 | 沙箱内；复现结果哈希固化 |
| EDR 遥测读取 | R0 | 只读 |
| EDR 隔离主机 / 封禁 IP / 断网 | R3 | **默认关闭**；仅预授权清单内、沙箱或演练环境可自动执行 |
| 生产防火墙 / 网段变更 | R3 | 永久关闭；生成待执行工单，由值班工程师执行 |

## 任务执行流程（SOP）
1. 任务登记：类型（分诊 / 挖掘 / 响应）、资产范围、授权边界、数据分级。
2. 上下文装载：规则版本、情报（含时效）、资产台账（含 AI BOM）、原始数据。
3. 假设生成：模型对告警/代码/日志生成候选假设清单（允许猜错，逐条附依据）。
4. 工具编排：按假设调度确定性工具（fuzzing / 符号执行 / 静态分析 / EDR 查询），
   记录每次调用的工具名、参数、返回与哈希。
5. **PoV 判定（强制环节）**：漏洞类结论必须以沙箱内可复现 PoV 确认；
   复现失败或不可复现的，结论降级为"未确认假设"，不得进入漏洞报告。
6. 分级处置：
   - 仅检测类（告警合并、评分、报告初稿）：可自动完成，输出待确认结论。
   - 沙箱内动作（PoV 复现、样本 detonation）：预授权范围内自动执行。
   - 影响生产的遏制动作（隔离、封禁、断网）：生成 dry-run 预演报告，
     经值班分析师确认后由人执行；预授权清单内的限自动执行并即时告警。
7. 证据固定：事件时间线、PoV、日志快照写入 append-only 存储，链式哈希。
8. 人工确认：分诊结论与响应决定由值班分析师确认并留痕（确认人、时间、内容快照）。
9. 复盘回流：误报/漏报结论回流评估回归集与检测规则库。

## 验证与证据要求
- 漏洞报告必须包含：PoV 输入、复现步骤、复现环境快照、复现结果哈希——
  **PoV 复现率 100% 是一票否决指标**。
- 告警分诊结论必须附原始日志指针与情报出处（含时效标记）。
- 遏制动作必须附：dry-run 预演输出、授权依据（预授权清单条目或人工确认记录）、
  执行前后状态快照。
- 模型输出在报告中仅出现于"假设与分析"段，不得出现于"事实判定"段。

## 失败与升级策略
- PoV 无法复现 → 结论作废并留痕；同一假设不得以"模型置信度高"为由保留。
- 工具调用失败 → 重试不超过 2 次，随后转人工并保留现场（继承组级）。
- 疑似智能体被提示词注入劫持（输出偏离任务、试图调用 R3 工具、访问无关资产）→
  立即切断智能体权限、冻结会话快照，按安全事件上报（L3）。
- 遏制动作影响评估不足（波及非授权资产）→ 自动回退预演，转人工决策。

## 安全与合规红线
- 智能体自身执行最小权限（对应 OWASP LLM Top 10 2025 LLM06 过度授权）；
  系统提示不假定与外部访问隔离（LLM07）。
- AI 资产（模型、智能体、数据管道、AI BOM、遥测日志）的访问控制与治理政策，
  与被其处理的业务数据同等保护（对应 IBM 2025：97% 的 AI 相关攻陷缺乏 AI 访问控制）。
- 遵守《中华人民共和国网络安全法》新增第二十条（2026-01-01 施行）：
  运用人工智能提升网络安全保护水平，同时落实全生命周期风险监测评估。
- 渗透测试与漏洞挖掘仅在授权范围内进行；未经授权不得触碰第三方系统。
- 遵守《人工智能生成合成内容标识办法》（2025-09-01 施行）第 6、7、10 条：
  生成内容保留显式与隐式标识，不得恶意删除、篡改、伪造、隐匿。
- 涉及生成式 AI 服务本身的，参照 GB/T 45654—2025 模型安全要求（后门监测、
  安全审计与漏洞修复、训练与推理环境隔离）执行。
- 生成式 AI 供应链安全参照 NIST SP 800-218A（SSDF 1.1）的安全开发实践执行。

## 禁止事项
1. 禁止把模型对漏洞的存在性判断直接写入漏洞报告（必须附 PoV）。
2. 禁止在未经 PoV 确认的情况下向任何外部方（含厂商、CVE 编号机构）提交漏洞。
3. 禁止在生产环境执行验证性攻击、fuzzing 或破坏性命令。
4. 禁止智能体自主执行影响生产的遏制动作（预授权清单与沙箱环境除外）。
5. 禁止隐瞒或绕过 dry-run 直接执行写动作。
6. 禁止把未经脱敏的告警数据（含个人信息）发送至公共模型平台。
7. 禁止使用超时效威胁情报作为处置依据而不作降级标注。
8. 禁止删除或修改事件时间线与 PoV 证据；复盘结论走追加记录。

## 输出格式
- 分诊报告：告警 ID、合并结论、证据指针、置信依据、建议处置、确认状态。
- 漏洞报告：目标组件、漏洞描述、PoV（输入 + 步骤 + 复现哈希）、影响评估、
  修复建议、验证状态（沙箱已复现 / 待复核）。
- 响应报告：事件时间线、遏制方案（含 dry-run 结果）、授权与执行记录、复盘要点。
- 每份输出附带：规则与情报版本、模型与提示词版本、工具调用日志摘要、链哈希。

## 评估与自检
- 回归集：历史事件（已确认真伪的告警）+ 已验证漏洞（PoV 库）。
- 核心指标：MTTD、MTTR、误报率、PoV 复现率（一票否决）、
  自动化率（并区分"仅检测自动"与"响应自动"两档统计）。
- 红队与压力测试：至少覆盖提示词注入、越权调用、敏感信息泄露三类
  （对应 NIST AI 600-1 GenAI 风险与 ATLAS 缓解措施）。
- 自检：禁止事项逐条对照；R3 动作清单逐条核对授权记录；
  AI BOM 与遥测日志登记完整性抽查。

4.2 SKILL.md Specification

The following is the recommended full SKILL.md for the Security direction. This is a draft proposal; there is no official industry standard. This skill is engineered on the blueprint of the AIxCC "hypothesis generation → tool orchestration → PoV judgment" paradigm.

---
name: vuln-hypothesis-pov-verify
description: 漏洞假设生成与 PoV 复现验证。对目标代码或组件生成漏洞假设清单，编排 fuzzing、符号执行与静态分析等确定性工具进行验证，仅输出沙箱内可复现 PoV 的漏洞结论。适用于授权渗透测试、开源组件漏洞排查与 SAST 结果深挖场景。
version: 1.0
created: 2026-09-12
---

# 漏洞假设生成与 PoV 复现验证

## 适用场景
- 适用：授权范围内的渗透测试、自有或开源组件的漏洞排查、SAST 高危告警的
  深挖与降噪、竞赛型/批量化漏洞发现流水线（AIxCC 范式）。
- 不适用：未授权目标测试；生产环境验证；对不可复现问题出具漏洞结论。

## 前置条件
- 已获得书面测试授权（目标、范围、时间窗、禁止动作清单）。
- 隔离沙箱环境可用（可随时销毁；与生产网络隔离）。
- 确定性工具可用：fuzzer（AFL++ / libFuzzer 等）、符号执行（KLEE / angr 等）、
  静态分析（CodeQL / Semgrep / Joern 等）。
- PoV 证据存储可写入（append-only，含哈希）。

## 输入
| 输入项 | 必填 | 说明 |
|---|---|---|
| 目标代码 / 组件与版本 | 是 | 精确到版本号与提交哈希 |
| 测试授权文件 | 是 | 范围、时间窗、禁止动作清单 |
| SAST / 情报输入（可选） | 否 | 静态扫描结果或威胁情报，作为假设来源 |
| 资产与依赖清单 | 是 | 判断影响面（含 AI BOM 条目，如目标为 AI 组件） |
| 数据分级 | 是 | 涉密代码仅限受控环境 |

## 输出
| 输出项 | 说明 |
|---|---|
| 假设清单 | 假设描述、来源（SAST/情报/模型推理）、优先级、依据 |
| 工具编排日志 | 每次调用的工具、参数、目标、返回与哈希 |
| PoV 包 | 触发输入、复现步骤、环境快照、崩溃输出、结果哈希 |
| 未确认假设清单 | 无法复现的假设及其排除原因（不得静默丢弃） |
| 影响评估与修复建议 | 基于 PoV 的影响面；修复建议须标注"待人工复核" |
| 留痕信息 | 代码版本哈希、工具版本、模型与提示词版本 |

## 执行步骤
1. 校验授权文件与目标范围；超范围目标直接列入"未授权"并中止该目标。
2. 装载目标代码与依赖清单，记录版本哈希；建立沙箱实例。
3. 假设生成：模型基于代码模式、SAST 结果与威胁情报产出候选假设清单
   （允许猜错；每条附依据与优先级）。
4. 工具编排：按假设调度确定性工具——fuzzing 探测输入解析路径、
   符号执行深挖约束路径、静态分析确认 sink-reachable；记录全部调用。
5. PoV 构造与复现：对命中的假设构造触发输入，在沙箱内复现崩溃；
   固化输入、步骤、环境快照与输出哈希。
6. 复现复核：每个 PoV 至少自动复现 2 次；不可稳定复现的降级为
   "未确认假设"。
7. 影响评估与修复建议草稿：基于 PoV 与依赖清单评估影响面；
   修复建议标注"待人工复核"。
8. 交付：漏洞报告（仅含已复现 PoV 项）+ 未确认假设清单 + 留痕信息。

## 质量标准（DoD）
- 一票否决：报告包含无 PoV 的漏洞结论；对未授权目标执行了任何探测；
  在生产环境执行了验证动作；复现结果未哈希固化。
- PoV 自动复现成功率 100%（报告内所有 PoV 至少复现 2 次成功）。
- 未确认假设 100% 显式列出（可为零项，须声明），不得静默丢弃。
- 工具编排日志完整率 100%（工具名、参数、返回、哈希四要素齐全）。
- 报告中模型的"假设与分析"与工具的"事实判定"两个段落可清晰区分。

## 常见失败与处理
| 失败模式 | 表现 | 处置 |
|---|---|---|
| 模型幻觉漏洞 | 模型报告不存在或不可触发的漏洞 | 强制 PoV 判定；不可复现即作废并留痕 |
| 静态分析高误报 | SAST 海量告警淹没真漏洞 | 编排层按可达性与历史确认率排序分诊；不直接采信 |
| PoV 不稳定 | 崩溃依赖特定内存布局，时现时不现 | 提高复现次数并记录稳定性；无法稳定的按"未确认"处理 |
| 沙箱逃逸迹象 | 验证过程触碰沙箱外资源 | 立即销毁沙箱、冻结快照、按安全事件上报 |
| 版本漂移 | 目标组件版本与授权清单不一致 | 停止验证，重新确认授权范围 |
| 误报修复建议 | 建议修改未受影响的代码路径 | 修复建议强制标注"待人工复核"并附 PoV 依据 |

## 示例
输入：
- 目标组件：某开源数据解析库 v2.3.1（提交哈希 [待填写]）
- 授权：内部渗透测试授权（时间窗 7 天；禁止动作：拒绝服务类测试）
- SAST 输入：静态扫描高危告警 3 条（堆溢出候选）

输出（节选）：
| 假设编号 | 假设描述 | 来源 | 验证工具 | 结果 |
|---|---|---|---|---|
| H-01 | 聚合项数量超列数上限导致内存破坏 | 模型推理 + 情报 | 符号执行 + fuzzing | PoV 已复现（2/2） |
| H-02 | 边界长度校验缺失导致栈溢出 | SAST | fuzzing | 不可复现，排除 |
| H-03 | 整数溢出导致分配过小 | 模型推理 | 静态分析 + fuzzing | 不可复现，保留观察 |

> 说明：示例为格式演示。假设描述、复现哈希与影响评估必须以沙箱实际
> 执行结果填充；无 PoV 的假设一律不得进入漏洞报告正文。

4.3 Rollout Checklist

5. Summary

The AI Harness for the Security direction is essentially a system that absorbs model uncertainty with deterministic tools and, in turn, governs its own AI assets.

Three conclusions:

First, "LLMs orchestrating classic tools, with PoV-confirmed reproducibility" is the verified architectural answer for the Security direction. DARPA AIxCC, with 54 million lines of code, 54 synthetic vulnerabilities, 18 real 0-days, and a per-task cost of about $152, proved this route can scale; the 37%→86% identification-rate leap came from orchestration improvements, not model upgrades. Google Big Sleep then proved the same paradigm also holds in the "threat-intelligence-driven" scenario, and that governance mechanisms (analyst verification before disclosure, a human-oversight whitepaper) can run in parallel with higher output. The direct corollary for Harness builders is: invest in the orchestration layer and tool contracts, rather than waiting for larger models.

Second, the Security direction is the dual pole of "judgment automation" and "authorization tightening". Vulnerability reality can be automatically reproduced by machines via PoV — the judgment stage can be made fully automatic; while the blast radius of destructive actions such as isolating hosts or blocking IPs is borne by assets and people — so the authorization stage must be the strictest. The AI SOC data further shows authorization should be gradual: analysts' recognition of capability (4.6/5.0) is significantly higher than their trust in autonomous responses (3.9/5.0), and "detect first, automatically respond later" is not conservatism but the route with the highest rollout success rate.

Third, AI itself is the largest new attack surface for the Security direction. Three figures from the IBM 2025 report — 13% of organizations reported AI breaches, 97% of breached organizations lack AI access controls, and 1 in 5 suffered a breach due to shadow AI — draw a defense line different from the traditional SOC. The three mitigations ATLAS gives (GenAI guardrails, AI BOM, AI telemetry logs), plus access controls and governance policies, constitute this direction's minimum L6 configuration. The depth to which security teams govern AI assets determines the true capability level of the AI security system.

Limitations that must be confronted: no domestically verifiable rollout case of "AI code review / SAST false-positive reduction" in the AppSec direction was obtained; the B-grade engineering details of AIxCC (smaller models outperforming much larger models, Theori's ~99.9% false positive rate, etc.) come from Chinese-community reviews and should be cited as such; the IBM report's "extensive use" lacks a unified definition, so the $1.9 million and 80 days should be read as between-group comparisons, not causal conclusions.

Information Gaps Statement

The following content could not be sourced from Grade A/B primary evidence; it is handled here under the "no fabrication" principle)Skip and must be independently verified when used:

1. Quantified rollout cases of incident response (IR) and SOC on the Chinese side — no Grade A/B primary source was obtained; the alert-noise-reduction and MTTR improvement figures claimed by domestic vendors (e.g., Sangfor, Qi-Anxin, QingTeng Cloud Security, etc.) are all Grade C and were not used in this document.
2. Verifiable rollout cases of "AI code review / SAST false-positive reduction" in the AppSec direction — none obtained; AIxCC is vulnerability discovery and patching validation and can partially substitute as evidence, but it is not an enterprise AppSec pipeline case.
3. Gartner's maturity-curve positioning of AI SOC Agents (2025 peak of inflated expectations, 1—5% adoption) — a Grade C retelling, not used in this document; Gartner's prediction that "40% of enterprise applications will feature task-specific AI agents by 2026" is likewise a retelling.
4. Details of ISO/IEC 27001:2022 Annex A controls and AI-related extensions — only the standard version and publication date were confirmed; clause details were not obtained.
5. Original English sources of AIxCC's B-grade engineering details — "8-billion-parameter-class models often outperform much larger models on code pattern recognition," "Theori used Infer with a ~99.9% false positive rate," "Buttercup's per-task cost was about $181," etc., come from secyanbao (secrss.com)'s Chinese review of the DARPA results and were not traced back item by item to each team's technical report.
6. The original survey-definition text of "widespread use of AI and automation" in the IBM 2025 report — the official press release confirmed the $1.9 million saved and 80 days shortened figures, but the questionnaire definition of "widespread use" was not sourced.
7. EU AI Act timeline version conflict — because of rumors of adjustments under the AI Omnibus (May 2026 provisional agreement) (e.g., new timelines of 2026-12-02 / 2027-12-02 / 2028-08-02), this document takes the timeline published by the EU's official service desk as the baseline, treats the Omnibus adjustments as "proposed," and defers to the EU's official text as final.
8. Chinese authoritative interpretations of and domestic implementation practices for NIST SP 800-218A — none obtained; this document only cites it as a practical reference for generative AI supply chain security.

6. References

1. Decision to Amend the Cybersecurity Law of the PRC (Presidential Order No. 61), newly added Article 20 — National People's Congress, 2025. http://www.npc.gov.cn/c2/c30834/202601/t20260105_450980.html
2. DARPA AI Cyber Challenge Finals Results (86%/68%, 18 0-days, $152) — DARPA, 2025. https://www.darpa.mil/news/2025/aixcc-results
3. Review of the AIxCC Finals Results: The Paradigm of LLMs Orchestrating Classic Tools and PoV Judgment — secyanbao (secrss.com), 2025. https://www.secrss.com/articles/86038
4. Cybersecurity updates: Summer 2025 (Big Sleep and CVE-2025-6965, Timesketch + Sec-Gemini, FACADE) — Google, 2025. https://blog.google/technology/safety-security/cybersecurity-updates-summer-2025/
5. Google's AI Big Sleep finds 5 new WebKit vulnerabilities — The Hacker News, 2025-11. https://thehackernews.com/2025/11/googles-ai-big-sleep-finds-5-new.html
6. IBM Report: 13% of Organizations Reported Breaches of AI Models or Applications — IBM / Ponemon Institute, 2025-07-30. https://newsroom.ibm.com/2025-07-30-ibm-report-13-of-organizations-reported-breaches-of-ai-models-or-applications,-97-of-which-reported-lacking-proper-ai-access-controls
7. OWASP Top 10 for LLM Applications (2025 edition) — OWASP GenAI Security Project. https://genai.owasp.org/llm-top-10/
8. MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) — MITRE. https://atlas.mitre.org
9. Secure AI v2 Release (ATLAS April 2026 update) — MITRE CTID, 2026-05-06. https://ctid.mitre.org/blog/2026/05/06/secure-ai-v2-release
10. Secure Software Development Practices for Generative AI and Dual-Use Foundation Models (NIST SP 800-218A) — NIST CSRC, 2024. https://csrc.nist.gov/pubs/sp/800/218/a/final
11. ISO/IEC 27001:2022, Information security, cybersecurity and privacy protection — ISMS — Requirements — ISO, 2022-10. https://www.iso.org/standard/27001
12. Interpretation of GB/T 45654—2025, Network Security Technology — Basic Requirements for Generative Artificial Intelligence Services — National Information Security Standardization Technical Committee (SAC/TC260). https://www.tc260.org.cn/tc260/hygd1/202403/b429d868525e48c3b7d12a0ec8f82e5e.shtml
13. Notice on Issuing the Measures for Labeling AI-Generated Content (Letter of the CAC No. 2 [2025]) — CAC and three other departments, 2025. https://www.cac.gov.cn/2025-03/14/c_1743654684782215.htm
14. CyberSentinel-LLM — Computers, Materials & Continua, vol. 89, no. 1, 2025, Tech Science Press. https://www.techscience.com/cmc/v89n1/68397/html
15. EU AI Act — Implementation Timeline (based on the EU official service desk) — European Commission AI Act Service Desk. https://ai-act-service-desk.ec.europa.eu/en/ai-act/timeline/timeline-implementation-eu-ai-act