Data Science Group

1. Introduction

1.1 Positioning of the Group

The Data Science group covers a class of agent tasks for which data and computation are the means of production: from the construction and governance of data pipelines, to question-answering and self-service analytics on top of metric definition basis, to literature surveys, statistical inference, high-performance job scheduling, and autonomous research aimed at scientific discovery.

Compared with the Software Engineering group and the Knowledge Collaboration group, the tasks of this group have three stable structural characteristics:

• The output is not text, but executable assertions. A SQL statement, a dbt model, a GCMC simulation, a causal-effect estimate — all run on real systems and can be directly validated or falsified.
• Errors are silent. A crashing pipeline immediately tells you it is broken; an incorrectly computed pipeline tells you nothing — it returns a well-formatted number complete with a chart. This is the most fundamental risk difference between this group and creative directions.
• Constraints come from the external world. Permission boundaries are set by data classification, correctness is set by physical laws or business definition basis, and cost is set by compute quotas. The model can neither authorize itself nor justify itself.

Therefore, this group's reliance on the AI Harness is extremely high: the model provides hypotheses and drafts, and the Harness provides trustworthiness.

It should be made clear that "AI Harness × data science," as a combined term, currently has no public standard or authoritative definition. The content in this document about "the emphasis of the six-layer capability model across the different directions" is all mapping analysis based on the Harness six-layer model definitions and publicly documented case facts; it is opinion and inference, not established fact, and should be distinguished from verifiable data and standard provisions when reading.

图 1-1｜六层 Harness 能力栈与六方向映射

数据来源：基于本文分析绘制的示意图。

1.2 Four Common Constraints of Data Science Tasks

1.2.1 Reproducibility Takes Priority Over Time-to-Output

An analytical conclusion that cannot be reproduced has zero or even negative value. All directions in this group require: version-traceable input data, rebuildable execution environments, controllable randomness (fixed random seeds), and recomputable conclusions. ACM gives a four-level terminology for reproducibility: Repeatability (same team, same experimental setup), Replicability (different team, same setup), Reproducibility (different team, different setup), Reusability (artifacts and documentation complete and reusable). Harness's L4 memory and state layer should use these four levels directly as the acceptance basis for memory structure design.

1.2.2 Data Classification Is a Hard Boundary

Data classification and masking are not soft constraints of the "just remind them in the prompt" kind; they are hard boundaries that must be enforced at the L6 governance and security layer: row- and column-level permissions (RLS/CLS), PII masking, cross-border transfer restrictions, and industry-specific supervisory requirements (the Data Security Law, the Personal Information Protection Law, and the Cybersecurity Law impose mandatory requirements on data classification and grading, privacy protection, and cross-border transfer). The model should never become the arbiter of permissions.

1.2.3 Executable Ground Truth Is the Most Valuable Asset

This group possesses an advantage rarely seen in other industry groups: ground truth is often executable. SQL result sets can be compared, unit tests can be run, physical conservation laws can be checked, and independent laboratories can perform synthetic validation. Harness's L5 evaluation layer should make this kind of executable validation a first-class citizen, rather than relying on the subjective judgment of "does it look right."

It must also be clearly understood that not every direction has ground truth. Deep research (03-research) and some statistical-inference tasks precisely lack ground truth, in which case one can only fall back on alternatives such as "citation verifiability rate," "competition dataset benchmarks," and "review agent grading + human oversight."

1.2.4 Cost and Compute Budget Are First-Class Constraints

The failure modes of data science tasks include one special category: ran correctly but ran out of money. Full-table scans, unconstrained job submission, and runaway literature-retrieval rounds can all push cost to unacceptable levels. Harness should provide core-hour quotas, query cost estimation, and budget guardrails at L6.

2. Definition and Positioning Comparison of the Six Directions

2.1 Directory of Directions

2.2 Four Discriminating Dimensions of Positioning Differences

These four dimensions mean that the center of gravity of Harness design differs completely across directions: for 01 the bottleneck is context and governance, for 02 context and evaluation, for 03 orchestration and source allowlists, for 04 tools and state persistence, for 05 tools and orchestration, and for 06 evaluation and orchestration. Fitting all six directions into one generic template is the easiest mistake this group can make.

3. Mapping the Six Harness Layers to the Six Directions

3.1 Mapping Matrix

The stars in the matrix below are a focus analysis based on publicly documented case facts, used to express the relative priority of design decisions; they do not represent a rating of any public standard.

3.2 Per-Layer Differences Within the Group

3.2.1 L1 Context Engineering Layer

For this group, context is not "documents" but the data assets themselves: schema, lineage, metric definition basis, data dictionaries, data samples, and literature evidence. The quality of context engineering directly determines success or failure — there are public reports that only 9% of data engineers are satisfied with the Airflow DAGs generated by general-purpose AI tools, with the top complaints being hallucination rate and outdated syntax; the root cause is not limited model capability, but the model's lack of context such as the project DAG, execution environment, and existing conventions (these figures come from second-hand reports; see the Information Gap Statement for details).

• 01 Data: context = project DAG + schema + existing model conventions + lineage + semantic layer.
• 02 Analytics: context = schema linking (should be treated as a RAG problem rather than stuffing the entire schema into the prompt) + a governed semantic layer. Enterprise repositories often contain more than 1,000 columns, so putting everything in is infeasible.
• 04 Science / 06 AI4S: context = protocols, data dictionaries, domain knowledge, failure records.

3.2.2 L2 Tools & Execution Layer

This group must have read-only sandboxes: SQL sandboxes, Jupyter/Colab kernels, HPC job submitters, and various MCP data and science tool servers. A repeatedly validated design insight is that agents should not call the scheduler directly, but should produce job descriptions that can be carried by a workflow engine. In Argonne National Laboratory's agentic material-screening work on Aurora, the MCP tools do not run simulations directly; instead they emit Parsl applications, and Parsl handles placement, concurrency, and fault tolerance — "the LLM never touches the scheduler." This is the best evidence for separating L2 and L3, and it should also be established as the default architecture in 05-hpc.

3.2.3 L3 Orchestration & Control Layer

DAGs and workflows are the native form of data science (Airflow/dbt/Parsl/Slurm), and Harness should reuse rather than reinvent them. Slurm is already a mature orchestration layer in itself: job arrays, job dependency building DAGs, and backfill scheduling. Harness's value lies in translating natural-language objectives into job graphs that these engines can digest.

Deep research, on the other hand, converges on a different path: the scope / research / write three-stage pipeline, and an orchestrator-worker multi-agent architecture. The core value of multi-agent here is context isolation — each sub-agent has its own independent context window, avoiding pollution of a single window.

3.2.4 L4 Memory & State Layer

The core memory of this group is artifacts and experiment records: notebook state, dataset snapshots, experiment run results, model versions, citation repositories, failure records. The 04 Science direction depends most heavily on this layer, where notebook variable state needs to persist across calls. The 06 AI4S direction, in turn, emphasizes that long-term memory should include failed experiments and their root-cause analysis — some argue that AI summarization tools can systematically index technical failures to form a searchable troubleshooting knowledge base.

3.2.5 L5 Evaluation & Observability Layer

This is the layer with the greatest differences across the group, and the one that best demonstrates Harness's value:

• 01/02: Executable ground truth is abundant. A regression set of verified queries should be built; in addition to execution accuracy (EX), the valid efficiency score (VES) should also be considered — a query that returns correct rows only through a full table scan is a production incident waiting to happen.
• 03: Ground truth is missing; only the citation verifiability rate, external benchmarks (such as DRACO, SealQA-style) and human review are available.
• 04: Some scenarios have no ground truth (one practitioner explicitly noted that "how to evaluate agent performance without real labeled data" is the core challenge); alternatives are competition datasets + review agent grading + process review.
• 05: There is physics-level ground truth (conservation, convergence), but evaluation cost is extremely high (a single simulation can take 1,600~4,400 seconds).
• 06: Ground truth is not human labels, but physical laws, experimental validation and conservation-law constraints — this is the most fundamental difference between AI4S and the other directions; see 06-ai4s.md for details.

3.2.6 L6 Governance & Security Layer

This group carries the heaviest governance burden. Beyond general permissions and audit, there are three categories of dedicated constraints:

• Data side: RLS/CLS enforcement, PII masking, pipeline-level data classification tagging, compliance checks built into the orchestration layer.
• Compute side: core-hour quotas, QoS fairness, GRES device isolation.
• Research side: research integrity (no fabrication of data or citations), experimental ethics review, biosafety, and the "submit scientific data first, then accept the science & technology program project" mechanism established by the Measures for the Administration of Scientific Data.

4. Industry Standards and the Standards Foundation

All six directions of this group share a set of cross-directional standards and policy foundations. The numbers and names of the following items all come from public sources; for the specific content of the provisions, the original text of the standards prevails.

4.1 Data Governance and Data Quality

4.2 Research Data and Reproducibility

4.3 Data Elements and Industry Policy

• Interim Provisions on the Accounting Treatment of Enterprise Data Resource-Related Matters (Cai Kuai [2023] No. 11): issued 2023-08-01, effective 2024-01-01. Items that meet the definition of intangible assets are recognized as intangible assets, and items that meet the definition of inventory are recognized as inventory; on the balance sheet, a "of which: data resources" line item is added under "inventory", "intangible assets", and "development expenditure"; a "mandatory disclosure + voluntary disclosure" approach is adopted.
• The "AI for Science" special deployment: launched jointly by the Ministry of Science and Technology and the National Natural Science Foundation of China in 2023-03, centering on key fields such as drug R&D, gene research, biological breeding, and new material research, advancing algorithmic model innovation, open sharing of research data, and construction of compute platforms, and supporting the heterogeneous integration and development of high-performance computing centers and intelligent computing centers.
• The "Data Elements × Three-Year Action Plan (2024–2026)": proposes implementing the "Data Elements × Sci-Tech Innovation" key action, using scientific data to support technological innovation, focusing on fields such as biological breeding, new material creation, and drug R&D.

5. The Three-Generation Architecture Evolution as It Applies to the Data Science Group

The key leap of the third generation is that evaluation has changed from an accessory into infrastructure. In the first two generations, which had no evaluation layer, the output of data science agents could only be checked by human effort; entering the third generation, executable validation sets, physical-constraint checks, and review agents become the default configuration, which is why "turning uncertainty into predictability" truly holds in data science scenarios.

6. Common Challenges and Countermeasures

7. Document Navigation

7.1 Group-Level Documents

7.2 Direction Documents

Each direction document uniformly includes: introduction, glossary, three case studies, practice standards (AGENTS.md spec / SKILL.md spec / landing checklist), summary, and references, with an Information Gap Statement given before the references.

7.3 Recommended Reading Paths

• Data platform and data warehouse practitioners: README.md → 01-data.md → 02-analytics.md
• Research and algorithm practitioners: README.md → 04-science.md → 06-ai4s.md
• Supercomputing and infrastructure practitioners: README.md → 05-hpc.md → 06-ai4s.md
• Research and intelligence practitioners: README.md → 03-research.md → 04-science.md
• Architecture and governance leads: README.md → AGENTS.md → SKILL.md → read the "practice standards" section of the direction documents as needed

8. Summary

The Data Science group is where the value of the AI Harness is easiest to quantify, and also easiest to underestimate.

Easy to quantify, because most ground truth here is executable: SQL can be compared, physical laws can be checked, and experiments can be reproduced. Every evaluation layer Harness adds translates directly into a drop in error rate and a rise in trust.

Easy to underestimate, because failures here barely make a sound. An incorrectly computed pipeline does not raise an error; a hallucinated survey appears complete and respectable with proper citation formatting; a candidate structure proposed by AI but not yet validated by physics quietly lies in the database waiting to be falsified.

Accordingly, the group's six direction documents jointly uphold one main thread: do not take "looks right" as "is right". 01 uses lineage and quality gates against silent errors, 02 uses a semantic layer and regression sets against definition drift, 03 uses source verifiability against hallucinated citations, 04 uses experiment design and review agents against spurious correlation, 05 uses the forced separation of reasoning and constraints against scheduling risk, and 06 uses physical laws and experimental validation against novelty hallucinations.

All six directions share the six Harness layers, but each has a different center of gravity. Any scheme that tries to cover all six directions with one general-purpose agent template will fail precisely where rigor is needed most.

Information Gap Statement

During the writing of this document, some content could not be confirmed through reliable sources; it has been marked in the body with [To be filled] or [To be verified], or is set out collectively in the table below:

1. An authoritative definition, market size, and adoption rate for "AI Harness × data science" — no reliable public source exists. This document gives no statement of the form "N% of organizations have adopted a data science Harness"; the related mapping matrices are analytical content based on the six-layer model definitions and publicly documented case facts, and are neither standards nor facts.
2. No official or industry-recognized source text found for the AGENTS.md / SKILL.md standards in the data science direction. The spec texts given in this group's AGENTS.md, SKILL.md, and sections 4.1 / 4.2 of the direction documents are all proposed standard drafts put forward by this document, and do not represent the position of any standards organization.
3. Some details of DCMM 2.0 (GB/T 36073-2025) remain to be verified: the two items "security weight raised from 15% to 25%" and "AI governance mandatory at level 4 and above" come from second-hand accounts and have not been verified in the public full-text system of the national standard; "capability domains expanded from 8 to 9 (adding data assets), capability items increased from 28 to 33, effective 2026-07-01" is a consistent account across multiple sources.
4. The exact year of ISO 8000-2 is to be verified (different sources give two versions, 2020 and 2022); this document writes only the number, not the year.
5. The page numbers and volume/issue of the original FAIR paper were not directly verified against the original page in this search; this document cites only the FAIR principles themselves and the survey data, not the paper's page numbers.
6. No officially published details were found for the specific funding amounts and number of projects of China's AI4S special program; the specific provision text of the three national standards for scientific data submission (GB/T 39908/39909/39912-2021) was not obtained.
7. All figures in this document that come from vendor self-reporting or second-hand accounts have been labeled with their source or marked [To be verified] at the point of appearance, and have not been used bare.

9. References

1. DAMA International, DAMA Data Management Body of Knowledge (DAMA-DMBOK) — DAMA International. https://www.longshidata.com/blog/c/c2026062301.html
2. Data Management Capability Maturity Assessment Model (GB/T 36073-2018); DCMM 2.0 (GB/T 36073-2025) — China national standards. https://ima.qq.com/wiki/?shareId=0258e6e62f0b37168d979fc9920268b9e4a313954973dbcf7e0469f132358841
3. Summary of the ISO 8000 data quality family of standards — ISO/TC 184/SC 4. https://quality.arc42.org/standards/iso-8000
4. ISO 8000 series item description — Wikipedia. https://en.wikipedia.org/wiki/ISO_8000
5. Open Data: Progress, Challenges and Institutions (including FAIR ten-year survey data) — Springer Nature, official blog. https://springernature.com/gp/librarians/the-link/open-science-blogpost/open-data-progress-challenges-institutions/52378334
6. ACM Artifact Evaluation terminology definitions (Repeatability / Replicability / Reproducibility / Reusability) — arXiv 1908.05986. https://arxiv.org/pdf/1908.05986.pdf
7. Measures for the Administration of Scientific Data (Guo Ban Fa [2018] No. 17) — General Office of the State Council. https://www.gov.cn/zhengce/content/2018-04/02/content_5279272.htm
8. Interim Provisions on the Accounting Treatment of Enterprise Data Resource-Related Matters (Cai Kuai [2023] No. 11) — Ministry of Finance. https://www.gov.cn/gongbao/2023/issue_10746/202310/content_6907744.html
9. Reports on the "AI for Science" special deployment and the "Data Elements × Three-Year Action Plan" — China Journalists Association Network. https://union.china.com.cn/cmdt/txt/2024-03/26/content_42737343.html
10. Explanation of the chain of scientific data management norms of the Chinese Academy of Sciences — National Basic Disciplines Public Science Data Center. https://cms.casdc.cn/article/312
11. OpenLineage official specification documentation — Linux Foundation. https://openlineage.io/
12. Overview of the W3C PROV conceptual model (as relayed) — W3C. https://www.w3.org/TR/prov-overview/
13. Agentic MOF Screening on Aurora (the "LLM never touches the scheduler" design) — supercomputing.news. https://www.supercomputing.news/hpc/agentic-mof-screening-aurora
14. AI for Data Engineering 2026 (data relayed from the Astronomer / dbt Labs report) — technical blog. https://baeseokjae.github.io/posts/ai-for-data-engineering-2026
15. Overview of Data Agent architectures: from architecture to practice — technical analysis blog. https://ibillxia.github.io/blog/2026/08/20/data-agent-servey-from-arch-to-practice