Knowledge Collaboration Group (Projects / Works / Docs / RAG / Knowledge / Workflow / BPM)

1. Introduction

1.1. Group Positioning and Background

The Knowledge Collaboration Group is AI Harness's empowerment blueprint in the knowledge and collaboration work domain. If the software engineering group addresses "how code is reliably written", then the Knowledge Collaboration Group addresses "how what the organization already knows is reliably extracted, used, managed, and made to flow".

The first batch of real scenarios where enterprises deploy large models is, in the overwhelming majority of cases, not about writing code but about: finding a document, answering a question, drafting material, processing a ticket, running a process end to end. What these scenarios share is that the capability bottleneck is not the model itself but the carrier layer between the model and the organization's existing knowledge assets and existing process assets. That carrier layer is precisely AI Harness.

This group splits this domain into seven interlocking directions:

Projects (project workspace) → Works (work tasks and tickets) → Docs (document creation and collaboration) → RAG (retrieval-augmented generation) → Knowledge (knowledge asset governance) → Workflow (workflow orchestration) → BPM (business process management)

These seven directions are not a parallel feature list but a progressive capability chain.

图 1-1｜知识协同组三层递进能力架构（信息可达 → 工作可编排 → 流程可治理）

数据来源：基于本文分析绘制的示意图。

1.2. Definition and Boundaries

Per the unified definition in the project parameter card, AI Harness (AI harness layer) sits between the foundation models and real business systems, and is responsible for converting the model's uncertainty into engineering predictability. In this group, this conversion manifests concretely as three kinds of predictability:

Boundaries with adjacent concepts (applied consistently throughout):

• This group does not cover model training, fine-tuning, or inference infrastructure (belonging to 06-Data Science Group and 04-Hardware R&D Group).
• This group does not cover code generation and the software-delivery pipeline (belonging to 03-Software Engineering Group), but shares the AGENTS.md / SKILL.md format and evaluation methodology.
• In this group, the compliance and audit part of the BPM direction shares conformance checking and documented-information requirements with the 08-Risk & Compliance Group.

1.3. Projection of the Three-Generation Architecture Evolution in This Group

According to the three-generation architecture evolution in the parameter card, the maturity distribution of each direction in this group is as follows:

The core change of the third generation is: the artifacts of knowledge and collaboration have shifted from "conversations" to "versionable, regressable, auditable engineering assets". This change directly determines that the documents for every direction in this group use "artifact contract + evaluation gate + governance red line" as their writing thread.

2. Positioning Comparison of the Seven Directions

2.1. Positioning Comparison Table

2.2. Progression and Dependency Between Directions

This group recommends understanding the relationship between the seven directions through a three-layer progression of "information accessibility → work orchestration → process governance", which is consistent with the path of Harness capabilities moving up from L1 to L3 and then to L6 (the following is this document's analysis):

Key dependency reminder: skipping Layer 1 and directly building Layer 3 is the most common deployment-failure path in this group. Without trustworthy retrieval and knowledge governance, process automation merely scales up the errors.

3. Mapping of the Seven Directions × Harness Six-Layer Capabilities

3.1. Mapping Matrix

The "primary/secondary layer" determination in the table below is an analytic conclusion of this document, not retrieved from research; please note this when citing.

Legend: ★★★ primary layer (the capability core of that direction) / ★★ secondary layer (important support) / ★ basic dependency.

3.2. Common Implementation Forms of Each Layer in This Group

One of the most important L6 conclusions in this group: permissions must be enforced at the retrieval layer, not only at the application layer. The official report of the UK government's M365 Copilot cross-department trial states this in first-hand terms — the tool "adopts the permissions of the end user and will only retrieve documents that a user could normally access", while the officials also warn that "Copilot's ability to search and query using a user's access may highlight issues when users have access to files they shouldn't" (GOV.UK official publication, R1). This means permission inheritance is not "safe once configured", but instead amplifies existing permission-configuration errors.

4. Common Challenges of This Group and Countermeasures

4.1. Challenge 1: Knowledge Accessibility and Hallucination Risk

Phenomenon: Without reliable corpus support, the model generates grammatically correct but factually wrong answers.

First-hand fact (R1): Anthropic's official blog post "Introducing Contextual Retrieval" states that, at the recall@20 metric, the baseline retrieval failure rate is 5.7%; it drops to 3.7% with Contextual Embeddings; to 2.9% when Contextual BM25 is added; and to 1.9% when reranking is added on top — the official wording is "reduce the number of failed retrievals by 49% and, when combined with reranking, by 67%". The one-time preprocessing cost is about $1.02 / million document tokens.

First-hand fact (R1): data.world's GenAI Benchmark I shows that across 43 business questions, a bare LLM achieves only 16.7% average execution accuracy, and 0% on the two high-schema-complexity categories "metrics and KPIs" and "strategic planning"; after connecting a knowledge graph, accuracy rises to 3×; with GenAI Benchmark II adding OBQC (Ontology-based Query Check) and LLM Repair, accuracy reaches 72.55%, a 4.2× improvement over pure SQL.

Countermeasures:

1. Retrieve first, generate later — never output a claim not backed by recall ("no answer without a source").
2. Set a recall-evaluation gate (Recall@K) on the retrieval chain rather than looking only at end-to-end answer quality.
3. For structured/high-complexity queries, prefer the knowledge-graph or ontology-validation path over plain-text RAG.

Balance note: the above 16.7% → 3× → 72.55% figures come from data.world's own benchmark (43 questions, 13 tables, a single insurance domain, GPT-4 zero-shot); third-party cautious assessments consider it not validated by large-scale third-party reproduction, so it should be treated as an upper-bound reference for that team's own method, not as an industry-wide expectation (see 05-knowledge.md).

4.2. Challenge 2: Permission Awareness and Data Classification

Phenomenon: Vector stores and LLMs are not inherently good at limiting the information scope by role; once the corpus is ingested, permission boundaries are easily lost in the retrieval step.

Countermeasures:

1. Permission-aware retrieval: filter results at the retrieval layer rather than only redacting after generation.
2. The knowledge graph can attach permission policies at the graph level, excluding unauthorized nodes and relations at the query layer.
3. Push data classification upstream into the ingestion step, making sensitivity level an index field rather than a post-hoc label.
4. Regularly run red-team tests with "over-permission retrieval" cases and fold the failures into the regression set.

4.3. Challenge 3: Physical Limitations of Long Context

Phenomenon: Even when the context window is large enough, the model's utilization of information in the middle remains markedly lower.

First-hand / authoritative facts:

• Lost in the Middle (Liu et al., TACL 2024): multi-document QA shows a U-shaped curve — accuracy is highest when the answer sits at the head or tail of the context and drops significantly in the middle; under some configurations, burying the relevant information in the middle even performs below the model's closed-book performance, meaning the recalled context is a net loss (R2).
• RULER (Hsieh et al., NVIDIA, COLM 2024): of the 17 long-context models evaluated, only about half maintain satisfactory performance at 32K length (R2).
• NoLiMa (Modarressi et al., ICML 2025): among 13 models claiming ≥128K support, 11 fall below half of their own short-context accuracy already at 32K; GPT-4o drops from 99.3% to 69.7% (R2).
• When RAG can be omitted (R1, Anthropic official wording): "If your knowledge base is smaller than 200,000 tokens (about 500 pages of material), you can just include the entire knowledge base in the prompt... with no need for RAG or similar methods."

Countermeasure: long context is not a substitute for RAG, but a complement to it. Context budget management, result reranking, and placing key information at the head and tail are essential L1 engineering actions.

4.4. Challenge 4: Missing Standards and Inconsistent Evaluation Criteria

Phenomenon: Among this group's seven directions, only Knowledge (ISO 30401) and BPM (BPMN/CMMN/DMN/IEEE 1849/OCEL) are backed by formal standard bodies; the RAG, Docs, and Works directions still have no corresponding ISO/IEEE formal standard.

Confirmed information gaps in this group (must be honestly declared in the relevant direction documents):

Countermeasures:

1. For directions with formal standards (Knowledge, BPM), use the standard clauses as the compliance baseline.
2. For directions without formal standards, adopt community/vendor de facto standards as engineering conventions, and clearly mark their non-standard nature — for example, AGENTS.md should be described as "a de facto open-source community standard, hosted by the Agentic AI Foundation under the Linux Foundation", not written as a standard number.
3. Establish an internal Golden Dataset and gate thresholds in-house, using proprietary data to compensate for the lack of public benchmarks.

4.5. Challenge 5: Engineering the Human-in-the-Loop

Phenomenon: Human review is often treated as "a patch at the end of the process", leading to high rollback costs and ambiguous accountability.

Countermeasures:

1. Human-intervention points (human-in-the-loop) should be written into the workflow definition as first-class nodes, not as a bypass.
2. High-risk nodes requiring human review should be explicitly enumerated: document finalization, knowledge publishing, process release, and actions involving money and legal obligations.
3. Review nodes must produce structured records (reviewer, time, conclusion, reason for change) for L5 observability and L6 audit to consume.
4. CMMN 1.1's case / plan item / sentry (entry & exit criteria) semantics are the closest standard model for expressing "knowledge-worker ad hoc decisions" (see 07-bpm.md).

5. Document Navigation

Suggested reading paths:

• Engineering rollout path: 04-rag.md → 05-knowledge.md → 01-projects.md → 03-docs.md
• Process automation path: 06-workflow.md → 02-works.md → 07-bpm.md
• Compliance / audit path: 05-knowledge.md → 07-bpm.md → AGENTS.md

6. Summary

The Knowledge Collaboration Group is the main battleground where AI Harness moves from "technology concept" to "organizational productivity". The group's seven directions form a complete chain from context boundaries (Projects) to task orchestration (Works / Docs), from retrieval augmentation (RAG) to knowledge asset governance (Knowledge), and on to workflows and business processes (Workflow / BPM).

The group's core engineering judgments can be summarized in five points:

1. Retrieval quality sets the ceiling for generation quality. Contextual Retrieval + Reranking can reduce the retrieval failure rate from 5.7% to 1.9% (-67%, R1), and this gain comes from the engineering layer rather than the model layer.
2. Structured knowledge is necessary, not optional, for complex queries. A bare LLM scores 0% on high-schema-complexity questions, and the lift from knowledge graphs and ontology validation is order-of-magnitude (R1).
3. Long context cannot replace context engineering. The U-shaped positional effect and evidence from RULER / NoLiMa show that piling on context can produce net-negative returns (R2).
4. Permissions must be pushed down to the retrieval layer. Permission inheritance amplifies existing configuration errors — this is the most typical and most easily overlooked L6 risk in this group (R1).
5. Missing standards are the norm; evaluation sets are the compensating device. Five of the group's 7 directions lack formal standards, so an internal Golden Dataset and gate thresholds should serve as the factual baseline.

To be candid, the group has obvious information gaps: most of the quantitative parameters for RAG chunking and retrieval come from second-hand aggregation sites, all Gartner market data is second-hand retelling, and the original text of ISO 30401 clauses cannot be directly cited due to the paywall. These gaps are listed item by item in the "information-gap declarations" of each direction document, with no fabricated compensation.

7. References

1. Introducing Contextual Retrieval — Anthropic, 2024. https://www.anthropic.com/news/contextual-retrieval
2. Generative AI Benchmark I: Increasing the Accuracy of LLMs in the Enterprise with a Knowledge Graph — data.world. https://data.world/blog/generative-ai-benchmark-increasing-the-accuracy-of-llms-in-the-enterprise-with-a-knowledge-graph
3. GenAI Benchmark II: Increased LLM Accuracy with Ontology-Based Query Checks and LLM Repair — data.world. https://data.world/blog/genai-benchmark-ii-increased-llm-accuracy-with-ontology-based-query-checks-and-llm-repair
4. ISO 30401:2018 Knowledge management systems — Requirements — ISO. https://www.iso.org/standard/68683.html
5. Microsoft 365 Copilot Experiment: Cross-Government Findings Report — GOV.UK, 2025. https://www.gov.uk/government/publications/microsoft-365-copilot-experiment-cross-government-findings-report/microsoft-365-copilot-experiment-cross-government-findings-report-html
6. AGENTS.md — A simple, open format for guiding coding agents. https://agents.md/
7. Equipping agents for the real world with Agent Skills — Anthropic Engineering. https://anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills
8. Building Effective Agents — Anthropic Research. https://www.anthropic.com/research/building-effective-agents
9. Serverless Workflow Specification — CNCF / serverlessworkflow. https://github.com/serverlessworkflow/specification
10. BPMN 2.0 — Object Management Group. https://www.omg.org/spec/BPMN/2.0/
11. CMMN 1.1 — Object Management Group. https://www.omg.org/spec/CMMN/1.1/About-CMMN
12. OCEL 2.0 — Object-Centric Event Log Standard. https://www.ocel-standard.org/
13. Lost in the Middle: How Language Models Use Long Contexts — Liu et al., TACL 2024. https://arxiv.org/abs/2307.03172
14. Model Context Protocol Specification Changelog — MCP. https://modelcontextprotocol.io/specification/2025-06-18/changelog