Repository navigation
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Feature description
Implement Scheme B: a Scope represents a project or workspace and can contain multiple rag Artifacts. Users create a rag before importing documents into it. Imported Sources remain immutable; document Artifacts own their processed content.
Implementation scope
- Register
raganddocumentthrough existing Artifact management and reuse configuration revisions, tags, provenance, and Scope authorization. - Allow multiple
ragArtifacts in one Scope. Creating or renaming a knowledge base must not create or rename its Scope. - Define explicit document ownership and logical document identity within a Scope and target
rag. Importing the same path into two knowledge bases creates separate documents. - Extend the OpenDAL/
opendalfsintegration for Markdown and independently imported images. Record the original path, immutable managed snapshot, digest, media type, and captured metadata. - Define a dedicated document Source type directly consumable only by the document pipeline. Enforce this policy during input selection, execution, and Artifact write validation.
- Persist target-specific import requests containing the
ragreference, exact SourceRef, logical document identity, configuration snapshot, and request/input generations. - Atomically record accepted target work and request background processing in the database transaction. Reusing an existing Source for another
ragmust still schedule that target's work. - Follow existing Source/Artifact API and client conventions; define explicit update, deactivation, removal, and downstream invalidation contracts.
Acceptance criteria
- Multiple
ragArtifacts can be created in the same Scope without a singleton constraint. - An empty
ragcan be created without imported document Sources or document-processing work. - Imports reject missing, inaccessible, inactive, or mismatched targets.
- Accepted snapshots remain readable after original files are overwritten or moved.
- Identical repeated imports are idempotent; importing an existing Source into another
ragcreates the required target work. - Interrupted ingestion cannot leave an accepted target import without recoverable processing work.
- Other Artifact families do not automatically consume dedicated document Sources.
- Include migrations, API contracts, authorization and behavior tests, and configuration documentation.
Dependencies and boundaries
Parent tracking issue: #2440.
Design: Scheme B RFC.
The knowledge-base family is rag; the content family is document. Parsing, OCR, chunking, and index execution belong to #2442.
Phase one supports Markdown and independently imported images only. Plain TXT, PDF, Office, and automatic OCR of images referenced by Markdown are excluded.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Type
Projects
- StatusShow more project fieldsTodo
Activity
[-]feat: add knowledge-base foundations and extensible document ingestion[/-][+]feat: add rag artifact management and targeted document ingestion[/+]