Skip to content

feat: add rag processing visibility, recovery controls, and three-backend validation #2444

Open

Description

@Zxf-xufeng

Feature description

Make the create-rag, import, document-processing, and index-publication lifecycle observable and recoverable, and validate the complete workflow on SQLite, SeekDB, and OceanBase.

Implementation scope

  • Expose rag identity, batch/item progress, current stage, failure reason, attempt, latest content revision, and searchable revision.
  • Distinguish an empty ready rag, an accepted import, completed parsing/OCR, and searchable content. Import acceptance is not processing success.
  • Provide supported retry and recovery operations that preserve the correct target and configuration without direct database edits.
  • Provide configurable concurrency, batch sizes, timeouts, execution budgets, index freshness, and processing/retrieval diagnostics without sensitive content in logs by default.
  • Use Markdown from a pinned oceanbase/oceanbase commit. Use separate OCR image fixtures with checked text and region capabilities.
  • Validate K1/K2 in the same Scope and K3 in another Scope, covering targeted scheduling, retrieval filters, independent lifecycles, and shared-snapshot references.
  • Document configuration, the selected OCR adapter/formats, supported backends, authorization granularity, and capability limitations.

Acceptance criteria

  • Users can distinguish empty-ready, accepted, processing, searchable, and failed states.
  • A first processing failure is inspectable and retryable even when no document Artifact exists yet.
  • Importing an existing Source into another rag schedules and completes the correct target's work.
  • Retries, restarts, configuration rebuilds, and late-worker handling preserve target and publication correctness.
  • Removing or rebuilding K1 does not affect K2, remove their Scope, or prematurely clean shared snapshots.
  • Markdown image references do not trigger automatic fetching or OCR.
  • Real OCR and the complete functional workflow are validated on all three backends.
  • Publish reproducible setup instructions, fixture references, results, and known limitations.

Dependencies and boundaries

Parent tracking issue: #2440. Depends on #2441, #2442, and #2443.

Design: Scheme B RFC.

A complete management UI and tens-of-thousands-of-documents scale evaluation are not required in this phase.

Feature-specific authorization, reliability, and tests ship with their implementing PRs. This issue provides the operational view and cross-component validation.

Activity

  1. AlexStocks commented on Nov 3, 2026

    @AlexStocks
    Contributor

    按当前 master(729dc4c5)与仓库既有门禁核对后的分析,供实现阶段确认。

    1. 可观测性字段不能另起一套
    AGENTS.md 的 Observability Changes 要求:新增结构化字段先登记对应 allowlist,并在中英文档同步其含义与单位;日志必须走 log_safely,失败不得改变业务结果。issue 里的「处理与检索延迟、计数、错误、模型用量」应复用 Server 现有 metrics 投影边界(readiness 白名单与 snapshot 异常是两条独立路径),而不是在 KB 模块内自建计数器或字段名。

    2. 「能区分四类失败」需要一个可断言的返回契约
    验收要求区分「解析缺口 / 索引不完整 / 过滤未命中 / 检索未命中」,但当前只描述了能力,没有可断言的形状。建议给出稳定的分类枚举或 reason code(并说明每一类在哪一层判定),否则这条验收只能靠人工读日志,测试无法保护。可参考现有 processing_diagnostics.py 的分层方式。

    3. 评估集要绑定代码身份
    仓库已有 evaluation/ 下的评估实践,报告需绑定 code_sha;若评估期间改动了运行时,必须在报告内标注或重算,否则报告不能作为该提交的证据。另外,本轮 #2388 给 SWE-bench Pro harness 引入了默认开启的 egress 允许列表(依赖安装期仅 pypi.org/files.pythonhosted.org/releases.astral.sh,agent 期仅模型端点),KB 的端到端/规模评估若复用该 harness,需要先确认所需主机在允许列表内或用 --allow-host 显式加入。

    4. e2e 预算是硬约束
    每个新增迁移 revision 都会乘进每个 e2e 用例的 bootstrap 预算;code-seekdb 分片还有 30 分钟步预算,超时表现为 cancelled 而不是失败。issue 里的「端到端覆盖 ingestion / 更新 / 删除 / 重建 / 重启恢复 / 组件替换」需要写清分片方式与预算,否则会以超时形式落到基线噪声里,无法归因。

    5. 「initial scale evaluation」需要阈值
    与 #2440 的规模问题同源:没有文档量级、chunk 量级、并发与延迟目标,「规模评估通过」不可判定。建议在 issue 内给出量级、阈值和可重复的执行命令。

  2. changed the title [-]feat: add knowledge-base operations and end-to-end validation[/-] [+]feat: add rag processing visibility, recovery controls, and three-backend validation[/+] on Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions