Skip to content

feat: add session-based context window management#38

Merged
Siddhant-K-code merged 4 commits intomainfrom
feat/session-management
Feb 24, 2026
Merged

feat: add session-based context window management#38
Siddhant-K-code merged 4 commits intomainfrom
feat/session-management

Conversation

@Siddhant-K-code
Copy link
Owner

@Siddhant-K-code Siddhant-K-code commented Feb 23, 2026

Stateful context windows with token budgets for long-running agent sessions. Entries are deduplicated on push, compressed through hierarchical levels as they age, and evicted when the budget is exceeded. Lowest-importance entries are compressed/evicted first.

Compression levels: full text → summary (~20%) → sentence → keywords. The preserve_recent setting keeps the N most recent entries at full fidelity.

Closes #31

Added

  • pkg/session — SQLite-backed session store with 13 tests
  • CLIdistill session create/push/context/delete
  • HTTP API/v1/session/create, /push, /context, /delete, /get (opt-in via --session flag)
  • MCP toolscreate_session, push_session, session_context, delete_session (opt-in via --session flag)

Docs

  • README — Session Management section with CLI/API/MCP examples, updated architecture diagram, roadmap, config example, Pipeline Modules
  • CHANGELOG[Unreleased] section with session management
  • FAQ — 4 new Q&As (What is Context Memory, What are Sessions, Memory vs Sessions, MCP tools list), updated roadmap status
  • mcp/README — 8 new tool docs (memory + session), updated config examples, 2 new integration patterns
  • examples/session_api.sh — Full API usage example
  • mcp/claude_desktop_config_full.example.json — Config with all features enabled

Usage

# API server
distill api --port 8080 --session

# MCP server
distill mcp --session

# CLI
distill session create --session-id task-42 --max-tokens 128000
distill session push --session-id task-42 --role tool --content "file contents..." --importance 0.8
distill session context --session-id task-42

Review Findings Addressed

All 10 findings from code review resolved:

  • P2: isDuplicate TODO, Create timestamp fix, enforceBudget loop
  • P3: CompressionSavings rename
  • Nits: removed unused ErrEmptyContent/CompressAge, sort.Slice, merged store constructors, word-boundary truncation

Stateful context windows with token budgets for long-running agent
sessions. Entries are deduplicated on push, compressed through
hierarchical levels as they age, and evicted when the budget is
exceeded. Lowest-importance entries are compressed/evicted first.

Compression levels: full text -> summary (~20%) -> sentence -> keywords.
The preserve_recent setting keeps the N most recent entries at full
fidelity.

- pkg/session: SQLite-backed session store with 13 tests
- CLI: distill session create/push/context/delete
- HTTP: /v1/session/* endpoints (opt-in via --session flag)
- MCP: create_session, push_session, session_context, delete_session
- README: session management docs, updated architecture + roadmap

Closes #31

Co-authored-by: Ona <no-reply@ona.com>
@Siddhant-K-code
Copy link
Owner Author

Code Review - PR #38

CI green (build + lint). 13 tests pass. 0 lint issues.

Verdict: Approve with fixes below

No blocking issues. Architecture follows the memory store patterns well.

Findings

Priority File Finding
P2 sqlite.go:437 isDuplicate full table scan per push entry. Add TODO noting O(n) scaling limit.
P2 sqlite.go:127-131 Create returns time.Now() twice instead of reusing the formatted timestamp. Subtle inconsistency with DB.
P2 sqlite.go:488-510 enforceBudget only compresses one level per push. A single large push can leave session over budget. Loop until within budget.
P3 sqlite.go:300-305 DedupSavings stat is misleading - rename to compression_savings.
Nit session.go:14-17 ErrEmptyContent declared but never used. Remove.
Nit session.go:148-152 CompressAge in Config never read. Remove or implement.
Nit sqlite.go:632-645 sortCandidates uses insertion sort. Use sort.Slice.
Nit session.go:193-225 sessionStoreFromFlags and sessionStoreFromConfig are nearly identical. Merge.
Nit sqlite.go:575-583 LevelSentence truncation at 50 chars can cut mid-word. Truncate at word boundary.
Nit session.go:224-225 Trailing blank line.

What's Good

  • Scan-then-close pattern used consistently (learned from memory store review)
  • preserve_recent keeps recent entries at full fidelity
  • Importance-based eviction ordering
  • handleContext accepts both GET and POST
  • handleDelete accepts both DELETE and POST
  • 13 tests covering all key paths

- P2: Add TODO to isDuplicate noting O(n) scaling limit
- P2: Fix Create timestamp consistency (reuse time.Time)
- P2: Loop enforceBudget until within budget or no progress
- P3: Rename DedupSavings to CompressionSavings
- Nit: Remove unused ErrEmptyContent and CompressAge
- Nit: Replace insertion sort with sort.Slice
- Nit: Merge sessionStoreFromFlags/sessionStoreFromConfig into newSessionStore
- Nit: Truncate at word boundary in LevelSentence
- Nit: Remove trailing blank line

Co-authored-by: Ona <no-reply@ona.com>
@Siddhant-K-code
Copy link
Owner Author

All 10 review findings addressed in 6302608:

Finding Fix
P2 isDuplicate full scan Added TODO comment
P2 Create timestamp inconsistency Reuse time.Time instead of calling time.Now() twice
P2 enforceBudget single-pass Loop until within budget or no progress
P3 DedupSavings misleading Renamed to CompressionSavings
Nit ErrEmptyContent unused Removed
Nit CompressAge unused Removed from Config
Nit Insertion sort Replaced with sort.Slice
Nit Duplicate store constructors Merged into newSessionStore
Nit Mid-word truncation Truncate at word boundary
Nit Trailing blank line Removed

All 13 tests pass, 0 lint issues.

Siddhant-K-code and others added 2 commits February 24, 2026 13:41
- CHANGELOG: add [Unreleased] section with session management
- FAQ: add memory, session, and MCP tools Q&As; update roadmap status
- examples/session_api.sh: full API usage example
- mcp/README: add memory + session tool docs, config examples,
  integration patterns (session tracking, cross-session memory)
- mcp/claude_desktop_config_full.example.json: config with all features

Co-authored-by: Ona <no-reply@ona.com>
- Add memory: and session: sections to distill.yaml config example
- Update Quick Start MCP to show --memory --session flags
- Add Memory and Session to Pipeline Modules section

Co-authored-by: Ona <no-reply@ona.com>
@Siddhant-K-code Siddhant-K-code merged commit a4b0724 into main Feb 24, 2026
2 checks passed
@Siddhant-K-code Siddhant-K-code deleted the feat/session-management branch February 24, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Product] Session-based incremental dedup - stateful context window management

1 participant