Skip to content

feat: Add support for Ingestion and Retrieval of Knowledge Bases #9088

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
edwinjosechittilappilly merged 187 commits into from
Aug 13, 2025
Merged

feat: Add support for Ingestion and Retrieval of Knowledge Bases #9088

Changes from 1 commit
Commits
Show all changes
187 commits
Select commit Hold shift + click to select a range
9be2d30
refactor: Standardize import statements and improve code readability …
deon-sanchez Jul 16, 2025
941bc81
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 16, 2025
4df3225
feat: Introduce new Files and Knowledge Bases page with tabbed interface
deon-sanchez Jul 17, 2025
c32d451
Create knowledgebase_utils.py
erichare Jul 17, 2025
75409c1
Push initial ingest component
erichare Jul 17, 2025
1c9a2aa
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
de3ade8
Create initial KB Ingestion component
erichare Jul 17, 2025
5ea7224
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
c22e59b
Fix ruff check on utility functions
erichare Jul 17, 2025
ccd0f79
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
b9f9e01
Some quick fixes
erichare Jul 17, 2025
c00f486
Update kb_ingest.py
erichare Jul 17, 2025
4ada462
Merge branch 'main' into feat-knowledge-bases
erichare Jul 17, 2025
cabf676
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
350461e
First version of retrieval component
erichare Jul 17, 2025
b0b62a3
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
7dad9d6
Update icon
erichare Jul 17, 2025
6a0f187
Update kb_retrieval.py
erichare Jul 17, 2025
8da44b2
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
0d25004
Merge branch 'lfoss-1813' into feat-knowledge-bases
deon-sanchez Jul 17, 2025
1247bed
Add knowledge bases feature with API integration and UI components
deon-sanchez Jul 17, 2025
66da30e
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
5951200
[autofix.ci] apply automated fixes (attempt 2/3)
autofix-ci[bot] Jul 17, 2025
d9c9cb9
Refactor imports and update routing paths for assets and main page co…
deon-sanchez Jul 17, 2025
75189e8
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 17, 2025
81367fb
Merge branch 'main' into feat-knowledge-bases
edwinjosechittilappilly Jul 17, 2025
d7940af
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
db49a96
Add CreateKnowledgeBaseButton, KnowledgeBaseEmptyState, and Knowledge…
deon-sanchez Jul 17, 2025
5503c78
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 17, 2025
845f0a7
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
ef94bcf
PoV: Add Parquet data retrieval to KBRetrievalComponent (#9097)
edwinjosechittilappilly Jul 17, 2025
6d82934
Fix some ruff issues
erichare Jul 17, 2025
79e3425
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2025
b43333f
Merge branch 'main' into feat-knowledge-bases
erichare Jul 17, 2025
109363c
Merge branch 'main' of https://github.com/langflow-ai/langflow into f…
deon-sanchez Jul 18, 2025
49c0db0
Merge branch 'main' into feat-knowledge-bases
erichare Jul 18, 2025
d7e5c33
Merge branch 'main' of https://github.com/langflow-ai/langflow into f…
deon-sanchez Jul 21, 2025
bd1d91f
feat: refactor file management and knowledge base components
deon-sanchez Jul 21, 2025
d5d2a5e
feat: implement delete confirmation modal for knowledge base deletion
deon-sanchez Jul 21, 2025
63dd4c9
feat: enhance knowledge base metadata with embedding model detection
deon-sanchez Jul 21, 2025
14b87c4
refactor: clean up tooltip and value getter comments in knowledge bas…
deon-sanchez Jul 21, 2025
8daab25
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 21, 2025
8268740
refactor: simplify KnowledgeBaseSelectionOverlay component
deon-sanchez Jul 21, 2025
c3d286b
feat: implement bulk and single deletion for knowledge bases
deon-sanchez Jul 21, 2025
388e98a
Merge branch 'main' into feat-knowledge-bases
erichare Jul 21, 2025
2c78dd0
Initial support for vector search
erichare Jul 21, 2025
2adcc77
feat: add KnowledgeBaseDrawer component for enhanced knowledge base d…
deon-sanchez Jul 21, 2025
c4bf9bf
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 21, 2025
3b88885
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 21, 2025
6b3a349
[autofix.ci] apply automated fixes (attempt 2/3)
autofix-ci[bot] Jul 21, 2025
4116cae
Fix ruff checks
erichare Jul 21, 2025
810c717
Update knowledge_bases.py
erichare Jul 21, 2025
c883ae1
feat: update mock data and enhance drawer functionality in KnowledgeB…
deon-sanchez Jul 21, 2025
24e7715
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 21, 2025
dd8855b
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 21, 2025
2c02cc0
Append scores column to rows
erichare Jul 21, 2025
0d36985
Merge branch 'main' into feat-knowledge-bases
erichare Jul 21, 2025
77bc57f
refactor: improve knowledge base deletion and UI components
deon-sanchez Jul 21, 2025
98766fc
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 21, 2025
9c7fb6a
refactor: standardize import statements and improve code readability …
deon-sanchez Jul 21, 2025
63fb9b9
feat: Add encryption for API keys in KB ingest and retrieval (#9129)
edwinjosechittilappilly Jul 22, 2025
049e39f
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 22, 2025
8ec1341
Merge branch 'main' into feat-knowledge-bases
erichare Jul 22, 2025
8adcd12
Merge branch 'main' into feat-knowledge-bases
erichare Jul 22, 2025
0ca5a67
Merge branch 'main' into feat-knowledge-bases
erichare Jul 22, 2025
f251c73
Merge branch 'main' into feat-knowledge-bases
erichare Jul 22, 2025
1def7f6
Fix import of auth utils
erichare Jul 22, 2025
9146f7e
Allow appending to existing knowledge base
erichare Jul 22, 2025
06211a6
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 22, 2025
d3a7120
Update kb_ingest.py
erichare Jul 22, 2025
67d5ae5
Update kb_ingest.py
erichare Jul 22, 2025
bc10c6e
Merge branch 'main' of https://github.com/langflow-ai/langflow into f…
deon-sanchez Jul 23, 2025
bad02f3
feat: enhance table component with editable Vectorize column function…
deon-sanchez Jul 23, 2025
fe36a36
New ingestion creation dialog
erichare Jul 23, 2025
d139d5b
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 23, 2025
4cb23b7
Clean up the creation process for KB
erichare Jul 23, 2025
6ece64b
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 23, 2025
69aed9a
Clean up names and descriptions
erichare Jul 23, 2025
bd4ae10
Update kb_retrieval.py
erichare Jul 23, 2025
1469ecf
Merge branch 'main' into feat-knowledge-bases
erichare Jul 23, 2025
a654109
chroma retrieval
erichare Jul 23, 2025
5d0916d
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 23, 2025
a8ea48e
Further KB cleanup
erichare Jul 23, 2025
4440e08
refactor: update KB ingestion component and enhance NodeDialog functi…
deon-sanchez Jul 23, 2025
93b5149
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 23, 2025
16555cd
Hash the text as id
erichare Jul 23, 2025
1e66ae2
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 23, 2025
1c4c209
Update kb_retrieval.py
erichare Jul 23, 2025
86c8e55
Merge branch 'main' into feat-knowledge-bases
erichare Jul 23, 2025
4b7de6d
Merge branch 'main' into feat-knowledge-bases
erichare Jul 23, 2025
4f49445
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 23, 2025
0a43c94
Make sure to write out the source parquet
erichare Jul 23, 2025
72d88c0
Remove unneeded old code
erichare Jul 23, 2025
2048c42
Merge branch 'main' into feat-knowledge-bases
erichare Jul 23, 2025
cf7d64d
Add ability to block duplicate ingestion chunks
erichare Jul 24, 2025
36fac5a
Merge branch 'main' into feat-knowledge-bases
erichare Jul 24, 2025
9341c41
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 24, 2025
45f14f7
[autofix.ci] apply automated fixes (attempt 2/3)
autofix-ci[bot] Jul 24, 2025
e6ab6cb
Rename retrieval component
erichare Jul 24, 2025
542984b
Better refresh mechanism for the retrieve
erichare Jul 24, 2025
4864640
Clean up some unused functionality
erichare Jul 24, 2025
3aeb0c5
Merge branch 'main' into feat-knowledge-bases
erichare Jul 24, 2025
8ab4368
Update kb_ingest.py
erichare Jul 24, 2025
80e223e
Fix dropdown component logic to include checks for refresh button and…
deon-sanchez Jul 24, 2025
9058976
Test the API key before saving knowledge
erichare Jul 24, 2025
03a8c2e
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 24, 2025
96ee3f4
Allow storing updated api keys if provided at ingest time
erichare Jul 24, 2025
896bf61
Merge branch 'main' into feat-knowledge-bases
erichare Jul 24, 2025
d3fc9e8
Add Knowledge Bases component and enhance Knowledge Base Empty State
deon-sanchez Jul 24, 2025
5718eb3
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 24, 2025
b33a3c9
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 24, 2025
602f39d
[autofix.ci] apply automated fixes (attempt 2/3)
autofix-ci[bot] Jul 24, 2025
502436d
Update Knowledge Bases.json
erichare Jul 24, 2025
00da454
Update Knowledge Bases configuration and enhance UI components
deon-sanchez Jul 24, 2025
76f0035
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 24, 2025
c9fbbdd
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 24, 2025
5dcf0b8
Implement feature flag for Knowledge Bases functionality
deon-sanchez Jul 24, 2025
14909d9
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 24, 2025
41ba6ec
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 24, 2025
3662d50
[autofix.ci] apply automated fixes (attempt 2/3)
autofix-ci[bot] Jul 24, 2025
20d4382
Refactor Knowledge Bases feature flag implementation
deon-sanchez Jul 24, 2025
de4edf7
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 24, 2025
1e7ffce
revert
deon-sanchez Jul 24, 2025
6e7b061
Merge branch 'main' of https://github.com/langflow-ai/langflow into f…
deon-sanchez Jul 24, 2025
8277cb6
Merge branch 'main' into feat-knowledge-bases
erichare Jul 25, 2025
ed009cd
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 25, 2025
8700133
Remove Knowledge Bases JSON configuration and clean up KnowledgeBases…
deon-sanchez Jul 25, 2025
aaaae03
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 25, 2025
02d4874
Enhance routing structure by adding admin and login routes with prote…
deon-sanchez Jul 25, 2025
ae0d378
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 25, 2025
43ef981
added template back
deon-sanchez Jul 25, 2025
9c21594
Use chroma for stats computation
erichare Jul 25, 2025
71eaf96
Fix ruff issue
erichare Jul 25, 2025
6ce2414
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 25, 2025
86334cf
Update Knowledge Bases.json
erichare Jul 25, 2025
d3d176f
Update Knowledge Bases.json
erichare Jul 25, 2025
dfcfe7b
Rename to just knowledge
erichare Jul 25, 2025
e072f0d
Merge branch 'main' into feat-knowledge-bases
erichare Jul 25, 2025
6645b25
Merge branch 'main' of https://github.com/langflow-ai/langflow into f…
deon-sanchez Jul 28, 2025
3efe3be
feat: enhance Jest configuration and add new tests for Knowledge Base…
deon-sanchez Jul 28, 2025
2dc9c55
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 28, 2025
8fa29e5
refactor: reorganize imports and clean up console log in Dropdown com…
deon-sanchez Jul 29, 2025
aacf468
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 29, 2025
f61689a
[autofix.ci] apply automated fixes (attempt 2/3)
autofix-ci[bot] Jul 29, 2025
6416d51
feat: add success callback for knowledge base creation in NodeDialog …
deon-sanchez Jul 29, 2025
b780edd
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Jul 29, 2025
d20c2c6
refactor: update table component to handle single-toggle columns
deon-sanchez Jul 29, 2025
8c40cf7
Merge branch 'main' of https://github.com/langflow-ai/langflow into f…
deon-sanchez Jul 29, 2025
5536a3d
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 29, 2025
2a4dba8
feat: Add unit tests for KBIngestionComponent (#9246)
edwinjosechittilappilly Jul 30, 2025
de843c8
Merge branch 'main' of https://github.com/langflow-ai/langflow into f…
deon-sanchez Jul 31, 2025
fb45847
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 31, 2025
c053983
fix: remove unnecessary drawer open state change in KnowledgePage
deon-sanchez Jul 31, 2025
3f24571
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 31, 2025
62a1023
[autofix.ci] apply automated fixes (attempt 2/3)
autofix-ci[bot] Jul 31, 2025
e80a68e
Remove kb_info output from KBIngestionComponent (#9275)
edwinjosechittilappilly Jul 31, 2025
663b819
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 31, 2025
414a7b9
Update Knowledge Bases.json
edwinjosechittilappilly Aug 1, 2025
6498a83
Use settings service for knowledge base directory
edwinjosechittilappilly Aug 1, 2025
60c6da5
Merge branch 'main' of https://github.com/langflow-ai/langflow into f…
deon-sanchez Aug 1, 2025
4516cca
Fix knowledge bases mypy issue
erichare Aug 1, 2025
9121c1d
test: Update file page tests for consistency and clarity
deon-sanchez Aug 1, 2025
9a9717a
test: Update expected title in file upload component test for accuracy
deon-sanchez Aug 1, 2025
1871c1d
Merge branch 'feat-knowledge-bases' of https://github.com/langflow-ai…
deon-sanchez Aug 1, 2025
d8f3d0f
[autofix.ci] apply automated fixes
autofix-ci[bot] Aug 1, 2025
7565e95
Fix tests on backend
erichare Aug 1, 2025
b62a7eb
Merge branch 'main' into feat-knowledge-bases
erichare Aug 1, 2025
706040f
Update kb_ingest.py
erichare Aug 1, 2025
4072499
[autofix.ci] apply automated fixes
autofix-ci[bot] Aug 1, 2025
4ace8d8
Merge branch 'main' into feat-knowledge-bases
erichare Aug 11, 2025
baeb113
Merge branch 'main' into feat-knowledge-bases
erichare Aug 11, 2025
11d7b17
Merge branch 'main' into feat-knowledge-bases
erichare Aug 11, 2025
dda21d7
Merge branch 'main' into feat-knowledge-bases
erichare Aug 11, 2025
fd1b2ae
Merge branch 'main' into feat-knowledge-bases
edwinjosechittilappilly Aug 12, 2025
b6b60fa
Merge branch 'main' into feat-knowledge-bases
erichare Aug 12, 2025
600e0e9
Merge branch 'main' into feat-knowledge-bases
erichare Aug 12, 2025
933233a
Merge branch 'main' into feat-knowledge-bases
erichare Aug 12, 2025
d88b479
Merge branch 'main' into feat-knowledge-bases
erichare Aug 12, 2025
fb5294c
Merge branch 'main' into feat-knowledge-bases
erichare Aug 12, 2025
ef664d8
Merge branch 'main' into feat-knowledge-bases
erichare Aug 12, 2025
9c90eeb
Merge branch 'main' into feat-knowledge-bases
erichare Aug 12, 2025
a37c8a8
Switch to two templates for KB
erichare Aug 12, 2025
0600f8c
Merge branch 'main' into feat-knowledge-bases
erichare Aug 13, 2025
f831d9b
Update names and descs
erichare Aug 13, 2025
71ef5f5
[autofix.ci] apply automated fixes
autofix-ci[bot] Aug 13, 2025
58044d0
Rename templates
erichare Aug 13, 2025
4d49c95
[autofix.ci] apply automated fixes
autofix-ci[bot] Aug 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into feat-knowledge-bases
  • Loading branch information
@erichare
erichare authored Aug 11, 2025
commit dda21d7544a2f4a23dc3813e8462130c7b429d26

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.