Skip to content

Cog 337 llama index support #186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
borisarzentar merged 21 commits into from
Nov 17, 2024
Merged

Cog 337 llama index support #186

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
ec5fd80
feat: Add support for LlamaIndex Document type
dexters1 Nov 6, 2024
b05a07f
docs: Add Jupyer Notebook for cognee with llama index document type
dexters1 Nov 6, 2024
93445bf
feat: Add metadata migration from LlamaIndex document type
dexters1 Nov 7, 2024
804ca26
refactor: Change llama index migration function name
dexters1 Nov 7, 2024
c151d7b
chore: Add llama index core dependency
dexters1 Nov 7, 2024
fb84aa0
Merge branch 'COG-337-llama-index-support' of github.com:topoteretes/…
dexters1 Nov 7, 2024
7d222ee
Feature: Add ingest_data_with_metadata task
dexters1 Nov 7, 2024
094e45d
docs: Add description on why specific type checking is done
dexters1 Nov 7, 2024
d5aa316
fix: Add missing parameter to function call
dexters1 Nov 7, 2024
98dc18c
refactor: Move storing of data from async to sync function
dexters1 Nov 7, 2024
2615b35
refactor: Pretend ingest_data was changes instead of having two tasks
dexters1 Nov 7, 2024
e4c39e8
refactor: Use old name for data ingestion with metadata
dexters1 Nov 7, 2024
733caa7
refactor: Return ingest_data and save_data_to_storage Tasks
dexters1 Nov 8, 2024
8c7fdc4
refactor: Return previous ingestion Tasks to add function
dexters1 Nov 8, 2024
f13146d
Merge branch 'main' of github.com:topoteretes/cognee into COG-337-lla…
dexters1 Nov 14, 2024
cc53693
fix: Remove dict and use string for search query
dexters1 Nov 14, 2024
af99e4f
refactor: Add changes request in pull request
dexters1 Nov 15, 2024
417b119
Merge branch 'main' of github.com:topoteretes/cognee into COG-337-lla…
dexters1 Nov 15, 2024
4c7a4e2
fix: Resolve issue with llama-index being mandatory
dexters1 Nov 15, 2024
c4700b1
fix: Add install of llama-index to notebook
dexters1 Nov 15, 2024
f6e8294
Merge branch 'main' into COG-337-llama-index-support
borisarzentar Nov 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 123 additions & 14 deletions poetry.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ boto3 = "^1.26.125"
botocore="^1.35.54"
gunicorn = "^20.1.0"
sqlalchemy = "2.0.35"
instructor = "1.6.3"
instructor = "^1.5.2"
Copy link
Copy Markdown
Contributor

@Vasilije1990 Vasilije1990 Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not fix the version? it is not a very reliable lib, as ti changed a few times already

networkx = "^3.2.1"
aiosqlite = "^0.20.0"
pandas = "2.0.3"
Expand All @@ -46,7 +46,7 @@ dlt = {extras = ["sqlalchemy"], version = "^1.2.0"}
aiofiles = "^23.2.1"
qdrant-client = "^1.9.0"
graphistry = "^0.33.5"
tenacity = "^9.0.0"
tenacity = "^8.4.1"
weaviate-client = "4.6.7"
scikit-learn = "^1.5.0"
pypdf = "^4.1.0"
Expand All @@ -70,6 +70,7 @@ asyncpg = "^0.29.0"
alembic = "^1.13.3"
pgvector = "^0.3.5"
psycopg2 = {version = "^2.9.10", optional = true}
llama-index-core = "^0.11.22"
Copy link
Copy Markdown
Contributor

@Vasilije1990 Vasilije1990 Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this optional please


[tool.poetry.extras]
filesystem = ["s3fs", "botocore"]
Expand Down