Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
d68a3be
feat: Add config support for pgvector
dexters1 Oct 11, 2024
c62dfdd
feat: Add PGVectorAdapter
dexters1 Oct 11, 2024
268396a
feature: Checkpoint during pgvector integration development
dexters1 Oct 11, 2024
9fbf2d8
feat: Add PGVector support
dexters1 Oct 17, 2024
9b9ae6c
refactor: Remove unused env parameter
dexters1 Oct 17, 2024
aa26eab
refactor: Remove echo for database
dexters1 Oct 17, 2024
02cd240
feat: Add batch search to PGVectorAdapter
dexters1 Oct 17, 2024
58e5854
Merge branch 'main' of github.com:topoteretes/cognee into COG-170-PGv…
dexters1 Oct 18, 2024
325e6cd
refactor: Rewrite search query
dexters1 Oct 18, 2024
2cd2557
refactor: Add formatting to PGVector Adapter
dexters1 Oct 18, 2024
7f7b015
refactor: Add formatting to create_vector_engine
dexters1 Oct 18, 2024
d2772d2
refactor: Formatting change for create_vector_engine
dexters1 Oct 18, 2024
240c660
refactor: Change raw SQL queries to SQLalchemy ORM for PGVectorAdapter
dexters1 Oct 21, 2024
05e4ef3
fix: Fix pruning of postgres database
dexters1 Oct 21, 2024
f8babba
test: Add test for PGVectorAdapter
dexters1 Oct 21, 2024
9f4b8f2
test: Add github action workflow to run PGVectorAdapter integration test
dexters1 Oct 21, 2024
4c381a3
chore: Add pgvector dependency
dexters1 Oct 21, 2024
9461ba0
chore: Add psycopg2 dependency
dexters1 Oct 21, 2024
2cedcbe
refactor: Change database name in PGVectorAdapter test and workflow
dexters1 Oct 21, 2024
71c1374
refactor: Move serialize_datetime function
dexters1 Oct 22, 2024
4a73505
refactor: Move create_db_and_tables module from vectors to pgvector
dexters1 Oct 22, 2024
a358168
refactor: Add setting of database configs through dictionary
dexters1 Oct 22, 2024
7b2022e
refactor: Move psycopg2 to an optional dependency
dexters1 Oct 22, 2024
88ded6e
Merge branch 'main' of github.com:topoteretes/cognee into COG-170-PGv…
dexters1 Oct 22, 2024
8002db7
chore: Add installing of depdendencies along with postgres group
dexters1 Oct 22, 2024
dbc86e2
chore: Add pgvector back to mandatory dependencies
dexters1 Oct 22, 2024
c7ed46d
fix: Change to new syntax for vector_engine_provider
dexters1 Oct 22, 2024
6b9a142
refactor: Fix spacing, remove unused config methods
dexters1 Oct 22, 2024
c78627f
chore: Remove postgres group from pyproject.toml install postgres dep…
dexters1 Oct 22, 2024
d30c337
refactor: Use SQLAlchemyAdapter create_database
dexters1 Oct 22, 2024
0e1533a
chore: Update how postgres dependencies are installed in integration …
dexters1 Oct 22, 2024
195929e
refactor: Fix typo
dexters1 Oct 22, 2024
dc46304
fix: Add missing await statement to LanceDBAdapter and PGVectorAdapter
dexters1 Oct 22, 2024
0c6f019
refactor: Remove broad exception handling from PGVectorAdapter
dexters1 Oct 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
feat: Add config support for pgvector
Added config support for using pgvector

Feature #COG-170
  • Loading branch information
dexters1 committed Oct 11, 2024
commit d68a3be32e5a4c720df45e07098496f63cd53e7b
6 changes: 3 additions & 3 deletions .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,19 @@ GRAPHISTRY_PASSWORD=

SENTRY_REPORTING_URL=

GRAPH_DATABASE_PROVIDER="neo4j" # or "networkx"
GRAPH_DATABASE_PROVIDER="neo4j" # "neo4j" or "networkx"
# Not needed if using networkx
GRAPH_DATABASE_URL=
GRAPH_DATABASE_USERNAME=
GRAPH_DATABASE_PASSWORD=

VECTOR_ENGINE_PROVIDER="qdrant" # or "weaviate" or "lancedb"
VECTOR_ENGINE_PROVIDER="qdrant" # or "qdrant", "pgvector", "weaviate" or "lancedb"
# Not needed if using "lancedb"
VECTOR_DB_URL=
VECTOR_DB_KEY=

# Database provider
DB_PROVIDER="sqlite" # or "postgres"
DB_PROVIDER="sqlite" # "sqlite" or "postgres"

# Database name
DB_NAME=cognee_db
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,11 +190,11 @@ Cognee supports a variety of tools and services for different operations:

- **Local Setup**: By default, LanceDB runs locally with NetworkX and OpenAI.

- **Vector Stores**: Cognee supports Qdrant and Weaviate for vector storage.
- **Vector Stores**: Cognee supports LanceDB, Qdrant, PGVector and Weaviate for vector storage.

- **Language Models (LLMs)**: You can use either Anyscale or Ollama as your LLM provider.

- **Graph Stores**: In addition to LanceDB, Neo4j is also supported for graph storage.
- **Graph Stores**: In addition to NetworkX, Neo4j is also supported for graph storage.

- **User management**: Create individual user graphs and manage permissions

Expand Down
2 changes: 1 addition & 1 deletion cognee/api/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ class LLMConfig(BaseModel):
apiKey: str

class VectorDBConfig(BaseModel):
provider: Union[Literal["lancedb"], Literal["qdrant"], Literal["weaviate"]]
provider: Union[Literal["lancedb"], Literal["qdrant"], Literal["weaviate"], Literal["pgvector"]]
url: str
apiKey: str

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ def create_vector_engine(config: VectorConfig, embedding_engine):
api_key = config["vector_db_key"],
embedding_engine = embedding_engine
)
elif config["vector_db_provider"] == "pgvector":
pass
else:
from .lancedb.LanceDBAdapter import LanceDBAdapter

Expand Down
3 changes: 3 additions & 0 deletions cognee/modules/settings/get_settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ def get_settings():
}, {
"value": "lancedb",
"label": "LanceDB",
}, {
"value": "pgvector",
"label": "PGVector",
}]

vector_config = get_vectordb_config()
Expand Down
2 changes: 1 addition & 1 deletion cognee/modules/settings/save_vector_db_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
class VectorDBConfig(BaseModel):
url: str
apiKey: str
provider: Union[Literal["lancedb"], Literal["qdrant"], Literal["weaviate"]]
provider: Union[Literal["lancedb"], Literal["qdrant"], Literal["weaviate"], Literal["pgvector"]]

async def save_vector_db_config(vector_db_config: VectorDBConfig):
vector_config = get_vectordb_config()
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ services:
- cognee-network

postgres:
image: postgres:latest
image: pgvector/pgvector:pg17
container_name: postgres
environment:
POSTGRES_USER: cognee
Expand Down