feat(api/repo): Allow to config repository implementation #21458

laipz8200 · 2025-06-25T02:03:17Z

Important

Make sure you have read our contribution guidelines
Ensure there is an associated issue and you have been assigned to it
Use the correct syntax to link this PR: Fixes #<issue number>.

Summary

related #20147

Screenshots

Before	After
...	...

Checklist

This change requires a documentation update, included: Dify Document
I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
I've updated the documentation accordingly.
I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

laipz8200 · 2025-06-27T05:47:22Z

Update: Nearly finished, with only some code related to WorkflowRun remaining.

laipz8200 · 2025-07-07T10:46:47Z

Hi @QuantumGhost. Do you have some time to take a look at this?

api/core/app/apps/common/workflow_response_converter.py

api/core/memory/token_buffer_memory.py

api/core/prompt/utils/get_thread_messages_length.py

api/core/repositories/factory.py

api/repositories/api_workflow_node_execution_repository.py

api/repositories/factory.py

QuantumGhost

LGTM

api/repositories/sqlalchemy_api_workflow_node_execution_repository.py

api/repositories/sqlalchemy_api_workflow_run_repository.py

api/services/workflow_draft_variable_service.py

api/tasks/remove_app_and_related_data_task.py

QuantumGhost · 2025-07-10T03:11:04Z

By the way, now that PostgreSQL is available in the CI pipeline, you may write integration tests for the repository implementations to ensure they behave as expected.

Signed-off-by: -LAN- <[email protected]>

…tion operations out of core. Signed-off-by: -LAN- <[email protected]>

Signed-off-by: -LAN- <[email protected]>

…on_repository Signed-off-by: -LAN- <[email protected]>

Signed-off-by: -LAN- <[email protected]>

…isolation Signed-off-by: -LAN- <[email protected]>

…run query for clarity Signed-off-by: -LAN- <[email protected]>

since we can utilize `Session.get_bind` to retrieve the correspond `Engine` for a session, there's no need to manually inject the `sessionmaker` for `WorkflowDraftVariableService`.

… to prevent session issues Signed-off-by: -LAN- <[email protected]>

…roves pagination logic for workflow runs Signed-off-by: -LAN- <[email protected]>

… note on tenant isolation responsibility Signed-off-by: -LAN- <[email protected]>

…ry.py): Add tenant isolation warning in method docstring Signed-off-by: -LAN- <[email protected]>

QuantumGhost

LGTM

luosenxinxi · 2025-07-14T12:24:29Z

Internal Server Error

* main: (533 commits) chore: bump nextjs to 15.3 (langgenius#22262) Provides a set of Kubernetes manifests supporting version 1.6.0 (langgenius#22287) improve opik workflow_trace span name to node name (langgenius#22356) fix: aliyun trace title&description (langgenius#22347) Update README.md (langgenius#22351) fix: close session before doing long latency operation (langgenius#22306) feat(api/repo): Allow to config repository implementation (langgenius#21458) Fix: Remove height and overflow style settings (langgenius#22327) Chore: Replace lodash/noop with lodash-es/noop (langgenius#22331) fix: error parsing object type parameters for code node (langgenius#22230) fix: prevent trigger form submit action when press 'enter' (langgenius#22313) Update README.md (langgenius#22305) test: add comprehensive unit tests for login decorator (langgenius#22294) fixing Enum part in backend and making it same as front end (langgenius#22296) test(web): add password regexp test case (langgenius#22308) fix(ui): prevent var icon hidden when only one var in list of start node (langgenius#22290) fix: tool's model selector and app selector not work (langgenius#22291) chore: bump ruff to 0.12.x (langgenius#22259) minor fix: wrong and (langgenius#22242) Add unit test for account service (langgenius#22278) ... # Conflicts: # web/app/(commonLayout)/apps/AppCard.tsx # web/app/(commonLayout)/apps/Apps.tsx # web/app/(commonLayout)/apps/page.tsx # web/app/(commonLayout)/datasets/Container.tsx # web/app/components/base/logo/dify-logo.tsx # web/app/components/base/markdown.tsx # web/app/components/header/account-dropdown/workplace-selector/index.tsx # web/app/components/header/explore-nav/index.tsx # web/app/components/header/index.tsx # web/app/components/header/nav/index.tsx # web/app/components/header/plugins-nav/index.tsx # web/app/components/header/tools-nav/index.tsx # web/app/components/tools/provider-list.tsx # web/app/components/workflow/nodes/_base/panel.tsx # web/app/reset-password/layout.tsx # web/app/signin/_header.tsx # web/app/signin/layout.tsx # web/app/signin/normalForm.tsx # web/i18n/zh-Hans/common.ts # web/i18n/zh-Hans/explore.ts # web/package.json # web/pnpm-lock.yaml # web/themes/dark.css # web/themes/light.css

…#21458) Signed-off-by: -LAN- <[email protected]> Co-authored-by: QuantumGhost <[email protected]> (cherry picked from commit 6eb155a)

…#21458) Signed-off-by: -LAN- <[email protected]> Co-authored-by: QuantumGhost <[email protected]>

laipz8200 force-pushed the feat/allow-to-config-repo-impl branch from e6ed5bf to 1fadcf5 Compare June 27, 2025 01:20

laipz8200 self-assigned this Jun 27, 2025

laipz8200 marked this pull request as ready for review June 27, 2025 05:43

laipz8200 marked this pull request as draft June 27, 2025 05:43

laipz8200 mentioned this pull request Jun 27, 2025

Proposal: Add in-memory repository to improve workflow api perf #20147

Closed

5 tasks

dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Jun 27, 2025

laipz8200 force-pushed the feat/allow-to-config-repo-impl branch from 2c73dd2 to edc0823 Compare July 1, 2025 05:49

laipz8200 mentioned this pull request Jul 6, 2025

feat: Add an asynchronous repository to improve workflow performance #20050

Merged

5 tasks

QuantumGhost self-requested a review July 7, 2025 03:56

laipz8200 force-pushed the feat/allow-to-config-repo-impl branch from edc0823 to 6a548cf Compare July 7, 2025 07:29

laipz8200 marked this pull request as ready for review July 7, 2025 10:45

dosubot bot added 💪 enhancement New feature or request 📚 documentation Improvements or additions to documentation labels Jul 7, 2025

laipz8200 mentioned this pull request Jul 8, 2025

[Tracker] 1.6.0 release blockers #22035

Closed

5 tasks

laipz8200 force-pushed the feat/allow-to-config-repo-impl branch from 6a548cf to 53cb15c Compare July 9, 2025 10:56

QuantumGhost reviewed Jul 9, 2025

View reviewed changes

QuantumGhost previously approved these changes Jul 10, 2025

View reviewed changes

dosubot bot added the lgtm This PR has been approved by a maintainer label Jul 10, 2025

laipz8200 dismissed QuantumGhost’s stale review via 539a609 July 10, 2025 06:30

laipz8200 added 8 commits July 14, 2025 14:09

feat(api/repo): Allow to config repository implementation

733386b

Signed-off-by: -LAN- <[email protected]>

feat: Create a DifyAPIRepositoryFactory to handle workflow node execu…

b2b4049

…tion operations out of core. Signed-off-by: -LAN- <[email protected]>

feat: Create a APIWorkflowRunRepository

b450acf

Signed-off-by: -LAN- <[email protected]>

test(workflow_service): Fix test

6a11105

Signed-off-by: -LAN- <[email protected]>

chore: add new configs to env example

641cabc

Signed-off-by: -LAN- <[email protected]>

test(test_workflow_draft_variable_service): Fix tests

8e71a9a

Signed-off-by: -LAN- <[email protected]>

feat: Integrates execution repository inheritance

6076240

Signed-off-by: -LAN- <[email protected]>

refactor(token_buffer_memory): Convert ValueError to AssertionError

63bbcaf

Signed-off-by: -LAN- <[email protected]>

laipz8200 and others added 7 commits July 14, 2025 14:09

refactor(workflow_response_converter): Remove unused workflow_executi…

cf2b5c7

…on_repository Signed-off-by: -LAN- <[email protected]>

refactor(factory): Move import statement to the top

f585431

Signed-off-by: -LAN- <[email protected]>

docs(api_workflow_node_execution_repository): Add notice about datat …

a0cfdb4

…isolation Signed-off-by: -LAN- <[email protected]>

refactor(api_workflow_node_execution_repository): Refactors workflow …

40a7089

…run query for clarity Signed-off-by: -LAN- <[email protected]>

refactor(api): Simplify the constructor for WorkflowDraftVariableService

00462a0

since we can utilize `Session.get_bind` to retrieve the correspond `Engine` for a session, there's no need to manually inject the `sessionmaker` for `WorkflowDraftVariableService`.

chore(api): improve type hints for DifyCoreRepositoryFactory

115c6e2

test(api): fix tests broken by refactor

00c50ed

laipz8200 force-pushed the feat/allow-to-config-repo-impl branch from b74360d to 00c50ed Compare July 14, 2025 06:09

laipz8200 added 4 commits July 14, 2025 14:14

refactor(remove_app_and_related_data_task): Remove 'expire_on_commit'…

7a47b86

… to prevent session issues Signed-off-by: -LAN- <[email protected]>

feat(api/repositories/sqlalchemy_api_workflow_run_repository.py): Imp…

42a195e

…roves pagination logic for workflow runs Signed-off-by: -LAN- <[email protected]>

docs(api/repositories/api_workflow_node_execution_repository.py): Add…

0d1b27f

… note on tenant isolation responsibility Signed-off-by: -LAN- <[email protected]>

docs(api/repositories/sqlalchemy_api_workflow_node_execution_reposito…

7cabcea

…ry.py): Add tenant isolation warning in method docstring Signed-off-by: -LAN- <[email protected]>

QuantumGhost approved these changes Jul 14, 2025

View reviewed changes

laipz8200 merged commit 6eb155a into main Jul 14, 2025
13 checks passed

laipz8200 deleted the feat/allow-to-config-repo-impl branch July 14, 2025 06:54

Nov1c444 mentioned this pull request Jul 23, 2025

chore(version): bump to 1.7.0 #22830

Merged

tutkun pushed a commit to tutkun/dify that referenced this pull request Aug 15, 2025

feat(api/repo): Allow to config repository implementation (langgenius…

dbd2f2f

…#21458) Signed-off-by: -LAN- <[email protected]> Co-authored-by: QuantumGhost <[email protected]>

tutkun pushed a commit to tutkun/dify that referenced this pull request Aug 16, 2025

feat(api/repo): Allow to config repository implementation (langgenius…

3c44241

…#21458) Signed-off-by: -LAN- <[email protected]> Co-authored-by: QuantumGhost <[email protected]>

feat(api/repo): Allow to config repository implementation #21458

feat(api/repo): Allow to config repository implementation #21458

Uh oh!

Conversation

laipz8200 commented Jun 25, 2025

Summary

Screenshots

Checklist

Uh oh!

laipz8200 commented Jun 27, 2025

Uh oh!

laipz8200 commented Jul 7, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

QuantumGhost left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

QuantumGhost commented Jul 10, 2025

Uh oh!

QuantumGhost left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

luosenxinxi commented Jul 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants