Iteration node slows everything down by A LOT

[DavideDelbianco]

Self Checks

• I have read the Contributing Guide and Language Policy.
• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.12.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Create a workflow with KB search
execute the search both outside and inside an iteration node (with 1 iteration)

✔️ Expected Behavior

There should be just a little overhead caused by using the iteration node

❌ Actual Behavior

The same search takes 5 seconds when executed inside the iteration vs the 1.2 sec from outside

[dosubot]

This is a known issue that's been reported and partially addressed. The iteration node slowdown stems from worker pool management overhead where the pool stops and restarts during execution, with each worker.join(timeout=10.0) call adding significant delays.

Fixes that should be in v1.12.1:

• PR #32789 (merged March 1, 2026) reduced thread join timeouts from 10.0s to 2.0s
• PR #32815 optimized database queries for iteration operations
• PR #31663 refactored worker pool configuration

Since you're on v1.12.1, these fixes should already be included. If you're still seeing this much overhead, you could try:

1. Check your actual image/deployment - ensure you're running a build that includes the March 2026 fixes
2. Switch to sync execution (though this was reported as ineffective for the worker pool issue):

   CORE_WORKFLOW_EXECUTION_REPOSITORY=core.repositories.sqlalchemy_workflow_execution_repository.SQLAlchemyWorkflowExecutionRepository
   CORE_WORKFLOW_NODE_EXECUTION_REPOSITORY=core.repositories.sqlalchemy_workflow_node_execution_repository.SQLAlchemyWorkflowNodeExecutionRepository
   

You might want to check the related issue #30390 for context, as your symptoms match what was reported there.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[DavideDelbianco]

Also notice the whole iteration duration is longer than the single node
It also seems like even if the nodes are in parallel in the graph, they were executed one after the other (it would explain the sum of 1.2+1.2+1.2 ~4.42

I am currently using:
GRAPH_ENGINE_MIN_WORKERS: "3"
CELERY_AUTO_SCALE: "true"

[EndlessLucky]

@DavideDelbianco Can I work on this issue?

[DavideDelbianco]

I have contributed in the past, I'm not a maintainer of this project, you should ask the Dify team @crazywoola @QuantumGhost

[EndlessLucky]

@fatelei May I work on this one?

[dosubot]

Hi, @DavideDelbianco. I'm Dosu, and I'm helping the dify team manage their backlog and am marking this issue as stale.

Issue Summary

• You reported a significant slowdown using an iteration node with KB search in version 1.12.1, with execution time increasing from 1.2s to 5s.
• The slowdown is linked to worker pool management overhead, causing nodes expected to run in parallel to execute sequentially.
• Several fixes were merged before v1.12.1 aimed at reducing timeouts and optimizing queries.
• You shared your current worker settings and noted the unexpected sequential execution behavior.
• Another user expressed interest in contributing, but you recommended contacting maintainers @crazywoola and @QuantumGhost for assignment.

Next Steps

• Please confirm if this issue is still relevant with the latest version of dify by commenting here to keep the discussion open.
• If no response is received within 15 days, this issue will be automatically closed to help keep the backlog manageable.

Thank you for your understanding and contribution!