feat: Graph Execution Debugging and Event System #10545
Conversation
- ExecutionTrace: Records execution with state deltas and loop tracking - ExecutionTracer: Captures run_manager state evolution at each step - GraphTrace: Interactive API for Jupyter debugging - GraphRecording: Time-travel debugging with timeline navigation - ComponentSnapshot: Before/after state capture for delta computation Features: - State delta tracking (shows only what changed) - Loop-specific iteration tracking - Works with Graph objects or JSON files - Before/after state capture for accurate deltas - Pickle-safe error storage
- test_execution_path_equivalence.py: Core tracing framework tests - test_recorder_debugging.py: 10 tests for timeline, navigation, save/load - test_tracer_demo.py: Demonstrates state deltas and loop tracking - test_all_starter_projects_equivalence.py: Framework for validating starter projects All tests passing: - 10/10 recorder tests - 3/3 execution path validation tests - Validates both execution paths run same components
- debug_graph_marimo.py: Reactive marimo notebook with timeline navigation - debug_programmatic_graph.ipynb: Jupyter examples for Graph objects Features: - Marimo: Slider navigation, state deltas, loop analysis - Jupyter: Build graphs from code, record without files - Examples show proper loop feedback pattern
Add comprehensive event system for tracking all graph state changes: - GraphMutationEvent: Serializable events with before/after state - Observer pattern: Register callbacks for all mutations - Before/after events: Complete state tracking per requirement - Zero overhead when no observers registered Async conversions: - Queue operations (extend_run_queue, get_next_in_queue) - Dependency management (add_dynamic_dependency) - Vertex state (mark_vertex, mark_branch, reset methods) - run_manager methods (remove_from_predecessors, etc) Hybrid compatibility: - Sync wrappers for backward compatibility - Async preferred, sync fallback available - Components use sync wrappers where needed Event capture working: - 210 events captured in LoopTest (105 before + 105 after) - Queue mutations: 136 events - Vertex state changes: 74 events Architecture: - All mutations go through centralized methods - Impossible to desync run_predecessors and run_map - Events enable advanced debugging and replay - Observer errors logged and collected
- Deleted unused debug utilities including `recorder.py`, `trace_data.py`, `tracer.py`, and associated example notebooks. - Cleaned up the `__init__.py` file in the debug directory. - This cleanup is part of the effort to streamline the debugging infrastructure and remove legacy code.
EventRecorder uses pure observer pattern instead of wrapping: - Observes GraphMutationEvent from centralized system - No build_vertex wrapping needed - Cleaner architecture and separation of concerns Features: - EventBasedRecording: Built from mutation events - Queue evolution tracking - Dependency change analysis - Event filtering and querying Tests: - 3 mutation event tests passing - 2 event recorder tests passing - 15 total debugger tests passing Benefits: - Simpler implementation (no wrapper complexity) - Events already contain before/after state - Extensible (any observer can subscribe) - Zero coupling to graph internals
Replace outdated programmatic example with event-focused notebook: - Shows dependency_added events with complete structure - Demonstrates queue operations - Shows vertex state transitions - Prints complete event JSON - All cells tested and produce actual output Focus on important events: - run_predecessors changes - run_map updates - Vertex state transitions - Complete graph snapshots All examples call APIs directly to trigger events.
Updated the event debugging notebook to improve clarity and functionality: - Reformatted markdown cells for better readability. - Added execution counts and outputs for all code cells. - Included detailed event outputs for dependency changes, queue operations, and vertex state transitions. - Ensured all examples produce actual output and demonstrate key event types effectively.
Added exclusion for Jupyter notebook files in the ruff configuration to streamline linting processes and avoid unnecessary warnings for debug examples.
Updated the codebase to use the new get_run_queue method for accessing the run queue, enhancing encapsulation and maintainability. Additionally, added type checking for GraphMutationEvent in unit tests to improve code clarity.
Modified the ruff configuration to streamline linting by refining the exclusion list for Jupyter notebook files. Additionally, added a new empty __init__.py file in the debug examples directory to facilitate package structure.
Marimo notebook had too many reactivity issues and wasn't working reliably. Focus on Jupyter notebook which is tested and working.
Deleted several outdated test files and debugging utilities, including test_all_starter_projects_equivalence.py, test_execution_path_equivalence.py, test_recorder_debugging.py, test_tracer_demo.py, and related modules. This cleanup aims to streamline the codebase and eliminate legacy components that are no longer in use.
Implemented methods to save and load event recordings using pickle. Added functionality to display event timelines and filter events by component, enhancing the usability of the EventBasedRecording class. Introduced constants for maximum event display limits to improve output management.
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughThis PR introduces an event-based debugging system for graph execution using an observer pattern to track mutations. Adds GraphMutationEvent, EventRecorder, and EventBasedRecording for capturing and analyzing mutations. Converts multiple graph operations to async variants, enhances loop component synchronization, improves error handling in custom components, and extends TYPE_CHECKING support in validation. Zero overhead when no observers are registered. Changes
Sequence Diagram(s)sequenceDiagram
participant App as Application
participant Graph
participant Observer as EventRecorder
participant Queue as Run Queue
Note over App,Observer: Event System Registration
App->>Graph: register_observer(EventRecorder)
Graph->>Graph: _enable_events = True
Note over App,Observer: Mutation Emission with Event
App->>Graph: extend_run_queue(vertices)
Graph->>Graph: _emit_event(before snapshot)
Note right of Graph: event_type="queue_extended"<br/>timing="before"
Graph->>Observer: on_event(before_event)
Observer->>Observer: collect event
Graph->>Queue: append vertices
Graph->>Graph: _emit_event(after snapshot)
Note right of Graph: timing="after"<br/>updated state
Graph->>Observer: on_event(after_event)
Observer->>Observer: collect event
Note over App,Observer: No Observer Path (Zero Overhead)
App->>Graph: extend_run_queue(vertices)
Note right of Graph: _enable_events=False<br/>fast path skips<br/>event emission
Graph->>Queue: append vertices
sequenceDiagram
participant LoopComponent
participant Graph as Graph Manager
participant RunQueue
Note over LoopComponent,RunQueue: Loop Dependency Synchronization
LoopComponent->>LoopComponent: item_output()
LoopComponent->>LoopComponent: update_dependency(dependency)
Note right of LoopComponent: sync path for<br/>run_map update
LoopComponent->>Graph: add_dynamic_dependency(vertex, predecessor)
Graph->>Graph: update run_predecessors
Graph->>Graph: update run_map<br/>(bidirectional)
Graph->>RunQueue: reflect new dependency
Note over LoopComponent,RunQueue: Reset for Fresh Execution
LoopComponent->>LoopComponent: reset_loop_state()
LoopComponent->>LoopComponent: clear initialized/index<br/>clear aggregated/data
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Areas requiring extra attention:
Possibly related PRs
Suggested labels
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
bot
added
enhancement
Frontend Unit Test Coverage ReportCoverage Summary
Unit Test Results
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #10545 +/- ##
==========================================
+ Coverage 33.22% 33.34% +0.12%
==========================================
Files 1391 1393 +2
Lines 65849 66076 +227
Branches 9745 9777 +32
==========================================
+ Hits 21877 22034 +157
- Misses 42850 42910 +60
- Partials 1122 1132 +10
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
github-actions
bot
added
enhancement
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/lfx/src/lfx/graph/graph/base.py (1)
1118-1137: Root vertex never gets markedThe first call into
_mark_branchskipsmark_vertex, so the branch root keeps its prior state. This is a regression: callers expect the starting vertex to be marked along with its descendants (e.g., LOOP/conditional routing). Mark the vertex before recursing so the entire branch updates consistently.Apply this diff:
async def _mark_branch( self, vertex_id: str, state: str, visited: set | None = None, output_name: str | None = None ) -> set: - if visited is None: - visited = set() - else: - await self.mark_vertex(vertex_id, state) - if vertex_id in visited: - return visited - visited.add(vertex_id) + if visited is None: + visited = set() + if vertex_id in visited: + return visited + await self.mark_vertex(vertex_id, state) + visited.add(vertex_id)
🧹 Nitpick comments (2)
src/backend/tests/unit/graph/test_graph_mutation_events.py (1)
14-30: Consider more detailed event content assertions.While the test correctly validates before/after timing and event count, it could be strengthened by asserting on the actual event content (e.g.,
state_before,state_after,changesfields) to ensure events carry the expected data.Example enhancement:
assert len(events) == 2 assert events[0].timing == "before" assert events[1].timing == "after" +assert events[1].changes["added"] == ["v1", "v2"] +assert events[1].changes["queue_size_after"] == 2src/backend/tests/unit/graph/test_event_recorder.py (1)
25-32: Consider more specific event content assertions.The test validates event counts and types, which is good, but could be strengthened by asserting on specific event properties (vertex IDs, state changes, etc.) to ensure events contain expected data for the LoopTest scenario.
Example enhancement:
assert len(recording.events) > 100 queue_events = recording.get_events_by_type("queue_extended") assert len(queue_events) > 0 +# Validate queue event structure +assert all(hasattr(e, 'state_before') and hasattr(e, 'state_after') for e in queue_events)
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (14)
PR_DESCRIPTION.md(1 hunks)PR_DESCRIPTION_SHORT.md(1 hunks)pyproject.toml(1 hunks)src/backend/tests/unit/graph/test_event_recorder.py(1 hunks)src/backend/tests/unit/graph/test_graph_mutation_events.py(1 hunks)src/lfx/src/lfx/components/logic/loop.py(2 hunks)src/lfx/src/lfx/custom/custom_component/component.py(1 hunks)src/lfx/src/lfx/custom/custom_component/custom_component.py(2 hunks)src/lfx/src/lfx/custom/validate.py(2 hunks)src/lfx/src/lfx/debug/__init__.py(1 hunks)src/lfx/src/lfx/debug/event_recorder.py(1 hunks)src/lfx/src/lfx/debug/events.py(1 hunks)src/lfx/src/lfx/graph/graph/base.py(13 hunks)src/lfx/src/lfx/graph/graph/runnable_vertices_manager.py(2 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
{src/backend/**/*.py,tests/**/*.py,Makefile}
📄 CodeRabbit inference engine (.cursor/rules/backend_development.mdc)
{src/backend/**/*.py,tests/**/*.py,Makefile}: Run make format_backend to format Python code before linting or committing changes
Run make lint to perform linting checks on backend Python code
Files:
src/backend/tests/unit/graph/test_graph_mutation_events.pysrc/backend/tests/unit/graph/test_event_recorder.py
src/backend/tests/unit/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/backend_development.mdc)
Test component integration within flows using create_flow, build_flow, and get_build_events utilities
Files:
src/backend/tests/unit/graph/test_graph_mutation_events.pysrc/backend/tests/unit/graph/test_event_recorder.py
src/backend/tests/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/testing.mdc)
src/backend/tests/**/*.py: Unit tests for backend code must be located in the 'src/backend/tests/' directory, with component tests organized by component subdirectory under 'src/backend/tests/unit/components/'.
Test files should use the same filename as the component under test, with an appropriate test prefix or suffix (e.g., 'my_component.py' → 'test_my_component.py').
Use the 'client' fixture (an async httpx.AsyncClient) for API tests in backend Python tests, as defined in 'src/backend/tests/conftest.py'.
When writing component tests, inherit from the appropriate base class in 'src/backend/tests/base.py' (ComponentTestBase, ComponentTestBaseWithClient, or ComponentTestBaseWithoutClient) and provide the required fixtures: 'component_class', 'default_kwargs', and 'file_names_mapping'.
Each test in backend Python test files should have a clear docstring explaining its purpose, and complex setups or mocks should be well-commented.
Test both sync and async code paths in backend Python tests, using '@pytest.mark.asyncio' for async tests.
Mock external dependencies appropriately in backend Python tests to isolate unit tests from external services.
Test error handling and edge cases in backend Python tests, including using 'pytest.raises' and asserting error messages.
Validate input/output behavior and test component initialization and configuration in backend Python tests.
Use the 'no_blockbuster' pytest marker to skip the blockbuster plugin in tests when necessary.
Be aware of ContextVar propagation in async tests; test both direct event loop execution and 'asyncio.to_thread' scenarios to ensure proper context isolation.
Test error handling by mocking internal functions using monkeypatch in backend Python tests.
Test resource cleanup in backend Python tests by using fixtures that ensure proper initialization and cleanup of resources.
Test timeout and performance constraints in backend Python tests using 'asyncio.wait_for' and timing assertions.
Test Langflow's Messag...
Files:
src/backend/tests/unit/graph/test_graph_mutation_events.pysrc/backend/tests/unit/graph/test_event_recorder.py
🧠 Learnings (7)
📚 Learning: 2025-06-24T19:10:17.195Z
Learnt from: edwinjosechittilappilly
Repo: langflow-ai/langflow PR: 7273
File: src/backend/base/langflow/graph/graph/base.py:875-875
Timestamp: 2025-06-24T19:10:17.195Z
Learning: In Langflow's graph execution, the `reset_inactivated_vertices()` method should only clear the tracking set (`inactivated_vertices`) and not reactivate vertices, because IF-ELSE components use `mark_branch()` to control flow by setting vertex states to INACTIVE/ACTIVE, and automatic reactivation would break conditional logic.
Applied to files:
src/lfx/src/lfx/custom/custom_component/custom_component.pysrc/lfx/src/lfx/graph/graph/base.py
📚 Learning: 2025-07-21T14:16:14.125Z
Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-07-21T14:16:14.125Z
Learning: Applies to src/backend/tests/**/*.py : Be aware of ContextVar propagation in async tests; test both direct event loop execution and 'asyncio.to_thread' scenarios to ensure proper context isolation.
Applied to files:
src/backend/tests/unit/graph/test_graph_mutation_events.pysrc/backend/tests/unit/graph/test_event_recorder.py
📚 Learning: 2025-07-21T14:16:14.125Z
Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-07-21T14:16:14.125Z
Learning: Applies to src/backend/tests/**/*.py : Test real-time event streaming endpoints in backend Python tests by consuming and validating NDJSON event streams.
Applied to files:
src/backend/tests/unit/graph/test_graph_mutation_events.pysrc/backend/tests/unit/graph/test_event_recorder.py
📚 Learning: 2025-07-21T14:16:14.125Z
Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-07-21T14:16:14.125Z
Learning: Applies to src/backend/tests/**/*.py : Test timeout and performance constraints in backend Python tests using 'asyncio.wait_for' and timing assertions.
Applied to files:
src/backend/tests/unit/graph/test_graph_mutation_events.py
📚 Learning: 2025-07-18T18:25:54.487Z
Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/backend_development.mdc:0-0
Timestamp: 2025-07-18T18:25:54.487Z
Learning: Applies to src/backend/tests/unit/**/*.py : Test component integration within flows using create_flow, build_flow, and get_build_events utilities
Applied to files:
src/backend/tests/unit/graph/test_graph_mutation_events.pysrc/backend/tests/unit/graph/test_event_recorder.py
📚 Learning: 2025-07-21T14:16:14.125Z
Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-07-21T14:16:14.125Z
Learning: Applies to src/backend/tests/**/*.py : Test flows using predefined JSON data and utility functions such as 'create_flow', 'build_flow', 'get_build_events', and 'consume_and_assert_stream' in backend Python tests.
Applied to files:
src/backend/tests/unit/graph/test_event_recorder.py
📚 Learning: 2025-07-21T14:16:14.125Z
Learnt from: CR
Repo: langflow-ai/langflow PR: 0
File: .cursor/rules/testing.mdc:0-0
Timestamp: 2025-07-21T14:16:14.125Z
Learning: Applies to src/backend/tests/**/*.py : Test both sync and async code paths in backend Python tests, using 'pytest.mark.asyncio' for async tests.
Applied to files:
src/backend/tests/unit/graph/test_event_recorder.py
🧬 Code graph analysis (8)
src/lfx/src/lfx/components/logic/loop.py (2)
src/lfx/src/lfx/custom/custom_component/component.py (2)
get_incoming_edge_by_target_param(224-239)ctx(284-288)src/lfx/src/lfx/custom/custom_component/custom_component.py (1)
graph(185-186)
src/lfx/src/lfx/debug/events.py (2)
src/lfx/src/lfx/graph/graph/base.py (1)
metadata(1026-1039)src/lfx/src/lfx/graph/graph/runnable_vertices_manager.py (2)
to_dict(13-20)from_dict(23-30)
src/lfx/src/lfx/custom/custom_component/custom_component.py (1)
src/lfx/src/lfx/graph/graph/base.py (1)
mark_branch_sync(1139-1147)
src/backend/tests/unit/graph/test_graph_mutation_events.py (2)
src/lfx/src/lfx/debug/events.py (1)
GraphMutationEvent(18-73)src/lfx/src/lfx/graph/graph/base.py (4)
Graph(57-2668)register_observer(199-208)extend_run_queue(1642-1686)add_dynamic_dependency(1688-1764)
src/lfx/src/lfx/graph/graph/runnable_vertices_manager.py (1)
src/lfx/src/lfx/graph/graph/base.py (2)
remove_from_predecessors(2611-2613)remove_vertex_from_runnables(2615-2617)
src/lfx/src/lfx/graph/graph/base.py (4)
src/lfx/src/lfx/debug/events.py (2)
GraphMutationEvent(18-73)to_dict(46-58)src/lfx/src/lfx/graph/graph/runnable_vertices_manager.py (3)
to_dict(13-20)remove_from_predecessors(102-107)remove_vertex_from_runnables(125-128)src/lfx/src/lfx/custom/custom_component/custom_component.py (1)
user_id(189-192)src/lfx/src/lfx/components/logic/loop.py (1)
reset_loop_state(139-155)
src/backend/tests/unit/graph/test_event_recorder.py (2)
src/lfx/src/lfx/debug/event_recorder.py (3)
record_graph_with_events(208-243)get_events_by_type(66-68)get_queue_evolution(81-95)src/lfx/src/lfx/graph/graph/base.py (2)
Graph(57-2668)from_payload(1316-1352)
src/lfx/src/lfx/debug/event_recorder.py (2)
src/lfx/src/lfx/debug/events.py (1)
GraphMutationEvent(18-73)src/lfx/src/lfx/graph/graph/base.py (3)
register_observer(199-208)async_start(471-502)unregister_observer(210-219)
🪛 markdownlint-cli2 (0.18.1)
PR_DESCRIPTION.md
68-68: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (14)
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 3
- GitHub Check: Run Backend Tests / LFX Tests - Python 3.10
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 4
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 5
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 2
- GitHub Check: Run Backend Tests / Integration Tests - Python 3.10
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 1
- GitHub Check: Test Docker Images / Test docker images
- GitHub Check: Lint Backend / Run Mypy (3.11)
- GitHub Check: Lint Backend / Run Mypy (3.13)
- GitHub Check: Lint Backend / Run Mypy (3.12)
- GitHub Check: Test Starter Templates
- GitHub Check: Update Component Index
- GitHub Check: Update Starter Projects
🔇 Additional comments (10)
src/lfx/src/lfx/custom/custom_component/custom_component.py (2)
145-145: LGTM - Correct usage of synchronous wrapper.The changes from
mark_branchtomark_branch_syncare correct and align with the new synchronous branch marking API introduced in the graph mutation system. The wrapper handles the async→sync conversion internally.Also applies to: 160-160
387-396: More resilient source code handling for dynamic environments.The updated exception handling gracefully handles environments where source code is unavailable (notebooks, REPL, dynamically defined classes) by using placeholder strings instead of raising errors. This is a good improvement for developer experience in interactive environments.
Verify that downstream code consuming
self._codecan handle placeholder strings:src/lfx/src/lfx/custom/custom_component/component.py (1)
387-396: Consistent with custom_component.py - resilient source handling.The exception handling changes here mirror those in
custom_component.py, providing consistent behavior when source code is unavailable. This enables component execution in notebooks and REPL sessions without hard failures.src/lfx/src/lfx/components/logic/loop.py (2)
90-104: Good defensive programming with synchronized updates.The refactored
update_dependencymethod now:
- Guards against duplicate dependencies (line 97 check)
- Synchronizes both
run_predecessorsandrun_mapstructures (critical forremove_from_predecessors()correctness)- Documents the fast path usage
The TODO comment appropriately acknowledges future async conversion.
139-155: Useful API for loop state management.The new
reset_loop_state()method provides a clean way to reset loop internal state between independent graph executions. The comprehensive cleanup of all context variables (initialization flag, index, aggregated results, and data) ensures fresh loop iterations.src/lfx/src/lfx/graph/graph/runnable_vertices_manager.py (1)
102-107: Clean async conversion aligned with mutation pattern.The conversion of
remove_from_predecessorsandremove_vertex_from_runnablesto async methods is consistent with the broader asynchronous mutation pathway introduced in this PR. The call site at line 128 correctly awaits the async operation.Also applies to: 125-128
src/lfx/src/lfx/debug/events.py (1)
17-77: Well-designed event structure for mutation tracking.The
GraphMutationEventdataclass provides a comprehensive, serializable event model with:
- Complete before/after state snapshots
- Structured change descriptions
- Temporal tracking via step counter
- Timing field for before/after differentiation
- Clean serialization support
The
GraphObservertype alias clearly documents the async observer signature.src/lfx/src/lfx/debug/event_recorder.py (3)
31-60: Pickle serialization appropriate for debug tooling.The use of
picklefor save/load is reasonable for a debugging tool where the recordings are not expected to cross trust boundaries. Thenoqa: S301on line 58 appropriately acknowledges the security consideration.
177-206: Clean observer implementation.The
EventRecorderclass follows the observer pattern correctly:
- Async
on_eventcallback signature matchesGraphObserver- Accumulates events without modifying graph state
- Tracks component executions for analysis
208-243: Robust helper with proper cleanup.The
record_graph_with_eventsfunction provides a clean API with proper observer lifecycle management:
- Registers observer before execution
- Unregisters in
finallyblock to ensure cleanup even on errors- Clear usage example in docstring
PR_DESCRIPTION.md
Outdated
| ### Architecture | ||
|
|
||
| ``` | ||
| Graph | ||
| ├── register_observer() / unregister_observer() | ||
| ├── _emit_event() - Emits events to all observers | ||
| └── All mutations → emit before/after events | ||
| ├── extend_run_queue() | ||
| ├── add_dynamic_dependency() | ||
| ├── mark_branch_sync() | ||
| └── remove_from_predecessors() | ||
| ``` |
There was a problem hiding this comment.
Specify a language on this fence to satisfy markdownlint.
markdownlint (MD040) is flagging this block because the fence lacks a language hint. Adding something like text keeps the ASCII diagram readable and unblocks the lint job.
-```
+```text
Graph
├── register_observer() / unregister_observer()
├── _emit_event() - Emits events to all observers
└── All mutations → emit before/after events
├── extend_run_queue()
├── add_dynamic_dependency()
├── mark_branch_sync()
└── remove_from_predecessors()
<details>
<summary>🧰 Tools</summary>
<details>
<summary>🪛 markdownlint-cli2 (0.18.1)</summary>
68-68: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
</details>
</details>
<details>
<summary>🤖 Prompt for AI Agents</summary>
In PR_DESCRIPTION.md around lines 66 to 77, the fenced code block containing the
ASCII architecture diagram lacks a language hint which triggers markdownlint
MD040; update the opening fence to include a language (for example change totext) so the block becomes a ```text fenced block, keep the diagram content
unchanged, save the file and re-run the linter to confirm the warning is
resolved.
</details>
<!-- fingerprinting:phantom:medusa:sabertoothed -->
<!-- This is an auto-generated comment by CodeRabbit -->
| test_file = Path("src/backend/tests/data/LoopTest.json") | ||
| data = json.loads(test_file.read_text()) |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
🧩 Analysis chain
Verify test data file path reliability.
The hard-coded relative path "src/backend/tests/data/LoopTest.json" assumes execution from repository root. This might break if tests are run from different directories.
Verify the test data file exists and the path resolves correctly:
Also applies to: 44-45
🏁 Script executed:
#!/bin/bash
# Verify test data file exists
fd LoopTest.json src/backend/tests/Length of output: 36
Now I need to examine the test file itself and any pytest configuration:Now let me examine the test file and conftest to understand how it's actually used:
Let me examine the actual test file context and project configuration:Now let me examine the actual test file and pytest configuration:
Based on the comprehensive analysis, I now have sufficient information to provide a final response. The hard-coded relative path is indeed a valid concern, but the original review comment needs clarification on whether this is specific to Langflow's testing conventions.
Use Path(__file__).parent for test-file-relative paths instead of hard-coded repository paths.
Using Path(__file__).parent is the recommended approach to retrieve files relative to the test module. The current code at lines 17-18 and 44-45 uses Path("src/backend/tests/data/LoopTest.json"), which assumes the working directory is the repository root and breaks if tests run from different directories.
Refactor to:
test_file = Path(__file__).parent.parent.parent / "data" / "LoopTest.json"This ensures the path resolves correctly regardless of where pytest is invoked from.
🏁 Script executed:
#!/bin/bash
# Examine test_event_recorder.py to understand path usage
head -50 src/backend/tests/unit/graph/test_event_recorder.pyLength of output: 1475
🏁 Script executed:
#!/bin/bash
# Check for pytest configuration
if [ -f pyproject.toml ]; then
echo "=== pyproject.toml ==="
grep -A 10 "\[tool.pytest" pyproject.toml
fi
if [ -f pytest.ini ]; then
echo "=== pytest.ini ==="
cat pytest.ini
fiLength of output: 422
🏁 Script executed:
#!/bin/bash
# Check if conftest.py exists and examine it
find . -name conftest.py -type f | head -5Length of output: 103
🏁 Script executed:
#!/bin/bash
# View the test_event_recorder.py file around lines 17-18 and 44-45
sed -n '1,60p' src/backend/tests/unit/graph/test_event_recorder.pyLength of output: 1705
Use Path(__file__).parent for test-file-relative paths instead of hard-coded repository paths.
The hard-coded relative path Path("src/backend/tests/data/LoopTest.json") assumes execution from the repository root. While pytest's testpaths configuration points to test directories, it does not change the working directory—tests execute from the repository root by default. This becomes fragile if tests run from subdirectories or if the working directory changes during execution.
Refactor both occurrences (lines 17-18 and 44-45) to use:
test_file = Path(__file__).parent.parent.parent / "data" / "LoopTest.json"This makes the path relative to the test file itself, ensuring it resolves correctly regardless of where pytest is invoked.
🤖 Prompt for AI Agents
In src/backend/tests/unit/graph/test_event_recorder.py around lines 17-18 and
44-45, the test currently uses a hard-coded repository-relative path for
LoopTest.json which breaks if tests run from a different working directory;
change both occurrences to build the path relative to the test file itself by
using Path(__file__).parent.parent.parent combined with the "data/LoopTest.json"
segments so the test always resolves the fixture file regardless of where pytest
is invoked.
| # Handle TYPE_CHECKING blocks - extract imports from inside | ||
| # TYPE_CHECKING blocks contain imports only needed for type hints | ||
| # but we need them available at runtime for get_type_hints() to work | ||
| for inner_node in node.body: | ||
| if isinstance(inner_node, ast.Import): | ||
| imports.append(inner_node) | ||
| elif isinstance(inner_node, ast.ImportFrom) and inner_node.module is not None: | ||
| import_froms.append(inner_node) |
There was a problem hiding this comment.
Keep TYPE_CHECKING imports optional at runtime.
By appending TYPE_CHECKING imports to the main imports/import_froms lists we now force them to succeed at execution time. That regresses the common pattern of keeping heavy or optional dependencies (e.g., pandas, torch, etc.) behind if TYPE_CHECKING: so custom components remain usable without those packages installed. Today such components load fine because those imports are never executed; with this change they start raising ModuleNotFoundError during validation.
Please treat TYPE_CHECKING imports as best-effort instead: either keep them out of the mandatory lists or make their import path swallow ModuleNotFoundError (and optionally stub placeholders) so we preserve backwards compatibility while still improving get_type_hints() support.
- imports = []
- import_froms = []
+ imports = []
+ import_froms = []
+ optional_imports = []
+ optional_import_froms = []
…
- elif isinstance(node, ast.If) and _is_type_checking_block(node):
+ elif isinstance(node, ast.If) and _is_type_checking_block(node):
…
- imports.append(inner_node)
+ optional_imports.append(inner_node)
…
- import_froms.append(inner_node)
+ optional_import_froms.append(inner_node)
…
+ for node in optional_imports:
+ for alias in node.names:
+ try:
+ module_obj = importlib.import_module(alias.name)
+ except ModuleNotFoundError:
+ continue
+ variable_name = alias.asname or alias.name.split(".")[0]
+ exec_globals[variable_name] = module_obj
+
+ for node in optional_import_froms:
+ module_names_to_try = [node.module]
+ if node.module and node.module.startswith("langflow."):
+ module_names_to_try.append(node.module.replace("langflow.", "lfx.", 1))
+ for module_name in module_names_to_try:
+ try:
+ imported_module = _import_module_with_warnings(module_name)
+ except ModuleNotFoundError:
+ continue
+ _handle_module_attributes(imported_module, node, module_name, exec_globals)
+ breakCommittable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In src/lfx/src/lfx/custom/validate.py around lines 364 to 371, the code
currently appends imports found inside TYPE_CHECKING blocks to the main
imports/import_froms lists which forces those optional dependencies to be
imported at runtime; instead, detect TYPE_CHECKING-block imports and do NOT add
them to the mandatory imports lists but collect them separately and, when
attempting to apply them later, import them in a try/except that catches
ModuleNotFoundError (and optionally ImportError), swallowing the error and
continuing (optionally logging a debug message) so TYPE_CHECKING dependencies
remain best-effort and do not break runtime validation.
| async def reset_inactivated_vertices(self) -> None: | ||
| """Resets the inactivated vertices in the graph (async).""" | ||
| for vertex_id in self.inactivated_vertices.copy(): | ||
| self.mark_vertex(vertex_id, "ACTIVE") | ||
| self.inactivated_vertices = set() | ||
| await self.mark_vertex(vertex_id, "ACTIVE") | ||
| self.inactivated_vertices = set() | ||
|
|
There was a problem hiding this comment.
Do not auto-reactivate inactivated vertices
reset_inactivated_vertices must only clear the bookkeeping set. The new implementation calls mark_vertex(..., "ACTIVE"), which reactivates branches that conditional components intentionally left inactive, breaking IF/ELSE flow control. Restore the previous behavior of just clearing the set.
Apply this diff:
async def reset_inactivated_vertices(self) -> None:
"""Resets the inactivated vertices in the graph (async)."""
- for vertex_id in self.inactivated_vertices.copy():
- await self.mark_vertex(vertex_id, "ACTIVE")
- self.inactivated_vertices = set()
+ self.inactivated_vertices.clear()🤖 Prompt for AI Agents
In src/lfx/src/lfx/graph/graph/base.py around lines 1054 to 1059, the current
implementation calls await self.mark_vertex(vertex_id, "ACTIVE") which
reactivates vertices and breaks intended IF/ELSE control flow; change the method
to only clear the bookkeeping set of inactivated_vertices without calling
mark_vertex (e.g., remove the loop and just clear or assign an empty set),
preserving the async signature and docstring.
… instead of printing directly
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
Summary
Introduces a comprehensive event-based debugging system for Langflow graph execution with zero overhead when not in use. Uses a pure observer pattern to track all graph state mutations during execution.
Key Changes
🎯 Event System
GraphMutationEventsystem tracking all graph state changesregister_observer()/unregister_observer()📊 Event Recording
EventRecordercaptures all graph mutationsEventBasedRecordingwith analysis methods (queue evolution, dependency changes, etc.)🔧 Graph Improvements
run_predecessorsandrun_mapdependenciesremove_from_predecessors()async, addedmark_branch_sync()for sync contextsTYPE_CHECKINGblock handling🧪 Testing
async_start()vsarun()equivalence)Usage
from lfx.debug.event_recorder import record_graph_with_events
recording = await record_graph_with_events(graph, "My Flow")
recording.show_summary()
recording.get_queue_evolution()
recording.save("recording.pkl")
Summary by CodeRabbit
Release Notes
New Features
Tests
Chores