Consolidate Geometry and Style into Layout Tree and Centralize Interaction Handling

[christian-byrne]

[Refactor] - Extract Interaction State Management from Renderers

Overview

Currently, LiteGraph tightly couples interaction handling with canvas rendering. We need to extract interaction state management and graph mutations into renderer-agnostic services, allowing different renderers to share the same interaction logic while handling their own layout and hit testing.

Parent Issue

Part of #4661 - Interactive Refactor Toward V3

Problem Statement

• LiteGraph owns both rendering AND interaction handling
• Cannot swap renderer implementations
• Interaction state (selection, dragging) is coupled to canvas renderer
• Graph mutations happen directly in renderer code
• No way to implement alternative renderers (Vue/DOM-based, WebGL, etc.)

Current Architecture

LiteGraph = Data Model + Layout + Rendering + Interaction Handling
                          └── All tightly coupled ──┘

Proposed Architecture

Data Model (pure graph structure)
    ↓
Graph Mutation Service (single point of access)
    ↓
Renderers (each handling own layout/hit-testing/interactions)
├── Canvas Renderer (LiteGraph) - owns canvas interactions
├── DOM/Vue Renderer - owns DOM interactions
└── WebGL Renderer - owns 3D interactions

Key Components

1. Graph Mutation Service

Single point of access for all graph modifications:

interface GraphMutationService {
  // Selection operations
  selectNode(nodeId: string, addToSelection?: boolean): void
  deselectNode(nodeId: string): void
  clearSelection(): void
  
  // Position operations (renderer-agnostic)
  moveNodesByDelta(nodeIds: string[], delta: { x: number, y: number }): void
  
  // Connection operations
  connectNodes(from: PortRef, to: PortRef): void
  disconnectPort(nodeId: string, portIndex: number, isOutput: boolean): void
  
  // Node operations
  addNode(type: string, position?: Point): string
  removeNodes(nodeIds: string[]): void
}

2. Shared Graph State

Only the truly shared state that all renderers need:

interface SharedGraphState {
  // Selection is shared across renderers
  selectedNodeIds: Set<string>
  
  // Active operation (so renderers don't conflict)
  activeOperation?: 'dragging' | 'connecting' | null
  activeRenderer?: string
}

// Each renderer can query but only active renderer can mutate
interface GraphStateService {
  getSelectedNodes(): Set<string>
  isOperationActive(): boolean
  
  // Only callable by active renderer
  claimOperation(renderer: string, operation: string): boolean
  releaseOperation(): void
}

3. Renderer Interface

Each renderer owns its complete interaction and layout handling:

interface GraphRenderer {
  // Render the graph
  render(graph: GraphModel): void
  
  // Each renderer handles its own events
  attachEventListeners(element: HTMLElement): void
  detachEventListeners(): void
  
  // Renderer decides when to call mutations
  setMutationService(mutations: GraphMutationService): void
  setStateService(state: GraphStateService): void
}

4. LiteGraph Internal Refactoring (Optional)

As part of making LiteGraph more maintainable, we could refactor it to use a layout tree internally:

// Internal to LiteGraph - not exposed
class LiteGraphLayoutTree {
  private nodes: Map<string, LayoutNode>
  
  // Extract hit testing logic
  hitTest(point: Point): LayoutNode | null {
    // Current logic spread across LGraph.getNodeOnPos, etc
  }
  
  // Extract bounds calculations
  updateNodeBounds(node: LGraphNode): void {
    // Current logic in LGraphNode
  }
}

// This makes LiteGraph's processMouseDown cleaner:
processMouseDown(e: PointerEvent) {
  const hit = this.layoutTree.hitTest(this.toGraphCoords(e))
  if (hit?.type === 'node') {
    // Handle node interaction
  }
}

This is purely an internal refactoring to make LiteGraph's code cleaner, not a shared abstraction.

5. Renderer Implementations

Each renderer handles its own interaction patterns:

// Canvas Renderer (current LiteGraph)
class CanvasGraphRenderer implements GraphRenderer {
  constructor(private canvas: LGraphCanvas) {}
  
  attachEventListeners(element: HTMLElement) {
    // LiteGraph handles its own events
    this.canvas.bindEvents()
  }
  
  render(graph: GraphModel) {
    // LiteGraph renders to canvas
    this.canvas.setGraph(graph)
    this.canvas.draw()
  }
  
  // When LiteGraph wants to mutate, it calls the service
  onNodeSelected(node: LGraphNode) {
    this.mutations.selectNode(node.id.toString())
  }
  
  onNodesMoved(nodes: LGraphNode[], delta: Point) {
    const nodeIds = nodes.map(n => n.id.toString())
    this.mutations.moveNodesByDelta(nodeIds, delta)
  }
}

// DOM/Vue Renderer  
class DOMGraphRenderer implements GraphRenderer {
  attachEventListeners(element: HTMLElement) {
    element.addEventListener('click', (e) => {
      const nodeEl = e.target.closest('[data-node-id]')
      if (nodeEl) {
        const nodeId = nodeEl.dataset.nodeId
        this.mutations.selectNode(nodeId, e.shiftKey)
      }
    })
    
    // Handle drag with DOM APIs
    // Each renderer implements its own interaction logic
  }
  
  render(graph: GraphModel) {
    // Render using Vue components
    // Uses its own layout strategy (CSS, flexbox, etc)
  }
}

## Example: Extracting Graph Mutations

```typescript
// Current: LiteGraph directly manipulates graph
canvas.processMouseDown = function(e) {
  const node = this.getNodeOnPos(e.canvasX, e.canvasY)
  if (node) {
    this.selected_nodes[node.id] = node // Direct mutation
    node.is_selected = true // Direct mutation
  }
}

// New: All mutations go through service
class LiteGraphCanvasRefactored {
  processMouseDown(e: PointerEvent) {
    const node = this.getNodeOnPos(e.canvasX, e.canvasY)
    if (node) {
      // Request mutation through service
      this.mutations.selectNode(node.id.toString(), e.shiftKey)
    }
  }
  
  processDrag(delta: Point) {
    if (this.selected_nodes.length > 0) {
      const nodeIds = this.selected_nodes.map(n => n.id.toString())
      // Request position update through service
      this.mutations.moveNodesByDelta(nodeIds, delta)
    }
  }
}

Implementation Plan

Phase 1: Graph Mutation Service

1. Create GraphMutationService interface
2. Implement basic operations (select, move, connect)
3. Start routing LiteGraph mutations through service
4. Keep direct mutations working during transition

Phase 2: Shared State Service

1. Create GraphStateService for truly shared state (selection)
2. Add operation locking to prevent renderer conflicts
3. Make selection queryable by all renderers

Phase 3: Refactor LiteGraph as a Renderer

1. Extract mutations from LiteGraph's interaction code
2. Route all graph changes through mutation service
3. Optional: Internal layout tree refactoring for cleaner code
4. LiteGraph becomes one renderer implementation

Phase 4: Alternative Renderer Proof of Concept

1. Create simple Vue/DOM renderer for minimap or property panel
2. Both renderers use same mutation service
3. Demonstrate coordinated updates between renderers

Related Issues

• Convert changeTracker to use CRDT-compatible immutable state with selectively reactive mutation log #4664 - Centralize Widget Type Registry System
• Implement Graph Mutation Service Architecture #4691 - Extract LiteGraph State Management into Vue Reactive System