Skip to content

refactor: optimize agent list payload and improve multimodal detection logic#12942

Merged
KevinHuSh merged 2 commits intoinfiniflow:mainfrom
eviaaaaa:fix/agent-list-dsl-and-multimodal-detection
Feb 2, 2026
Merged

refactor: optimize agent list payload and improve multimodal detection logic#12942
KevinHuSh merged 2 commits intoinfiniflow:mainfrom
eviaaaaa:fix/agent-list-dsl-and-multimodal-detection

Conversation

@eviaaaaa
Copy link
Copy Markdown
Contributor

@eviaaaaa eviaaaaa commented Feb 2, 2026

Description

This PR focuses on API performance optimization and refining the model capability detection logic in the Agent/Canvas module.

1. Performance Optimization (Backend)

  • Changes: Removed cls.model.dsl from query fields in UserCanvasService.get_by_tenant_ids.
  • Reasoning: The dsl object is large and unnecessary for the Agent list view. Excluding it reduces the payload size of the /v1/canvas/list API, leading to faster serialization and reduced network latency.
  • Consistency: Full DSL data remains accessible via the individual /v1/canvas/get/<id> endpoint used in the detail view.

2. Multimodal Detection Refinement (Frontend)

  • Changes: Replaced model_type === LlmModelType.Image2text with tags?.includes('IMAGE2TEXT').
  • Reasoning: In RAGFlow, model_type defines the primary role of a model (e.g., chat). However, many advanced Chat models are also vision-capable. Since model_type is a single-value field, it cannot represent these multiple capabilities.
  • Solution: Utilizing the tags field (which supports multiple attributes) to check for IMAGE2TEXT ensures that models like gpt-5.2-pro correctly display multimodal input options.

Type of Change

  • Bug fix (logic correction for multimodal detection)
  • Optimization (performance improvement for list API)

Main Changes

  • api/db/services/canvas_service.py: Optimized DB query by excluding heavy DSL fields.
  • web/src/pages/agent/form/agent-form/index.tsx: Enhanced capability detection using the tags system.

Verification

  • Verified Agent list loads faster with reduced response payload.
  • Confirmed that chat models with the IMAGE2TEXT tag now correctly enable the multimodal input UI.

Since model_type only represents the primary category (e.g., 'chat'),
it cannot capture auxiliary capabilities. Switching to 'IMAGE2TEXT' tag
detection allows multimodal support for versatile models like gpt-5.2-pro.
The dsl field in the agent list is typically large and causes unnecessary
network overhead. DSL is now only fetched in the detail view.
@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. 🌈 python Pull requests that update Python code 🐞 bug Something isn't working, pull request that fix bug. 🧰 typescript Pull requests that update Typescript code labels Feb 2, 2026
@KevinHuSh KevinHuSh added the ci Continue Integration label Feb 2, 2026
@KevinHuSh KevinHuSh marked this pull request as draft February 2, 2026 07:57
@KevinHuSh KevinHuSh marked this pull request as ready for review February 2, 2026 07:57
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 2, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 44.46%. Comparing base (23bdf25) to head (ffa09cf).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #12942   +/-   ##
=======================================
  Coverage   44.46%   44.46%           
=======================================
  Files          43       43           
  Lines        9266     9266           
  Branches      107      107           
=======================================
  Hits         4120     4120           
  Misses       5127     5127           
  Partials       19       19           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@KevinHuSh KevinHuSh merged commit 2e5a186 into infiniflow:main Feb 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🐞 bug Something isn't working, pull request that fix bug. ci Continue Integration 🌈 python Pull requests that update Python code size:XS This PR changes 0-9 lines, ignoring generated files. 🧰 typescript Pull requests that update Typescript code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants