Skip to content

Conversation

@fang-xing-esql
Copy link
Member

@fang-xing-esql fang-xing-esql commented Dec 4, 2025

Resolves: #138106

The implicit limit appended to each subquery adds limitations for subquery, especially when working with lookup join+ remove index patterns, and inline stats. It may also cause non-deterministic results from subqueries when there is no sort in the subquery to ensure the order of the intermediate results.

This PR attempts to remove the implicit limit append to each subquery branch, and enables more capabilities of subquery. There is potential risk that the intermediate results returned by a subquery is huge, and they need to be processed in batches or CB if it is too big to process.

The major changes are listed here, more investigation on heap attack tests is needed before this is ready for review:

  • HeapAttackSubqueryIT is added to catch potential OOM or CBE caused by large intermediate results from subqueries. The tests hit OOM, instead of CBE, more investigation is needed, this is a prerequisite for this PR to be merged. More heap attack tests might be needed.
  • PushDownFilterAndLimitIntoUnionAll is updated accordingly after Analyzer does not add the implicit limit to each subquery branch.
  • Subquery related tests are updated accordingly after removing the implicit limit.

@elasticsearchmachine
Copy link
Collaborator

Hi @fang-xing-esql, I've created a changelog YAML for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ES|QL] Remove the implicit LIMIT command added to each subquery (UnionAll branch)

2 participants