[ES|QL] Remove implicit limit appended for each subquery branch
#139058
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves: #138106
The implicit
limitappended to each subquery adds limitations for subquery, especially when working withlookup join+ remove index patterns, andinline stats. It may also cause non-deterministic results from subqueries when there is nosortin the subquery to ensure the order of the intermediate results.This PR attempts to remove the implicit
limitappend to each subquery branch, and enables more capabilities of subquery. There is potential risk that the intermediate results returned by a subquery is huge, and they need to be processed in batches or CB if it is too big to process.The major changes are listed here, more investigation on heap attack tests is needed before this is ready for review:
HeapAttackSubqueryITis added to catch potential OOM or CBE caused by large intermediate results from subqueries. The tests hit OOM, instead of CBE, more investigation is needed, this is a prerequisite for this PR to be merged. More heap attack tests might be needed.PushDownFilterAndLimitIntoUnionAllis updated accordingly afterAnalyzerdoes not add the implicitlimitto each subquery branch.limit.