[Cosmos] Adds support for non streaming ORDER BY #35468
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
|
API change check APIView has identified API level changes in this PR and created following API reviews. |
neildsh
reviewed
May 3, 2024
sdk/cosmos/azure-cosmos/azure/cosmos/_execution_context/query_execution_info.py
Outdated
Show resolved
Hide resolved
neildsh
reviewed
May 3, 2024
sdk/cosmos/azure-cosmos/azure/cosmos/_execution_context/non_streaming_order_by_aggregator.py
Show resolved
Hide resolved
neildsh
reviewed
May 3, 2024
sdk/cosmos/azure-cosmos/azure/cosmos/_execution_context/document_producer.py
Outdated
Show resolved
Hide resolved
neildsh
reviewed
May 3, 2024
sdk/cosmos/azure-cosmos/azure/cosmos/_execution_context/document_producer.py
Outdated
Show resolved
Hide resolved
simorenoh
requested review from
annatisch,
bambriz,
kushagraThapar and
xinlian12
as code owners
xinlian12
reviewed
May 3, 2024
Member
Author
|
/azp run python - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
/azp run python - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
/azp run python - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
/azp run python - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
/azp run python - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
/azp run python - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
/azp run python - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Python follow-up to the .NET PR: Azure/azure-cosmos-dotnet-v3#4362
Using the flag for
nonStreamingOrderBythat is now present in the query plan, we choose to create a separate query execution context for these types of operations.The process starts as a normal order-by query, creating one document producer per physical partition involved. However since there's no ordering guarantees, in this case we first need to drain the results from these document producers. The current approach is to initialize a priority queue that will serve as the ordering mechanism, receiving a document producer's batch of items one at a time, and then re-balancing with every new document producer being processed. This makes it so we hold 2*items_per_partition items in memory at most at any given time. Once fully drained, we return a priority queue with only the top k items in it.
This PR includes changes for the following:
This branch was made on top of this one that includes just the changes for the vector policies: #34882 please ignore those files