Skip to content

[Internal] Query: Adds Split Support for Ode #3572

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ealsur merged 41 commits into from
Jan 20, 2023
Merged

[Internal] Query: Adds Split Support for Ode #3572

Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
1dbfc8a
Added tests to test different aspects of merge/split support with Opt…
akotalwar Oct 19, 2022
2fc0b89
Added gone exception simulation tests.
akotalwar Oct 26, 2022
36d18bb
Added new tests and improved test infra
akotalwar Nov 4, 2022
924b24a
Removed ParalleContEvocation test. Fixed comments
akotalwar Nov 9, 2022
095a44d
Removed CreateParallelCrossPartitionPipelineStateAsync() as it is not…
akotalwar Nov 9, 2022
9115fcf
Removed while loop in CreateDocumentContainerAsync()
akotalwar Nov 10, 2022
c054909
Fixed comments.
akotalwar Nov 11, 2022
5807da9
Merge branch 'master' into users/akotalwar/TestsForMergeSplitSupport
akotalwar Nov 11, 2022
bfb4110
Updated ExecuteGoneExceptionOnODEPipeline()
akotalwar Nov 12, 2022
ea0aac4
Added type Assert for ExecuteGoneExceptionOnODEPipeline()
akotalwar Nov 12, 2022
b886f40
Replaced try-catch with if statement in MoveNextAsync()
akotalwar Nov 17, 2022
04389e0
Added delegate to access TryCreateCoreContextAsync()
akotalwar Nov 30, 2022
4af2036
Added check to confirm Ode pipeline is not called in fallback plan
akotalwar Nov 30, 2022
1257ad3
Updated method name from OptimisticDirectExecutionContext() to TryCre…
akotalwar Nov 30, 2022
bd4c195
Using delegate instead of Func<>.
akotalwar Dec 1, 2022
661b4f8
Ode fallback plan always calls Specialized pipeline
akotalwar Dec 2, 2022
039fbe8
Using ServiceInterop/Gateway to get QueryPlan for Specialized Pipeline
akotalwar Dec 7, 2022
60392ad
Added new test to check handling of failing fallback pipeline
akotalwar Dec 18, 2022
f9fbd8a
Code cleanup
akotalwar Dec 19, 2022
c230837
Added logic for handling non ODE continuation tokens
akotalwar Dec 21, 2022
c66be0f
Moved delegate away from member variables
akotalwar Dec 21, 2022
758c07f
Added tests for Merge case
akotalwar Dec 22, 2022
d442b99
Updated method names
akotalwar Dec 28, 2022
d1dcd7e
Added checks for tryCatch
akotalwar Dec 30, 2022
294d173
Updated SetCancellationToken() to use Try
akotalwar Jan 3, 2023
0621090
Updated TryUnwrapContinuationToken()
akotalwar Jan 4, 2023
87c9916
Removed changes in FlakyDocumentContainer.cs
akotalwar Jan 6, 2023
c4b0546
Removed unused imports
akotalwar Jan 6, 2023
3de111c
Updated comments
akotalwar Jan 6, 2023
7543ec7
Fixed comments and cleaned up test code
akotalwar Jan 9, 2023
134dcc8
Added CosmosElement null check in TryUnwrapContinuationToken()
akotalwar Jan 9, 2023
9d37c12
Removed FlakyDocumentContainer.cs from pull request
akotalwar Jan 9, 2023
a009e53
Removed unused imports
akotalwar Jan 9, 2023
400d4f5
Updated TryUnwrapContinuationToken()
akotalwar Jan 9, 2023
6179112
Update MoveNextAsync() call in OptimisticDirectExecutionQueryBaseline…
akotalwar Jan 10, 2023
c94db62
Made MergeTestUtil.IsFailedFallbackPipelineTest a readonly property
akotalwar Jan 11, 2023
2416a89
Added IsPartitionSplitException() overload to take CosmosElement
akotalwar Jan 13, 2023
bc8fa42
Resolved git conflicts
akotalwar Jan 13, 2023
c129afe
Fixed bug regarding syntax error queries
akotalwar Jan 18, 2023
a68dd2c
Merge branch 'master' into users/akotalwar/SplitSupport
akotalwar Jan 18, 2023
0d5390e
Merge branch 'master' into users/akotalwar/SplitSupport
akotalwar Jan 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions Microsoft.Azure.Cosmos/src/Query/Core/Monads/TryCatch{TResult}.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,38 @@ public async Task<TryCatch<T>> TryAsync<T>(
return matchResult;
}

public async ValueTask<TryCatch<T>> TryAsync<T>(
Func<TResult, ValueTask<T>> onSuccess)
{
TryCatch<T> matchResult;
if (this.Succeeded)
{
matchResult = TryCatch<T>.FromResult(await onSuccess(this.either.FromRight(default)));
}
else
{
matchResult = TryCatch<T>.FromException(this.either.FromLeft(default));
}

return matchResult;
}

public TryCatch<T> Try<T>(
Func<TResult, TryCatch<T>> onSuccess)
{
TryCatch<T> matchResult;
if (this.Succeeded)
{
matchResult = onSuccess(this.either.FromRight(default));
}
else
{
matchResult = TryCatch<T>.FromException(this.either.FromLeft(default));
}

return matchResult;
}

public TryCatch<TResult> Catch(
Action<Exception> onError)
{
Expand Down
18 changes: 18 additions & 0 deletions Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CosmosExceptionExtensions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
// ------------------------------------------------------------
// Copyright (c) Microsoft Corporation. All rights reserved.
// ------------------------------------------------------------

namespace Microsoft.Azure.Cosmos.Query.Core.Pipeline
{
using System;

internal static class CosmosExceptionExtensions
{
public static bool IsPartitionSplitException(this Exception ex)
{
return ex is CosmosException cosmosException
&& (cosmosException.StatusCode == System.Net.HttpStatusCode.Gone)
&& (cosmosException.SubStatusCode == (int)Documents.SubStatusCodes.PartitionKeyRangeGone);
}
}
}
326 changes: 240 additions & 86 deletions Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CosmosQueryExecutionContextFactory.cs
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 1 addition & 3 deletions ...src/Query/Core/Pipeline/CrossPartition/OrderBy/OrderByCrossPartitionQueryPipelineStage.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1153,9 +1153,7 @@ private static bool IsSplitException(Exception exception)
exception = exception.InnerException;
}

return exception is CosmosException cosmosException
&& (cosmosException.StatusCode == HttpStatusCode.Gone)
&& (cosmosException.SubStatusCode == (int)Documents.SubStatusCodes.PartitionKeyRangeGone);
return exception.IsPartitionSplitException();
}

public void SetCancellationToken(CancellationToken cancellationToken)
Expand Down
6 changes: 3 additions & 3 deletions ...ery/Core/Pipeline/OptimisticDirectExecution/OptimisticDirectExecutionContinuationToken.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ public static CosmosElement ToCosmosElement(OptimisticDirectExecutionContinuatio
public static TryCatch<OptimisticDirectExecutionContinuationToken> TryCreateFromCosmosElement(CosmosElement cosmosElement)
{
CosmosObject cosmosObjectContinuationToken = cosmosElement as CosmosObject;
if (cosmosObjectContinuationToken == null)
if (cosmosObjectContinuationToken == null || !cosmosObjectContinuationToken.ContainsKey(OptimisticDirectExecutionToken))
{
return TryCatch<OptimisticDirectExecutionContinuationToken>.FromException(
new MalformedChangeFeedContinuationTokenException(
message: $"Malformed Continuation Token"));
new MalformedContinuationTokenException(
message: $"Malformed Continuation Token: Expected OptimisticDirectExecutionToken\r\n"));
}

TryCatch<ParallelContinuationToken> inner = ParallelContinuationToken.TryCreateFromCosmosElement(cosmosObjectContinuationToken[OptimisticDirectExecutionToken]);
Expand Down
319 changes: 211 additions & 108 deletions ...ry/Core/Pipeline/OptimisticDirectExecution/OptimisticDirectExecutionQueryPipelineStage.cs
View file
Open in desktop

Large diffs are not rendered by default.

26 changes: 26 additions & 0 deletions ...e/OptimisticDirectExecutionQueryBaselineTests.NegativeOptimisticDirectExecutionOutput.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,30 @@
<ExecuteAsOptimisticDirectExecution>false</ExecuteAsOptimisticDirectExecution>
</Output>
</Result>
<Result>
<Input>
<Description>Single Partition Key with Parallel continuation token</Description>
<Query>SELECT * FROM c</Query>
<PartitionKeys>
<Key>/pk</Key>
</PartitionKeys>
<PartitionKeyType>Hash</PartitionKeyType>
</Input>
<Output>
<ExecuteAsOptimisticDirectExecution>false</ExecuteAsOptimisticDirectExecution>
</Output>
</Result>
<Result>
<Input>
<Description>Single Partition Key with OrderBy continuation token</Description>
<Query>SELECT * FROM c ORDER BY c._ts</Query>
<PartitionKeys>
<Key>/pk</Key>
</PartitionKeys>
<PartitionKeyType>Hash</PartitionKeyType>
</Input>
<Output>
<ExecuteAsOptimisticDirectExecution>false</ExecuteAsOptimisticDirectExecution>
</Output>
</Result>
</Results>
38 changes: 32 additions & 6 deletions ...e/OptimisticDirectExecutionQueryBaselineTests.PositiveOptimisticDirectExecutionOutput.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<Results>
<Result>
<Input>
<Description>Partition Key + Value and Distinct</Description>
<Query>SELECT DISTINCT c.key FROM c</Query>
<Description>Single Partition Key and Distinct</Description>
<Query>SELECT DISTINCT c.age FROM c</Query>
<PartitionKeys>
<Key>/pk</Key>
</PartitionKeys>
Expand All @@ -14,8 +14,8 @@
</Result>
<Result>
<Input>
<Description>Partition Key + Value and Min Aggregate</Description>
<Query>SELECT VALUE MIN(c.key) FROM c</Query>
<Description>Single Partition Key and Min Aggregate</Description>
<Query>SELECT VALUE MIN(c.age) FROM c</Query>
<PartitionKeys>
<Key>/pk</Key>
</PartitionKeys>
Expand All @@ -27,8 +27,34 @@
</Result>
<Result>
<Input>
<Description>Partition Key + Value Fields</Description>
<Query>SELECT c.key FROM c</Query>
<Description>Single Partition Key and Value Field</Description>
<Query>SELECT c.age FROM c</Query>
<PartitionKeys>
<Key>/pk</Key>
</PartitionKeys>
<PartitionKeyType>Hash</PartitionKeyType>
</Input>
<Output>
<ExecuteAsOptimisticDirectExecution>true</ExecuteAsOptimisticDirectExecution>
</Output>
</Result>
<Result>
<Input>
<Description>Single Partition Key and Value Field</Description>
<Query>SELECT * FROM c</Query>
<PartitionKeys>
<Key>/pk</Key>
</PartitionKeys>
<PartitionKeyType>Hash</PartitionKeyType>
</Input>
<Output>
<ExecuteAsOptimisticDirectExecution>true</ExecuteAsOptimisticDirectExecution>
</Output>
</Result>
<Result>
<Input>
<Description>Single Partition Key and Ode continuation token</Description>
<Query>SELECT * FROM c</Query>
<PartitionKeys>
<Key>/pk</Key>
</PartitionKeys>
Expand Down
Loading