[WIP] RateLimitPolicy sketch

CHANGELOG.md

@@ -1,3 +1,6 @@
+## 7.2.0
+- Add RateLimit policy
+
 ## 7.1.0
 - Add SourceLink debugger support.
 - Bug fix: PolicyRegistry with .NET Core services.AddPolicyRegistry() overload (affects Polly v7.0.1-3 only)

GitVersionConfig.yaml

@@ -1 +1 @@
-next-version: 7.1.0
+next-version: 7.2.0

README.md

@@ -1,6 +1,6 @@
 # Polly
 
-Polly is a .NET resilience and transient-fault-handling library that allows developers to express policies such as Retry, Circuit Breaker, Timeout, Bulkhead Isolation, and Fallback in a fluent and thread-safe manner.  
+Polly is a .NET resilience and transient-fault-handling library that allows developers to express policies such as Retry, Circuit Breaker, Timeout, Bulkhead Isolation, Rate-limiting and Fallback in a fluent and thread-safe manner.  
 
 Polly targets .NET Standard 1.1 ([coverage](https://docs.microsoft.com/en-us/dotnet/standard/net-standard#net-implementation-support): .NET Framework 4.5-4.6.1, .NET Core 1.0, Mono, Xamarin, UWP, WP8.1+) and .NET Standard 2.0+  ([coverage](https://docs.microsoft.com/en-us/dotnet/standard/net-standard#net-implementation-support): .NET Framework 4.6.1, .NET Core 2.0+, and later Mono, Xamarin and UWP targets).
 
@@ -29,7 +29,8 @@ Polly offers multiple resilience policies:
 |**Retry** <br/>(policy family)<br/><sub>([quickstart](#retry)&nbsp;;&nbsp;[deep](https://github.com/App-vNext/Polly/wiki/Retry))</sub>|Many faults are transient and may self-correct after a short delay.| "Maybe it's just a blip" |  Allows configuring automatic retries. |
 |**Circuit-breaker**<br/>(policy family)<br/><sub>([quickstart](#circuit-breaker)&nbsp;;&nbsp;[deep](https://github.com/App-vNext/Polly/wiki/Circuit-Breaker))</sub>|When a system is seriously struggling, failing fast is better than making users/callers wait.  <br/><br/>Protecting a faulting system from overload can help it recover. | "Stop doing it if it hurts" <br/><br/>"Give that system a break" | Breaks the circuit (blocks executions) for a period, when faults exceed some pre-configured threshold. | 
 |**Timeout**<br/><sub>([quickstart](#timeout)&nbsp;;&nbsp;[deep](https://github.com/App-vNext/Polly/wiki/Timeout))</sub>|Beyond a certain wait, a success result is unlikely.| "Don't wait forever"  |Guarantees the caller won't have to wait beyond the timeout. | 
-|**Bulkhead Isolation**<br/><sub>([quickstart](#bulkhead)&nbsp;;&nbsp;[deep](https://github.com/App-vNext/Polly/wiki/Bulkhead))</sub>|When a process faults, multiple failing calls backing up can easily swamp resource (eg threads/CPU) in a host.<br/><br/>A faulting downstream system can also cause 'backed-up' failing calls upstream.<br/><br/>Both risk a faulting process bringing down a wider system. | "One fault shouldn't sink the whole ship"  |Constrains the governed actions to a fixed-size resource pool, isolating their potential  to affect others. |
+|**Bulkhead Isolation**<br/><sub>([quickstart](#bulkhead)&nbsp;;&nbsp;[deep](https://github.com/App-vNext/Polly/wiki/Bulkhead))</sub>|When a process faults, multiple failing calls can stack up (if unbounded) and can easily swamp resource (threads/ CPU/ memory) in a host.  <br/><br/>This can affect performance more widely by starving other operations of resource, bringing down the host, or causing cascading failures upstream. | "One fault shouldn't sink the whole ship"  |Constrains the governed actions to a fixed-size resource pool, isolating their potential  to affect others. |
+|**Rate-limit**<br/><sub>([quickstart](#ratelimit)&nbsp;;&nbsp;[deep](https://github.com/App-vNext/Polly/wiki/RateLimit))</sub>|Limiting the rate a system handles requests is another way to control load. <br/><br/> This can apply to the way your system accepts incoming calls, and/or to the way you call downstream services. | "Slow down a bit, will you?"  |Constrains executions to not exceed a certain rate. |
 |**Cache**<br/><sub>([quickstart](#cache)&nbsp;;&nbsp;[deep](https://github.com/App-vNext/Polly/wiki/Cache))</sub>|Some proportion of requests may be similar.| "You've asked that one before"  |Provides a response from cache if known. <br/><br/>Stores responses automatically in cache, when first retrieved. |
 |**Fallback**<br/><sub>([quickstart](#fallback)&nbsp;;&nbsp;[deep](https://github.com/App-vNext/Polly/wiki/Fallback))</sub>|Things will still fail - plan what you will do when that happens.| "Degrade gracefully"  |Defines an alternative value to be returned (or action to be executed) on failure. | 
 |**PolicyWrap**<br/><sub>([quickstart](#policywrap)&nbsp;;&nbsp;[deep](https://github.com/App-vNext/Polly/wiki/PolicyWrap))</sub>|Different faults require different strategies; resilience means using a combination.| "Defence in depth"  |Allows any of the above policies to be combined flexibly. | 
@@ -607,6 +608,12 @@ Bulkhead policies throw `BulkheadRejectedException` if items are queued to the b
 
 For more detail see: [Bulkhead policy documentation](https://github.com/App-vNext/Polly/wiki/Bulkhead) on wiki.
 
+### Rate-Limit
+
+**TODO: Documentation to be completed**
+
+<br/>
+
 ### Cache
 
 ```csharp

src/Polly.Specs/Bulkhead/BulkheadSpecsHelper.cs

@@ -1,4 +1,5 @@
 using System;
+using System.Diagnostics;
 using System.Threading;
 using System.Threading.Tasks;
 using FluentAssertions.Execution;
@@ -33,7 +34,7 @@ public BulkheadSpecsHelper(ITestOutputHelper testOutputHelper)
         /// <param name="actionContainingAssertions">The action containing fluent assertions, which must succeed within the timespan.</param>
         protected void Within(TimeSpan timeSpan, Action actionContainingAssertions)
         {
-            DateTime timeoutTime = DateTime.UtcNow.Add(timeSpan);
+            Stopwatch watch = Stopwatch.StartNew();
             while (true)
             {
                 try
@@ -45,7 +46,7 @@ protected void Within(TimeSpan timeSpan, Action actionContainingAssertions)
                 {
                     if (!(e is AssertionFailedException || e is XunitException)) { throw; }
 
-                    TimeSpan remaining = timeoutTime - DateTime.UtcNow;
+                    TimeSpan remaining = timeSpan - watch.Elapsed;
                     if (remaining <= TimeSpan.Zero) { throw; }
 
                     statusChanged.WaitOne(remaining);

src/Polly.Specs/Helpers/Bulkhead/TraceableAction.cs

@@ -38,8 +38,10 @@ public TraceableAction(int id, AutoResetEvent statusChanged, ITestOutputHelper t
             _testOutputHelper = testOutputHelper;
         }
 
-    // Note re TaskCreationOptions.LongRunning: Testing the parallelization of the bulkhead policy efficiently requires the ability to start large numbers of parallel tasks in a short space of time.  The ThreadPool's algorithm of only injecting extra threads (when necessary) at a rate of two-per-second however makes high-volume tests using the ThreadPool both slow and flaky.  For PCL tests further, ThreadPool.SetMinThreads(...) is not available, to mitigate this.  Using TaskCreationOptions.LongRunning allows us to force tasks to be started near-instantly on non-ThreadPool threads.
-    // Similarly, we use ConfigureAwait(true) when awaiting, to avoid continuations being scheduled onto a ThreadPool thread, which may only be injected too slowly in high-volume tests.
+        // Note re TaskCreationOptions.LongRunning: Testing the parallelization of the bulkhead policy efficiently requires the ability to start large numbers of parallel tasks in a short space of time.
+        // The ThreadPool's algorithm of only injecting extra threads (when necessary) at a rate of two-per-second however makes high-volume tests using the ThreadPool both slow and flaky.  In the days of PCL, further, ThreadPool.SetMinThreads(...) was not available to mitigate this.
+        // Using TaskCreationOptions.LongRunning allows us to force tasks to be started near-instantly on non-ThreadPool threads.
+        // Similarly, we use ConfigureAwait(true) when awaiting, to avoid continuations being scheduled onto a ThreadPool thread, which may only be injected too slowly in high-volume tests.
 
         public Task ExecuteOnBulkhead(BulkheadPolicy bulkhead)
         {

src/Polly.Specs/Helpers/RateLimit/IRateLimiterExtensions.cs

@@ -0,0 +1,40 @@
+using System;
+using FluentAssertions;
+using Polly.RateLimit;
+
+namespace Polly.Specs.Helpers.RateLimit
+{
+    internal static class IRateLimiterExtensions
+    {
+        public static void ShouldPermitAnExecution(this IRateLimiter rateLimiter)
+        {
+            (bool permitExecution, TimeSpan retryAfter) canExecute = rateLimiter.PermitExecution();
+
+            canExecute.permitExecution.Should().BeTrue();
+            canExecute.retryAfter.Should().Be(TimeSpan.Zero);
+        }
+
+        public static void ShouldPermitNExecutions(this IRateLimiter rateLimiter, long numberOfExecutions)
+        {
+            for (int execution = 0; execution < numberOfExecutions; execution++)
+            {
+                rateLimiter.ShouldPermitAnExecution();
+            }
+        }
+
+        public static void ShouldNotPermitAnExecution(this IRateLimiter rateLimiter, TimeSpan? retryAfter = null)
+        {
+            (bool permitExecution, TimeSpan retryAfter) canExecute = rateLimiter.PermitExecution();
+
+            canExecute.permitExecution.Should().BeFalse();
+            if (retryAfter == null)
+            {
+                canExecute.retryAfter.Should().BeGreaterThan(TimeSpan.Zero);
+            }
+            else
+            {
+                canExecute.retryAfter.Should().Be(retryAfter.Value);
+            }
+        }
+    }
+}

src/Polly.Specs/Helpers/RateLimit/ResultClassWithRetryAfter.cs

@@ -0,0 +1,21 @@
+using System;
+
+namespace Polly.Specs.Helpers.RateLimit
+{
+    internal class ResultClassWithRetryAfter : ResultClass
+    {
+        public TimeSpan RetryAfter { get; }
+
+        public ResultClassWithRetryAfter(ResultPrimitive result)
+        : base(result)
+        {
+            RetryAfter = TimeSpan.Zero;
+        }
+
+        public ResultClassWithRetryAfter(TimeSpan retryAfter)
+        : base(ResultPrimitive.Undefined)
+        {
+            RetryAfter = retryAfter;
+        }
+    }
+}

src/Polly.Specs/RateLimit/AsyncRateLimitPolicySpecs.cs

@@ -0,0 +1,49 @@
+using System;
+using System.Threading.Tasks;
+using Polly.RateLimit;
+using Polly.Specs.Helpers;
+using Polly.Specs.Helpers.RateLimit;
+using Polly.Utilities;
+using Xunit;
+
+namespace Polly.Specs.RateLimit
+{
+    [Collection(Polly.Specs.Helpers.Constants.SystemClockDependentTestCollection)]
+    public class AsyncRateLimitPolicySpecs : RateLimitPolicySpecsBase, IDisposable
+    {
+        public void Dispose()
+        {
+            SystemClock.Reset();
+        }
+
+        protected override IRateLimitPolicy GetPolicyViaSyntax(int numberOfExecutions, TimeSpan perTimeSpan)
+        {
+            return Policy.RateLimitAsync(numberOfExecutions, perTimeSpan);
+        }
+
+        protected override IRateLimitPolicy GetPolicyViaSyntax(int numberOfExecutions, TimeSpan perTimeSpan, int maxBurst)
+        {
+            return Policy.RateLimitAsync(numberOfExecutions, perTimeSpan, maxBurst);
+        }
+
+        protected override (bool, TimeSpan) TryExecuteThroughPolicy(IRateLimitPolicy policy)
+        {
+            if (policy is AsyncRateLimitPolicy typedPolicy)
+            {
+                try
+                {
+                    typedPolicy.ExecuteAsync(() => Task.FromResult(new ResultClassWithRetryAfter(ResultPrimitive.Good))).GetAwaiter().GetResult();
+                    return (true, TimeSpan.Zero);
+                }
+                catch (RateLimitRejectedException e)
+                {
+                    return (false, e.RetryAfter);
+                }
+            }
+            else
+            {
+                throw new InvalidOperationException("Unexpected policy type in test construction.");
+            }
+        }
+    }
+}

src/Polly.Specs/RateLimit/AsyncRateLimitPolicyTResultSpecs.cs

@@ -0,0 +1,67 @@
+using System;
+using System.Threading.Tasks;
+using Polly.RateLimit;
+using Polly.Specs.Helpers;
+using Polly.Specs.Helpers.RateLimit;
+using Polly.Utilities;
+using Xunit;
+
+namespace Polly.Specs.RateLimit
+{
+    [Collection(Polly.Specs.Helpers.Constants.SystemClockDependentTestCollection)]
+    public class AsyncRateLimitPolicyTResultSpecs : RateLimitPolicyTResultSpecsBase, IDisposable
+    {
+        public void Dispose()
+        {
+            SystemClock.Reset();
+        }
+
+        protected override IRateLimitPolicy GetPolicyViaSyntax(int numberOfExecutions, TimeSpan perTimeSpan)
+        {
+            return Policy.RateLimitAsync<ResultClassWithRetryAfter>(numberOfExecutions, perTimeSpan);
+        }
+
+        protected override IRateLimitPolicy GetPolicyViaSyntax(int numberOfExecutions, TimeSpan perTimeSpan, int maxBurst)
+        {
+            return Policy.RateLimitAsync<ResultClassWithRetryAfter>(numberOfExecutions, perTimeSpan, maxBurst);
+        }
+
+        protected override IRateLimitPolicy<TResult> GetPolicyViaSyntax<TResult>(int numberOfExecutions, TimeSpan perTimeSpan, int maxBurst,
+            Func<TimeSpan, Context, TResult> retryAfterFactory)
+        {
+            return Policy.RateLimitAsync<TResult>(numberOfExecutions, perTimeSpan, maxBurst, retryAfterFactory);
+        }
+
+        protected override (bool, TimeSpan) TryExecuteThroughPolicy(IRateLimitPolicy policy)
+        {
+            if (policy is AsyncRateLimitPolicy<ResultClassWithRetryAfter> typedPolicy)
+            {
+                try
+                {
+                    typedPolicy.ExecuteAsync(() => Task.FromResult(new ResultClassWithRetryAfter(ResultPrimitive.Good))).GetAwaiter().GetResult();
+                    return (true, TimeSpan.Zero);
+                }
+                catch (RateLimitRejectedException e)
+                {
+                    return (false, e.RetryAfter);
+                }
+            }
+            else
+            {
+                throw new InvalidOperationException("Unexpected policy type in test construction.");
+            }
+        }
+
+        protected override TResult TryExecuteThroughPolicy<TResult>(IRateLimitPolicy<TResult> policy, Context context, TResult resultIfExecutionPermitted)
+        {
+            if (policy is AsyncRateLimitPolicy<TResult> typedPolicy)
+            {
+                return typedPolicy.ExecuteAsync(ctx => Task.FromResult(resultIfExecutionPermitted), context).GetAwaiter().GetResult();
+            }
+            else
+            {
+                throw new InvalidOperationException("Unexpected policy type in test construction.");
+            }
+        }
+    }
+}

src/Polly.Specs/RateLimit/LockBasedTokenBucketRateLimiterTests.cs

@@ -0,0 +1,11 @@
+using System;
+using Polly.RateLimit;
+
+namespace Polly.Specs.RateLimit
+{
+    public class LockBasedTokenBucketRateLimiterTests : TokenBucketRateLimiterTestsBase
+    {
+        internal override IRateLimiter GetRateLimiter(TimeSpan onePer, long bucketCapacity)
+            => new LockBasedTokenBucketRateLimiter(onePer, bucketCapacity);
+    }
+}

src/Polly.Specs/RateLimit/LockFreeTokenBucketRateLimiterTests.cs

@@ -0,0 +1,11 @@
+using System;
+using Polly.RateLimit;
+
+namespace Polly.Specs.RateLimit
+{
+    public class LockFreeTokenBucketRateLimiterTests : TokenBucketRateLimiterTestsBase
+    {
+        internal override IRateLimiter GetRateLimiter(TimeSpan onePer, long bucketCapacity)
+            => new LockFreeTokenBucketRateLimiter(onePer, bucketCapacity);
+    }
+}

src/Polly.Specs/RateLimit/RateLimitPolicySpecs.cs

@@ -0,0 +1,48 @@
+using System;
+using Polly.RateLimit;
+using Polly.Specs.Helpers;
+using Polly.Specs.Helpers.RateLimit;
+using Polly.Utilities;
+using Xunit;
+
+namespace Polly.Specs.RateLimit
+{
+    [Collection(Polly.Specs.Helpers.Constants.SystemClockDependentTestCollection)]
+    public class RateLimitPolicySpecs : RateLimitPolicySpecsBase, IDisposable
+    {
+        public void Dispose()
+        {
+            SystemClock.Reset();
+        }
+
+        protected override IRateLimitPolicy GetPolicyViaSyntax(int numberOfExecutions, TimeSpan perTimeSpan)
+        {
+            return Policy.RateLimit(numberOfExecutions, perTimeSpan);
+        }
+
+        protected override IRateLimitPolicy GetPolicyViaSyntax(int numberOfExecutions, TimeSpan perTimeSpan, int maxBurst)
+        {
+            return Policy.RateLimit(numberOfExecutions, perTimeSpan, maxBurst);
+        }
+
+        protected override (bool, TimeSpan) TryExecuteThroughPolicy(IRateLimitPolicy policy)
+        {
+            if (policy is RateLimitPolicy typedPolicy)
+            {
+                try
+                {
+                    typedPolicy.Execute(() => new ResultClassWithRetryAfter(ResultPrimitive.Good));
+                    return (true, TimeSpan.Zero);
+                }
+                catch (RateLimitRejectedException e)
+                {
+                    return (false, e.RetryAfter);
+                }
+            }
+            else
+            {
+                throw new InvalidOperationException("Unexpected policy type in test construction.");
+            }
+        }
+    }
+}