An/queue based scheduling #1701
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently Amigo will rebake recipes based on a Quartz cron schedule. Unfortunately, bakes can fail or timeout if too many are running at once, which means that users sometimes need to hunt for a free slot, or adjust the scheduling of many recipes to get them all to play nicely. We've had similar problems in the Grid when people try to upload many images all in one go. The architectural solution we've implemented there is to queue jobs, allowing the app to restrict its concurrency and limit the number of jobs running at any one time. Once we're doing this though, there seems little need for the precision of a Cron expression - we always schedule weekly bakes, so besides the choice of day do we really care exactly what hour/minute it starts? Instead, only prompt the user to select a day, and then shortly after midnight, have Amigo find and schedule the bakes for that day's recipes.
Contributor
andrew-nowak
force-pushed
the
an/queue-based-scheduling
branch
from
6f6d6f4 to
74d51a5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reopening of #1670
Currently Amigo will rebake recipes based on a Quartz cron schedule. Unfortunately, bakes can fail or timeout if too many are running at once, which means that users sometimes need to hunt for a free slot, or adjust the scheduling of many recipes to get them all to play nicely.
We've had similar problems in the Grid when people try to upload many images all in one go. The architectural solution we've implemented there is to queue jobs, allowing the app to restrict its concurrency and limit the number of jobs running at any one time.
Once we're doing this though, there seems little need for the precision of a Cron expression - we always schedule weekly bakes, so besides the choice of day do we really care exactly what hour/minute it starts? Instead, only prompt the user to select a day, and then shortly after midnight, have Amigo find and schedule the bakes for that day's recipes.
What does this change?
How to test
What is the value of this?
Have we considered potential risks?