-
-
Notifications
You must be signed in to change notification settings - Fork 4.7k
feat(objectstore): add configurable S3 retry attempts #56502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
/backport to stable31 |
|
/backport to stable32 |
| // optional: Maximum number of retry attempts for failed S3 requests | ||
| // Default: 5 | ||
| 'retriesMaxAttempts' => 5, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we add that to the existing entry line 1905?
server/config/config.sample.php
Lines 1905 to 1936 in f2f9cb7
| 'objectstore' => [ | |
| 'class' => 'OC\\Files\\ObjectStore\\Swift', | |
| 'arguments' => [ | |
| // trystack will use your Facebook ID as the username | |
| 'username' => 'facebook100000123456789', | |
| // in the trystack dashboard, go to user -> settings -> API Password to | |
| // generate a password | |
| 'password' => 'Secr3tPaSSWoRdt7', | |
| // must already exist in the objectstore, name can be different | |
| 'container' => 'nextcloud', | |
| // prefix to prepend to the fileid, default is 'oid:urn:' | |
| 'objectPrefix' => 'oid:urn:', | |
| // create the container if it does not exist. default is false | |
| 'autocreate' => true, | |
| // required, dev-/trystack defaults to 'RegionOne' | |
| 'region' => 'RegionOne', | |
| // The Identity / Keystone endpoint | |
| 'url' => 'http://8.21.28.222:5000/v2.0', | |
| // uploadPartSize: size of the uploaded chunks, defaults to 524288000 | |
| 'uploadPartSize' => 524288000, | |
| // required on dev-/trystack | |
| 'tenantName' => 'facebook100000123456789', | |
| // dev-/trystack uses swift by default, the lib defaults to 'cloudFiles' | |
| // if omitted | |
| 'serviceName' => 'swift', | |
| // The Interface / URL Type, optional | |
| 'urlType' => 'internal', | |
| // Maximum amount of data that can be uploaded | |
| 'totalSizeLimit' => 1024 * 1024 * 1024, | |
| ], | |
| ], | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But that is for swift not s3?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, weird that we have an entry for swift only.
We should also add some documentation here then: https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/primary_storage.html#simple-storage-service-s3
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's keep your change then :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to document all supported options here too then: for instance totalSizeLimit, objectPrefix, use_path_style, concurrency, proxy, connect_timeout, timeout, uploadPartSize, putSizeLimit, useMultipartCopy, copySizeLimit, legacy_auth, version, verify_bucket_exists, autocreate, storageClass (and there's even more in S3ConnectionTrait).
Maybe just a bunch of common ones?
Some of there are not in the documentation as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I've overlooked the comment 🙈
Sounds good to me 👍
Add retriesMaxAttempts parameter to S3 objectstore configuration to allow customization of AWS SDK retry behavior for handling unreliable network conditions or proxy issues. Defaults to 5 retries (AWS SDK default) if not specified. Signed-off-by: nfebe <[email protected]>
1274580 to
3030783
Compare
kesselb
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you 🙏
Add retriesMaxAttempts parameter to S3 objectstore configuration to allow customization of AWS SDK retry behavior for handling unreliable network conditions or proxy issues.
Defaults to 5 retries (AWS SDK default) if not specified.