Skip to content

Conversation

@juliusknorr
Copy link
Member

@juliusknorr juliusknorr commented Aug 20, 2021

Follow up to #27877 to further address the comments from @kesselb

Could we use a higher value for single put operations? Like 25 or 50 MB. I understand the buffer uses the memory hence we should not pick a value to high. However most pictures have more than 5 MB nowadays.

100 MB was picked as the default, as per AWS S3 recommendation to make use of multipart upload for files larger than that:

In general, when your object size reaches 100 MB, you should consider using multipart uploads instead of uploading the object in a single operation.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html

Memory overhead is the actual file size which would fit our recommended php memory limit of 512 MB still fairly easy.

Base automatically changed from enh/s3-putObject to master August 20, 2021 16:54
@juliusknorr juliusknorr marked this pull request as ready for review December 29, 2021 11:06
@juliusknorr juliusknorr added 3. to review Waiting for reviews enhancement labels Dec 29, 2021
@juliusknorr juliusknorr added this to the Nextcloud 24 milestone Dec 29, 2021
use Aws\Credentials\Credentials;
use Aws\Exception\CredentialsException;
use Aws\S3\Exception\S3Exception;
use Aws\S3\MultipartUploader;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused, it seems

Copy link
Contributor

@artonge artonge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some php-cs warnings, else, looks good.

@juliusknorr
Copy link
Member Author

Failure unrelated

@juliusknorr juliusknorr merged commit de2b86a into master Dec 29, 2021
@juliusknorr juliusknorr deleted the enh/s3-put-limit branch December 29, 2021 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3. to review Waiting for reviews enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants