Skip to content

Measure upload performance on S3 #7829

@arielshaqed

Description

@arielshaqed

Our PathPartitionProvider picks names for new uploaded objects. These go into (descending...) partition prefixes that are periodically replaced.

Measure how such a scheme interacts with S3 access performance. Measure how many uploads/s we can get from 1, 3, and possibly more concurrent PathPartitionProviders. Do we get slowdowns from S3?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions