Skip to content

Updated Default Concurrency count #54428

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nickliu-msft wants to merge 5 commits into
base: feature/storage/stg101base
Choose a base branch
from
Open

Updated Default Concurrency count #54428

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
837e65c
Updated default concurrency to processor count + compat switch logic
nickliu-msft Dec 10, 2025
449d18e
updated default concurrency
nickliu-msft Dec 11, 2025
374bc66
updated changelogs
nickliu-msft Dec 11, 2025
330d533
slight revision
nickliu-msft Dec 11, 2025
14889a1
More descriptive changelogs
nickliu-msft Dec 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Updated default concurrency to processor count + compat switch logic
  • Loading branch information
@nickliu-msft
nickliu-msft committed Dec 10, 2025
commit 837e65c02e3d55d666669352a6ccab5b593e8fae
1 change: 1 addition & 0 deletions sdk/storage/Azure.Storage.Blobs/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- Added support for Server-side Encryption Rekeying.
- Added cross-tenant support for Principal-Bound User Delegation SAS.
- Added support for Dynamic User Delegation SAS.
- Updated default concurrency transfer count to be the number of processors on the machine.

### Bugs Fixed
- Added BlobErrorCode.IncrementalCopyOfEarlierSnapshotNotAllowed, deprecated BlobErrorCode.IncrementalCopyOfEarlierVersionSnapshotNotAllowed.
Expand Down
4 changes: 3 additions & 1 deletion sdk/storage/Azure.Storage.Blobs/src/PartitionedDownloader.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,9 @@ public PartitionedDownloader(
}
else
{
_maxWorkerCount = Constants.Blob.Block.DefaultConcurrentTransfersCount;
_maxWorkerCount = CompatSwitches.UseLegacyDefaultConcurrency
? Constants.Blob.Block.LegacyDefaultConcurrentTransfersCount
: Environment.ProcessorCount;
}

// Set _rangeSize
Expand Down
1 change: 1 addition & 0 deletions sdk/storage/Azure.Storage.Common/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

### Features Added
- This release contains bug fixes to improve quality.
- Updated default concurrency transfer count to be the number of processors on the machine.

## 12.26.0-beta.1 (2025-11-17)

Expand Down
5 changes: 5 additions & 0 deletions sdk/storage/Azure.Storage.Common/src/Shared/CompatSwitches.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,10 @@ internal static class CompatSwitches

public static bool DisableExpectContinueHeader => _disableExpectContinueHeader
??= AppContextSwitchHelper.GetConfigValue(Constants.DisableExpectContinueHeaderSwitchName, Constants.DisableExpectContinueHeaderEnvVar);

private static bool? _useLegacyDefaultConcurrency;

public static bool UseLegacyDefaultConcurrency => _useLegacyDefaultConcurrency
??= AppContextSwitchHelper.GetConfigValue(Constants.UseLegacyDefaultConcurrencySwitchName, Constants.UseLegacyDefaultConcurrencyEnvVar);
}
}
16 changes: 14 additions & 2 deletions sdk/storage/Azure.Storage.Common/src/Shared/Constants.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

using System;
using System.Collections.Generic;
using System.ComponentModel;

namespace Azure.Storage
{
Expand Down Expand Up @@ -129,6 +130,9 @@ internal static class Constants
public const string DisableExpectContinueHeaderSwitchName = "Azure.Storage.DisableExpectContinueHeader";
public const string DisableExpectContinueHeaderEnvVar = "AZURE_STORAGE_DISABLE_EXPECT_CONTINUE_HEADER";

public const string UseLegacyDefaultConcurrencySwitchName = "Azure.Storage.UseLegacyDefaultConcurrency";
public const string UseLegacyDefaultConcurrencyEnvVar = "AZURE_STORAGE_USE_LEGACY_DEFAULT_CONCURRENCY";

public const string DefaultScope = "/.default";

/// <summary>
Expand Down Expand Up @@ -241,7 +245,9 @@ internal static class Append

internal static class Block
{
public const int DefaultConcurrentTransfersCount = 5;
[EditorBrowsable(EditorBrowsableState.Never)]
public const int DefaultConcurrentTransfersCount = LegacyDefaultConcurrentTransfersCount;
Copy link
Member

@jaschrep-msft jaschrep-msft Dec 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get that this is to avoid altering a ton of files but i think it's probably better for code health to do that. We don't want to be confused about whether we're using old or new values. We want to know it's legacy when we see it. I'd rather this const just not exist.

Copy link
Member Author

@nickliu-msft nickliu-msft Dec 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So Constants.DefaultConcurrentTransfersCount is no longer being referenced at all, we are replacing all usage with the new Constants.LegacyDefaultConcurrentTransfersCount. The reason why we hid it rather than removed it is to avoid a breaking change

public const int LegacyDefaultConcurrentTransfersCount = 5;
public const int DefaultInitalDownloadRangeSize = 256 * Constants.MB; // 256 MB
public const int Pre_2019_12_12_MaxUploadBytes = 256 * Constants.MB; // 256 MB
public const long MaxUploadBytes = 5000L * Constants.MB; // 5000MB
Expand Down Expand Up @@ -378,7 +384,13 @@ internal static class DataLake
/// <summary>
/// Default concurrent transfers count.
/// </summary>
public const int DefaultConcurrentTransfersCount = 5;
[EditorBrowsable(EditorBrowsableState.Never)]
public const int DefaultConcurrentTransfersCount = LegacyDefaultConcurrentTransfersCount;

/// <summary>
/// Legacy default concurrent transfers count.
/// </summary>
public const int LegacyDefaultConcurrentTransfersCount = 5;

/// <summary>
/// Max upload bytes for less than Service Version 2019-12-12.
Expand Down
4 changes: 3 additions & 1 deletion sdk/storage/Azure.Storage.Common/src/Shared/PartitionedUploader.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,9 @@ public PartitionedUploader(
}
else
{
_maxWorkerCount = Constants.Blob.Block.DefaultConcurrentTransfersCount;
_maxWorkerCount = CompatSwitches.UseLegacyDefaultConcurrency
? Constants.Blob.Block.LegacyDefaultConcurrentTransfersCount
: Environment.ProcessorCount;
}

// Set _singleUploadThreshold
Expand Down
Loading