-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Fix: Enable throttling parameters in BackupRepository repositoryConfig #9351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
469a1ec to
ff9e49c
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9351 +/- ##
==========================================
+ Coverage 59.67% 60.24% +0.57%
==========================================
Files 383 386 +3
Lines 44050 35942 -8108
==========================================
- Hits 26286 21653 -4633
+ Misses 16218 12715 -3503
- Partials 1546 1574 +28 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
ff9e49c to
5e643a4
Compare
The `uploadBytes`, `downloadBytes`, and other throttling parameters specified in BackupRepository's `repositoryConfig` were being silently filtered out and not applied to Kopia repositories. This occurred because these parameters were missing from the whitelist in `getStorageVariables()`.
**Code:**
- Added throttling options to the valid parameters whitelist in `pkg/repository/provider/unified_repo.go`:
- `uploadBytes` - upload bandwidth throttling (bytes/sec)
- `downloadBytes` - download bandwidth throttling (bytes/sec)
- `readOPS` - read operations throttling (ops/sec)
- `writeOPS` - write operations throttling (ops/sec)
- `listOPS` - list operations throttling (ops/sec)
**Documentation:**
- Updated `site/content/docs/main/backup-repository-configuration.md` with throttling parameters descriptions and examples
**Tests:**
- Added test case in `pkg/repository/provider/unified_repo_test.go` to verify throttling parameters are properly passed through
```yaml
apiVersion: velero.io/v1
kind: BackupRepository
spec:
repositoryConfig:
uploadBytes: "838860800" # ~800 MB/s
downloadBytes: "838860800" # ~800 MB/s
```
These parameters are now properly applied to Kopia during file system backups performed by node-agent.
Signed-off-by: Andrei Kvapil <[email protected]>
| ThrottleOptionListOps = "listOPS" | ||
| ThrottleOptionUploadBytes = "uploadBytes" | ||
| ThrottleOptionDownloadBytes = "downloadBytes" | ||
| ThrottleOptionReadOps = "ThrottleReadOPS" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for misleading of my previous comment. We add throttle for each of the options, but as the convention, we need to make the first word lowercase, like:
ThrottleOptionReadOps = "throttleReadOPS"
ThrottleOptionWriteOps = "throttleWriteOPS"
ThrottleOptionListOps = "throttleListOPS"
ThrottleOptionUploadBytes = "throttleUploadBytes"
ThrottleOptionDownloadBytes = "throttleDownloadBytes"
The throttling parameters specified in BackupRepository's
repositoryConfigwere being silently filtered out and not applied to Kopia repositories. This occurred because these parameters were missing from the whitelist ingetStorageVariables().Code:
Added throttling options to the valid parameters whitelist in
pkg/repository/provider/unified_repo.go:ThrottleUploadBytes- upload bandwidth throttling (bytes/sec)ThrottleDownloadBytes- download bandwidth throttling (bytes/sec)ThrottleReadOPS- read operations throttling (ops/sec)ThrottleWriteOPS- write operations throttling (ops/sec)ThrottleListOPS- list operations throttling (ops/sec)Updated parameter names in
pkg/repository/udmrepo/repo_options.goto use "Throttle" prefix for better clarity:readOPS→ThrottleReadOPSwriteOPS→ThrottleWriteOPSlistOPS→ThrottleListOPSuploadBytes→ThrottleUploadBytesdownloadBytes→ThrottleDownloadBytesDocumentation:
site/content/docs/main/backup-repository-configuration.mdwith throttling parameters descriptions and examples using the new parameter namesTests:
pkg/repository/provider/unified_repo_test.goto verify throttling parameters are properly passed throughExample Usage:
These parameters are now properly applied to Kopia during file system backups performed by node-agent.
Signed-off-by: Andrei Kvapil [email protected]