Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.
Merged
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
34577eb
service: storage monitor added
michalkucharczyk Jan 5, 2023
b5f970b
Merge remote-tracking branch 'origin/master' into mku-storage-monitor
michalkucharczyk Jan 6, 2023
37bbba4
Cargo.lock updated
michalkucharczyk Jan 6, 2023
ba0f28f
misspell
michalkucharczyk Jan 9, 2023
e360685
fs events throttling added
michalkucharczyk Jan 9, 2023
fd5679f
minor updates
michalkucharczyk Jan 9, 2023
217234f
filter out non mutating events
michalkucharczyk Jan 9, 2023
8b3310b
misspell
michalkucharczyk Jan 9, 2023
931339c
".git/.scripts/commands/fmt/fmt.sh"
Jan 10, 2023
c200159
Update client/service/src/storage_monitor.rs
michalkucharczyk Jan 12, 2023
f5501f2
storage-monitor crate added
michalkucharczyk Jan 13, 2023
45171dc
cleanup: configuration + service builder
michalkucharczyk Jan 13, 2023
f346394
storage_monitor in custom service (wip)
michalkucharczyk Jan 13, 2023
eb49400
copy-paste bad desc fixed
michalkucharczyk Jan 13, 2023
8619766
notify removed
michalkucharczyk Jan 16, 2023
a0cd44d
storage_monitor added to node
michalkucharczyk Jan 16, 2023
d3f143c
Merge remote-tracking branch 'origin/master' into mku-storage-monitor
Jan 16, 2023
a79af9a
fix for clippy
michalkucharczyk Jan 16, 2023
00e7264
publish = false
michalkucharczyk Jan 16, 2023
6c3eb1c
Update bin/node/cli/src/command.rs
michalkucharczyk Jan 17, 2023
58dcf55
Apply suggestions from code review
michalkucharczyk Jan 23, 2023
95b576b
crate name: storage-monitor -> sc-storage-monitor
michalkucharczyk Jan 23, 2023
0b8697e
error handling improved
michalkucharczyk Jan 23, 2023
c0fc2ca
Merge remote-tracking branch 'origin/master' into mku-storage-monitor
michalkucharczyk Jan 23, 2023
de35dda
Apply suggestions from code review
michalkucharczyk Jan 23, 2023
4e32f3d
publish=false removed
michalkucharczyk Jan 24, 2023
0578588
Merge remote-tracking branch 'origin/master' into mku-storage-monitor
Jan 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fs events throttling added
  • Loading branch information
michalkucharczyk committed Jan 9, 2023
commit e36068584c6f365c11d995c0935e5727e888b86d
20 changes: 15 additions & 5 deletions client/service/src/storage_monitor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,13 @@ use futures::StreamExt;
use nix::{errno::Errno, sys::statvfs::statvfs};
use notify::{Config, Event, RecommendedWatcher, RecursiveMode, Watcher};
use sc_utils::mpsc::{tracing_unbounded, TracingUnboundedReceiver};
use std::path::{Path, PathBuf};
use std::{
path::{Path, PathBuf},
time::{Duration, Instant},
};

const LOG_TARGET: &str = "storage-monitor";
const THROTTLE_PERIOD: std::time::Duration = Duration::from_secs(2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also wondering how did you end up choosing 2 sec? why not 5 or 10s? or even 60s? do you know the avg disk space growth rate in polkadot?

Copy link
Contributor Author

@michalkucharczyk michalkucharczyk Jan 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a guess. The syscall is cheap, should not affect the system. I did not measure the usage by node, but there also can be other consumers of storage running on the machine.

I also added a param, so anyone can adjust if required.


pub struct StorageMonitorService {
/// watched path
Expand All @@ -32,6 +36,8 @@ pub struct StorageMonitorService {
stream: TracingUnboundedReceiver<Result<Event, notify::Error>>,
/// number of bytes that shall be free and available on the filesystem for watched path
threshold: u64,
/// timestamp of the most recent check
recent_check: Instant,
/// keeps the ref for file system watcher
_watcher: RecommendedWatcher,
}
Expand Down Expand Up @@ -81,6 +87,7 @@ impl StorageMonitorService {
path: path.to_path_buf(),
stream,
threshold,
recent_check: Instant::now(),
_watcher: watcher,
})
},
Expand All @@ -92,8 +99,11 @@ impl StorageMonitorService {
while let Some(watch_event) = self.stream.next().await {
match watch_event {
Ok(_) =>
if Self::check_free_space(&self.path, self.threshold).is_err() {
break
if self.recent_check.elapsed() >= THROTTLE_PERIOD {
self.recent_check = Instant::now();
if Self::check_free_space(&self.path, self.threshold).is_err() {
break
}
},
Err(e) => {
log::error!(target: LOG_TARGET, "watch error: {:?}", e);
Expand All @@ -108,8 +118,8 @@ impl StorageMonitorService {
fs_stats.map(|stats| stats.blocks_available() * stats.block_size() / 1_000_000)
}

// checks if the free space for given `path` is below given `threshold`.
// If not error is returned.
// Checks if the amount of free space for given `path` is below given `threshold`.
// If it dropped below, error is returned.
// System errors are silently ignored.
fn check_free_space(path: &Path, threshold: u64) -> Result<(), Error> {
match StorageMonitorService::free_space(path.as_ref()) {
Expand Down