Skip to content

Conversation

@BourgoisMickael
Copy link
Contributor

  • improve some logs
  • warn early for event listener leaks
  • avoid send a remove to zk when a watcher notify the removed bucket config
  • separate list and get configs

Below a sequence diagram showing in red a few problems fixed here

bucket-notif-zookeeper drawio

Fix `key: { type: 'Buffer', data: [ 104, 101, 108, 108, 111 ] }`
To `key: 'hello'`
The watcher NODE_DELETED already calls removeConfig with false
Because it's already removed from zk if the event is received
There is no need to send a request to zk
This kind of error can happen if data is undefined
when the read is triggered too late when the znode
is already removed
@bert-e
Copy link
Contributor

bert-e commented Nov 14, 2025

Hello bourgoismickael,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request TBA
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@codecov
Copy link

codecov bot commented Nov 14, 2025

Codecov Report

❌ Patch coverage is 84.61538% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.04%. Comparing base (354107d) to head (694086a).

Files with missing lines Patch % Lines
...tification/configManager/ZookeeperConfigManager.js 84.61% 4 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

Files with missing lines Coverage Δ
...ensions/notification/NotificationQueuePopulator.js 98.13% <ø> (ø)
lib/BackbeatConsumer.js 92.64% <ø> (+0.27%) ⬆️
...tification/configManager/ZookeeperConfigManager.js 83.16% <84.61%> (-0.35%) ⬇️

... and 3 files with indirect coverage changes

Components Coverage Δ
Bucket Notification 80.30% <84.61%> (-0.04%) ⬇️
Core Library 80.32% <ø> (-0.56%) ⬇️
Ingestion 70.28% <ø> (ø)
Lifecycle 78.67% <ø> (ø)
Oplog Populator 85.06% <ø> (ø)
Replication 58.55% <ø> (-0.04%) ⬇️
Bucket Scanner 85.76% <ø> (ø)
@@                 Coverage Diff                 @@
##           development/9.0    #2687      +/-   ##
===================================================
- Coverage            74.27%   74.04%   -0.24%     
===================================================
  Files                  201      201              
  Lines                13444    13458      +14     
===================================================
- Hits                  9986     9965      -21     
- Misses                3448     3483      +35     
  Partials                10       10              
Flag Coverage Δ
api:retry 9.45% <0.00%> (-0.02%) ⬇️
api:routes 9.26% <0.00%> (-0.01%) ⬇️
bucket-scanner 85.76% <ø> (ø)
ft_test:queuepopulator 8.91% <0.00%> (-1.30%) ⬇️
ingestion 12.51% <0.00%> (-0.02%) ⬇️
lib 7.47% <0.00%> (-0.01%) ⬇️
lifecycle 18.67% <0.00%> (-0.02%) ⬇️
notification 1.04% <0.00%> (-0.01%) ⬇️
replication 18.47% <0.00%> (-0.02%) ⬇️
unit 50.03% <84.61%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bert-e
Copy link
Contributor

bert-e commented Nov 14, 2025

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

@scality scality deleted a comment from bert-e Nov 14, 2025
Copilot finished reviewing on behalf of BourgoisMickael November 14, 2025 18:12
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR addresses event listener leaks in the ZookeeperConfigManager by improving watcher management and separating list/get config operations. The key improvements include early detection of watcher leaks, preventing redundant ZooKeeper remove operations when watchers notify of deleted bucket configs, and enhanced logging for better debugging.

  • Added early warning system for ZooKeeper watcher leaks to detect issues before hitting EventEmitter limits
  • Separated getConfig and listConfigs event handlers to avoid unnecessary bucket list operations
  • Modified removeConfig to optionally skip ZooKeeper emission when triggered by a watcher notification

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
lib/BackbeatConsumer.js Enhanced logging to safely convert Buffer keys to strings for better debugging
extensions/notification/configManager/ZookeeperConfigManager.js Core fixes for watcher leak detection, separated list/get config handlers, and added conditional ZK emission for removeConfig
extensions/notification/NotificationQueuePopulator.js Corrected typo in comment from "deleter" to "deleted"

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Otherwise we need a few buckets and bucket operations
to trigger the event emitter listener limit warning at 10 listeners
The getConfig should not trigger a listing with watcher
as it duplicates an already existing watcher.
This can have an exponential effect of adding even more watchers
when events are triggered.

A clear separation prevent this effect:
- only NODE_CHILDREN_CHANGED should trigger a new listing to reapply a watcher
- getConfig should be contained to a single bucket (discovered by listing, or from watcher NODE_DATA_CHANGED)
@BourgoisMickael BourgoisMickael force-pushed the improvement/BB-727-zk-config branch from 62f6510 to dc1fe12 Compare November 14, 2025 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants