v2.3

v2.3

This release introduces a significant upgrade in automated testing coverage, improving both back-end and front-end reliability. The goal is to ensure that every new feature or modification in Watcher is backed by robust, automated unit tests. This update directly strengthens the stability and maintainability of the project across all modules.

Update Procedure

No breaking change or configuration is required for this release. However, contributors must ensure they follow the updated test commands and conventions detailed in the updated documentation.

What’s Changed

Test Coverage

Unit Tests (Back-End)

• 99 Django unit tests across the main back-end modules:

  • common/tests.py
  • watcher/tests.py
  • Individual tests.py files per module

End-to-End Tests (Front-End with Cypress)

• 4 full-featured Cypress suites covering the entire front-end application:
  • DataLeak.cy.js: 31 tests
  • DnsFinder.cy.js: 32 tests
  • SiteMonitoring.cy.js: 26 tests
  • ThreatsWatcher.cy.js: 31 tests
• Total: 120 Cypress tests successfully passed with no failures or pending cases.

CI/CD Integration

• All tests are automatically executed in our CI/CD pipeline using GitHub Actions:

  • Triggered on: Push, Pull Requests, and manual workflow dispatch
  • Execution: Both back-end and front-end tests run automatically
  • Coverage: Full test suite validation before code integration

• The CI/CD workflow ensures that:

  • No broken code reaches the main branch
  • All new features are properly tested

Developer Notes

All test commands must now be run from the Watcher/Watcher directory:

cd Watcher/Watcher

Back-End Tests

To run all Django unit tests:

python manage.py test

Front-End Tests

Before running front-end tests, you need to create a test superuser:

python manage.py shell -c "
from django.contrib.auth.models import User
User.objects.create_superuser('Watcher', 'cypress@watcher.com', 'Watcher', first_name='Unit-Test Cypress', last_name='Watcher')"

To run all Cypress Test Runner:

npm run test:e2e

If you want more information about the commands and other instructions, please refer to the documentation.

Why it matters

This update lays the foundation for a more robust and scalable Watcher ecosystem. By enforcing test-driven development and automated validation, we ensure faster releases, fewer regressions, and a cleaner development experience for contributors.

From now on: All Pull Requests must include tests for new functionality. PRs without adequate test coverage may be rejected

Full Changelog: v2.2.0...v2.3

v2.2.0

v2.2.0

This release focuses on a complete overhaul of the MISP integration, improved code modularity, and the resolution of several front-end and dependency issues. It also addresses important issues such as #2 (creating MISP objects instead of attributes) and #206 (HTTP 500 errors when exporting IOC), improving overall export reliability and functionality. These updates aim to increase stability, maintainability, and flexibility when using Watcher with MISP.

Update Procedure

Please follow this process :

1. Pull the latest Docker image from the repository.
2. Apply any migrations:

   docker compose down
   docker compose run watcher bash
   python manage.py migrate

3. Update your .env file:
   • Check MISP_URL, MISP_KEY, and MISP_VERIFY_SSL.
4. Rebuild and restart containers with :

   docker compose down
   docker compose up

[WARNING] If you were using custom scripts for MISP export, you will need to adapt them to the new common.misp module.

What’s Changed

MISP Integration Redesign

• New centralized MISP logic: The common module now handles all MISP-related interactions by @ygalnezri in #207
• Shared MISP module across all features (e.g. dns_finder, website_monitoring), avoiding code duplication.
• Possibility to specify a MISP Event UUID when exporting IOCs:

  • If the UUID exists, Watcher will detect and update the corresponding event.

  • Full support for both automatic and manual updates.

  • Enhanced export logic for MISP objects with support for object creation (e.g., domain, ip-port).

    This redesign addresses key issues including #2 (creating MISP objects instead of attributes) and #206 (HTTP 500 error when exporting IOC in Twisted DNS finder).

For more details on the new MISP integration, the MISP Export section of the documentation has been fully updated: MISP Export

• Change your variable name based on the new .env format: the setting has been renamed from ALLOWED_HOST to ALLOWED_HOSTS. You must now explicitly define it in .env (e.g., ALLOWED_HOSTS=localhost,127.0.0.1) to match your environment's domain list. This update ensures proper host validation in settings.py and prevents runtime errors during container startup by @ygalnezri in #185

Dependency and Security Updates

• Fixed minor display issues detected during development
• Bump @babel/runtime from 7.26.0 to 7.27.0 in /Watcher by @dependabot #189
• Bump axios from 1.7.9 to 1.8.2 in /Watcher by @dependabot #190
• Bump react-router and react-router-dom in /Watcher by @dependabot #191

Full Changelog: v2.1.4...v2.2.0

v2.1.4

v2.1.4

This update focuses on improving stability in the site monitoring process and fixing environment configuration issues related to host validation.

Update Procedure

Please follow this process.

What’s Changed

• Change your variable name based on the new .env format: the setting has been renamed from ALLOWED_HOST to ALLOWED_HOSTS. You must now explicitly define it in .env (e.g., ALLOWED_HOSTS=localhost,127.0.0.1) to match your environment's domain list. This update ensures proper host validation in settings.py and prevents runtime errors during container startup by @ygalnezri in #185
• Resolved an issue in SiteSerializer and monitoring_init function, allowing for smooth execution of the site monitoring process without blocking subsequent domain additions by @ygalnezri in #185

Full Changelog: v2.1.3...v2.1.4

v2.1.3

v2.1.3

This update improves domain name validation and fixes issues related to regex validation for Domain Name & Ticket ID. Additionally, a bug in the search functionality for the Alert model in the DNS Finder and Data Leak modules has been resolved.

Update Procedure

Please follow this process.

What’s Changed

• Fixed regex validation for Domain Name & Ticket ID on the front-end, ensuring accurate input validation by @ygalnezri
• Improved domain name validation with tldextract, enhancing reliability in domain handling by @ygalnezri
• Fixed a bug in search functionality for the Alert model, resolving issues in the DNS Finder and Data Leak modules on the admin interface by @ygalnezri
• Refactored the site creation method in Site Monitoring module by @radomir-mijovic

New Contributors

• @radomir-mijovic made their first contribution in #179

Full Changelog: v2.1.2...v2.1.3

v2.1.2

v2.1.2

This update fixes the handling of parent domain actions in DNS Finder, improving tagging accuracy in TheHive. It also enhances TheHive integration by resolving an issue that prevented the creation of grouped alerts for DNS Finder. Additionally, RSS sources have been updated, with obsolete sources removed and new, relevant cybersecurity sources added. Lastly, a time-based verification mechanism has been introduced in Website Monitoring to prevent duplicate alerts from being generated unnecessarily.

Update Procedure

[WARNING] RSS Sources Update:

We have removed obsolete RSS sources and replaced them with new sources related to cybersecurity. To populate the new RSS sources, run the following command:

python manage.py populate_db

Refer to the updated documentation for details: Update Watcher.

What’s Changed

• Fixed incorrect action handling for parent domains in DNS Finder, improving tagging accuracy in TheHive by @ygalnezri
• Resolved the issue preventing group alerts from being created in TheHive for DNS Finder by @ygalnezri
• Updated RSS sources, adding relevant ones and removing obsolete ones by @ygalnezri
• Fixed duplicate alerts in Website Monitoring, adding a time interval check to prevent unnecessary duplicates by @ygalnezri
• Fixed domain identification logic in DNS Finder, ensuring proper handling of TLDs with multiple segments (e.g., second-level TLDs) by @ygalnezri
• v2.1.2 by @ygalnezri in #171

Full Changelog: v2.1.1...v2.1.2

v2.1.1

v2.1.1

This release improves TheHive integration, refining TLP, PAP, Severity, and observable tags, while enhancing alert and case management. The system now verifies existing entries before updating them with new observables, preventing duplicates. Additionally, Dockerfile optimizations enhance container performance and security.

Watcher now has the ability to automatically feed cases and alerts by adding DNS Finder alerts linked to a monitored domain in Website Monitoring. Watcher will add subdomains to the case or alert of the parent domain and automatically update it.

---

Update Procedure

Please follow this process.

• If you want, you can update the .env configuration to include credentials and endpoints for TheHive and other notification channels. Refer to the updated documentation for details: Update Watcher.

---

What’s Changed

• This release enhances TheHive integration, ensuring alerts and cases are automatically updated with new observables for better incident tracking by @ygalnezri.
  • Watcher now automatically feeds cases and alerts by adding DNS Finder alerts linked to a monitored domain in Website Monitoring. Subdomains are added to the case or alert of the parent domain, ensuring they are automatically updated.
• Refined TLP, PAP, Severity, and observable tags, improving classification and response accuracy by @ygalnezri.
• Optimized the Dockerfile following best practices, improving performance and security by @0xlildoudou in #163
• Bump django from 5.0.10 to 5.0.11 in /Watcher by @dependabot in #168

---

New Contributors

• @0xlildoudou made their first contribution in #163

---

Full Changelog: v2.1...v2.1.1

v2.1

v2.1

This release focuses on a major transformation of the notification system, introducing new integrations with TheHive, Citadel, and Slack, alongside an enhancement to email notifications. Users can now automate alert creation in TheHive, send notifications via Citadel's APIs, and streamline team communication through Slack. Additionally, email notifications have been upgraded to SMTPS for improved security. These updates aim to enhance communication, security, and collaboration across multiple platforms.

Update Procedure

[MANDATORY] Update Watcher:
This version includes significant changes to the notification system. It is essential to follow these steps:

1. Pull the latest Docker image from the repository.
2. Update your docker-compose.yml file as per the latest version on GitHub.
3. Apply migrations for the newly created module:

   python manage.py migrate

4. Update configurations in the .env file to include credentials and endpoints for the new notification channels. Refer to the updated documentation for details: Update Watcher.

New Features

• Common Module Implementation:
  A new Django app has been introduced to centralize generic functions shared by multiple modules. This update simplifies the codebase and enhances maintainability.

• Notification System Creation:

  • Email Notifications via SMTPS:
    Transitioned from SMTP to SMTPS for enhanced security. More details and information about this: Configure your Email notifications by @ygalnezri.
  • TheHive Integration:
    Automatic alert creation in TheHive via APIs. More details and information about this: Configure your TheHive notifications by @ygalnezri.
  • Citadel Integration:
    Notifications are now supported through the enterprise application Citadel via APIs. More details and information about this: Configure your Citadel notifications by @ygalnezri.
  • Slack Notifications:
    Notifications can now be delivered directly through Slack via APIs for better team collaboration. More details and information about this: Configure your Slack notifications by @ygalnezri.

What’s Changed

• Redesigned email templates for improved clarity and user experience by @ygalnezri.
• Refactored the docker-compose.yml file to use env_file for better readability and reduced redundancy by @ygalnezri.
• Updated the Dockerfile to align with the latest best practices for Django applications by @ygalnezri.
• Revised documentation to provide detailed setup instructions for the new notification system and its integrations by @ygalnezri
• v2.1 by @ygalnezri in #166

Full Changelog: v2.0.2...v2.1

v2.0.2

What's Changed

• Bump path-to-regexp and react-router-dom in /Watcher by @dependabot in #156
• Bump django from 5.0.8 to 5.0.9 in /Watcher by @dependabot in #158

Full Changelog: v2.0.1...v2.0.2

v2.0.1

What's Changed

• Bump django from 5.0.7 to 5.0.8 in /Watcher by @dependabot in #149
• Bump webpack from 5.76.0 to 5.94.0 in /Watcher by @dependabot in #153
• Bump axios from 1.7.2 to 1.7.4 in /Watcher by @dependabot in #154

Full Changelog: v2.0...v2.0.1

v2.0

This release aims to make the existing API easily usable, in order to facilitate communication with Watcher for other software. It also aims to correct several existing anomalies.

Update Procedure

[MANDATORY] Please follow this process:
This version includes breaking changes, so it is mandatory to follow this process: Update Watcher

[WARNING] RSS-Bridge Removal:
RSS-Bridge is not needed anymore. We used it to obtain RSS flow from X (Twitter). Due to changes in the pricing of the X API, this is no longer functional.
Therefore, we have removed the RSS-Bridge container and replaced it with 200+ new RSS sources related to cybersecurity. Make sure to populate the new RSS sources with the following command:

python manage.py populate_db

Please remove the RSS-Bridge container from your docker-compose.yml file with the latest version available on GitHub.
You can also delete all RSS sources associated with the old RSS-Bridge (They are not needed anymore). Here is an example of the formatting: http://10.10.10.7/?action=display&bridge=Twitter&context=By+username&u...

[WARNING] MySQL Update:
If you have a version of MySQL >= 8.1.X, please keep your version as it is.
For new installations, please use MySQL version 8.0.39 as specified in the latest version of the docker-compose.yml file. This version is compatible, maintained, and stable.
MySQL does not authorize downgrades (MySQL 8.1.X -> MySQL 8.0.39).
All versions >= 8.2.X are not compatible with Watcher and may impair its functionality.

New Feature

• Added the ability for users to create one or more API keys (admin). (265e73d)
  • Added the ability for administrators to create one or more API keys per user to better manage access to API features. This update will allow for more granular permission management and enhance system security. More details and information about this: API Key Creation & Management by @ygalnezri in #113

What’s Changed

• Added new relevant RSS sources, removed obsolete ones, and improved the "banned words" filters for optimized detection of cyber trends by @ygalnezri in #107

• Fixed a bug related to creating or modifying a ticket with a free format, allowing for smoother integration and precise traceability by @ygalnezri in #129

• Fixed an issue generating false positives in the "Website monitoring" module by @ygalnezri in #137

• Removed the RSS-Bridge container and its related dependencies such as react-twitter-widgets by @ygalnezri in #115

• Fixed an issue allowing duplicate sources to be added in the "threats_watcher" section.

• Updated the docker-compose.yml file to accommodate the new versions by @ygalnezri in #112

• Updated the Dockerfile, bump python-nodejs:python3.9-nodejs18 to python-nodejs:python3.11-nodejs18 by @ygalnezri.

• Revise documentation to include Docker setup instructions, add explanatory notes for the API Key section in the admin section, update the Update Watcher tab, and address other minor corrections by @ygalnezri in #138

• v2.0 by @ygalnezri in #144

• Bump django from 4.1.4 to 4.1.7 in /Watcher by @dependabot in #93

• Bump webpack from 5.75.0 to 5.76.0 in /Watcher by @dependabot in #94

• Bump django from 4.1.7 to 4.1.10 in /Watcher by @dependabot in #98

• Bump semver from 6.3.0 to 6.3.1 in /Watcher by @dependabot in #99

New Contributors

• @ygalnezri made their first contribution in #106
• @PoloOctopus made their first contribution in #105

Full Changelog: v1.2.13...v2.0