Skip to content

Conversation

Signor1
Copy link

@Signor1 Signor1 commented Oct 4, 2025

Motivation

The EF state tests runner currently generates reports in the root directory with static filenames (success_report.txt and
failure_report.txt), which makes it difficult to track different test runs and compare results over time. Additionally, failure reports use
plain text formatting that can be hard to read when debugging test failures.

Description

This PR polishes the test reports for the EF state tests runner by implementing the following improvements:

  • Reports directory: All reports are now generated in a reports/ directory that is automatically created if it doesn't exist
  • Timestamped filenames: Report files now include timestamps in the format YYYY-MM-DD_HH-MM-SS (e.g.,
    failure_report_2025-10-03_14-30-45.txt), making it easy to identify and compare different test runs
  • Improved formatting: Failure reports now use prettytable-rs to display test information and errors in formatted tables, significantly improving readability
  • Cleanup command: Added make clean-reports command to easily delete all generated reports
  • Updated .gitignore: Added reports/ directory to prevent generated reports from being committed

These changes make it easier to debug failing EF tests by providing better-organized, timestamped, and more readable reports.

Closes #3886

  - Move reports to reports/ directory with automatic creation
  - Add timestamps to report filenames (YYYY-MM-DD_HH-MM-SS)
  - Improve failure report readability using prettytable-rs
  - Add make clean-reports command to delete all reports
  - Update .gitignore to exclude reports/ directory
@Signor1 Signor1 requested a review from a team as a code owner October 4, 2025 01:29
@Signor1 Signor1 mentioned this pull request Oct 6, 2025
Copy link
Contributor

@JereSalo JereSalo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that one report is being generated per test? get_report_paths() gets a new timestamp when called. Wouldn't that generate a ton of files if we run all the tests? Maybe it would be better to have less files.

If you have them could you provide an example of success and failure report so that everybody can see the final result in the PR? Thanks

@Signor1
Copy link
Author

Signor1 commented Oct 8, 2025

Is it possible that one report is being generated per test? get_report_paths() gets a new timestamp when called. Wouldn't that generate a ton of files if we run all the tests? Maybe it would be better to have less files.

If you have them could you provide an example of success and failure report so that everybody can see the final result in the PR? Thanks

Alright, thanks.

@Signor1
Copy link
Author

Signor1 commented Oct 8, 2025

Is it possible that one report is being generated per test? get_report_paths() gets a new timestamp when called. Wouldn't that generate a ton of files if we run all the tests? Maybe it would be better to have less files.

If you have them could you provide an example of success and failure report so that everybody can see the final result in the PR? Thanks

I just updated it to generate single timestamped report per test run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Polish tests reports
2 participants