Skip to content

Migrate performance tests to Playwright #51084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
WunderBart merged 53 commits into from
Jun 27, 2023
Merged

Migrate performance tests to Playwright #51084

Changes from 1 commit
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
cdc5b1a
Create performance config
WunderBart May 30, 2023
58c9171
Define WP_ARTIFACTS_PATH in global setup
WunderBart May 30, 2023
35c35ef
Move perf utils and reporter
WunderBart May 30, 2023
b251340
Migrate Site Editor perf spec
WunderBart May 30, 2023
9837b4a
Try running in CI
WunderBart May 30, 2023
117ea63
Refactor reporter to Playwright
WunderBart May 30, 2023
c91913c
Ignore performance tests from the base E2E config
WunderBart May 30, 2023
cb1f49f
Delete the obsolete reporter
WunderBart May 30, 2023
33ea9c1
Migrate Post Editor perf specs
WunderBart May 30, 2023
4f05e97
Migrate the classic theme perf spec
WunderBart Jun 7, 2023
96344ac
Migrate the block theme perf spec
WunderBart Jun 7, 2023
fdf9eeb
Move WP_ASSETS_PATH env var declaration to the base config
WunderBart Jun 7, 2023
d9db7fe
Move the sample loop inside
WunderBart Jun 7, 2023
7d9f84e
Make the test independent from preceding steps
WunderBart Jun 7, 2023
c36eacc
Push missing util
WunderBart Jun 7, 2023
2f163a3
Small cleanup
WunderBart Jun 7, 2023
788548c
Make the perf config independent from the e2e one
WunderBart Jun 9, 2023
4bf1c77
Try tracing warnings to see what's wrong with CI
WunderBart Jun 9, 2023
7718268
Merge remote-tracking branch 'origin' into refactor/playwright-perfor…
WunderBart Jun 13, 2023
262d5be
Undo changes to the e2e config
WunderBart Jun 13, 2023
a0e8e4c
Rename perf reporter class
WunderBart Jun 13, 2023
ce36f27
Make tests as 1:1 to current ones as possible
WunderBart Jun 14, 2023
9556478
Add linefeed at the end of the custom report
WunderBart Jun 14, 2023
3649b25
Isolate performance tests from e2e tests
WunderBart Jun 14, 2023
c107fda
Fix esm/cjs mixup
WunderBart Jun 14, 2023
74ba986
Build packages correctly
WunderBart Jun 14, 2023
baef167
Merge branch 'trunk' into refactor/playwright-performance-tests
WunderBart Jun 14, 2023
cc06c99
Make site editor loading test take sample from a new page as before
WunderBart Jun 15, 2023
94dd81a
Try running a different test
WunderBart Jun 15, 2023
9424aeb
Try printing from the child process
WunderBart Jun 15, 2023
c7ac136
Make sure Playwright is installed
WunderBart Jun 15, 2023
07afec7
Revert "Try printing from the child process"
WunderBart Jun 15, 2023
40ddc30
Revert "Try running a different test"
WunderBart Jun 15, 2023
f53273d
Leave only chromium for perf testing
WunderBart Jun 15, 2023
a3679e8
Try stdout again
WunderBart Jun 15, 2023
71e673d
Do not run retries
WunderBart Jun 15, 2023
2596967
Create new page for loading spec as before
WunderBart Jun 15, 2023
7885bf3
Make naming consistent
WunderBart Jun 15, 2023
07e4d34
Remove warning tracing
WunderBart Jun 15, 2023
1ebe284
Revert "Try stdout again"
WunderBart Jun 15, 2023
5412032
Update config according to review comments
WunderBart Jun 16, 2023
875a4f1
tmp: Compare Puppeteer vs. Playwright in CI
WunderBart Jun 16, 2023
d9f2672
Revert "Revert "Try stdout again""
WunderBart Jun 16, 2023
740360a
Increase global timeout to 10 minutes
WunderBart Jun 19, 2023
5934795
Remove unused dep
WunderBart Jun 19, 2023
c8e6639
Extend more timeouts to address flaky CI
WunderBart Jun 19, 2023
d0aee00
Revert "Revert "Revert "Try stdout again"""
WunderBart Jun 19, 2023
115d731
Revert "tmp: Compare Puppeteer vs. Playwright in CI"
WunderBart Jun 20, 2023
ffcd340
Add Playwright script, but keep Puppeteer as a default runner
WunderBart Jun 20, 2023
e77d0b7
tmp: Run Puppeteer vs. Playwright comparison 100 times
WunderBart Jun 22, 2023
6bb53d8
Revert "tmp: Run Puppeteer vs. Playwright comparison 100 times"
WunderBart Jun 26, 2023
637035a
Revert "Make sure Playwright is installed"
WunderBart Jun 27, 2023
c5266ba
Revert "Build packages correctly"
WunderBart Jun 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update config according to review comments
  • Loading branch information
@WunderBart
WunderBart committed Jun 16, 2023
commit 5412032fee975b9cae48ef4f28f4c4d6644dc86f
3 changes: 1 addition & 2 deletions test/performance/playwright.config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ const config = defineConfig( {
actionTimeout: 10_000, // 10 seconds.
trace: 'retain-on-failure',
Copy link
Member

@kevin940726 kevin940726 Jun 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will have some minor performance impact. Probably we don't need to enable it in performance test?

Copy link
Member Author

@WunderBart WunderBart Jun 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we mostly care about changes in performance, so the impact itself might not be that important if we keep it switched on. The question is whether we need the trace or not. It did help me with some debugging just recently!

screenshot: 'only-on-failure',
video: 'on-first-retry',
video: 'off',
},
webServer: {
command: 'npm run wp-env start',
Expand All @@ -58,7 +58,6 @@ const config = defineConfig( {
{
name: 'chromium',
use: { ...devices[ 'Desktop Chrome' ] },
grepInvert: /-chromium/,
},
],
} );
Expand Down