Skip to content

Performance tests: Make theme versions consistent cross-env #50905

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
WunderBart merged 7 commits into from
Jun 19, 2023
+20 −7
Merged

Performance tests: Make theme versions consistent cross-env #50905

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b315dd1
Upload [email protected] and [email protected] to test themes
WunderBart May 24, 2023
ebdefe0
Use uploaded themes in performance tests
WunderBart May 24, 2023
a96092c
Be explicit about which themes we use for perf tests
WunderBart May 24, 2023
af5d1c9
Map zip URLs instead of uploading
WunderBart May 25, 2023
6980769
Remove bundled themes
WunderBart May 25, 2023
80ed46e
Use fixed twentytwentyone for post editor perf specs
WunderBart Jun 19, 2023
2a18422
Merge remote-tracking branch 'origin' into refactor/upload-perf-test-…
WunderBart Jun 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .wp-env.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@
"wp-content/plugins/gutenberg": ".",
"wp-content/mu-plugins": "./packages/e2e-tests/mu-plugins",
"wp-content/plugins/gutenberg-test-plugins": "./packages/e2e-tests/plugins",
"wp-content/themes/gutenberg-test-themes": "./test/gutenberg-test-themes"
"wp-content/themes/gutenberg-test-themes": "./test/gutenberg-test-themes",
"wp-content/themes/gutenberg-test-themes/twentytwentyone": "https://downloads.wordpress.org/theme/twentytwentyone.1.7.zip",
"wp-content/themes/gutenberg-test-themes/twentytwentythree": "https://downloads.wordpress.org/theme/twentytwentythree.1.0.zip"
}
}
}
Expand Down
11 changes: 9 additions & 2 deletions bin/plugin/commands/performance.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,8 +337,6 @@ async function runPerformanceTests( branches, options ) {
performanceTestDirectory,
'test/emptytheme'
),
'https://downloads.wordpress.org/theme/twentytwentyone.1.7.zip',
'https://downloads.wordpress.org/theme/twentytwentythree.1.0.zip',
],
env: {
tests: {
Expand All @@ -352,6 +350,15 @@ async function runPerformanceTests( branches, options ) {
performanceTestDirectory,
'packages/e2e-tests/plugins'
),
'wp-content/themes/gutenberg-test-themes':
path.join(
performanceTestDirectory,
'test/gutenberg-test-themes'
),
'wp-content/themes/gutenberg-test-themes/twentytwentyone':
'https://downloads.wordpress.org/theme/twentytwentyone.1.7.zip',
'wp-content/themes/gutenberg-test-themes/twentytwentythree':
'https://downloads.wordpress.org/theme/twentytwentythree.1.0.zip',
},
},
},
Expand Down
3 changes: 1 addition & 2 deletions packages/e2e-tests/specs/performance/front-end-block-theme.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,12 @@ describe( 'Front End Performance', () => {
};

beforeAll( async () => {
await activateTheme( 'twentytwentythree' );
await activateTheme( 'gutenberg-test-themes/twentytwentythree' );
Copy link
Member

@oandregal oandregal May 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noting here what I've observed, so it's documented somewhere.

This PR maps the specific versions in use within the gutenberg-test-themes folder in the theme root (wp-content/themes) of the testing WordPress environment. As a result, the environment has two instances of the TT1 and TT3 themes: one whose stylesheet is twentytwentyone (latest version) and another with stylesheet gutenberg-test-themes/twentytwentyone (the bundled version, here's at 1.7 for TT1).

You can confirm this by going to the "Appearance > Themes" page (inspect the "activate" button of each instance to know its stylesheet):

Captura de ecrã de 2023-05-25 10-03-53

Also by asking docker: docker exec -it `docker ps -f name=tests-wordpress_1 -q` ls -lat /var/www/html/wp-content/themes.

Copy link
Member Author

@WunderBart WunderBart May 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for noting this down! 🙇 Just wanted to add that we can check the available test env themes also via the following wp-cli command:

npx wp-env run tests-cli 'wp theme list'

For this branch, it should return the following:

+-----------------------------------------+----------+--------+---------+
| name                                    | status   | update | version |
+-----------------------------------------+----------+--------+---------+
| emptytheme                              | inactive | none   | 1.0     |
| gutenberg-test-themes/emptyhybrid       | inactive | none   | 1.0     |
| gutenberg-test-themes/style-variations  | inactive | none   | 1.0     |
| gutenberg-test-themes/twentytwentyone   | inactive | none   | 1.7     |
| gutenberg-test-themes/twentytwentythree | active   | none   | 1.0     |
| twentyeleven                            | inactive | none   | 4.3     |
| twentyfifteen                           | inactive | none   | 3.4     |
| twentyfourteen                          | inactive | none   | 3.6     |
| twentynineteen                          | inactive | none   | 2.5     |
| twentyseventeen                         | inactive | none   | 3.2     |
| twentysixteen                           | inactive | none   | 2.9     |
| twentyten                               | inactive | none   | 3.8     |
| twentythirteen                          | inactive | none   | 3.8     |
| twentytwelve                            | inactive | none   | 3.9     |
| twentytwenty                            | inactive | none   | 2.2     |
| twentytwentyone                         | inactive | none   | 1.8     |
| twentytwentythree                       | inactive | none   | 1.1     |
| twentytwentytwo                         | inactive | none   | 1.4     |
+-----------------------------------------+----------+--------+---------+

As a comparison, we can list the themes used in dev env, which should not return the gutenberg-test-themes/* ones, for example:

> npx wp-env run cli 'wp theme list'

+-------------------+----------+--------+---------+
| name              | status   | update | version |
+-------------------+----------+--------+---------+
| emptytheme        | inactive | none   | 1.0     |
| twentyeleven      | inactive | none   | 4.3     |
| twentyfifteen     | inactive | none   | 3.4     |
| twentyfourteen    | inactive | none   | 3.6     |
| twentynineteen    | inactive | none   | 2.5     |
| twentyseventeen   | inactive | none   | 3.2     |
| twentysixteen     | inactive | none   | 2.9     |
| twentyten         | inactive | none   | 3.8     |
| twentythirteen    | inactive | none   | 3.8     |
| twentytwelve      | inactive | none   | 3.9     |
| twentytwenty      | inactive | none   | 2.2     |
| twentytwentyone   | inactive | none   | 1.8     |
| twentytwentythree | active   | none   | 1.1     |
| twentytwentytwo   | inactive | none   | 1.4     |
+-------------------+----------+--------+---------+

Copy link
Member

@oandregal oandregal May 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Brilliant. Thanks for the tip!

await logout();
} );

afterAll( async () => {
saveResultsFile( __filename, results );
await activateTheme( 'twentytwentyone' );
} );

it( 'Report TTFB, LCP, and LCP-TTFB', async () => {
Expand Down
2 changes: 1 addition & 1 deletion packages/e2e-tests/specs/performance/front-end-classic-theme.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ describe( 'Front End Performance', () => {
};

beforeAll( async () => {
await activateTheme( 'twentytwentyone' );
await activateTheme( 'gutenberg-test-themes/twentytwentyone' );
await logout();
} );

Expand Down
6 changes: 6 additions & 0 deletions packages/e2e-tests/specs/performance/post-editor.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import path from 'path';
* WordPress dependencies
*/
import {
activateTheme,
createNewPost,
saveDraft,
insertBlock,
Expand Down Expand Up @@ -83,6 +84,11 @@ describe( 'Post Editor Performance', () => {

let traceResults;

beforeAll( async () => {
// See https://github.com/WordPress/gutenberg/pull/50905/files#r1209014677;
await activateTheme( 'gutenberg-test-themes/twentytwentyone' );
} );

afterAll( async () => {
saveResultsFile( __filename, results );
deleteFile( traceFilePath );
Expand Down
1 change: 0 additions & 1 deletion packages/e2e-tests/specs/performance/site-editor.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@ describe( 'Site Editor Performance', () => {
saveResultsFile( __filename, results );
await deleteAllTemplates( 'wp_template' );
await deleteAllTemplates( 'wp_template_part' );
await activateTheme( 'twentytwentyone' );
} );

// Number of loading measurements to take.
Expand Down