Skip to content

[ci] Add emulator tests on older Android version #4616

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
auto-submit merged 8 commits into from
Oct 19, 2023
Merged

[ci] Add emulator tests on older Android version #4616

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
39a6b0f
Test 21 with existing tasks
stuartmorgan-g Jul 31, 2023
e62bc42
Try 22, which should be available
stuartmorgan-g Aug 1, 2023
2701cf2
Merge branch 'main' into android-legacy-emulator
stuartmorgan-g Oct 18, 2023
a5c1a48
Add a new exclusion file and test file
stuartmorgan-g Oct 18, 2023
3b8c115
More exclusions
stuartmorgan-g Oct 18, 2023
a7f5cbe
Add as actual new tasks
stuartmorgan-g Oct 19, 2023
4c5de9b
Typo fix
stuartmorgan-g Oct 19, 2023
997e2a9
Move emulator definition to new base config
stuartmorgan-g Oct 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -529,6 +529,90 @@ targets:
{"dependency": "android_virtual_device", "version": "33"}
]

- name: Linux_android android_platform_tests_legacy_api_shard_1 master
recipe: packages/packages
timeout: 60
bringup: true # New target
properties:
target_file: android_legacy_emulator_tests.yaml
channel: master
version_file: flutter_master.version
package_sharding: "--shardIndex 0 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "22"}
]

- name: Linux_android android_platform_tests_legacy_api_shard_2 master
recipe: packages/packages
timeout: 60
bringup: true # New target
properties:
target_file: android_legacy_emulator_tests.yaml
channel: master
version_file: flutter_master.version
package_sharding: "--shardIndex 1 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "22"}
]

- name: Linux_android android_platform_tests_legacy_api_shard_3 master
recipe: packages/packages
timeout: 60
bringup: true # New target
properties:
target_file: android_legacy_emulator_tests.yaml
channel: master
version_file: flutter_master.version
package_sharding: "--shardIndex 2 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "22"}
]

- name: Linux_android android_platform_tests_legacy_api_shard_4 master
recipe: packages/packages
timeout: 60
bringup: true # New target
properties:
target_file: android_legacy_emulator_tests.yaml
channel: master
version_file: flutter_master.version
package_sharding: "--shardIndex 3 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "22"}
Copy link
Contributor

@reidbaker reidbaker Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of scope for this pr but I wish we could use variables for these shards since they cant actually have different dependencies.

Copy link
Collaborator Author

@stuartmorgan-g stuartmorgan-g Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could make a new config for legacy emulator Android tests, but without a way to have one platform_properties entry inherit from another one that sort of just moves the problem (since I'd have to instead duplicate all the core Android dependencies, which can't really be different either). Would you prefer that? I can see an argument for it being less error-prone.

Copy link
Contributor

@reidbaker reidbaker Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is out of scope for this change but anyplace we can make it clear in code that dependencies that have to be the same are the same I think it is a win for clarity and maintainability.

Copy link
Collaborator Author

@stuartmorgan-g stuartmorgan-g Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a whole new set of shards with configs, it seems in scope to have it start the way you'd prefer it; I'll leave the existing ones alone for now though (especially so I don't mess up Camille's WIP PR).

Copy link
Collaborator Author

@stuartmorgan-g stuartmorgan-g Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know what you think of this updated version :)

]

- name: Linux_android android_platform_tests_legacy_api_shard_5 master
recipe: packages/packages
timeout: 60
bringup: true # New target
properties:
target_file: android_legacy_emulator_tests.yaml
channel: master
version_file: flutter_master.version
package_sharding: "--shardIndex 4 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "22"}
]

- name: Linux_android android_platform_tests_legacy_api_shard_6 master
recipe: packages/packages
timeout: 60
bringup: true # New target
properties:
target_file: android_legacy_emulator_tests.yaml
channel: master
version_file: flutter_master.version
package_sharding: "--shardIndex 5 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "22"}
]

# Device versions of Android integration tests, run via FTL.
# TODO(stuartmorgan): Revisit whether physical device tests are redundant once
# we have more data about emulator tests; see
Expand Down
16 changes: 16 additions & 0 deletions .ci/targets/android_legacy_emulator_tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
tasks:
- name: prepare tool
script: .ci/scripts/prepare_tool.sh
infra_step: true # Note infra steps failing prevents "always" from running.
- name: download Dart and Android deps
script: script/tool_runner.sh
infra_step: true
args: ["fetch-deps", "--android", "--supporting-target-platforms-only", "--exclude=script/configs/exclude_integration_android_legacy_emulator.yaml"]
# Only these two tests are run because they are the only ones that use the
# emulator. Other tests would just duplicate android_platform_test.yaml tests.
- name: native integration tests
script: script/tool_runner.sh
args: ["native-test", "--android", "--no-unit", "--exclude=script/configs/exclude_integration_android_legacy_emulator.yaml"]
- name: drive examples
script: script/tool_runner.sh
args: ["drive-examples", "--android", "--exclude=script/configs/exclude_integration_android_legacy_emulator.yaml,script/configs/exclude_integration_android.yaml,script/configs/exclude_integration_android_emulator.yaml"]
13 changes: 13 additions & 0 deletions script/configs/exclude_integration_android_legacy_emulator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Tihs plugin no-ops below API 25, which is newer than the legacy emulator.
# This can be removed once the legacy emulator test is using at least
# API level 25.
- quick_actions
- quick_actions_android
# Fails on the legacy emulator. See https://github.com/flutter/flutter/issues/136823
# TODO(stuartmorgan): Remove this if the limitation can be worked around.
- video_player
- video_player_android
# Hangs on the legacy emulator. See https://github.com/flutter/flutter/issues/136824
# TODO(stuartmorgan): Remove this if the hang can be fixed.
- webview_flutter
- webview_flutter_android