Allow to define more than one service deployer and choose the deployer via configuration setting #2638

mrodm · 2025-06-05T15:23:24Z

Relates #1302

This PR allows us to define more than one service deployer in the _dev/deploy folder (at package or data stream level).

If there are more than one service deployer defined, it must be added to the test system configuration file a new setting deployer. This new setting helps us to choose which specific specific service deployer will be used by each test. Example:

Test using docker deployer:

deployer: docker
service: nginx
vars: ~
data_stream:
  vars:
    paths:
      - "{{SERVICE_LOGS_DIR}}/access.log*"

Test using terrafirn deployer:

deployer: tf
wait_for_data_timeout: 2m
vars: ~
data_stream:
  vars:
    paths:
      - "{{SERVICE_LOGS_DIR}}/file.log*"

If this new setting is not present in the configuration file, there must be zero or one service deployers defined (same behavior as before).

Added a new test package where this new feature is used: nginx_multiple_services
This test package is a subset of the nginx test package (just access and error data-stream and no kibana assets).

Author's checklist

Test packages in integrations repositiory with this elastic-package: Test elastic-package#2638 - DO NOT MERGE integrations#14180
Add a test package (nginx_multiple_services) defining both docker and terraform service deployers.

From now on, there could be more than one service deployer defined in the `_dev/deploy` folder. This new setting helps us to choose which specific specific service deployer will be used by each test. If this new setting is not present in the configuration file, there must be zero or one service deployers defined.

mrodm · 2025-06-06T11:32:24Z

/test

mrodm · 2025-06-06T15:09:57Z

test integrations

elastic-vault-github-plugin-prod · 2025-06-06T15:16:29Z

Created or updated PR in integrations repository to test this version. Check elastic/integrations#14180

mrodm · 2025-06-09T13:21:56Z

internal/agentdeployer/factory.go

+	devDeployPath, err := servicedeployer.FindDevDeployPath(servicedeployer.FactoryOptions{
+		DataStreamRootPath: options.DataStreamRootPath,
+		DevDeployDir:       options.DevDeployDir,
+		PackageRootPath:    options.PackageRootPath,
+	})


Used the same functions as in the servicedeployer module.

mrodm · 2025-06-09T13:22:54Z

internal/testrunner/runners/system/test_config.go

+	// Not included in package-spec validation for deployer name
+	if c.Deployer != "" && !slices.Contains(allowedDeployerNames, c.Deployer) {
+		return nil, fmt.Errorf("invalid deployer name %q in system test configuration file %q, allowed values are: %s", c.Deployer, configFilePath, strings.Join(allowedDeployerNames, ", "))
+	}


Added validation here instead of package-spec. In package-spec this file has additionalProperties: true

mrodm · 2025-06-09T16:00:06Z

/test

jsoriano

Looks good.

internal/testrunner/runners/system/test_config.go

jsoriano · 2025-06-19T16:07:42Z

internal/agentdeployer/factory.go

-
-func findAgentDeployers(devDeployPath string) ([]string, error) {
-	fis, err := os.ReadDir(devDeployPath)
+func findAgentDeployers(devDeployPath, expectedDeployer string) ([]string, error) {


Nit. This method returns now a single deployer or none, right? We could change it to return a single value, and we could return a named error in case of no deployer found.

Suggested change

func findAgentDeployers(devDeployPath, expectedDeployer string) ([]string, error) {

func findAgentDeployer(devDeployPath, expectedDeployer string) (string, error) {

Updated method to just return one single deployer or none (empty string).
And it has been changed it to fail if deployer is set in the test configuration file and that expected agent deployer is not found in the deploy folder.

jsoriano · 2025-06-19T16:11:53Z

test/packages/parallel/nginx_multiple_services/manifest.yml

Nit. Could we also have a failing case, with a package that defines multiple deployers but only defines one?

Not sure what you mean for this failing case.

One data stream that in its system test configuration sets "docker" as deployer_name setting but that deployer is not present under any _dev/deploy folder?

Oh sorry, I mispelled, I meant a package that defines multiple deployers, but uses none. So that this error is reproduced: expected to find only one agent deployer in...

Added a test package nginx_missing_deployer with two errors:

data_stream access requires tf deployer but it is not present in the _dev/deploy folder from the data stream.

data_stream error uses the _dev/deploy folder (from package level) where there are two deployers defined, but it does not set deployer field in the system test configuration.

elasticmachine · 2025-06-20T12:49:50Z

💚 Build Succeeded

Buildkite Build
Commit: 97676f4

History

💔 Build #5527 failed 02b4144
💚 Build #5526 succeeded ba96214
💚 Build #5524 succeeded 15e83cc
💔 Build #5523 failed 358e938
💔 Build #5521 failed 3c764bd

cc @mrodm

jsoriano

Looks good, thanks for adding the test cases!

jsoriano · 2025-06-23T08:56:57Z

test/packages/false_positives/nginx_missing_deployer.expected_errors

mrodm self-assigned this Jun 5, 2025

mrodm added 6 commits June 5, 2025 18:50

Add test package

3ef9cb9

Remove variants file

924e8ad

Update readme new package

85e4b24

Override service name in system test configuration files

cd59c57

Add validation for deployer test system configuration

1264d69

Add comments

6c1b880

elastic-vault-github-plugin-prod bot mentioned this pull request Jun 6, 2025

Test elastic-package#2638 - DO NOT MERGE elastic/integrations#14180

Closed

mrodm commented Jun 9, 2025

View reviewed changes

mrodm marked this pull request as ready for review June 9, 2025 16:25

mrodm requested a review from a team June 9, 2025 16:26

jsoriano approved these changes Jun 19, 2025

View reviewed changes

mrodm added 10 commits June 20, 2025 09:15

Add false positive test

d4bdfa0

Fix findAgentDeployers

159c9ca

Remove last new line from expected errors

3c764bd

Update regex in expected error

ae4142b

Add another false positive error

358e938

Fix regex for the error of the second false positive test

15e83cc

Update docs about system testing

c48a988

Return just one element in findAgentDeployer

ba96214

Fail agent setup if unexpected deployers are found

02b4144

Update false positive tests for nginx_missing_deployer

97676f4

mrodm requested a review from jsoriano June 20, 2025 12:57

jsoriano approved these changes Jun 23, 2025

View reviewed changes

test/packages/false_positives/nginx_missing_deployer.expected_errors

Copy link

Member

jsoriano Jun 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

mrodm merged commit 9c8276d into elastic:main Jun 23, 2025
3 checks passed

mrodm deleted the allow_multiple_service_deployers branch June 23, 2025 10:35

mrodm mentioned this pull request Jun 24, 2025

Support deploying multiple services in system tests #1302

Closed

ShourieG mentioned this pull request Jun 25, 2025

[Imperva Cloud WAF] Add system test for the CEL code elastic/integrations#10357

Closed

efd6 mentioned this pull request Jul 9, 2025

[Netskope] Add Alerts v2 data stream elastic/integrations#14443

Merged

5 tasks

	func findAgentDeployers(devDeployPath, expectedDeployer string) ([]string, error) {
	func findAgentDeployer(devDeployPath, expectedDeployer string) (string, error) {

Allow to define more than one service deployer and choose the deployer via configuration setting #2638

Allow to define more than one service deployer and choose the deployer via configuration setting #2638

Uh oh!

Conversation

mrodm commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Author's checklist

Uh oh!

mrodm commented Jun 6, 2025

Uh oh!

mrodm commented Jun 6, 2025

Uh oh!

elastic-vault-github-plugin-prod bot commented Jun 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrodm commented Jun 9, 2025

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrodm Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jun 20, 2025

💚 Build Succeeded

History

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mrodm commented Jun 5, 2025 •

edited

Loading

mrodm Jun 19, 2025 •

edited

Loading