Add test package with docker service deployers using terraform too

docs/howto/system_testing.md

@@ -69,7 +69,7 @@ or the data stream's level:
 
 `<service deployer>` - a name of the supported service deployer:
 * `docker` - Docker Compose
-* `agent` - Custom `elastic-agent` with Docker Compose
+* `agent` - (Deprecated) Custom `elastic-agent` with Docker Compose
 * `k8s` - Kubernetes
 * `tf` - Terraform
 
@@ -110,6 +110,69 @@ volumes:
   mysqldata:
 ```
 
+#### Run provisioner tool along with the Docker Compose service deployer
+
+Along with the Docker Compose service deployer, it could be added other services in the docker-compose scenario
+to run other provisioner tools. For instance, the following example shows how it could be used Terraform with the
+Docker Compose service deployer, but other tools could be used too.
+
+**Please note**: this is not officially supported by `elastic-package`. Package owners are responsible for maintaining
+their own provisioner Dockerfiles and other resources required (e.g. scripts).
+
+There is an example in the [test package `nginx_multiple_services`](../../test/packages/parallel/nginx_multiple_services/).
+
+For that, you need to add another `terraform` service container in the docker-compose scenario as follows:
+```yaml
+version: '2.3'
+services:
+  nginx:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    ports:
+      - 80
+    volumes:
+      - ${SERVICE_LOGS_DIR}:/var/log/nginx
+    depends_on:
+      terraform:
+        condition: service_healthy
+  terraform:
+    tty: true
+    stop_grace_period: 5m
+    build:
+      context: .
+      dockerfile: Dockerfile.terraform
+    environment:
+      - TF_VAR_TEST_RUN_ID=${TEST_RUN_ID:-detached}
+      - TF_VAR_CREATED_DATE=${CREATED_DATE:-unknown}
+      - TF_VAR_BRANCH=${BRANCH_NAME_LOWER_CASE:-unknown}
+      - TF_VAR_BUILD_ID=${BUILD_ID:-unknown}
+      - TF_VAR_ENVIRONMENT=${ENVIRONMENT:-unknown}
+      - TF_VAR_REPO=${REPO:-unknown}
+    volumes:
+      - ./tf/:/stage/
+      - ${SERVICE_LOGS_DIR}:/tmp/service_logs/
+```
+
+Two other files required are to define this `terraform` service:
+- Dockerfile to build the terraform container.
+- Shellscript to trigger the terraform commands.
+
+Those files are available in the [test package](../../test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/)
+([Dockerfile.terraform](../../test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/Dockerfile.terraform) and
+[terraform.run.sh](../../test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/terraform.run.sh))
+and they could be used as a basis for other packages.
+
+Along with that docker-compose definition, it is required to add the terraform files (`*.tf`) within the `docker` folder.
+They can be placed in a `tf` folder for convenience.
+
+This terraform container must be used as a helper for the main service defined in the docker-compose to create the required resources for the main service (e.g. `nginx` in the example above).
+In this example, Terraform creates a new file in `/tmp/service_logs` and it is used by the `nginx` service.
+
+Currently, there is no support to use outputs of terraform via this service deployer to set parameters in the test configuration file (variables),
+as it is available in the terraform service deployer.
+
+
 ### Agent service deployer
 
 **NOTE**: Deprecated in favor of creating [new Elastic Agents in each test](#running-a-system-test). These
@@ -619,6 +682,7 @@ The `SERVICE_LOGS_DIR` placeholder is not the only one available for use in a da
 | `Port` | int | Alias for `Ports[0]`. Provided as a convenience. |
 | `Logs.Folder.Agent` | string | Path to integration service's logs folder, as addressable by the Agent. |
 | `SERVICE_LOGS_DIR` | string | Alias for `Logs.Folder.Agent`. Provided as a convenience. |
+| `TEST_RUN_ID` | string | Unique identifier for the test run, allows to distinguish each run. Provided as a convenience. |
 
 Placeholders used in the `test-<test_name>-config.yml` must be enclosed in `{{{` and `}}}` delimiters, per Handlebars syntax.
 

internal/servicedeployer/compose.go

@@ -80,6 +80,7 @@ func (d *DockerComposeServiceDeployer) SetUp(ctx context.Context, svcInfo Servic
 		variant:  d.variant,
 		env: []string{
 			fmt.Sprintf("%s=%s", serviceLogsDirEnv, svcInfo.Logs.Folder.Local),
+			fmt.Sprintf("%s=%s", testRunIDEnv, svcInfo.Test.RunID),
 		},
 	}
 

test/packages/parallel/nginx_multiple_services/_dev/build/docs/README.md

@@ -29,6 +29,15 @@ Access logs collects the nginx access logs.
 
 {{fields "access"}}
 
+### Access Docker & TF Logs
+
+Access logs collects the nginx access logs.
+
+{{event "access_docker_tf"}}
+
+{{fields "access_docker_tf"}}
+
+
 ### Error Logs
 
 Error logs collects the nginx error logs.

test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/Dockerfile

@@ -0,0 +1,6 @@
+ARG SERVICE_VERSION=${SERVICE_VERSION:-1.19.5}
+FROM nginx:${SERVICE_VERSION}
+RUN sed -i "/jessie-updates/d" /etc/apt/sources.list
+RUN apt-get update && apt-get install -y curl
+HEALTHCHECK --interval=1s --retries=90 CMD curl -f http://localhost/server-status
+COPY ./nginx.conf /etc/nginx/

test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/Dockerfile.terraform

@@ -0,0 +1,26 @@
+FROM --platform=linux/amd64 ubuntu:24.04
+ENV TERRAFORM_VERSION=1.9.6
+
+# Based on this Dockerfile:
+# https://github.com/elastic/elastic-package/blob/d8e4300715af43b3b792d25960d75c3382948dcd/internal/servicedeployer/_static/Dockerfile.terraform_deployer
+# This dockerfile just installs terraform and sets up the environment to run terraform commands.
+
+RUN apt-get -qq update \
+  && apt-get install -yq curl apt-transport-https ca-certificates gnupg \
+  && apt-get clean
+
+RUN curl -fsSL https://apt.releases.hashicorp.com/gpg | gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg \
+  && echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com noble main" | tee /etc/apt/sources.list.d/hashicorp.list \
+  && apt-get update -qq \
+  && apt-get install -yq terraform=${TERRAFORM_VERSION}-1 \
+  && apt-get clean
+
+HEALTHCHECK --timeout=3s CMD sh -c "[ -f /tmp/tf-applied ]"
+
+ENV TF_IN_AUTOMATION=true
+ENV TF_CLI_ARGS="-no-color"
+ADD terraform.run.sh /run.sh
+RUN chmod +x /run.sh
+WORKDIR /workspace
+
+ENTRYPOINT exec /run.sh

test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/docker-compose.yml

@@ -0,0 +1,29 @@
+version: '2.3'
+services:
+  nginx:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    ports:
+      - 80
+    volumes:
+      - ${SERVICE_LOGS_DIR}:/var/log/nginx
+    depends_on:
+      terraform:
+        condition: service_healthy
+  terraform:
+    tty: true
+    stop_grace_period: 5m
+    build:
+      context: .
+      dockerfile: Dockerfile.terraform
+    environment:
+      - TF_VAR_TEST_RUN_ID=${TEST_RUN_ID:-detached}
+      - TF_VAR_CREATED_DATE=${CREATED_DATE:-unknown}
+      - TF_VAR_BRANCH=${BRANCH_NAME_LOWER_CASE:-unknown}
+      - TF_VAR_BUILD_ID=${BUILD_ID:-unknown}
+      - TF_VAR_ENVIRONMENT=${ENVIRONMENT:-unknown}
+      - TF_VAR_REPO=${REPO:-unknown}
+    volumes:
+      - ./tf/:/stage/
+      - ${SERVICE_LOGS_DIR}:/tmp/service_logs/

test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/nginx.conf

@@ -0,0 +1,33 @@
+user  nginx;
+worker_processes  1;
+
+error_log  /dev/stderr warn;
+pid        /var/run/nginx.pid;
+
+
+events {
+    worker_connections  1024;
+}
+
+
+http {
+    include       /etc/nginx/mime.types;
+    default_type  application/octet-stream;
+
+    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
+                      '$status $body_bytes_sent "$http_referer" '
+                      '"$http_user_agent" "$http_x_forwarded_for"';
+    access_log  /var/log/nginx/access.log main;
+
+    server {
+        listen [::]:80;
+        listen 80;
+        server_name localhost;
+
+        location /server-status {
+            stub_status on;
+        }
+    }
+
+    include /etc/nginx/conf.d/*;
+}

test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/terraform.run.sh

@@ -0,0 +1,33 @@
+#!/usr/bin/env bash
+
+set -euxo pipefail
+
+# Terraform code may rely on content from other files than .tf files (es json, zip, html, text), so we copy all the content over
+# See more: https://github.com/elastic/elastic-package/pull/603
+# NOTE: must copy hidden files too (supported by "/.")
+# See more: https://github.com/elastic/package-spec/issues/269
+cp -r /stage/. /workspace
+
+cleanup() {
+  r=$?
+
+  set -x
+  terraform destroy -auto-approve
+
+  exit $r
+}
+trap cleanup EXIT INT TERM
+
+terraform init
+terraform plan
+terraform apply -auto-approve
+
+mkdir -p /output
+terraform output -json > /output/tfOutputValues.json
+
+touch /tmp/tf-applied # This file is used as indicator (healthcheck) that the service is UP, and so it must be placed as the last statement in the script
+
+echo "Terraform definitions applied."
+
+set +x
+while true; do sleep 1; done # wait for ctrl-c

test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/tf/files/example.log

@@ -0,0 +1,21 @@
+::1 - - [03/Jun/2025:14:41:33 +0000] "GET /server-status-tf HTTP/1.1" 200 97 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:34 +0000] "GET /server-status-tf HTTP/1.1" 200 97 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:35 +0000] "GET /server-status-tf HTTP/1.1" 200 97 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:36 +0000] "GET /server-status-tf HTTP/1.1" 200 97 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:37 +0000] "GET /server-status-tf HTTP/1.1" 200 97 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:38 +0000] "GET /server-status-tf HTTP/1.1" 200 97 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:39 +0000] "GET /server-status-tf HTTP/1.1" 200 97 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:40 +0000] "GET /server-status-tf HTTP/1.1" 200 97 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:41 +0000] "GET /server-status-tf HTTP/1.1" 200 97 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:42 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:43 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:45 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:46 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:47 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:48 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:49 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:50 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:51 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:52 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:53 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"
+::1 - - [03/Jun/2025:14:41:54 +0000] "GET /server-status-tf HTTP/1.1" 200 100 "-" "curl/7.64.0" "-"

test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/tf/main.tf

@@ -0,0 +1,37 @@
+# Remember to set default tags in case the provider allows for that.
+# Example for the "aws" provider
+# provider "aws" {
+#   region = "us-east-1"
+#   default_tags {
+#     tags = {
+#       environment  = var.ENVIRONMENT
+#       repo         = var.REPO
+#       branch       = var.BRANCH
+#       build        = var.BUILD_ID
+#       created_date = var.CREATED_DATE
+#     }
+#   }
+# }
+
+resource "local_file" "log" {
+  source          = "./files/example.log"
+  filename        = "/tmp/service_logs/file.log"
+  file_permission = "0777"
+}
+
+locals {
+  items ={
+    environment  = "${var.ENVIRONMENT}"
+    repo         = "${var.REPO}"
+    branch       = "${var.BRANCH}"
+    build        = "${var.BUILD_ID}"
+    created_date = "${var.CREATED_DATE}"
+    test_run_id  = "${var.TEST_RUN_ID}"
+  }
+}
+
+resource "local_file" "log_variables" {
+  content         = format("%s\n", jsonencode(local.items))
+  filename        = "/tmp/service_logs/file_vars.log"
+  file_permission = "0777"
+}

test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/deploy/docker/tf/variables.tf

@@ -0,0 +1,26 @@
+variable "BRANCH" {
+  description = "Branch name or pull request for tagging purposes"
+  default     = "unknown-branch"
+}
+
+variable "BUILD_ID" {
+  description = "Build ID in the CI for tagging purposes"
+  default     = "unknown-build"
+}
+
+variable "CREATED_DATE" {
+  description = "Creation date in epoch time for tagging purposes"
+  default     = "unknown-date"
+}
+
+variable "ENVIRONMENT" {
+  default = "unknown-environment"
+}
+
+variable "REPO" {
+  default = "unknown-repo-name"
+}
+
+variable "TEST_RUN_ID" {
+  default = "detached"
+}

test/packages/parallel/nginx_multiple_services/data_stream/access_docker_tf/_dev/test/system/test-default-config.yml

@@ -0,0 +1,12 @@
+deployer: docker
+service: nginx
+wait_for_data_timeout: 2m
+assert:
+  min_count: 50
+vars: ~
+data_stream:
+  vars:
+    paths:
+      - "{{SERVICE_LOGS_DIR}}/file.log*"
+      - "{{SERVICE_LOGS_DIR}}/access.log*"
+      - "{{SERVICE_LOGS_DIR}}/access-{{TEST_RUN_ID}}.log*"