DRAFT: Add Packer build and upload workflow for Proxmox templates

container-creation/intern-phxdc-pve1/packer/.github/workflows/build-templates.yml

@@ -0,0 +1,45 @@
+name: Build and Upload Templates
+on:
+  schedule:
+    - cron: "0 4 * * *" # Nightly at 4 AM
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    env:
+      PROXMOX_API_URL: ${{ secrets.PROXMOX_API_URL }}
+      PROXMOX_TOKEN_ID: ${{ secrets.PROXMOX_TOKEN_ID }}
+      PROXMOX_TOKEN_SECRET: ${{ secrets.PROXMOX_TOKEN_SECRET }}
+      # Define the template version ONCE for the whole job
+      TEMPLATE_VERSION: ${{ format('{0:yyyyMMdd}', github.run_started_at) }}
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install dependencies
+        run: sudo apt-get update && sudo apt-get install -y packer ansible zstd python3-requests
+
+      # --- Build Debian 12 ---
+      - name: Build Debian 12 Template
+        run: |
+          packer build \
+            -var "template_version=${TEMPLATE_VERSION}" \
+            debian12.pkr.hcl
+
+      - name: Upload Debian 12 Template
+        run: |
+          python3 api/proxmox_upload.py \
+            --file /tmp/output/debian12-fungible_${TEMPLATE_VERSION}.tar.xz
+
+      # --- Build Rocky 9 ---
+      - name: Build Rocky 9 Template
+        run: |
+          packer build \
+            -var "template_version=${TEMPLATE_VERSION}" \
+            rocky9.pkr.hcl
+
+      - name: Upload Rocky 9 Template
+        run: |
+          python3 api/proxmox_upload.py \
+            --file /tmp/output/rocky9-fungible_${TEMPLATE_VERSION}.tar.xz

container-creation/intern-phxdc-pve1/packer/README.md

@@ -0,0 +1,150 @@
+Here is a README.md file that explains how your project works.
+
+-----
+
+# Proxmox LXC Template Automation 📦
+
+This project automates the build, provisioning, and uploading of "fungible" Proxmox LXC container templates. It uses **Packer** to build the images, **Ansible** to provision them, and **Python** to upload them directly to your Proxmox cluster via the API.
+
+This system is designed to run automatically (e.g., nightly via GitHub Actions) to ensure your base container templates are always up-to-date with the latest patches and common configurations.
+
+## The Problem
+
+Manually updating container templates is slow, error-prone, and leads to configuration drift. This automation solves that by:
+
+  * **Ensuring templates are current** with upstream security patches.
+  * **Pre-applying standard configuration** (like common packages, Wazuh, MOTD, etc.) before a container is ever created.
+  * **Reducing manual workload** and enabling faster, more consistent automated deployments.
+
+-----
+
+## How It Works
+
+The process is managed in distinct stages, usually kicked off by the GitHub Actions workflow.
+
+### 1\. Trigger (GitHub Actions)
+
+  * **File:** `.github/workflows/build-templates.yml`
+  * **Action:** On a schedule (e.g., nightly) or a manual trigger, a new runner spins up.
+  * **Steps:**
+    1.  Installs the required tools: `packer`, `ansible`, `python3-requests`, and `zstd`.
+    2.  Sets environment variables (like API secrets and a `TEMPLATE_VERSION`).
+
+### 2\. Build (Packer)
+
+  * **File:** `debian12.pkr.hcl` / `rocky9.pkr.hcl`
+  * **Action:** The workflow runs the `packer build ...` command.
+  * **Steps:**
+    1.  **Download:** A `shell` provisioner downloads the official Proxmox base template (a `.tar.zst` file) from `download.proxmox.com`.
+    2.  **Extract:** The file is decompressed and extracted into a temporary directory, `/tmp/rootfs`. This folder now contains the entire offline file system of the container.
+
+### 3\. Provision (Ansible)
+
+  * **File:** `provisioners/ansible/site.yml`
+  * **Action:** Packer's `ansible` provisioner takes over.
+  * **Steps:**
+    1.  **The `chroot` Connection:** Ansible is told to use `--connection=chroot` and its "inventory" is just the `/tmp/rootfs` directory.
+    2.  **Run Playbook:** Ansible runs all tasks *inside* that directory as if it were a running system. This is where it:
+          * Installs packages (`vim`, `curl`, `git`, etc.).
+          * Sets the "Message of the Day" (`/etc/motd`).
+          * Copies over placeholder scripts for services like Wazuh. (These are designed *not* to run during the build, but rather on the container's first real boot).
+          * Cleans up temporary files.
+
+### 4\. Package (Packer)
+
+  * **File:** `debian12.pkr.hcl`
+  * **Action:** Packer runs its final `shell` provisioner.
+  * **Steps:**
+    1.  **Compress:** It `cd`s into the *modified* `/tmp/rootfs` directory.
+    2.  **Create Tarball:** It creates a new, compressed `.tar.xz` file (e.g., `debian12-fungible_20251024.tar.xz`) containing the fully-provisioned file system.
+
+### 5\. Upload (Python)
+
+  * **File:** `api/proxmox_upload.py`
+  * **Action:** The GitHub workflow's final step calls this Python script.
+  * **Steps:**
+    1.  **Authenticate:** The script reads the `PROXMOX_API...` environment variables and authenticates with the Proxmox API.
+    2.  **Find Nodes:** It calls the `/nodes` API endpoint to get a list of all nodes in the cluster (e.g., `intern-phxdc-pve1`, `pve2`).
+    3.  **Find Storage:** For *each* node, it calls the `/nodes/{node}/storage` endpoint to find available storage.
+    4.  **Upload:** It intelligently picks the best `local`-type storage (falling back to `local`) and uploads the `.tar.xz` file to it.
+    5.  **Repeat:** It repeats this process for *every node*, ensuring the template is available cluster-wide.
+
+-----
+
+## Prerequisites
+
+To run this, you will need:
+
+### Tools
+
+  * [Packer](https://www.packer.io/downloads)
+  * [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html)
+  * [Python 3](https://www.python.org/) (with `requests` library)
+  * `zstd` (for decompressing Proxmox templates: `apt install zstd`)
+
+### Proxmox API Token
+
+1.  In your Proxmox GUI, go to **Datacenter** -\> **Permissions** -\> **API Tokens**.
+2.  Create a token (e.g., for `root@pam` or a dedicated automation user).
+3.  **Permissions:** The token needs at minimum:
+      * `Nodes.View` (to find nodes)
+      * `Storage.View` (to find storage)
+      * `Storage.Upload` (to upload the template)
+      * `Datastore.AllocateTemplate` (implicitly used by upload)
+4.  Copy the **Token ID** and **Token Secret** immediately.
+
+-----
+
+## Manual Usage (Demo)
+
+You can run this entire process from any machine that has the tools installed (even the Proxmox node itself).
+
+1.  **Clone the Repository:**
+
+    ```bash
+    git clone <your-repo-url>
+    cd <your-repo-name>
+    ```
+
+2.  **Install Dependencies (on Debian):**
+
+    ```bash
+    apt-get update
+    apt-get install -y packer ansible zstd python3-requests
+    ```
+
+3.  **Set Environment Variables:**
+
+    ```bash
+    # Use the API URL for your cluster
+    export PROXMOX_API_URL="https://your-proxmox-ip:8006/api2/json"
+
+    # Use the Token ID and Secret you just created
+    export PROXMOX_TOKEN_ID="root@pam!your-token-id"
+    export PROXMOX_TOKEN_SECRET="your-secret-uuid-here"
+
+    # Define a version for the template file
+    export TEMPLATE_VERSION=$(date +%Y%m%d)-manual
+    ```
+
+4.  **Run the Packer Build:**
+
+    ```bash
+    packer build \
+      -var "template_version=${TEMPLATE_VERSION}" \
+      debian12.pkr.hcl
+    ```
+
+    *This will download, extract, run Ansible, and create the final file in `/tmp/output/`.*
+
+5.  **Run the Python Upload:**
+
+    ```bash
+    python3 api/proxmox_upload.py \
+      --file /tmp/output/debian12-fungible_${TEMPLATE_VERSION}.tar.xz
+    ```
+
+    *This will upload the file to all nodes in your cluster.*
+
+6.  **Verify:**
+    Log in to your Proxmox GUI. Go to any node's `local` storage and click the **CT Templates** tab. You will see your new template, ready for cloning.

container-creation/intern-phxdc-pve1/packer/api/proxmox_upload.py

@@ -0,0 +1,48 @@
+#!/usr/bin/env python3
+import sys, argparse, warnings
+from proxmox_utils import get_nodes, get_storages, upload_template, choose_default_storage
+
+# Suppress insecure request warnings (e.g., if PROXMOX_API_URL has verify=False)
+warnings.filterwarnings("ignore", category=requests.urllib3.exceptions.InsecureRequestWarning)
+
+def main():
+    parser = argparse.ArgumentParser(description="Upload Proxmox LXC template to all nodes.")
+    parser.add_argument("--file", required=True, help="Path to the .tar.xz template file.")
+    args = parser.parse_args()
+
+    print(f"Starting template upload for: {args.file}")
+
+    try:
+        nodes = get_nodes()
+        if not nodes:
+            print("Error: No Proxmox nodes found.")
+            sys.exit(1)
+
+        print(f"Found nodes: {', '.join(nodes)}")
+
+        for node in nodes:
+            print(f"--- Processing Node: {node} ---")
+            storages_list = get_storages(node)
+
+            # Use the utility function to pick the best 'local' storage
+            storage = choose_default_storage(storages_list)
+
+            if not storage:
+                print(f"Warning: No suitable storage found on node {node}. Skipping.")
+                continue
+
+            print(f"Uploading to {node}:{storage}...")
+            try:
+                result = upload_template(node, storage, args.file)
+                print(f"Successfully uploaded to {node}:{storage}. Task: {result.get('data')}")
+            except Exception as e:
+                print(f"Error uploading to {node}:{storage}: {e}")
+
+    except Exception as e:
+        print(f"An unexpected error occurred: {e}")
+        sys.exit(1)
+
+    print("Template upload process finished.")
+
+if __name__ == "__main__":
+    main()

container-creation/intern-phxdc-pve1/packer/api/proxmox_utils.py

@@ -0,0 +1,49 @@
+# api/proxmox_utils.py
+import os
+import requests
+
+API_URL = os.environ.get("PROXMOX_API_URL")
+TOKEN_ID = os.environ.get("PROXMOX_TOKEN_ID")
+TOKEN_SECRET = os.environ.get("PROXMOX_TOKEN_SECRET")
+
+HEADERS = {
+    "Authorization": f"PVEAPIToken={TOKEN_ID}={TOKEN_SECRET}"
+}
+
+def get_nodes():
+    """Return a list of node names from the Proxmox cluster."""
+    r = requests.get(f"{API_URL}/nodes", headers=HEADERS, verify=False)
+    r.raise_for_status()
+    data = r.json().get("data", [])
+    return [n["node"] for n in data]
+
+def get_storages(node):
+    """Return available storages for a node."""
+    r = requests.get(f"{API_URL}/nodes/{node}/storage", headers=HEADERS, verify=False)
+    r.raise_for_status()
+    return [s["storage"] for s in r.json().get("data", [])]
+
+def upload_template(node, storage, filepath):
+    """Upload a template tarball to a specific node and storage."""
+    with open(filepath, "rb") as f:
+        files = {"content": f}
+        data = {
+            "content": "vztmpl",
+            "filename": os.path.basename(filepath)
+        }
+        r = requests.post(
+            f"{API_URL}/nodes/{node}/storage/{storage}/upload",
+            headers=HEADERS,
+            files=files,
+            data=data,
+            verify=False
+        )
+        r.raise_for_status()
+        return r.json()
+
+def choose_default_storage(storages):
+    """Pick the first local-type storage, or fallback to 'local'."""
+    for s in storages:
+        if s == "local" or "local" in s.lower():
+            return s
+    return storages[0] if storages else None

container-creation/intern-phxdc-pve1/packer/debian12.pkr.hcl

@@ -0,0 +1,58 @@
+packer {
+  required_plugins {
+    ansible = {
+      version = ">=1.1.0"
+      source  = "github.com/hashicorp/ansible"
+    }
+  }
+}
+
+variable "template_name" {
+  default = "debian12-fungible"
+}
+
+source "null" "local_build" {
+  communicator = "none"
+}
+
+build {
+  name    = "debian12-template"
+  sources = ["source.null.local_build"]
+
+  provisioner "shell" {
+    inline = [
+      "mkdir -p /tmp/rootfs",
+      "wget -O /tmp/base.tar.zst http://download.proxmox.com/images/system/debian-12-standard_12.12-1_amd64.tar.zst",
+      "unzstd -d /tmp/base.tar.zst -o /tmp/base.tar",
+      "tar -xf /tmp/base.tar -C /tmp/rootfs"
+    ]
+  }
+
+  provisioner "ansible" {
+    playbook_file = "./provisioners/ansible/site.yml"
+    ansible_env_vars = [
+      "ANSIBLE_CONFIG=./provisioners/ansible/ansible.cfg"
+    ]
+    extra_arguments = [
+      "--connection=chroot",
+      "--inventory", "/tmp/rootfs,",
+    ]
+  }
+
+  provisioner "shell" {
+    inline = [
+      "set -eux", # Good practice from your rocky file
+      "mkdir -p /tmp/output",
+      "cd /tmp/rootfs",
+      # Use the variables to build the filename
+      "tar -cJf /tmp/output/${var.template_name}_${var.template_version}.tar.xz .",
+      "ls -lh /tmp/output"
+    ]
+  }
+
+  variable "template_version" {
+  type    = string
+  default = "latest"
+  }
+
+}