-
Notifications
You must be signed in to change notification settings - Fork 2
DRAFT: Add Packer build and upload workflow for Proxmox templates #80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
cmyers-mieweb
wants to merge
6
commits into
main
Choose a base branch
from
cmyers_issue63
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
ca05763
DRAFT: Add Packer build and upload workflow for Proxmox templates
28b0c52
Add README and update Rocky9 template name
ff63249
Migrate Proxmox template upload API from Python to Node.js
4f53869
Enforce SSL verification for Proxmox API requests
3975740
Move packer files to top-level directories
6d61e47
Merge branch 'main' into cmyers_issue63
cmyers-mieweb File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next
Next commit
DRAFT: Add Packer build and upload workflow for Proxmox templates
Introduces a draft GitHub Actions workflow to build and upload Debian 12 and Rocky 9 LXC templates for Proxmox. Adds supporting Packer HCL files, Ansible provisioning playbook, Python API utilities for uploading templates, and variable files for template customization.
- Loading branch information
There are no files selected for viewing
45 changes: 45 additions & 0 deletions
45
container-creation/intern-phxdc-pve1/packer/.github/workflows/build-templates.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| name: Build and Upload Templates | ||
| on: | ||
| schedule: | ||
| - cron: "0 4 * * *" # Nightly at 4 AM | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| PROXMOX_API_URL: ${{ secrets.PROXMOX_API_URL }} | ||
| PROXMOX_TOKEN_ID: ${{ secrets.PROXMOX_TOKEN_ID }} | ||
| PROXMOX_TOKEN_SECRET: ${{ secrets.PROXMOX_TOKEN_SECRET }} | ||
| # Define the template version ONCE for the whole job | ||
| TEMPLATE_VERSION: ${{ format('{0:yyyyMMdd}', github.run_started_at) }} | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install dependencies | ||
| run: sudo apt-get update && sudo apt-get install -y packer ansible zstd python3-requests | ||
|
|
||
| # --- Build Debian 12 --- | ||
| - name: Build Debian 12 Template | ||
| run: | | ||
| packer build \ | ||
| -var "template_version=${TEMPLATE_VERSION}" \ | ||
| debian12.pkr.hcl | ||
|
|
||
| - name: Upload Debian 12 Template | ||
| run: | | ||
| python3 api/proxmox_upload.py \ | ||
| --file /tmp/output/debian12-fungible_${TEMPLATE_VERSION}.tar.xz | ||
|
|
||
| # --- Build Rocky 9 --- | ||
| - name: Build Rocky 9 Template | ||
| run: | | ||
| packer build \ | ||
| -var "template_version=${TEMPLATE_VERSION}" \ | ||
| rocky9.pkr.hcl | ||
|
|
||
| - name: Upload Rocky 9 Template | ||
| run: | | ||
| python3 api/proxmox_upload.py \ | ||
| --file /tmp/output/rocky9-fungible_${TEMPLATE_VERSION}.tar.xz |
48 changes: 48 additions & 0 deletions
48
container-creation/intern-phxdc-pve1/packer/api/proxmox_upload.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| #!/usr/bin/env python3 | ||
| import sys, argparse, warnings | ||
| from proxmox_utils import get_nodes, get_storages, upload_template, choose_default_storage | ||
|
|
||
| # Suppress insecure request warnings (e.g., if PROXMOX_API_URL has verify=False) | ||
| warnings.filterwarnings("ignore", category=requests.urllib3.exceptions.InsecureRequestWarning) | ||
|
|
||
| def main(): | ||
| parser = argparse.ArgumentParser(description="Upload Proxmox LXC template to all nodes.") | ||
| parser.add_argument("--file", required=True, help="Path to the .tar.xz template file.") | ||
| args = parser.parse_args() | ||
|
|
||
| print(f"Starting template upload for: {args.file}") | ||
|
|
||
| try: | ||
| nodes = get_nodes() | ||
| if not nodes: | ||
| print("Error: No Proxmox nodes found.") | ||
| sys.exit(1) | ||
|
|
||
| print(f"Found nodes: {', '.join(nodes)}") | ||
|
|
||
| for node in nodes: | ||
| print(f"--- Processing Node: {node} ---") | ||
| storages_list = get_storages(node) | ||
|
|
||
| # Use the utility function to pick the best 'local' storage | ||
| storage = choose_default_storage(storages_list) | ||
|
|
||
| if not storage: | ||
| print(f"Warning: No suitable storage found on node {node}. Skipping.") | ||
| continue | ||
|
|
||
| print(f"Uploading to {node}:{storage}...") | ||
| try: | ||
| result = upload_template(node, storage, args.file) | ||
| print(f"Successfully uploaded to {node}:{storage}. Task: {result.get('data')}") | ||
| except Exception as e: | ||
| print(f"Error uploading to {node}:{storage}: {e}") | ||
|
|
||
| except Exception as e: | ||
| print(f"An unexpected error occurred: {e}") | ||
| sys.exit(1) | ||
|
|
||
| print("Template upload process finished.") | ||
|
|
||
| if __name__ == "__main__": | ||
| main() |
49 changes: 49 additions & 0 deletions
49
container-creation/intern-phxdc-pve1/packer/api/proxmox_utils.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| # api/proxmox_utils.py | ||
| import os | ||
| import requests | ||
|
|
||
| API_URL = os.environ.get("PROXMOX_API_URL") | ||
| TOKEN_ID = os.environ.get("PROXMOX_TOKEN_ID") | ||
| TOKEN_SECRET = os.environ.get("PROXMOX_TOKEN_SECRET") | ||
|
|
||
| HEADERS = { | ||
| "Authorization": f"PVEAPIToken={TOKEN_ID}={TOKEN_SECRET}" | ||
| } | ||
|
|
||
| def get_nodes(): | ||
| """Return a list of node names from the Proxmox cluster.""" | ||
| r = requests.get(f"{API_URL}/nodes", headers=HEADERS, verify=False) | ||
| r.raise_for_status() | ||
| data = r.json().get("data", []) | ||
| return [n["node"] for n in data] | ||
|
|
||
| def get_storages(node): | ||
| """Return available storages for a node.""" | ||
| r = requests.get(f"{API_URL}/nodes/{node}/storage", headers=HEADERS, verify=False) | ||
| r.raise_for_status() | ||
| return [s["storage"] for s in r.json().get("data", [])] | ||
|
|
||
| def upload_template(node, storage, filepath): | ||
| """Upload a template tarball to a specific node and storage.""" | ||
| with open(filepath, "rb") as f: | ||
| files = {"content": f} | ||
| data = { | ||
| "content": "vztmpl", | ||
| "filename": os.path.basename(filepath) | ||
| } | ||
| r = requests.post( | ||
| f"{API_URL}/nodes/{node}/storage/{storage}/upload", | ||
| headers=HEADERS, | ||
| files=files, | ||
| data=data, | ||
| verify=False | ||
| ) | ||
| r.raise_for_status() | ||
| return r.json() | ||
|
|
||
| def choose_default_storage(storages): | ||
| """Pick the first local-type storage, or fallback to 'local'.""" | ||
| for s in storages: | ||
| if s == "local" or "local" in s.lower(): | ||
runleveldev marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| return s | ||
| return storages[0] if storages else None | ||
58 changes: 58 additions & 0 deletions
58
container-creation/intern-phxdc-pve1/packer/debian12.pkr.hcl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| packer { | ||
| required_plugins { | ||
| ansible = { | ||
| version = ">=1.1.0" | ||
| source = "github.com/hashicorp/ansible" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| variable "template_name" { | ||
| default = "debian12-fungible" | ||
| } | ||
|
|
||
| source "null" "local_build" { | ||
| communicator = "none" | ||
| } | ||
|
|
||
| build { | ||
| name = "debian12-template" | ||
| sources = ["source.null.local_build"] | ||
|
|
||
| provisioner "shell" { | ||
| inline = [ | ||
| "mkdir -p /tmp/rootfs", | ||
| "wget -O /tmp/base.tar.zst http://download.proxmox.com/images/system/debian-12-standard_12.12-1_amd64.tar.zst", | ||
| "unzstd -d /tmp/base.tar.zst -o /tmp/base.tar", | ||
| "tar -xf /tmp/base.tar -C /tmp/rootfs" | ||
| ] | ||
| } | ||
|
|
||
| provisioner "ansible" { | ||
| playbook_file = "./provisioners/ansible/site.yml" | ||
| ansible_env_vars = [ | ||
| "ANSIBLE_CONFIG=./provisioners/ansible/ansible.cfg" | ||
| ] | ||
| extra_arguments = [ | ||
| "--connection=chroot", | ||
| "--inventory", "/tmp/rootfs,", | ||
| ] | ||
| } | ||
|
|
||
| provisioner "shell" { | ||
| inline = [ | ||
| "set -eux", # Good practice from your rocky file | ||
| "mkdir -p /tmp/output", | ||
| "cd /tmp/rootfs", | ||
| # Use the variables to build the filename | ||
| "tar -cJf /tmp/output/${var.template_name}_${var.template_version}.tar.xz .", | ||
| "ls -lh /tmp/output" | ||
| ] | ||
| } | ||
|
|
||
| variable "template_version" { | ||
| type = string | ||
| default = "latest" | ||
| } | ||
|
|
||
| } |
56 changes: 56 additions & 0 deletions
56
container-creation/intern-phxdc-pve1/packer/provisioners/ansible/site.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # packer/provisioners/ansible/site.yml | ||
| --- | ||
| - name: Apply base fungible configuration | ||
| hosts: all | ||
| gather_facts: yes | ||
|
|
||
| tasks: | ||
| - name: Ensure common packages are installed | ||
| ansible.builtin.package: | ||
| name: | ||
| - vim | ||
| - curl | ||
| - sudo | ||
| - wget | ||
| - ca-certificates | ||
| - nano | ||
| - git | ||
| state: present | ||
|
|
||
| - name: Set up /etc/motd branding | ||
| ansible.builtin.copy: | ||
| dest: /etc/motd | ||
| content: | | ||
| This will be replaced by logic to deploy container information. | ||
| This container image was built automatically via Packer and Ansible. | ||
runleveldev marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| - name: Configure system DNS (common) | ||
| ansible.builtin.copy: | ||
| dest: /etc/resolv.conf | ||
| content: | | ||
| nameserver 1.1.1.1 | ||
| nameserver 8.8.8.8 | ||
runleveldev marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| when: ansible_os_family in ['Debian', 'RedHat'] | ||
|
|
||
| - name: Copy pown.sh (disabled until first boot) | ||
| ansible.builtin.copy: | ||
| src: files/pown.sh | ||
| dest: /usr/local/bin/pown.sh | ||
| mode: '0755' | ||
|
|
||
| - name: Copy Wazuh placeholder registration script | ||
| ansible.builtin.copy: | ||
| dest: /etc/wazuh-agent/init_disabled | ||
| content: | | ||
| #!/bin/bash | ||
| # Registration disabled in build context; runs on first boot only. | ||
runleveldev marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| mode: '0755' | ||
|
|
||
| - name: Clean temporary or build-specific files | ||
| ansible.builtin.file: | ||
| path: "{{ item }}" | ||
| state: absent | ||
| loop: | ||
| - /var/lib/apt/lists/* | ||
| - /var/cache/dnf | ||
| - /tmp/* | ||
60 changes: 60 additions & 0 deletions
60
container-creation/intern-phxdc-pve1/packer/rocky9.pkr.hcl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| packer { | ||
| required_plugins { | ||
| ansible = { | ||
| version = ">=1.1.0" | ||
| source = "github.com/hashicorp/ansible" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| variable "template_name" { | ||
| default = "rocky9-lxc" | ||
| } | ||
|
|
||
| source "null" "local_build" { | ||
| communicator = "none" | ||
| } | ||
|
|
||
| build { | ||
| name = "rocky9-template" | ||
| sources = ["source.null.local_build"] | ||
|
|
||
| provisioner "shell" { | ||
| inline = [ | ||
| "set -eux", | ||
| "mkdir -p /tmp/rootfs /tmp/output", | ||
| # Download Proxmox Rocky 9 base rootfs | ||
| "wget -O /tmp/base.tar.zst http://download.proxmox.com/images/system/rockylinux-9-standard_9.4-1_amd64.tar.zst", | ||
| # Extract base | ||
| "unzstd -d /tmp/base.tar.zst -o /tmp/base.tar", | ||
| "tar -xf /tmp/base.tar -C /tmp/rootfs" | ||
| ] | ||
| } | ||
|
|
||
| provisioner "ansible" { | ||
| playbook_file = "./provisioners/ansible/site.yml" | ||
| ansible_env_vars = [ | ||
| "ANSIBLE_CONFIG=./provisioners/ansible/ansible.cfg" | ||
| ] | ||
| extra_arguments = [ | ||
| "--connection=chroot", | ||
| "--inventory", "/tmp/rootfs,", | ||
| ] | ||
| } | ||
|
|
||
| provisioner "shell" { | ||
| inline = [ | ||
| "set -eux", | ||
| "cd /tmp/rootfs", | ||
| "tar -cJf /tmp/output/${var.template_name}_$(date +%Y%m%d).tar.xz .", | ||
| "ls -lh /tmp/output" | ||
| ] | ||
| } | ||
|
|
||
| post-processor "shell-local" { | ||
| inline = [ | ||
| "echo '✅ Rocky 9 LXC rootfs built successfully'", | ||
| "ls -lh /tmp/output" | ||
| ] | ||
| } | ||
| } |
4 changes: 4 additions & 0 deletions
4
container-creation/intern-phxdc-pve1/packer/vars/debian12.auto.pkrvars.hcl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| debian_release = "bookworm" | ||
| template_name = "debian12-lxc" | ||
| rootfs_dir = "/tmp/debian12-rootfs" | ||
| hostname = "template-debian12" |
1 change: 1 addition & 0 deletions
1
container-creation/intern-phxdc-pve1/packer/vars/rocky9.auto.pkrvars.hcl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| template_name = "rocky9-fungible" |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.