Table of Contents
Overview
Proxmox-GitOps implements a self-contained GitOps environment for provisioning and orchestrating Linux Containers (LXC) on Proxmox VE.
Encapsulating infrastructure within an extensible monorepository - recursively resolved from Git submodules at runtime - it provides a comprehensive Infrastructure-as-Code (IaC) abstraction for an entire, automated container-based infrastructure.
Architecture
The architecture is based on a multi-stage pipeline capable of recursively deploying and configuring itself as a self-managed control plane.
Initial bootstrapping is performed via a local Docker environment, with subsequent deployments targeting Proxmox VE.
Core Concepts
Proxmox-GitOps standardizes stateless infrastructure and automates container-based deployment on Proxmox VE.
| Concept | Approach | Reasoning |
|---|---|---|
| Desired State | Monorepository as Single Source of Truth represents the entire infrastructure state. | Deterministic bootstrap from code over version history. |
| Self-Containment | The composite monorepository is pushed to a local container, triggering a pipeline that provisions onto Proxmox. | Fully automated infrastructure deployment mirroring local development. |
| Dynamic Configuration | Imperative logic (e.g. config/recipes/repo.rb) used for dynamic, cross-layer state management. |
Declarative approach intractable for dynamic cross-layer changes (e.g. submodule remote rewriting). |
| Monorepository | Centralizes infrastructure as a single code artifact, utilizing submodules for modular composition. | Provides modular container base; dynamically resolved for container-specific workflow control. |
Design
-
Decoupled Architecture: Containers operate independently, allowing for runtime replacement and detached operation.
-
Headless container configuration: By convention, Ansible is used for provisioning (
community.proxmoxupstream); Cinc (Chef) handles modular, recursive desired state complexity. -
Integrated Baseline: The
baserole standardizes container configuration defaults. Proxmox-GitOps leverages this baseline and built-in infrastructure libraries to deploy itself, establishing a reproducible operational pattern to reuse for containerlibs.
Trade-offs
-
Complexity vs. Autonomy: Self-containment increases complexity to achieve automated bootstrap and reproducible behavior.
-
Git as State Engine: Uses Git as a state engine rather than for versioning in volatile, stateless contexts. Monorepository representation, however, encapsulates the entire infrastructure as a self-contained asset suited for version control.
-
API Token Restriction vs. Automation: With Proxmox 9, stricter privilege separation prevents privileged containers from mounting shares via API token; automation capabilities, however, are mainly within the root user context. As a consequence, root user-based API access takes precedence over token-based authentication.
Usage
Requirements
- Docker
- Proxmox VE 8.4-9.1
- See Getting Started and Wiki for recommendations
Deployment
-
Set Proxmox VE configuration and authorization in
./local/config.json. -
Ensure centralized environment configuration in
globals.json. -
Adjust container resource allocation in
container.env. -
Run
./local/run.shto spin up the local Docker environment.-
Accept the Pull Request at
http://localhost:8080/main/configto deployProxmox-GitOpson Proxmox VE, from which the deployment of the respective container library can then be triggered in the same way. -
If
AUTO_DEPLOYis set totrueinglobals.json, it will automatically deployProxmox-GitOpsand container libraries (/libs) to PVE.
-
Files and Configuration
For configuration, cascading overrides are used to separate infrastructure defaults:
-
Global environment variables can be set in
globals.json. -
container.stage.envis sourced for forked-repository deployments. -
.localfiles can be used to structure versioning (e.g.,globals.local.json,container.local.envor10-assistant.local.caddy).
See Configuration Reference for details.
Lifecycle
Default Pipeline
- run
release: creates and configures a container. - run
main: configures an existing container. - run
snapshot: creates a snapshot leveragingUtils.snapshot. - run
rollback: rolls back configuration changes.
Self-Containment
git clone --recurse-submodules, e.g., for Version-Controlled Mirroring.
-
./local/share/can be used for persistence. -
Backup, Update and Rollback: See Self-Containment, which mirrors the system's architecture, implying lifecycle operations emerge from the principle itself.
Development and Extension
Reusable container definitions are stored in the libs folder.
Getting Started
Copy an example container (like libs/broker or libs/proxy) as a template, or create a new container lib from scratch and follow these steps:
- Add
container.envto your container's root directory (e.g../libs/apache):
IP=192.168.178.42
ID=42
CORES=2
MEMORY=2048
SWAP=512
DISK=local-lvm:8
BOOT=yes
- Add your cookbook to the container definition root:
# libs/apache/recipes/default.rb
package 'apache2'
file '/var/www/html/index.html' do
content "<h1>Hello from #{Env.get(node, 'login')}</h1>"
mode '0644'
owner Default.user # see base/roles/base/tasks/main.yml
group Default.group # each container is configured identically
end
Common.application(self, 'apache2') # provided by convention
-
Add to Monorepository and redeploy.
-
Container
libscan be tested locally by running./local/run.sh [container]:
Environment
- Optionally, use
Env.get()andEnv.set()to access environment variables, initially set by globals.