Proxmox VE

Proxmox VE (Virtual Environment) is a free, open-source server virtualization platform. It is a Type-1 (bare-metal) hypervisor, meaning it runs directly on the physical hardware rather than inside a host operating system. Proxmox VE combines two virtualization technologies under a single web-based management interface:

• KVM (Kernel-based Virtual Machine) — for full hardware virtualization of complete operating systems. See Hardware Assisted Virtualization.
• LXC (Linux Containers) — for lightweight, OS-level virtualization where containers share the host Linux kernel.

The project is developed by Proxmox Server Solutions GmbH, an Austrian company, and is licensed under the GNU AGPL v3.

Why Proxmox exists

Enterprise server virtualization has historically been dominated by VMware’s vSphere/ESXi stack. While powerful, VMware is proprietary and carries significant per-socket or per-core licensing costs that put it out of reach for small businesses, educational institutions, and homelabs.

Proxmox VE was created to provide a comparable feature set on top of a fully open-source stack — Debian Linux, KVM, LXC, and Corosync (a cluster communication engine that provides group membership and reliable messaging between nodes). It offers:

• No per-socket license fees — the base product is free; optional paid support subscriptions exist.
• Full access to the underlying Debian OS — no locked-down appliance.
• A web UI and REST API that rival commercial alternatives.

Adoption accelerated sharply after Broadcom acquired VMware in late 2023 and restructured VMware’s licensing model, eliminating perpetual licenses and bundling products into expensive subscription tiers. Many organizations began evaluating Proxmox as a migration target.

Architecture

Proxmox VE is built on Debian GNU/Linux (currently Debian 12 “Bookworm” for Proxmox VE 8.x). The key components are layered as follows:

┌──────────────────────────────────────────────────────┐
│                   Web UI (:8006/HTTPS)               │
│                   REST API (same port)               │
├──────────────────────────────────────────────────────┤
│  pveproxy / pvedaemon / pvestatd (Proxmox daemons)  │
├──────────────────────────────────────────────────────┤
│    KVM + QEMU         │         LXC                  │
│    (full VMs)         │    (containers)               │
├──────────────────────────────────────────────────────┤
│         Linux kernel (with KVM module)               │
├──────────────────────────────────────────────────────┤
│              Debian GNU/Linux (bare metal)            │
├──────────────────────────────────────────────────────┤
│                    Hardware                           │
│         (CPU with AMD-V or Intel VT-x)               │
└──────────────────────────────────────────────────────┘

Key subsystems

• KVM + QEMU: KVM is a kernel module that turns the Linux kernel into a hypervisor. QEMU (Quick Emulator) runs in userspace and provides device emulation (disk controllers, network cards, display). Together, they give each VM its own virtual hardware and allow guest CPU instructions to execute directly on the physical CPU at near-native speed. See Hardware Assisted Virtualization for the full mechanism.
• LXC: Linux Containers use kernel namespaces and cgroups (control groups — a Linux kernel feature for limiting and isolating resource usage of process groups) to create isolated Linux environments that share the host kernel. No hardware emulation is involved, so overhead is minimal.
• Web UI: Served over HTTPS on port 8006. Provides a dashboard for creating VMs and containers, managing storage, configuring networking, viewing logs, and accessing a built-in VNC/SPICE console for each guest.
• REST API: The same port 8006 exposes a full REST API. Every action available in the web UI can be automated via the API using tools like curl, Ansible, or Terraform.
• Corosync: When multiple Proxmox nodes are joined into a cluster, Corosync handles node discovery, quorum voting (a majority of nodes must be online for the cluster to accept changes), and reliable message delivery between nodes. Quorum prevents split-brain scenarios where two halves of a partitioned cluster make conflicting decisions.

Hardware requirements

The CPU must support hardware virtualization extensions: AMD-V for AMD processors or Intel VT-x for Intel processors. These must be enabled in the BIOS/UEFI firmware. For PCI passthrough (e.g., passing a GPU to a VM), you additionally need IOMMU support: Intel VT-d or AMD-Vi.

How to check if your CPU supports virtualization and IOMMU

Before installing Proxmox (from any Linux live USB or existing Linux install):

# Check CPU virtualization support
grep -c 'vmx' /proc/cpuinfo   # Intel VT-x (result > 0 means supported)
grep -c 'svm' /proc/cpuinfo   # AMD-V / SVM (result > 0 means supported)
 
# Check if IOMMU is active in the kernel
dmesg | grep -i iommu
# Intel: look for "DMAR: IOMMU enabled"
# AMD:   look for "AMD-Vi: Found IOMMU at..."
# If nothing appears, IOMMU may be disabled in BIOS or missing from kernel boot params

If IOMMU is not active, it usually means it is disabled in BIOS/UEFI:

1. Reboot into BIOS/UEFI setup (typically Del, F2, or F12 at boot).
2. Look for a setting named “Intel VT-d”, “AMD IOMMU”, “SVM Mode”, or “IOMMU” — enable it.
3. Add the kernel boot parameter: edit /etc/default/grub, add intel_iommu=on (Intel) or amd_iommu=on (AMD) to GRUB_CMDLINE_LINUX_DEFAULT, run update-grub, and reboot.
4. Verify: dmesg | grep -i iommu should now show the IOMMU is active.

Check your specific CPU model: search for your CPU on ark.intel.com (Intel) or the AMD product page. Look for “VT-d” (Intel) or “AMD-Vi” (AMD) in the features list. Most Intel CPUs since Sandy Bridge (2011) and most AMD CPUs since Phenom II (2009) support IOMMU, but budget chipsets sometimes disable it.

See IOMMU for a full explanation of how IOMMU works, IOMMU groups, and common GPU passthrough pitfalls.

VMs vs LXC containers

Choosing between a full VM (KVM) and an LXC container depends on the workload. They are not interchangeable — each has distinct trade-offs.

Aspect	KVM Virtual Machine	LXC Container
Isolation	Full hardware boundary — own kernel, own virtual devices	Shares host kernel; isolation via namespaces and cgroups
Guest OS	Any OS: Linux, Windows, FreeBSD, etc.	Linux only (must be compatible with host kernel version)
Overhead	Higher — QEMU device emulation, separate kernel in RAM	Near-native — no emulation layer, no extra kernel
Startup time	10-30 seconds (full boot)	1-3 seconds
RAM usage	Each VM runs its own kernel (adds ~200-500 MB baseline)	No extra kernel; only the application’s memory
Hardware passthrough	Yes — GPU, NIC, USB via IOMMU	No — limited device access
Security boundary	Stronger — hypervisor enforces isolation at hardware level	Weaker — kernel vulnerability affects all containers
Use cases	Windows, non-Linux OS, untrusted workloads, GPU compute	DNS, web servers, monitoring, lightweight Linux services

Decision heuristic

Use a KVM VM when:

• The guest needs to run Windows, BSD, or another non-Linux OS.
• You need to pass through a GPU or other PCI device (e.g., for AI/ML workloads or video transcoding).
• The workload is untrusted and you need the strongest possible isolation.

Use an LXC container when:

• The workload is Linux-based and you want minimal resource overhead.
• Fast startup and density matter (many small services on one host).
• You trust the workload (or it runs as unprivileged — Proxmox supports unprivileged LXC containers that map container root to a non-root UID on the host).

Storage options

Proxmox VE supports a wide range of storage backends. The choice affects snapshot capability, thin provisioning, and performance.

Local storage

• ext4 / XFS on LVM: LVM (Logical Volume Manager) provides volume management on top of physical disks. Simple and well-understood. Supports LVM snapshots, but they are copy-on-write at the block level and can degrade performance under heavy write loads.
• ZFS: ZFS (Zettabyte File System) is a combined filesystem and volume manager originally developed by Sun Microsystems. It is popular on Proxmox for several reasons:
  • Checksumming: Every block is checksummed, detecting silent data corruption (bit rot).
  • Copy-on-write snapshots: Snapshots are instantaneous and space-efficient.
  • RAID-Z: ZFS’s software RAID implementation (RAID-Z1 = single parity, RAID-Z2 = double parity, RAID-Z3 = triple parity), eliminating the need for a hardware RAID controller.
  • ARC (Adaptive Replacement Cache): An intelligent caching layer that uses available RAM to cache frequently and recently accessed blocks.
  • Trade-off: ZFS is RAM-hungry. A common guideline is 1 GB of RAM per 1 TB of storage, plus ECC (Error-Correcting Code) RAM is strongly recommended.
• LVM-thin: Thin-provisioned LVM volumes that allocate disk space on demand rather than upfront. Supports efficient snapshots.

Network storage

• NFS (Network File System): A distributed filesystem protocol that allows a client to access files over a network as if they were local. Simple to set up. Used for shared storage across cluster nodes (e.g., ISO images, VM backups).
• iSCSI (Internet Small Computer Systems Interface): A protocol that carries SCSI (Small Computer System Interface) commands over TCP/IP networks, presenting remote block devices as if they were locally attached disks. Lower latency than NFS for block-level workloads.
• Ceph: A distributed storage system providing object, block, and file storage in a single cluster. Proxmox has built-in Ceph integration — you can deploy a Ceph cluster directly from the Proxmox UI across your nodes. Ceph RBD (RADOS Block Device) is the typical backend for VM disks in multi-node clusters, providing replication and self-healing.

Key features

Live migration

A running VM can be moved from one cluster node to another with zero downtime. The mechanism:

1. VM memory pages are copied to the destination node while the VM continues running on the source.
2. Dirty pages (pages modified during the copy) are re-sent iteratively until the delta is small enough.
3. The VM is briefly paused (typically milliseconds), the final dirty pages are transferred, and the VM resumes on the destination.

This requires shared storage (so both nodes can access the VM’s disk) or local storage with Proxmox’s built-in storage migration.

Snapshots and backups

• Snapshots: Capture the VM or container state (disk + optionally RAM) at a point in time. Useful for pre-upgrade rollback. ZFS and LVM-thin provide efficient snapshot implementations.
• Proxmox Backup Server (PBS): A separate product (also free/open-source) purpose-built for backing up Proxmox VE guests. It supports incremental backups with client-side deduplication, encryption, and integrity verification. PBS can be installed on a dedicated machine or even inside a VM on the same Proxmox host for homelabs.

High Availability (HA)

In a cluster of three or more nodes, Proxmox can automatically restart a VM or container on a surviving node if its original node fails. HA requires:

• A cluster with Corosync quorum (minimum 3 nodes, or 2 nodes + a QDevice — a lightweight quorum arbitrator).
• Shared storage accessible from all nodes.
• The VM or container must be marked as an HA resource.

Firewall

Proxmox includes a built-in firewall configurable at the datacenter, node, and individual VM/container level. Rules are managed via the web UI or API and are applied using iptables/nftables on the host.

GPU passthrough (IOMMU)

A physical GPU (or other PCI device) can be passed through to a VM so the guest has direct, exclusive access to the hardware. This is essential for:

• AI/ML workloads using CUDA or ROCm.
• Video transcoding with hardware encoders.
• Windows gaming VMs in a homelab.

Requirements: CPU and motherboard must support IOMMU (Intel VT-d or AMD-Vi), and IOMMU must be enabled in BIOS/UEFI and in the kernel boot parameters (intel_iommu=on or amd_iommu=on).

cloud-init support

cloud-init is an industry-standard tool for automating the initial setup of a VM on first boot (setting hostname, SSH keys, network configuration, running scripts). Proxmox can attach a cloud-init drive to a VM, allowing templated, hands-off VM provisioning — create a template once, clone it, and cloud-init configures each clone with unique settings.

Typical homelab setup

A common homelab configuration runs Proxmox VE on a single mini PC or server, hosting a mix of VMs and LXC containers:

Proxmox VE (bare metal on mini PC, e.g., Intel N100 or AMD Ryzen)
│
├── VM: GitLab Runner (Ubuntu 22.04, 4 CPU, 4 GB RAM)
│       CI/CD runner for personal projects
│
├── VM: Windows 11 (4 CPU, 8 GB RAM, GPU passthrough)
│       For testing or specific Windows-only software
│
├── LXC: Pi-hole (1 CPU, 256 MB RAM)
│       DNS-level ad blocking for the home network
│
├── LXC: Ollama + Open WebUI (4 CPU, 8 GB RAM)
│       Local LLM inference
│
└── LXC: Monitoring (2 CPU, 1 GB RAM)
        Prometheus + Grafana stack

In this layout, the two VMs consume roughly 12 GB of RAM (including their guest kernels), while the three LXC containers together use under 10 GB. The LXC containers start in seconds and share the host kernel, making them ideal for always-on lightweight services.

For a multi-node homelab (3 nodes), you gain HA clustering and can run Ceph across the nodes for replicated storage, eliminating the need for a separate NAS.

See also

• Homelab Monitoring
• IOMMU — required for GPU passthrough; explains IOMMU groups and how to verify support
• AMD-V
• Intel VT
• Hardware Assisted Virtualization
• QEMU
• Virtio