It’s not an either-or scenario. Running services in Docker/Podman is great and makes a lot of sense, as you’ve found. But there’s no reason the OS running those Docker containers can’t be a VM on a hypervisor like Proxmox. Then you get the simplicity of Docker, in addition to the isolation and segmentation (network and process) provided by VMs, and snapshot-based incremental backups from PBS. It’s the best of both worlds. You wouldn’t have a VM per service like you ran before, instead you’d have a VM per group of related services with common networking and security requirements. For example, all of your publicly exposed services can run in Docker in their own isolated VM that’s walled off from the rest of your network, while your internal-only services also run in Docker, but on a separate VM on your internal network.

I’ll die on this hill.

Containers run on “bare metal” in the same way that other processes on your system run.

I like the flexibility that proxmox provides me. I do this as a hobby and I’m self taught. I can try a bunch of things and if I mess up I can start over without much hassle.

Proxmox running LXC’s and VM’s, all of them also have docker installed

Security is a big reason to use VMs. Containers share the kernel with the host. That means that any of the kernel vulnerabilities from this year (like DirtyFrag and CopyFail) could have been used to compromise your host. Once the host is compromised, the only way to be safe is to literally buy a new machine. Seriously. Viruses can bury deep and even infect the motherboard firmware to persist indefinitely.

Kernel vulnerabilities are frequent enough that I find VMs worth it. I still use containers in my VMs though.

I like Proxmox for it’s first-class treatment of ZFS. You can easily run Docker/Podman on top of it or in a VM. Proxmox has some other advantages like replication and clustering, but I’d say you don’t need that in a home setup (at least I don’t).

If I had to redo my setup today, I’d probably give TrueNAS SCALE a chance though.

I use Proxmox with a VM for each service I provide and I still use docker compose.

This way I have nice VM backups (with Proxmox BS) + dockerized services deployed via GitLab CI pipelines.

Proxmox with LXC’s and a few VM’s.
Often with Portainer as well.

It’s just nice to use and easy to automate backups.

I used to run barebone on Debian, but I accepted that I’m not good enough in the terminal and to used to graphical solutions to go back.

I moved away from proxmox about 3 years ago to incus, primarily because Proxmox has a lot going on that I don’t need, but also incus is much more amenable to configuration with ansible. Also, and maybe this has changed recently, but passing through PCI devices to incus is much easier.

When I got started I saw a lot of suggestions for proxmox. For me, it was an extra layer I didn’t feel like I needed. I installed Open Media Vault (just Debian with a web interface) and I have my docker containers for running the apps and eventually setup VMs for various other things. Staring out, Docker was a bit of a headache but now that I’ve got practice with it docker compose is fairly simple.

I appreciate that these days there are a bunch of really good tools out there and options.

I use Proxmox as a VDI server. I understand that this isn’t homelab territory, but selfhosted VDIs have proven exponentially more reliable and easier to manage than cloud based ones (after the initial PitA setup). Flawless copy/paste, screen resize, and most importantly, file transfers. When you connect to 12 different clients, each with a different set of security requirements, system hardening, monitoring, and VPN access, being able to console amd fully interact with a sandboxed desktop becomes priceless.

Proxmox exclusively with unprivileged LXCs, which themselves host other docker containers/stacks managed with Dockhand (a more modern alternative to both Portainer and Dockge). I have a very tiny low power machine so I have to be kind with my resources, therefore no VMs. But LXCs are simpler anyway. I can easily pass devices like /dev/tun for tailscale and i forget the name of the iGPU for accelerated workloads (for Immich, Nextcloud).

Reasoning: with containers it’s easy to make mistakes without dire consequences (just docker compose down -v and start over). The LXCs are easy to back up and restore. They hold internal running state of whatever docker containers I run in them.

Pro tip: don’t map host paths using the UI (mp0 etc), use lxc.mount instead. This will let you continue to have snapshots of your LXCs whereas the other approach makes your LXC incompatible with snapshots.

Good luck!

Having to manually start and stop your VMs is very atypical, proxmox can absolutely handle that itself

because sometimes i need a vm (home assisstant has more features thsn home assisstant container), sometimes a container (i mostly use lxc).

its not really an extra lauer because proxmox is an os.you use nixos i use proxmox. everything is out of the box. proxmox has comminntiu scripts to run everything and set them up. there is an i stall script for about everything https://community-scripts.org/

sure podman has similar setup but i also need vms and they are all mamaged in one place

EDIT: you can read the script, its text

There are different reasons to run different things. Some apps are not able to be run as containers, or don’t make sense to run containerized.

VMs offer better isolation between systems, which is important for high-security applications.

Traditional Hypervisors also allow you to easily run fleets of completely different OSes on the same hardware, which you can’t do with containers.

On the other hand, if you want/need high scalability, extremely efficient resource usage, and you don’t need to run radically different OS environments simultaneously, containers, especially Docker containers or Podman containers are a great choice.

Incus containers are a great middle ground that I’m a fan of more than Docker/podman containers or traditional VMs. They are system containers, vs application containers like Docker/Podman.

That means they are actually full-fledged Linux distros by default, but are still much smaller than a traditional VM.

Also, for any of these solutions, if you’re constantly doing admin tasks manually, you’re doing it wrong. All modern platforms have APIs or other ways to interact with them via code.

For my own setup, I have everything. My main server runs XCP-ng for the hypervisor, and I have several traditional VMs on it, including my primary NAS. I also run a virtual Incus host that I have my Minecraft server running in, and some random other system containers. I have a Docker host VM, but I don’t really use it.

I also have my home media server which is a standalone system running TrueNAS on bare metal with Tailscale and Jellyfin running as Docker containers within TrueNAS.

My setup is messy because it has been built slowly over several years, and I don’t have enough time or energy to rip it all down and put it back together in a more optimal way.