Every migration story you read online starts at the destination. Here's the Talos config, here's the cluster, here's the beautiful YAML. What gets skipped is the part that actually explains the decision: what the thing looked like before, and what specifically made the owner finally snap. So before I show you any metal, you get the archaeology - the Proxmox estate I was actually running, the drag it generated, and why the answer turned out to be an immutable, API-driven, deliberately minimal Linux I'd never heard of three months before I installed it.

And the story is far more embarrassing, frankly, but the lessons are the reward in their own way.

What I actually had

The honest inventory of my homelab circa "before": a couple of Proxmox hosts running Fedora VMs, and inside those VMs, Docker - deployed the way most people deploy Docker inside VMs, which is docker-compose up -d over SSH, with the container configs living partially in git, partially in /etc, and mostly in the one place that always loses: my memory. What flags did I pass to that container in 2022? Good question. The container knows. Ask it after the next reboot.

And this wasn't laziness on my part - it's what the software ecosystem made the setup be. Ten years ago, a homelab sysadmin installed services from packages. .deb, .rpm, systemd unit shipped by the maintainer, apt install, done. The distro was the packaging layer, and the packaging layer was someone else's job. Today, almost everything self-hostable ships as a Docker image and only as a Docker image. Which means the sysadmin job hasn't shrunk - it's just moved. Now you are the packaging layer: you write the compose file, you map the volumes, you wire it into systemd (or don't, and let the Docker daemon be your init system, which is its own genre of regret), you handle the update cadence by hand. Every new app is a small packaging project, and every one of those projects is another piece of hand-maintained glue that will silently rot.

Backups, meanwhile, were real but asymmetric. The databases were covered - proper snapshots, taken on a schedule, tested more than never. What was not covered was the system around them: the OS state, the compose files, the eighteen months of small undocumented decisions. I could restore my data. What I could not restore was the machine that served it, because that machine was a sediment layer, and sediment doesn't restore.

If any of this sounds familiar, congratulations, you have a normal homelab. None of it was broken in the "things are down" sense. It was broken in the worse sense: it worked, and the price of keeping it working was me being a full-time, unpaid SRE for a platform nobody but me used.

The real drag

Homelab drag doesn't announce itself. It accumulates the way silt does - each individual deposit is trivial.

It's the Friday night you spend upgrading containerd on one VM because you vaguely remember an advisory, and by the end you're three VMs divergent with no idea which version is running where. It's wanting to try something new - a proper observability stack, say - and realizing the cost isn't learning the new thing, it's re-hand-integrating it with the old sediment. It's apt upgrade on a VM you forgot about, watching it reboot, and feeling genuine dread because you don't actually know what lived on that box anymore. It's the snowflake problem in its purest form: every host is special, and every host being special means every host is a liability.

The real killer, though, was a subtler one. I was a professional platform person spending evenings doing artisanal, hand-crafted infrastructure work - the kind of thing I'd flag in a customer's environment review, gently, with a diagram. I had built, at home, the exact setup I spent my day job walking customers away from: a system whose reliability depended on one specific human remembering things. I was the snowflake. The homelab wasn't under version control; I was the version control, and I had no rollback strategy.

And what made it a conviction rather than a nagging feeling: Kubernetes fixes the interesting half of this - the scheduling, the healing, the declarative state - but it does nothing about the host underneath. A hand-rolled Ubuntu VM running Kubernetes is still a hand-rolled Ubuntu VM. It still drifts. It still has eighteen months of undocumented apt decisions in it. The old joke is that the hardest part of Kubernetes isn't Kubernetes, it's Linux - and if your Linux is artisanal, you've just put a very sophisticated scheduler on top of a very unsophisticated base and hoped the base never embarrasses you. It will. It's Linux. That's the deal.

Enter Talos

While it has drawbacks, getting on the Kubernetes train late also has its advantages: tens of projects, with the successful ones surviving. And lots of presentations provided me plenty of insights on what I should now be looking for. And this is how, I went looking for an operating system that behaves like infrastructure instead of like a pet. The requirements were simple but quite blunt:

1. The host is not mine to mess with. If I can SSH in and hand-edit things, I will, and in three months it's sediment again. I needed an OS where that door is welded shut - not by policy, by design.
2. Everything is declared somewhere, and "somewhere" is git. The node's state should be a file I can diff, not a history I have to excavate.
3. Upgrades should be boring. Atomic, image-based, testable - closer to "flash a new appliance firmware" than "apt dist-upgrade and pray."
4. Minimal surface. If the host is only ever a Kubernetes node, it doesn't need a package manager, a shell, or opinions. Every extra component is another advisory to track forever.

Talos Linux is basically those four requirements turned into a product. It's an immutable, minimal, API-only Linux that runs Kubernetes and nothing else - no SSH, no shell, no package manager. You don't administer a Talos node; you talk to it over an API, and the node's entire configuration is a YAML file. Want to change something? Edit the file, apply it, and the node converges. Want to upgrade? Push a new image version - the OS is itself a container image, more or less, which is a pleasing bit of recursion. There is no state on the node worth preserving, which means there is no state on the node worth fearing.

But here's the part I want to underline, because it's the part everyone skips: I didn't put it on metal. Not first. I installed Talos inside my existing Proxmox VMs - same hosts, same VLANs, same network plumbing, everything reused. The VMs were my training wheels, deliberately.

The reason is boring and physical. I don't own a KVM. I don't own a portable monitor. My "cluster" lives on hardware in a corner, and the moment something goes wrong at the hardware layer - a bad boot, a bricked config, an EFI quirk - the recovery procedure is: unplug the machine, carry it to a desk, find a monitor and a keyboard, sit there with a USB stick like it's 2004. Every bare-metal mistake is a field trip. A VM mistake is a qm destroy and ten seconds. When you're learning a new OS - and Talos makes you relearn things, from disk layout to how you even read logs - you want your mistakes to be cheap, repeatable, and performed from a chair. Proxmox snapshots gave me exactly that: I could break a Talos VM, roll it back, and break it again differently, all without standing up once.

So the graduation path was deliberate: prove the cluster design inside the VMs - the control plane, the CNI, the config surface, the upgrade path - and only move to bare metal once the process was boring. If you're planning the same jump, steal that ordering. The VM phase isn't a detour; it's the rehearsal for the part where mistakes cost a desk trip.

The security posture was what sold me, honestly. I'll write a whole post on it later, but the short version: my prior homelab's attack surface included every box I'd ever forgotten about. Talos' attack surface is an API endpoint, mTLS, and a kernel that ships without a shell to abuse. For someone who cares about data sovereignty and runs his own IAM and email on bare metal, an OS that structurally can't be casually logged into is not a nice-to-have. It's the point.

I want to be fair to Proxmox here, because Proxmox is good software and plenty of people run beautiful clusters on it. The failure wasn't Proxmox's. The failure was mine: Proxmox gives you more ways to touch the system by hand, and I had demonstrated over several years that when given those ways, I used them. The fix wasn't a better hypervisor. The fix was removing the temptation - making the drift physically impossible rather than merely discouraged. (And to be clear: Proxmox stayed. It's a fine tool. It just stopped being the substrate of the thing I actually cared about.)

And so there we are

This is the blog's spine showing up in a specific decision. Bare-metal Kubernetes is worth it to me because it trades a kind of convenience I was abusing for a kind of control I actually wanted - and Talos is the same trade one layer down. Mutable OS with a shell: convenient, in the way credit cards are convenient. Immutable OS with an API: a little friction up front, and in exchange the system can't quietly become something you no longer understand. That's the whole deal. Power and sovereignty, paid for in time and patience.