Part 16: Two vaults, one rule
TL;DR
- Bitwarden, self-hosted because I can and recommended to anyone regardless: the human-facing vault, where I keep the credentials I type. It leans on Authentik for sign-in.
- Because of that dependency, the two systems must not trap each other: I don't keep Authentik-only credentials in the vault that needs Authentik to unlock. No circular lock-ins.
- OpenBao is the machine-facing vault: everything I don't want systems reading directly lives there. External Secrets Operator syncs from OpenBao into Kubernetes Secrets, so apps consume ESO output while source of truth stays in the vault.
- Same rule as identity: source of truth is a role, and it belongs to the service built for it.
Behind the door: keys. Two kinds of them, as it turns out, because the credentials a human types and the credentials a machine reads are two different problems wearing similar uniforms.
The one I'd recommend to anyone: Bitwarden
Let me get the disclaimer out of the way early: this one isn't a homelab Compromise, it's just a good product. Bitwarden is the password manager I'd recommend to people who have never heard of a homelab, and the fact that I can self-host it is a bonus stacked on top of a product that stands on its own. Great software that happens to be open source is rare enough that when it shows up, you use it and say thank you.
So there's a Bitwarden running in the cluster. It holds the credentials I type: logins, cards, notes, the stuff of daily life. It's an everyday essential in the most literal sense - I open it several times a day, and if it's down, I notice before any dashboard tells me. That's why it sits on the critical storage class from the fire map, same as mail and Authentik.
And it leans on Authentik to work. Single sign-on through the door from last post, second factor from the same policies. Which is convenient, and convenience creates a trap if you're not paying attention.
The circular lock-in
If my vault requires Authentik, and my Authentik credentials live in the vault, then losing access to either one locks me out of both. Forget the password to the vault: reset mail flows through Authentik, which is fine. Get locked out of Authentik somehow, or lose the hardware key that unlocks it: the vault now holds every credential except the one I need to get back in. A dependency loop, with my entire digital life inside it.
The fix is a rule, and it's boring the way good rules are: never store a system's own access credentials inside a system that depends on that system. Authentik's break-glass credentials, the vault's, the cluster's - they live outside the loop, on paper or in a place no federation can touch. The setup is self-hosted, but the escape hatch is manual, and that's on purpose. The day I need it is the day every automated path is broken anyway.
The machine-facing one: OpenBao
The other half of the problem never gets typed by a human at all. Applications need credentials: database passwords, API tokens, keys for the things that talk to other things. The naïve k8s answer is to put them in Secrets and move on, but a Secret is a config object, not a vault - it lives with the manifests, it gets copied into git-shaped places, and its access story is "whoever can read the cluster, more or less."
Here I have opinions, and one of them is historical: HashiCorp changed the license on Vault, and I don't trust a vendor that moves the goalposts mid-game. The open fork, OpenBao, is what I run instead. It's the machine-facing vault: everything I do not want systems to read directly lives there, with real access control on the way in.
The glue is External Secrets Operator. ESO watches the vault and syncs the right values into Kubernetes Secrets, so an app that speaks plain k8s Secrets keeps speaking plain k8s Secrets - it just receives its credentials from something that actually guards them. The sync lands in the cluster, but the source of truth never does. Rotate a credential in OpenBao and the cluster catches up. Delete the cluster and the vault shrugs: nothing of value was lost, because nothing of value was stored there.
The rule, again
Notice that this is the same rule as last post, wearing different clothes. Authentik is the source of truth for identity because it's built for identity. OpenBao is the source of truth for machine credentials because it's built for that. Bitwarden is where my human-scale credentials live because it's built for that too. Three systems, each doing its one job, none of them borrowing another's role, and the dependency arrows drawn so no loop closes.
The alternative - one tool doing all three jobs badly - is how you end up trusting a generalist with your rotation story and a vendor's roadmap with your license. Ask me how I know HashiCorp's roadmap looks.