I don’t, and I know that I’m fucked one day.

The whole deployment is done via ansible, so the ansible source is my documentation.

I use the notes sections in proxmox preeettty heavily. Lots of links to the helper scripts, youtube videos and other resources i used to get er’ goin’.

In the near future I’m really hoping I can set up Netbox to help me document the network and equipment I’m putting in my homelab. a nice thing is that I went through a divorce a while ago and I’m getting to start from scratch. You’d be surprised at just how much you’ve learned since starting to self host and I think there’s this sunk cost fallacy that gets a lot of us to keep going with what we’ve got already set up because we’ve “already put so much work into it” and the concern of what we might lose by scrapping it and starting over.

Also, not what you asked… but if you’re still relatively new with proxmox you should check out the ProxmoxVE helper scripts. Lots of good automated scripts from doing a post-install to setting up various LXC containers and VM’s

NixOS’s declarative configurations basically document themsleves: add some comments and you’re good to go and can back then up to wherever whenever

I’ve moved to an “infrastructure as code” approach, not using any fancy tools in particular, primarily just bash shell scripts. Basically almost everything I setup or do gets documented via shell scripts, I write them as I go when I’m learning to install something new, and before I commit to something to new, I take extra care to make sure the scripts are idempotent so that when I want to do make any changes, all I need to do is add it to the appropriate script and re-run it.

The idempotent part takes some effort sometimes, but is not actually as hard as it seems, particularly if you don’t mind that it sometimes spends some wasted time doing things that have already been done, and occasionally spits out some harmless error messages because something is already done, but I also try to minimize that when I can. The consequences of doing too much by re-running are rarely serious. Yeah sometimes the scripts can break, but as long as they fail properly (set -euo pipefail) it’s usually pretty obvious how to fix it and it won’t leave too much of a mess.

Doing this has transformed my homelab from a mess of unknowable higgledy-piggledy spaghetti-services that was always teetering one small failure away from total collapse and frantic rebuilding, into something repeatable and reproducible that I can actually … wait for it … test. Just firing up a Linux ISO in a VM is all I need to test everything I’m doing in a perfect sandbox, and I can throw it away when I’m done with no regrets. Plus it makes rolling out new servers, and more importantly, decommissioning old ones, a breeze, you know exactly what’s on them and how it was set up, because it was all in your scripts. Combined with good data backups (which are also set up in the scripts) and restores (which I also test with scripts) it really takes the drama and stress out of migrations and even hardware failures.

Yeah there are probably easier ways to accomplish what I’m doing using some of the technologies like terraform, ansible and nix/flake that people have mentioned, and I’ve dabbled with those, but for me, the shell script approach strikes a nice balance of not just documenting but also learning the process myself so that I understand enough of what it’s doing to effectively debug it when something goes wrong, and it works on almost everything and in most cases requires no installation or setup. Bash is everywhere. I even have an infrastructure-as-code setup for my Steam Deck to install stuff and get it set up the way I want.

I download the YouTube tutorials I followed, upload them to my UAT Jellyfin server, and then when my server is having issues I can’t get to the videos!

A flawless system really

I have NixOS running on mine and I write everything in comments in my configuration.nix. Usually I’ll write notes about why I set a certain option above the line where I do it but I also have a section at the start that describes anything to be aware of if I’m moving this configuration to another machine. For example “This assumes a drive is mounted on /mnt/backup0”

I build my infrastructure with the terraform, Ansible and helm charts. The code is it’s own documentation as well as comments in that code explaining why I’ve done things if it’s not obvious.

Jokes aside, I do keep some harder to remember stuff written down in a README.md in my repo, but mainly most things are undocumented