Here is a more or less automated install for root on zfs. Need at least three hdds preferably in an hba and can withstand the loss of at least one drive.

https://github.com/Reddimes/ubuntu-zfsraid10/tree/debian-raidz1

There is also an Ubuntu ZFS RAID 10 branch.

I gotta give LibreElec a 👍. If your tv can do HDMI control, then using the remote to control Kodi works by default.

Jellyfish plugin works incredibly easily too. YouTube, not so much. Google API is not up to spec.

I would like to put in something that I over heard about the pi5, it apparently can run Android tv. Which would solve all your requirements.

Heads up, this is going to be an incredibly detailed comment, sorry. So, at the time I stood up that cluster, it was not in Ceph. I had setup the host to run Ubuntu 24.04 with Root on ZFS and the host was simply connected to itself via NFS.

Here is the Github I created for the Root on ZFS installation, I’m not sure if you are familiar with ZFS but it is an incredibly feature rich filesystem. Similar to BTRFS, you can take snapshots of the server so basically if your host goes down you have a backup at least. On top of that, you get L2ARC caching. Basically, any time it reads or writes to my zpool, that is handled in the background by my NVMe SSD. It also caches the most frequently used files so that it doesn’t have to read from a HDD everytime. I will admit that ZFS does use a lot of memory, but the L2ARC kinda saved me from that on this server.

Ultimately that cluster was not connected to CEPH, but simply NFS. Still, I created a Github repository which is basically just one command to get Ubuntu 24.04 installed with Root on ZFS. https://github.com/Reddimes/ubuntu-zfsraid10. Its not prefect, if it seems like it is frozen, just hit enter a couple times, I don’t know where it is getting hung up and I’m too lazy to figure it out. After that, I followed this guide for turning it into a Cloudstack Host: https://rohityadav.cloud/blog/cloudstack-kvm/.

That was my initial setup. But now I have it setup significantly differently. I rebuilt my host, installed Ubuntu 24.04 to my NVMe drive this time. Did some fairly basic setup with Cephadm to deploy the osds. After the OSD’s were deployed, I followed this guide for getting it setup with cloudstack: https://www.shapeblue.com/ceph-and-cloudstack-part-1/. The only other issue is that you do need a secondary storage server as well. I’ve personally decided to use NFS for that similar to my original setup. Now Ceph does use a LOT of memory. It is currently the only thing running on my host and I’ve attached a screenshot. 77GB!!! OoooWeee… A bit high. Admittedly, this is likely because I am not running just the Rados image store, but also an *arr stack in cephfs on it. And though I have 12 HDDS, some of them have smart poweron time exceeding 7 years. So ignore the scrubbing, please.

I do potentially see some issues, with ceph, the data is supposed to be redundant, but I’ve only provided one ip for it for the moment until I figure out the issues I’m having with my other server. That is some exploration that I’ve not done yet.

Finally takes a breath Anyways, the reason I choose Cloudstack was to delve into the DevOps space a little bit except home built and self-hosted. It is meant to be quite large, and be used by actual cloud providers. In fact, it is meant to have actual public IP addresses which get assigned to the Centos Firewalls that it creates for each network. In a homelab, I had to get a little creative and setup a “public” network on a vlan controlled by my hardware firewall. This does mean that if I actually want something to be public that I need to actually forward it from my hardware firewall, but otherwise, no issue. Going back to the DevOps learning path, not only can you set up linux servers with cloud-init user data, but Terraform works by default and it acts quite similar to Terraform and AWS.

The thing that is interesting about K8S deployments is that it is just the click of a single button. Sure, first you have to download the iso, or build your own with the built-in script, but Cloudstack manipulates the cloud-init user data of each node in the cluster to set it up automatically whether it is a control node, or a worker node. After that, you do need to update the virtual machines running it. I’m sure there is a proper way to use Ansible, but I’ve run into a couple of issue with it and did it manually via ssh.

Edit: Yes, those nodes were all VMs.

Yeah my storage was beefed up at the time. Zfs raid 10. But I’ve changed to using ceph for shared redundant storage.

I’m a little curious what you are using for a hypervisor. I’m using Apache Cloudstack. Apache Cloudstack had a lot of the same features as AWS and Azure. Basically, I have 1000 vlans prepared to stand up virtual networking. Cloudstack uses Centos to stand up virtual firewalls for the ones in use. These firewalls not only handle firewall rules, but can also do load balancing which I use for k8s. You can also make the networks HA by just checking a box when you stand it up. This runs a second firewall that only kicks in if the main one stops responding. The very reason I used Cloudstack was because of how easy it is to setup a k8s cluster. Biggest cluster I’ve stood up is 2 control nodes and 25 worker nodes, it took 12 minutes to deploy.

Yeah, those data questions are really loaded. I don’t host for privacy or what not. It’s because of a learning objective, to study, experiment, and run automated stock trading algorithms. I don’t exactly have anything to hide from private companies.