Trying to figure out how to get Fedora CoreOS VMs provisioned on my cluster using . I can't figure out how to do the network configuration using ignition 😓

I spent all day yesterday trying to get to provision some CoreOS and VMs, but I think I'm going to give up on getting Terraform to generate the ignition files, and injecting them through oVirt. I think the oVirt Ignition support is still a bit buggy.

Instead I'm going to just use terraform to build some blank VMs, and PXE boot them and point them to the ignition scripts.

Finally! got a FCOS VM built. It's an ugly process, but it works for now.
Now to repeat a few more times, and then try kubespray'ing it

Kubespray playbook is doing its thing. I'll be quite happy if this works without having to do much troubleshooting.

Yay, kubespray worked without any issues, other than having to reboot the nodes once due to how rpm-ostree installs packages.
Shiny new cluster is running. Using cri-o as well, instead of docker. :)

@mhamzahkhan apparently the way to go is either putting files into the right place or use the default config, maybe you can also use the kernel commandline.

The config spec doesn't mention any network section, which is quite poor :/

@sheogorath I know how to get the ignition file into CoreOS, but the problem I'm having is how to get it to work with terraform. For example, normally I could put a nic_configuration block in my terraform manifest with the static IP to assign, and that is converted to a cloud-init config file, but that doesn't work with ignition. There is an ignition plugin for terraform, but the network config bit isn't "nice" as the cloud-init counter part

@mhamzahkhan oof, yes, that's really poor. Not sure if there is anything nicer.

@mhamzahkhan just out of interest, how do you handle backups in your clusters? It's currently one of my biggest concerns especially since I don't want to develop an own backup mechanism for each application in the cluster.

@sheogorath I'm using to backup K8S resources, which drops them into a S3 bucket. It's supposed to be able to backup persistent volume as well, but as I'm using TrueNAS for my persistent storage, I just create a ZFS snapshot of all the PVs, and send the snapshots to another machine.

Sign in to participate in the conversation

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!