CLOUD NATIVE

Most of my home network runs on my Raspberry Pi Kubernetes cluster, and for the most part it’s rock solid. However, applications being applications, sometimes they become less responsive than they should (or for example, when my Synology updates itself and reboots, any mounted NFS volumes can cause the running pods to degrade in performance). This isn’t an issue with service liveliness, which can be mitigated with a liveness probe that restarts the pod if a service isn’t running.

As more services go live on my Kubernetes clusters and more people start relying on them, I get nervous. For the most part, I try and keep my applications and configurations stateless - relying on ConfigMaps for example to store application configuration. This means with a handful of YAML files in my Git repository I can restore everything to working order. Sometimes though, there’s no choice but to use a PersistentVolume to provide some data persistance where you can’t capture it in a config file.

If you’re anything like me, your home lab is constantly changing, evolving, breaking, rebuilding. For the last year or so I’ve been running all my home kubernetes workloads on a Raspberry Pi cluster - and it’s been working really well! I’ve been through several iterations - for example firstly running on SD cards (tl;dr - it’s bad, they wear out really fast with Kubernetes on board!), then PxE booting them from my Synology to it’s now current state of booting directly from SSDs.

I love Raspbery Pis - I have done since they first released them, all the way up to the present iteration, the Raspberry Pi 4. They’re phenominal little bits of kit, endlessly hackable and because of their really low price, they open up computing to a huge number of people who otherwise wouldn’t get the opportunity. One of the irritating things, though, about running Raspberry Pis is that they typically boot from an SD card.

Since I started learning Kubernetes the Certified Kubernetes Administrator (CKA) exam has been a target for me, but it’s always seemed to be out of reach. The whole Kubernetes ecosystem is a vast and nebulous beast, with new projects rising to the fore all the time, and old projects fading from favour. The size and rapid development that make the field so interesting and powerful, are the same properties that make the learning curve so steep, and the entry bar so high.

Up until recently I’ve been running a Windows Server Core VM with Active Directory, DNS and Certificate Services deployed to provide some core features in my home lab. However, I’ve also been conscious that running a lab on old hardware doesn’t exactly have much in the way of green credentials. So, in an effort to reduce my carbon footprint (and electricity bill) I’ve been looking for ways to shut down my lab when it’s not in use.

I run quite a few applications in Docker as part of my home network - there’s a small selection below, but at any one time there might be 10-15 more apps I’m playing around with: plex - Streaming media server unifi - Ubiquiti Network Controller homebridge - Apple Homekit compatible smart home integration influxdb - Open source time series database grafana - Data visualization & Monitoring pihole - internet tracking and ad blocker vault - Hashicorp secret management Until recently a single PhotonOS VM with Docker was all I needed to run - everything shared the same host IP, stored it’s configuration locally or on an NFS mount and generally ran fine.

Following on from me recent post deploying Kubernetes with the NSX-T CNP, I wanted to extend my environment to make use of the vSphere Cloud Provider to enable Persistent Volumes backed by vSphere storage. This allows me to use Storage Policy to create Persistent Volumes based on policy. For example, I’m going to create two classes of storage, Fast and Slow - Fast will be vSAN based and Slow will be NFS based.

I’ve done a fair amount of work learning VMware PKS and NSX-T, but I wanted to drop down a level and get more familiar with the inner workings for Kubernetes, as well as explore some of the newer features that are exposed by the NSX Container Plugin that are not yet in the PKS integrations. The NSX-T docs are…not great, I certainly don’t think you can work out the steps required from the official NCP installation guide without a healthy dollop of background knowledge and familiarity with Kubernetes and CNI.

When I started my blog back in May 2007 (12 years ago!) I was running Wordpress, then switched to DotNetNuke, then BlogEngine, then finally back to Wordpress - which I’ve used since 2010. Today I’ve cut over to a new architecture based on Hugo and hosted on AWS using a combination of Route53, Cloudfront and S3. Why the change? If it ain’t broke… You may well ask why I’ve made the move, or you may not…I’m going to tell you anyway…