Allowing outside access to Home Kubernetes Cluster

After I created a home kubernetes cluster, I immediately wanted to allow external access to pods/services/ingresses hosted inside the cluster. One must be aware that in bare metal environments, there is no receiver of an api call to create a load balancer. Since there is not a scriptable environment available to kubernetes, kubernetes cannot request external IP addresses or provision resources that one has come to expect in cloud environments such as AWS. This is a huge bummer – especially since dynamically built environments are fun to have.

To route traffic to web services inside of kubernetes, you have to options available: ingress and service. Services can be exposed via NodePort, LoadBalancer, or ClusterIP. In bare metal, LoadBalancer would never work (unless you coded your own API call to configure a load balancer outside of kubernetes). ClusterIP might work if you want to manage a routing table somewhere inside your network, and NodePort will work if you want to manage a port forwarding table on your router. None of these options are fun for home labs on bare metal. An Ingress is like a layer 7 firewall in that it reads the hostname and path of the incoming HTTP request and can route to applicable services. This works great for a dynamic environment where I am going to host multiple http endpoints.

The overall view of this traffic is going to be: Internet > Router > k8s Ingress > k8s Service > Pod(s).

To create an ingress in kubernetes, you have to make it a Service. In cloud environments, the Ingress is created as type LoadBalancer in home labs, we create this as type NodePort and port forward on the router to any node in the kubernetes cluster.

$ kubectl get svc -n ingress-nginx
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
default-http-backend   ClusterIP   10.102.173.184           80/TCP                       3d
ingress-nginx          NodePort    10.110.162.247           80:30746/TCP,443:32641/TCP   3d

In my home lab, I am port forwarding on my router 0.0.0.0:80 -> <any_node>:30746 and 0.0.0.0:443 -> <any_node>:32641.

Since I have a non-traditional home router (a Linux server with two nics), I had to either enter these into iptables or I could improve upon that by setting up a load balancer such as nginx. nginx will allow me to port-forward load balance across all my nodes and have an easy config file to edit. Because I also want to use cert-manager with Let’s Encrypt free SSLs, I chose to use the TCP stream server of nginx.

Another hiccup (so to speak) in home based labs, is that ISPs give DHCP addresses. So when my internet IP changes, I need to update the DNS of all my http endpoints. Rather than doing that, I have all my home urls (*.wplr.rocks) CNAME to a single hostname which I update with a script with the correct IP.