Kuberentes Infrastructure Overview

I've posted several blog entries to setup various parts of an on-premise Kubernetes installation. This is meant as a summary referencing code posted to github for easy access. You can clone the entire repository, edit the required files and use deploy.sh/cleanup.sh scripts, or run the deployment directly from github as documented below. Each of the headers below is a link to the corresponding blog describing the process in detail.

If you'd like to clone the code run this command.

[root@kube-master ~]# git clone https://github.com/mike-england/kubernetes-infra.git

Cluster Install

While this can be automated through templates or tools like terraform, for now, I recommend following the post specifically for this.

Logging

This setup can be almost entirely automated, but unfortunately you'll need to modify the elasticsearch output in the config file

[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/logging/fluent-bit-role.yaml
[root@kube-master ~]# wget https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/logging/fluent-bit-configmap.yaml
<modify output server entry elasticsearch.prod.int.com entry and index to match your kubernetes cluster name>
[root@kube-master ~]# kubectl create -f fluent-bit-configmap.yaml
[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/logging/fluent-bit-daemon-set.yaml

Load Balancing

Installation from metallb is straight forward. As with logging, you'll need to modify the config map, this time changing the IP range. If you're running a cluster with windows nodes, be sure to patch the metallb daemonset so it doesn't get deployed to any of those nodes.

[root@kube-master ~]# kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.7.3/manifests/metallb.yaml
[root@kube-master ~]# wget https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/load_balancer/metal-config.yaml
<modify ip address range>
[root@kube-master ~]# kubectl create -f metal-config.yaml
if you're running a mixed cluster with windows nodes
[root@kube-master ~]# wget https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/load_balancer/node-selector-patch.yaml
[root@kube-master ~]# kubectl patch ds/speaker --patch "$(cat node-selector-patch.yaml)" -n=metallb-system

Monitoring

Assuming you have the load balancer installed above, you should be able to deploy monitoring without any changes.

[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/monitoring/clusterRole-prometheus.yaml
[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/monitoring/prometheus-config-map.yaml
[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/monitoring/prometheus-server.yaml
[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/monitoring/prometheus-node-exporter.yaml
[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/monitoring/clusterRole-kube-state.yaml
[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/monitoring/prometheus-kube-state.yaml

DNS Services

Again, with the load balancer in place, this should be deployable as is.

[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/external_dns/dns-namespace.yaml
[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/external_dns/etcd.yaml
[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/external_dns/external-dns.yaml
[root@kube-master ~]# kubectl create -f https://raw.githubusercontent.com/mike-england/kubernetes-infra/master/external_dns/coredns.yaml