Ingress controller tutorial

(Source code)

Building something from scratch is always a nice approach to really understand a subject. Hence, I recently build a simple ingress controller. Here I want to give a recap of the relevant tasks of an ingress controller and why it is needed in Kubernetes.

Why do we need Ingress controller?

The problem

Kubernetes has to solve a problem that all container orchestrations have to solve: How to route traffic between and to pods?

The Kubernetes networking model requires that every Kubernetes networking implementation guarantees that every pod gets its own IP and that communication between pods as well as node agents works without NAT. This heavily simplifies the port management as only port collisions inside a pod (collection of containers) have to be avoided.

Kube-proxy: Cluster-internal calls

For cluster-internal calls Kubernetes uses the following method:

• Direct pod to pod communications works out of the box.
• Usually pods are called in the form of Service API Resources. These group pods to provide load balancing between multiple pods.
• These pods can then be called via the domain name '{service-name}.{namespace}.svc.cluster.local'. The Kubernetes internal DNS resolution resolves this to a separate IP. Calls to this IP are then load balanced to the individual pod on the network layer using e.g. iptables on Linux. To keep these load balancing networking layer rules up to date with the pods deployed in the Kubernetes cluster kube-proxy runs on every node. It listens to the Kubernetes API for service and pod updates and updates the networking layer rules accordingly.

Ingress: Cluster-external calls

For calls from outside the Kubernetes clusters there is no way for callers to

• resolve the service urls using the Kubernetes internal DNS resolution
• route the requests to the service or pod IP addresses.Hence, a new Kubernetes API resource and corresponding controller is required. This is the ingress.

In contrast to kube-proxy, an ingress is placed in the application layer and limited to the HTTP/HTTPS protocols. It receives requests and proxies them to the relevant backend services. For this mapping especially the request host and request URL is used to match requests to backend services. This allows to map, e.g. the paths '/' and '/api' to different backend services. This purpose is one main reason why the ingress has been placed in the application layer.

For the actual network routing the ingress acts as an application layer reverse proxy. The actual network route for the calls made by the reverse proxy is then handled by the pod network (as they are cluster-internal calls).

Ingress: Implementation details

The ingress has two separate components:

• A client of the Kubernetes API that collects the current status and status-updates for:
  • ingress: The ingress resource defines all relevant ingress definitions.
  • service: The service resource is required to determine the relevant service ports for the proxied calls.
  • secrets: Only if the ingress is used for TLS termination. The secret holds for this case the TLS-certificates.
• A reverse proxy that listens for HTTP/HTTPS calls and proxies them to the relevant backend services (which uses then Kubernetes internal DNS combined with kube-proxy to determine the actual routing via the network layer).

The current status determines the configuration for the reverse proxy. Updates of the Kubernetes resources cause a reconfiguration of the reverse proxy.