Networking challenges

• Decoupled microservices based applications rely heavily on networking in order to mimic the tight-coupling once available in the monolithic era.

• Networking, in general, is not the easiest to understand and implement.

• Kubernetes is no exception - as a containerized microservices orchestrator it needs to address a few distinct networking challenges:

  • Container-to-Container communication inside Pods.

  • Pod-to-Pod communication on the same node and across cluster nodes.

  • Service-to-Pod communication within the same namespace and across cluster namespaces.

  • External-to-Service communication for clients to access applications in a cluster.

Container to Container

• Making use of the underlying host operating system's kernel virtualization features, a container runtime creates an isolated network space for each container it starts.

• On Linux, this isolated network space is referred to as a network namespace.

  • A network namespace can be shared across containers, or with the host operating system.

• When a grouping of containers defined by a Pod is started, a special infrastructure Pause container is initialized by the Container Runtime for the sole purpose of creating a network namespace for the Pod.

• All additional containers, created through user requests, running inside the Pod will share the Pause container's network namespace so that they can all talk to each other via localhost.

Pod to Pod

• In a Kubernetes cluster Pods, groups of containers, are scheduled on nodes in a nearly unpredictable fashion.

• Regardless of their host node, Pods are expected to be able to communicate with all other Pods in the cluster, all this without the implementation of Network Address Translation (NAT).

• This is a fundamental requirement of any networking implementation in Kubernetes.

• The Kubernetes network model aims to reduce complexity, and it treats Pods as VMs on a network, where each VM is equipped with a network interface - thus each Pod receiving a unique IP address. This model is called IP-per-Pod and ensures Pod-to-Pod communication, just as VMs are able to communicate with each other on the same network.

• Containers share Pod's network namespace and must coordinate ports assignment inside the Pod just as applications would on a VM, all while being able to communicate with each other on localhost - inside the Pod.

• However, containers are integrated with the overall Kubernetes networking model through the use of the Container Network Interface (CNI) supported by CNI plugins.

• CNI is a set of specifications and libraries which allow plugins to configure the networking for containers.

  • While there are a few core plugins, most CNI plugins are 3rd-party Software Defined Networking (SDN) solutions implementing the Kubernetes networking model.

  • In addition to addressing the fundamental requirement of the networking model, some networking solutions offer support for Network Policies. Flannel, Weave, Calico, and Cilium are only a few of the SDN solutions available for Kubernetes clusters.

• The container runtime offloads the IP assignment to CNI, which connects to the underlying configured plugin, such as Bridge or MACvlan, to get the IP address.

• Once the IP address is given by the respective plugin, CNI forwards it back to the requested container runtime.

External to Pod Communication

• A successfully deployed containerized application running in Pods inside a Kubernetes cluster may require accessibility from the outside world.

• Kubernetes enables external accessibility through Services, complex encapsulations of network routing rule definitions stored in iptables on cluster nodes and implemented by kube-proxy agents.

• By exposing services to the external world with the aid of kube-proxy, applications become accessible from outside the cluster over a virtual IP address and a dedicated port number.

References

• Cluster Networking