Kubernetes networking ensures seamless communication between pods, services, and external systems. The system is designed to be flexible and works across diverse infrastructure, whether on-premises, cloud, or hybrid environments.
Key Principles of Kubernetes Networking
- Flat Network: All pods in a Kubernetes cluster must be able to communicate with each other without NAT (Network Address Translation).
- Pod-to-Service Communication: Pods need to connect to services via stable IPs, regardless of the pod’s dynamic lifecycle.
- Service Discovery: Kubernetes uses DNS or environment variables to enable pods to discover and communicate with other services.
Networking Components
Pods
- Each pod is assigned a unique IP address.
- Pods communicate directly using their IPs within the cluster.
Services
- Services provide a stable endpoint for accessing a group of pods, even as individual pod IPs change.
CNI Plugins
Kubernetes relies on Container Network Interfaces (CNIs) for implementing the networking layer. These plugins handle IP assignment, routing, and network policy enforcement.
Common CNI Options
Kubernetes supports a variety of CNI plugins, each designed for specific use cases:
1. Amazon VPC CNI (VpcCniAddOn)
- AWS-native solution for Amazon EKS.
- Assigns VPC-native IPs to pods for direct integration with AWS services.
- Uses ENIs to scale pods on a node.
2. Calico
- Focuses on network policies for security and isolation.
- Supports both overlay and non-overlay modes, depending on the infrastructure.
3. Cilium
- Leverages eBPF for efficient packet filtering and observability.
- Integrates well with Kubernetes NetworkPolicies and service meshes.
4. Flannel
- Simple overlay network implementation.
- Uses protocols like VXLAN for pod-to-pod communication. (See Overlay Networking for details.)
5. Weave Net
- Easy to deploy and manage, with optional encryption.
- Suitable for small to medium-sized clusters.
Kubernetes Networking Modes
VPC-Native Networking
- Pods receive VPC IPs directly from the underlying network.
- Supported by plugins like VpcCniAddOn in AWS EKS.
Overlay Networking
- Creates a virtual network on top of the physical network.
- Encapsulates packets for pod-to-pod communication across nodes.
- Implemented by plugins like Flannel and Calico in overlay mode.
Network Policies
- Define rules for controlling traffic flow to and from pods.
- Supported by CNI plugins like Calico and Cilium.
Service Mesh Integration
- Tools like Istio or Linkerd add advanced features like traffic routing, observability, and encryption.
Multi-Cluster Networking
- Enables communication between pods in different Kubernetes clusters.
- Often implemented with tools like Submariner or Cilium.