Networking Models
TCP/IP Model (Internet Protocol Suit)
• Link Layer, corresponds to L2 in OSI model.
• Internet Layer (IP), networking interface or internetworking, addressing and routing for TCP/IP
• Transport Layer (UDP/TCP)
• Application Layer (HTTP, FTP, etc) corresponds to L5/L6/L7 in OSI model.
Open Systems Interconnection (OSI) Model
• L1 – Physical
• L2 – Data Link (Ethernet, PPP, VLAN, etc)
• L3 – Network (NAT, IP, IPSEC etc)
• L4 – Protocol Suites (NetBIOS, SCTP, TCP, UDP, etc)
• L5 – Session (RPC, SDP, SMB, SOCKS, etc)
• L6 – Presentation (TLS, SSL, FTP, IMAP, SSH, etc)
• L7 – Application (SOAP, DHCP, DNS, HTTP, HTTPS, NFS, POP3, SMTP, FTP, IRC, SSH, etc)
OSI Model for Kubernetes
L1: bare metal, VMs, cloud (CPU, RAM, disk, network)
L2: OS, Machine Application Binary Interface (ABI) (system calls, device drivers, etc), kernel + [system, cgroups/namespaces, jails, zones) (RHEL, CoreOS)
L3: Cluster Consensus (etcd (consensus db built with Raft consensus algorithm), Apache ZooKeeper, consul) Etcd uses Raft, a consensus algorithm for distributing configuration, state and metadata information within a cluster and monitoring for any changes to the data stack
L4: Cluster Resources (CNI, CSI)
L5: Container Orchestration (CO) (Kubernetes, OpenShift, CloudFoundry)
L6: Containers (Rocket, Docker, Podman, CRIO, containerd)
Etcd: The etcd project’s focus is safely storing critical data of a distributed system and it demonstrated its quality early on. etcd was created by the CoreOS team in 2013, inspired by Chubby, a key-value store created for Google’s internal cluster infrastructure. Etcd was contributed to the CNCF
Software Defined Networking (SDN)
Software Defined Networking (SDN) uses software-based controllers or APIs to create and control a virtual network or traditional hardware.
Network virtualization segments different virtual networks within a single physical network or connect devices on different physical networks to create a single virtual network.
SDN is a software defined way to control the traffic or routing of data packets through a server.
• Using an open standard software-based controller, using a single protocol to communicate with different hardware devices.
• Configure network services and allocate virtual resources to change network infrastructure in real time, to optimize data flow.
• Visibility into the entire network and creation of separate zones for devices with different security levels.
Parts of SDN architecture:
• Applications,
• Controllers,
• Networking devices.
Network Functions Virtualization (NFV) abstracts network functions from hardware and supports SDN by providing the infrastructure on which SDN software runs.
Network Service Chaining, also known as Service Function Chaining (SFC) uses SDN to create a chain of connected network services, such as L4-7 services like firewalls, network address translation (NAT), and intrusion protection.
Network Functions Virtualization (NFV)
Network Functions Virtualization (NFV) is a way to virtualize network services such as routers, firewalls, and load balancers.
The NFV architecture is proposed by the European Telecommunications Standards Institute (ETSI), NFV consists of
• Virtualized Network Functions (VNF),
• Network Functions Virtualization infrastructure (NFVi), such as Hypervisors, and
• Management, Automation and Network Orchestration (MANO).
Container Networking
Container Networking is based for a large part on Linux networking. Containers use Linux partitioning features as Cgroups and Namespaces that allow container processes to map to network, storage and other namespaces. Container networking uses network namespaces of the Linux kernel.
Network namespaces:
• A container isolated network uses its own virtual interfaces like routing tables, L2 isolation, sockets and IPTABLE rules.
• Containers use iproute2 to interact with network namespaces.
• Network namespaces are stored in /var/run/netns
• Two types of network namespaces:
– Root namespace (ip link)
– Non-root namespace (ip netns, ip link)
A container network interface belongs to only one network namespace. Multiple containers require multiple interfaces or using pseudo-interfaces like:
• Virtual bridge, using Virtual Ethernet device (veth) pairs as a tunnel between container and root namespace, and a Linux bridge or OpenvSwitch (OVS) for connectivity between containers and external (real) interfaces.
• Multiplexing, using a network device with multiple virtual interfaces and packet forwarding rules (MACVLAN, IPVLAN),
• Hardware networks like SR-IOV (supported by most Network Interface Controllers (NIC)).
Docker networking uses pluggable drivers. Several drivers exist by default and provide core networking functionality.
Bridge networks
Default Docker networking mode, a Link Layer device which forwards traffic between network segments, using a software bridge, which allows containers on the same bridge network on the same host to communicate. The Docker bridge driver automatically installs rules in the host machines.
Docker0: a Linux bridge created by docker daemon
• 1 interface to host namespace
• All containers on the host are attached to docker0 via veth-pair
• Container gets private IP address assignment, with random MAC generation to avoid MAC collissions and ARP problems.
docker run –d –P –net=bridge training/webapp python app.py
docker ps
docker exec <containerid> ip a
Host networks
Use host’s network and share the host’s networking namespace. Container does not get its own IP-address.
Basic port mapping on host network to reach services
ifconfig | grep –A 2 eth0
docker run –d –net=host training/webapp python app.py
docker ps
docker exec <containerid> ip a | grep –A 2 eth0
Overlay
Creates a distributed network across multiple hosts to connect multiple containers. When you initialize a container host, two new networks are created:
• An overlay network called ingress
• A bridge network called docker_gwbridge
All pods on the host are attached to docker0 via veth-pair
Macvlan
Connect directly to physical network, assigning a MAC address to a container’s virtual network interface, making it appear as a physical device on the network. You can also use ipvlan, which uses L3 and L2 mode, and use an L2 bridge.
No Networking
Disable all networking, usually in conjunction with a custom network driver.
Container Network Interface (CNI)
CNI is a vendor-neutral interface between container runtimes and networks, independent of the Container Orchestration (CO) platform, e.g. Kubernetes, OpenShift, Docker, rkt, Mesos, CloudFoundry, podman, CRI-O, Kurma, Amazon ECS. CNI (Container Network Interface) originates at CoreOS as part of rkt. Now a CNCF project.
CNI is responsible for network connectivity of containers and removing allocated resources when the container is deleted. It consists of a specification and libraries for writing plugins to configure network interfaces. CNI defines a basic execution flow and JSON-based configuration format for network operations. Plugins are executables in the host, and Kubelet is the runtime on the node.
The container runtime creates the Network Namespace, identifies the network to attach containers, and invokes the plugin.
The CNI plugin or bridge program implements:
1. Create veth Pairs
2. Create Bridge Network/Interface
3. Attach vEth to Namespace
4. Attach other vEth to Bridge
5. Assign IP address
6. Bring up interfaces
7. Enable NAT
Basic plugin commands:
• ADD
• DEL
• CHECK
• VERSION
E.g. the container runtime invokes `bridge add <cid> <namespace>` to add a container to a network.
Container Networking is based for a large part on Linux networking.
Multi-networking
CNI supports multi-networking, but Kubernetes by default does not: a Kubernetes pod is exposed only to 1 interface (and loopback). Multus by Intel supports multi-networking using CRD-based network objects.
Reference plugins: bridge, loopback, vlan, macvlan, ipvlan, host-device, ptp (point-to-point), Windows bridge, Windows overlay.
IPAM (IP Address Management): host-local, DHCP, static
Meta (chained) plugins: bandwidth, firewall, flannel, portmap, source-based routing, tuning
3rd Party plugins: Calico, Weave, Cilium, Multus, Romana, SR-IOV (Single Root I/O Virtualization), Nuage, Amazon ECS, Linen, Silk (CF)
* IBM Cloud Kubernetes implements Calico
* OpenShift supports using SR-IOV hardware on nodes
That's a wrap........