Linux Networking

Almost every protocol and interface in Linux are based on public available specifications published by the Internet Engineering Task Force that makes all those RFc freely available at https://datatracker.ietf.org

Tip

For long time ifconfig was the command to use to retrieve and change network parameters in Linux. It has been replaced by ip which can be used in multiple ways:

ip link show shows the NIC available

ip neigh shows MAC address → IP mapping on the local network

Virtual network interfaces

Linux provides several kernel primitives for creating virtual network interfaces without physical hardware. They appear as normal interfaces to the rest of the network stack (ip link show lists them) but move packets in software.

veth pairs

A veth (virtual Ethernet) pair is two virtual NICs wired back-to-back: whatever enters one end exits the other. They are always created in pairs and are the standard mechanism for connecting two different network namespaces.

namespace A              namespace B
┌────────────┐           ┌────────────┐
│  veth0     │ ←———————→ │  veth1     │
└────────────┘           └────────────┘

Container runtimes (Docker, k3s/containerd) create one veth pair per container:

container namespace          host namespace
┌────────────────┐           ┌─────────────────────────┐
│  eth0 (vethXXX)│ ←———————→ │ vethYYY ——→ cni0/docker0│
└────────────────┘           └─────────────────────────┘

The host-side end attaches to a bridge (cni0, docker0, flannel.1) that routes to the real NIC.

Why containers use veth and not TAP/TUN

This is not obvious. TAP/TUN look like they could work, but veth is faster — not slower — for containers, and TAP/TUN would be the wrong tool architecturally.

The key difference is what sits on either side of the interface:

A container shares the host kernel. It runs inside a network namespace, which is a kernel construct. Its network stack is the kernel’s network stack — just isolated. There is no userspace intermediary needed. A veth pair connects two kernel namespaces directly, entirely in kernel space: the packet crosses the veth and arrives in the host namespace without ever leaving the kernel.
A VM has its own independent kernel that does not share the host kernel and cannot participate in host network namespaces. The only way to get packets out is through a hypervisor process (QEMU) running in host userspace. TAP gives that process a file descriptor to exchange frames with the host kernel.

If you gave a container a TAP interface, some userspace forwarder process would have to sit in the middle reading packets from the TAP fd and forwarding them — adding two context switches per packet (kernel → userspace → kernel). That is strictly worse than veth, which stays in the kernel the whole time.

veth (container):   container ns ——[kernel]——→ host ns ——→ bridge ——→ NIC
                    zero userspace hops, no copies

TAP (VM):           guest kernel ——→ virtio-net ——→ QEMU (userspace) ——→ TAP fd ——→ host kernel ——→ bridge ——→ NIC
                    one mandatory userspace hop because QEMU must be there

TAP/TUN exist specifically to bridge the gap when a userspace process needs to participate in the packet path — for VMs (QEMU must emulate devices) or VPNs (the daemon must encrypt). Containers don’t need that; veth keeps everything in-kernel.

TAP and TUN interfaces

TUN and TAP are a Linux kernel feature — a driver built into the kernel that lets a userspace process own one end of a virtual network interface. They are completely separate from veth pairs. With veth, both ends are inside the kernel (in different namespaces). With TUN/TAP, one end is inside the kernel (as a normal network interface) and the other end is a file descriptor held by a userspace process.

Normal NIC:  kernel network stack  ←→  hardware (wire / radio)
veth pair:   kernel namespace A    ←→  kernel namespace B
TUN / TAP:   kernel network stack  ←→  fd  ←→  userspace process

The names come from what they intercept:

TUN = network TUNnel. Operates at L3: the fd carries raw IP packets (no Ethernet header). The interface gets an IP address, not a MAC address.
TAP = network TAP (as in a wiretap). Operates at L2: the fd carries full Ethernet frames, including MAC headers, ARP, VLAN tags — everything a real NIC would see.

	TUN	TAP
Layer	L3 — IP	L2 — Ethernet
fd carries	Raw IP packets	Full Ethernet frames
Has MAC address?	No	Yes
Can be bridged?	No	Yes (acts like a real NIC)
Typical use	VPNs	VM networking

/dev/net/tun — the device node

Both TUN and TAP are created through the same character device: /dev/net/tun. The name is confusing (it says “tun” but handles TAP too); think of it as the control node for the entire TUN/TAP driver.

Creating an interface requires CAP_NET_ADMIN (root, or a process with that capability). The steps a process takes:

int fd = open("/dev/net/tun", O_RDWR);        // open the driver
 
struct ifreq ifr = {};
ifr.ifr_flags = IFF_TUN;                      // or IFF_TAP for TAP
strncpy(ifr.ifr_name, "vpn0", IFNAMSIZ);      // requested name; kernel fills it if empty
 
ioctl(fd, TUNSETIFF, &ifr);                   // creates the interface, returns fd

After this:

ip link show lists vpn0 as a real interface
ip link set vpn0 up && ip addr add 10.8.0.1/24 dev vpn0 configures it like any other NIC
The kernel has no idea the other end is a user process — it treats vpn0 exactly like a hardware NIC

The interface lives as long as the fd is held open. When the process closes the fd (or dies), the kernel tears the interface down automatically.

Any process with CAP_NET_ADMIN can create TUN/TAP interfaces. Unprivileged users can be granted access to a pre-created interface using ioctl(fd, TUNSETOWNER, uid) — this is how tools like OpenVPN let non-root users connect to a VPN: the privileged helper creates the interface once and grants ownership to the user process.

Packet flow through the fd

The fd is a bidirectional byte pipe to the kernel network stack:

read(fd) → the kernel hands you the next packet/frame that was routed to this interface (outbound from the kernel’s perspective — traffic leaving the host via this interface)
write(fd, data) → the kernel processes the packet/frame as if it had arrived on this interface (inbound — goes through netfilter, routing, delivered to sockets or forwarded)

TUN in practice — VPN

WireGuard, OpenVPN, and most VPN clients use TUN:

app sends to 10.8.0.5
  → kernel routing table: "10.8.0.0/24 via tun0"
  → kernel puts IP packet in tun0's queue
  → VPN daemon read(fd) → gets the raw IP packet
  → daemon encrypts it, sends over real UDP socket to VPN server

VPN server sends encrypted reply over UDP
  → daemon recv, decrypts → plain IP packet
  → daemon write(fd, packet) → kernel processes as arriving on tun0
  → delivered to the app

The VPN daemon never touches Ethernet; it only sees IP packets. That is why TUN (L3) is the right tool here.

TAP in practice — VMs

QEMU uses TAP to give a VM a virtual Ethernet card. The guest OS needs to do ARP, send Ethernet broadcasts, get DHCP — all L2 operations — so TAP (L2) is required:

guest (VM)           host userspace           host kernel          host network
┌─────────────┐     ┌──────────────────┐     ┌──────────────────┐
│ virtio-net  │←───→│  QEMU            │←fd─→│ vm-<name> (TAP)  │──→ microvm-br ──→ NAT → wlp1s0
└─────────────┘     │  (Ethernet       │     └──────────────────┘
                    │   frames r/w)    │
                    └──────────────────┘

Guest sends an Ethernet frame → virtio-net → QEMU write(tap_fd, frame) → kernel processes the full Ethernet frame on vm-<name>
Host receives a frame → bridge forwards to vm-<name> → QEMU read(tap_fd, frame) → delivered to guest’s virtio-net

The TAP interface is bridged to microvm-br, which is in turn NAT’d to the physical NIC (wlp1s0).

ip link show type tun   # list TUN interfaces
ip link show type tap   # list TAP interfaces
ip tuntap               # create/delete TUN/TAP interfaces

Quick comparison

Primitive	Connects	Used by
veth pair	two kernel namespaces	containers (Docker, k3s)
TAP	kernel ↔ userspace (L2)	VMs (QEMU, VirtualBox)
TUN	kernel ↔ userspace (L3)	VPNs (WireGuard, OpenVPN)

See also NetworkManager for how these interfaces interact with NetworkManager and why containers/VMs should be added to the unmanaged list.

Sockets

Linux implements The TCP Stack but also provides an higher level interface, called sockets We can use ss to display socket-related information ```


$ ss -s 1
Total: 913 (kernel 0)
TCP:   10 (estab 4, closed 1, orphaned 0, synrecv 0, timewait 1/0), ports 0 2

Transport Total     IP        IPv6 3
*         0         -         -
RAW       1         0         1
UDP       10        8         2
TCP       9         8         1
INET      20        16        4
FRAG      0         0         0
1

lsof can also be used to show open files in combination with -c to select a process name

$ lsof -c chrome -i udp | head -5 1
COMMAND   PID USER   FD  TYPE   DEVICE     NODE NAME
chrome   3131  mh9  cwd   DIR      0,5   265463 /proc/5321/fdinfo
chrome   3131  mh9  rtd   DIR      0,5   265463 /proc/5321/fdinfo
chrome   3131  mh9  txt   REG    253,0  3673554 /opt/google/chrome/chrome
chrome   3131  mh9  mem   REG    253,0  3673563 /opt/google/chrome/icudtl.dat
chrome   3131  mh9  mem   REG    253,0 12986737 /usr/lib/locale/locale-archive

Edmondo's Vault

Explorer

Linux Networking

Virtual network interfaces

veth pairs

Why containers use veth and not TAP/TUN

TAP and TUN interfaces

/dev/net/tun — the device node

Packet flow through the fd

TUN in practice — VPN

TAP in practice — VMs

Quick comparison

Sockets

Graph View

Table of Contents

Backlinks