Design Youtube

Streaming a Video: Step-by-Step Process

When a user plays a video, the client (e.g., the YouTube player) performs the following steps:

Requesting the Manifest File:
- The client first requests a manifest file from the CDN or origin server. This file contains metadata about the video, such as available bitrates, resolutions, and chunk URLs. Without the manifest, the client cannot know how the video is segmented or where the chunks are stored.
Example of an HTTP Live Streaming (HLS) manifest (M3U8 format):
```
#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1500000,RESOLUTION=1280x720
https://cdn-edge.com/videos/1234/720p/manifest.m3u8
```
The client parses the manifest to understand the available chunks and formats.
Requesting Specific Chunks:
- Based on the manifest, the client requests the appropriate chunk:
  - It uses the video ID, start position, and the desired resolution/bitrate (initially determined by available bandwidth) to identify which chunk to fetch.
  - Example request:
```
GET https://cdn-edge.com/videos/1234/720p/segment_03.ts
```
The client continues requesting chunks sequentially as the video plays.
Dynamic Resolution Switching:
- The client monitors the user’s bandwidth and adjusts the requested bitrate dynamically. For instance, if bandwidth drops, the client switches from 720p chunks to 360p chunks.
CDN Edge Caching:
- If the requested chunk is cached at the CDN edge, it is served directly. Otherwise, the edge server retrieves the chunk from an origin server, caches it locally, and then serves it to the client.

Object Storage (e.g., S3):
- While services like S3 support cross-region replication, it is asynchronous, which introduces the risk of losing data during a regional disaster. To mitigate this:
  - Synchronous Multi-Region Writes: Use a storage system like DynamoDB Global Tables or a custom replication service to ensure synchronous replication to multiple regions. Each write is acknowledged only when it is committed to multiple regions.
  - Multi-Master Architecture: Employ a multi-master storage architecture where uploads can be directed to any region, and updates are synchronized globally.
Example: Videos are uploaded to a “hot” region but are immediately replicated to backup regions synchronously.
Metadata Server:
- The metadata server stores critical details about videos, such as manifest locations, chunk paths, and resolutions. To ensure availability:
  - Distributed Database: Use a globally distributed database (e.g., Google Spanner, CockroachDB) to store metadata. These systems provide strong consistency and cross-region replication.
  - Conflict Resolution: Employ mechanisms like version vectors or timestamps to resolve conflicts when writes occur in multiple regions simultaneously.