Design Youtube

Requirements

Functional requirements

1. User can upload videos
2. User can stream (watch) the video

Out of scope:

• Users can view information about a video, such as view counts.
• Users can search for videos.
• Users can comment on videos.
• Users can see recommended videos.
• Users can make a channel and manage their channel.
• Users can subscribe to channels.

Non functional requirements

1. System should be highly available (availability >> consistency)
2. System should support uploading and streaming large video (10s of GB)
3. System should allow low latency streaming of video even in low bandwidth
4. System should scale to a high number of videos uploaded and watched per day (1M videos uploaded per day, 100M videos watched per day)
5. Support resumable uploads

[!note]
If the functional requirement is small, non-functional requirement is even more important to pin down

Core identity

1. User
2. Video: actual video
3. VideoMetadata: the metadata of the video

API

1) user can upload a video
   
POST /v1/upload
{
    Video,
    VideoMetadata
}

2) user stream/watch the video
   
GET /v1/videos/:videoId -> Video & VideoMetadata

[!note]
In here remember to mention that you're outlining some simple apis, but may come back and improve them as we get deeper in the design

High level design

Background — Video streaming

1. Video Codec:
   • use to compress and decompress digital video. It reduce the size while preserving the quality
   • Common codec: H.264, H.265 (HEVC), VP9, AV1, MPEG-2 and MPEG-4
   • H.264 has the device support while H.265, VP9 compressed better but requires more encoding works
   • VP9 is more common for browser
2. Video container:
   • Use to store the video data (frames, audio and metadata or even transcript)
   • Common ones are .mp4, .webm, .mkv and .mov — .mp4 has the best support across browsers
3. Bit rate
   • Bit transmit over a period of time, typically in kbps (kilobit per second) or mbps (megabit per seconds)
   • High resolution & high frame rate requires more bit rate
   • Compression via codecs also have effective on bit rate: the higher the compress, the smaller the video, the better
4. Manifest file
   • Include 2 type of file:
     • Primary manifest: The root file, store all the version of the video
     • Manifest file: in each version, store the segment of the video,
   • Example:
     • Primary manifest
       • 480p media manifest → segment1, segment2, segment3
       • 720p media manifest → segment1, segment2, segment3
       • 1080p media manifest → segment1, segment2, segment3
   • This is typically for adaptive streaming, the segment is normally a .mp4

1) User can upload videos

We first need to find the way to store the video metadata. In order to do this, we can consider using Cassandra since the metadata of a video is structured. We already know how we're gonna query, Cassandra allow very high throughput in writing due to it's natural LSM datastructure incoming write goes to Memtable

For uploading we can use multi-part uploading, this will requires changing our endpoint from POST /v1/upload to POSt /v1/presigned_url

Extend for video streaming

For video streaming, we would need to split our video into smaller segments (a few seconds in length), convert each segment into playable format on each device. We would then able to support adaptive video streaming later on.

2) User can watch the videos

In here we can use Adaptive bit rate streaming. For this to work, we need to store the video in different format. It will work like this

1. Client will fetch the VideoMetadata which has the URL point to the manifest file in S3
2. Client will download the manifest file
3. Depends on the network conditions and user settings, client retrieve the URL for the suitable format and download that segment
4. Client will play that segment first and begin to download more segment
5. If client detect network condition are slowing down or improving, it will adjust the format of the video segments it is downloading. If the Network get worse it will attempt to download lower resolution segment

[!important]
If the interviewer ask how we would code such client we can refer to using some open source library of adaptive streaming player i.e Shaka player