Key Technology

Core database

[!danger]
Many candidate trip themselves by trying to compare between NoSQL or SQL, i.e I need relational database because there are relationships or I use NoSQL because i need scale and performance are often flagged as inexprience

Instead focus more about what the database bring to you, focus on the feature you're familiar with i.e: I'm using postgres here because ACID allow me maintain data integrity

There are 2 types

1. Relational database: Use for transaction data, default choice for product design interview
   1. The interview where we desgin a product (shop or something) not design the consistent hashing one
2. NoSQL database: Use when these condition happen
   1. Flexible data models without fixed schema
   2. Application needs to scale horizontally across many servers to accomodate large amount of data or high load
   3. Your application dealing with large volumes of data, unstructured data, applications require real-time data processing and analytics

For NoSQL, the two flexible options are:

1. DynamoDB: breadth of feature
2. Cassandra: Good choice for write-heavy workload

[!NOTE]
The thing NoSQL can do does not mean SQL cannot do it. For example Postgres can do JSON columns with flexible schema, SQL can also scale horizontally

Blob storage

Normally for blob storage, our core database will store a pointer to the blob storage references. The blob storage has the following attribute

1. Durability: uses Erasure Coding
2. Scalability: AWS S3 can be considered as inidifinitely scale, you dont need to care about scalability here
3. Security: built-in security like encryption at rest, access control features
4. Upload & download directly at client: use presigned URL to do so
5. Chunking: when upload large files, we can enable chunking which allow to resume upload

Search optimized database

Search optimised index will use Inverted Index. Which is very fast to do reverse look up to find which items/documents contains the word. For example if we want to search for which doc contains word1 inverted index will give us something like

{
  "word1": [doc1, doc2, doc3],
  "word2": [doc2, doc3, doc4],
  "word3": [doc1, doc3, doc4]
}

Thing you should know

1. Inverted index
2. Tokenisation: breaking a piece of text into different words
3. Stemming: Reducing words to their root form i.e we can reduce "running" and "runs" to "run"
4. Fuzzy Search: Allow some slight mispelling. By using algorithm like distance calculation — measure how many letters need to be changed to transform the word
5. Scaling: Search optimised database just scale by adding more shards

Example ElasticSearch, Postgres GIN indexes that support full text search

API Gateway

API gateway responsible to route GET /users/123 to your Users microservice

nearly in all product design, it's good idea to include API gateway in the design as your first point of contact for your client

Normally, API gateways are AWS API Gateway, KongAPI, sometimes we have nginx as well.

[!NOTE]
All most all the system design interview we will put the API gateway there

Load balancer

Use to distribute traffic across multiple machine (horizontal scaling). The reality is you need a loadbalancer before every service that horizontally scale. However in system design, either omit load balancer from your design all together, or add one only to the front of the design as an abstraction, otherwise it's too much.

This is common setup for authenticated horizontally scale service. Sometimes, if you need sticky sessions or persistent connections, use L4 or L7 load balance.

• If you have persistent connections like websockets, you likely want to use L4 Load balancer
• Otherwise L7 offer more flexibility

Common load balancer are: AWS Elastic Load Balancer, nginx, HAProxy. Note if you need extreme traffic, you would need hardware load balancers which will outperform software load balancers — this is related to networking like T0 to server node.

Queue

Queue provide a pool of worker to process messages in their own pace. Using a queue we can

1. Smooth out the load on the system
2. Decouple the producer and the consumer, allow to scale separately

[!danger]
If the workload is synchronous, and you have strong latency requirement (<500ms) you nearly guarantee to break that latency constraint

The 2 common use case is

1. Buffer for bursty traffic
2. Distribute work across a system

Things to note:

1. Message ordering: most queue are FIFO, however some queues like Kafka allow more complex ordering per priority or time
2. Retry mechanism: Many queues built-in retry mechanism, between attempts and maximum attempts
3. Dead Letter Queue: use to store message that cannot be process for debug and auditing
4. Scale with partitions: Queue can scale with multiple servers. Each partition can be processed by different set of workers
5. Backpressure: slow down the queue when it's overwhelm, help to prevent the queue from becoming bottle neck. If the queue is full, we reject new messages or slow down the rate at which message are accepted

Common technology: Kafka, SQS

Stream / Event sourcing

• Good when dealing with real-time processing a large amount of data.
• Event sourcing means we store the application state as the sequence of event, allow us to replay

Use case:

1. Process large data in real time
2. Event replay (Bank transaction, deposits, withdrawer)
3. Support multiple consumers reading from the same stream (good for pubsub system)
   1. i.e Chat system where a lot of people login and chat on the same room

How to scale:

1. Partitioning: Partition the stream across multiple servers. You specify the partition key to ensure the events are from the same partition
2. Multiple consumer groups: allowing differnt consumer to read from same stream independently
3. Windowing: batching grouping events together based on the time.

Fault tolerance:

1. Replication: replicate multiple server

Example: Kafka, Flink, Kinesis

Distributed Lock

Good when we need to claim the period of lock for a time, for example: ticketmaster - locking a ticket/seat etc.

Normally, this can be done using either Redis or Zookeeper. We use the atomicity feature of the key-value store to ensure only 1 proccess to lock at a time

The lock can then set to be expire after a certain amount of time

Common technology: Redis, Zookeeper

Distributed cache

Caching. Use to offload and increase speed of query to database.

We need to pay attention to

1. Eviction Policy: how to save space

   1. Least Recently Used (LRU): Evicts least recently accessed item first
   2. First in first out (FIFO): evicts in the order they were added
   3. Least Frequently Used: remove the one least frequently accessed

2. Cache Invalidation strategy: how to discard/invalid stale data

   • Invalidate per database update

3. Write strategy

   1. Write through: write both cache and datastore simultaneously. Consistency but slow write
   2. Write around: Write data to datastore, skip cache. Cache will update next fetch time
   3. Write Back (Write Behind): Write to cache and then asynchronously write to datastore

[!note]
When using cache, be specific on what data type and how you store the data in cache. Many candidate missed this and this is a good opportunity

Common technology: Redis, Memcache

CDN

CDN - cache content on the server edge, if the content is not cache, CDN will fetch the content and return it to the user

1. CDN can cache both static and dynamic content
   1. For Dynamic, when the source content change, the app will invoke the CDN provider API to invalidate the cache. Another way is use short TTL
2. CDN can cache the API response
3. CDN has eviction policies - determine when cached content is removed, we can set TTL

Example: Cloudflare, Akamai, Amazon Cloud Front