Core Concept

Network

• Default HTTP over TCP, handle 90% of use cases
• Need real time update? use WebSocket or Server-Sent Events (SSE)
  • Prioritise Server-Sent Events (SSE) and HTTP Long polling. Only fall back WebSocket when need bi-directional
  • Since this is stateful, we cant simply throw it under a load balancer
• For internal to internal services, use gRPC if performance is critical.
  • Don't use for public facing since it's not natively support in browser yet
• For low latency globally, do regional deployment with data replicated or partitioned by geography, for this case, use CDNs

API Design

• 90% of the time default to REST.
• Do not spend too much time, only 4-5 key endpoints in couple minutes
• If you're returning large result sets, need pagination.
  • Real-time: using cursor base pagination so we know where we are at
    • i.e: ?cursor=post_123
  • Non-real-time: offset base pagination to fetch n amount of item for each page
    • i.e ?offset=20&limit=10
• Authentication, use:
  • User session: JWT token
  • Service to service: API key

Data Modeling

• There are 2 main choices, either NoSQL or SQL
• SQL works great for structured data, when there is a clear relationship and need strong consistency:
  • i.e user account links to order, links to product
  • In SQL, we need to consider Database Denormalisation and Database Normalisation
    • Normalisation: great for consistency but need to do join to complete the data
    • Denormalisation: bad for write but good for read
  • NOTE: always start with Database Normalisation in the interview and Database Denormalisation the specific hot part. Don't propose Database Denormalisation upfront unless you have the clear reasons
• NoSQL allows you to have flexible schemas (data structure changes frequently), or you need to scale horizontally across many servers without complex joins
  • For NoSQL, queries like "get all posts that mentions hashtag Y" require scanning entire table.
  • Therefore you need to design the table right upfront which requires you to know your query upfront
  • NOTE: for NoSQL you cannot add Dynamo Local Secondary Index (LSI) after the table is created but for SQL you can using CREATE INDEX.
    • Therefore changing the structure of the table is often more difficult for NoSQL

Database indexing

• Most relational database create B-tree index B Tree and B+ Tree
  • Hash index is fast for exact match but it doesn't support range, so they're less common
• If you need specialised index for example, full-text search and location queries, you will need external system:
  • Full text search: ElasticSearch
  • Geospatial: PostGIS
• These external system often sync from your primary database using CDC, therefore there will be an acceptable slight lag behind. The trade-off is often worth it since it lets you search in a way your database cannot handle

Caching

• A cache on redis take a round 1ms compare to 20-50ms for a database query, that's already 20-50x speedup
• Normally, 90% of the time we use Cache aside: if the data is there, return, if not, query database, store the result in the cache with TTL and returns. Works for most heavy-read system
• To Invalidate the cache, there are a few strategy
  1. Write through > Write through with invalidation: invalidate entry after write
  2. Use short TTL and accept some staleness
  3. Combination of these 2
• For cache stampedes (when popular cache entry expires, many concurrent requests miss at same time and hits db), we can do
  • Locking: if one requests detect a cache miss, it will claim a lock to generate the cache entry for it. Other requests wait for this cache entry to generate instead of hitting the database
  • Early cache recomputation: recompute the cache before it expires
  • Random TTLs so they dont expire the same time