Common Pattern

Real time updates

Start with long polling → SSE → Websocket. Pub/sub is also a good way to decouple the publisher and subscriber. Long-polling doesnt serve the need anymore if it creates too much delay or wasted requests. If user need near-instant update.

Manage long-running tasks

Use queue base, return the job id. However if the job is small and affort to wait, it's better to just let the client wait then adding a queue.

Dealing with contention

Prevent race conditions when multiple user try to access the same resource. In here we have 2 choice:

1. Use database level approach locking

   1. Database Transaction

      BEGIN;
      
      UPDATE tickets
      SET status = 'sold'
      WHERE id = 42 AND status = 'available';
      
      INSERT INTO orders (user_id, ticket_id)
      VALUES (7, 42);
      
      COMMIT;
      

   2. CAS

      1. Only update if the version correct, UPDATE tickets SET status = 'sold', version = version + 1 WHERE id = 42 AND status = 'available' AND version = 7
      2. Database itself will make sure it transaction, by default, for the same row, only 1 write can happen at a time
2. Implement a distributed locks (Redis, Zookeeper)

We can start with database transaction resolution first before we scale to distributed lock.

NOTE:

• If the data is changes across multiple row, use Transaction.
• If the data is just updating one record, use CAS

[!danger]
Given this problem, if we break out database to smaller database, we have the responsibility to deal with our own transaction cross database. This should be heavily considered and dont do it prematurely

Read scale

Read traffic grows much faster than write

We should follow these orders:

1. Optimise read performance within database through Database Denormalisation and Database indexing
2. Scale horizontally with Read replica pattern
   1. If scale with read replica, we need to know how to handle Database Replica lag solution
3. Add external cache layers like Redis and CDNs

Write scale

Options:

1. Sharding (horizontally)
2. Verticle scale
3. Burst write goes through queue

Large blob handling

Use CDN or S3 and use presigned url so that the client can download from CDN or S3 directly

Multi-step fanout

We can use AWS Step Functions or Temporal for stage management. We can also use Kafka to split from an event to multiple steps worker to start processing parallely

Proximity-based service

For map based service, we need to use Geospatial index, PostgreSQL PostGIS, Redis Geospatial. For this, we dont build index since it's big, we better just manually scan the items