Regional Broadcasting Async Reconcilation

To sync the redis across region, we can do the following pattern: Local counters + async gossip — each region tracks its own count and broadcasts deltas to other regions every few seconds. Each region sums local + last-known-remote to approximate the global count.

graph LR
  User[Mobile User] -->|req| US[US-East Rate Limiter + Redis]
  User -->|req| EU[EU-West Rate Limiter + Redis]
  US -->|"async delta sync (every 1-5s)"| EU
  EU -->|"async delta sync (every 1-5s)"| US

Option to sync

Option 1: Redis Pub/Sub (simplest, but unreliable)

Each region's rate limiter publishes deltas to a cross-region Redis Pub/Sub channel. Other regions subscribe.

Problem: Pub/Sub is fire-and-forget — if a subscriber is disconnected when the message is published, it misses it forever. No persistence, no replay. Fine for best-effort, bad if you need guaranteed delivery.

Option 2: Redis Streams (better)

Each region writes deltas to a Redis Stream (XADD) in a shared/replicated Redis. Other regions consume with XREADGROUP. Streams are persistent — consumers can catch up after disconnection.

graph LR
  US[US-East Limiter] -->|"XADD region:us-east {user123: -5}"| Stream[(Cross-Region Redis Stream)]
  EU[EU-West Limiter] -->|"XREADGROUP"| Stream
  EU -->|"XADD region:eu-west {user123: -3}"| Stream
  US -->|"XREADGROUP"| Stream

Option 3: Kafka / SQS / dedicated message bus (production-grade)

Each region publishes deltas to a Kafka topic (partitioned by user key). Other regions consume. This is what large-scale systems actually use because:

  • Durable, replayable
  • Handles cross-region replication natively (Kafka MirrorMaker, or a global Kafka cluster)
  • Decouples regions — if EU-West is down, deltas queue up and get processed on recovery