Dropbox - File Sharing System

Functional requirement

Core requirements:

User should be able to upload a file from any device
User should be able to download a file from any device
User should be able to share a file with ohter users and view files shared with them
User can automatically sync file across device

Out of scope:

User should be able to edit files
User should be able to view files without downloading them

Non functional requirement

Core requirement:

System should be highly available
System should support file as large as 50 GB
System should be secure and reliable. We should be able to recover files if they lost or corrupted
System should make upload/download, sync time as fast as possible (low latency)

[!note]
In all most all scenario, we consider availability > consistency. We only consider consistency if every read must receive the most recent write i.e stock trading app, bank system.
For Dropbox system, it's okay if user see stale file

Core identity

Now we need to go through the functional requirement one by one to detect the main identity, so far we can see

File: raw data that users will be uploading/downloading and sharing
FileMetadata: Metadata associated with the file, include info like files name, size, mime type, who uploade it
User: the user of the system

[!note]
Our objective for the system design should be meeting all the functional and non functional requirements. We start with functional first and then lay the non-functional one on top

API or system interface

Lets start simple to just finish the functional requirement. Uploading a file, we may have something like

POST /v1/files
Request: {
    File, Metadata
}

To download the file, we can have something like

GET /files/{fileId} -> File & FileMetadata

To share the file, we can have

POST /files/{fileId}/share

Requests:
{
    User[] // Other user that also have this file
}

[!danger]
These apis might change later, it's important to let the interviewer know that this is for the first stage and you may come back and improve them

To get the changes of the file, we need to do

GET /files/changes?since={timestamp} -> ChangeEvent[]

Each ChangeEvent will include fileId, the type of change created, updated or deleted and the updated Metadata. We support ?since={} params so that the client can only fetch the changes from the last seen.

[!note]
For user information, we need to go from the headers side (either via session token or JWT) to ensure that alll the users are authenticated and authorised. We should not pass user information in request body as it's easy to manipulated

High level design

1) User should be able to upload a file from any device

We need to take care of

File Metadata
File (Raw byte)

For metadata, we can use NoSQL like DynamoDB since the metadata is likely to change and loosly structured. However PostgresSQL would work as well here, so either option should be fine

The schema can be simple like

{
    "id": "123",
    "name": "file.txt",
    "size": 1000,
    "mimeType": "text/plain",
    "uploadedBy": "user1"
}

Option 1: File service split file into Blob Stroage and Metadata DB

The FileService once receive the file from the user can store the metadata part in the metadata db, and the actual raw part in the S3 Blob Storage

Challenges

We need to handle the case where metadata is saved but the file is not uploaded.
1. Solution: Transactional approach, only save the metadata is file is uploaded
We technically upload twice:
1. From client to File Service
2. From File Service to S3
  This would double the bandwidth and redundant

Option 2: Let the user upload the file directly to blob storage

From the previous option, we need to upload twice. The better way is let the user to upload to S3 directly using PresignedURL. We can do the following

POST /v1/files -> PresignedUrl
Request:
{
    FileMetadata
}

The post now only take the file metadata, after user received the presigned url they can upload using PUT

PUT PresignedUrl 
Request:
{
    File
}

Once the file is uploaded, we will send a notification to our backend using S3 Notification. The backend then update the file metadata with status "uploaded"

2) Users should be able to download a file from any device

Download from blob storage

The user can download directly from the blob storage. They can request a presigned url from our backend

GET /files/{fileId} -> PresignedURL

The user can use the presigned url to download the file from blobstorage directly

Challenge

The user in different geography will have a slow problem to download. How you address this

Download from CDN

This method is to address the geography problem stated above. We can use a CDN to cache the file closer to the user location. This helps reduce the latency and speed up download time

When a user requests a file, we can use CDN to serve the file from the closest server to user. We can generate a signed URL for the user to download directly from CDN

Challenges: CDN are expensive

Solution:
- We only cache specific files that downloaded frequently, and specify how long the file should be cached on CDN
- We use cache invalidation to remove files from CDN as they're updated or deleted

For this we can create a table contains which file is shared with which user.

[!note]
We avoid to keep track of the sharedUser in the file metadata because the cost of keeping its in sync is high, we better store this in a separate table and lookup when necessary

We can have a table SharedFiles like this

`userId` (parititon key)	`fieldId` (sort key)
user1	fileId1
user1	fileId2
user1	fileId3

In this case, (userId, fileId) is Composite Primary Key. Using this, we no longer need the sharedUser in the file metadata, we can simply query this SharedFiles table

4) Users can sync files across device

We need to handle 2 way of syncing

Local → Remote
Remote → Local

Local → Remote

From local side, we need to have a sync agent, which can use the following apis:

MacOS: FSEvents
Windows: FileSystemWatcher
Linux: inotify

These would let the client to sync the data to the remote machine detect for file change. The step are:

Using the OS api to monitor the file
When detect the change, it queues and upload the modified file
The change then send to our server with updated metadata
Conflict resolves as last write wins — who most recent edit has the latest version of the file

Remote → local

Client needs to ask the server for new update to poll in. There are 2 way to do this

Polling: periodically ask the server "anything changes since my last sync?" Server would query the DB to see if any files has a updatedAt timestamp
Websocket or SSE: server maintain an open connection with each client and pushes notification when changes occur. This is more complex but provides real-time update

We can use a hybrid approach:

Active notification: server pushes change through websocket or SSE for real-time sync
Periodic polling: Client poll in background (every few minutes using GET /v1/files/changes?since={timestamp}) to catch the missed message to ensure eventual consistency