Austin's SWE Notes 😭

GitHub

main

Home
Vault

Alexu System Design Interview

1. Scale from zero to millions of users

Cache Tier
CDN Consideration
Database Replication
Database Scaling
DataCenter
Horizontal Scale User Session
Horizontal Vs Vertical
Load Balancer
Logging
Message Queue
Scale From 0 To Millions

10. Design a Notification System

Android Push Notification
Design A Notification System
Email Notification
IOS Push Notification
SMS Push Notification

11. Design a new feed system

Design A New Feed System

12. Design a chat system

Design A Chat System
HTTP Long Polling
HTTP Short Polling
Server-Sent Events (Sse)
WebSocket

13. Design a search autocomplete system

Design A Search Auto Complete System

14. Design youtube

DAG Config File
DAG Scheduler Pattern
DASH
Design Youtube
HLS
RTMP
RTSP

15. Design google drive

Block Servers
Design Google Drive
Resumable Upload
Sync Conflict Resolution

2. Back-of-the-envelope estimation

Back-of-envelop Estimation

3. Framework for system design interviews

System Design Framework

4. Design a rate limiter

Design Rate Limiter
Fixed Window Counter Algorithm
Leaking Bucket Algorithm
Sliding Window Counter Algorithm
Sliding Window Log Algorithm
Token Bucket Algorithm

5. Design Consistent Hashing

Consistent Hashing PseudoCode
Design Consistent Hashing

6. Design Key-Value store

Design Key-Value Store

7. Design Unique ID generator

Design Unique ID Generator In Distributed System
Multi-master Replication ID Generator
Ticket Server
Twitter Snowflake ID Generator
Universally Unique Identifer (Uuid)

8. Design a URL shortener

Design A URL Shortener

9. Design a web crawler

Design A Web Crawler

Algorithms

Arrays

Array Rotation
Boyer-Moore Majority Voting Algorithm
Cyclic Sort
Encode And Decode
Largest Number
Longest Consecutive Sequence
Merge Intervals
Partition Equal Subset Sum
Perfect Rice Bags
Pigeonhole Principle
PrefixPostFix Product
TwoSum (Hashtable Way)

Traversal

1D Traversal
Spiral Traversal

Backtracking

Combination Sum
Combination, Permutation And Subset
Generate Parentheses
Letter Combination
N-Queens
Palindrome Partitioning
Permutation
Subset
WordSearch

Dynamic Programming

1-D Dynamic Programming

Climbing Stairs
Coin Change II
Coin Change
Decode Ways
House Robbery II
House Robbery
Longest Increasing Subsequence
Maximum Product Subarray
Word Break

2-D Dynamic Programming

Longest Common Subsequence
Longest Palindrome Substring
Palindromic Substrings
Traverse DP Table
Unique Paths

Graph

Alien Dictionary
Clone Graph
Course Schedule
Graph Traversal
Graph Valid Tree
Number Of Connected Component In A Graph
Number Of Islands
Pacific Atlantic Water Flow
Rotting Oranges
Topological Sort (Kahn'S Algorithm)
Union Find

Greedy

Greedy Algorithm
Jump Game
Maximum Subarray (Kadane'S Algorithm)
Meeting Rooms II
Non Overlapping Intervals

HashMap

LRUCache

Heaps

Find Median From Data Stream
Merge K Sorted Linked List
Merge K Sorted Lists
Top K Elements

LinkedList

Find Half Of A Linked List
In-place Reversal Linked List
LinkedList Cycle
Odd Even Linked List
Remove Nth Node From End Of List
Reorder List
Swap Nodes In Pairs

Misc

Anagram Check
Max Product Vs Max Sum
Palindrom Check
Trick
Tricky Time Complexity

Things to watchout

Default Variable Gotcha
Gotcha
Happy Condition First Before Failure
Keep Track Of Global Variable In Recursion
Recursion Base Condition Mistake

Pointers

3Sum
Container With Most Water
Fast And Slow Pointers
Floyd's Loop Detection
Two Pointers
TwoSum (Two Pointers Way)

Recursions

Tail Vs Non-Tail Recursion

Search

Binary Search In Rotated Sorted Array
Binary Search

Sliding Windows

Longest Contiguous Sub Array Less Than Or Equal To K
Longest Repeating Character Replacement
Longest String Without Repeating Characters
Minimum Swaps To Group All 1'S Together
Minimum Window Substring
Sliding Windows

Stack

Decode String
Largest Rectangle In Histogram

Tree

Binary Tree Maximum Path Sum
Construct Binary Tree From Preorder And Inorder Traversal
Kth Smallest Element In A BST
Lowest Common Ancestor
Maximum Depth Of Binary Tree
Same Tree
Serialise And Deserialise Binary Tree
Subtree Of Another Tree
Tree BFS (Level-By-Level)
Tree DFS
Validate Binary Search Tree

Tries

Design Add And Search Words DataStructure
Implement Trie Prefix Tree
WordSearch II

Resources

assets

EtherChannel 2024-08-04 21.44.01.Excalidraw
Switching Loop 2024-08-04 22.07.02.Excalidraw

AWS Certified Developer (DVA-C01)

Advanced Identity

Advanced IAM
AWS Directory Services
Pass Role And Get Role
STS Overview
Trust Policy

API Gateway

API Gateway - HTTP API Vs REST API
API Gateway Authoisation And Authentication
API Gateway Caching
API Gateway Canary Deployment
API Gateway Common HTTP Errors
API Gateway CORS
API Gateway Endpoint Types
API Gateway Integration Types
API Gateway Monitoring, Logging And Tracing
API Gateway Resource Policy
API Gateway Stage And Deployment
API Gateway Swagger & Open API Spec
API Gateway Throttling
API Gateway Usage Plans & API Keys
API Gateway WebSocket API
API Gateway

AWS Amplify

AWS Amplify

AWS AppSync

AppSync Security
AWS AppSync

AWS CDK

AWS CDK (Infrastructure As Code)

AWS CLI

AWS CLI Profile
AWS Credentials Provider Chain
AWS Signature V4 Signing
Dryrun
MFA With CLI
STS Decode

AWS Cognito

AWS Cognito Identity Pools (Federated Identitiy)
AWS Cognito Sync
AWS Cognito User Pools Vs Identity Pools
AWS Cognito User Pools
AWS Cognito

AWS Lambda

AWS Lambda & CloudWatch Events (Eventbridge)
AWS Lambda
AWS Lambda@Edge
Lambda Aliases
Lambda And CloudFormation
Lambda And CodeDeploy
Lambda And VPC
Lambda Asynchronous Invocation
Lambda Cold Start
Lambda Concurrency And Throttling
Lambda Container Image
Lambda Destinations
Lambda Environment Variables
Lambda Event Queue
Lambda Event Source Mapping
Lambda Execution Context
Lambda Execution Role (Iam Role)
Lambda External Dependencies
Lambda Function Performance
Lambda Integration With ALB
Lambda Integration With S3
Lambda Layers
Lambda Limits To Know
Lambda Monitoring & X-Ray Tracing
Lambda Provisioned Concurrency
Lambda Reserved Concurrency
Lambda Resource Based Policies
Lambda Synchronous Invocation
Lambda Version

AWS SAM

AWS SAM And CodeDeploy
AWS SAM CLI
AWS SAM Policy Templates
AWS SAM Summary
AWS SAM
AWS SAR

AWS Security and Encryption

AWS KMS Encryption Patterns
AWS KMS Key Policy
AWS KMS Key
AWS KMS Limitations
AWS KMS Symmetric API Summary
AWS KMS
AWS Secrets Manager
AWS SSM CLI
AWS SSM Parameter Store
AWS SSM Parameters Policies
Envelope Encryption
SSM Parameter Store Vs Secret Manager

AWS Step Functions

AWS Step Functions Error Handling
AWS Step Functions States
AWS Step Functions Task State
AWS Step Functions

CloudFormation

ChangeSets
CloudFormation Drift
CloudFormation Rollbacks
CloudFormation
Conditions
Functions
Mappings
Nested Stacks
Outputs
Parameters
References
Resources
Stacksets

CloudFront

CloudFront Advanced Concepts
CloudFront Caching
Cloudfront Security
Cloudfront Signed URL - Signed Cookies
CloudFront

Development Services

CodeArtifact
CodeBuild
CodeCommit
CodeDeploy
CodeGuru
CodePipeline Artifact
CodePipeline
CodeStar

DynamoDB

Dynamo Basic APIs
Dynamo Local Secondary Index (Lsi)
DynamoDB Accelerator (Dax)
DynamoDB And S3
DynamoDB CLI
DynamoDB Global Secondary Index (Gsi)
DynamoDB Hash And Range Primary Key
DynamoDB Hash Primary Key
DynamoDB On-Demand
DynamoDB Operations
DynamoDB Optimistic Locking
DynamoDB PartiQL
DynamoDB Partition Strategies
DynamoDB Partitions Internal
DynamoDB Provisioned
DynamoDB Security And Other
DynamoDB Streams
DynamoDB Throttling
DynamoDB Time To Live (Ttl)
DynamoDB Transactions
DynamoDB Write Type
DynamoDB

ECS

ECR CLI
ECS Rolling Updates
ECS Task Placement
Task Defnitions

Elastic BeanStalk

BeanStalk Integration With HTTPS
CLI
Cloning
Components
Custom Platform
Deployment Options
Deployment Process
Elastic Beanstalk
Extensions
Integration With Docker
Lifecycle Policy
Migration
RDS With Elastic Beanstalk
Supported Platforms
Update Options
Web Server Tier
Worker Tier

ElasticCache

ElasticCache Redis Cluster Mode
ElasticCache Strategy

Integration & Messaging

Monitoring & Audit

CloudTrail Insights
CloudTrail Vs CloudWatch Vs X-Ray
CloudTrail
CloudWatch Alarms
CloudWatch Custom Metrics
CloudWatch Events
CloudWatch Logs Agent
CloudWatch Logs Insight
CloudWatch Logs Metric Filter
CloudWatch Logs Subscription
CloudWatch Logs
CloudWatch Metrics
CloudWatch Unified Agent
CloudWatch
EventBridge Schema Registry
EventBridge
X-Ray & ECS
X-Ray & Elastic Beanstalk
X-Ray Read & Write Apis
X-Ray Sampling
X-Ray

Others

AWS Amazon Certificate Manager (Acm)
AWS CloudMap
AWS Databases Summary
AWS DataSync
AWS Fault Injection Simulator (Fis)
AWS SES
Exponential Backoff & Service Limits Increase
Exponential Backoff (Any AWS Service)

S3

Athena
S3 Bucket Key
S3 Encryption Advance
SSE-C
Strong Consistency Model

VPC

3 Tier Solution Architecture
NACL, SG, VPC Flow Logs
VPC Peering, Endpoints, VPN, DX
VPC, Subnets, IGW, And NAT

Exam Summary
Notes
README

Azure

App Service
Application Insights
ARM (Azure Resource Manager)
ASG (Application Security Group)
Computer Vision
Container Registry
Content Moderator
Endpoint
Function App
IFX (Instrumental Framework)
Key Vault
Logic App
MA (Geneva Monitoring Agent)
MDM (Multi-Dimensional Metrics)
MDS (Monitoring & Diagnostics Service)
Network Interface
NSG (Network Security Group)
Resource Group
SFMC (Service Fabric Managed Cluster)
SQL Database
SQL Server
Storage Account
Virtual Machine
Virtual Network

Container

Docker

Creating A New Network Bridge
Default Bridge Network
Docker Default Gateway
How Docker Works
Mac Docker Limitation

Data Structure

Circular Buffer

Circular Buffer

CleanCode

Summary

GeoHash

Quad Tree

Graph

Adjacency List
Directed Acyclic Graph
Undirected Cyclic Graph
Vertex

HashMap

Chaining (Hash Collision)
Hash Collision
HashMap
Load Factor And Resize
Open Addressing (Hash Collision)
Probing Sequence

Heap

Heap Implementation
Heap Python
Heap Theory

Probabilistic Datastructure

Bloom Filter
Count-min Sketch (Cms)
Skip List

Queue

Primary Queue

Stack

Monotonic Stack

Ways To Sort A Hash Map

Database

ACID (Consistency)
ACID (Isolation)
ACID
B Tree And B+ Tree
BASE
Cassandra
Clustered Index Vs Non-Clustered Index
Database Denormalisation
Database Index (Secondary Key)
Database Race Condition
Database Scaling
Database Sharding
Database Transaction
DocumentDB And MongoDB
Graph Database
LSM Tree (Log Structured Merge Tree)
LSM Tree Vs B Tree
NoSQL
Partition Key Vs Primary Key Vs Sort Key Vs Clustering Key
Range Lock
RDS Proxy
Read Lock
SQL Vs NoSQL
SQL
WAL (Write A-head Logging)
Write Lock

Hardware

Blocking IO Vs Non-blocking IO
DWDM (Dense Wavelength Division Multiplexing)
How Fiber Optics Work

Kubernetes

Api Server
Container Runtime
Controller Manager
Etcd
K8s (Kubernetes)
K8s Cluster
K8S Control Plane
Kube-Proxy
Kubelet
Node
Pod
Scheduler

Lesson Learned

JPA Saveandflush() Doesn'T Reflect Immediately In Database
Kafka Live Lock
SQS Event Duplication When Autoscale

Linux

Namespace
Netlink

Networking

Cisco

Adding IP Address For An Interface
Assign IP To VLANS
L3 Switch Configure Switch To Run As L2 Or L3

Network Design

Campus LAN
Datacenter Desgin
SOHO (Small Office, Home Office)
Tier 2 Design (Collapsed Core)
Tier 3 Design
WAN (Wide Area Network)

OSI Layers

Header Vs Trailer
OSI Layers
Physical Layer

P4

BMv2
P4

Protocol

2. Data Link Layer Protocol

Data Link Layer
LLDP (Link Layer Discovery Protocol)
PPP (Point-To-Point Protocol)

3. Network Layer Protocol

ARP (Address Resolution Protocol)
ICMP (Internet Control Message Protocol)
IP (Internet Protocol)
IPv4 Address
IPv6 Address
IRDP (Icmp Router Discovery Protocol)
NAT (Network Address Translation)
Network Layer

IPSec protocols

AH (Authentication Header Protocol)
ESP (Encapsulating Security Payload)
IKE (Internet Key Exchange)
IPSec Protocols

4. Transport Layer Protocol

BGP (Border Gateway Protocol)
Networking Commands
QUIC
TCP (Transmission Control Protocol)
Transport Layer
UDP (User Diagram Protocol)

5. Session Layer Protocol

NFS (Network File System Protocol)
PAP (Password Authentication Protocol)
PPTP (Point-To-Point Tuneling)
Session Layer

6. Presentation Layer Protocol

Presentation Layer
SSL (Secure Sockets Layer)
TLS (Transport Layer Security)

7. Application Layer protocol

Application Layer
DHCP (Dynamic Host Configuration Protocol)
DNS (Domain Name System) Protocol
FTP (File Transfer Protocol)
HTTP (Hyptertext Transfer Protocol)
HTTP 1.1
HTTP 1
HTTP 2
HTTP 3
HTTPS (Hypertext Transfer Protocol Secure)
IMAP (Internet Message Access Protocol)
Network Time Protocol (Ntp)
POP (Post Office Protocol)
SFTP (Secure File Transfer Protocol)
SMTP (Simple Mail Protocol)
SNMP (Simple Network Management Protocol)
SSH (Secure Shell Protocol)

SONiC

DASH (Disaggregated API For SONiC Hosts)
SONiC (Software For Open Networking In The Cloud)

Wireless

AP (Access Point)
BSS (Basic Service Set)
EAP (Extensible Authentication Protocol)
Wireless Security
Wireless
WLC (Wireless Lan Controller)
WPA (Wifi Protected Access)

Operating System

Disk Vs Device
High Memory Address
Kernel Space
Low Memory Address
MMU (Memory Management Unit)
Physical Memory
Program Structure In Memory
Stack Vs Heap
User Space
Virtual Memory

Programming

Ansible

Adhoc Command
Ansible Encryption Decryption
Ansible Variables
Ansible-Doc
Ansible.Cfg
Background Task Execution
Control Node
Get Priveledge
Inventory
Main Components
Managed Node
Parallelism
Play
Playbook
Tasks
Variables

module

Setup

Structure

Folder Structure
Play
Playbook
Role
Tasks

Bash

GDB (Gnu Project Debugger)
Gzip
LLDB (Low Level Debugger)
Make

C

Argc Vs Argv
Char
Const Keyword
Header File
How Variable Work In C
Include
Lvalue
Macro
Object File
Printf Format Type
Puts Vs Fputs
Rvalue
Signed Vs Unsigned
String In C

Array

2D Array
Array Arithmetic
Array Declaring
Array Initialiser
Array Supscripting
Copy An Array
Pass Array To Function
Retrieve Array Length

Pointer

Pointer And Array
Pointer Arithmetic
Pointer Subscripting
Pointer Variable
Void Pointer

String

A More Optimal Way Of Looping Through String
Copy A String
Remember To Malloc When Return A String
String In C

Struct

Struct Example
Struct
Typedef Struct

Design Patterns

Abstract Factory Pattern
Adapter Pattern
Builder Pattern
Facade Pattern
Factory Pattern
Iterator Pattern
Observer Pattern
Repository Pattern
Singleton Pattern
Strategy Pattern

GraphQL

Aliases
Arguments
Directives
Entry Point
Fields
Fragments
Mutation
Query
Resolver
Schema
Summary
Variables

Java

DelayQueue
VersionControlSystem

AOP

AspectJ

Concepts

== Vs Equals()
Abstract
ArrayDeque Vs Linkedlist
Checked Exception Vs Unchecked Exception
Checked Exception
Comparator Chaining
Comparator Vs Comparable
Composition Vs Aggregation
Constructor
DAO
Domain
DTO
Field Initialisation Vs Constructor Initialisation
Final
HashCode
Jar
Java SPI Vs @Autoservice
JavaBean
Javax AbstractProcessor
Misc
POJO
Serialisation
Servelet
Static
Thread State
Type Parameter
Unchecked Exception
Volatile
Wildcard

Gradle

Implementation Vs Api
Publish Maven Local

JPA

Flush() Vs Commit()
Generate Primary Key
Save() Vs Saveandflush()

JVM

JVM Stack And Heaps
Manifest File
Object Expiration
RMI (Remote Method Invocation)

Logging

Slf4j Vs JUL Vs Logback Vs Log4j
Slf4j

Multithreading

CAS Pattern
CAS Vs Synchronisation
Common Pool
Compare And Swap (Cas)
Consumer Producer Implementation
Notify And Wait
Process Vs Thread Vs Coroutine
Processor Thread Scheduler
Virtual Thread

Concurrency Collection

ArrayBlockingQueue
Atomic Operation
Concurrency Package And Synchronisation
ConcurrentHashMap Vs HashTable
ConcurrentSkipListMap
CopyOnWriteArrayList
CountDownLatch
CyclicBarrier
ForkJoinPool
Future And Callable
Future
Java Scheduler
LinkedBlockingQueue
Mutex Vs Semaphore
Synchronised Collections Vs CopyOnWriteArray
SynchronousQueue
TransferQueue Vs SynchronousQueue
TransferQueue

Reactive Programming

ReactiveStream API

Reactive Stream

Reactor Core

Adapt From-To Other Classes
Blocking To Flux
Concat Vs Merge
Error Handling
Flux Vs Stream
Flux
Logging
Mono
Multithread
Never Vs Empty
Publisher And Subscriber
Reactor Framework
Reactor Map Vs Flatmap
StepVerifier
Subscribeon Vs Publishon
Zip

RxJava

Flowable
Main Concept
Processor
Publisher (Observable)
Single
Subscriber (Observer)
Subscription

Thread based programming

Instinct Lock
Synchronisation
Thread

Spring

Annotation @Autoconfiguration
Annotation @Autowired
Annotation @Configurationproperty
Annotation @Import
Annotation @Profile
Annotation @Springbootapplication
Annotation @Value
Async
Dependency Management
Environment Variable
Everything About Dependency Injection
Export To Jar
Full @Configuration Vs Lite Bean
Gradle
Lazy Initialise
Naming Convention For Test Packages
Spring Context
Spring External Configuration
Spring Property File
Spring Property Profile
Spring Scheduler
Starters
Why Avoid @Autowired Field Injection

API

API Exception Handler
Path Variable
RequestBody
ResponseBody
ResponseEntity
RestController
Transactional

Events

Default Events

Redis

Redis With SpringBoot

SpringBatch

Chunk Size Vs Page Size
Transaction

SpringBoot

SpringBoot With Redis

Stream

Intermediate Operation
Terminal Operation

Javascript

How Async Await Works
Javascript Is Single Thread
WebAPI
When Not To Use Async Await

Micro Frontends

Micro Frontends

React

Hydration
React Context
Suspense

Serverside Rendering

Server Component
Serverside Rendering (Ssr)

Service Worker

Offline Cache Implementation
Service Worker

Testing

Jest

Transpilers

BalbelJS
BrowserifyJS
SystemJS
Transpilers Code
Webpack

Module API Standard

AMD (Asynchronous Module Definition)
CommonJS (Cjs)
CommonJS Vs ESM Vs AMD Vs UMD
ECMAScript Module (Esm)
RequireJS
UMD

WebRTC

ICE Trickling
WebRTC Quick Start

Kafka

ClientID, Consumer Group ID And Consumer ID
Consumer Group Vs Partition
Kafka Broker Leader
Kafka Broker Replica
Kafka Brokers
Kafka Cluster
Kafka Connect
Kafka Consumer Group
Kafka Delivery Model
Kafka Events (Message, Record)
Kafka Offset
Kafka Partition
Kafka Redundancy
Kafka Stream
Kafka Topic
Kafka
Why Kafka

Spring Kafka

Kafka Exception Handling (Retry)
Spring Kafka Quick Start

Listeners

Concurrency Attribute
Listening To Message

Producers

Partition Strategy
Producing Message

Python

Asyncio
Counter
Custom Comparator For Sort
Custom Decorator
Dataclasses
Defaultdict
Deque
Generator
GIL
Heapq
Kwargs Vs Args
Python Caching
Python Gotcha
Python Module
Python
Slicing
Unpacking

pytest

Class Test
Collect
Conftest
Fixture
Function Test
Mark
Overview
Parameterised Test

Redis

Sorted Set

Typescript

Decorator

Security

Digital Signatures
Firewall Vs Intrusion Prevention System
Symmetric Vs Asymmetric Encryption

System Designs

Software migration pattern

Strangler Fig

System Design Pattern

CQRS (Command Query Responsibility Segregation)
Event Sourcing
Leader Election
Pubsub

System Design Template

Chat System
Design Google Drive
Design Youtube
Frequency Count (Top K Hitters)
Kubernetes
Message Queue
New Feed System
Notification System
Offline Synchronisation
Search System
Websocket Architecture

Cache Improvement

Cache Scaling
LRU Cache System

Database Improvement

Database Connection
Database Sharding NoSQL
Database Sharding SQL
Request Coalescing

File upload

Service Upload To Blob Storage
User Uploads To Blob Storage

Job Scheduler

DAG Scheduler Pattern
DAG Scheduler
Resource Manager

Web crawler

Web Crawler Frontier
Web Crawler

Transaction Management

Saga Architecture Pattern
Saga Vs TCC Vs 2Pc
TCC - Try Confirm Cancel
Three Phase Commit
Two Phase Commit

Terms

The Pragmatic Programmer

0. Preface
1. Pragmatic Philosophy

On this page

Kafka Connect

Pasted image 20230903101337.png

We source the data from different database into a kafka cluster.

← Kafka Cluster

Kafka Consumer Group →