Design a Chat System

Step 1: Requirements

Functional:

1. 1:1 chat and group chat (say, up to 500 members).
2. Send and receive text messages in real time. Media (images and voice notes) is also supported.
3. Message history: scroll back and search recent conversations.
4. Multi-device sync: the same account on phone, tablet and web.
5. Online presence ("online", "last seen") and typing indicators (the "typing…" sign).
6. Delivery and read receipts.
7. Push notifications when the user is offline or the app is in the background.

Out of scope (not covered here): voice and video calls, stories, payments, and channels with millions of followers (these need a different "broadcast" design).

Non-functional:

• Low latency: a message reaches an online recipient in less than 500 ms (p99, which means 99 out of 100 messages), and ideally in about 100 ms inside one region.
• Durability: no message is lost once the sender sees "sent" (✓).
• Ordering: messages in a conversation appear in the same order for everyone.
• High availability: the service is almost always up, because chat is the main feature of the app.
• Scale: tens of millions of daily users, and millions of connections at the same time (concurrent connections).
• Security and privacy: encryption in transit (while data travels), and ideally end-to-end encryption, where only the sender and the receiver can read the message (post 51).

Step 2: Estimation

Assumptions:

• 50M daily active users (DAU, people who use the app each day), each sending 40 messages per day.
• Peak concurrent connections: at the busiest time, about 20% of DAU are connected, so 10M are online at once.
• Average message size: ~200 bytes of text + metadata. Media is stored separately in object storage.
• Average group size for delivery: about 5 recipients per message (most chats are 1:1 or small groups).

So this means:

• Storage with many writes and reads in time order for each conversation. This is a great fit for wide-column / LSM-based databases, split by conversation (posts 20 and 22). A wide-column database stores rows in partitions, and each partition holds rows sorted by a key. LSM (log-structured merge tree) is a storage design that is very fast at writing, because it only appends data.
• Many connections that stay open, so we need a separate layer of WebSocket gateway servers that scales by adding more machines (post 33). A gateway here is a server that only holds the connections.
• High fan-out for groups. Fan-out means one message must go to many receivers. So we need an efficient routing / pub-sub layer between servers. Pub/sub (publish and subscribe) means a sender publishes a message to a topic, and every subscriber of that topic receives it.

Step 3: API, protocol and data model

Connection: each client keeps a WebSocket open to a gateway (post 33), and sends and receives small JSON or binary frames (frames are single messages on the connection):

// Client → server: send a message{ "type": "send", "clientMsgId": "c-7f3a…", "conversationId": "conv_881",  "body": "Lunch at 1?", "sentAt": "2027-05-06T08:00:01Z" }// Server → sender: acknowledgement (✓ sent: the message is safely stored){ "type": "ack", "clientMsgId": "c-7f3a…", "messageId": "1789012345678901234", "seq": 1042 }// Server → recipients: new message{ "type": "message", "conversationId": "conv_881", "messageId": "1789012345678901234",  "seq": 1042, "senderId": "42", "body": "Lunch at 1?", "createdAt": "…" }// Client → server: receipts{ "type": "delivered", "conversationId": "conv_881", "upToSeq": 1042 }{ "type": "read",      "conversationId": "conv_881", "upToSeq": 1042 }// Reconnect and catch up (sync){ "type": "sync", "conversations": [ { "id": "conv_881", "afterSeq": 1038 } ] }

REST APIs (normal web requests) handle actions that are not real-time: create a group, add members, upload media (using pre-signed URLs, which are temporary links that allow one upload, post 17), and load older history pages (cursor pagination, which means "give me the next page after this marker", post 34).

Data model:

conversations:        conversation_id, type (direct/group), created_at, last_seqconversation_members: conversation_id, user_id, role, joined_at,                      last_delivered_seq, last_read_seqmessages:             PARTITION KEY (conversation_id [, time_bucket])                      CLUSTERING KEY seq (or message_id) DESC                      sender_id, body / media_ref, created_at, type, edited/deleted flagsuser_inbox (optional):user_id → list of (conversation_id, last_seq, unread_count) for the chat listdevices:              user_id, device_id, push_token, platform, last_seen

• Messages are partitioned by conversation, so "get the latest 50 messages in conv_881" is one fast read of one partition, newest first.
• Very active conversations can grow huge. So add a time bucket to the partition key (for example, one partition per conversation per 10 days) to keep each partition from growing without limit (post 25).

Step 4: High-level design

Sending a message, step by step:

1. Asha's app sends send over its WebSocket to gateway 3, with a client message ID (used to remove duplicates, called dedup).
2. The gateway forwards it to the chat service, which checks that Asha is a member of conv_881 (this is authorisation: checking what a user is allowed to do, post 49).
3. The chat service gives the message the next sequence number in the conversation and a message ID, and writes the message to the message store.
4. Once the message is safely stored, the server sends an ack (acknowledgement, a "got it" reply) to Asha (✓ sent).
5. The chat service looks up the recipients' sessions in the registry ("Ravi is on gateway 7, on 2 devices") and publishes the message to those gateways through pub/sub.
6. Gateway 7 sends it to Ravi's connected devices. Ravi's app replies delivered (✓✓), and later read (blue ✓✓).
7. If Ravi has no active connection, the notification service sends a push notification (post 62).

Step 5: Deep dives

Deep dive 1: The connection (gateway) layer

• Gateways are stateful (stateful means a server remembers data between requests, here the open connections) but thin: they only do authentication (checking who the user is), heartbeats and forwarding. The business logic lives in stateless chat services, which keep no connection data. Stateless means a server remembers nothing between requests, so any server can handle any request (post 7).
• Scale horizontally (add more machines). Estimate how many connections one server can hold (post 33), and spread connections with least-connections routing, which sends each new connection to the server that has the fewest.
• Session registry: when a device connects, save user → {gateway_id, device_id} in Redis (a fast in-memory database) with a TTL. Each heartbeat (every 30 s or so) renews it. If a gateway dies, its entries expire.
• Reconnect storms: when a gateway restarts, its thousands of clients reconnect all at once. Use exponential backoff with jitter on the client (wait longer after each failed try, plus a random extra delay, post 41). Also drain gateways slowly during deploys, which means moving their clients away a few at a time.
• How mobile phones really behave: networks switch between Wi-Fi and mobile data, and the system pauses apps in the background. Clients must reconnect and sync without problems.

Deep dive 2: Message IDs, sequence numbers and ordering

The problem: messages from different senders, through different servers, can arrive in different orders. The clocks on phones can also be wrong (post 28).

The solution: the server assigns an order per conversation.

• A per-conversation sequence number (1, 2, 3…) is given by the chat service when it stores the message. It gives a clear order and makes gaps easy to find ("I have 1038, the server says the latest is 1042, so I am missing 1039–1041").
  • The sequence needs one single writer for each conversation, or an atomic counter (one that cannot be interrupted), like a Redis INCR on seq:conv_881, or conditional writes in the database.
• Globally unique, time-ordered IDs (a Snowflake ID is a 64-bit number built from a timestamp, a machine number and a counter, so IDs sort by time; post 60) give each message a unique ID that is roughly sorted by time, and the servers do not need to talk to each other to make them. Many systems use both: a Snowflake-style message_id, plus a per-conversation seq for ordering and sync.
• Never trust client timestamps for ordering. You can show them as "sent at", but order the messages by the server's sequence.

Deep dive 3: Delivery guarantees, with no loss and no duplicates

• Store first, then acknowledge. The sender's ✓ means "safely stored", so a server crash after that cannot lose the message.
• Client retries. If the sender does not get an ack (the network dropped), it sends the message again with the same clientMsgId. The server removes duplicates by checking (sender, clientMsgId) and returns the first ack again (post 34). So there are no duplicate messages.
• Delivery to recipients is at-least-once. This means a message always arrives, but it may arrive twice. The pub/sub push may fail or be repeated, so recipients remove duplicates by message ID.
• Catch-up sync is the safety net. Each time the client connects or reconnects, it sends its last seen seq per conversation. The server returns everything newer from the message store. Even if a real-time push was lost, sync fills the gap.

Deep dive 4: Storage

Why wide-column (Cassandra/ScyllaDB) or similar: Cassandra is an open-source wide-column database that spreads data over many machines and is very fast at writes. ScyllaDB is a database that works like Cassandra but is written in C++ to be faster. A partition key is the column value that decides which machine and partition holds a row.

• Very many writes (tens of thousands per second), and LSM storage handles them well (post 22).
• The access pattern (how the data is read) is always "messages in conversation X, newest first, a page at a time". This is one partition, read in clustering order, which is the sort order inside the partition (post 20).
• Horizontal scaling and copying data to several data centres (replication) are built in (post 24).

-- Messages are split into partitions by conversation (plus a time bucket for very busy groups),-- and stored newest first. So "load the latest 50 messages" reads just one partition.PRIMARY KEY ((conversation_id, bucket), seq)  WITH CLUSTERING ORDER BY (seq DESC)

Other storage pieces:

• Conversation lists and unread counts (the chat list screen): a per-user inbox table, or a cache in Redis. It is updated when messages arrive.
• Media: kept in object storage (a service for big files), uploaded directly with pre-signed URLs, and served through a CDN (servers near the user that keep copies of files). Messages only store a reference to the file (post 17).
• Search: you can search recent history with a search index. It is filled asynchronously, which means a little later (post 19). This is not possible on the server if messages are end-to-end encrypted (see below).
• Retention: rules that delete or archive old messages, and features like "disappearing messages".

Deep dive 5: Group chats and fan-out

For a group of N members, one message becomes N deliveries:

• Small and medium groups (up to hundreds): the chat service looks up the members (from a cache), finds their sessions, and publishes to their gateways. It is the same as 1:1 chat, repeated for each member.
• Store once, deliver many. The message is stored once in the conversation's partition, not copied for each member. Each member keeps their own last_read_seq (the last message they read), so unread count = latest seq − last_read_seq.
• Large groups and channels (thousands of members or more) change the design. Pushing every message to every member in real time is expensive. So systems often push only to people who are looking at the channel now, and let the others pull (fetch the messages themselves) when they open the channel. This is similar to the news-feed trade-offs in post 64.
• Limit the group size for groups with all features (many apps do), and offer "broadcast channels" for huge audiences.

Deep dive 6: Presence and typing indicators

Presence ("online / last seen"):

• Based on heartbeats: while a device's WebSocket is alive and sending heartbeats, the user is online. When the heartbeats stop (after, say, 60 seconds), the user becomes offline, and last_seen is updated.
• The fan-out problem: a user with 500 contacts who goes online or offline would cause 500 updates each time, and phones reconnect all the time. Ways to reduce this:
  • Lazy presence: only fetch or subscribe to presence for contacts that are visible on screen right now (the chat list, or an open chat).
  • Debounce: wait a little before reacting, and do not announce very short disconnects (like switching from Wi-Fi to mobile data).
  • Coarse "last seen" ("last seen today at 10:30") instead of second-by-second updates.
• Presence is soft state: it is data that may be a little old and that is fine to lose. It becomes correct after a short time (eventually consistent). It is stored in memory (Redis) with TTLs, and never in the main message database.

Typing indicators are ephemeral (they last only a moment). They are sent over the WebSocket, never stored, rate-limited (for example, at most one event every few seconds), and dropped if delivery fails. Here at-most-once delivery (it may be lost, but it is never repeated) is fine (post 37).

Deep dive 7: Multi-device sync and receipts

• Each device has its own connection and its own sync cursor. A cursor is a marker of how far the device has read: last_seq for each conversation.
• Messages that the user sends from their phone are also delivered to their other devices (they are receivers too).
• Read state belongs to the user (not to a device): reading on the laptop clears the unread badge on the phone. Sync last_read_seq to all devices.
• Receipts are just small events that update last_delivered_seq and last_read_seq for each member. In groups, show a total like "read by 12", instead of storing a receipt for every member and every message.

Deep dive 8: Security and end-to-end encryption

• TLS (Transport Layer Security, a protocol that encrypts data between two machines; it is the encryption used by HTTPS) for all connections, and authenticated WebSockets (a token is checked when the connection starts, and it is renewed from time to time).
• Check permission on every action: may this user post in this conversation, or add members?
• Abuse controls: limits on how fast a user can send (post 43), spam detection, and tools to block and report users.
• End-to-end encryption (E2EE) (post 51): messages are encrypted on the sender's device, and only the recipients' devices can decrypt them. The server stores and routes encrypted blobs (unreadable chunks of data) that it cannot read.
  • It is widely done with the Signal Protocol, an open set of rules that lets two devices share secret keys and encrypt messages so that nobody in the middle can read them. Groups often use "sender keys", so a message is encrypted once for the whole group.
  • Trade-offs: no server-side search or content moderation (checking messages for bad content) of message content, more complex multi-device support (each device has its own keys), and backups must also be encrypted.

Deep dive 9: Reliability and operations

• Multi-AZ (several separate data centres) for gateways, chat services, pub/sub, and a message store that keeps copies of the data (post 40).
• Graceful degradation: if one part fails, the important parts keep working. If presence or typing indicators fail, messaging still works. If search is down, chats still load (post 44).
• Backpressure: this means handling the case where a receiver is slower than the sender. If a recipient's gateway is slow, do not block senders. Queue messages for a short time, drop short-lived events, and rely on sync.
• Observability (being able to see what the system is doing): the full message latency (from send to delivered), ack latency, connection counts for each gateway, reconnect rates, sync lag and push success (posts 46–47).
• SLOs (service level objectives, reliability targets), for example "99% of messages delivered to online recipients within 500 ms".

Wrap-up

• Key decisions:
  • WebSocket gateways + stateless chat services + pub/sub routing + a session registry.
  • Store first, then deliver, with per-conversation sequence numbers and Snowflake-style IDs.
  • Client-side dedup IDs plus sync-on-reconnect, for no loss and no duplicates.
  • A wide-column message store partitioned by conversation (plus time buckets).
  • Lazy, heartbeat-based presence, and push notifications for offline users.
• Trade-offs: E2EE gives privacy but removes server features (search, moderation). Real-time fan-out is faster, but pull is cheaper for large groups. We keep strict ordering inside one conversation, and we do not need one global order.
• Next steps: voice and video calls (WebRTC, a browser technology for live audio and video), large broadcast channels, message search that works with E2EE, and multi-region active-active (several regions all serving users at once).