Part 6 · Async & Event-Driven Systems
Event-Driven Architecture for Beginners
The order service calls seven other services and fails whenever any of them does. Event-driven architecture flips it: announce what happened and let whoever cares react. Events vs commands, the four patterns called “event-driven”, choreography vs orchestration, and sagas.
On this page
The Problem
Your order service has become too big and too busy. When an order is placed, its code calls these services directly, one after another:
- 1orderService.placeOrder()waits for every call, in order
- 2paymentService.charge()
- 3inventoryService.reserve()
- 4shippingService.createShipment()
- 5emailService.sendConfirmation()
- 6loyaltyService.addPoints()slow → checkout is slow
- 7analyticsService.track()down → checkout fails
Every new feature means changing the order service. If the loyalty service is slow, checkout is slow. If analytics is down, checkout fails. Seven teams have to agree on every change to one file. This tight link between services is called coupling: when one part changes or fails, the other parts are hurt too.
Event-driven architecture (EDA) is a way to build software where services talk by sending events. An event is a short message that says something has happened. EDA turns the design around. The order service no longer tells everyone what to do. It only announces what happened: "An order was placed". Any service that cares reacts.
The Core Idea
Think about a wedding announcement.
The command style: the family personally phones the caterer ("prepare food for 300"), the decorator ("set up flowers"), the photographer ("be there at 6") and 200 relatives. If the photographer does not pick up, the family is stuck on the phone. Adding a new vendor means one more call to make.
The event style: the family posts one announcement: "The wedding is on 14 Feb at 6 PM, at this venue." The caterer, decorator, photographer and guests each see it and do their own part. A new vendor who hears about it can join without the family doing anything.
- A command is a message that says "do this" to one specific receiver. It expects the work to be done.
- An event is a message that says "this happened" to anyone who is interested. The sender does not care who reacts.
How It Works
Events, commands and messages
| Command | Event | |
|---|---|---|
| Meaning | "Please do X" | "X happened" |
| Tense | Imperative: ChargeCard, SendEmail | Past: OrderPlaced, PaymentFailed |
| Receivers | One specific service | Zero, one or many |
| Can be refused? | Yes ("card declined") | No, it is a fact about the past |
| Coupling | Sender knows the receiver | Sender does not know who listens |
Both travel as messages over queues or streams (posts 35–36). A publisher is the service that sends an event. A subscriber is a service that asks to receive it.
Name events as past-tense business facts: OrderPlaced, PaymentCaptured, ShipmentDispatched, UserEmailChanged. Avoid names that describe technical details, like OrderTableRowUpdated.
The basic event-driven flow
- 1Order servicesaves the order + writes OrderPlaced to its outbox
- 2Event bus / Kafkadurable (it keeps events safely), replayable
- 3Inventoryreserves stock
- 4Emailsends the confirmation
- 5Loyaltyadds points — if it's down, events simply wait
- 6Analyticsrecords the sale
- 7Fraud check (new)just subscribes — no change to the order service
- The order service is now simple: save the order, publish the event (using the outbox pattern, post 37).
- New features just subscribe. Adding a fraud check means no change to the order service.
- A slow or broken loyalty service no longer affects checkout. Its events wait in the stream until it recovers.
Four patterns that are all called "event-driven"
Martin Fowler pointed out that "event-driven" can mean four different things. Here they are one by one.
1. Event notification. The event is small. It only says that something happened. Consumers call back to ask for the details if they need them.
{ "type": "OrderPlaced", "orderId": "5521" }- ✅ Small events, loose coupling.
- ❌ Consumers call the order service's API. This adds load, and the consumers now depend on the order service while they run.
2. Event-carried state transfer. The event carries all the data that consumers need.
{ "type": "OrderPlaced", "orderId": "5521", "userId": 42, "items": [{"productId": 991, "qty": 1, "price": 129900}], "total": 129900, "city": "Pune" }- ✅ Consumers do not need to call back. They can even keep their own local copy of the data.
- ❌ Bigger events and duplicated data (post 23). Schema evolution (changing the event format over time without breaking readers) becomes important.
3. Event sourcing. Normally you store only the current state (for example, the balance). With event sourcing you store every event that ever happened. You work out the current state by replaying (re-reading) them in order. A bank statement works like this.
- AccountOpenedbalance 0
- MoneyDeposited+10,000
- MoneyWithdrawn−2,500
- MoneyDeposited+1,000
- Replaycurrent balance = 8,500 — and the state on any past day
- ✅ A full audit history (a record of who did what and when), and "time travel" (what was the state last Tuesday?). You can also build new views by replaying events.
- ❌ More complex, and changing the format of old events is tricky. Long histories need snapshots (a saved copy of the state at one moment, so you do not replay everything). To ask "what is the current state?" you need projections (tables built by replaying the events).
- Good for: ledgers, banking, and areas with many audit rules. Too much for most CRUD apps (apps that only create, read, update and delete records).
4. CQRS (Command Query Responsibility Segregation). This is a design that uses separate models for writing (commands) and reading (queries). Writes go to a model built for correctness. Events then update one or more read models built for fast queries: a search index, a denormalised feed (data copied into one ready-to-read shape), or a reporting table (post 23).
- Commands
- Write modelnormalised, correct
- Read modelssearch · cache · feed tables
- Queries
Event sourcing and CQRS are often used together. You can also use either one alone.
Choreography vs orchestration
When a business process spans several services (like checkout), who is in charge of the steps?
Choreography means services react to each other's events, like dancers who each know their own moves. Nobody is in charge.
- OrderPlaced
- PaymentCaptured
- StockReserved
- ShipmentBooked
- Decoupled, but “where is order 5521 stuck?” means reading four services' logs
- ✅ Very decoupled, with no central bottleneck.
- ❌ You cannot see the whole flow. It is spread across many services, so it is hard to answer "where is order 5521 stuck?". Circular dependencies (A waits for B, and B waits for A) can appear by accident.
Orchestration means a central coordinator (the orchestrator) tells each service what to do with commands, and tracks the progress. It is like a conductor of an orchestra.
- Charge
- Charged
- Reserve
- Reserved
- Book courier
- Booked
- ↻ mark order confirmed
- ✅ The flow is clear and in one place. It is easy to monitor, change and debug.
- ❌ The orchestrator is an extra part to run, and the services stay somewhat tied to it.
Rule of thumb: use choreography for simple reactions that are only loosely related (send an email, update analytics). Use orchestration for important business processes with many steps (checkout, loan approval, onboarding). Workflow engines like Temporal or AWS Step Functions are often used as orchestrators (post 38).
Sagas: transactions across services
A monolith is one big application with one database. There, checkout can be one database transaction (a group of steps that all succeed or all fail, post 21). With many services, each with its own database, you cannot do that. A saga is a sequence of local transactions (one per service). Each step has a compensating action, which is a step that undoes it if a later step fails.
Example: the checkout saga when payment fails.
- Step 1create order (PENDING) — compensation: cancel order
- Step 2reserve stock — compensation: release stock
- Step 3charge payment FAILS
- Compensate 2release stock ✅
- Compensate 1cancel order ✅ and tell the user “payment failed”
Important points:
- Compensations are business actions, not automatic rollbacks (a rollback is a database undo). Examples: "refund", "release", "cancel", "send apology email".
- Every step and compensation must be idempotent (safe to run twice, post 37), because they may be retried.
- Users may see in-between states ("Order pending…"), so design the screens for that.
- Sagas can be choreographed (via events) or orchestrated (via a coordinator). Orchestration is usually easier to reason about for sagas.
Change Data Capture (CDC) as an event source
Sometimes you cannot, or do not want to, change old applications so that they publish events. Change Data Capture (CDC) is a technique that reads the database's change log (post 21) and turns every insert, update and delete into an event. CDC tools like Debezium send these events to Kafka. CDC is also a reliable way to build the outbox pattern (post 37).
Event schemas and evolution
Events are a contract between teams, just like APIs (post 34). A contract is a promise about the shape of the data:
- use a schema (a written description of the fields in an event, for example in Avro, Protobuf or JSON Schema) and a schema registry (a service that stores schemas and rejects changes that would break readers),
- make changes backward compatible, so old readers still work. Add optional fields. Never remove a field that consumers use, and never give it a new meaning,
- include standard metadata: event ID (for dedup), type, version, timestamp, source, and a correlation ID (one ID shared by all events from the same user action, used for tracing). The CloudEvents standard defines a common format for this.
{ "id": "evt-8f2a1c", "type": "com.shop.order.placed", "specversion": "1.0", "source": "order-service", "time": "2027-02-11T10:15:00Z", "datacontenttype": "application/json", "data": { "orderId": "5521", "userId": 42, "total": 129900 }}The challenges
- Eventual consistency. This means data in different places is not updated at the same moment, but it becomes the same after a short time. After you place an order, the loyalty points appear a second later. The UI must show "processing" states.
- Debugging and visibility. One user action starts a chain of events across services. To follow it you need correlation IDs and distributed tracing (tools that show one request as it moves through many services, Part 8).
- Ordering. Use per-key ordering (events with the same key, such as the order ID, go to the same partition) and version numbers in events (posts 35–37).
- Duplicates. At-least-once delivery means a message may arrive twice. So idempotent consumers are a must.
- Replay side effects. Replaying events to rebuild a read model is useful. But make sure a replay does not resend emails or charge cards again.
- "Event soup". This means too many badly named events that overlap and have no owner. Keep an event catalogue (a list of all events) with a clear owner for each.
Trade-offs
- ✅ Loose coupling: add features without touching the publisher.
- ✅ Resilience (the ability to keep working when parts fail): slow or broken consumers do not break the core flow.
- ✅ Scalability: each consumer can grow on its own.
- ✅ History and replay, especially with Kafka and event sourcing.
- ❌ Eventual consistency, and harder reasoning about "what state is the system in right now?".
- ❌ Harder debugging. You need tracing, good logs and tools.
- ❌ More infrastructure: brokers, schema registry, outbox relays, DLQs.
- ❌ Event sourcing and CQRS add big complexity. Use them only where the benefits (audit, history, many kinds of read models) are clear.
When not to go event-driven:
- small systems with one team,
- flows where the user needs an immediate, consistent answer,
- or when a simple synchronous API call (the caller waits for the answer) is clearer and good enough.
In the Real World
E-commerce order pipelines. Large retailers commonly publish events like "order placed", "payment captured" and "shipment dispatched". Warehouses, delivery partners, notifications, customer support tools and analytics each subscribe independently. This is why your order confirmation, tracking updates and loyalty points arrive at slightly different times.
Uber and ride events. Trip lifecycle events (requested, accepted, started, completed) flow through Kafka to many systems: pricing, payments, driver earnings, fraud detection, maps and analytics. Each system handles them on its own, at a very large scale.
Banking and ledgers. Financial systems naturally fit event sourcing: an account's balance is the result of every deposit and withdrawal, and the full history must be kept for audits. Many modern banking and payment platforms use ledgers that are immutable (never changed) and append-only (you only add new records at the end).
Git as an event-sourcing analogy. Git keeps every commit, not just the latest files. You can go back to any point in history, create branches, and get the current state from the log of commits. This is the core idea of event sourcing, in a tool most developers use every day. (The match is not exact: Git commits are saved snapshots, not small events.)
Workflow orchestration. Companies running complex flows (loan applications, insurance claims, e-commerce fulfilment) often use orchestrated sagas with tools like Temporal or AWS Step Functions, so every step, retry and compensation is visible and can be checked later.
Interview Questions
Q1What's the difference between an event and a command?
A command is an instruction to one specific receiver (“ChargeCard”), and it can be refused. An event is a fact in the past tense (“OrderPlaced”) that goes to anyone who is interested. The publisher does not know or care who reacts.
Q2Choreography or orchestration for a checkout flow?
Usually orchestration: checkout is a critical multi-step process where you need to see and control where each order is, handle timeouts and run compensations. Choreography suits simple, loosely related reactions like emails and analytics.
Q3What is a saga?
A way to keep data consistent across services without a distributed transaction: a sequence of local transactions, each with a compensating action (refund, release, cancel) that undoes it if a later step fails. Every step and compensation must be idempotent.
Q4What is event sourcing, and when is it worth it?
You store every change as an event that is never edited. You work out the current state by replaying the events. It pays off for ledgers and other areas that need full history and time travel. For ordinary CRUD it adds extra work (projections, snapshots, event versions) for little gain.
Q5Event notification vs event-carried state transfer?
Notification sends a thin event and consumers call back for details, creating runtime coupling and extra load. Event-carried state transfer includes the data consumers need, so they can act or keep a local copy without calling back, at the cost of bigger events and duplicated data.
Q6What makes event-driven systems hard to operate?
Eventual consistency in the UI, following one user action across many services (you need correlation IDs and tracing), per-key ordering, duplicate deliveries that demand idempotency, replays that must not resend side effects, and event schemas that need versioning and ownership.
Key Takeaways
- Commands say "do this" to one receiver. Events say "this happened" to anyone interested. Name events as past-tense business facts.
- The four event-driven patterns are event notification, event-carried state transfer, event sourcing and CQRS. Use the heavier ones only where they pay off.
- Choreography (react to events) is decoupled but hard to see. Orchestration (a central coordinator) is explicit and easier for important processes.
- Sagas replace cross-service transactions with local steps plus compensating actions, all idempotent.
- Treat events as contracts (schemas, versioning, IDs, CloudEvents), publish reliably (outbox / CDC), and invest in tracing and idempotency.
Further Reading
- Martin Fowler's articles "What do you mean by 'Event-Driven'?", "Event Sourcing" and "CQRS"
- microservices.io pattern "Saga"
- Azure Architecture Center "Event-driven architecture style"
- The original paper "Sagas" by Hector Garcia-Molina and Kenneth Salem (1987)
- The Debezium documentation and the CloudEvents specification
- Building Event-Driven Microservices by Adam Bellemare
This wraps up Part 6. Next up, Part 7: Reliability & Resilience, starting with "Single Points of Failure, Redundancy & Failover".
Published by Jainik · Last updated · How these lessons are made
Keep learning
Related series