Event-Driven Architecture with Apache Kafka and Spring Boot

Most distributed systems fail not because a service is slow, but because services are too tightly coupled — one synchronous call chains into another until a single slow dependency takes everything down. Event-driven architecture (EDA) breaks that chain. Services publish facts ("an order was placed") to a log, and other services react on their own schedule. Apache Kafka is the durable, partitioned log that makes this practical at scale.

When event-driven is the right call

EDA is not a default — it's a trade-off. Reach for it when you have real decoupling needs:

• Independent scaling: the service that places orders and the service that emails receipts have very different load profiles.
• Resilience: if the email service is down, orders still succeed; events wait in the log.
• Fan-out: one event ("payment captured") triggers many reactions — fulfilment, analytics, fraud checks — with no change to the producer.
• Auditability: the event log is a replayable history of everything that happened.

If your operations are simple request/response and strongly consistent, a synchronous REST or gRPC call is simpler and you should use it. Don't add Kafka for its own sake.

Producing and consuming with Spring Boot

Spring for Apache Kafka makes producers and consumers a few lines each. A producer publishes to a topic; a consumer joins a consumer group, and Kafka spreads the topic's partitions across the group members so you scale by adding instances.

@Service
public class OrderEvents {
  private final KafkaTemplate<String, OrderPlaced> kafka;

  public void publish(OrderPlaced event) {
    // key by orderId so all events for one order keep their order
    kafka.send("orders.placed", event.orderId(), event);
  }
}

@KafkaListener(topics = "orders.placed", groupId = "fulfilment")
public void onOrder(OrderPlaced event) {
  fulfilmentService.reserve(event);   // react to the fact
}

The patterns that keep it reliable

EDA's failure modes are different from synchronous systems. Three patterns matter most:

• Idempotent consumers. Kafka guarantees at-least-once delivery, so the same event can arrive twice. Make handlers idempotent — track processed event IDs, or use upserts — so a redelivery is a no-op.
• Partition keys for ordering. Kafka only orders messages within a partition. Key events by the entity ID (e.g. orderId) so all events for one order land in the same partition and stay ordered.
• The transactional outbox. Never write to your database and publish to Kafka in two separate steps — one can succeed while the other fails. Write the event to an outbox table in the same DB transaction, then relay it to Kafka. This guarantees the event fires if and only if the data was committed.

Dead letters and observability

Some events can't be processed — bad data, a downstream that's permanently rejecting. Route those to a dead-letter topic after a bounded number of retries so a single poison message doesn't block the partition. Then instrument everything: consumer lag is the single most important metric in a Kafka system. Rising lag means consumers can't keep up, and it's your earliest warning before users feel it.

Key takeaways

• Use EDA to decouple services that scale or fail independently — not as a default for every call.
• Key messages by entity ID for ordering, and make every consumer idempotent.
• Use the transactional outbox to avoid dual-write inconsistency, and monitor consumer lag above all.

We design and run Kafka-based event-driven systems in production. Read how in our case studies, or talk to an architect about your platform.