Designing reliable CDC-driven indexing pipelines
Keeping a search index faithful to its source database when the only link between them is a change stream.
Technical writing
Notes on the problems behind the systems: consistency, event processing, search and operability.
Articles
Nothing is published yet. The topics below are in preparation.
In preparation
Topics I’m writing about. Each will appear above once it’s finished.
Handling duplicate and out-of-order events in Kafka
What delivery guarantees actually promise, and what the consumer still has to do itself.
Designing idempotent event consumers
Patterns for making reprocessing safe, and the cases where idempotency is harder than it looks.
Hybrid search: lexical and semantic retrieval
Combining keyword relevance with vector similarity, and the ranking problems that appear where they meet.
Understanding consistency in distributed systems
A working engineer’s map of consistency models, and how to tell which one a system actually provides.
Engineering observability into distributed applications
Designing the signals an on-call engineer will need before the system ships, not after the first incident.