Stream Processing and Event Streaming Questions

Designing and operating systems that ingest, process, and serve continuous event streams with low latency and high throughput. Core areas include architecture patterns for stream native and event driven systems, trade offs between batch and streaming models, and event sourcing concepts. Candidates should demonstrate knowledge of messaging and ingestion layers, message brokers and commit log systems, partitioning and consumer group patterns, partition key selection, ordering guarantees, retention and compaction strategies, and deduplication techniques. Processing concerns include stream processing engines, state stores, stateful processing, checkpointing and fault recovery, processing guarantees such as at least once and exactly once semantics, idempotence, and time semantics including event time versus processing time, watermarks, windowing strategies, late and out of order event handling, and stream to stream and stream to table joins and aggregations over windows. Performance and operational topics cover partitioning and scaling strategies, backpressure and flow control, latency versus throughput trade offs, resource isolation, monitoring and alerting, testing strategies for streaming pipelines, schema evolution and compatibility, idempotent sinks, persistent storage choices for state and checkpoints, and operational metrics such as stream lag. Familiarity with concrete technologies and frameworks is expected when discussing designs and trade offs, for example Apache Kafka, Kafka Streams, Apache Flink, Spark Structured Streaming, Amazon Kinesis, and common serialization formats such as Avro, Protocol Buffers, and JSON.

MediumSystem Design

38 practiced

Design a sink connector that writes streaming events into Postgres while guaranteeing no duplicates despite producer retries (upstream at-least-once). Discuss using upserts, idempotency keys, and a buffered transactional approach. Include failure handling and recovery steps you would operationalize as the SRE.

HardTechnical

49 practiced

Design SLOs and an error budget policy for a streaming pipeline that feeds customer-facing dashboards and alerts. Define measurable indicators (event freshness, completeness, duplicates), concrete thresholds, error budget burn actions (e.g., stop non-critical jobs), and how to automate enforcement and communicate with stakeholders.

EasyTechnical

35 practiced

You're designing partition keys for a topic that stores user events and telemetry. Describe factors an SRE should consider when choosing a partition key (e.g., cardinality, skew, ordering requirements, hot partitions, re-sharding implications). Propose key selection strategies for low-latency per-user ordering vs high-throughput aggregated processing.

MediumTechnical

45 practiced

You run a schema-registry-backed Avro stream used by many consumers. A producer needs to add fields to the schema. Describe how to plan, test, and roll out schema changes to ensure backward and forward compatibility. Include compatibility checks, consumer testing, and rollback strategies an SRE should implement.

MediumTechnical

43 practiced

Define SLOs for a streaming pipeline that powers a real-time dashboard: include metrics for event freshness (max end-to-end latency), completeness (percentage of events processed), and correctness (duplicate rate). Propose alerting thresholds, runbooks, and automated remediation steps when error budgets are breached.

Unlock Full Question Bank

Get access to hundreds of Stream Processing and Event Streaming interview questions and detailed answers.

Join thousands of developers preparing for their dream job.