Stream Processing and Event Streaming Questions

Designing and operating systems that ingest, process, and serve continuous event streams with low latency and high throughput. Core areas include architecture patterns for stream native and event driven systems, trade offs between batch and streaming models, and event sourcing concepts. Candidates should demonstrate knowledge of messaging and ingestion layers, message brokers and commit log systems, partitioning and consumer group patterns, partition key selection, ordering guarantees, retention and compaction strategies, and deduplication techniques. Processing concerns include stream processing engines, state stores, stateful processing, checkpointing and fault recovery, processing guarantees such as at least once and exactly once semantics, idempotence, and time semantics including event time versus processing time, watermarks, windowing strategies, late and out of order event handling, and stream to stream and stream to table joins and aggregations over windows. Performance and operational topics cover partitioning and scaling strategies, backpressure and flow control, latency versus throughput trade offs, resource isolation, monitoring and alerting, testing strategies for streaming pipelines, schema evolution and compatibility, idempotent sinks, persistent storage choices for state and checkpoints, and operational metrics such as stream lag. Familiarity with concrete technologies and frameworks is expected when discussing designs and trade offs, for example Apache Kafka, Kafka Streams, Apache Flink, Spark Structured Streaming, Amazon Kinesis, and common serialization formats such as Avro, Protocol Buffers, and JSON.

MediumTechnical

64 practiced

Describe a practical testing strategy for streaming pipelines. Cover unit tests for operators (function-level), integration tests with embedded brokers (e.g., Kafka TestContainers), property-based/time-based tests for event-time semantics, and end-to-end tests that validate correctness during failure scenarios. Include how to assert watermarks and late-event behavior.

EasyTechnical

49 practiced

Explain what a commit log is and how a distributed commit-log system such as Apache Kafka stores messages, maintains ordering, and exposes offsets to consumers. Describe how partitions enable parallelism while preserving per-partition ordering, and briefly explain retention and log compaction and when you would use each.

MediumTechnical

36 practiced

Coding (Python): Implement a simple event-time tumbling window aggregator function aggregate_by_minute(events, allowed_lateness_seconds) that consumes a list of events {'event_time':int,'value':int}. Simulate a watermark that is max_seen_event_time - allowed_lateness, emit sums for each 60s window when watermark passes the window end. Show sample out-of-order input and expected output.

EasyTechnical

37 practiced

Compare Avro, Protocol Buffers (Protobuf), and JSON as serialization formats for event streams. Explain trade-offs in size, parsing speed, schema evolution capabilities, and runtime compatibility. When and why would you use a schema registry?

MediumTechnical

46 practiced

Explain at-least-once, at-most-once, and exactly-once processing guarantees in streaming systems. For each guarantee, describe how producers, brokers, and stream processors typically implement them (e.g., idempotent producers, transactions, offset commit semantics), and practical limitations in real systems.

Unlock Full Question Bank

Get access to hundreds of Stream Processing and Event Streaming interview questions and detailed answers.

Join thousands of developers preparing for their dream job.