Stream Processing and Event Streaming Questions

Designing and operating systems that ingest, process, and serve continuous event streams with low latency and high throughput. Core areas include architecture patterns for stream native and event driven systems, trade offs between batch and streaming models, and event sourcing concepts. Candidates should demonstrate knowledge of messaging and ingestion layers, message brokers and commit log systems, partitioning and consumer group patterns, partition key selection, ordering guarantees, retention and compaction strategies, and deduplication techniques. Processing concerns include stream processing engines, state stores, stateful processing, checkpointing and fault recovery, processing guarantees such as at least once and exactly once semantics, idempotence, and time semantics including event time versus processing time, watermarks, windowing strategies, late and out of order event handling, and stream to stream and stream to table joins and aggregations over windows. Performance and operational topics cover partitioning and scaling strategies, backpressure and flow control, latency versus throughput trade offs, resource isolation, monitoring and alerting, testing strategies for streaming pipelines, schema evolution and compatibility, idempotent sinks, persistent storage choices for state and checkpoints, and operational metrics such as stream lag. Familiarity with concrete technologies and frameworks is expected when discussing designs and trade offs, for example Apache Kafka, Kafka Streams, Apache Flink, Spark Structured Streaming, Amazon Kinesis, and common serialization formats such as Avro, Protocol Buffers, and JSON.

EasyTechnical

41 practiced

List and explain at least six operational metrics you would monitor for a streaming pipeline that feeds business dashboards. For each metric, explain why it matters to freshness, correctness, or cost (examples: consumer lag, throughput, processing latency, checkpoint age, error rate, GC pause time).

MediumTechnical

49 practiced

You have a Kafka topic of user profile update events (user_id, profile_json, updated_at). Explain when you would use log compaction vs time-based retention for this topic. Discuss schema evolution concerns and how you would make downstream analytics consumers resilient to fields being added or removed.

HardTechnical

42 practiced

You must migrate a critical real-time analytics pipeline from on-prem Kafka to a managed cloud Kafka (Confluent Cloud or AWS MSK). Outline migration steps that minimize downtime and ensure data consistency: cross-cluster mirroring, updating producers/consumers, handling schema registry migration, and validating post-cutover metrics.

HardTechnical

42 practiced

As a senior Data Analyst asked to prioritize streaming investments across product lines (lowering latency, improving exactly-once, adding stream joins, better monitoring), describe a framework to evaluate business ROI, criteria for prioritization, and a proposed phased roadmap. Include stakeholders and KPIs you would use to justify investments.

HardTechnical

33 practiced

Given a stream of financial trades with schema (trade_id STRING, symbol STRING, quantity INT, price DOUBLE, trade_time TIMESTAMP), design a streaming job to compute VWAP (volume-weighted average price) per symbol with sub-second latency and fault tolerance. Explain how to handle out-of-order/late trades, duplicates, and exactly-once writes to downstream storage.

Unlock Full Question Bank

Get access to hundreds of Stream Processing and Event Streaming interview questions and detailed answers.

Join thousands of developers preparing for their dream job.