Stream Processing and Event Streaming Questions

Designing and operating systems that ingest, process, and serve continuous event streams with low latency and high throughput. Core areas include architecture patterns for stream native and event driven systems, trade offs between batch and streaming models, and event sourcing concepts. Candidates should demonstrate knowledge of messaging and ingestion layers, message brokers and commit log systems, partitioning and consumer group patterns, partition key selection, ordering guarantees, retention and compaction strategies, and deduplication techniques. Processing concerns include stream processing engines, state stores, stateful processing, checkpointing and fault recovery, processing guarantees such as at least once and exactly once semantics, idempotence, and time semantics including event time versus processing time, watermarks, windowing strategies, late and out of order event handling, and stream to stream and stream to table joins and aggregations over windows. Performance and operational topics cover partitioning and scaling strategies, backpressure and flow control, latency versus throughput trade offs, resource isolation, monitoring and alerting, testing strategies for streaming pipelines, schema evolution and compatibility, idempotent sinks, persistent storage choices for state and checkpoints, and operational metrics such as stream lag. Familiarity with concrete technologies and frameworks is expected when discussing designs and trade offs, for example Apache Kafka, Kafka Streams, Apache Flink, Spark Structured Streaming, Amazon Kinesis, and common serialization formats such as Avro, Protocol Buffers, and JSON.

EasyTechnical

63 practiced

Explain event sourcing and how it differs from traditional CRUD storage. For an ML feature pipeline, when would event sourcing be useful? Describe how replayability of events might be used to rebuild features or retrain models.

EasyTechnical

59 practiced

You're building an ML inference pipeline that ingests telemetry events in real time. Explain the key architectural and semantic differences between a traditional message-queue (e.g., RabbitMQ) and a commit-log system (e.g., Apache Kafka). Discuss ordering guarantees, retention and replay capabilities, consumer semantics, and typical failure/recovery trade-offs. When would you choose one over the other for model feature ingestion and why?

MediumSystem Design

36 practiced

You need to enrich a stream of click events with user profile attributes stored in a slowly changing table. Design a stream-to-table join strategy that supports near-real-time updates while tolerating occasional profile updates arriving late. Discuss caching, TTL for profile entries, consistency trade-offs, and how to implement this in Kafka Streams or Flink.

EasyTechnical

40 practiced

Implement a simple in-memory deduplication layer in Python for a streaming consumer that may receive duplicate events. Each event is a JSON object with an 'event_id' and 'timestamp'. Requirements: deduplicate within a sliding time window of 10 minutes, bound memory usage, and explain time/space complexity. You may use standard Python libraries.

MediumTechnical

45 practiced

A downstream ML training job can't keep up with the incoming event rate and needs graceful backpressure handling. Describe producer and consumer-side strategies to handle backpressure and flow control in a streaming pipeline (e.g., Kafka + Flink). Include rate-limiting, buffering, adaptive batching, flow control signals, and how to avoid data loss while maintaining acceptable latency.

Unlock Full Question Bank

Get access to hundreds of Stream Processing and Event Streaming interview questions and detailed answers.

Join thousands of developers preparing for their dream job.