Data Architecture and Pipelines Questions
Designing data storage, integration, and processing architectures. Topics include relational and NoSQL database design, indexing and query optimization, replication and sharding strategies, data warehousing and dimensional modeling, ETL and ELT patterns, batch and streaming ingestion, processing frameworks, feature stores, archival and retention strategies, and trade offs for scale and latency in large data systems.
MediumTechnical
0 practiced
Design a lakehouse architecture on object storage (S3/GCS) using Delta Lake / Iceberg / Hudi. Explain how ACID transactions are implemented (transaction log), how schema evolution and time travel work, the role of compaction/optimization (vacuum/cleanups), and how query engines like Spark and Trino/Presto interact with it. Discuss trade-offs between these table formats.
HardTechnical
0 practiced
Design online feature storage for extremely high-cardinality entities (billions of keys) requiring low-latency lookups (<10ms) with constrained memory (e.g., <64GB). Discuss storage engines (LSM-based stores, RocksDB), sharding, compression/serialization formats, caches, TTLs, and strategies to reduce read amplification and cost.
HardSystem Design
0 practiced
Design an analytics platform that supports hundreds of concurrent ad-hoc BI users querying petabytes of data while ensuring predictable latency for critical dashboards. Include storage layout, query engine choices (Presto/Trino/BigQuery), caching layers, materialized views, workload isolation, admission control, and how you'd auto-scale components.
HardTechnical
0 practiced
Discuss the trade-offs between eventual consistency and strong consistency for analytics pipelines. For use cases such as near-real-time dashboards, financial reconciliation, and fraud detection, recommend consistency models and architectural patterns (e.g., materialized views, change logs, two-phase commits) that meet each requirement.
EasyTechnical
0 practiced
Define idempotency in the context of data pipelines and ingestion systems. List practical strategies to make a pipeline idempotent (examples: upserts, deduplication, idempotent writes, checkpoints, message dedup keys) and discuss trade-offs for each approach.
Unlock Full Question Bank
Get access to hundreds of Data Architecture and Pipelines interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.