Covers designing and operating machine learning systems to handle growth in data volume, model complexity, and traffic. Topics include distributed training strategies such as data parallelism, model parallelism, and pipeline parallelism; coordination and orchestration approaches like parameter servers, gradient aggregation, and framework tools such as PyTorch distributed, Horovod, and TensorFlow strategies; data pipeline and I O considerations including sharding, efficient formats, preprocessing bottlenecks, streaming and batch ingestion; serving and inference scaling including model sharding, batching for throughput, autoscaling, request routing, caching, and latency versus throughput tradeoffs. Also includes monitoring, profiling, checkpointing and recovery, reproducibility, cost and resource optimization, and common bottleneck analysis across network, storage, CPU preprocessing, and accelerator utilization.
MediumTechnical
28 practiced
Write a PySpark transformation that computes per-user daily aggregates from an events dataset while minimizing shuffle. Assume events(partitioned by event_date) with columns (user_id, event_type, value). Show how you would use partitioning and map-side combiners to reduce shuffle volume.
MediumTechnical
26 practiced
You observe that distributed training jobs underutilize GPUs: GPU utilization is 20% while CPU utilization is 80% and data loader queue is empty. List likely root causes and propose concrete fixes (code, infra, and configuration) to increase GPU utilization for a PyTorch job reading image data from S3.
EasyTechnical
29 practiced
Define data sharding and partitioning in the context of ML training data. Describe three common sharding strategies (hash-based, range-based, round-robin) and for each give an example use case and one drawback that a data engineer should watch for.
MediumTechnical
36 practiced
Explain strategies to handle preemptible or spot instances for distributed training: checkpoint policies, job orchestration (e.g., using Kubernetes, Ray, or SLURM), and checkpoint storage choices. Describe how to minimize lost work while keeping costs low.
HardTechnical
50 practiced
You manage a training platform with 10 PB of dataset storage and hundreds of hyperparameter tuning runs. Propose a roadmap to reduce overall training cost by 3x while keeping model quality similar. Consider data tiering, incremental datasets, search strategies, spot instances, caching, and infrastructure changes.
Unlock Full Question Bank
Get access to hundreds of AI System Scalability interview questions and detailed answers.
