Covers designing and operating machine learning systems to handle growth in data volume, model complexity, and traffic. Topics include distributed training strategies such as data parallelism, model parallelism, and pipeline parallelism; coordination and orchestration approaches like parameter servers, gradient aggregation, and framework tools such as PyTorch distributed, Horovod, and TensorFlow strategies; data pipeline and I O considerations including sharding, efficient formats, preprocessing bottlenecks, streaming and batch ingestion; serving and inference scaling including model sharding, batching for throughput, autoscaling, request routing, caching, and latency versus throughput tradeoffs. Also includes monitoring, profiling, checkpointing and recovery, reproducibility, cost and resource optimization, and common bottleneck analysis across network, storage, CPU preprocessing, and accelerator utilization.
HardTechnical
0 practiced
Discuss model quantization and pruning as techniques to reduce inference cost. For a production deployment, quantify expected improvements (latency and memory), explain the impact on accuracy and calibration, and list operational changes required to include quantized models in CI/CD and monitoring.
MediumTechnical
0 practiced
Compare Horovod, PyTorch Distributed Data Parallel (DDP), and TensorFlow's MirroredStrategy in terms of ease of integration into existing training code, performance at scale, fault tolerance, and ecosystem/tooling support. Which would you recommend for fast-prototyping vs large-scale training?
MediumTechnical
0 practiced
Using PyTorch's torch.distributed API, write concise Python pseudocode (or real code) that initializes a distributed training process across N processes and performs a simple gradient averaging step using all-reduce after backward(). Include initialization, model wrapping, and the all-reduce call for gradients.
HardTechnical
0 practiced
A production training job diverged recently: loss suddenly exploded after a dataset ingestion pipeline change. Outline a debugging plan to determine whether the cause is data corruption, schema drift, preprocessing changes, or code regressions. List the exact checks and lightweight reproducible experiments you would run to isolate the root cause.
HardSystem Design
0 practiced
Architect a multi-region ML training and serving platform for a company serving users across North America, Europe, and APAC. Requirements: model training can be centralized but serving must be regional with <100ms latency; model updates are frequent (daily); regulatory constraints require that raw user data never leaves origin region. Describe data replication strategy, model artifact distribution, and how you ensure consistent feature computation across regions.
Unlock Full Question Bank
Get access to hundreds of AI System Scalability interview questions and detailed answers.
