Debugging and Troubleshooting AI Systems Questions
Covers systematic approaches to find and fix failures in machine learning and artificial intelligence systems. Topics include common failure modes such as poor data quality, incorrect preprocessing, label errors, data leakage, training instability, vanishing or exploding gradients, numerical precision issues, overfitting and underfitting, optimizer and hyperparameter problems, model capacity mismatch, implementation bugs, hardware and memory failures, and production environment issues. Skills and techniques include data validation and exploratory data analysis, unit tests and reproducible experiments, sanity checks and simplified models, gradient checks and plotting training dynamics, visualizing predictions and errors, ablation studies and feature importance analysis, logging and instrumentation, profiling for latency and memory, isolating components with canary or shadow deployments, rollback and mitigation strategies, monitoring for concept drift, and applying root cause analysis until the underlying cause is found. Interviewers assess the candidate on their debugging process, ability to isolate issues, use of tools and metrics for diagnosis, trade offs in fixes, and how they prevent similar failures in future iterations.
EasyTechnical
0 practiced
Describe three practical techniques (automated and manual) to detect label errors in a supervised dataset of 10 million examples intended for a binary classifier. For each technique explain why it surfaces label errors, how you prioritize samples for human review given limited labeling budget, and an example metric that would guide triage.
MediumTechnical
0 practiced
Design a minimal set of unit and integration tests for an ML training pipeline that SREs and ML engineers can run in CI to catch regressions likely to cause production failures (examples: data schema drift, training crash, sharp metric regression). For each test specify runtime/resource budget and the failure signal it detects.
EasyTechnical
0 practiced
Explain finite-difference gradient checking for small neural networks: why it helps, how to implement it, and what limitations make it impractical for large or stochastic networks. Provide short Python-like pseudocode (3-6 lines) that demonstrates the main numerical gradient check loop.
HardTechnical
0 practiced
A vulnerability in your model-serving endpoint allows crafted inputs to leak training data via model outputs. Describe immediate mitigations (rate-limiting, sanitize outputs, block suspicious inputs), how to assess the scope of exfiltration, and long-term fixes including adversarial testing, output filtering, and privacy-preserving training. Also include how SREs would evidence and monitor for repeat attacks.
MediumTechnical
0 practiced
You maintain SLOs: P99 latency < 500ms and weekly AUC > 0.85. A new model improves AUC to 0.88 but raises P99 to 650ms. As SRE, describe how you would analyze trade-offs, use the error budget to make a rollout decision, and propose mitigations (architectural or operational) to attempt to keep improved AUC without violating latency SLOs.
Unlock Full Question Bank
Get access to hundreds of Debugging and Troubleshooting AI Systems interview questions and detailed answers.
