AI and Machine Learning Background Questions
A synopsis of applied artificial intelligence and machine learning experience including models, frameworks, and pipelines used, datasets and scale, production deployment experience, evaluation metrics, and measurable business outcomes. Candidates should describe specific projects, roles played, research versus production distinctions, and technical choices and trade offs.
EasyTechnical
0 practiced
Explain why offline evaluation metrics (AUC, F1) may not directly map to production business metrics (conversion, revenue). From an SRE standpoint, which production signals should you monitor to detect model-induced business regressions and how would you instrument them?
MediumTechnical
0 practiced
Write a Python script to parse model inference logs where each line is JSON like {"timestamp":"2025-01-01T12:00:00Z","request_id":"abc123","latency_ms":123,"status":"ok"}. The script should compute p50 and p95 latency and the overall error rate for the last 24 hours, assuming logs are stored in a file. Describe how you'd scale this for very large files.
MediumTechnical
0 practiced
You observe an increase in tail latency (p99) for model inferences correlated with a spike in feature-store read latency. As SRE, design an investigation plan, immediate mitigations, and a permanent fix path that addresses both infra and software changes.
HardTechnical
0 practiced
Implement a Python function compute_kl(p_counts, q_counts, smoothing=1e-6) that computes the Kullback-Leibler divergence between two histograms represented as lists of counts. Handle zero counts robustly via smoothing and return the divergence in nats. Explain numerical-stability decisions and complexity.
EasyTechnical
0 practiced
A production model's reported accuracy drops by 10% overnight. As on-call SRE, list the prioritized checklist you would run in the first 30 minutes to identify likely causes and temporary mitigations to reduce user impact (e.g., roll back, rate-limit, serve fallback model).
Unlock Full Question Bank
Get access to hundreds of AI and Machine Learning Background interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.