Covers the design and operation of incident response programs and the creation and maintenance of actionable runbooks and playbooks for production systems. Candidates should be able to explain the incident lifecycle from detection and classification through investigation, escalation, remediation, and post incident analysis. Topics include severity definitions and assessment, escalation procedures, team roles and responsibilities, communication protocols during incidents, on call rotations, alert triage, and coordination across teams during outages. Also includes designing automated remediation steps where appropriate, integrating runbooks with monitoring and alerting systems, maintaining playbooks for common failure modes such as malware, data exfiltration, denial of service, and account compromise, and conducting blameless post incident reviews and continuous improvement. Candidates should be able to discuss metrics for measuring response effectiveness such as mean time to detect, mean time to repair, and response success rate, and describe approaches to improve those metrics over time.
MediumTechnical
0 practiced
Create an actionable runbook template (in markdown) for the scenario: 'scheduled Spark ETL job failing with executor OOM'. Include detection signals, immediate mitigation commands (YARN or Kubernetes), safe configuration changes to try, validation checks after restart, rollback considerations, communication steps, and placeholders for cluster/job identifiers and logs.
MediumTechnical
0 practiced
Propose an operational process for maintaining runbooks in a medium-sized data engineering org: define runbook ownership, review cadence, testing expectations, deprecation policy, linking to CI/CD pipelines, and incentives for engineers to update runbooks after incidents.
MediumTechnical
0 practiced
Draft an incident communication protocol template for a data-platform outage that affects downstream analytics. Include initial notification content, cadence of status updates, stakeholders to notify (engineers, analytics, product, customers), recommended communication mediums, and when to open a bridge call or involve leadership.
MediumTechnical
0 practiced
Design a rule-based method to automatically assign incident severity for data pipeline alerts using inputs such as data freshness delay, consumer lag, failed-records count, and SLA breaches. Describe threshold design, combination rules (e.g., weighted scoring, rule precedence), hysteresis windows, and safeguards to prevent misclassification during noisy periods.
EasyTechnical
0 practiced
Describe best practices for designing on-call rotations for a data engineering team that supports both batch ETL and streaming systems. Include frequency and length of shifts, handover/checklist, fatigue mitigation, escalation policies, compensation/expectations, and how on-call engineers access runbooks and tooling during an incident.
Unlock Full Question Bank
Get access to hundreds of Incident Response and Runbook Design interview questions and detailed answers.
