Metric Selection & Product Instrumentation Questions
Techniques for turning vague business questions into measurable, actionable product metrics. Includes identifying leading vs. lagging indicators, upstream vs. downstream metrics, aligning metrics with company strategy, balancing multiple stakeholders (user satisfaction, business growth, content value), and recognizing when metrics can be misleading or require multiple signals to capture impact.
MediumTechnical
75 practiced
An A/B experiment shows a reported 10% increase in a 'new_event_count' metric for treatment but no change in conversion. Describe step-by-step how you'd investigate whether the instrumentation caused the spike (false positive) versus a real behavioral change. Include SQL checks, event-schema audits, and suggestions for short-term and long-term fixes.
EasyTechnical
100 practiced
Compare measuring 'time on page' using client-side timers (e.g., JavaScript activity tracking) versus server-side approximations based on request timestamps. List the pros and cons of each approach and recommend which you'd implement for a news website where article engagement is a key success metric.
MediumTechnical
82 practiced
Write SQL (standard SQL dialect) to compute 7-day rolling retention for new users: for each cohort defined by signup_date, compute percentage of users who were active on day 7 since signup. Use tables: users(user_id, signup_at TIMESTAMP) and events(user_id, occurred_at TIMESTAMP, event_name). Explain how your query handles timezone normalization and users with missing events.
MediumTechnical
75 practiced
When should you report absolute counts versus rates versus weighted metrics? Give examples in product analytics where each is appropriate, and explain pitfalls of relying on absolute counts without normalization. Include a short rule-of-thumb checklist for choosing the right representation.
HardTechnical
80 practiced
You observed DAU increased 12% after a UI overhaul, but revenue per user dropped 8%. Design an analysis plan to find the root cause and identify missing instrumentation. Include SQL queries or pseudo-code fragments you would run, segments to analyze, and hypotheses to test (e.g., new users, session quality).
Unlock Full Question Bank
Get access to hundreds of Metric Selection & Product Instrumentation interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.