InterviewStack.io LogoInterviewStack.io

Hypothesis and Test Planning Questions

End to end practice of generating clear testable hypotheses and designing experiments to validate them. Candidates should be able to structure hypotheses using if change then expected outcome because reasoning ground hypotheses in data or qualitative research and distinguish hypotheses from guesses. They should translate hypotheses into experimental variants and choose the appropriate experiment type such as A and B tests multivariate designs or staged rollouts. Core skills include defining primary and guardrail metrics that map to business goals selecting target segments and control groups calculating sample size and duration driven by statistical power and minimum detectable effect and specifying analysis plans and stopping rules. Candidates should be able to pre register plans where appropriate estimate implementation effort and expected impact specify decision rules for scaling or abandoning variants and describe iteration and follow up analyses while avoiding common pitfalls such as peeking and selection bias.

MediumTechnical
0 practiced
Design an end-to-end experiment plan to improve activation rate for new users (activation = first key action within 7 days). Include: a clear hypothesis, primary and guardrail metrics, target segment, required sample size estimate (justify assumptions), rollout plan, stopping rules, instrumentation checklist, and decision criteria for scaling.
EasyTechnical
0 practiced
Given a Postgres events table: events(user_id bigint, event_type text, occurred_at timestamptz, variant text). Write a SQL query to compute 7-day conversion rate (percentage of users who triggered 'purchase' within 7 days of signup) by variant using users' first-event as cohort. Show per-variant sample size and conversion rate.
EasyTechnical
0 practiced
Rewrite this vague hypothesis into a testable hypothesis: 'We think redesigning the homepage will increase engagement.' Use the if...then...because template and specify the target segment, primary metric, measurement window, and at least one guardrail.
MediumTechnical
0 practiced
You have 12 proposed experiments and two engineers. Explain how you would prioritize which experiments to run using a framework (RICE, ICE, or custom). Provide a filled example scoring for the top three experiments and justify your choices.
HardTechnical
0 practiced
You expect conversion to rise from about 2.0% to 2.2%. Using a Beta-Binomial approach with a prior Beta(2,98) reflecting a 2% baseline, explain conceptually how you would compute the posterior probability that the variant is better than control. What decision threshold (posterior probability) would you recommend for scaling and why?

Unlock Full Question Bank

Get access to hundreds of Hypothesis and Test Planning interview questions and detailed answers.

Sign in to Continue

Join thousands of developers preparing for their dream job.