Hypothesis Testing and Inference Questions

Fundamental framework and application of hypothesis testing and statistical inference. Topics include formulating null and alternative hypotheses, understanding Type I and Type II errors, interpreting p values and confidence intervals, selecting and applying common tests such as t tests, chi square tests, analysis of variance, and non parametric alternatives, checking test assumptions, and discussing statistical versus practical significance. Candidates should explain power, significance levels, effect sizes, and common pitfalls such as misinterpreting p values or violating independence assumptions. At more advanced levels, discuss limitations of null hypothesis significance testing, alternatives such as Bayesian inference, and guidance for when different approaches are appropriate.

HardTechnical

31 practiced

Design a simulation study to evaluate the Type I error and power of a new nonstandard test statistic under realistic data-generating processes that include heteroskedasticity, skewness, and missingness. Describe how you would select scenarios, generate data, run repeated simulations, compute evaluation metrics (Type I error, power, coverage), and present results to stakeholders.

MediumTechnical

48 practiced

Write a Python function that accepts a contingency table (2D numpy array) and automatically selects and runs the appropriate test: chi-square test when expected counts are adequate, Fisher's exact test for 2x2 small tables, or Monte Carlo chi-square approximation when table is larger with low counts. Return the test used, test statistic if applicable, and p-value.

MediumTechnical

25 practiced

You are designing an A/B test to detect a 5% relative increase in conversion rate from a baseline of 10%, with 80% power and alpha 0.05. Describe how you would calculate the required sample size per variant, what assumptions the calculation relies on, and alternatives if those assumptions do not hold (for example, cluster randomization or rare events).

HardTechnical

28 practiced

In high-dimensional testing with correlated features, standard Benjamini-Hochberg may not control FDR as intended. Explain statistical approaches to handle dependence: Benjamini-Yekutieli correction, permutation-based FDR estimation, hierarchical testing, and empirical Bayes methods such as Storey's q-values. Discuss trade-offs in power, computational cost, and interpretability.

EasyTechnical

29 practiced

Describe when to use a chi-square test of independence on categorical variables. Walk through a concrete example with a 2x2 contingency table of device type (mobile/desktop) versus conversion (yes/no). Explain the expected count requirement and alternatives when expected counts are low.

Unlock Full Question Bank

Get access to hundreds of Hypothesis Testing and Inference interview questions and detailed answers.

Join thousands of developers preparing for their dream job.