Statistical Inference and Hypothesis Testing Questions

Reasoning about uncertainty in data and drawing formal conclusions from samples. Covers probability rules and common distributions, the Central Limit Theorem, sampling, standard error, confidence intervals, and Bayesian reasoning, together with the significance-testing framework: null and alternative hypotheses, p-values, statistical power, Type I and Type II errors, effect sizes, and choosing the right test (t-test, chi-square, non-parametric). Emphasizes correctly interpreting statistical results and avoiding common misreadings of significance in business and product contexts rather than memorizing formulas.

MediumTechnical

26 practiced

Given the contingency table of product preference by region: North A=30 B=70; South A=20 B=30; East A=50 B=50; West A=10 B=20, outline the steps to perform a chi-square test of independence. Show how to compute expected counts, the chi-square statistic, degrees of freedom, and p-value. When is Fisher's exact test more appropriate? Mention an effect size measure you would report.

MediumTechnical

33 practiced

You observe purchase amounts with heavy right skew and outliers across two groups in an A/B test. Describe how you would analyze the data: compare t-test on raw data, log-transform + t-test, trimmed means, or a nonparametric test (Mann–Whitney). Explain trade-offs in interpretability, robustness, and estimation of effect size and confidence intervals.

Sample Answer

Framework / first steps:- Explore distribution (histogram, boxplot, quantiles), check group sample sizes, missingness, and whether extreme purchases are genuine or errors.- Decide the estimand: average treatment effect on mean spend, percent change, or median/typical customer? That choice drives method selection.

Method comparisons and trade-offs:1) t-test on raw data- Interpretability: directly tests difference in means (dollars), easy to explain.- Robustness: very sensitive to skew and outliers; long tails inflate variance and can make p-values unreliable.- CIs: mean CI may be wide and influenced by a few extreme purchases.

2) Log-transform + t-test- Interpretability: tests difference in log-mean → approximately tests multiplicative change; exponentiate to report geometric mean ratio (e.g., % change).- Robustness: reduces skew and effect of outliers; improves normality for t-test assumptions.- CIs: can back-transform CIs to multiplicative scale, but interpretation becomes about median/typical multiplicative change, not arithmetic mean.

3) Trimmed means (e.g., 10–20%) + t-test for trimmed means- Interpretability: estimates mean excluding extremes — represents "typical spend among non-extreme customers".- Robustness: much more robust to outliers than raw mean; good trade-off between mean and median.- CIs: can use robust standard errors or bootstrap; gives interpretable dollar difference for trimmed population.

4) Nonparametric test (Mann–Whitney)- Interpretability: tests stochastic dominance (probability one random from A > B), not difference in means; often misinterpreted.- Robustness: robust to outliers and skew.- CIs: doesn’t directly give mean difference CI; use rank-based effect sizes or bootstrap to estimate median or other quantiles.

Recommended practical approach:- Report multiple complementary metrics aligned to business questions: arithmetic mean (for revenue), median or 10% trimmed mean (for typical customer), and geometric mean ratio from log-transform (for multiplicative effects).- Use bootstrap to estimate CIs and p-values for means, trimmed means, and medians—bootstrapping is robust to skew and gives direct CIs on original scale.- If you present hypothesis tests, be explicit about null hypotheses and what the test measures (mean vs median vs shift in distribution).- For small samples or heavy tails, prefer trimmed means + bootstrap or log-transform + bootstrap depending on whether multiplicative interpretation is meaningful.

Example communication to stakeholders:- "Mean spend increased by $X (95% CI $A–$B) — sensitive to top 1% spenders. Median spend changed by $Y (95% CI $C–$D), indicating the typical customer saw ... . Geometric mean ratio = 1.12 (95% CI 1.03–1.21), suggesting ~12% multiplicative uplift after accounting for skew."

Key takeaways:- Choose metric to match business goal (revenue vs typical user).- Use transforms or robust estimators to control outlier influence.- Use bootstrap for CIs and effect-size estimation on the original scale for clearer stakeholder interpretation.

EasyTechnical

31 practiced

You run an A/B test and observe a statistically significant conversion lift with p=0.01 but the absolute lift is only 0.5 percentage points. As a data analyst, how would you explain the difference between statistical significance and practical (business) significance, and what recommendations would you give the product team?

MediumTechnical

42 practiced

You have two highly correlated features in a linear regression model (e.g., marketing_spend_tv and marketing_spend_radio). How would you detect multicollinearity, and what remedies would you apply? Discuss pros/cons of dropping variables, combining them, using PCA, and using regularization.

Sample Answer

Detecting multicollinearity- Start with domain check: high correlation between marketing_spend_tv and marketing_spend_radio suggests collinearity.- Quantitative checks: correlation matrix (Pearson r close to ±1), Variance Inflation Factor (VIF > 5 or >10 is problematic), condition number (>30 indicates instability).- Inspect coefficient signs and large SEs: unstable or counterintuitive coefficients and wide CIs are red flags.

Remedies and trade-offs1) Drop a variable- Pros: simple, restores stability and interpretability of remaining coeffs.- Cons: loses information about the dropped channel; biased if the dropped variable has unique causal effect.- Use when one variable is redundant for the business question or data collection shows it’s noisy.

2) Combine variables (sum or index)- Pros: preserves total marketing exposure, easy to explain to stakeholders, reduces collinearity.- Cons: loses ability to estimate channel-specific ROI; only valid if channels are substitutable or you only care about aggregate spend.

3) PCA (feature extraction)- Pros: creates orthogonal components; solves multicollinearity while keeping most variance.- Cons: principal components are linear combinations—less interpretable for business decisions; need to explain PC meaning to stakeholders.

4) Regularization (Ridge, Lasso)- Ridge: shrinks correlated coefficients toward each other, reduces variance without dropping features — good if prediction is goal.- Lasso: can zero out one of correlated features (feature selection), but arbitrary if they’re equally predictive.- Pros: maintain all variables, often best for predictive accuracy.- Cons: coefficients are biased; tuning required (cross-validation); interpretability suffers compared to unpenalized estimates.

Recommended approach for a Data Analyst- First decide objective: inference (understand channel effects) vs prediction.- If inference: prefer combining variables or carefully drop one based on business logic; report sensitivity (models with each option) and show VIF/SE changes.- If prediction: use Ridge or Elastic Net with CV and report feature importance via standardized coefficients or permutation importance.- Always communicate trade-offs to stakeholders, show model diagnostics, and, when possible, validate with A/B tests or uplift analyses to attribute channel effects reliably.

HardTechnical

34 practiced

Explain the conceptual difference between a 95% confidence interval and a 95% Bayesian credible interval. Provide a short, precise example where their numerical values could differ and explain why interpretations differ for decision-makers.

Unlock Full Question Bank

Get access to hundreds of Statistical Inference and Hypothesis Testing interview questions and detailed answers.

Join thousands of developers preparing for their dream job.