Experimental Design and Analysis Pitfalls Questions

Covers the principles of designing credible experiments and the common errors that invalidate results. Topics include defining clear hypotheses and control and treatment groups, randomization strategies, blocking and stratification, sample size and power calculations, valid run length and avoiding early stopping, and handling unequal or missing samples. It also covers analysis level pitfalls such as multiple comparisons and appropriate corrections, selection bias and nonrandom assignment, data quality issues, seasonal and temporal confounds, network effects and interference, and paradoxes such as Simpson paradox. Candidates should be able to critique flawed experiment designs, identify specific failure modes, quantify their impact, and propose concrete mitigations such as pre registration, A and B testing best practices, adjustment methods, intention to treat analysis, A over A checks, cluster randomization, and robustness checks.