Data Science & Analytics Topics
Statistical analysis, data analytics, big data technologies, and data visualization. Covers statistical methods, exploratory analysis, and data storytelling.
Statistical Foundations for Experimentation
Core statistical concepts and inference needed to design analyze and interpret experiments. Topics include hypothesis testing p values confidence intervals Type One and Type Two errors the relationship between sample size variability and interval width statistical power minimum detectable effect and effect size versus practical significance. Candidates should be able to choose and explain common statistical tests such as t tests and chi square tests contrast Bayesian and frequentist approaches at a conceptual level and describe variance estimation and variance reduction techniques. The topic covers corrections for multiple comparisons sequential testing and the risks of peeking and p hacking common misconceptions about p values and limitations of inference such as confounding and selection bias. Candidates should also be able to translate statistical findings into clear language for non technical stakeholders and explain uncertainty and limitations.
Metrics, Guardrails, and Evaluation Criteria
Design appropriate success metrics for experiments. Understand primary metrics, secondary metrics, and guardrail metrics. Know how to choose metrics that align with business goals while avoiding unintended consequences.
Data Storytelling and Insight Communication
Skills for converting quantitative and qualitative analysis into a clear, persuasive narrative that guides stakeholders from findings to action. This includes leading with the headline insight, defining the business question, selecting the most relevant metrics and visual evidence, and structuring a concise story that explains what happened, why it happened, and what the recommended next steps are. Candidates should demonstrate tailoring of language and technical depth for diverse audiences from engineers to product managers to executives, summarizing trade offs and uncertainty in plain language, distinguishing correlation from causation, proposing follow up experiments or investigations, and producing concise executive summaries and status reports with an appropriate cadence. Interviewers evaluate the ability to persuade and align cross functional partners, answer questions about data validity and methodology, synthesize qualitative signals with quantitative results, and adapt presentation format and level of detail to the decision maker.
Probability and Statistical Inference
Covers fundamental probability theory and statistical inference from first principles to practical applications. Core probability concepts include sample spaces and events, independence, conditional probability, Bayes theorem, expected value, variance, and standard deviation. Reviews common probability distributions such as normal, binomial, Poisson, uniform, and exponential, their parameters, typical use cases, computation of probabilities, and approximation methods. Explains sampling distributions and the Central Limit Theorem and their implications for estimation and confidence intervals. Presents descriptive statistics and data summary measures including mean, median, variance, and standard deviation. Details the hypothesis testing workflow including null and alternative hypotheses, p values, statistical significance, type one and type two errors, power, effect size, and interpretation of results. Reviews commonly used tests and methods and guidance for selection and assumptions checking, including z tests, t tests, chi square tests, analysis of variance, and basic nonparametric alternatives. Emphasizes practical issues such as correlation versus causation, impact of sample size and data quality, assumptions validation, reasoning about rare events and tail risks, and communicating uncertainty. At more advanced levels expect experimental design and interpretation at scale including A B tests, sample size and power calculations, multiple testing and false discovery rate adjustment, and design choices for robust inference in real world systems.
Data Analysis and Insight Generation
Ability to convert raw data into clear, evidence based business insights and prioritized recommendations. Candidates should demonstrate end to end analytical thinking including data cleaning and validation, exploratory analysis, summary statistics, distributions, aggregations, pivot tables, time series and trend analysis, segmentation and cohort analysis, anomaly detection, and interpretation of relationships between metrics. This topic covers hypothesis generation and validation, basic statistical testing, controlled experiments and split testing, sensitivity and robustness checks, and sense checking results against domain knowledge. It emphasizes connecting metrics to business outcomes, defining success criteria and measurement plans, synthesizing quantitative and qualitative evidence, and prioritizing recommendations based on impact feasibility risk and dependencies. Practical communication skills are assessed including charting dashboards crafting concise narratives and tailoring findings to non technical and technical stakeholders, along with documenting next steps experiments and how outcomes will be measured.
Experimentation Metrics and Strategy
Designing experiments and selecting appropriate primary, secondary, and guardrail metrics to evaluate hypotheses while protecting long term user value. This includes choosing metrics that reflect both short term signal and long term outcomes, reasoning about metric interactions and potential unintended consequences, and applying statistical considerations such as minimum detectable effect, sample size and power analysis, test duration, and external validity across segments and platforms. Candidates should also discuss experiment risk mitigation, stopping rules, and how to operationalize experiment results into product decisions.
Problem Definition and Hypothesis Formation
Break down ambiguous business questions into specific, answerable analytics problems and define what success looks like. Ask clarifying questions about business context, constraints, stakeholder expectations, and acceptance criteria. Use structured diagnosis and root cause analysis to isolate where a problem occurs by segmenting users, products, time periods, or geographies. Generate multiple testable hypotheses that explain observed outcomes, distinguish correlation from causation, and prioritize hypotheses by likelihood, potential impact, and ease of validation. Frame measurable metrics for each hypothesis and propose high level validation approaches or experiments to confirm or reject the hypotheses.
Hypothesis Testing and Inference
Fundamental framework and application of hypothesis testing and statistical inference. Topics include formulating null and alternative hypotheses, understanding Type I and Type II errors, interpreting p values and confidence intervals, selecting and applying common tests such as t tests, chi square tests, analysis of variance, and non parametric alternatives, checking test assumptions, and discussing statistical versus practical significance. Candidates should explain power, significance levels, effect sizes, and common pitfalls such as misinterpreting p values or violating independence assumptions. At more advanced levels, discuss limitations of null hypothesis significance testing, alternatives such as Bayesian inference, and guidance for when different approaches are appropriate.
Experiment Design Analysis and Causal Methods
Design and analysis of experiments and causal inference methods for when randomization is not possible. Candidates should know strategies to ensure randomization and evaluate experiment quality compute sample size and minimum detectable effect select and interpret primary and guardrail metrics and design appropriate test duration. Analysis skills include hypothesis testing p values confidence intervals effect size estimation variance estimation and variance reduction segmentation and interaction analysis and robust reporting of uncertainty. This topic covers observational and quasi experimental approaches such as propensity score matching difference in differences and regression discontinuity how to reason about confounding and selection bias and when to prefer a quasi experimental approach over a randomized test. Candidates should be able to translate causal conclusions into actionable guidance recommend follow up analyses and triangulate evidence across methods.