Evaluating Reports Based on Data - Statistics

Every same-size sample has an equal chance of selection. SRS ensures unbiased representation of the population.

Selection bias; sample likely differs from the target population. Convenience samples are not randomly selected from the population.

Strong selection bias; results may not represent the population. Volunteers differ systematically from non-volunteers.

Random assignment to treatment and control groups. Randomization eliminates confounding variables by balancing groups.

Was the study a randomized experiment (not just observational). Only randomized experiments can establish causation, not observational studies.

Mean is not resistant; median is more appropriate. Outliers inflate the mean but don't affect median.

Median. Median resists the pull of high-income outliers.

Season/temperature is a confounding variable. Both increase in summer; correlation doesn't imply causation.

Self-selection confounding; groups may differ before treatment. Without randomization, treatment groups aren't comparable.

Association only; causation is not justified without random assignment. Observational studies can't control for all confounders.

Statistically significant at $\alpha=0.05$ (interval excludes $0$). All values in CI are positive, showing a real difference.

Not statistically significant at $\alpha=0.05$ (interval includes $0$). CI contains null value $0$, so fail to reject $H_0$.

Truncated axis exaggerates visual differences. Makes small differences appear larger than they are.

Relative change can exaggerate impact without the absolute difference. "$200%$ increase" from $1$ to $3$ is only $2$ units.

Significance is about chance; importance is about effect size/impact. Small effects can be significant with large samples.

Probability of data at least as extreme, assuming $H_0$ is true. Not the probability that $H_0$ is true.

Reject $H_0$; results are statistically significant at level $\alpha$. Small $p$-values indicate data unlikely under null hypothesis.

A $p$-value (or enough information to compute it). Compare $p$ to $\alpha$ to determine significance.

The method captures the true value in $95%$ of repeated samples. Not that we're $95%$ sure about this specific interval.

Voluntary response bias. Self-selected respondents don't represent the population.

Association does not imply a cause-and-effect link. Correlation doesn't prove one variable causes changes in another.

A result with $p \le \alpha$. The p-value must be less than or equal to the significance level.

Probability of data at least as extreme, assuming $H_0$. Measures how likely the observed data would occur if the null hypothesis were true.

Reject $H_0$. Since $p < \alpha$, the result is statistically significant.

Fail to reject $H_0$. Since $p > \alpha$, the result is not statistically significant.

$95%$ of such intervals capture the true parameter. The method produces intervals that contain the parameter 95% of the time.

Reject if $\theta_0$ is not in the $95%$ CI. If the null value falls outside the CI, it's unlikely given the data.

Reject $H_0$ at $\alpha=0.05$. Since $0$ is outside $[2,5]$, the null hypothesis is rejected.

Fail to reject $H_0$ at $\alpha=0.05$. Since $0$ is inside $[-1,4]$, the null hypothesis is not rejected.