Understanding t-Tests

A t-test compares a sample mean to a reference value or compares two sample means, using the Student t distribution to account for sampling variability when population variance is unknown.

The test statistic measures how far the observed mean is from the null hypothesis value in units of estimated standard error.

Use a one-sample t-test to check whether a single population mean equals a hypothesized value. For example, testing whether a manufacturing process produces parts with a target average diameter.

The test statistic is t = (x̄ − μ₀) / (s / √n), compared against the t distribution with n − 1 degrees of freedom.

The two-sample (independent) t-test compares the means of two separate groups. The pooled version assumes equal variances; Welch's version does not and is generally preferred as a default.

Common applications include A/B testing, treatment vs control comparisons, and comparing performance metrics across two cohorts.

A paired t-test applies when observations come in natural pairs, such as before-and-after measurements on the same subjects. It tests whether the mean difference within pairs is zero.

By working with differences, the paired test removes subject-level variability and often has greater power than an independent two-sample test on the same data.

All t-tests assume approximately normal data or a large enough sample for the central limit theorem to apply. They are sensitive to outliers, which inflate the standard error estimate and can mask or fabricate effects.

Check normality with QQ plots and consider non-parametric alternatives like the Wilcoxon test when assumptions are clearly violated or sample sizes are very small.