AI Grounds
Open AI Grounds on a desktop
These interactive lessons need a larger screen. Please continue on a desktop or laptop computer.
AI Grounds
These interactive lessons need a larger screen. Please continue on a desktop or laptop computer.
Guided discovery
Compare each activation with its batchmates—or a frozen reference.
Each row is the same feature’s activation—a numeric signal—for a different example. Batch size includes the first N values of a fixed dataset; it does not randomly sample.
Include the first 4–8 examples. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.
Use this batch’s mean and population variance.
Gamma multiplies z; this demo uses positive scale. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.
Beta adds the same offset to each output. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.
Fixed numeric axes −8 to 8 keep changes in spread visible; rows link the same example. x and y are activation values; z is dimensionless. Outlined dots mark the selected example. Exact values are available in the table below.
z = (2.000000 − (3.666667)) / √(1.205556 + 0.001) = -1.517313
y = 1.00 × (-1.517313) + (0.00) = -1.517313
Displayed substitutions are rounded; calculations use full precision. Epsilon (0.001) is a small positive term protecting the denominator.
Mean of the current batch.
3.6667
mean = Σx / N
Average squared distance from the batch mean.
1.2056
variance = Σ(x−mean)² / N
Denominator √(variance+0.001) = 1.098433
Population standard deviation around the current output mean.
0.9996
std(y) = |Scale| × std(z)
mean(y) = Scale × mean(z) + Shift = 0.000000
| Example | Input x | Normalized z | Output y |
|---|---|---|---|
| 1 · selected | 2.000000 | -1.517313 | -1.517313 |
| 2 | 2.700000 | -0.880041 | -0.880041 |
| 3 | 3.400000 | -0.242770 | -0.242770 |
| 4 | 4.000000 | 0.303463 | 0.303463 |
| 5 | 4.800000 | 1.031773 | 1.031773 |
| 6 | 5.100000 | 1.304889 | 1.304889 |
Current batch: mean 3.666667, population variance 1.205556, std 1.097978.
Frozen preset: mean 4.150000, variance 2.624400, std 1.620000. These illustrative estimates do not update.
Normalized mean 0.000000, std 0.999586. Standard deviation is measured around this actual mean, including in Inference.
One scalar feature across examples, using population variance (divide by N) for the training forward calculation. Epsilon makes Training std(z)=√(variance/(variance+epsilon)); a constant batch would give z=0 and y=Shift. Scale and Shift correspond to gamma and beta: you set them manually here. Real learnable scales can also be zero or negative.
The Inference path represents BatchNorm with tracked running statistics. Real PyTorch tracks running mean and variance during training; its running variance update uses an unbiased batch estimate (divide by N−1). This lesson keeps explicit illustrative references frozen and does not simulate running updates, optimizer steps or training quality. BatchNorm configured without tracked statistics can use batch statistics in evaluation too.
This demonstrates normalization mechanics and parameter effects; it does not establish faster optimization or higher accuracy. PyTorch BatchNorm1d reference.
Shifted, training, 6 examples. Used mean 3.666667, variance 1.205556. Example 1: x 2.000000, z -1.517313, y -1.517313.