AI Grounds
Open AI Grounds on a desktop
These interactive lessons need a larger screen. Please continue on a desktop or laptop computer.
AI Grounds
These interactive lessons need a larger screen. Please continue on a desktop or laptop computer.
Guided discovery
Change the weights. Compare the directions.
Choosing a reference changes P while keeping Q and direction fixed. KL adds source probability × ln(source / target); ln is the natural logarithm and the units are nats.
This bucket keeps the chosen whole percent. Others redistribute with at least 1% each; Q stays 100%. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.
This bucket keeps the chosen whole percent. Others redistribute with at least 1% each; Q stays 100%. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.
This bucket keeps the chosen whole percent. Others redistribute with at least 1% each; Q stays 100%. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.
This bucket keeps the chosen whole percent. Others redistribute with at least 1% each; Q stays 100%. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.
P and Q share a fixed 0–100% scale. Current contribution: P × ln(P / Q). The direction changes the formula, keeping the probabilities fixed.
Contribution = 0.7 × ln(0.7 / 0.45) ≈ +0.3093 nats.
Contribution = 0.15 × ln(0.15 / 0.25) ≈ -0.0766 nats.
Contribution = 0.1 × ln(0.1 / 0.2) ≈ -0.0693 nats.
Contribution = 0.05 × ln(0.05 / 0.1) ≈ -0.0347 nats.
Current direction: P supplies the weights and numerator. Units: nats.
0.1287
Σ P × ln(P / Q)
Signed bucket terms add to this nonnegative total.
Opposite direction: Q supplies the weights and numerator. Units: nats.
0.1368
Σ Q × ln(Q / P)
The same two distributions can have different KL totals.
Values use actual probabilities without clipping. Displayed terms are rounded; the total uses full precision.
| Bucket | Source P | Target Q | Ratio | Term (nats) |
|---|---|---|---|---|
| A | 0.7 | 0.45 | 1.5556 | +0.3093 |
| B | 0.15 | 0.25 | 0.6000 | -0.0766 |
| C | 0.1 | 0.2 | 0.5000 | -0.0693 |
| D | 0.05 | 0.1 | 0.5000 | -0.0347 |
| Sum | 0.1287 | |||
The controls keep each bucket positive. Mathematically, a zero source contributes zero, including when both values are zero. Positive source mass against a zero target gives infinite KL.
Fixed −0.5 to 4.5 nats scale covers the available controls. Negative terms extend left of zero; positive terms extend right. This chart scale is not a universal maximum for KL.
Individual terms can be negative, while normalized KL is nonnegative. It is zero when the distributions match, generally directional, and has no universal maximum.
Source P, target Q. A: P 70%, Q 45%, contribution +0.3093 nats; B: P 15%, Q 25%, contribution -0.0766 nats; C: P 10%, Q 20%, contribution -0.0693 nats; D: P 5%, Q 10%, contribution -0.0347 nats. KL 0.1287 nats, opposite 0.1368 nats.