AI Grounds

Open AI Grounds on a desktop

These interactive lessons need a larger screen. Please continue on a desktop or laptop computer.

Guided discovery

Backpropagation Inspector

Follow one prediction backward, then test a weight update.

Your training case

h1 and h2 are hidden activations: outputs cached from an earlier layer. We inspect one sigmoid output unit. Probability p means P(target = 1); binary cross entropy measures its prediction error.

Type exact values from 0 to 1, or use ↑ / ↓ in an activation field. Editing starts free exploration; Resume experiment restores its starting case.

Visual view

h1 0.80 × w1 +1.400

+ h2 0.35 × w2 -0.600

+ bias −0.25 → z 0.660

sigmoid(z) → p 0.659

Compare with target 1 → loss 0.417

Forward: cached activations become a probabilityh1 0.8 times weight 1 1.4, plus h2 0.35 times weight 2 -0.6, plus fixed bias -0.25 gives z 0.660. Sigmoid gives probability 0.659.h10.80h20.35z0.660p0.659w1 = +1.400w2 = -0.600sigmoidbias = −0.25Target y1Loss0.417
→ Forward computes a prediction. The target is used to measure loss, not to compute p.

Build the prediction

z = 1.40 × 0.80 + (-0.60) × 0.35 − 0.25 = 0.660
p = sigmoid(z) = 1 / (1 + exp(−z)) = 0.659
L = −ln(p) = 0.417

The sigmoid maps the weighted sum z to a probability. ln is the natural logarithm. Switch to Backward to ask how a small change would affect loss.

Matrix view

One example: h is a 2 × 1 column of cached activations; W is a 1 × 2 row of weights. W × h gives one value, z. Bias b, probability p, and target y are scalars. Each entry matches the corresponding value in the visual view above.

z = W × h + b

W · 1 × 2×h · 2 × 1+ b (-0.250) = z 0.660

p = sigmoid(z) = 0.659
L = −ln(p) = 0.417 · y = 1

The target enters the loss calculation, not W × h or the prediction.

Probability

Starting prediction for target 1.

0.659

p = sigmoid(z)

Observed target: 1

Loss

Starting binary cross entropy.

0.417

L = −y ln(p) − (1 − y) ln(1 − p)

Lower is better for this example.

Output gradient

Starting loss sensitivity to z.

—

δ = dL/dz = p − y

Reveal with Backward.

Bias and cached activations stay fixed during each previewed update. A full network normally also learns bias and earlier-layer parameters. Displayed values are rounded; calculations use full precision.

forward stage. h1 0.8, h2 0.35, target 1. Probability 0.659; loss 0.417.