One parameter θ, one deterministic loss L(θ) = θ²/2, minimum at zero. Its gradient g = θ is the local uphill slope; −g points locally downhill. A finite update can still raise loss.

Learning rate η

Scales the current negative gradient. Editing keeps position and previous update, and pauses Run. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.

Momentum β

Carries a fraction of the previous signed parameter update. Editing keeps that stored update. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.

Presets restart at θ = 2.4 with velocity 0 and load both parameters. Restart path preserves the parameters. Run stops at step 60 or when both |θ| and |previous update| are below 0.002; this is a convenience threshold, not exact convergence. Step remains available.

Loss curve and actual update

Step 0 · θ = 2.4 · Previous update v = 0 · Gradient g = +2.4

● Current point · ◇ Next point · Dashed line: recorded steps. Axis: θ from -3.2 to 3.2; loss from 0 to 5.53. Axes expand when needed to include actual positions.

vNext = −η × g + β × vPrevious

= −0.08 × (+2.4) + 0 × (0)

= (-0.192) + (0) = -0.192

θNext = 2.4 + (-0.192) = 2.208

Current loss

Loss at current θ = 2.4.

2.88

L(θ) = θ²/2

Distance from minimum: 2.4.

Next loss

Proposed loss if you press Step with the current parameters.

2.437632

L(2.208) = θNext²/2

Lower next loss. The next step does not cross zero.

Step history

Each row records the position, previous signed update and loss. Display rounds numbers; the recurrence uses full precision.

Recorded position, previous signed update and loss. The start has no prior update.
Step	θ	Previous update	Loss
0	2.4	0	2.88

Preset comparisons

All three original presets start at θ = 2.4, velocity 0 and run 12 updates. Presets change both η and β, so this does not isolate either parameter.

Preset	η	β	θ after 12	Loss after 12
Creep	0.08	0	0.88239933	0.38931429
Converge	0.42	0.18	4.1e-7	8.489e-14
Overshoot	1.55	0.36	-0.00551495	0.00001521

On this particular quadratic, all three fixed presets converge in the long run; Overshoot crosses the minimum without diverging. These results do not guarantee behavior for other losses or changing parameters.

Velocity here stores the previous signed parameter update. Momentum can reinforce or oppose the current negative gradient. The full update, not gradient direction alone, determines the next position and loss.

Step 0; θ 2.4, previous update 0, gradient +2.4. Next update -0.192, next θ 2.208. Current loss 2.88, next loss 2.437632.

Open AI Grounds on a desktop

Gradient Descent Playground

Loss curve and actual update

Current loss

Next loss