Gradient descent · Mathematics

GRADIENT DESCENT SIMULATION

▸ Building the loss-function landscape f(x, y)…

▸ Painting the contour map (heatmap)

▸ Computing the gradient ∇f = (∂f/∂x, ∂f/∂y)

▸ Dropping the ball downhill along −∇f

▸ Marking the local & global minima

▸ Ready — Online. ✅

Iteration state

⛰️ Descending

−∇f = steepest descent direction

Iteration—

Position (x, y)—

Loss f—

|∇f| (steepness)—

Learning rate η—

Status—

Tip

The ball always moves against the gradient (−∇f): the gradient points UPHILL, so subtracting it goes DOWN. Each step: x ← x − η∇f. Too large an η → blow-up/oscillation; too small → slow crawl. With many basins → the ball can get stuck in a local minimum.

—

Click the map to drop the ball there · drag Learning rate η to see slow crawl ↔ blow-up · switch the Scenario (single basin · many basins · big step · small step · narrow valley · momentum)

Your browser has canvas disabled.

Loss f & steepness |∇f| by iteration loss f|∇f| steepness