The robot gets no map and nobody tells it which move is right; every step hands it a single number, a reward or a penalty. From that alone a Q-table records the value of each cell-and-direction pair, updated by δ = r + γ·max Q(s′,·) − Q(s,a) with a discount factor around 0.9, and value seeps backwards from the goal cell across the whole grid. Keeping ε high leaves the robot wandering everywhere, slow to learn but rarely trapped in a poor route; a low ε glues it to the first path it stumbles onto. On a slippery grid where roughly one step in five slides sideways, the best policy is to stay clear of the edges of the pits.
The robot gets no map and nobody tells it which move is right; every step hands it a single number, a reward or a penalty. From that alone a Q-table records the value of each cell-and-direction pair, updated by δ = r + γ·max Q(s′,·) − Q(s,a) with a discount factor around 0.9, and value seeps backwards from the goal cell across the whole grid. Keeping ε high leaves the robot wandering everywhere, slow to learn but rarely trapped in a poor route; a low ε glues it to the first path it stumbles onto. On a slippery grid where roughly one step in five slides sideways, the best policy is to stay clear of the edges of the pits.
Related topics in Mind & Machine