A trained neural network feels like a black box, and the word "learning" makes it feel like more of one. Open the box and the thing inside is humbler than the word suggests. To see it, you have to drop one question into the middle of the machine and follow where it leads.
The question is this: how much would a little nudge here change the result there? That is all a derivative measures, a rate of change. Speed is the rate at which your position changes. A derivative inside the network asks the same kind of thing about a single internal dial. If I turn this dial a hair, does the network's error go up or down, and by how much?
A network has thousands of those dials, the weights it tunes during training. For each one you can ask the question separately, holding the rest fixed. The answers together form a compass. Where a dial pushing the error up means turn it the other way, and a dial pushing it down means keep going. Point every dial downhill, take a small step, and the error drops a little. Then ask again. Step again. That loop, repeated, is training.
Nothing is understood along the way. The network only feels the local slope under its feet and steps toward lower ground.
There is one honest wrinkle. The downhill direction is only ever the direction right where you stand, and the lowest point in sight is not always the lowest point there is. To reach the true bottom of the valley you sometimes have to climb a little upward first, out of a dip that was never the floor.
So when we say a network learns, this is what we mean. A blind, patient walk downhill, one small step at a time, with no map and no idea where the bottom is.