Units sometimes dont retreat correctly

I had it multiple times now that my units don't really retreat.
I press the retreat and they don't retreat and flee for a short distance but they can be controlled immediately. However while I can move them they can still be hit by the melee units they were in a fight with even though it shows as if they have some distance between them already. Pretty annoying if you want to retreat units that are only 2-3 man strong and you loose them without the option to do anything against it...