The motor should work fine even at very slow speeds. Is your belt a little too tight? There should be some flex in the belt.
Servo motors are constant torque devices, changing to a smaller pulley has no effect on low end torque.
No one lives forever, the trick is creating something that will.