Changing PhD Programs/Universities and the gap between

If one plans to quit one's current PhD program and apply to other PhD programs in other universities, is it recommended that one tries to get to the new program as fast as possible? Waiting could make it possible to apply to several places and choose which suits one best.

Would working outside of academia a period of time before applying or before the new PhD program starts hurt one's chances of getting accepted (or damage one's career later in some other way)? I imagine getting recommendation letters might be an issue if the period is long.

I remember reading that if one should try to find out what work outside academia is like the best point in time to do so is before graduate studies. Myself I have studied my whole life and for many years my only career goal was in academic research, but thinking about the job prospects and sacrifices associated to academia I have become mildly curious about life outside academia. I think some time outside could bring some perspective and a point of comparison. If it matters my own area is pure math.