Hii aspirant..

If you are having three years of gap in my academics it is not going to effect your chance of getting admissions at IIM or XLRI.

But instead the work experience you were having while working for these three years is going to act as a boon for you during the selection process in these colleges. 

During the selection process of IIMs and XLRI not only your CAT score will be playing the important part but instead your previous academics, your work experience, your interpersonal skill is going to play a very important part. 

Thus, instead of worrying you should know that the gap between your academics and your work experience during it is going to work positively for you. 

Hope it helps. 

Best of luck for your future.