This is a very good idea and many people in the past have pulled this off. As far as the fees of VIT Vellore is considered, you only have to pay the fees up to till when you studied there. Once you leave VIT for IIT or some other of your choice, you'll not have to pay anything to VIT after that, be it tuition fees or hostel or mess fees.

So, answering your question, yes, it is possible to leave VIT without paying the fees for rest of the semesters.

Hope I have answered your question. All the best!