Philip,
You do use a little bit of rigidity when you go over to a QCTP, but if you can make sure the rest of your machine gibs etc are set up correctly, you won't find it a problem.
The main advantage is that once the tool is set up in it's holder and spot on centre, it really is just a 'drop it on and use straight away' system.
A lot of people nowadays make their own holders, in bulk, from standard barstock, and if you have a little experience, you can make them for a few bucks each.
I personally have about 30 holders now, and a few more specials are to be made when I get the chance. You make them to fit whatever tooling you are using, make a special for holding a boring bar or DTI, if you can get them to centre, you can make them to hold larger or smaller tooling than you normally use (that is what I have to do, I am using a lot of 1/8" & 3/16" tooling in a 5/8" holder, not ideal).
As already mentioned, it is really the first major upgrade people do to their lathes. It makes life just that little easier.
John