Apologies if my answer was misleading. I was assuming a full set of collets in 1/32nds.
With my own set of 1/32nd ER32s, I've never had a problem finding a suitable size. My set specifies .040" compression, which is 1/25", so there should be some overlap between all adjacent 1/32" sizes.