Longboy, what size of set screw was in use?
Probably it was too small anyway, compared to flywheel weight ???.
Even smaller engines can give enough kick to loosen the set screw, if the axle material is too soft and/or set screw is too small, to provide enough contact surface between it and the axle.