Well to each his own. Maybe it's the equipment but in the 50+ years I've been drilling holes, I've NEVER had a problem with straightness and I've never started a drilled hole by boring. However there may be a variation in methodologies.
You are correct, if the starter is off, the drill will...