To drill deep holes without the drill wandering, rotate the work piece and keep the drill stationary - if the drill starts to wander, the drilling forces push it back on to the centre of rotation of the work piece...... Regular drill tips can drift off just a bit especially on longer (depth:diameter) holes & deviation can feel like a slightly tighter fit to the length of a valve stem. A reamer will basically follow this pilot hole so you are chasing your tail. ....
The easiest way to set this up is to mount the work piece on a faceplate in the lathe and drill from the tailstock. Holding the work in a four-jaw independent chuck is an option, but the faceplate is usually quicker and easier
All the best,