The standard, I believe, for using very small drills was an Albrecht (uncertain if this is the correct spelling) chuck.
Edit: Delta ( and probably others) offered a high-speed bench-top drill press many years ago (thinking the 60's or 70's). Back then, this was part of their industrial tool catalog - not the home stuff. Expensive then but second hand today... who knows.
There is an article in Pop Mech, Dec 1941 with plans to build a small modelmaker's drill press powered by a sewing machine or vacuum power brush head motor.
Elektor magazine had a construction article - early 2001, I think - but I cant locate any more info.