Drill the corners 1st staying perhaps .010" away from your finish dim and staying off the bottom. use a flat bottom drill (easily ground) to finish the bottom.
Rough the pocket, Then plunge corners to finish with the end mill. then finish the pocket with the end mill.