Two answers- maybe more???
You could be hitting inclusions- of carbides from scrap tooling. Well you will have people exposing the virtues of such things.#
I supposed that the only solutions are two harder things like CBN and diamond!
The other solution of actual chilling is to take it up to red heat or hotter and soak it over night.
The classic thing was a Beehive oven. Workington in Cumberland was full of them.
Does it help or should you start again?
Norman