Fill the whole hole if you are going to re-drill it as Mike says. And make sure that when that drill first starts meetin wood, it hits a perfectly square surface. You'll need to re-align the ramrod groove by making it deeper at the muzzle and carefully make sure that the drill is not springing in any way when it is aligned right. I've heard tell that a fella can pull the muzzle end of the blank to direct the drill but prefer to change the ramrod groove.