I sight in with a clean barrel, but find that with my three rifles and one smoothbore, none of them shoot very far off when fouled...maybe an inch or so at most. Generally just a tad lower with the first shot, then with following shots. I don't get much or any left-right/windage variation between clean and dirty.
Generally when hunting you "should" only need one shot, but the fun thing about hunting is that anything can, and sometimes does happen. However, I think it's fairly safe to say that if you knock something down, and have to shoot it again, you'll be pretty close for the follow up shot, and one, two or even three inches won't make any meaningful difference...POI between a fouled and clean barrel really won't cause a miss. Remember that the kill zone on a deer is a good 12" circle. Makes sense to me to be sighted in with a clean barrel, because your first shot is more likely to be the long one.
I also believe that a clean rifle is more reliable, that's more of an issue to me, than fear of the bore rusting...but a rusting bore is another chance I'd rather not take. But those are two good reasons, for me, not to hunt with a fouled bore and breech.
Getting a rifle sighted in for a clean barrel can be time consuming, but I don't mind. Just plan the whole day at the range, take your time, and enjoy shooting. Also cleaning just the bore alone, with just one shot's worth of fouling, is pretty fast and easy. Don't have to cook up five gallons of hot water, (how did the old timers do THAT on the trail?) take the rifle apart, dance around it three times counter-clockwise and all that.
And of course sighting in the rifle with a clean barrel is the very last thing I do with said rifle, long after the load is perfected, desired accuracy level is achieved, and I'm sure I'll stick with the powder charge chosen...after all the experimentation is done.
RAT