If you aren't certain you are going to nail the animal where you want, you are absolutely correct in letting them pass.
If you want to use a rifle, then use a rifle. You will never hear me try and tell others what they have to use. It's a trade off. A rifle, in particular a muzzleloading rifle is accurate to a decent range, a good shot should be able to be effective to about 125 yards. The drawback is that now you have a weapon that makes it mighty tough to hit a moving animal. A muzzleloading shotgun with buckshot is only effective to 35-40 or so yards, a little more with tight chokes. Despite the range limitation, you can make the close in shots that a rifle cant.
You clearly are content to allow running deer to pass, in exchange for a rifle with it's extended range abilities. Or maybe you shoot a smooth bore that is accurate with a ball to 50-60 yards. It's only marginally farther than buckshot is effective, but more is more.
I'm not happy that I'm limited to what I can use. The farthest that I have ever taken a shot on a whitetail is 60 yards, and there is a good chance he would have kept coming in closer if I let him. I would gladly trade extended range, so that I can make these close in shots. 75% of the deer I am forced to pass on due to having a rifle, are absolutely killable by a shotgun. Still, I get it done every year, and as of yet, I've not had to miss a monster buck because of the limitation. Sometimes it sure would be nice to just use the best tool for the job, rather than use a hammer for everything.