Allot of things come into mind on your 5" groups.... We need more information from you.
Rifle type & barrel twist, ball dia., charge, powder brand, patch, lube, etc.
First thing that hits me is the hundred yard thing. I have a strict rule for myself & that is I don't shoot at anything past my limitations. My limitation is a 3" group max. off a bench, thus normally 75 yards is my limit off hand. If I had a really good shot & something for a rest, then I may attempt a 100 yard shot but it would have to be the optimum situation & shot. I have shot lots of under 2" groups off a bench at 100 yards, but anymore these are rare as my eyesight continues to get worse with age.. But I have some of the young bucks at the hunting camp shoot them sometimes & the rifles are capable of it, if the marksman is.
If I could not get the gun to perform around a 3" group I think am not doing my part, as I think just about all of the rifles will do that or better than that, if the person wants to work hard enough to get there.
Also, one of the first things I do when I shoot a 100 yard target is I shoot a 3" Black dot & I hold the flat of my rear sight at 6 o/c. This give me a full view of the target & it stays the same, not trying to split it or whatever, I have a full dot there to shoot at. I am sshooting for a group, not a dead center bull hit. I can adjust the sights anytime, first I must find a group. If it is 3" off right or left, makes no dif, hold the Same Sight Picture & shoot for the group. Acctually best if it don't even hit the dot, as that way it doesn't change the sight picture.
Also you mention the commotion going on in front of you..... Forget all that mess & concentrate on what the objective is. Forget the type of ignition, BP, style of gun....all of that. Concentrate on the sights.... Make yourself "Keep Your Head Down" till long after you have shot, then slowly raise your head.
Lots of times I see shooters pop that head up immediately after shooting. Bout 10 shots later & now they are popping their heads up as they are shooting & also pulling the shot off. Keep it down & follow thru...... This is Most important if you are shooting a flintlock and even all the more important if the ignition is not fast.