Low and to the right would be a pretty good indicator of a flinch for a right handed shooter, Lenny. And, yes, when you have a flinch, you can throw the shots fairly consistently, albeit into a wider group. Shoot the gun from a rest, first at 25 yds, and then at 50 yds, with the gun braced both under the forestock, and at the buttstock, so that you only have to support the butt with your shoulder, and then pull the trigger. If you are still shooting low and left, you have a problem with your sights. Ask a buddy to shoot 3-5 shots to see where the gun shoots for him. If it is also shooting low and left, adjust your sights.
Then, back that load down to one used to shoot targets. 50 shots in an afternoon is a lot of shooting, Lenny, with a muzzle loader. This is not a .22 rimfire where shooting a brick of ammo in an afternoon is considered a good workout!