Not to insult you, but I think you need someone else to shoot the rifle til you find a load. Obviously, what you are doing is not working.
First thing is eliminate shooter error.
Shoot off a Solid bench with a Sand Bag under the entrypipe and a Sand Bag under the Butt. Sandbags do all the supporting, you just line it up. The larger the bench& more stable it is, the better the groups will generally be, IMHO.
Start at 45 grains of 3F, .440 ball, .015 PT and patch grease.
1" Black dot bull @ 25 yards.
shoot 3 shots.
Change targets. Go to 50 grains. 3 shots.
Change targets. Go to 55 grains. 3 shots.
Change targets. Go to 60 grains. 3 shots.
Change targets. Go to 65 grains. 3 shots.
One of those targets should be grouping close & I mean 3 cutting or real close to cutting.
Shoot that group again & insure it is doing that.
If they are not cutting, change lube or ball size. & do it all over, keep changing til you find the best group.
Sometimes this happens in a few hours, sometimes in a few days, sometimes takes a few weeks.
Documentation is Most Important, so you don't repeat yourself.
LOTS of things have to be Consistent.
The amount of lube you use Each Shot.
Powder charge on Each Shot.
Patches all the same thickness & Same Brand or Same Batch of PT. You cannot guess, you have to Mic the material. DO NOT go by what the supplier said it is... Mic it.
Balls from the same supplier & check to insure they are the same dia.
Swabbing........ Must be the same Every Time.... You are not cleaning the barrel. you are making the barrel Consistent from shot to shot.
If you wet the patch, it must be the same wetness Every Time.
If you swabbed down & back One Time, then on the next shot went down & back 2 times.... you just screwed up, as you Changed The Barrel Condition..... That will change the grouping....
Flinching....... If You KNOW when the rifle is going to go off...... you will not shoot good groups..... If you Know it is going off, you are anticipating the ignition & will flinch or pull....
Every time the rifles fires.. you should be surprised it went off, thus you did not anticipate the ignition.....
Just my opinion on it all....... and just an opinion....
Keith Lisle