I don't understand why you are NOT cleaning between each shot. It takes very little time, and insures that you have consistency for each shot, that is, a consistent barrel condition for each shot. In a smoothbore, consistent condition if vital to having any kind of accuracy at all! All it take me is to wet a cleaning patch with spit, lightly, run it down the barrel of my 20 gauge in stages on a jag, ( I push down 1/4 the length of the barrel, pull it back a couple of inches, then go down to half the barrel, pull back, etc. ) That patch comes out pretty black and cruddy. I then use a second patch, dry, to dry the barrel from my spit, and any moisture absorbed by the powder residue. Again, I do this is staged strokes. I don't want a dry patch STUCK in my barrel, because I loaded it with residue too quickly. If the patch looks particularly wet, or dirty, I flip it over, and run it down the barrel a second time.
I am looking for the absence of oily or damp crud from the Breechplug face. That tells me that the powder chamber is dry, and relatively clean. Now I am ready to put the new powder charge in the barrel, and finish reloading. I use an OP wad on top the powder, then a PRB. Then I run a lubed cleaning patch down the barrel to lube its entire length in front of the PRB. I wipe the priming pan, frizzen and flint off, then pick a hole in my powder charge with my vent pick, then prime, close the frizzen, and check my hammer to see that it is on half cock. MY last glance is to check the condition of the edge of the flint, and the distance it stands from the face of the frizzen when the hammer is at half cock. If all is in order, I am ready to continue my hunt.
With a 325 grain round ball, you do need a good patch to insure compression, and a high enough chamber pressure to insure clean burning of the powder, regardless of brand. I found that using the OP wad did a far better job of sealing the gases, and increasing both chamber pressure AND velocity. However, the velocity change was not as important as the substantial reduction in SDV, even in a smoothbore fowler. I don't have to worry about the condition of my bore for the 3rd or 4th shots, because It will be the same as my second shot, every time.
For safety sake, please consider running a damp cleaning patch down the barrel after ever shot, to put out any glowing embers OUT! That is the primary mission of the damp patch. The second, dry patch does the removal of residue, moisture and crud. MY experience with my smoothbore is that all the powders leave a residue in the barrel that instantly draws moisture from the air. By lubing the barrel after seating the PRB, I have found that the residue is much softer and easier to clean with the cleaning technique described. Before I began using that lubed patch down the barrel, I would often find a hard crud ring about 4-6 inches back of the muzzle. It melted if dealt with quickly with a damp patch, but it still slowed the cleaning, and was a constant concern because of its obvious affect on accuracy for subsequent rounds.