What is the Spent Patch for that first shot telling you?? READ THAT PATCH, and then compare it to the next 4! That should tell you a lot about what is going on in the barrel.
I Suspect that you are NOT USING the CORRECT Patch/lube/ball diameter combination in the gun. Measure both the Land-to-Land( bore) diameter, and then measure the groove depth(groove diameter). You need a patch thick enough that when compressed, will still fill those grooves to seal back gases. A thicker patch will also hold MORE LUBE, and that also helps seal the gases behind your PRB, and keeps the patch from tearing and burning.
For hunting, ZERO the gun for that first shot- not the next 4! In most cases, you will only get that first shot.
You SHOULD BE CONCERNED about what allowing residue to remain for long periods of time in the bore of the gun. That is how RUST occurs. BTDT. Use an vegetable based, or mineral oil based lube- not something with water in it, and not petroleum based oils.
You can get away with the water based patch lubes on shooting ranges, when you will be shooting the gun fairly quickly after the gun is loaded. That doesn't work too well in the field, particularly when the temperature is below freezing, and will the dry air will suck the moisture right out of your patch! :shocked2: :idunno: :surrender: :hmm: In cold weather, a bit of extra lube in your patch, that will grease the entire length of your bore, both lands and grooves, goes a long way to protecting the bore from rust during the long hours of a hunt.
In using a chronograph to measure velocities, I found that MV increased consistently if I ran a greased cleaning patch down my barrel AFTER seating my PRB, in both my rifle, and smoothbores. That grease in the barrel seems to prevent powder residue from Sticking to the sides of the bore, making it much easier to clean, or load the next round. :hmm: