Bob : Try cleaning the breech area of the barrel with alcohol before you load any powder. Then, do work on sealing the bore properly with tight fitting patches. Its my experience that the red residue is an indicator of incomplete burning, and that should just not be happening with your gun. You are using FFFg powder to raise the pressure in the chamber to burn up the powder, when you should be using FFg powder and a good tight wad to raise the pressure. If your gun is not fitted with one of Jim Champer's parabolic vent liners, do that. The parabolic curve focuses both heat and pressure back into the chamber as gases are trying to escape out the vent hole, which raises both the temperature and chamber pressure for the brief instant when you need it to fully consume the powder. Residue should be most carbon, as that is the largest element in black powder. But in your 20 ga., all you should have to do is run a bore bursh down the barrel to loosen the carbon, and then upend the barrel to knock and pour the carbon chunks out of the barrel. On a humid day, you then may want to follow up with a slightly dampened patch, ( I use spit unless its below freezing) , followed up with a dry patch on a cleaning jag to dry the barrel. Give the jag a clockwise turn when it butts against the breechplug. to help wipe the face of the plug off of any residue.
I have used my vent pick to scrape crud off the face of the breechplug. A better made gun will leave about 1/8" gap between the vent hole, and the face of the breech, so that there is some room for crud to build up where it will not interfere with your vent. However, cleaning between shots is the best way to keep that vent cleared. The next best thing to do is always use a vent pick to poke a hole in the powder charge in the barrel to make it possible for the heat and flame of your priming charge to reach more than one granule of powder in the barrel on ignition.
If you will use a chronograph with that FFFg powder charge, you will find spikes in velocities, from one shot to another, and from one day at the range to another. That happens much less when you use FFg powder, so you don't have the balls shooting to different POI from one range sesson to the next, unless its very windy out. i don't have any argument with people who use FFFg in a 20 ga. gun, if it produces a consistent load for them. When it doesn't the problem is very easily seen by using a chronograph. I found it took much more careful attention to loading when I used FFFg powder than when I used FFg powder to get consistency in my shots. I was not expecting that, however, as my best friend had done exhaustive testing with a .62 cal. rifle, and finally settled on a FFFg powder load for the gun, because it gave him the most consistent accuracy, day in and day out. In fact, someone talked him into selling that gun for a nice profit after seeing the several targets he had saved from shooting it over many days. I was expecting the same result in my 20 ga. smoothbore. So, please don't think I am just prejudiced for or against a given powder in making these comments.