Your cleaning method is missing pulling water through the flame channel. A brass brush isn't going to seal and push/pull water through it.
A jag and cleaning patch will clean it out better. Just stick the breech end in a bucket of water with a couple drops of dish soap. I prefer hot water because the bore dries so much faster when done. Run a jag with patch up/down the bore, it will pull water up through the barrel clear to the muzzle. Each time back and forth is pushing/pulling water through the flame channel. The brush is just getting the bore where it touches and not flushing the flame channel at all, the only cleaning you're getting of the patent breech/flame channel is soaking in water with barely any agitation. After cleaning, dump the water and do again with just water for a rinse. Patch on the jag should stay clean for this one mostly. No need for the PVC pipe treatment. Using alcohol to help dry is a good idea. Running a pipe cleaner through flame channel is good. Pour some alcohol down the muzzle and let it go through the flame channel, then blast some air through the flame channel from the nipple end. I just use the little cans of air, like the ones sold for dusting a keyboard.
I started out using a nylon brush through the bore as part of my cleaning routine but abandoned it as it simply doesn't seem to be needed at all for me to get my bore nice and clean. Patch on a jag does the job just fine. I do believe not needing a brush is dependent on cleaning method and type of lube being used.
Sounds like you have a rust preventative that works for you now after the first one failed.
To me a wedge pin that can be removed/installed by hand is a bit loose. I have to tap mine out and do the same to put it back in. I'm with Brit about stock to barrel fit. Brit also mentioned in his first post that 56 grains is a "rabbit" charge for a .50 cal., he is not wrong. My .54 TC barely showed paper plate groups until 70 grains. I realize it is a bigger caliber but still. Many say they get decent groups with a light "target" charge, but my TC shoots horrible groups until I get up to 70 grains. At 70 it still isn't anything I'm going to take a picture of. 75 jumps to "minute of deer heart" and at 80 it'd put them all in the same hole if the nut behind the gun could manage it.
Many, many folks report best accuracy between 70-90 grains.
Your swabbing method isn't softening the fouling at all, the brush is just knocking some of it loose. A lot is being left in the grooves and stuck to the walls. If brushing with the muzzle pointing up then the fouling knocked loose is dropping down into the patent breech and causing the hang fires. The dry patch is likely pushing even more fouling down into it too. Use a barely damp patch. Water, spit, alcohol, several things work fine. Push down, pause for fouling to soften a bit, then remove again. I do two passes with damp and one with dry when working on load development or wanting tiny groups on paper. Pop a cap after to ensure flame channel is clear before loading. Throw a dry patch, leaf, etc. on the ground and point the barrel at it, when the cap pops it should make it move. If at a range then simply pop a cap per the rules with it pointed down range, one almost always does the job in my experience. No more hang fires, that alone will improve groups drastically.
Shallow groove 1:48 twist TC's tend to prefer a short overall length conical, short being around 1". Hornady Great Plains Bullets often shoot extremely well in them. If you read somewhere they don't make them anymore they are talking about other calibers, they are still made in .50 cal. They are a hollow base bullet, so no wad with them. May be tough to find right now in the current ammo hoarding rampage.
Most everything I've read about accurate PRB loads in a TC from people is that a pretty tight fit is needed. Most report a minimum of an .018" thick patch with a RB .010 under bore size really show good accuracy. My .54 was no exception, I see best groups with a .530 RB with an .018" pillow ticking patch or a .535" RB with a .015" patch. You could also be dealing a bit with patch cutting since the bore was unfired, but your steel wool treatment should've helped that out.
My starting point with your rifle.
Conical I'd start at 75 grains of 2f and work up in 5 grain increments from there. I'd use a Hornady GPB or a maxi-ball (not maxi-hunter). The maxi-ball I'd use with only the narrower bottom groove lubed, leave the upper empty. Shoot a group, increase charge 5 grains, shoot a group, etc., etc. I'd swab with two damp patch passes and one dry every single shot, pop a cap before loading the next charge. While it's important to know where your foul shot lands if you're a hunter, I wouldn't include it in my group measurements during load development. Don't exceed max charge recommendation of course.
PRB I'd start at 65 grains of 2f and work up, using your .490 RB and .015" patch since it showed some promise. Like Okie, I experience great results with mink oil or Frontier's patch lube (there are others that worked for me but those two have stood out accuracy wise for me). Swab between shots or every other shot if you prefer, it sounds like it did okay until the 3rd shot without a swab.
I swab every shot. I like, in order; the better accuracy I see, ease of loading the next shot, and knowing that possible embers are out. Your comment that you're first two PRB shots are touching and then the third jumps out is exactly what mine does if I don't swab. The fourth shot without swabbing is clear off a paper plate and I use an even tighter PRB combo than you. You could try swabbing every other shot and see if your PRB set up shoots well that way.
If using Pyrodex I'd use mag caps and also ensure nipple has at least a .031" flash hole. Something like a Treso nipple with a .028" flash hole wouldn't be a very good choice with a sub powder.
As with most anything there is more than one way to skin a cat. I'm not saying something different won't work as well as what I do and prefer, but I do believe I could have your rifle shooting nicely in short order with the above.