your barrel has (most probably, anyway) a 1:48 twist - that is, the ball will spin once in every 48 inches of forward travel. Most barrels designed to shoot 50 caliber roundball are in the 1:60 range.
as a general rule, you must spin a heavier (given the same caliber) projectile at a faster rotational rate in order to stabilize it in flight.
there are usually two sweet spots, where the groups will tighten to their minimum size as you change the amount of powder ... one of these will be a fairly light charge (most use this for their 'target load') and then, often at pretty close to double that charge, you'll find the groups tighten up again - this is the hunting load.
it is said (but I have not proved it empirically) that a barrel with slower twist is more forgiving as to the variations in the charge. in other words, the quantity of powder need not be as precise, and the greater the variation you can get away with, the better the barrel will appear to shoot, because the mechanics are forgiving small errors in measurement of the charge.
so, here you are stuck in a tough spot: you want a nice slow rate of twist so there's a bigger sweet spot, but you want to spin a heavier projectile (say, a REAL bullet) fast enough to keep it from doing goofy stuff as it goes downrange.
the 1:48 is, therefore, something of a compromise. having said that, I would disagree with those who claim that these barrels aren't capable of decent accuracy: I've had a T/C Renegade for three and a half decades, and it's always printed groups tighter than I can hold it.
I agree that a tighter patch is a good start point ... (and patching that you got for yourself, not some premade stuff which might have come from the factory last week, or might have been in a warehouse for a decade, and you have no way of knowing)
also, I would try a few of the many lube recipies (moose snot and the one using Ballistol come to mind) in your new patching. then, you should be able to recover your patches and see what's going on(i.e. cutting, burn through or whatever).
don't be embarrassed to use a big board and get close when you're working up a new load ... think 4x8 plywood and newspaper at 25 yards, if that's what it takes.
patience is a virtue, and only change one variable at a time.
good luck, and make good smoke!