With #4 buck, at .24" in diameter, you can get two balls to sit in one layer across the bore of your .54 smoothie( .550 Diameter bore?), but not three of them. You would have to drop down to birdshot to get three pellets across, Probably down to #2-#4 shot to get three pellets to lay across on the same layer. I don't think that is what you are looking to do.
If you want to use buck shot, then stick with the #4, and put two layers, two lined in one direction, separated by a OS card, and then the next two at 90 degrees to the first two, followed by one or two OS cards to hold all the stuff in the barrel. If you put an OS card on top of the large bore diameter RB, to separate the Ball from the buckshot, you should get a bit tighter pattern with the buckshot.
Basically, you are shooting a very small bore shotgun, about 28 gauge, and there is a reason why you don't find buckshot loads made and sold in that gauge. Its too small for this. Most Buck and Ball guns are 16 gauge, or larger, bores( .69, .72, .75, .80, etc.) Even a .62 caliber/ 20 gauge gun is difficult to load with a buck and ball charge.
I think the 60 grain charge of either FFg or FFFg is a good place to start, also. YOu might take it back down to 55 grains( 2 drams ), to see if that improves the group. Considering the effective range of the small buckshot you are forced to use, You have more than enough power even with the lighter loads.
Instead of using small buckshot, and a ball, you might be better to consider using two balls loaded on top of each other, each patched, with the same small charge of powder, and see how they perform. They are more likely to put the two balls close on a target at even 35 to 50 yards than any buck and ball load you are likely to make. Keep the powder charge low, because the gun is going to be knocked about a bit with two balls in the barrel. Remember that chamber pressure tends to get higher, faster, in small bore guns, for the same proportional increase in powder charge.