I don't know what a BBTOH is, so can't help with that.
I can tell you that nominal caliber is not always true to bore size. I have a Hawken built around a Green River Rifle Works .54 caliber barrel. These barrels were very nicely made but were frequently undersized. Mine is closer to .53 caliber, and I shoot .520" balls in it.
My opinion is that this should be fun. Life can be such a battle sometimes, and I just don't want to be fighting with my rifle trying to get a ball down the bore when I get some time on the range. I would recommend trying a smaller ball. Don't be afraid to go down .020" below bore size for your round balls. You may need some thicker patches, but thicker fabric holds more lube. More lube is always good. A felt wad saturated with lube between the patched ball and the powder will protect the patch from burning and help keep fouling soft, which will also ease loading.
If you can afford it, order some .520", .526", and .530" balls for your .54 caliber rifle, in addition to some .54 caliber wool felt wads, and patches in every thickness (.005", .010", .015", .017" ticking, and .020"), and a tin of lube. Track of the Wolf has everything you need. Most of us here on the forum don't like pre-lubed patches. The "Wonderful Lube" in pre-lubed patches appears to deteriorate after less time than you would think. I believe something like beef or lamb tallow, mink, or SPG is better. Heat the tin of lube in an improvised double boiler arrangement and drop in a few patches and wads. They soak up the melted lube immediately. Pull them out with tweezers and lay them on a cookie sheet or waxed paper until they cool. Put them in another tin or a butter dish, and you're good to go.
I don't want to start another lube thread. We all have our favorites, but they all work. Just pick one from a reliable vendor and go with it.
Best regards,
Notchy Bob