This has been debated at length before. Everyone has an opinion and most are based on the "bigger just has to be better" line of thinking. But everyone agrees that there is a reasonable limit to the "bigness". I doubt that the difference between .50 and .54 would ever be noticeable in actual use.
The current Lyman BP Handbook shows for 32" barrels,
.50 cal, .490" ball, 120 gr.3f, 2045 fps and 1644 ft.lb at the muzzle, with 525 ft.lb remaining at 100 yards.
.54 cal, .535" ball, 120 gr.3f, 1735 fps and 1538 ft.lb at the muzzle, with 580 ft.lb at 100 yds.
As you see, the .50 has about 7% more energy at the muzzle and the .54 has about 10% more remaining at 100 yards. Hardly anything to get worked up over and certainly nothing to lead one to imagine that "the greater mass of the .54 will help to compensate for imperfect shot placement".
But people will insist on believing that their "big .54" can work wonders.
The three most important factors in making clean kills on game are: shot placement, SHOT placement and shot PLACEMENT. If the flatter trajectory and slightly lessened recoil of a .50 enables one to place their shot just a tiny bit better I'd say that far outweighs just a tiny bit bigger ball. :grin: