Being able to shoot a conical accurately may depend on the depth of your rifling and if it's square or round bottomed. I have had three .54 caliber rifles with square groove, relatively shallow 1-65 twist rifling (all Pedersoli). They all shot a Hornaday Great Plains bullet very well. Crisp cut holes out past 100 yards in targets...so no wobbling. They also shot PRB very well.
If the rifling is a bit deeper and round, you could try a tight-fitting wad behind a conical to see if that helps the accuracy.
If you do "have" to shoot a PRB, don't feel like your horribly disadvantaged. I have shot a number of deer with those big Hornady's and a number with .530 & .535 PRB. That includes big, heavy, mature northern whitetails with live weights in the upper 200's. Whether conical or round ball, they both pretty much put a big hole through and through with NO difference in killing power whatsoever. At the distances most of us are shooting game with these guns, the conical is not a necessity in that .50.