Hmmm. Won't group, eh? That's very strange, as many of us over here in the UK and the rest of Europe shoot these shallow groove guns at ranges out to 600 yards in the BP military match competition, as our Mr Minshall, who does just that on a weekly basis, would tell you. He might also be able to put you onto the article that either he or Mr Curtis wrote about making the P53 shoot, as well.
But not with round ball, as you note, they were not designed to shoot such stuff, but rather the 535gr Minie bullet, on top of between 55 and 65gr of whatever you can get where YOU live.
There are at least four different bullet moulds to try, but the one that shoots best will probably be one that throws a Minie bullet about 5 thou less then the bore - they seem to upset into the shallow rifling best, as indeed they were meant to in the first place.
A few years ago we tried RB in my Whitworth, just for a larf, as well as my pal's P-H P53. Never saw where the bullets went, but they sure as he** didn't hit the 3 foot square target on our 100yard sighting-in range.
Shoot what it is meant to shoot, and all with be well, I'm sure.
tac