Old Charlie: Is the muzzle square to the bore: You might be able to raise that POI by simply filing the bottom of the muzzle so that it releases the PRB sooner than the top. Have you also tried increasing the powder charge, and/or using tighter wads over the powder to get a better seal? Both will tend to raise the POI.
I found, for instance, that using a proper diameter wad in my " 20 ga. fowler" that required using 19 ga. wads because the bore is oversized, to immediately raise the POI on my round ball loads. I was having the same problem you had with the balls shooting close, but very low.