For elevation, you can experiment with other loads, and patch and ball combinations to see if you can find one that shoots closer to your POA. For windage, you can file the front sight, or if it has a dovetail, you can move it the opposite direction you want the ball to land. So, if it is shooting to the right, drift the sight to the right, to make the gun shoot to the left. Got it?
If you still can't find a load to shoot to elevation, you can replace the front sight with a taller on, or file the existing one if you are trying to raise the point of impact. Some people will solder on to the top of the blade some extra metal to lower the impact point. I have seen this done more frequently with BP revolvers and pistol, than with smoothbores, but the principle is the same. Civil War revolvers were sighted to hit a man at 75 yds, a very long distance considering the pistol skills of the average soldier at the time. Even today, that would stretch the skills of most soldiers. When you shoot those guns, and replicas, at 25 yds, the balls hit at the very top of the target, if they hit the paper at all. Only raising the height of the front sight can bring the POI down onto the paper.