You sound like you may be a little new to muzzleloading. With that in mind here are a couple of other pointers that you will want to remember.
Don't short load the ball, ensure it is all the way down on the powder charge. Short loading can damage the barrel or if everything goes wrong, damage you.
Start out with no more than 25 grains of Fffg.
Using the balls recommended above, use a good cotton patch and a good patch lube like T/C Bore Butter or something similar.
If you are having to force the ball down the barrel as hard as you were saying, something is wrong. Pistols are a bit of a bugger to load if you don't have a loading stand but, it should never be so difficult that you have to hammer a ball down the barrel.
Run a cleaning patch moistened with your saliva up and down the barrel a few times between each shot, gets that fouling out, makes loading easier.
Go slow, these old guns are fun, but they certainly are a different animal.