Start at 70 grains of FFg and work up by 10 grain increments to see what shoots the best groups at 50 yards. The 3/4 oz ball is going to put a huge hole through any deer you shoot, and kill it quickly, so you don't need to dig trenches on the other side of the deer with the ball after it exits the deer. Without knowing your gun, or its bore dimensions, and other loading information, its only a guess what velocity you are getting. But, Hodgdon's Data Manual indicates that 60 grains is giving you about 972 fps. MV, while 70 grains will produce 1054 fps. MV. That may not sound like much velocity with a modern 30 caliber deer rifle, but you are shooting a .58 caliber rifle, with a ball that weighs more than 2 times the weight of any 30 caliber bullet. Its the size of the hole, and the weight of the ball that is going to kill that deer, and at 50 yards, max, the deer is going to die quickly with such a large hole in it, dropping its blood pressure almost instantly.
Concentrate on producing an accurate load, rather than a fast load. Plenty of deer have been killed with what you now consider your target load. If you are still worried, do some penetration testing. Use bundles of newspapers and fire your target load, and then any greater powder charge load to see what, if any difference in penetration takes place. Bundles of newspaper are easy to acquire, easy to take apart to find your slugs, and easily disposed of. Just take a tape measure, or yardstick with you, and some extra string to retie the bundles when you.
On a broadside shot of deer, most deer are less than 8 inches deep across the chest. You don't need 20 inches of penetration to kill a deer. 4-6 inches of penetration is all that is necessary with that large caliber rifle.