Hard to say. I once calculated my bow throws about 45 ft pounds of energy with the arrows I use, yet it kills as well as a muzzleloader or slug shotgun out to 30 yards or so.
If you're punching a hole in one side of an animal and out the other through a vital organ in the middle you have enough energy.
We are used to reading tables for jacketed bullets at relatively high speed that have to expand for killing performance. A .490 or .530 round ball is already a large "mushroom" viewed front on. I would take a W.A. guess and say the range at which you hold 400 ft lbs of energy with a ball of .490" or larger is as far as you should attempt a deer sized target; and that may be stretching it. A LOT depends on shot placement. With modern bullets that would be 1,000 or 1,200 ft lbs or so.
"Optimal energy for marginal hits"? Darned if I know. You would need a HEAT round from a 120mm tube to take out a good buck cleanly with a rear left hoof hit. :hmm: Might not even be good enough for a tail hit with that.
Seriously, energy can't make up for poor hits. You cannot drive a round lead ball fast enough to impart a killing shock wave through soft tissue (i.e. "knock 'em down" with a gut shot). Round balls kill by hemmorage, just like a bow, or by snapping bones to disable them long enough for hemmorage to work. Spine & brain shots, of course, drop 'em flat. Better to go slower and hit what your're aiming for. In realistic terma, I like at least 80 gr FFg in either a .50 or .54 for shots out to 75 yards on deer. If I expected to need much farther I'd load hotter or move up in size. I use a .54 with 84 gr FFFg in my flintlock and that is plenty for whitetail where I am comfortable using the open sights.