Gus, that's not what's happening. In fact the opposite would be the result when hunting at black powder velocities. The lower speed ball that went deeper would kill the animal faster than the ball that flattened sooner at the higher velocity...IF the animal was large enough.
"Energy transfer" is not what kills animals, but actual tissue damage. Energy transfer is part of the idea of hydrostatic shock, and it doesn't apply at black powder velocities, if it applies at all.
The reason that the higher charge reduced the penetration is that the friction against the ball spiked higher, sooner. The hydraulic pressure in the jug created upon ball impact, made the jug harder due to fluid compression, thus deforming the ball more, and causing it to slow down at a higher rate with more deformation than the first shot. The fragmentation then reduced the overall mass, so the parts slowed down even faster. The jug itself could not stand the pressure so it shattered. There would be different results in an open trough that equaled the length of the closed jugs, where the pressure would not spike nearly as high.
Say we drop two objects of the same mass and diameter from the same height, say 8 stories onto the ground. Galileo demonstrated that they would impact at the same time. Mine is a solid sphere of glass, yours is a ball of sand of the same diameter and weight. Mine would go deeper since yours would deform. You then go to the 12th floor, and thus have more energy, but the results would be similar... the sand would deform wider, faster, and not go as deep as the sphere that didn't deform.
LD