Sounds like you're onto something w the bare ball.
Before you pull the trigger on a new mold here's a tip I used...not just read about, not heard someone talk about , but actually have a few years experience with.
I bought a .575 minie mold that was too small, it patterned more than grouped.
Being too cheap (and broke) to buy a new mold I read on Lee Precision's site about shimming a mold. I used some plain old receipt paper, greased it real good, and in about 10 minutes of tinkering had a mold throwing a .580 minie.
Lee says you can shim up to .010 and still be accurate. You can remove the paper shim in seconds and no harm to
mold.
This turned my musket from shooting a 7 moa group to 3
moa with those lousy mil/spec sights.
Something to consider.......