If you go with a harder alloy you will want to use a bit more undersized ball than normal. A thicker patch will make up the difference and still provide a good group.
There was a fella (Roundball I think it was) who did some test shooting of brass bearings on this forum many years ago. And brass is a sight more hard than any alloy lead. He got good results. Now I don't recommend it, but it can be done and hard balls are not going to perform worse than lead. Just who wants a hard metal ball stuck in the barrel? Not like you can use a ball puller on such ammunition.
There was a fella (Roundball I think it was) who did some test shooting of brass bearings on this forum many years ago. And brass is a sight more hard than any alloy lead. He got good results. Now I don't recommend it, but it can be done and hard balls are not going to perform worse than lead. Just who wants a hard metal ball stuck in the barrel? Not like you can use a ball puller on such ammunition.