This is so true. I use .530 ball in my Colerain, and I use pillow ticking from the local fabric shop. I never measured it as I was lucky in that it works great. It does matter though. Colerain uses round bottom rifling, which seems to me to be just a tad more forgiving on patching material.
Also when one washes the material in hot water and then machine dries it, it can thicken a bit.
When I tried a .389 ball and the same ticking in my new .40..., Green mountain barrel and squared rifling..., it didn't like the fit so much. Tried the same ticking with a .395 ball; wouldn't load. Tried the .395 with thin cotton cloth; loaded, but again not very good groups. Then I doubled the thin cotton, got a snugger fit, and better groups. So I will be going to the fabric store today with a micrometer, to see if I can find cloth that's the same thickness as two pieces of the thin cotton, so it will work for the .40. I also have a mold for .380 ball, and will in a few weeks try them with some really thick material.
So there are lots of variables, and then if you are used to loading a ball made of a certain type of metal, lead, lead alloy, wheel weight, antimony, the diameter can change just enough if you switch to really change how the rifle shoots.
LD