EXACTLY!!
Shops in larger towns/cities did carry ball molds in various sizes, but I don't know about trading posts along the frontier. I can't document this, but I bet those shop keepers cast one ball and left it in the mold so customers could try the cast ball in the muzzle/s of their guns. HOWEVER, I just can't see most frontier riflemen going to the expense of two ball molds for his rifle
The way I figure they could have separated two sizes of balls is as follows. The normal size balls would most likely have remained in the rifleman's shot pouch. A small "ball pouch" or "belt pouch" like a lemuel lyman pouch could carry the smaller balls "before" or on a leather waist belt in front of his body for easy and fast access for close in fighting. Below is one by our own Capt. Jas.
View attachment 138325
However, I'm not comfortable at all to suggest they did carry two sizes of balls.
Gus