If your balls are within 1 grains, plus or minus, they should be good enough for most shooting situations. Only when shooting target matches involving bench rests will you usually see a difference in group size.
The general question about how much of a variance is okay for cast bullets needs a slightly broader answer. It really depends on the caliber and weight of the RB. 1 grain makes much more of a difference when shooting a .390 RB in a .40 caliber rifle, when the ball only weighs about 85 grains. By contrast, in a .58 caliber, when the RB weighs more like 280 grains, 1 grain is far less significant a factor. Think not in terms of a specific weight for your tolerance factor, but in a percentage of the total weight that you can accept within your tolerance range.
A good friend who was shooting a "Slug Gun " at Friendship that was .69 caliber, and fired a 2-piece bullet that weighed 1760 grains sorted his bullets so there was NO--NONE-- difference in the weight of 10 rounds he had carefully packaged up in a wooden box he made for them. Zero Tolerance. Even he admitted he was probably being a bit Anal about the subject, but he had the time to sort his bullets, and the equipment( electronic scale) to do the job. Each bullet was in turn wrapped in paper toweling to protect its outer shape, and his loading jags were custom made to exactly mirror the outside diameter of the nose of his bullets. Those huge slugs were loaded with a false muzzle, were paper patched, with sperm whale oil, and slid down the barrel like water on glass! A drop tube was inserted in the barrel to load the powder in the base of the barrel, without leaving any dust or granules on the sides of the bore. He had a photocopy of a 10 shot group he fired at 500 yds. that measure 5.26". That is what consistency means to target shooters.
The presense of air pockets in the balls, wrinkles on the surface, and distorting the surface with an ill-fitted loading jag on your ramrod are more likely to affect the group size. Some casters have begun to use case tumblers to round their cast RBs, by putting the balls in the tumblers, or vibrators, without any grit, and letting the balls pound against each other. With pure lead, this will often reduce the size and effect of any air bubbles or pockets in the balls, and the tumblers tend to remove sprues, and make the balls uniform in diameter, if not weight.
Balls should be sorted by both weight and diameter if you intend to do serious target shooting with them. It helps if you sort the balls so that all of the balls used to shoot a given target, or " string " of targets are the same weight, and the same diameter, regardless of what the actual weight may be.
You can, and will have to, adjust your powder charge based on relative humidity, and temperatures at the range each time you shoot in a target match. Most target shooter keep notebooks to record every shot taken, weather conditions, etc.
Fortunately, most of us mere mortals can have plenty of fun shooting what we get, either from a mold, or from a commercial supplier, without bothering to measure them by diameter or weight, and without tumbling the balls to get rid of sprues. There is accuracy, and then there is ACCURACY. Unless you are going to get into the bench rest shooting game, Extreme Accuracy with a RB is just not necessar, IMHO.