IMO filler is only needed if the powder charge is too light and won’t take up enough space in the chamber to insure that, when the projectile is seated, it’s in firm contact with the powder. In other words the rammer will bottom out before the ball is home.
Some shooters, real marksmen, hold...