So I replaced the sites on my TVM Flintlock to smaller lower sights than what came on it. My problem is is that it's a swamped Barrel so the breech end of the barrel is slightly larger in diameter than the muzzle of the barrel. But the front and rear sights are the same height. Does that mean I'm probably going to have to file down the rear sight and has anybody ever really had to file down the rear sight instead of the front sight?