The closer the sights are to the barrel, the faster mirage will begin to distort your sight picture as the barrel heats up.
So on a multiple shot woodswalk match, a taller sight will usually give you an "edge". Unless you're trying to participate in novelty events, like splitting horizontal playing cards, etc... you'll want be as close to the bore as possible for such precise work.
A tall(er) sight is more likely to be damaged, especially on a hunt trip. And we're seldom lucky/unlucky enough to heat up a barrel from multiple shots on hunt. Sure sucks to look down while on the hunt of a lifetime and see a sight all bashed, bent or missing..lol
The short sights have served me just about perfectly on my Colonials.