I'm going to speak on purely a practical standpoint. I know nothing on which was more common in the 18th century. From what I can tell from the 19th century, the 10 gauge was the all around shotgun gauge up until smokeless powder came about. The 16 is a favorite of mine, but I believe its popularity was more due to the fact that in a SXS, it was much less bulky than a 10, especially in a flintlock.
The difference between the 20 and 12 gauges is that the 12 can send more shot down range effectively. For the most part, a 20 gauge you are looking at about a 3/4 to 1 ounce load for general purpose shooting. I've seen where some have gone as heavy as 1 1/2 ounces of shot. At 1 1/2 ounces, you are not going to have much velocity if shooting a safe powder charge, although it would probably make a good turkey load. A 12 gauge on the other hand can easily handle a 1 1/2 ounce payload with good velocity. It is my opinion that the idea of a square load is a simple way of eliminating having to bring another shot measure. While using equal volumes of shot and powder often produces a good load, I've never found it to be anything special. "Less powder, more lead, shoots farther, kills dead." That's a saying that holds a lot of merit. I'm basing this on both the pattern board and hunting. Shot stringing is another area some look at. I'm not particularly concerned about it as much as some, and in the case of 20 vs 12 gauge, the differences are not as great as you would think. See the following link for tests done by Federal Premium.
The Truth About Shotgun Ammo: 6 Questions Answered at Federal's High-Tech Range.
Take note that in 12 gauge, the 3 1/2" load carrying more shot did not have a notable difference in shot string length than the 3". The same is true for the 1 ounce 20 gauge vs 1 ounce 12 gauge. While I realize this is modern shotgun ammunition, the same principles apply to muzzleloaders. You might notice that the 12 gauge patterned tighter than the 20 gauge, but I think this is likely more to do with the particular gun. Change guns, and the results could be swapped.
So what's stopping you from putting 2 ounces of shot in a 20 gauge? Velocity, at least from a fowler barrel. To use such a heavy payload, would require a low amount of powder to keep things safe. The result would be pathetic velocity. The 12 gauge on the other hand could handle a 2 ounce payload, although it is getting to be about topped out there. My Pedersoli 10 gauge, is in reality an 11 gauge. With 2 ounces of shot, and 100 grains of powder, velocity is 900 fps at the muzzle. That load worked great on turkeys, and would work on flying birds too, but it is getting on the slow side.
The other factor is choke, which is a post civil war invention. To remain truely PC, would require a cylinder bore. That Pedersoli I mentioned is a cylinder bore in both barrels, and let me say that I needed all the shot I could get for turkeys. I ended up shooting my turkey real close at about 10 yards, so anything would have worked. If it hadn't though, my maximum range was 25 yards, and not one yard past that. I really doubt you are going to be able to get an effective turkey hunting pattern from a cylinder bore 20 gauge at that, and even 20 yards might be pushing it. As a wing gun though, you don't need nearly the same pattern, and a 20 would be quite effective in that role.
The only real drawback to the 12 is weight and bulk. In a single barreled fowler, the bulk wouldn't really be a factor either. I'd say go for the 12 gauge, and wouldn't discount a 10 either.