You just gave two reasons to use 4f as a main charge and answered your own question.
Not everyone has access to Swiss and 4f will give more energy than "crappy Goex" 3f with the economy of more shots per pound. Add to that there is less fouling with 4f and the reasons to use it are piling up, while the reasons not to are non existent.
As far as Swiss goes, their 4f is marketed as a main charge powder, not a priming powder, so there is that.
It makes me wonder why someone would use a powder not marketed as for use as a pan powder, as a pan powder?
Some of us folks like to think outside the 4f priming, 3F pistol, 2f Musket, shotgun box, that is a modern invention and not period or historically correct, which kind of cuts to the heart of this forum.