Several of us do it all the time. In both 50 and 54 cal with charges up to 60 grains of 3f or Pyro P it's pretty hard to tell the difference from slow twist barrels out to 50 yards, our normal shooting range. Heck, with small game loads down at 30-35 grains you can stack the balls on top of each other.
Over 60 grains the groups do tend to open up. But only enough to be a concern to target shooters. I'll hunt deer all day long with a ball over 80 grains of powder delivering a 2-3" group at 50. That's cuzz our shooting is usually at 50 yards or less. I'm not worried about what balls are doing at 100 yards from fast twists, and in fact I've never tried it. No need and no worries.
Most of my guns have slow twists and that's fine too because I don't usually hunt with conicals. But I'm not worried enough about the fast twist models to go to the expense of switching them over to slow twist.
If you're using fast twist at longish ranges, then yeah, it's worth testing hunting loads out there. If they're not performing to your specs, by all means invest in a slow twist. If they're shooting fine, no one will shame you about fast twist unless you tell them you've got the dreaded condition.