I've lived most of my life on the Continental Divide where, depending on which direction I went after leaving home, I could be hunting Bighorn sheep, Shiras moose, grumbly and black bears, elk, mulies and whitetails within about an hour at the most. I am a casual muzzlestuffer hunter compared to most; my time gets equally consumed with long bows, my grandfather's 1895 Winchester in 30 U.S. fed a diet of cast bullets, and somewhat arcane centerfire rifles like 30 Newtons and 358 Norma Magnums. So I've shot a lot of stuff, but not a lot of it with muzzlestuffers. All my BP hunting is with conicals.
I got my .45 as a gift. I shot a big muley with it; good bullet placement, but my future suppers didn't expire as quickly as I expected and wanted. A guy I knew had a .50 that he felt kicked to hard, so we swapped. I shot a couple of deer with it and an elk; terminal performance was what I was looking for. And then I shot a moose and a black bear within a few weeks of each other. Performance on both did not leave me happy.
Bought a .58 because... well because. As most of my big game hunting at the time was with the .358 Norma Magnum, seemed to be logical. That rifle and I never got along; I think it hated me. And tune up shooting at gophers through the summer taught me I could probably throw rocks with a flatter trajectory.
So one last gun swap... a guy in town was determined to go to the Yukon and shoot a massive grumbly bear with a muzzlestuffer. He'd bought one of T/C's kit Hawken rifles in .54 that he hadn't laid a finger on yet; traded it to me including a combination mould that threw a round ball out of one cavity, a flat bottomed conical out of the other. Along with a simple T/C rear aperture sight; I put that on before I even fired it the first time... haven't taken it off since.
Been whacking gophers all the way up to moose with it ever since. Stupid accurate, and that flat bottomed conical has left everything I hit with it DRT. I am finding the recoil of the big conicals with maximum charges to be getting towards uncomfortable now that I'm getting into my senior citizen years. But there's no law that says I have to shoot maximum charges with heavy conicals either. And I only need to fire it once to get the desired results.
Anyways, as a casual enthusiast that has drifted back and forth between calibers, living where big game hunting is available, I've concluded that the .54 is it. You may not need all that thump; if so you can download.
However, the .54 fits the mantra I learned in the military: better to have and not need, than need and not have. In my casual experience and opinion, if you're going to choose just one for all hunting, .54 is it.