Like mrfishnhunt, I take ten .32 PRBs in a bullet board for squirrel, and 3 .50 cal speedloaders plus a load in the pipe for whitetail. I'm never more than 30 minutes from my house or truck -- if I were going deep woods or for an overnight trip, I'd bring much more for just-in-case.
In my effort to minimize gear on squirrel hunts, I've narrowed it down to:
1. a teardrop capper
2. a cap tin full of cleaning patches
3. the small bullet board
4. 10 or so 3-inch segments of plastic drinking straw, masking taped at both ends and filled with 20 grains 3f
5. a homemade short starter/ramrod handle (one end of the starter has a hole to accept the thin .32 ramrod -- it gives a bit more grip and control when ramming down the ball).
These items fit nicely in the two pockets of my camo game vest. The reduced bulk is liberating, and my loading speed has significantly increased (only an issue after a shot on game, when I reload as quickly as possible in case a follow up shot is needed, which it has not been so far).
As a largely self-taught hunter, I had to learn a few things the hard way. I started squirreling about 10 years ago with a pump shotgun, but had no idea how many shots it took to fill a limit of 6 squirrel. As such, I returned from my first hunt with one squirrel and 26 #6 shells, 25 of which were in the absurd shotshell belt I wore that one time. Ha!