I've had 3 or 4, 32 cal. over the years and currently own 2, 36 cal., one flint and one percussion. I much prefer the 36's. No reason other than I like the 36 cal. better.
For small game I don't believe a person could find anything wrong with either caliber. 36 is probably just a wee bit more versatile. If you haven't heard already you probably will hear that the 32 is more cantankerous, sensitive to fouling, kinda difficult to find "stuff" for....and all of that is a little true. I never had any real problems with any of the above mentioned, but, the fact is sub-bores require just a bit more attention to detail than the larger bores. I believe they're worth that little extra effort, both calibers!!
Vic