It mostly depends on what youre used to or can get used to. I used to carry a 7 1/2" Ruger Super Blackhawk regularly, it didnt bother me in my 20s. I carried a 4" Smith 29 or 4 5/8" Ruger 44 or 45 SA daily for 20 or so years with no discomfort, they still my primary mountain carry and dog walking guns. The length being such that it wasnt in the way sitting in vehicles, chairs, on the ground, cutting firewood or working. This was with a 2 1/2" belt and a 2" belt. The weight of all those is quite tolerable for all day, day in day out carry. Much more does begin to be wearisome to me.
Id also suggest the 1860 as a good balance point for power and weight. The 5 1/2" versions should be a nice size. Id love to cut mine down to that size and put a Navy size steel grip frame on it.