one thing to consider in regards to riflemen (both regular and irregular) is that they were often supported by infantry (as Gus mentioned above). How many circumstances existed when riflemen were rushed by the opposing infantry ?
Strategically riflemen fought from distances greater than 100 yards and were likely able to retreat long before any enemy advanced on them.
The need to reload in a hurry or rush without having a difficult round to pound down seems like a rare occurrence.
In-fact one of the only battles i can think of where this may have been an actual problem would have been the Alamo as many of the Alamo were civilian volunteers armed with rifles. At the Alamo men were armed with multiple muskets and pistols as well as rifles as it was a very desperate situation for them. Having balls of many diameters might have been their only option too having been outnumbered and under siege.
Actually, situations where riflemen were at a disadvantage due to their slow reloading and lack of bayonets happened often enough to be noteworthy in the AWI. In
The Book of the Continental Soldier it has a page of period quotes about the riflemen, including some that were highly critical:
Colonel John Simcoe, Queens Rangers: "their not being armed with bayonets, permitted their opponents to take liberties with them which otherwise would have been quite improper."
Another British officer (Kindig also quotes this in
Thoughts on the Kentucky Rifle: "..about twilight is the best season for hunting rebels in the woods, at which time their rifles are of little use; and they are not found so serviceable as a body of musketry, a rest being requisite at all times, and before they are able to make a second discharge, it frequently happens that they find themselves run through the body by the push of a bayonet, as a rifleman is not entitled to any quarter."
Another: "Riflemen as riflemen only are a very feeble foe and not to be trusted alone...they must ever be supported by regulars, or they will constantly be beaten in, and compelled to retire."
There were more, but I'm tired of typing them. Most critical comments came from British officers, and so could be interpreted (as Kindig did) as evidence of that they felt intimidated by the riflemen and felt the need to belittle them. But they do seem to provide evidence that it was not unheard of for British troops to meet unsupported bodies of American riflemen.
Basically, I agree with everything you say about how riflemen could be used to best advantage, except that it might not have been as rare as you think that riflemen not used in that way - and found themselves in a vulnerable position in which they would need to reload quickly. This also would explain the need for the folding pikes that some riflemen were issued with.