I have both and shoot them side by side a lot. What I've found is that the Patriot has slightly better inherent accuracy shot from a rest, but the Trapper is easier to shoot offhand and will generally shoot with, or slightly outshoot the Patriot. The Patriot has better balance and fits the hand better. The Trapper can be adjusted to have a really sweet, light and crisp trigger though. The sights are good on both, with a slight edge to the Trapper because of the thin front blade.
One thing I've learned on the Trapper is to grip it higher up than you normally would. This lets it point a little more naturally and has helped my consistency.
Brett is right that the sights can shift. A reference mark is a good idea. So is tightening the dovetail. I built mine from a kit, so I tightened it when I built it. You can either use a pointed punch to stipple the bottom of the dovetail slot, or lightly tap down the tops of the dovetail. The stippling raises metal inside the dovetail and tightens the fit.
Mine seems to like a fairly hot load. I generally run 30 grains FFFG Goex, .490 riding in a .018 pillow ticking patch using Ballistol or Hoppe's #9 black powder lube/solvent. It's more accurate with Ballistol, but it fouls less with Hoppe's.
One big negative with the Trapper, at least with mine, is that the lands are sharp and cut patches when the gun was new. Also, the crown had a sharp "rim" all the way around. I'd suggest a slight recrowning to break this rim if yours has this problem. It took a bit of lapping to get rid of the sharp edges on the lands, but they would have smoothed out after a couple hundred shots anyway.
I would be hard pressed to pick between the two guns for target shooting. That's the reason I shoot them both together so much. I love the Patriot, but considering that they're hard to find and expensive, I'd probably give a nod to the Trapper. I score better with it anyway.