Check the keys and the escutcheons for burrs. Put some kind of marking dye on the keys and then see where they are rubbing. Often, a few strokes with a Jeweler's file( you can buy a cheap set at Harbor Freight) is all that is needed to get these keys to go into the escutcheons correctly. Make the keys work smoothly before you try them through the barrel hanger. That way , you will know if the hangers also need a little " Polishing".
Some keys are made to be too wide, and therefore have no "play" front and back to allow for expansion and contraction of the wood stock as humidity changes. You don't want loose sizing of the escutcheons, but you do want some "slop" or "play" in those barrel hanger slots. The Play should be front and back, parallel to the barrel, not play up and down. Keys with too large slots, from barrel to bottom can be tightened by hammering on the bottom of the key to close the "loop", or "slot" somewhat.
If you find a hanger is so tall that its pushing against your ramrod in its channel, File a bit of material off it so that your ramrod is Not affected. You will be surprised how much that affects the accuracy of your gun.