If "nipple passage" means the tiny hole thru the nipple, leave it alone. Do not enlarge it.
It is small for a reason and enlarging it can damage your lock by allowing the powder gasses to blow the hammer back.
If you want to do anything to a passage, buy some old fashioned "pipe cleaners" and scour the small hole that connects the bottom of the nipple with the powder chamber in the bore.
That little hole may have a bunch of fouling in it that's causing the problem.
(The pipe cleaners I'm talking about are used to clean the stem on a tobacco pipe.)
Also, when you are loading the gun, make sure the hammer is at half cock and no cap (or remnants of it) are left on the nipple.
You want a clear passageway for the air under the ball or Minie' to escape thru when the projectile is being rammed down the barrel.
If the connecting passageway is clear, the escaping air should automatically blow fresh powder to the base of the nipple doing, in effect the same thing you are doing by adding extra powder to that area.