Question on blueing

[Bigdaddykane]

good morning muzzleloaders of my republican controlled republic! I have a newly purchased cva 50 cal I am extremely happy with fires accurately, looks great. It does have some very lite rusting on bbl, but I notice when loading her, that where I grasp the bbl with my left hand, just under front sight, when I am loading, the bbl has a gritty feel to it. The rest of the length feels smooth has a baby butt. My ? Is this a sign of impending rust problems, or can I just "buff" it out. I always keep a layer of moose milk on external barrel between shooting sessions. Opinions please.