Your going to have to re-Brown it, Right?
Go after the top 3 flats either draw file or 220 grit on a block.
Then I'd mark the side flats with a pencil while it's in the barrel, make a measurement so you have an idea where it's at and only sand (block sand) that section of the barrel that's above the stock.
I would not do the entire side flat,, the part of the barrel that meets and is below the forearm isn't seen anyways.
If a tiny amount of the original rough browning is above the rail it won't be that noticeable.
As said you probably won't have to take that much off.
I would not take so much off the sides that' I'd have too try to build the channel back up again,
That, would be noticeable.