I can only speak for me, but during the building of a rifle, I will have installed and removed the pinned barrel not less than 15 times.
If the pin holes don't go thru the stock, removing the pins and the barrel would challenge a saint (darned hard to drive the pin out if there is no thru hole).
I use 5/64 diameter pins thru the stock and barrel lugs and I use a 1/16 diameter pin as a punch to remove them.
I drill the holes with a 5/64 dia drill bit.
If you drill straight thru, backing straight out to clear the chips every so often, the hole will be a snug fit to the pins.
If you allow the drill to wallow or move from side to side while your drilling, the pins will be loose. This of course is unacceptable, but if it happens, a few light hammer blows with the edge of the hammer face on the outside of the pins will create little raised areas. This will keep them tight in the hole.