Wow! I can't believe all of the great responses! Thank you so much.
Update: I took a spring loaded punch and put a few dimples on the bottom of the front and rear sights and that definitely helped keep them in place.
Next Issue: I'm not sure if the sights were like this or I ended up doing it by putting the dimples in them, but both the front and rear sights are now sitting at an angle (as opposed to perpendicular to the barrel.) If you're looking down the sights both are at about -10° (assuming top is 0°)
