I wouldn't think of building a rifle without a vice. Mainly because I am allergic to drill bits & chisel tips in my legs ..... :grin:
Yes, it can be done, but it is a heck of a lot harder.
My suggestion is get the tools first, then the parts. That way you can do it correctly and safely and also not make a screwup ya cannot repair or hide. :redface:
I already had the vice, but the drillpress is the first thing I bought. $ 129. at Lowes for a Delta benchtop. And there is no telling how many Thousands of holes I have drilled with it & have never regretted the purchase. You can get one cheaper at Harbor Freight & it will get ya by, but saving a bit & getting one from Lowes would be much better. (IMHO)