I bought a set of Buck Bros. chisels at Home Depot for less than $30, they work well. Also, I got an oilstone and mineral oil for sharpening. I made a strop block by gluing leftover leather and extra hardwood together, and loaded half with Jeweler's rouge. Also, a $12 sharpening guide from Woodcraft helps keep the chisels flat on the stone.
Also, I purchased a small Japanese woodcarving set from an art supply store. These are various chisels and gouges that are very small, and allow for precision work. I think I paid $15 for a set of 10. Very handy.
If you have extra chunks of hardwood leftover from sawing a blank, you can make a small mallet to chase your chisels with.