YOu remove scratches left by your abrasive by going to a finer abrasive, and progressing down until you have the "LOOK" you want. If you put oil on the finest emery cloth, you can usually polish out scratches left by the dry emery cloth.
There are fine polishing compounds that you can buy at paint stores, like Pumice, and Rottenstone. These are often used to smooth finishes on furniture, where streaks are being polished out of varnished finishes. Used with a rag dipped in oil these will polish out your scratches. The various abrasives and rouges come in different size from course to fine, so get some help from the store personnel the firs time you are buying these product to find out what the grades of the product mean, compared to the emery cloth you are using.
Some brush finishes are made used a wire brush on a powered shaft. These can be quite harsh on brass, so I don't use them, but I have seen them used. There are also brass brushes sold to fit grinding arbors, for this kind of work. Check Woodcraft to see what they have.