The stone you are probably describing is usually made of Carborundum, a man made material. But it can work well in keep the edge of an axe true, and sharp, if you know how to use it. Rest the stone on a solid surface, and use both hands on the axe to control your angles. A throwing axe can best work with a 20-30 degree approach bevel, made by using a draw file, and then using a 45 degree angle on that stone, or a good arkansas stone, fine, to created what is called a " supported edge ". The fine stone work will remove the feather or " burr " left by filing the two sides of the blade to make that approach bevel. But in sharpening that second bevel at 45 degrees, another much smaller burr, or feather, will be created. Use a belt, or strip of leather glued to a board to strop that feather off the edge before using the axe on anything. This three-stage process is fully described in The Razor Edge Book of Sharpening, and in literature written by Roger Needham, a long time supplier of arkansas stones to NMLRA members at Friendship. Roger's stand is just inside the gate on the West side of the drive.
A throwing axe takes a pretty good beating, because their can be any number of things hidden in the wooden backstops, and when you miss, and it hits the ground, their can be rocks, or metal that will ding the edge in a heartbeat. Keep the edge sharp, and a good stone ready to make any repairs, and you will have a hawk that serves you well.
I do not use oil on my stones, but rather use Liquid Wrench or Break Free, both kerosene based oxydizers that help dissolved bits of steel clogging the surface of the stones, facilitating the removal of the clogs with a rag or patch soaked in the solutions and rubbed on the stones in a circular motion, first one direction, and then counter clockwise. If those clogs are not removed, the stone cannot reach down and cut the steel from the edge of your axe to give you a new edge. My fine Arkansas, which I use for making that supported edge bevel, is used dry, and cleaned frequently. Otherwise, those fine bits of steel hang in solution, and dull one side of the edge as I am sharpening the other side of the edge. A sharp axe, like a sharp knife is a precision tool, not to be misused or given to a fool to use. If you ever get to see one of the log rolling rallies where the use of axes are still a fascinating sport, try to get a look at the axes that the competitors use. The edges they put on them just before cutting logs at speed is razor sharp, but also strong enough to take the pounding that is required to remove huge chunks of wood.