You can usually shoot either but you will have to select a thinner patch for the .535 ball. It is one of the fun things a shooter must do to determine exactly what your particular gun prefers. If you happen to find that your gun prefers the .535 ball with, for instance, a .010 patch, you may find that the thinner patches are burning through. If you find that happening, you can load a lubed felt OP wad to protect your patch from burn through or you can use another method and after settling your powder in your bore, add about a 10 to 20 grain volume of corn meal, grits or Cream of Wheat on top of your powder before seating your ball. The cornmeal or whatever acts as an inert buffer between the burning powder and the patch. I know it sounds odd but it is a relative common practice and will do absolutely no harm to your rifle. The cornmeal will get a bit toasted as it exits the muzzle and you may get a few folks asking who's baking cornbread. Anyway, give it a try if you decide on a thinner patch and are getting burn through.
I'll relate a humorous story about using corn meal as a filler in the black powder cartridges for my Sharps .45/120 rifle. I was shooting in a cross sticks match and did not want to take the punishment of a fully loaded .45/120 cartridge. So, I loaded them down to only 70 grains of 3f black powder. You can never leave a void in a black powder cartridge so I finished filling them to the top with corn meal. As I was shooting, the wind was blowing directly back toward me. There were a few guys standing behind me watching me shoot. after I had finisned my allotted number of rounds, one of them asked me "What the heck are you shooting in that gun?" "70 rains of 3f Goex with a corn meal buffer." I replied. "Corn meal, huh. No wonder every time you fired, I got hungry for cornbread." :haha: