I would be concerned about bounce-back as well as plastic residue in the barrel. Paintballs, I believe, are normally shot with compressed air. Black powder generates a good deal of heat, which may affect the structural integrity of the projectile. I wouldn't shoot them in any of my guns, nor would I want to be nearby if someone else is shooting them. The result could very well be harmless, or even humorous. However, we don't know that.
This may be a good time to point out that steel ball bearings are also very dangerous, due to the risk of ricochet and bounce-back. There is some cautious experimentation in the industry with non-lead projectiles, due to lead prohibitions in some areas. Plastic-saboted "Eco-Slugs" and the TomBob ITX (iron-tungsten matrix) balls are examples, but these can be very hard on your rifle bore if the sabot or patch slips.
We had a thread going a few months ago about shooting rock salt and even okra seed out of smooth bored muzzleloaders. These are lightweight, so won't carry far or bounce back, and won't melt in your barrel. However, these are typically "special purpose" loads used to deter pests.
As
@necchi pointed out, some people apparently feel a compulsion to poke things down muzzleloader barrels. The park staff at Castillo de San Marcos in St. Augustine, here in Florida, have described finding trash, golf balls, tennis balls, and all sorts of junk stuffed in the cannon barrels. These guns are used for blank firings over the waterway, and the have to be carefully examined before every demonstration, primarily because of the idiotic pranks of tourists.
I'm not one to rain on anybody's parade, but I am a "traditionalist." Rubber or paintball projectiles in a muzzleloader may very well be harmless, or might even provide material for some good jokes and stories. However, I think I'll stick with lead.
Best regards,
Notchy Bob