I heard this statement repeated in many places. Not specifically about ballistol, but other less-natural products. Personally I use mostly neatsfoot, jojoba, coconut oils and lard, ballistol being the only "artificial" oil. However, I use a lot of it. I use ballistol water (20:1) for cleaning and 7:1 as patch lubricant (I let the water evaporate mostly). One thing I can say for sure is there is no detriment whatsoever to using ballistol (mixed with water) around burnt black powder. I noticed no difference in the sludge resulting from using artificial or natural oil.
I also encountered people who added motor oil to their black powder lube and claimed it worked perfectly fine for them.
So I would be very interested in finding out where did this "petroleum+black powder =sludge" came from and if there is any actual experience backing it up.