YOu forget that the shot pellets are breaking a " trail " through the air in front of that big, flat, heavy wad. That is why the pellets that have rubbed flats on their sides while traveling down the barrel slow down more rapidly, and begin to break to the outside of the main column of shot. YOu can see these pellets with their flat sides if you shoot through paper at 10 feet, then 15, 20, 25, and 30. At each distance more and more pellets are breaking out of the pattern. At ten yards, the pattern should be no larger than a large softball, but you will often find pellets hitting outside even this large pattern. If you look closely, you may see pellets that went through the paper with a flat on the side, instead of in front or in back. You will get a nice arc on the paper around a portion of the hole the pellet made, and then a " flat " on one side of the hole. That tells you what is happening with your unprotected shot loads.
As to the Overpowder or cushion wads following the shot, there is no doubt that it does, as you often will see large holes through paper at 25 yds. Again, if you shoot through paper at shorter ranges, as described above, you see what the wads are doing, in relation to the hole made by the load of shot. As the lead pellets slow in that first 20 yards, the wads " catch up"< mostly because for half that distance the wads are riding behind the shot in a vacuum created by the shot. As the shot opens up, however, the vacuum disappears, and air now begins to also slow the wads. However, the damage is already done. As soon as that shot slows enough to strike the wads trailing behind, the shot begins to split away from the POA, with the wads being the base of a shallow cone of shot that remains on line. The shot in that shallow cone is what is left to fill the center of your pattern, and even that begins to travel sideways because of the interference with the aerodynamics of each pellet by the proximity of the wad. Remember, as that wad slows, it is also pushing air in front of it, and if the shot is close to that wad, it will be receiving and reacting to air resistence not only on its front, but on its rear. Something has to give. Its usually that pellet, and it gives by getting out of the way, ie. going sideways.
I hope that helps you understand why heavy wads tend to blow patterns of shot, even though the shot load is much heavier, and perhaps the wad only weighs a little bit more than an individual pellet. The wad at first flies fast because it takes advantage of the vacuum created in front of it by the pellets. As that load of shot begins to dispurse, and slow, the vacuum disappears, and not the wad is pushing air into the back of the shot. The wad itself does not have to mechanically hit a pellet to make it move out of the line of fire. Air will do that quite nicely, just like when you blow out a candle.