I have trouble believing that there would have been enough barm produced from brewing to make all the bread on a daily basis
Not sure why you pose that "all the bread" had to be made the same way, nor that all of the bread had to always be made on a daily basis.
Every single family in Europe or in the UK didn't have access to a bread oven, nor did they personally bake bread on a daily basis. In fact some households didn't necessarily bake bread. Commercial bakeries are the most likely customers for barm from a brewhouse. IF it wasn't close to that sort of situation, then there wouldn't be a law requiring brewers to sell barm to bakers, and to set a fixed price.
In The House-Keepers Pocket-Book and Complete Family Cook by Sarah Harrison 1739, she has a recipe for French Bread that uses " Ale yeast", and also a recipe for buns that uses "fresh barm"...
To make French Bread
Beat two Eggs with a little Salt, lay to them half a Pint of Ale Yeast, or more, then put to it three Pounds of fine Flour, and put into it as much blood-warm Milk as will make it soft and light ; then make it into Loaves or Rolls, and when bak’d and cold, rasp or grate all the outside off, and the it is fit to set at Table.
To make Buns
Take, to three Pounds of Flour well dry’d before the Fire, two Pounds and a half of Butter, a Pound of Sugar, and ten Ounces of Carraway-comfits ; melt your butter in warm Water upon the Fire, with six Spoonfuls of Rose-water, a few more Caraway-seeds, if you please, and a Pint of new Barm ; knead all these together, and set your Buns into the Oven, after white Bead is drawn.
By 1767 The Art of Cookery Made Plain and Easy by A Lady [Hannah Glasse] has a recipe To make white bread, after the London way, which uses yeast. The next recipe is To make French Bead which calls for a pint and a half of "good ale yeast". Then she follows two pages later with, A receipt for making bread without barm with the help of a leaven, which uses yeast in a dough-sponge from a previous baking, with instructions on keeping it from being too sour. This recipe she credits to The Dublin Society. Directly following that recipe..., she documents the method for drying yeast for later use, as well as drying the same yeast for use in beer.
So there were a lot of ways known to use yeast, but it wasn't uncommon to use barm, and the yeast that they were drying for use later..., is an ale yeast.
LD