You can open the hole from the pot where the lead comes out, and you can also open up the holes into your two cavity mold to allow more lead to enter them faster. The Lee Mold are aluminum Alloys, and both heat up fast and cool down fast( er) than a iron or steel mold would do. Try putting the mold into the molten lead between pours to heat it up.
Heat transferrance is always more efficient when you have the two metals in contact, rather than leaving air between them.
You do have to get HOT lead into the mold, and you need a large sprue on top of the cut off plate on the mold block, holding it to the spout of the pot until the color changes as the mold cools off, and a small dimple appears in the middle of the sprue puddle. The lead drippings can be picked up with needle nose pliers and dropped back into the pot, along with the sprues, and wrinkled balls.
Casting is hot, sweaty work, when done right. You should not be working in a cold room, where the only source of heat in the room is your lead pot. That allows the mold to cool too fast, and cools the lead too fsst, too. I used to cast in a cold basement shop, and had to put an electric space heater at my feet before I could get a consistent run of good bullets out of the pot. I recall working about 4 hours and threw back more than 300 bullets before the evening ended, with only a handful of bullets that were keepers. I think I got the space heater into the shop the next night.