Living in Alberta we do a lot of shooting and hunting in weather that is anywhere from 0 to minus 40 F. The coldest I have hunted in is caribou and seal in the Territories, it was minus 55 F.
Cold weather, we never bring the guns into a warm room or vehicle. This avoids condensation and other problems. Guns are lubed with dry graphite or shot completely dry. We use a lot of stainless but at minus 20 and below, you won't get rust even on chrome moly. On CFs, all loads are made with magnum primers. Percussion, same.
With CFs, at -20 you will see your POI drop about an inch at 200, from what it did at +70. With Black powder and open sights and the relatively short distances, I have never noticed any change in POI.
A few things you will find, regular oil gets a lot thicker as you get close to freezing and turns to a thick glue at -20 and concrete at minus 35. Your larger groups are far more likely the result of the slower moving lock parts than any effect on the powder at +30. If you are going to shoot at freezing to minus 20 use a good synthetic oil/graphite lube, or full synthetic oil, 0-20 Mobile 1 and Amsoil are common for barrel internal and external) for lube. Below -20, dry graphite or no lube at all on any moving parts and full synthetic on anything that is just a surface, though bore we leave dry.