You may have the same problem I once had. Your length of pull may be too long with "thick" hunting clothing on. I missed with my custom rifle way too many times on deer I could have killed with my longbow. In order to reach the trigger I must have moved the barrelunconsciously while pulling the trigger. After the 3rd missed deer in 5 days of hunting I really was mad. I was ready to throw the $2000 rifle in the next garbage can, i was so disgusted.
Shot the rifle at the range. 1" groups at 65 yards. Everything was fine. Gave the rifle to a national champion, the rifle was dead on. But while talking to him we talked about LOP and although he is really tall, he prefers a shorter LOP as well.
After dropping from 13.25" LOP to 12.6" length of pull, the problem was solved. First shot offhand at 45 hard with the 0.62 jaeger literally smashed the deer's heart.
Although think about the influence in fouling versus a clean and "oiled barrel". My 275 year old jaeger shoots the first shot about 5" higher at 75 yards. The second and so on is dead on....