You aren't alone. I'm pretty new to flintlocks - picked up my first one about a year and a half ago. Funny thing is I went to my first NMLRA regional shoot about a year ago and took second place in offhand shooting with it! Of course, I don't know how many people entered...
Anyway I shoot once a month with some NMLRA guys I met at the regional. We shoot scored targets for fun. I can't hit the broad side of a barn off hand. I have gotten some very good groups off a bench, but the delay in the firing gets me, I guess.
And I can shoot. I have been shooting caplock Civil War guns for a decade in the N-SSA and have more medals than I can shake a stick at.
I'm shooting a Pedersoli Pennsylvania flintlock in .50. Using Tom Fuller flints from Track of the Wolf.
Last weekend I did some very careful load workups from 40 grains to 80 grains in 5-grain increments. I weighed all my bullets (Lee cast .490 round ball) and discarded any outside of +/- .5 grains from average. I switched to using .02" thick cotton duck canvas (Given to me by one of my NMLRA shooter friends). I lubricated the canvas using a mixture of 1:3 Ballistol/Water and let the water evaporate out. So I was using cut patches instead of pre-cut patches. Walter Cline says in his book you can't get as good of good groups with pre-cut patches as you can with cutting at the muzzle.
My best group seems to be 65 grains. But it is hard to say for certain because on some of the targets (like 55 grains) some holes went through the same hole, at least I am pretty sure. It doesn't seem likely as "on target" as I was that I could have missed the paper entirely. But in the On Target software, I only input the holes I was sure of. So the group size would be smaller for some of the targets with only 4 holes visible (except for 55 grains where I know for sure the first 2 shots went in the same hole just to the left of the X ring).
So for me, it's all about flinching and the delay of ignition from the flintlock. I can definitely hear the "ca-CHACK-WHOOSH" before the gun goes off most of the time. I have found that it is better to underfill the pan than to fill it to the rim. I'm using 3F to prime (I don't have any 4F). A sharp flint also makes more sparks and so you get faster ignition.
I have also purchased a .45 ACP caliber laser "cartridge". These are for boresighting cartridge firearms. I wrapped it in a few layers of painter's tape and set it in the muzzle of my flinter. Now I can practice pointing it at a spot on the wall and holding it. The gun feels very muzzle heavy compared to my military caplocks.