What your seeing, happens to all rifles but is most notable with Black Powder guns. The first shot out of all of my guns goes wide of the mark.
This is the reason when people are in a target match they are given a "fouling shot" which doesn't count towards the score.
This seems to occure worst in guns whech were taken apart to clean them and then reassembled.
When a gun is shot there are all kinds of wierd vibrations, expansions, and osculations happening to the barrel so my own theory is "the first shot seats the barrel back where it was when I sighted it in last".
To see if my theory is pure bunk you might take your gun out and shoot a few shots. Then using rags to protect the wood, clean the gun without taking it apart (at the range or where ever your shooting), dry and lightly oil the bore, reload and take your best shot to see if it is still hitting where it was before cleaning.
If it is still hitting the same place your problem is solved.
Note, I have never tried this so I don't know how it will work.
Any one else out there with any ideas??